Nullability Parent of Nested Models

[DaanRademaker]

Good afternoon!

I am trying to implement the following models

from sparkdantic import SparkModel

class ChildSchema(SparkModel):
    ncol1: str
    ncol2: str

class ParentSchema(SparkModel):
    col1: ChildSchema

I would assume that because ChildSchema is not defined as optional that the spark schema would reflect this so that ChildSchema struct nullability is False.

However, for ParentSchema defined above this is the output of ParentSchema.model_spark_schema().jsonValue()

{
    'type': 'struct', 
    'fields': [
        {'name': 'col1', 
         'type': 
            {
                'type': 'struct', 'fields': [
                    {'name': 'ncol1', 'type': 'string', 'nullable': False, 'metadata': {}}, 
                    {'name': 'ncol2', 'type': 'string', 'nullable': False, 'metadata': {}}
                ]
            }, 
        'nullable': True, 'metadata': {}}]
}

Should 'nullable' be false for the case above? Or is this behavior on purpose?

[mitchelllisle]

@DaanRademaker definitely a bug - have fixed and cut a new release to fix this in v0.20.1

[DaanRademaker]

Thanks for the quick Fix! This was the final open thing for us to move completely to your library.