sv-tools/marshmallow-objects

non-deterministic order when using NestedModel

lorencarvalho opened this issue · 7 comments

lorencarvalho commented

Hi there, me again :)

Looks like we have some issue retaining order when using Meta.ordered = True with nested models.

Here's some code to illustrate what I mean:
test.py

from marshmallow_objects import *


test_data = """
{"subfields": {
    "test": {
        "field_one": "one",
        "field_two": "two"
        }
    }
}
"""


class Base(Model):
    class Meta:
        ordered = True


class SubModel(Base):
    field_one = fields.Str()
    field_two = fields.Str()


class TestModel(Base):
    subfields = fields.Dict(keys=fields.Str(), values=NestedModel(SubModel))


print(
    TestModel.load_json(test_data)
)

 $ python3 ./test.py
OrderedDict([('subfields', {'test': {'field_two': 'two', 'field_one': 'one'}})])
 $ python3 ./test.py
OrderedDict([('subfields', {'test': {'field_two': 'two', 'field_one': 'one'}})])
 $ python3 ./test.py
OrderedDict([('subfields', {'test': {'field_one': 'one', 'field_two': 'two'}})])

Note the random field ordering.

I'm really not sure what is causing this, but it seems to be limited to NestedModel being used as the values of a fields.Dict field. I have a hunch that this is caused by the fact that the NestedModel field relies on deserializing into models via the model's __init__ method, which just passes in the incoming dict as kwargs (and therefore casting it from an OrderedDict to a regular one) but I am not certain.
Any help would be appreciated!! I'll update this thread if I find anything else.

SVilgelm commented

Hi, thank you, I need a time to figure out how to fix it

lorencarvalho commented

I believe I've narrowed it down to something happening in the schema property of the Nested field type (in Marshmallow upstream, not your library, though it maybe a combination of things).

https://github.com/marshmallow-code/marshmallow/blob/3.0.0b20/src/marshmallow/fields.py#L432

Namely, this line:

self.__schema.ordered = getattr(self.parent, 'ordered', False)

I think that self.parent never has an 'ordered' attribute when using marshmallow-objects.

However, it also appears that in the latest marshmallow release that line has been removed! I can confirm that I can no longer repro using the latest Marshmallow (3.0.0rc6) and master of marshmallow-objects. Do you mind cutting a new release to pypi with the compatibility fixes for rc6? Thanks!

SVilgelm commented

Hi

Do you mind cutting a new release to pypi with the compatibility fixes for rc6? Thanks!
I did it yesterday, but it wasn't uploaded to pypi :(

SVilgelm commented

figured out, the regex for branches was changed by mistake

lorencarvalho commented

thanks so much!

  • 😄1