Compressed JsonProperty within a repeated StructuredProperty raises AttributError
aetherknight opened this issue · 2 comments
While converting a complex AppEngine code base from python 2.7 to python 3.x, I ran into an unexpected bug in the new google.cloud.ndb that does not exist in the legacy google.appengine.ext.ndb.
Environment details
python-ndb 1.11.1
Python 2.7.18
Discovered and tested on MacOS 12.6
Steps to reproduce
See the toy example below. But to summarize: A repeated StructuredProperty that contains a compressed JsonProperty field. The bug probably apples to other property types that inherit from BlobProperty.
Code example
from google.cloud import ndb
class SomeStructuredProperty(ndb.Model):
compressed_json = ndb.JsonProperty(compressed=True)
class ParentModel(ndb.Model):
repeated_sp = ndb.StructuredProperty(SomeStructuredProperty, repeated=True)
m = ParentModel(repeated_sp=[SomeStructuredProperty(compressed_json={})])
m.put() # Raises AttributeError with google.cloud.ndb
Stack trace
Traceback (most recent call last):
File "/.../test.py", line 247, in test_problem
m.put() # Raises AttrbuteError with google.cloud.ndb
File "/.../lib/google/cloud/ndb/_options.py", line 102, in wrapper
return wrapped(*pass_args, **kwargs)
File "/.../lib/google/cloud/ndb/utils.py", line 121, in wrapper
return wrapped(*args, **new_kwargs)
File "/.../lib/google/cloud/ndb/utils.py", line 153, in positional_wrapper
return wrapped(*args, **kwds)
File "/.../lib/google/cloud/ndb/model.py", line 5383, in _put
return self._put_async(_options=kwargs["_options"]).result()
File "/.../lib/google/cloud/ndb/tasklets.py", line 214, in result
self.check_success()
File "/.../lib/google/cloud/ndb/tasklets.py", line 161, in check_success
raise self._exception
AttributeError: 'list' object has no attribute 'startswith'
Possible fix?
The exception actually happens in BlobProperty._to_datastore, but under python 2.7 the stack trace doesn't show the original stack trace. If I understand that method properly, it checks self._repeated
, but it neglects the repeated
input argument when determining how to mutate or read the data
dict that it is constructing for datastore. The exception is raised because the field itself is not self._repeated
, but it is part of a repeated StructuredProperty.
Fixed in google-cloud-ndb==2.3.0
Based on my initial testing, this does seem to be fixed in 2.3.0 and 2.3.1.
(Although we can't upgrade until this version of NDB can properly work with our test suite. It can't be used with the datastore emulator, and the firestore emulator lacks some fairly critical features of the datastore emulator at this time.)