googleapis/python-ndb

Why are non-compressed `JsonProperty`s opaque base64 strings in the datastore console web UI?

snarfed opened this issue · 1 comments

Hi all! First off, as always, thank you all for maintaining ndb. Much appreciated!

Is your feature request related to a problem? Please describe.

I've always wondered why non-compressed JsonPropertys don't show up as human-readable JSON strings in https://console.cloud.google.com/datastore/entities . Is that intentional? That console is hugely useful, enough that I manually serialize JSON into StringPropertys so I can read them in it, but I'd obviously much rather use JsonProperty.

Reading the code, it just serializes the JSON and encodes as ASCII, and if I use compressed=False, BlobProperty doesn't do anything else beyond that. But when I do eg:

class Foo(ndb.Model):
  j = ndb.JsonProperty(compressed=False)

with ndb.Client().context():
  Foo(j={'x': 'y', 'z': [3,4]}).put()

That property is visible in the web console as a BLOB with value eyJ4IjoieSIsInoiOlszLDQsNV19. Is this the web console's fault? Is there anything I can do to avoid that?

image

Describe the solution you'd like

Web console shows JsonProperty values as human-readable (and ideally editable) serialized JSON.

Thanks in advance!

Looks like this is indeed due to blob handling. I tried making my own JsonProperty that subclasses TextProperty instead, and it does what I want: values show up as readable serialized JSON in the web console with type STRING.

class JsonProperty(ndb.TextProperty):
    def _to_base_type(self, value):
        as_str = json.dumps(value, separators=(",", ":"), ensure_ascii=True)
        return as_str.encode("ascii")

    def _from_base_type(self, value):
        if not isinstance(value, six.text_type):
            value = value.decode("ascii")
        return json.loads(value)