wmv/appengine-ndb-experiment

Non-UTF-8-encoded names are allowed in some places

Opened this issue · 1 comments

NDB accidentally allows 8-bit strings that aren't valid UTF-8 in a few places; 
in particular:

key names (string id of a Key)
property names
namespaces
app ids (not too important)

This was never intended but felt through the cracks, and we seem to have some 
apps that accidentally write such key names.  These cannot be displayed by the 
admin console's data viewer because the other datastore client libraries don't 
support these.

We should probably start rejecting these, but because we're in GA we can't just 
change the rules.  Proposal:

1. Add a warning for this case, plus a tri-state flag that can suppress the 
warning or turn it into an error.  The warning should point out the problem and 
the existence of the flag.

2. (In a few releases:) Change the flag default to generate an error.

3. (Some or many releases later:)  Remove the flag altogether, always 
generating the error.

In perpetuity, *reading* such values should not be an error.  The behavior 
should be similar to that of TextProperty, which rejects 8-bit strings that 
aren't valid UTF-8 in its validation, but allows them when reading from the 
datastore.

Original issue reported on code.google.com by guido@google.com on 29 Aug 2012 at 11:06

Original comment by pcoste...@google.com on 5 Dec 2013 at 2:36