apple/swift-protobuf

Can't JSON encode something the library JSON Decoded

thomasvl opened this issue · 3 comments

thomasvl commented

Another fuzz testing find: https://oss-fuzz.com/testcase-detail/4827369236463616

Given the input {"wktFieldMask":"{H̱ܻ̻ܻ̻ܶܶAܻD"}, when the tester tries to generate JSON out of it, it fails and throws. If we accepted it.

So odds are something in the parse isn't validating the same way and should have failed to decode.

thomasvl commented

@tbkka should FieldMask+Extensions's ProtoToJSON/JSONToProto follow https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/util/internal/field_mask_utility.h for how it does things?

tbkka commented

I'll start working through it. I believe that any valid field mask will correctly round-trip today. So I think the next steps may be:

  1. Refuse to decode anything that can't be re-encoded. This will fix the immediate crash issue.
  2. Reject field masks that are invalid according to the C++ code.

The more ambitious path management and union/intersection handling in https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/util/field_mask_util.h can follow later.

tbkka commented

That was simpler than I thought! Since fieldmasks are proto field names, we should never be permitting non-ASCII values.

By allowing non-ASCII values, the fuzz tester was able to explore some of the strangeness that is Unicode. For example, this particular string has a character that matches "A"..."Z" but when converted to lowercase, the result does not match "a"..."z".