CamelCase Responses with snake_case Protobuf

[arranf]

The change to make everything camel case has had a side effect I wasn't anticipating and I wanted to run my ideas past you!

I'm looking to rewrite an existing service which has an existing data contract which uses snake_case for its responses. The current implementation of TwirpScript generates types which are camelCase, which it makes it impossible to replace the existing service without causing a breaking change without some "trickery".

With the current implementation of TwirpScript you can "trick" the JSON into sending variables in the current format by using as however the protobuf implementation uses the generated types with the writer and as a result will output msg.camelCase unless you alter all the generated types.

Additionally - it means that the Protobuf definition is no longer a valid source of truth for other services as the message on the wire doesn't match the protobuf definition.

I'm not sure what the solution is here going forward:

• Have types use camelCase and map them to snake_case?
• Revert to types using snake_case

I'd really appreciate your thoughts!

[tatethurston]

Hey @arranf,

Does your existing service respond with JSON, protobuf, or both? If you own the client and the server, and your clients only use protobuf, the casing difference won't break any clients or servers, because the field names aren't sent over the wire.

If you need JSON support in production, I could prioritize some JSON work in TwirpScript. The JSON serialization/deserialization for TwirpScript is out of spec with the published Protobuf3 spec, and resolving that would open up some more options for us.

Another idea:

Let's call your existing serivce FooService. You could publish a temporary service, eg FooServicev2 that contains all the same business logic as FooService. Then you migrate all clients to using FooServicev2. You could then deprecate and remove FooService. You could follow those same steps again if you want to rename FooServicev2 back to FooService.

[arranf]

@tatethurston Hey!

Thanks for pointing out I was being an idiot with the Protobuf field names!

I think that leaves the main issue being that the service we're placing only supports JSON. We don't own the clients to the degree that we can do an easy breaking change here - we need parity in the payload that's sent back.

We can currently get away with reshaping the object and then using as (described in my original issue description) for the JSON side, but I'd prefer to keep that as a short term hack!

[tatethurston]

Two options I can think of are:

• Adding an option to TwirpScript to use the proto field names exactly as written for JSON (de)serialization. This would apply to all files generated by the TwirpScript compiler (read: not on a proto by proto basis, it would be all or nothing). I think I would keep the generated code interface field names the same (camel cased) and only have the JSON (de)serialization impacted.

• Implement the json_name option described here. That would enable you to specify these field names as snake cased in your proto file, eg:

message Foo {
  int32 my_field = 1 [json_name="my_field"];
}

Do those options sound like they would work for you? And if so, which do you think you would use?

Out of curiosity, are you replacing an existing twirp service? And if so, is that service written in another language? If you could point me to the library, I'd like to take a look.

[arranf]

Two options I can think of are:

* Adding an option to TwirpScript to use the `proto` field names exactly as written for JSON responses. This would apply to all output from the TwirpScript compiler. I think I would keep the generated code interface field names the same (camel cased) and only have the JSON serialization impacted.

* Implement the `json_name` option described [here](https://developers.google.com/protocol-buffers/docs/proto3#json). That would enable you to define these field names as snake cased in your `proto` file, eg:

message Foo {
  int32 my_field = 1 [json_name="my_field"];
}

Out of curiosity, are you replacing an existing twirp service? And if so, is that service written in another language? If you could point me to the library, I'd like to take a look.

I'm happy with either option and I'm happy to help contribute to this as I understand either are significant work!

We're not replacing an existing twirp service here - we're replacing an old standard restful service but putting a reverse proxy in front of it to redirect traffic to the new Twirp service. We're hoping to use this pattern where possible to avoid breaking changes to clients.

[tatethurston]

Awesome, thanks for the context. I’ll plan on implementing JSON serializers and accepting the json_name option

[tatethurston]

I implemented json_name in #116. Try it out and LMK if you run into any issues!

[arranf]

Thanks so much @tatethurston !!!!
Will take a look :)