ctrl-Felix/cosmospy-protobuf

Google Protobufs Dependency/Versioning

Opened this issue · 1 comments

ctrl-Felix commented

I will make this issue here to provide a central place to track and discuss the current issue we're facing with the google protobufs
Sources: PR: #16, Issue: #15

I think @xloem proposed to make a separate Descriptor Pool if I understood you correctly? I have to admit I am not deep enough into Protobuf to fully understand if that would be the best solution in this case.

Regarding the Pull request #16. You've added the google protobuf files but I think they are mainly unused. When compiling the protobufs the compiler will always translate google.protobuf to the google package and not the files in the repository.

The alternative would be to remove the protobuf dependency completely, which I believe would be the best solution. However, we need to figure out a way for the descriptor pool before I guess.

All in all I still find these protobufs very confusing...

xloem commented

A descriptor pool change maybe like #15 (comment) might theoretically make the code work with cosmpy and other differently versioned packages without issue.

I’m thinking maybe with more time and exposure of the google situation we can come up with a good approach. Protoletariat can be configured to load package-local or global google. If packages use different versions of the same proto, they can’t be imported simultaneously, unless some change like the separate descriptor pool is made.

EDIT: I've updated my descriptor pool post to make the idea more clear. Here it is below. I have not tested it so it likely needs iteration for troubleshooting:

  • The pb2 files contain lines like from google.protobuf import descriptor_pool as _descriptor_pool. Change these to instead import a package-specific file like from ...descriptor_pool import DESCRIPTOR_POOL.
  • The package-specific file can contain a single line like DESCRIPTOR_POOL = DescriptorPool(). I guess an import from google.protobuf or such would be needed too, to access the class.
  • The pb2 files contain lines like DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n*cosmos/base/tendermint/v1beta1/query.proto\x12\x1 .... Change these to instead use the package-specific pool like DESCRIPTOR = DESCRIPTOR_POOL.AddSerializedFile(b'\n