Explore new serialization library used for communication between horaedb-server
Closed this issue · 3 comments
jiacai2050 commented
Describe This Problem
Our RemoteEngineService use gRPC to communicate between different servers, and adopt protobuf as encode/decode methods, protobuf is not a very high performance library, and we observe high CPU usage in our production env.
Proposal
Explore new serialization library, which support following features:
- Zero copy, which means its wire format is the same as memory layout.
- Native rust binding
- Support integration with tonic
Additional Context
Some choices:
jiacai2050 commented
zealchen commented
Okay. Those libs look super awesome, let me build a benchmark to demonstrate performance first.
jiacai2050 commented
Our benchmarks show flatbuffers has the best perf, and it can be integrated with tonic, so let's refactor our remote service.
Currently we can only refactor the write
API, since the CPU cost of it is the most obvious.