Benchmarks against Serialization Systems
![Build Status](https://camo.githubusercontent.com/6dc7ba45cef10f12730dfeb67865160ed674bdea1290db40a6ed2fdf55570b3b/68747470733a2f2f7472617669732d63692e6f72672f7361696e74313939312f73657269616c697a6174696f6e2d62656e63686d61726b2e7376673f6272616e63683d6d6173746572)
|
Protocol Buffers (proto3) |
Thrift (compact protocol) |
Avro |
CSV |
JSON |
MessagePack |
Schema-less? |
No |
No |
No |
No |
Yes |
Yes |
Require compiling schema in advance? |
Yes |
Yes |
No |
No |
No |
No |
where $project is one of the name of sbt sub project (e.g. avroBench)
Sample data file generation
The schemas used in this benchmark are under schema
NOTE: This benchmark is taken place under the specific condition, results may be different under the other conditions.
- OS: Ubuntu 16.04 TLS
- CPU: Intel(R) Xeon(R) CPU E5-2680 v3 2.50GHz, 4 cores
- Memory: 14GB
- JDK 1.8.0_171, Java HotSpot(TM) 64-Bit Server VM, 25.171-b11
Average time to encode 100,000 records in milli seconds.
- 20 warming up iteration
- Average of 20 iteration
Protocol Buffers (proto3) |
Thrift (compact protocol) |
Avro |
CSV |
JSON (with jsoniter-scala) |
JSON (with circe) |
MessagePack (jackson-module-msgpack) |
MessagePack (msgpack4z) |
43.0 |
235.8 |
232.6 |
116.8 |
74.6 |
488.7 |
354.8 |
358.0 |
Average time to decode 100,000 binary records in milli seconds.
- 20 warming up iteration
- Average of 20 iteration
Protocol Buffers (proto3) |
Thrift (compact protocol) |
Avro |
CSV |
JSON (with jsoniter-scala) |
JSON (with circe) |
MessagePack (jackson-module-msgpack) |
MessagePack (msgpack4z) |
139.5 |
162.9 |
586.0 |
160.8 |
151.3 |
503.5 |
414.9 |
609.5 |
Data size of 100,000 encoded records in MB.
Protocol Buffers (proto3) |
Thrift (compact protocol) |
Avro |
CSV |
JSON |
MessagePack |
18.5 |
18.3 |
16.7 |
17.4 |
32.2 |
25.7 |