tendermint/go-amino

Investigate amino performance/mem usage

Opened this issue · 2 comments

While looking into adding amino to this list: https://github.com/alecthomas/go_serialization_benchmarks (see this branch: https://github.com/Liamsi/go_serialization_benchmarks/tree/add_amino), it seems like amino is quite slow compared to other similar libraries:

BenchmarkAminoMarshal-12                         300000              5492 ns/op            4138 B/op         72 allocs/op
BenchmarkAminoUnmarshal-12                       1000000              1081 ns/op             194 B/op          7 allocs/op
BenchmarkProtobufMarshal-12                      2000000               676 ns/op             200 B/op          7 allocs/op
BenchmarkProtobufUnmarshal-12                    3000000               593 ns/op             192 B/op         10 allocs/op
BenchmarkGoprotobufMarshal-12                    5000000               325 ns/op              96 B/op          2 allocs/op
BenchmarkGoprotobufUnmarshal-12                  3000000               498 ns/op             200 B/op         10 allocs/op
BenchmarkGogoprotobufMarshal-12                 20000000               119 ns/op              64 B/op          1 allocs/op
BenchmarkGogoprotobufUnmarshal-12               10000000               166 ns/op              96 B/op          3 allocs/op

Hopefully, we should be able to vastly improve this performance without completely reworking the structure.

Here is what the profiler tells us about memory/cpu while running above benchmarks for unmarshaling/marshaling:



marshal_cpu_profile


marshal_mem_profile


unmarshal_cpu_profile


unmarshal_mem_profile


Zaki suggested to add a hint in the readme about performance issue and in which cases users would might want to refrain from using amino.

Two things are not fair for your amino case.

  1. Before marshalling, you need to reset timer or stop timer before generating testing data.
func BenchmarkAminoMarshal(b *testing.B) {
        // b.StopTimer()
	data := generateAmino()
	b.ReportAllocs()
	b.StartTimer() 
	s := AminoSerializer{amino.NewCodec()}
	for i := 0; i < b.N; i++ {
			s.MustMarshalBinaryBare(data[rand.Intn(len(data))])
	}
}
  1. the type of BirthDay is time.Time while in other cases it is int64

good point @rickyyangz. Did you you re-benchmark with your suggested changes? I would assume the performance to still be much slower than generated protobuf.