hamba/avro

Performance degradation in v2.19.0

Closed this issue · 4 comments

In the benchmark test I noticed a decrease performance in CPU and memory:

var (
	schema = `{"type":"record","name":"test","namespace":"appo","doc":"","fields":[{"name":"ts","type":{"type":"long","logicalType":"timestamp-micros"},"doc":""},{"name":"uid","type":"long","doc":""},{"name":"parentLog","type":"string","doc":"","default":""},{"name":"parentTs","type":{"type":"long","logicalType":"timestamp-micros"},"doc":"","default":0},{"name":"resourceList","type":{"type":"array","items":{"type":"record","name":"Resource","doc":"","fields":[{"name":"id","type":"int","doc":"","default":0},{"name":"type","type":{"type":"int","logicalType":"uint16"},"doc":"","default":0},{"name":"value","type":"long","doc":"","default":0},{"name":"inc","type":"long","doc":"","default":0}]}},"doc":""},{"name":"dar","type":["null","int"],"doc":"","default":null},{"name":"level","type":["null","int"],"doc":"","default":null},{"name":"vipLevel","type":["null","int"],"doc":"","default":null},{"name":"uniqSessionId","type":["null",{"type":"long","logicalType":"uniqSessionId"}]},{"name":"actionTs","type":["null",{"type":"long","logicalType":"actionTs"}]},{"name":"accountId","type":["null",{"type":"long","logicalType":"accountId"}],"doc":""}]}` //nolint:gochecknoglobals
	data   = []byte{206, 209, 246, 250, 201, 202, 132, 6, 190, 131, 227, 11, 18, 113, 117, 101, 115, 116, 70, 97, 114, 109, 154, 208, 246, 250, 201, 202, 132, 6, 1, 10, 2, 30, 158, 19, 120, 0, 2, 2, 2, 56, 2, 0, 2, 138, 234, 251, 133, 128, 13, 2, 220, 215, 2, 2, 224, 132, 218, 10}
)

func BenchmarkHambaDecode(b *testing.B) {
	s := avro.MustParse(schema)

	b.ReportAllocs()
	b.ResetTimer()

	for i := 0; i < b.N; i++ {
		var value map[string]any
		_ = avro.Unmarshal(s, data, &value)
	}
}

in v2.18.0
BenchmarkHambaDecode-10 611534 1956 ns/op 1636 B/op 31 allocs/op

in v2.19.0
BenchmarkHambaDecode-10 75986 15417 ns/op 4089 B/op 136 allocs/op

This is likely as we have had to rebuild generic decoding to make it compatible with composite schemas. I will be looking into this further in the next releases, but should you want the old behaviour, you can use Reader.ReadNext.

Can you try the main branch to see if this solves your issue please.

My results is

BenchmarkHambaDecode-10 381804 3089 ns/op 3661 B/op 69 allocs/op

better, but still worse than before

Unfortunately it will likely always be worse than before, due to the new way the decoding needs to happen for composite schema support. All the additional time is spent making to primitives, which is why you see an increase in allocs/op.

I don't really have a good way around this at the moment, but something that should be explored further in future.