`bufio` in `WriteTo`/`ReadFrom` causes a lot of GC allocations
omerfirmak opened this issue · 0 comments
omerfirmak commented
(pprof) omer@omer-ThinkPad-E14-Gen-3:~/Documents/juno$ go tool pprof -alloc_space http://localhost:9080/debug/pprof/heap
Fetching profile over HTTP from http://localhost:9080/debug/pprof/heap
Saved profile in /home/omer/pprof/pprof.juno.alloc_objects.alloc_space.inuse_objects.inuse_space.008.pb.gz
File: juno
Build ID: 951dffa00ff1dc027e7f19c22ffa115310c93eb0
Type: alloc_space
Time: Apr 22, 2023 at 6:09pm (+03)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 441.63GB, 93.58% of 471.93GB total
Dropped 677 nodes (cum <= 2.36GB)
Showing top 10 nodes out of 102
flat flat% sum% cum cum%
216.42GB 45.86% 45.86% 216.42GB 45.86% bufio.NewWriterSize (inline)
199.21GB 42.21% 88.07% 199.21GB 42.21% bufio.NewReaderSize
3.96GB 0.84% 88.91% 3.96GB 0.84% reflect.New
3.73GB 0.79% 89.70% 3.74GB 0.79% io.ReadAll
3.63GB 0.77% 90.47% 3.63GB 0.77% github.com/cockroachdb/pebble.(*Batch).grow
3.36GB 0.71% 91.18% 4.62GB 0.98% bytes.(*Buffer).grow
3.11GB 0.66% 91.84% 4.07GB 0.86% encoding/json.(*decodeState).objectInterface
3.09GB 0.65% 92.49% 3.09GB 0.65% github.com/bits-and-blooms/bitset.New
2.64GB 0.56% 93.05% 2.64GB 0.56% github.com/cockroachdb/pebble.(*Batch).newInternalIter
2.49GB 0.53% 93.58% 224.33GB 47.53% github.com/bits-and-blooms/bitset.(*BitSet).MarshalBinary
Buffered counterparts of Reader and Writer interfaces allocate 4K each time.
bufio provides no benefit either, words are read individually in to a local buffer.