Compressed trace files
cgaebel opened this issue · 1 comments
cgaebel commented
Trace files are big and compress really well, 5x-ish in my experience. Could magic-trace write out compressed traces instead of raw traces?
Perfetto already has support for loading gzipped traces. But gzip is really slow (fingers crossed for Sapphire Rapids, but that's years from being commonplace).
As far as I can tell, we have two options here:
- teach magic-trace to write gzipped traces, or
- teach magic-trace to write
zstd'd traces, and also teach perfetto to load them.
The first option is easier, but the second option would be faster to save and load. FWIW, Tudor thinks zstd is overkill because we're so slow at decoding otherwise.
- Perfetto's gzip_trace_parser.cc
- zstd ocaml bindings
- gzip ocaml bindings
- libdeflate has the tall claim of gzipping faster that zstd