TRI-AMDD/beep

Automatically serialize BEEPDatapaths to .json.gz, not json

Closed this issue · 0 comments

New BEEPDatapaths are a bit bigger than older ProcessedCyclerRuns since they include complete data, including the raw data.

Running the TestMaccorDatapath.test_get_diagnostic and checking that file:

processed_diagnostic.json       249.3MB

Compressing it

processed_diagnostic.json.gz    18.6MB

Compressing and decompressing even large files is computationally trivial in comparison with interpolation. It seems like the default surely should be compressing these saved files.