Automatically serialize BEEPDatapaths to .json.gz, not json
Closed this issue · 0 comments
ardunn commented
New BEEPDatapaths are a bit bigger than older ProcessedCyclerRuns since they include complete data, including the raw data.
Running the TestMaccorDatapath.test_get_diagnostic
and checking that file:
processed_diagnostic.json 249.3MB
Compressing it
processed_diagnostic.json.gz 18.6MB
Compressing and decompressing even large files is computationally trivial in comparison with interpolation. It seems like the default surely should be compressing these saved files.