NREL/buildstockbatch

Support msgPack format

rajeee opened this issue · 4 comments

rajeee commented

Is your feature request related to a problem? Please describe.
ResStock supports a variety of output format for timeseries and annual results. Currently, BSB assumes that the timeseries results will be in csv format and annual results will be in json. However, for increased precision, msgPack format is better suited - especially to avoid rounding errors in timeseries file.

Describe the solution you'd like
The solution I am seeking is for BSB to be able to deal with msgPack output files in addition to standard csv/json files.

Describe alternatives you've considered
This could be somewhat fixed by increasing the number of decimals used in the results_timeseries.csv file, but an even better fix is to do away with csv files altogether and work in binary file format (such as msgPack) which can preserve the full precision of the EnergyPlus output.

Additional context
We have noticed up to 20% difference (for some enduses) in the annual savings calculated using the annual results vs calculated using the timeseries. The root cause of this issue was the 3 decimal places rounding we were doing for the results_timeseries.csv file causing a systematic error - especially for idle energy consumption periods. This error % is magnified when doing savings calculation.

nmerket commented

We will want to maintain backwards compatibility to read the existing csv files for older versions of ResStock and ComStock. So the logic would be:

  1. Look for msgpack file and read it.
  2. If not msgpack, look for csv and read it (as is currently done).
  • 👍1
shorowit commented

Currently, BSB assumes that the timeseries results will be in csv format and annual results will be in json.

It probably doesn't matter, but that doesn't sound right. The requested output format (csv, json, msgpack) in ReportSimulationOutput currently applies to both annual and timeseries outputs.

rajeee commented

Currently, BSB assumes that the timeseries results will be in csv format and annual results will be in json.

It probably doesn't matter, but that doesn't sound right. The requested output format (csv, json, msgpack) in ReportSimulationOutput currently applies to both annual and timeseries outputs.

BSB currently reads the data_point_out.json file for the annual results and I thought that was the only source for annual result. Looks like there is now also the results_annual.csv file - I wasn't aware of.

So, I guess, the results_annual.csv is impacted by the output_format and data_point_out.json will always exist as a json? @shorowit

shorowit commented

Ah, I forgot about data_point_out.json. Yes, that sounds right.