Add support for binary data files
Closed this issue · 8 comments
Description
Add support for binary data files
The eva package is going to be used to generate plots in the next generation of DA monitoring. The current DA monitoring packages use binary files as an interim storage mechanism. Extend eva to read these files and format the data into the correct xarray structures for eva image generation. These files will be added to eva in order to accomplish this:
- eva/src/eva/data/mon_data_space.py This will be similar to gsi_obs_space.py, but capable of handling binary data files and corresponding control files as input.
- eva/src/eva/tests/config/testMonDataSpaceHirs4Metop-A.yaml A yaml file for testing mon_data_space.py.
- eva/src/eva/tests/data/time.hirs4_metop-a.ctl A GrADS type control file describing the binary file format as currently produced by the RadMon DA monitor.
- eva/src/eva/tests/data/time.hirs4_metop-a.*.ieee_d At least two binary data files as currently produced by the RadMon DA monitor. More than one file will be necessary to confirm correct handling of multi-cycle data sets.
Requirements
Given a yaml file with data inputs in binary format, produce data plots similar to those currently generated by the RadMon DA monitor.
Acceptance Criteria (Definition of Done)
Requirements above are met.
Dependencies
None
@EdwardSafford-NOAA will work on this but I can't assign myself to it. Maybe @CoryMartin-NOAA can do that?
Progress to date: Able to read control file and multiple binary data files (from RadMon) and load into a dataset. Have figured out how to adjust dataset to fit with requested elements in yaml file including dropping vars and unrequested channels (a coordinate).
Will next add the ability to drop specific regions. (Note that "regions" here is something of a misnomer; they are actually surface types. Region 1 is all, and that's primarily what's of interest.)
Closed with PR #70 .
@EdwardSafford-NOAA best to not close the issue until the PR is merged, and putting closes #68 in the PR will do so automatically upon merge!
Sorry, I must have missed something. I got email notification that PR #70 had been merged and saw this issue was still open. Thought I was tidying up by closing this. I was ignorant of the closes phasing for automatic closure; so noted.
ah! #70 is not yet merged, I merged in develop into that branch (to get it up to date) but that branch is not yet merged to develop (too many merges in that sentence!)
Ah, I guess I read it too quickly. So noted again. I'll stop assuming I already know what I'm reading.
No worries, the PR has been merged now, thanks for this work!