Example for creating numpy arrays of data
Closed this issue · 2 comments
The current examples describe how to get individual data items and inspect them, but I suspect most people are wanting to fill arrays or some other data structures from the data. On IMAP we have a utility function to handle creating xarray.Dataset objects for each apid. I suspect a utility function like this would be valuable to other missions as well, but I don't think we'd want to add large dependencies to the library. Maybe we could add optional dependencies hidden behind try/except ImportError blocks to support some of these utilities? Or the easiest would be to add examples of how to do this and let everyone copy/paste the examples.
Example code for creating a mapping of apid: Dataset
https://github.com/IMAP-Science-Operations-Center/imap_processing/blob/373f9f382260188d94366007853b673982ba1275/imap_processing/utils.py#L298-L332
As long as the code is compact enough (e.g. in its own module), hiding things behind import errors works for me. We can add the dependencies in an "extra" dependency set.
@greglucas Let's upstream your code from IMAP to support conversion to an xarray DataSet. I'd like to improve the handling of derived values if possible. I think for numeric values we can just default to float64 (Splines and Polynomials both return python floats) and for enums, we could use a Unicode String ("U") type or a Python Object "O" type. This isn't memory efficient but without knowing the full list of possible values a priori, I'm not sure how else we could approach it. We can also always include an option to only create arrays of raw values.