clessig/atmorep

How do I read the ZipStore files comprising the predictions, targets, etc.?

Closed this issue · 3 comments

dancivitarese commented

I tried to read the content as:

z = zarr.ZipStore('output/id1jh2qvrx/results_id1jh2qvrx_epoch00000_pred.zarr')

It creates a storage like <zarr.storage.ZipStore at ...>, but I don't know how to read the values, groups, etc.

iluise commented

Hi @dancivitarese,
for a quick inspection you can retrieve the fields with:

root =  zarr.open_group(z)
root.tree() ---> plots the internal structure
root['field/sample=XXXXX/data'][:]

For a more in depth analysis we have written an interface to easily retrieve the data as xarrays. it is still being finalised, but you can find some examples here: https://github.com/iluise/atmorep_analysis/tree/main
if you ran in global_forecast mode I suggest you to look at forecasting/forecasting_zarr.py, while if you ran in BERT mode you can look at trainings/.
The interface in itself is in utils/read_atmorep_data.py.

Best,
Ilaria

  • 👍1
dancivitarese commented

Thank you, @iluise for such quick response. The first option already solved my issue, but I will also test the other repo.

iluise commented

Great.
just one thing I forgot to add: if you ran with main you might get incorrect outcomes if you plot through lat/lons coordinates. please use the branch iluise-fix-dims for now.
best,

Ilaria

  • 👍1