Provide a more informative error when using `open_consolidated`
andersy005 opened this issue · 1 comments
andersy005 commented
Problem description
When using open_consolidated
on stores that we were created without consolidating the metadata, the user gets a KeyError
. I think for folks familiar with zarr, it's easy to diagnose what's going on. However, for new users, the error can be too cryptic (especially when using xarray which exposes zarr.open_consolidated()
via a consolidated
keyword argument).
I'm wondering if it would be reasonable (if feasible) to provide a more informative error. I am imagining something along these lines:
FileNotFoundError: Couldn't find consolidated metadata file: '.zmetadata'. Try opening the store with `zarr.open(...)`
instead of
KeyError: '.zmetadata'
I don't know if my suggestion is generic enough to warrant a change in zarr. I'm happy to bring up the issue upstream in xarray.
Minimal, reproducible code sample, a copy-pastable example if possible
In [43]: import zarr
In [44]: import numpy as np
In [45]: store = zarr.DirectoryStore("dataset.zarr")
In [46]: ds = zarr.array(np.arange(10))
In [47]: zarr.save(store, ds)
In [48]: zarr.open(store)
Out[48]: <zarr.core.Array (10,) int64>
In [49]: zarr.open_consolidated(store)
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-49-f77b4b56ec4c> in <module>
----> 1 zarr.open_consolidated(store)
/glade/work/abanihi/opt/miniconda/envs/playground/lib/python3.8/site-packages/zarr/convenience.py in open_consolidated(store, metadata_key, mode, **kwargs)
1176
1177 # setup metadata store
-> 1178 meta_store = ConsolidatedMetadataStore(store, metadata_key=metadata_key)
1179
1180 # pass through
/glade/work/abanihi/opt/miniconda/envs/playground/lib/python3.8/site-packages/zarr/storage.py in __init__(self, store, metadata_key)
2767
2768 # retrieve consolidated metadata
-> 2769 meta = json_loads(store[metadata_key])
2770
2771 # check format of consolidated metadata
/glade/work/abanihi/opt/miniconda/envs/playground/lib/python3.8/site-packages/zarr/storage.py in __getitem__(self, key)
844 return self._fromfile(filepath)
845 else:
--> 846 raise KeyError(key)
847
848 def __setitem__(self, key, value):
KeyError: '.zmetadata'
Version and installation information
Please provide the following:
- Value of
zarr.__version__
: 2.8.3 - Value of
numcodecs.__version__
: 0.7.3 - Version of Python interpreter: 3.8.8
- Operating system: Linux
- How Zarr was installed: conda