zarr-developers/zarr-python

Provide a more informative error when using `open_consolidated`

andersy005 opened this issue · 1 comments

Problem description

When using open_consolidated on stores that we were created without consolidating the metadata, the user gets a KeyError. I think for folks familiar with zarr, it's easy to diagnose what's going on. However, for new users, the error can be too cryptic (especially when using xarray which exposes zarr.open_consolidated() via a consolidated keyword argument).

I'm wondering if it would be reasonable (if feasible) to provide a more informative error. I am imagining something along these lines:

FileNotFoundError: Couldn't find consolidated metadata file: '.zmetadata'. Try opening the store with `zarr.open(...)`

instead of

KeyError: '.zmetadata'

I don't know if my suggestion is generic enough to warrant a change in zarr. I'm happy to bring up the issue upstream in xarray.

Minimal, reproducible code sample, a copy-pastable example if possible

In [43]: import zarr

In [44]: import numpy as np

In [45]: store = zarr.DirectoryStore("dataset.zarr")

In [46]: ds = zarr.array(np.arange(10))

In [47]: zarr.save(store, ds)

In [48]: zarr.open(store)
Out[48]: <zarr.core.Array (10,) int64>
In [49]: zarr.open_consolidated(store)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-49-f77b4b56ec4c> in <module>
----> 1 zarr.open_consolidated(store)

/glade/work/abanihi/opt/miniconda/envs/playground/lib/python3.8/site-packages/zarr/convenience.py in open_consolidated(store, metadata_key, mode, **kwargs)
   1176 
   1177     # setup metadata store
-> 1178     meta_store = ConsolidatedMetadataStore(store, metadata_key=metadata_key)
   1179 
   1180     # pass through

/glade/work/abanihi/opt/miniconda/envs/playground/lib/python3.8/site-packages/zarr/storage.py in __init__(self, store, metadata_key)
   2767 
   2768         # retrieve consolidated metadata
-> 2769         meta = json_loads(store[metadata_key])
   2770 
   2771         # check format of consolidated metadata

/glade/work/abanihi/opt/miniconda/envs/playground/lib/python3.8/site-packages/zarr/storage.py in __getitem__(self, key)
    844             return self._fromfile(filepath)
    845         else:
--> 846             raise KeyError(key)
    847 
    848     def __setitem__(self, key, value):

KeyError: '.zmetadata'

Version and installation information

Please provide the following:

  • Value of zarr.__version__: 2.8.3
  • Value of numcodecs.__version__: 0.7.3
  • Version of Python interpreter: 3.8.8
  • Operating system: Linux
  • How Zarr was installed: conda

closed by #2113