nsidc/earthaccess

Error downloading files to local directory

theobarnhart-USGS opened this issue ยท 6 comments

Hello this looks like a really cool package!

I followed the how to's to get a list of granules to download, but the download function gives the following error message. Any suggestions to fix this?

results = earthaccess.search_datasets(short_name = 'MYD10A1F', temporal = ('2005-07-01','2010-11-01'), bounding_box = (67.00390352, 38.80432187, 67.81368289, 39.29232868), )

Datasets found: 1

downloaded = earthaccess.download(results, local_path='./data/MYD10A1F')

Input In [40], in <cell line: 1>()
----> 1 downloaded = earthaccess.download(results, local_path='./data/MYD10A1F')

File ~/mambaforge/envs/pangeo/lib/python3.10/site-packages/earthaccess/api.py:188, in download(granules, local_path, provider, threads)
    186     granules = [granules]
    187 try:
--> 188     results = earthaccess.__store__.get(granules, local_path, provider, threads)
    189 except AttributeError as err:
    190     print(err)

File ~/mambaforge/envs/pangeo/lib/python3.10/site-packages/earthaccess/store.py:475, in Store.get(self, granules, local_path, provider, threads)
    472     local_path = Path(local_path)
    474 if len(granules):
--> 475     files = self._get(granules, local_path, provider, threads)
    476     return files
    477 else:

File ~/mambaforge/envs/pangeo/lib/python3.10/site-packages/multimethod/__init__.py:373, in multimethod.__call__(self, *args, **kwargs)
    371 """Resolve and dispatch to best method."""
    372 self.evaluate()
--> 373 func = self.dispatch(*args)
    374 try:
    375     return func(*args, **kwargs)

File ~/mambaforge/envs/pangeo/lib/python3.10/site-packages/multimethod/__init__.py:368, in multimethod.dispatch(self, *args)
    366 matches = {key for key in list(self) if isinstance(key, signature) and key.instances(*args)}
    367 matches -= {ancestor for match in matches for ancestor in match.parents}
--> 368 return self.select(types, matches)

File ~/mambaforge/envs/pangeo/lib/python3.10/site-packages/multimethod/__init__.py:352, in multimethod.select(self, types, keys)
    350 if len(funcs) == 1:
    351     return funcs.pop()
--> 352 raise DispatchError(f"{self.__name__}: {len(keys)} methods found", types, keys)

DispatchError: ('_get: 0 methods found', (<class 'earthaccess.store.Store'>, <class 'list'>, <class 'pathlib.PosixPath'>, <class 'NoneType'>, <class 'int'>), set())```

Hey, thanks for the report! Can you please report which version of earthaccess you have installed (earthaccess.__version__)?

The earthaccess.search_datasets function returns "collections", and you can't feed those to earthaccess.download. You probably want the earthaccess.search_data function (see docs), which returns "granules". Those you can download.

@mfisher87 If I'm right on this, perhaps we could provide a more helpful error message?

Great catch, Ian! We'd discussed using a tool like @pydantic.validate_call to add runtime validation to our top-level API functions to catch things like this early and give the user more useful error message. I found the open issue for that: #420

@theobarnhart-USGS apologies for the poor messaging. Can you let us know if Ian's suggestion gets you back on track? Feel free to close this issue if so, and we will track the larger work needed to improve the error message in #420.

Thank you @itcarroll! My brain was not seeing search_data vs search_datasets. Things seem to be working much better now.

Thank you @itcarroll! My brain was not seeing search_data vs search_datasets. Things seem to be working much better now.

You're not alone, I made the same mistake when I read this issue ๐Ÿ˜† Maybe we should think about the the top-level API; are these good names for these functions? Why not use common terminology like search_collections and search_granules? "Dataset" is often used to refer to a single data file. ๐Ÿค”

Thank you @itcarroll! My brain was not seeing search_data vs search_datasets. Things seem to be working much better now.

You're not alone, I made the same mistake when I read this issue ๐Ÿ˜† Maybe we should think about the the top-level API; are these good names for these functions? Why not use common terminology like search_collections and search_granules? "Dataset" is often used to refer to a single data file. ๐Ÿค”

I agree on the need for better function names here. I like search_collections and search_granules, and I'd like to add find_collections and find_granules as another pair of names for consideration.