Check_availability returning NoneType in 0.16.0

[TDFSE]

What happened?

Checking availability of various resolutions in USGS 3DEP database to allow quick selection of highest-resolution data.

What did you expect to happen?

Return a response in the form of:

{'1m': True, '3m': True, '5m': False, '10m': True, '30m': True, '60m': False, 'topobathy': False}

Minimal Complete Verifiable Example

import py3dep

bbox = (-82.536901, 34.541368, -82.508202, 34.563319)

avail = py3dep.check_3dep_availability(bbox=bbox)

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

Traceback (most recent call last):
  File "~\poly-val\_test_del.py", line 5, in <module>
  File "~\AppData\Local\pypoetry\Cache\virtualenvs\poly-val-L9A71YTm-py3.10\lib\site-packages\py3dep\py3dep.py", line 664, in check_3dep_availability
    avail = {
  File "~\AppData\Local\pypoetry\Cache\virtualenvs\poly-val-L9A71YTm-py3.10\lib\site-packages\py3dep\py3dep.py", line 665, in <dictcomp>
    res: "Failed" if "error" in r else bool(r.get("count")) for res, r in zip(res_layers, resps)
TypeError: argument of type 'NoneType' is not iterable

Anything else we need to know?

This function is working using py3dep version 0.14.0

Environment

SYS INFO

commit: bff6686d5dcbb3c778832c2f298a12e27bfe4531
python: 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 154 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('English_United States', '1252')

PACKAGE VERSION

async-retriever 0.16.0
pygeoogc 0.16.0
pygeoutils 0.16.0
py3dep 0.16.0
pynhd N/A
pygridmet N/A
pydaymet N/A
hydrosignatures N/A
pynldas2 N/A
pygeohydro N/A
aiohttp 3.9.1
aiohttp-client-cache 0.10.0
aiosqlite 0.19.0
cytoolz 0.12.2
ujson 5.9.0
defusedxml 0.7.1
joblib 1.3.2
multidict 6.0.4
owslib 0.29.3
pyproj 3.6.1
requests 2.31.0
requests-cache 1.1.1
shapely 2.0.2
url-normalize 1.4.3
urllib3 2.1.0
yarl 1.9.4
geopandas 0.12.2
netcdf4 1.6.5
numpy 1.24.4
rasterio 1.3.9
rioxarray 0.14.1
scipy 1.11.4
xarray 2023.12.0
click 8.1.7
pyflwdir N/A
networkx 3.2.1
pyarrow N/A
folium N/A
h5netcdf 1.3.0
matplotlib 3.8.2
pandas 2.1.4
numba 0.57.1
bottleneck N/A
py7zr N/A
pyogrio N/A

[cheginit]

Thanks for reporting the issue!

I think the web service was down when you were testing it, since I just ran the code and it worked:

Can you try again?

I should improve the error message to make it clear the issue is with the service, not py3dep.

[TDFSE]

It seems this was part of the issue. 3 of 4 for successful responses, but the response time for this particular function going from 0.14.0 to 0.16.0 was also substantially slower. Oddly, now that it is working, response time has increased - first run the availability call took 15 seconds (versus <1s in 0.14.0 in the control run), but subsequent calls are now <1s in 0.16.0.

Seems to support that it was an issue on the web service side even from a response time view. Thank you for your help!

[cheginit]

For web calls, the response time, for the most part, depends on the web service capacity. The reason for the second time being faster, is caching. By default, HyRiver packages cache web requests/responses for a week, so you'd get the same response if it exists in the cache database, without making a web call. Essentially, the second time that you ran the function, you measured the runtime of the function itself, since it's not waiting for a response from the web service. It just queries the response from the local cache database in ./cache/aiohttp_cache.sqlite or as defined by HYRIVER_CACHE_NAME environmental variable. By the way I released Py3DEP 0.16.1, that addresses the issue of not returning Failed and throwing an exception.