NOAA-OWP/hydrotools

Determine feasibility of _restclient's continued dependence on `aiohttp_cache_client`

aaraney opened this issue · 5 comments

See #237, specifically this comment. The long and short of it is a transitive dependency of _restclient may require _restclient.RestClient to change its public api. That is not desirable and likely not feasible. _restclient may need to change caching providers to avoid public api changes.

Just a quick update to this. I started exploring viable alternatives to aiohttp yesterday. The main criteria are:

  • Supports async requests
  • Either a caching solution exists for the library or is viable to implement and maintain
  • Support for retries and exponential backoff

httpx paired with hishel for caching are promising candidates. httpx supports injecting a custom request response making it really easy to implement caching, backoffs, and other desirable middleware (see custom transports) and also supports attaching custom middleware which could provide a way to extend custom application logging to downstream clients for example. I need to explore more the implications of moving to hishel as a caching backend. For starters, sqlite is not a supported caching backend. Only file based caches are supported at the local filesystem level. I am not overly concerned about the potential change, but I need to consider the potential ramifications for moving to a different cache storage strategy.

Ive been working through the transitive issues in aiohttp_client_cache with the author of that library and we are nearing a solution. Progress on that issue can be tracked requests-cache/aiohttp-client-cache#187 and requests-cache/aiohttp-client-cache#189.

jarq6c commented

Thanks for following up on this Austin. Just to clarify, would this issue cause nwis_client to fail when running for the first time with a "no event loop in the Main thread" error?

@jarq6c, I dont believe so. I think the program wont exit.

jarq6c commented

@jarq6c, I dont believe so. I think the program wont exit.

Follow-up. It seems to only happen when importing/using nwis_client and nwm_client_new in the same script. I'll have to dig deeper.