3.7 Tests failing: xarray EntryPoints has no attribute get

[aaraney]

The 3.7 action runner started failing on my submission of #218. A nwm_client_new test, test_get_dataset is the source. Here is the trackback from the runner:

python/nwm_client_new/tests/test_NWMFileProcessor.py F
... omitted for viewing purposes ...

    def test_get_dataset():
>       ds = NWMFileProcessor.get_dataset(input_directory)

python/nwm_client_new/tests/test_NWMFileProcessor.py:8: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/hydrotools/nwm_client_new/NWMFileProcessor.py:59: in get_dataset
    ds = xr.open_mfdataset(file_list, engine="netcdf4")
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/xarray/backends/api.py:908: in open_mfdataset
    datasets = [open_(p, **open_kwargs) for p in paths]
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/xarray/backends/api.py:908: in <listcomp>
    datasets = [open_(p, **open_kwargs) for p in paths]
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/xarray/backends/api.py:[48](https://github.com/NOAA-OWP/hydrotools/actions/runs/5008064198/jobs/8975487805?pr=218#step:5:49)1: in open_dataset
    backend = plugins.get_backend(engine)
/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/xarray/backends/plugins.py:161: in get_backend
    engines = list_engines()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    @functools.lru_cache(maxsize=1)
    def list_engines():
>       entrypoints = entry_points().get("xarray.backends", ())
E       AttributeError: 'EntryPoints' object has no attribute 'get'

/opt/hostedtoolcache/Python/3.7.16/x[64](https://github.com/NOAA-OWP/hydrotools/actions/runs/5008064198/jobs/8975487805?pr=218#step:5:65)/lib/python3.7/site-packages/xarray/backends/plugins.py:105: AttributeError

I've not looked into the xarray source to see where entry_points() is coming from, but it seems pretty safe at this point to assume this is an xarray bug.

Version information:

OS: Ubuntu 22.0 LTS
Runner Image: ubuntu-22.04:20230507.1
Python: 3.7.16
xarray: 0.20.2

[aaraney]

I just re-ran the runners. What changed?

• On submission of #218 the 3.9 runner did not run because 3.7 failed and 3.9 had not started yet. All tests passed in 3.9.

More and more this is looking like an issue in xarray. Below are the xarray versions for each python runner:

• 3.7: xarray-0.20.2
• 3.8: xarray-2023.1.0
• 3.9: xarray-2023.4.2

All tests passed in the 3.8 and 3.9 runners, but 3.7 is failing.

[aaraney]

So xarray dropped support for 3.7 in v0.21.0 and dropped support for 3.8 in v2023.02.0. That's odd to me that the 3.7 runner is not picking up v0.21.0. Looking into that now.

[aaraney]

Got slightly side tracked and need to work on something else for a bit. Ill come back to this either later this afternoon or tomorrow morning.

[aaraney]

The issue is importlib.metadata. In short, importlib.metadata was added in python 3.8. importlib-metadata was introduced for backwards compatibility with python < 3.8. xarray is using the importlib.metadata,entry_points() function which per the documentation returns

a dictionary of all entry points, keyed by group.

The xarray function that is calling this function tries to use the get method on the dictionary. In importlib-metadata versions<= 4.13.0, this works. A breaking change was introduced in importlib-metadata==5 that changes what is returned by the entry_points() function.

[aaraney]

The two obvious options are:

1. drop nwm_client_new's python <= 3.7 support
2. pin importlib-metadata<=4.13.0 in nwm_client_new

@jarq6c thoughts? Im pretty impartial on this one. I think pinning importlib-metadata as a bandaid fix until we fully drop python<=3.7 support for all hydrotools might be the easiest.

[aaraney]

repeat of #211. closing.