Allow newer versions of xarray
Closed this issue · 3 comments
heikoklein commented
Is your feature request related to a problem? Please describe.
xarray is currently restricted to <=2022.10. pyaerocom should work with latest releases of xarray.
This issue is related to #1066 .
Describe the solution you would like to see
A clear and concise description of what you want to happen.
- xarray should only depend on a minimum, but not on a maximum version, e.g.
xarray >= 2020.10
thorbjoernl commented
Upgrading to the most recent release of xarray (2024.3.0) runs into a Segmentation Fault:
Fatal Python error: Segmentation fault
[...]
Current thread 0x00007f74d81a4140 (most recent call first):
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/xarray/backends/file_manager.py", line 217 in _acquire_with_cache_info
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/xarray/backends/file_manager.py", line 199 in acquire_context
File "/modules/rhel8/user-apps/fou-modules/python/python_3.11.3/lib/python3.11/contextlib.py", line 137 in __enter__
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 412 in _acquire
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 418 in ds
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 356 in __init__
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 409 in open
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/xarray/backends/netCDF4_.py", line 646 in open_dataset
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/xarray/backends/api.py", line 573 in open_dataset
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/xarray/backends/api.py", line 749 in open_dataarray
File "/lustre/storeB/users/thlun8736/python/pyaerocom/pyaerocom/colocateddata.py", line 1287 in read_netcdf
File "/lustre/storeB/users/thlun8736/python/pyaerocom/pyaerocom/colocateddata.py", line 1361 in open
File "/lustre/storeB/users/thlun8736/python/pyaerocom/pyaerocom/colocateddata.py", line 135 in validate_data
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pydantic/main.py", line 175 in __init__
File "/lustre/storeB/users/thlun8736/python/pyaerocom/tests/fixtures/collocated_data.py", line 253 in <lambda>
File "/lustre/storeB/users/thlun8736/python/pyaerocom/tests/fixtures/collocated_data.py", line 265 in coldata
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/fixtures.py", line 913 in call_fixture_func
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/fixtures.py", line 1132 in pytest_fixture_setup
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 102 in _multicall
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 119 in _hookexec
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 501 in __call__
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/fixtures.py", line 1079 in execute
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/fixtures.py", line 647 in _compute_fixture_value
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/fixtures.py", line 570 in _get_active_fixturedef
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/fixtures.py", line 541 in getfixturevalue
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/fixtures.py", line 687 in _fillfixtures
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/python.py", line 1775 in setup
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/runner.py", line 512 in setup
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/runner.py", line 158 in pytest_runtest_setup
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 102 in _multicall
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 119 in _hookexec
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 501 in __call__
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/runner.py", line 240 in <lambda>
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/runner.py", line 340 in from_call
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/runner.py", line 239 in call_and_report
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/runner.py", line 128 in runtestprotocol
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/runner.py", line 115 in pytest_runtest_protocol
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 102 in _multicall
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 119 in _hookexec
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 501 in __call__
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/main.py", line 364 in pytest_runtestloop
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 102 in _multicall
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 119 in _hookexec
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 501 in __call__
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/main.py", line 339 in _main
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/main.py", line 285 in wrap_session
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_callers.py", line 102 in _multicall
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_manager.py", line 119 in _hookexec
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 501 in __call__
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 174 in main
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 197 in console_main
File "/lustre/storeB/users/thlun8736/python/pyaerocom/venv/bin/pytest", line 8 in <module>
This happens at this call to xr.open_dataarray().
Setting cache=False prevents the Segmentation Fault so I think it may be related to this open issue on the xarray repo, in particular this comment.
thorbjoernl commented
Another fix appears to be engine="h5netcdf" also appears to resolve it. This is consistent with the table here.
thorbjoernl commented
Fixed in #1150