incompatible with the netCDF4 stack
xiki-tempula opened this issue · 15 comments
I think there might be some incompatibility issue with the most recent netCDF4 stack. I have pulled in the latest openmmtools conda package and run it on a script that used to run. However, I'm getting some netCDF4 related issues.
Please find the conda list, and input files attached.
Traceback (most recent call last):
File "/sharedfs-home/zwu/.conda/envs/openmmtools/lib/python3.10/site-packages/openmmtools/multistate/multistatereporter.py", line 1800, in _write_dict
nc_variable = self._resolve_nc_path(path, storage_name)
File "/sharedfs-home/zwu/.conda/envs/openmmtools/lib/python3.10/site-packages/openmmtools/multistate/multistatereporter.py", line 1465, in _resolve_nc_path
return nc_group.variables[path_split[-1]]
KeyError: 'options'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/sharedfs-home/zwu/C/vac/prod/lambda_0/OpenMMTools_script.py", line 56, in <module>
simulation.create(
File "/sharedfs-home/zwu/.conda/envs/openmmtools/lib/python3.10/site-packages/openmmtools/multistate/multistatesampler.py", line 606, in create
self._initialize_reporter()
File "/sharedfs-home/zwu/.conda/envs/openmmtools/lib/python3.10/site-packages/mpiplus/mpiplus.py", line 271, in _wrapper
return run_single_node(rank, task, *args, **kwargs)
File "/sharedfs-home/zwu/.conda/envs/openmmtools/lib/python3.10/site-packages/mpiplus/mpiplus.py", line 220, in run_single_node
result = task(*args, **kwargs)
File "/sharedfs-home/zwu/.conda/envs/openmmtools/lib/python3.10/site-packages/openmmtools/multistate/multistatesampler.py", line 1182, in _initialize_reporter
self._store_options()
File "/sharedfs-home/zwu/.conda/envs/openmmtools/lib/python3.10/site-packages/openmmtools/multistate/multistatesampler.py", line 1256, in _store_options
self._reporter.write_dict('options', self.options)
File "/sharedfs-home/zwu/.conda/envs/openmmtools/lib/python3.10/site-packages/openmmtools/multistate/multistatereporter.py", line 1104, in write_dict
self._write_dict(path, data, storage_name=storage_name)
File "/sharedfs-home/zwu/.conda/envs/openmmtools/lib/python3.10/site-packages/openmmtools/multistate/multistatereporter.py", line 1812, in _write_dict
nc_variable = storage_nc.createVariable(path, variable_type,
File "src/netCDF4/_netCDF4.pyx", line 2945, in netCDF4._netCDF4.Dataset.createVariable
File "src/netCDF4/_netCDF4.pyx", line 4184, in netCDF4._netCDF4.Variable.__init__
File "src/netCDF4/_netCDF4.pyx", line 2014, in netCDF4._netCDF4._ensure_nc_success
RuntimeError: NetCDF: Filter error: bad id or parameters or duplicate filter
I see the same error in a fresh conda environment (mamba create --name test python=3.10 openmmtools mpi4py).
THanks for reporting this issue. I have encountered the same behavior, I have worked around this by using hdf5=1.12.x instead of the latest 1.14.x in my environments. This has been creating some issues for other projects as well, as far as I can tell. My suggestion is that you try to pin the 1.12.x version for hdf5 in your environments. We will be doing that in our conda-forge feedstock soon.
@ijpulidos Thanks for the help. I have created a new env with mamba create --name test python=3.10 openmmtools mpi4py hdf5=1.12. However, this seems to be giving the same error.
@xiki-tempula Oh, yes, apparently that's not enough. I now remember that we applied a fix in #654 that will be included in our next release (hopefully soon, we are finishing some details for its release).
For the moment you could try installing openmmtools from the main branch using something like the following:
- Sadly when you install with pip and conda things can get mixed. I suggest to uninstall current openmmtools with
conda remove --force openmmtools - install with
pipfrom main branch:pip install git+https://github.com/choderalab/openmmtools.git
Alternatively, you could do a local install, that is cloning the repo and running pip install . or similar.
@ijpulidos Thanks. Weirdly, that this was working last week. I wonder if you know what has happened in the past few days? Could I somehow use the old version until the new version has been released? Thank you.
@xiki-tempula Actually I think you could if you explicitly install netcdf4=1.5.8 along side the current released version of openmmtools (that is, 0.21.5). That is working on my side.
I don't really know the reasons but we have been also facing weird behavior on our CI, and I have the feeling it is related to some subtle changes in the netcdf and hdf packages and builds upstream.
This solved my problem Thanks
@ijpulidos Hi, sorry for troubling you again. I wonder if the current version of conda openmmtools would pin the netcdf version?
@xiki-tempula Yes, we already pinned the netcdf4/hdf5 dependencies in https://anaconda.org/conda-forge/openmmtools/files/modal/info/6418e634f21024c0d026d840 but I've found out that since there's a previous build without these, conda/mamba could opt out to fall back to that one. You should try to use the pyhd8ed1ab_1 build, you can specify it when doing the install. Apologies for the inconveniences but this is something that changed upstream (in the netcdf world).
Soon we will be releasing a new openmmtools version that will partly solve this problem (because there's more than one breaking change upstream).
@ijpulidos Thank you very much for the help. I wonder if it is possible to deprecate the pyhd8ed1ab_0 build and only preserve the pyhd8ed1ab_1 build? It seems that the old build is always being pulled in.
@xiki-tempula I just created conda-forge/admin-requests#700 to mark that build of the package as broken. Hopefully it will get removed from conda-forge soon.
@ijpulidos Thank you very much for the help.
@xiki-tempula that build has been marked as broken in https://anaconda.org/conda-forge/openmmtools/files?channel=broken
Even so, I recommend using mamba when installing openmmtools, that is instead of conda, because I have seen conda not getting the most up to date version of it when you try to install it with something like conda create -n test-env -c conda-forge openmmtools. Whereas changing conda with mamba or micromamba in that command seems to work just as expected. I hope this helps.
@ijpulidos Thank you very much. I only use mamba as otherwise, it is just too slow.
Thank you. I can verify that it works perfectly now!