Compatibility issues with xsdata==21.12 and h5py==2.9.0

[kspaceKelvin]

There appears to be an backend dependency on xsdata 22.2 that conflicts with our pinned 21.12 version.

This can be reproduced with

docker run -it --rm python:3.10.2-slim  /bin/bash
apt-get update && apt-get install --no-install-recommends -y git
mkdir -p /opt/code && cd /opt/code && git clone https://github.com/ismrmrd/ismrmrd-python.git && cd /opt/code/ismrmrd-python
pip3 install --no-cache-dir .

which produces the error:

ERROR: Some build dependencies for file:///opt/code/ismrmrd-python conflict with the backend dependencies: xsdata==22.2 is incompatible with xsdata[cli]==21.12.

There also appears to be an issue with h5py==2.9.0 on Debian-based images (like python above):

docker run -it --rm python:3.10.2-slim  /bin/bash
apt-get update && apt-get install --no-install-recommends -y gcc
pip install h5py==2.9.0

which produces error:

    In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/syslimits.h:7,
                     from /usr/lib/gcc/x86_64-linux-gnu/10/include/limits.h:34,
                     from /usr/local/include/python3.10/Python.h:11,
                     from /tmp/pip-install-zqc5v665/h5py_816fe9a0a47c42d2863dea3f6f7d4c14/h5py/defs.c:47:
    /usr/lib/gcc/x86_64-linux-gnu/10/include/limits.h:195:15: fatal error: limits.h: No such file or directory
      195 | #include_next <limits.h>  /* recurse down to the real one */
          |               ^~~~~~~~~~
    compilation terminated.
    error: command '/usr/bin/gcc' failed with exit code 1

[hansenms]

I am not quite sure what the build situation is in that slim image is, but:

docker run -it --rm python:3.10.2-slim  /bin/bash
pip3 install ismrmrd==1.9.6

works and I have validated, that the ismrmrd package imports and works in the slim image. We also build and install with those dependencies on conda packages, etc. So I think they can work.

That said, I am not opposed to moving dependencies forward, I think it is a good idea such as suggested in #48, but I don't think it necessarily has to be motivated by this.

[hansenms]

So a few different issues going on here. On that slim image, if you try to do pip3 install h5py==2.9.0, it will actually build the package, which requires a bunch of dependencies (in the right places) like hdf5.h so that is not guaranteed to succeed. Not totally clear what the steps are to get those build dependencies in the right place on that image. The other issue is that you need to install xsdata[cli]==21.12 to get that requirement pinned correctly before doing the install. Otherwise, some background dependency will cause a problem. So to work around those two issues, you can just install whatever version of h5py will come pre-built and pull that in with xsdata before building. This works:

First start the container:

docker run -it --rm python:3.10.2-slim  /bin/bash

apt-get update && apt-get install -y git
mkdir -p /opt/code && cd /opt/code && git clone https://github.com/ismrmrd/ismrmrd-python
cd ismrmrd-python/
pip3 install xsdata[cli]==21.12 h5py
python3 setup.py install

Again, I am all for updating versions, etc. But I do not think that this is the issue that should make us do it.

[kspaceKelvin]

Agreed that there are two different issues here.

For xsdata, I think that the backend dependency conflict is an actual problem. While installing a pip release is a good idea and manually installing xsdata 21.12 before running setup.py is a good workaround, I think it's reasonable for someone to expect that they can run setup.py wtihout needing to manually specify an xsdata version beforehand.

As for h5py, it's not clear to me why pip installing 2.9.0 builds the package and 3.6.0 doesn't (both on slim). In practice, I use the setup.py which specifies h5py>=2.3, so it'll install the new version anyways. The release notes for 3.0 note some performance improvements that could benefit us for large datasets, but there are also potentially some breaking changes that need to be checked.

Neither are blocking issues, but as you said, it would be good to update versions it's not too much effort.

[hansenms]

Fixed in #48