[BUG] conda-forge installation doesn't come with brainglobe_atlasapi
alessandrofelder opened this issue · 12 comments
Describe the bug
Running brainreg CLI on sample-data in fresh conda environment with brainreg from conda-forge fails with brainglobe_atlasapi not found (at least on M2)
To Reproduce
Run this script (adapt local paths to conda/mamba and to sample data)
#!/usr/bin/env bash -l
mamba env remove -n brainreg-conda-forge
mamba create -n brainreg-conda-forge python=3.10 -y
mamba activate brainreg-conda-forge
mamba install -c conda-forge brainreg -y
which python
which pip
which brainreg
brainreg ~/Downloads/brainreg-test-data/input/brain/ ~/Downloads/brainreg-metapackage-output/ -v 5 2 2 --orientation sal --atlas allen_mouse_25umand observe this output:
/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/bin/python
/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/bin/pip
/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/bin/brainreg
Traceback (most recent call last):
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/bin/brainreg", line 6, in <module>
from brainreg.core.cli import main
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/cli.py", line 14, in <module>
from brainreg.core.main import main as register
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/main.py", line 4, in <module>
from brainglobe_atlasapi import BrainGlobeAtlas
Expected behaviour
brainreg CLI terminates successfully and gives at least an approximately reasonable registration output.
Log file
Please attach the log file if relevant (e.g. output/cellfinder_2019-12-17_13-29-46.log)
Screenshots
\
Computer used (please complete the following information):
- OS: macOS
- Version: Sonoma 14.5
- Hardware specs: M2
Additional context
\
Just so we're not going completely insane, what's the output of running pip list after mamba install -c conda-forge brainreg -y? Presumably brainglobe-atlasapi is missing from the list.
mamba list should also hopefully give the same indication that it's missing.
Yes, output of mamba list | grep brain is:
# packages in environment at /Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge:
brainglobe-napari-io 0.3.3 pyhd8ed1ab_0 conda-forge
brainglobe-segmentation 1.2.2 pyh9208f05_0 conda-forge
brainglobe-space 1.0.2 pyhd8ed1ab_0 conda-forge
brainglobe-utils 0.4.0 pyhd8ed1ab_0 conda-forge
brainreg 1.0.7 pyhd8ed1ab_0 conda-forge
If I add pip install brainglobe-atlasapi to the bash script, right after the installation of brainreg from conda-forge, I get an error with pip's dependency resolver which references brainreg==0.0.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
brainreg 0.0.0 requires brainglobe-utils>=0.4.2, but you have brainglobe-utils 0.4.0 which is incompatible.
Successfully installed PyMCubes-0.1.4 SimpleITK-2.3.1 brainglobe-atlasapi-2.0.6 contourpy-1.2.1 cycler-0.12.1 fonttools-4.53.0 loguru-0.7.2 matplotlib-3.9.0 pyparsing-3.1.2 vedo-2024.5.1 vtk-9.3.0 xmltodict-0.13.0
and mamba list | grep brain gives
# packages in environment at /Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge:
brainglobe-atlasapi 2.0.6 pypi_0 pypi
brainglobe-napari-io 0.3.3 pyhd8ed1ab_0 conda-forge
brainglobe-segmentation 1.2.2 pyh9208f05_0 conda-forge
brainglobe-space 1.0.2 pyhd8ed1ab_0 conda-forge
brainglobe-utils 0.4.0 pyhd8ed1ab_0 conda-forge
brainreg 1.0.7 pyhd8ed1ab_0 conda-forge
Further down in the script, the brainreg CLI fails in a different way:
Stack trace
``` INFO:root:Starting affine registration INFO 2024-06-13 11:52:10 AM - INFO - MainProcess run.py:103 - Starting affine run.py:103 registration Traceback (most recent call last): File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainglobe_utils/general/system.py", line 302, in safe_execute_command subprocess.check_call( File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/bin/nifty_reg/osX/reg_aladin -ln 6 -lp 5 -flo "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/brain_filtered.nii" -ref "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/downsampled_filtered.nii" -aff "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine_matrix.txt" -res "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine_registered_atlas_brain.nii" -omp 4' returned non-zero exit status 127.During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/backend/niftyreg/registration.py", line 67, in register_affine
safe_execute_command(
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainglobe_utils/general/system.py", line 314, in safe_execute_command
raise SafeExecuteCommandError(
brainglobe_utils.general.system.SafeExecuteCommandError:
Process failed:
/bin/sh: /Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/bin/nifty_reg/osX/reg_aladin: No such file or directory
please read the logs at /Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine.log and /Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine.err
command: /Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/bin/nifty_reg/osX/reg_aladin -ln 6 -lp 5 -flo "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/brain_filtered.nii" -ref "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/downsampled_filtered.nii" -aff "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine_matrix.txt" -res "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine_registered_atlas_brain.nii" -omp 4
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/bin/brainreg", line 10, in
sys.exit(main())
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/cli.py", line 288, in main
register(
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/main.py", line 65, in main
run_niftyreg(
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/backend/niftyreg/run.py", line 104, in run_niftyreg
brain_reg.register_affine()
File "/Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/backend/niftyreg/registration.py", line 73, in register_affine
raise RegistrationError(
brainreg.core.backend.niftyreg.registration.RegistrationError: Affine registration failed;
Process failed:
/bin/sh: /Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/bin/nifty_reg/osX/reg_aladin: No such file or directory
please read the logs at /Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine.log and /Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine.err
command: /Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/bin/nifty_reg/osX/reg_aladin -ln 6 -lp 5 -flo "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/brain_filtered.nii" -ref "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/downsampled_filtered.nii" -aff "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine_matrix.txt" -res "/Users/alessandrofelder/Downloads/brainreg-metapackage-output/niftyreg/affine_registered_atlas_brain.nii" -omp 4
</details>
I get an error with pip's dependency resolver which references brainreg==0.0.0
⁉️
Urgh I feel like I hit this weird behaviour right back when I started the tool reorganisation - pip and conda get their package information (inc. version) from different places. Could be the cause of the bug because I'm fairly sure brainreg 0.0.0 doesn't list atlasapi as a dependency.
What's the output of pip list | grep brainglobe in the envrionment? Guessing it should say brainreg is at v0.0.0.
As for the new problem you're encountering, the problem of
/bin/sh: /Users/alessandrofelder/mambaforge/envs/brainreg-conda-forge/lib/python3.10/site-packages/brainreg/core/bin/nifty_reg/osX/reg_aladin: No such file or directoryimplies that the niftyreg binaries are not being found. That I do have the answer to: our conda-forge installation is meant to depend on the conda-forge niftyreg package, but it looks like it doesn't. So adding niftyreg to the runtime dependencies should fix that... assuming conda installs them to a sensible place.
Thanks @willGraham01
What's the output of pip list | grep brainglobe in the envrionment? Guessing it should say brainreg is at v0.0.0.
You guess correctly:
brainglobe-atlasapi 2.0.6
brainglobe-napari-io 0.3.3
brainglobe-segmentation 1.2.2
brainglobe_space 1.0.2
brainglobe-utils 0.4.0
brainreg 0.0.0
Installing brainglobe-atlasapi from conda-forge in the script gives another error:
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versionsThe following specifications were found to be incompatible with your system:
- feature:/osx-arm64::__osx==14.5=0
- feature:/osx-arm64::__unix==0=0
- feature:|@/osx-arm64::__unix==0=0
- brainglobe-atlasapi -> click -> __unix
- brainglobe-atlasapi -> click -> __win
- brainglobe-atlasapi -> numpy -> __osx[version='>=10.9|>=11.0']
- python=3.10 -> libsqlite[version='>=3.45.2,<4.0a0'] -> __osx[version='>=10.9|>=11.0']
Your installed version is: 14.5
So adding niftyreg to the runtime dependencies should fix that... assuming conda installs them to a sensible place
Let's try
The following specifications were found to be incompatible with your system:
This doesn't fill me with joy though. The click dependency look suspicious, I found this open issue: conda-forge/click-feedstock#45
Local testing suggests this is not quite fixed yet. While things on conda-forge look good (there is a new build which will include niftyreg there: brainreg 1.0.9, build 1), an older version of brainreg (1.0.7) is installed for me on Macbook M2. Explicitly calling
mamba install brainreg=1.0.9fails with
Looking for: ['brainreg=1.0.9']
conda-forge/osx-arm64 Using cache
conda-forge/noarch Using cache
Pinned packages:
- python 3.10.*
Could not solve for environment specs
The following package could not be installed
└─ brainreg 1.0.9** is not installable because it requires
└─ brainglobe-atlasapi, which requires
└─ pymcubes, which does not exist (perhaps a missing channel)
It looks like pymcubes does not have an ARM package on conda-forge.
Additionally, I don't think we need pymcubes as a dependency of brainglobe-atlasapi - it's only needed for atlas packaging AFAICT.
conda-forge/brainglobe-atlasapi-feedstock#6 is now merged, which in particular removes the pymcubes dependency. So 🤞 the conda-forge install of brainreg should now be possible on ARM?
Yes, I hope so too. I am testing locally now.
I can locally install brainreg on M2 and run it on our tutorial sample data through both CLI and napari plugin.