gschramm/parallelproj

Installation of parallelproj fails

Closed this issue · 15 comments

jjgomezcadenas commented

Hello,
I am trying to install parallelproj in an ubuntu 22.03 machine with a NVIDIA GPU and CUDA libraries installed. I have followed the installation in the docs (using conda) to install parallelproj, cupy and pytorch. Everything is fine, but the first example fails.

>>> import array_api_compat.cupy as xp
>>> import parallelproj
/home/investigator/anaconda3/envs/epprj/lib/python3.12/site-packages/parallelproj/backend.py:18: UserWarning: The numpy.array_api submodule is still experimental. See NEP 47.
  from numpy.array_api._array_object import Array
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/investigator/anaconda3/envs/epprj/lib/python3.12/site-packages/parallelproj/__init__.py", line 1, in <module>
    from .backend import (
  File "/home/investigator/anaconda3/envs/epprj/lib/python3.12/site-packages/parallelproj/backend.py", line 171, in <module>
    raise ImportError(
ImportError: Cannot find parallelproj cuda lib. Consider settting the environment variable PARALLELPROJ_CUDA_LIB.

I have checked that, indeed, the library libparallelproj_c is installed but not libparallelproj_cuda (this is the reason why the error appears).

Thanks for your consideration and congratulations for what appears to be a very nice (and necessary) piece of software.

jjgomezcadenas commented

BTW, I tried to compile from source and got this error:

 69%] Built target parallelproj_c
nvcc fatal   : Unsupported gpu architecture 'compute_86+PTX'
nvcc fatal   : Unsupported gpu architecture 'compute_86+PTX'
nvcc fatal   : Unsupported gpu architecture 'compute_86+PTX'

My GPU is a NVIDIA GeForce RTX 4090

Thanks again

gschramm commented

Hm. That is strange.

  1. Ca you send the outout of:
    conda list in your env?
  2. Which GPU do you have?
  3. Which cuda version is installed (i) system wide (ii) in your conda env?
jjgomezcadenas commented

Hello:
For GPU :NVIDIA GeForce RTX 4090 .
CUDA Version: 12.2

CUDA is installed system wide, not in my environment.

Below the conda list (is long).

Thanks

$ conda list -n epprj

packages in environment at /home/investigator/anaconda3/envs/epprj:

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_kmp_llvm conda-forge
anyio 4.3.0 pyhd8ed1ab_0 conda-forge
argon2-cffi 23.1.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py312h98912ed_4 conda-forge
array-api-compat 1.4.1 pyhd8ed1ab_0 conda-forge
arrow 1.3.0 pyhd8ed1ab_0 conda-forge
asttokens 2.4.1 pyhd8ed1ab_0 conda-forge
async-lru 2.0.4 pyhd8ed1ab_0 conda-forge
attrs 23.2.0 pyh71513ae_0 conda-forge
babel 2.14.0 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.12.3 pyha770c72_0 conda-forge
bleach 6.1.0 pyhd8ed1ab_0 conda-forge
brotli 1.1.0 hd590300_1 conda-forge
brotli-bin 1.1.0 hd590300_1 conda-forge
brotli-python 1.1.0 py312h30efb56_1 conda-forge
bzip2 1.0.8 hd590300_5 conda-forge
ca-certificates 2024.2.2 hbcca054_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
certifi 2024.2.2 pyhd8ed1ab_0 conda-forge
cffi 1.16.0 py312hf06ca03_0 conda-forge
charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge
comm 0.2.1 pyhd8ed1ab_0 conda-forge
contourpy 1.2.0 py312h8572e83_0 conda-forge
cuda-nvrtc 12.3.107 hd3aeb46_0 conda-forge
cuda-version 12.3 h32bc705_2 conda-forge
cupy 13.0.0 py312h5add188_3 conda-forge
cupy-core 13.0.0 py312h7d03b9e_3 conda-forge
cycler 0.12.1 pyhd8ed1ab_0 conda-forge
debugpy 1.8.1 py312h30efb56_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.2.0 pyhd8ed1ab_2 conda-forge
executing 2.0.1 pyhd8ed1ab_0 conda-forge
fastrlock 0.8.2 py312h30efb56_2 conda-forge
filelock 3.13.1 pyhd8ed1ab_0 conda-forge
fonttools 4.49.0 py312h98912ed_0 conda-forge
fqdn 1.5.1 pyhd8ed1ab_0 conda-forge
freetype 2.12.1 h267a509_2 conda-forge
fsspec 2024.2.0 pyhca7485f_0 conda-forge
h11 0.14.0 pyhd8ed1ab_0 conda-forge
h2 4.1.0 pyhd8ed1ab_0 conda-forge
hpack 4.0.0 pyh9f0ad1d_0 conda-forge
httpcore 1.0.4 pyhd8ed1ab_0 conda-forge
httpx 0.27.0 pyhd8ed1ab_0 conda-forge
hyperframe 6.0.1 pyhd8ed1ab_0 conda-forge
icu 73.2 h59595ed_0 conda-forge
idna 3.6 pyhd8ed1ab_0 conda-forge
importlib-metadata 7.0.1 pyha770c72_0 conda-forge
importlib_metadata 7.0.1 hd8ed1ab_0 conda-forge
importlib_resources 6.1.2 pyhd8ed1ab_0 conda-forge
ipykernel 6.29.3 pyhd33586a_0 conda-forge
ipython 8.22.1 pyh707e725_0 conda-forge
isoduration 20.11.0 pyhd8ed1ab_0 conda-forge
jedi 0.19.1 pyhd8ed1ab_0 conda-forge
jinja2 3.1.3 pyhd8ed1ab_0 conda-forge
json5 0.9.17 pyhd8ed1ab_0 conda-forge
jsonpointer 2.4 py312h7900ff3_3 conda-forge
jsonschema 4.21.1 pyhd8ed1ab_0 conda-forge
jsonschema-specifications 2023.12.1 pyhd8ed1ab_0 conda-forge
jsonschema-with-format-nongpl 4.21.1 pyhd8ed1ab_0 conda-forge
jupyter-lsp 2.2.3 pyhd8ed1ab_0 conda-forge
jupyter_client 8.6.0 pyhd8ed1ab_0 conda-forge
jupyter_core 5.7.1 py312h7900ff3_0 conda-forge
jupyter_events 0.9.0 pyhd8ed1ab_0 conda-forge
jupyter_server 2.12.5 pyhd8ed1ab_0 conda-forge
jupyter_server_terminals 0.5.2 pyhd8ed1ab_0 conda-forge
jupyterlab 4.1.2 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.3.0 pyhd8ed1ab_1 conda-forge
jupyterlab_server 2.25.3 pyhd8ed1ab_0 conda-forge
kiwisolver 1.4.5 py312h8572e83_1 conda-forge
lcms2 2.16 hb7c19ff_0 conda-forge
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libabseil 20230802.1 cxx17_h59595ed_0 conda-forge
libblas 3.9.0 21_linux64_openblas conda-forge
libbrotlicommon 1.1.0 hd590300_1 conda-forge
libbrotlidec 1.1.0 hd590300_1 conda-forge
libbrotlienc 1.1.0 hd590300_1 conda-forge
libcblas 3.9.0 21_linux64_openblas conda-forge
libcublas 12.3.4.1 hd3aeb46_0 conda-forge
libcufft 11.0.12.1 hd3aeb46_0 conda-forge
libcurand 10.3.4.107 hd3aeb46_0 conda-forge
libcusolver 11.5.4.101 hd3aeb46_0 conda-forge
libcusparse 12.2.0.103 hd3aeb46_0 conda-forge
libdeflate 1.19 hd590300_0 conda-forge
libexpat 2.5.0 hcb278e6_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 13.2.0 h807b86a_5 conda-forge
libgfortran-ng 13.2.0 h69a702a_5 conda-forge
libgfortran5 13.2.0 ha4646dd_5 conda-forge
libgomp 13.2.0 h807b86a_5 conda-forge
libhwloc 2.9.3 default_h554bfaf_1009 conda-forge
libiconv 1.17 hd590300_2 conda-forge
libjpeg-turbo 3.0.0 hd590300_1 conda-forge
liblapack 3.9.0 21_linux64_openblas conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libnvjitlink 12.3.101 hd3aeb46_0 conda-forge
libopenblas 0.3.26 pthreads_h413a1c8_0 conda-forge
libparallelproj 1.7.3 cpu_h1b64f48_0 conda-forge
libpng 1.6.43 h2797004_0 conda-forge
libprotobuf 4.25.1 hf27288f_2 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libsqlite 3.45.1 h2797004_0 conda-forge
libstdcxx-ng 13.2.0 h7e041cc_5 conda-forge
libtiff 4.6.0 ha9c0a0a_2 conda-forge
libtorch 2.1.2 cpu_mkl_hcefb67d_101 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libuv 1.47.0 hd590300_0 conda-forge
libwebp-base 1.3.2 hd590300_0 conda-forge
libxcb 1.15 h0b41bf4_0 conda-forge
libxcrypt 4.4.36 hd590300_1 conda-forge
libxml2 2.12.5 h232c23b_0 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
llvm-openmp 17.0.6 h4dfa4b3_0 conda-forge
markupsafe 2.1.5 py312h98912ed_0 conda-forge
matplotlib-base 3.8.3 py312he5832f3_0 conda-forge
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mistune 3.0.2 pyhd8ed1ab_0 conda-forge
mkl 2023.2.0 h84fe81f_50496 conda-forge
mpmath 1.3.0 pyhd8ed1ab_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
nbclient 0.8.0 pyhd8ed1ab_0 conda-forge
nbconvert-core 7.16.1 pyhd8ed1ab_0 conda-forge
nbformat 5.9.2 pyhd8ed1ab_0 conda-forge
ncurses 6.4 h59595ed_2 conda-forge
nest-asyncio 1.6.0 pyhd8ed1ab_0 conda-forge
networkx 3.2.1 pyhd8ed1ab_0 conda-forge
notebook-shim 0.2.4 pyhd8ed1ab_0 conda-forge
numpy 1.26.4 py312heda63a1_0 conda-forge
openjpeg 2.5.1 h488ebb8_0 conda-forge
openssl 3.2.1 hd590300_0 conda-forge
overrides 7.7.0 pyhd8ed1ab_0 conda-forge
packaging 23.2 pyhd8ed1ab_0 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
parallelproj 1.7.3 pyha770c72_200 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pexpect 4.9.0 pyhd8ed1ab_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 10.2.0 py312hf3581a9_0 conda-forge
pip 24.0 pyhd8ed1ab_0 conda-forge
pkgutil-resolve-name 1.3.10 pyhd8ed1ab_1 conda-forge
platformdirs 4.2.0 pyhd8ed1ab_0 conda-forge
prometheus_client 0.20.0 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.42 pyha770c72_0 conda-forge
psutil 5.9.8 py312h98912ed_0 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.17.2 pyhd8ed1ab_0 conda-forge
pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge
pysocks 1.7.1 pyha2e5f31_6 conda-forge
python 3.12.2 hab00c5b_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-fastjsonschema 2.19.1 pyhd8ed1ab_0 conda-forge
python-json-logger 2.0.7 pyhd8ed1ab_0 conda-forge
python_abi 3.12 4_cp312 conda-forge
pytorch 2.1.2 cpu_mkl_py312h5fa05df_101 conda-forge
pytz 2024.1 pyhd8ed1ab_0 conda-forge
pyyaml 6.0.1 py312h98912ed_1 conda-forge
pyzmq 25.1.2 py312h886d080_0 conda-forge
readline 8.2 h8228510_1 conda-forge
referencing 0.33.0 pyhd8ed1ab_0 conda-forge
requests 2.31.0 pyhd8ed1ab_0 conda-forge
rfc3339-validator 0.1.4 pyhd8ed1ab_0 conda-forge
rfc3986-validator 0.1.1 pyh9f0ad1d_0 conda-forge
rpds-py 0.18.0 py312h4b3b743_0 conda-forge
scipy 1.12.0 py312heda63a1_2 conda-forge
send2trash 1.8.2 pyh41d4057_0 conda-forge
setuptools 69.1.1 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sleef 3.5.1 h9b69904_2 conda-forge
sniffio 1.3.1 pyhd8ed1ab_0 conda-forge
soupsieve 2.5 pyhd8ed1ab_1 conda-forge
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
sympy 1.12 pyh04b8f61_3 conda-forge
tbb 2021.11.0 h00ab1b0_1 conda-forge
terminado 0.18.0 pyh0d859eb_0 conda-forge
tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge
tk 8.6.13 noxft_h4845f30_101 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
tornado 6.4 py312h98912ed_0 conda-forge
traitlets 5.14.1 pyhd8ed1ab_0 conda-forge
types-python-dateutil 2.8.19.20240106 pyhd8ed1ab_0 conda-forge
typing-extensions 4.10.0 hd8ed1ab_0 conda-forge
typing_extensions 4.10.0 pyha770c72_0 conda-forge
typing_utils 0.1.0 pyhd8ed1ab_0 conda-forge
tzdata 2024a h0c530f3_0 conda-forge
uri-template 1.3.0 pyhd8ed1ab_0 conda-forge
urllib3 2.2.1 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.13 pyhd8ed1ab_0 conda-forge
webcolors 1.13 pyhd8ed1ab_0 conda-forge
webencodings 0.5.1 pyhd8ed1ab_2 conda-forge
websocket-client 1.7.0 pyhd8ed1ab_0 conda-forge
wheel 0.42.0 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.11 hd590300_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
yaml 0.2.5 h7f98852_2 conda-forge
zeromq 4.3.5 h59595ed_1 conda-forge
zipp 3.17.0 pyhd8ed1ab_0 conda-forge
zstd 1.5.5 hfc55251_0 conda-forge

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.154.05 Driver Version: 535.154.05 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4090 ... Off | 00000000:01:00.0 On | N/A |
| N/A 34C P8 4W / 150W | 77MiB / 16376MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 2315 G /usr/lib/xorg/Xorg 69MiB |
+---------------------------------------------------------------------------------------+

jjgomezcadenas commented

BTW I installed parallelproj in my Mac and it works. Instead, it doesn't work in my Ubuntu laptop even using the numpy end.

gschramm commented

Ok. Let's try to debug this step by step (1st CUDA ubuntu problem, then ubuntu laptop problem).

From the conda list output I see that the CPU-build (lacking the CUDA lib) of parallelproj has been installed by conda.
I suspect this is because your CUDA version is too new (12.2). Right now, we have cuda build for versions 11.2, 11.8, 12.0 - see here.

I suggest to try the following:

  1. conda create -n pp-test1 parallelproj cudatoolkit=12.0
    This forces conda to install cudatoolkit 12.0 where we have a cuda build.
  2. If (1) does not work: conda create -n pp-test2 parallelproj cudatoolkit=11.8

To check whether the cuda lib gets installed, run
conda list | grep libparallelproj in your env. The build string should be sth like libparallelproj-1.7.3-cuda.... If it says libparallelproj-1.7.3-cpu..., then conda still install the cpu version.

gschramm commented

BTW, I tried to compile from source and got this error:

 69%] Built target parallelproj_c
nvcc fatal   : Unsupported gpu architecture 'compute_86+PTX'
nvcc fatal   : Unsupported gpu architecture 'compute_86+PTX'
nvcc fatal   : Unsupported gpu architecture 'compute_86+PTX'

My GPU is a NVIDIA GeForce RTX 4090

Thanks again

Can post the output of the following commands:

  1. nvcc --version
  2. which nvcc
  3. echo $PATH
  4. echo $LD_LIBRARY_PATH

I suspect that you could have an nvcc version on your PATH that is too old and does not support compute 86.

jjgomezcadenas commented

Here it is:
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

$ which nvcc
/usr/bin/nvcc

$ echo $PATH
/home/investigator/Software/gate9_3qt/bin:/home/investigator/anaconda3/envs/epprj/bin:/home/investigator/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin

$ echo $LD_LIBRARY_PATH
:/home/investigator/Software/root_v6_3_2/lib:/home/investigator/Software/geant4v11qt/lib:/home/investigator/Software/coin3d/soxt_install/lib:/home/investigator/Software/coin3d/coin_install/lib

ls /usr/lib/x86_64-linux-gnu/cuda
/usr/lib/x86_64-linux-gnu/libcudadebugger.so.1 /usr/lib/x86_64-linux-gnu/libcuda.so
/usr/lib/x86_64-linux-gnu/libcudadebugger.so.535.161.07 /usr/lib/x86_64-linux-gnu/libcuda.so.1
/usr/lib/x86_64-linux-gnu/libcudadevrt.a /usr/lib/x86_64-linux-gnu/libcuda.so.535.161.07
/usr/lib/x86_64-linux-gnu/libcudart.so /usr/lib/x86_64-linux-gnu/libicudata.a
/usr/lib/x86_64-linux-gnu/libcudart.so.11.0 /usr/lib/x86_64-linux-gnu/libicudata.so
/usr/lib/x86_64-linux-gnu/libcudart.so.11.5.117 /usr/lib/x86_64-linux-gnu/libicudata.so.70
/usr/lib/x86_64-linux-gnu/libcudart_static.a /usr/lib/x86_64-linux-gnu/libicudata.so.70.1

gschramm commented

Your system nvcc and libcudart seem to be v11.5, but above you wrote that you are running CUDA v12.2.
Did you install multiple system versions of nvcc and the cudatoolkit? How did you install the system nvcc and cuda?

one more thing: Can you upload the ouput of the first call cmake .. in a separate file? This shows which nvcc is being used.

jjgomezcadenas commented

Sorry about the trouble. Apparently I have a mess with CUDA installation (I am a MAC user, just moved to ubuntu, since our current work involves PET reconstruction, thus parallelproj). This is what I get from

nvidia-smi command

Thu Feb 29 09:03:05 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.07 Driver Version: 535.161.07 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4090 ... Off | 00000000:01:00.0 On | N/A |
| N/A 38C P8 4W / 150W | 74MiB / 16376MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 3019 G /usr/lib/xorg/Xorg 69MiB |
+---------------------------------------------------------------------------------------+

Here is the output of cmake

cmake ../parallelproj/
-- CMAKE_VERSION: 3.22.1
-- BUILDING VERSION: 1.7.3
-- The C compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/bin/nvcc
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- linking against libm
-- The CXX compiler identification is GNU 11.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The CUDA compiler identification is NVIDIA 11.5.119
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- CMAKE_CUDA_ARCHITECTURES NOT DEFINED
-- Using CMAKE_CUDA_ARCHITECTURES: 89
-- GPUS RES: 0
-- Detected GPUS: NVIDIA GeForce RTX 4090 Laptop GPU

-- Configuring done
-- Generating done
-- Build files have been written to: /home/investigator/Projects/pprj_build
/usr/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

jjgomezcadenas commented

This does not work directly:

conda create -n pp-test2 parallelproj cudatoolkit=11.8

It says it does not find paralleproj.
Instead this works

conda create -n pp-test2 cudatoolkit=11.8
and I can later install paralleproj from source_forge but still seems that the cuda libray is not in.

gschramm commented

Thanks for all the feedback + the 2 tests. I think it is possible that the "inconsistencies" in the system cuda/nvcc versions can cause the issues (compilation fail + conda not able to see that you have cuda).

Would it be possible to remove + re-install the system CUDA+nvcc?
(only if that is not affecting any other of your cuda projects).

I recommend to follow the official nvidia cuda install guide here

jjgomezcadenas commented

I tried this:
conda create --name tpproj parallelproj cudatoolkit=11.8 -c conda-forge

Got this:# Package Plan ##

environment location: /home/investigator/anaconda3/envs/tpproj

added / updated specs:
- cudatoolkit=11.8
- parallelproj

The following NEW packages will be INSTALLED:

_libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
_openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-2_gnu
array-api-compat conda-forge/noarch::array-api-compat-1.4.1-pyhd8ed1ab_0
brotli conda-forge/linux-64::brotli-1.1.0-hd590300_1
brotli-bin conda-forge/linux-64::brotli-bin-1.1.0-hd590300_1
bzip2 conda-forge/linux-64::bzip2-1.0.8-hd590300_5
ca-certificates conda-forge/linux-64::ca-certificates-2024.2.2-hbcca054_0
certifi conda-forge/noarch::certifi-2024.2.2-pyhd8ed1ab_0
contourpy conda-forge/linux-64::contourpy-1.2.0-py312h8572e83_0
cudatoolkit conda-forge/linux-64::cudatoolkit-11.8.0-h4ba93d1_13
cycler conda-forge/noarch::cycler-0.12.1-pyhd8ed1ab_0
fonttools conda-forge/linux-64::fonttools-4.49.0-py312h98912ed_0
freetype conda-forge/linux-64::freetype-2.12.1-h267a509_2
kiwisolver conda-forge/linux-64::kiwisolver-1.4.5-py312h8572e83_1
lcms2 conda-forge/linux-64::lcms2-2.16-hb7c19ff_0
ld_impl_linux-64 conda-forge/linux-64::ld_impl_linux-64-2.40-h41732ed_0
lerc conda-forge/linux-64::lerc-4.0.0-h27087fc_0
libblas conda-forge/linux-64::libblas-3.9.0-21_linux64_openblas
libbrotlicommon conda-forge/linux-64::libbrotlicommon-1.1.0-hd590300_1
libbrotlidec conda-forge/linux-64::libbrotlidec-1.1.0-hd590300_1
libbrotlienc conda-forge/linux-64::libbrotlienc-1.1.0-hd590300_1
libcblas conda-forge/linux-64::libcblas-3.9.0-21_linux64_openblas
libdeflate conda-forge/linux-64::libdeflate-1.19-hd590300_0
libexpat conda-forge/linux-64::libexpat-2.5.0-hcb278e6_1
libffi conda-forge/linux-64::libffi-3.4.2-h7f98852_5
libgcc-ng conda-forge/linux-64::libgcc-ng-13.2.0-h807b86a_5
libgfortran-ng conda-forge/linux-64::libgfortran-ng-13.2.0-h69a702a_5
libgfortran5 conda-forge/linux-64::libgfortran5-13.2.0-ha4646dd_5
libgomp conda-forge/linux-64::libgomp-13.2.0-h807b86a_5
libjpeg-turbo conda-forge/linux-64::libjpeg-turbo-3.0.0-hd590300_1
liblapack conda-forge/linux-64::liblapack-3.9.0-21_linux64_openblas
libnsl conda-forge/linux-64::libnsl-2.0.1-hd590300_0
libopenblas conda-forge/linux-64::libopenblas-0.3.26-pthreads_h413a1c8_0
libparallelproj conda-forge/linux-64::libparallelproj-1.7.3-cuda118_he266856_200
libpng conda-forge/linux-64::libpng-1.6.43-h2797004_0
libsqlite conda-forge/linux-64::libsqlite-3.45.1-h2797004_0
libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-13.2.0-h7e041cc_5
libtiff conda-forge/linux-64::libtiff-4.6.0-ha9c0a0a_2
libuuid conda-forge/linux-64::libuuid-2.38.1-h0b41bf4_0
libwebp-base conda-forge/linux-64::libwebp-base-1.3.2-hd590300_0
libxcb conda-forge/linux-64::libxcb-1.15-h0b41bf4_0
libxcrypt conda-forge/linux-64::libxcrypt-4.4.36-hd590300_1
libzlib conda-forge/linux-64::libzlib-1.2.13-hd590300_5
matplotlib-base conda-forge/linux-64::matplotlib-base-3.8.3-py312he5832f3_0
munkres conda-forge/noarch::munkres-1.1.4-pyh9f0ad1d_0
ncurses conda-forge/linux-64::ncurses-6.4-h59595ed_2
numpy conda-forge/linux-64::numpy-1.26.4-py312heda63a1_0
openjpeg conda-forge/linux-64::openjpeg-2.5.2-h488ebb8_0
openssl conda-forge/linux-64::openssl-3.2.1-hd590300_0
packaging conda-forge/noarch::packaging-23.2-pyhd8ed1ab_0
parallelproj conda-forge/noarch::parallelproj-1.7.3-pyha770c72_200
pillow conda-forge/linux-64::pillow-10.2.0-py312hf3581a9_0
pip conda-forge/noarch::pip-24.0-pyhd8ed1ab_0
pthread-stubs conda-forge/linux-64::pthread-stubs-0.4-h36c2ea0_1001
pyparsing conda-forge/noarch::pyparsing-3.1.1-pyhd8ed1ab_0
python conda-forge/linux-64::python-3.12.2-hab00c5b_0_cpython
python-dateutil conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0
python_abi conda-forge/linux-64::python_abi-3.12-4_cp312
readline conda-forge/linux-64::readline-8.2-h8228510_1
scipy conda-forge/linux-64::scipy-1.12.0-py312heda63a1_2
setuptools conda-forge/noarch::setuptools-69.1.1-pyhd8ed1ab_0
six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
tk conda-forge/linux-64::tk-8.6.13-noxft_h4845f30_101
tzdata conda-forge/noarch::tzdata-2024a-h0c530f3_0
wheel conda-forge/noarch::wheel-0.42.0-pyhd8ed1ab_0
xorg-libxau conda-forge/linux-64::xorg-libxau-1.0.11-hd590300_0
xorg-libxdmcp conda-forge/linux-64::xorg-libxdmcp-1.1.3-h7f98852_0
xz conda-forge/linux-64::xz-5.2.6-h166bdaf_0
zstd conda-forge/linux-64::zstd-1.5.5-hfc55251_0

It seems to work now Will test with notebooks.

gschramm commented

I tried this: conda create --name tpproj parallelproj cudatoolkit=11.8 -c conda-forge

Got this:# Package Plan ##

environment location: /home/investigator/anaconda3/envs/tpproj

added / updated specs: - cudatoolkit=11.8 - parallelproj

The following NEW packages will be INSTALLED:

_libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge _openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-2_gnu array-api-compat conda-forge/noarch::array-api-compat-1.4.1-pyhd8ed1ab_0 brotli conda-forge/linux-64::brotli-1.1.0-hd590300_1 brotli-bin conda-forge/linux-64::brotli-bin-1.1.0-hd590300_1 bzip2 conda-forge/linux-64::bzip2-1.0.8-hd590300_5 ca-certificates conda-forge/linux-64::ca-certificates-2024.2.2-hbcca054_0 certifi conda-forge/noarch::certifi-2024.2.2-pyhd8ed1ab_0 contourpy conda-forge/linux-64::contourpy-1.2.0-py312h8572e83_0 cudatoolkit conda-forge/linux-64::cudatoolkit-11.8.0-h4ba93d1_13 cycler conda-forge/noarch::cycler-0.12.1-pyhd8ed1ab_0 fonttools conda-forge/linux-64::fonttools-4.49.0-py312h98912ed_0 freetype conda-forge/linux-64::freetype-2.12.1-h267a509_2 kiwisolver conda-forge/linux-64::kiwisolver-1.4.5-py312h8572e83_1 lcms2 conda-forge/linux-64::lcms2-2.16-hb7c19ff_0 ld_impl_linux-64 conda-forge/linux-64::ld_impl_linux-64-2.40-h41732ed_0 lerc conda-forge/linux-64::lerc-4.0.0-h27087fc_0 libblas conda-forge/linux-64::libblas-3.9.0-21_linux64_openblas libbrotlicommon conda-forge/linux-64::libbrotlicommon-1.1.0-hd590300_1 libbrotlidec conda-forge/linux-64::libbrotlidec-1.1.0-hd590300_1 libbrotlienc conda-forge/linux-64::libbrotlienc-1.1.0-hd590300_1 libcblas conda-forge/linux-64::libcblas-3.9.0-21_linux64_openblas libdeflate conda-forge/linux-64::libdeflate-1.19-hd590300_0 libexpat conda-forge/linux-64::libexpat-2.5.0-hcb278e6_1 libffi conda-forge/linux-64::libffi-3.4.2-h7f98852_5 libgcc-ng conda-forge/linux-64::libgcc-ng-13.2.0-h807b86a_5 libgfortran-ng conda-forge/linux-64::libgfortran-ng-13.2.0-h69a702a_5 libgfortran5 conda-forge/linux-64::libgfortran5-13.2.0-ha4646dd_5 libgomp conda-forge/linux-64::libgomp-13.2.0-h807b86a_5 libjpeg-turbo conda-forge/linux-64::libjpeg-turbo-3.0.0-hd590300_1 liblapack conda-forge/linux-64::liblapack-3.9.0-21_linux64_openblas libnsl conda-forge/linux-64::libnsl-2.0.1-hd590300_0 libopenblas conda-forge/linux-64::libopenblas-0.3.26-pthreads_h413a1c8_0 libparallelproj conda-forge/linux-64::libparallelproj-1.7.3-cuda118_he266856_200 libpng conda-forge/linux-64::libpng-1.6.43-h2797004_0 libsqlite conda-forge/linux-64::libsqlite-3.45.1-h2797004_0 libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-13.2.0-h7e041cc_5 libtiff conda-forge/linux-64::libtiff-4.6.0-ha9c0a0a_2 libuuid conda-forge/linux-64::libuuid-2.38.1-h0b41bf4_0 libwebp-base conda-forge/linux-64::libwebp-base-1.3.2-hd590300_0 libxcb conda-forge/linux-64::libxcb-1.15-h0b41bf4_0 libxcrypt conda-forge/linux-64::libxcrypt-4.4.36-hd590300_1 libzlib conda-forge/linux-64::libzlib-1.2.13-hd590300_5 matplotlib-base conda-forge/linux-64::matplotlib-base-3.8.3-py312he5832f3_0 munkres conda-forge/noarch::munkres-1.1.4-pyh9f0ad1d_0 ncurses conda-forge/linux-64::ncurses-6.4-h59595ed_2 numpy conda-forge/linux-64::numpy-1.26.4-py312heda63a1_0 openjpeg conda-forge/linux-64::openjpeg-2.5.2-h488ebb8_0 openssl conda-forge/linux-64::openssl-3.2.1-hd590300_0 packaging conda-forge/noarch::packaging-23.2-pyhd8ed1ab_0 parallelproj conda-forge/noarch::parallelproj-1.7.3-pyha770c72_200 pillow conda-forge/linux-64::pillow-10.2.0-py312hf3581a9_0 pip conda-forge/noarch::pip-24.0-pyhd8ed1ab_0 pthread-stubs conda-forge/linux-64::pthread-stubs-0.4-h36c2ea0_1001 pyparsing conda-forge/noarch::pyparsing-3.1.1-pyhd8ed1ab_0 python conda-forge/linux-64::python-3.12.2-hab00c5b_0_cpython python-dateutil conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0 python_abi conda-forge/linux-64::python_abi-3.12-4_cp312 readline conda-forge/linux-64::readline-8.2-h8228510_1 scipy conda-forge/linux-64::scipy-1.12.0-py312heda63a1_2 setuptools conda-forge/noarch::setuptools-69.1.1-pyhd8ed1ab_0 six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0 tk conda-forge/linux-64::tk-8.6.13-noxft_h4845f30_101 tzdata conda-forge/noarch::tzdata-2024a-h0c530f3_0 wheel conda-forge/noarch::wheel-0.42.0-pyhd8ed1ab_0 xorg-libxau conda-forge/linux-64::xorg-libxau-1.0.11-hd590300_0 xorg-libxdmcp conda-forge/linux-64::xorg-libxdmcp-1.1.3-h7f98852_0 xz conda-forge/linux-64::xz-5.2.6-h166bdaf_0 zstd conda-forge/linux-64::zstd-1.5.5-hfc55251_0

It seems to work now Will test with notebooks.

Glad to see that :)
I missed -c conda-forge which is essential, since parallelproj is only availalbe on the conda-forge channel (community conda channel.

Btw: mamba is a much more efficient implementation of conda that by default uses conda-forge as channel ;)

jjgomezcadenas commented

Hi,

  1. THANKS SO MUCH for your help.
  2. Tried several notebooks. All work now.
  3. I know well numpy and pytorch, from previous work. Not cupy. My tendency would be to use pytorch as default, but is cupy more efficient for GPU?
  4. Do you think I need to do something about my messy CUDA installation? No experience here. Of course, happy to work on environments.
  5. Read your paper. Impressive. Would love to discuss more about using parallelproj for a full (lightweight) python (bindings) implementation of PET reconstruction (lot of the stuff seems to be there already). If you have the time drop me a line to jjgomezcadenas@dipc.org (see hhttps://next.ific.uv.es/next/ and ttps://neutrino.dipc.org/ if you are curious)
    Thanks again
gschramm commented

Glad to hear that the notebooks are running :)

  1. If you are "happy" with cuda 11.8, no need to change anything. For parallelproj cuda 11.8 is fine.
  2. There should be no performance difference between using pytorch GPU tensors vs cupy gpu arrays.

I will close the issue now.