quantumlib/qsim

Unable to build with cuQuantum (Linux)

ochapman-dphil opened this issue · 8 comments

ochapman-dphil commented

I'm trying to build qsim for use with cuQuantum but am having difficulty getting the build to complete.

I get the following logs when running make in the qsim directory

make -C apps/ qsim
make[1]: Entering directory '/home/usr8221/qsim/apps'
g++ -o ./qsim_amplitudes.x qsim_amplitudes.cc -O3 -fopenmp -march=native
g++ -o ./qsim_base.x qsim_base.cc -O3 -fopenmp -march=native
g++ -o ./qsim_von_neumann.x qsim_von_neumann.cc -O3 -fopenmp -march=native
g++ -o ./qsimh_amplitudes.x qsimh_amplitudes.cc -O3 -fopenmp -march=native
g++ -o ./qsimh_base.x qsimh_base.cc -O3 -fopenmp -march=native
make[1]: Leaving directory '/home/usr8221/qsim/apps'
make -C pybind_interface/ pybind
make[1]: Entering directory '/home/usr8221/qsim/pybind_interface'
g++ basic/pybind_main_basic.cpp -o ../qsimcirq/qsim_basic`python3-config --extension-suffix` -O3 -fopenmp -Wall -shared -std=c++17 -fPIC `python3 -m pybind11 --includes`
g++ sse/pybind_main_sse.cpp -o ../qsimcirq/qsim_sse`python3-config --extension-suffix` -O3 -fopenmp -msse4.1 -Wall -shared -std=c++17 -fPIC `python3 -m pybind11 --includes`
g++ avx2/pybind_main_avx2.cpp -o ../qsimcirq/qsim_avx2`python3-config --extension-suffix` -O3 -fopenmp -mavx2 -mfma -Wall -shared -std=c++17 -fPIC `python3 -m pybind11 --includes`
g++ avx512/pybind_main_avx512.cpp -o ../qsimcirq/qsim_avx512`python3-config --extension-suffix` -O3 -fopenmp -mavx512f -mbmi2 -Wall -shared -std=c++17 -fPIC `python3 -m pybind11 --includes`
nvcc cuda/pybind_main_cuda.cpp -o ../qsimcirq/qsim_cuda`python3-config --extension-suffix` -O3 -std=c++14 -x cu -Xcompiler "-Wall -shared -fPIC `python3 -m pybind11 --includes`"
cuda/../pybind_main.cpp(53): warning #177-D: handler parameter "exp" was declared but never referenced

cuda/../pybind_main.cpp(61): warning #177-D: handler parameter "exp" was declared but never referenced

cuda/../pybind_main.cpp(70): warning #177-D: handler parameter "exp" was declared but never referenced

nvcc custatevec/pybind_main_custatevec.cpp -o ../qsimcirq/qsim_custatevec`python3-config --extension-suffix` -O3 -I/home/usr8221/cuquantum-linux-x86_64-0.1.0.30-archive/include -L/home/usr8221/cuquantum-linux-x86_64-0.1.0.30-archive/lib64 -lcustatevec -lcublas -std=c++14 -x cu -Xcompiler "-Wall -shared -fPIC `python3 -m pybind11 --includes`"
custatevec/../pybind_main.cpp(53): warning #177-D: handler parameter "exp" was declared but never referenced

custatevec/../pybind_main.cpp(61): warning #177-D: handler parameter "exp" was declared but never referenced

custatevec/../pybind_main.cpp(70): warning #177-D: handler parameter "exp" was declared but never referenced

/apps/system/easybuild/software/binutils/2.37-GCCcore-11.2.0/bin/ld.gold: error: cannot find -lcustatevec
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:55: pybind-custatevec] Error 1
make[1]: Leaving directory '/home/usr8221/qsim/pybind_interface'
make: *** [Makefile:48: pybind] Error 2

Prior to running this I have:

  • set up a new conda environment conda create -n env-cuquantum python=3.8 pybind11 and activated it
  • exported the cuQuantum directory export CUQUANTUM_ROOT=/home/usr8221/cuquantum-linux-x86_64-0.1.0.30-archive and export CUQUANTUM_DIR=/home/usr8221/cuquantum-linux-x86_64-0.1.0.30-archive (different tutorials refer to ROOT and DIR so both assigned)
  • created the library path export LD_LIBRARY_PATH=${CUQUANTUM_ROOT}/lib64:${LD_LIBRARY_PATH}
leofang commented

Could you try export LD_LIBRARY_PATH=${CUQUANTUM_ROOT}/lib:${LD_LIBRARY_PATH} please? (lib64->lib)

ochapman-dphil commented

I've given that a go and I seem to be getting the same result...

make -C apps/ qsim
make[1]: Entering directory '/home/usr8221/qsim/apps'
g++ -o ./qsim_amplitudes.x qsim_amplitudes.cc -O3 -fopenmp -march=native
g++ -o ./qsim_base.x qsim_base.cc -O3 -fopenmp -march=native
g++ -o ./qsim_von_neumann.x qsim_von_neumann.cc -O3 -fopenmp -march=native
g++ -o ./qsimh_amplitudes.x qsimh_amplitudes.cc -O3 -fopenmp -march=native
g++ -o ./qsimh_base.x qsimh_base.cc -O3 -fopenmp -march=native
make[1]: Leaving directory '/home/usr8221/qsim/apps'
make -C pybind_interface/ pybind
make[1]: Entering directory '/home/usr8221/qsim/pybind_interface'
g++ basic/pybind_main_basic.cpp -o ../qsimcirq/qsim_basic`python3-config --extension-suffix` -O3 -fopenmp -Wall -shared -std=c++17 -fPIC `python3 -m pybind11 --includes`
g++ sse/pybind_main_sse.cpp -o ../qsimcirq/qsim_sse`python3-config --extension-suffix` -O3 -fopenmp -msse4.1 -Wall -shared -std=c++17 -fPIC `python3 -m pybind11 --includes`
g++ avx2/pybind_main_avx2.cpp -o ../qsimcirq/qsim_avx2`python3-config --extension-suffix` -O3 -fopenmp -mavx2 -mfma -Wall -shared -std=c++17 -fPIC `python3 -m pybind11 --includes`
g++ avx512/pybind_main_avx512.cpp -o ../qsimcirq/qsim_avx512`python3-config --extension-suffix` -O3 -fopenmp -mavx512f -mbmi2 -Wall -shared -std=c++17 -fPIC `python3 -m pybind11 --includes`
nvcc cuda/pybind_main_cuda.cpp -o ../qsimcirq/qsim_cuda`python3-config --extension-suffix` -O3 -std=c++14 -x cu -Xcompiler "-Wall -shared -fPIC `python3 -m pybind11 --includes`"
cuda/../pybind_main.cpp(53): warning #177-D: handler parameter "exp" was declared but never referenced

cuda/../pybind_main.cpp(61): warning #177-D: handler parameter "exp" was declared but never referenced

cuda/../pybind_main.cpp(70): warning #177-D: handler parameter "exp" was declared but never referenced

nvcc custatevec/pybind_main_custatevec.cpp -o ../qsimcirq/qsim_custatevec`python3-config --extension-suffix` -O3 -I/home/usr8221/cuquantum-linux-x86_64-0.1.0.30-archive/include -L/home/usr8221/cuquantum-linux-x86_64-0.1.0.30-archive/lib64 -lcustatevec -lcublas -std=c++14 -x cu -Xcompiler "-Wall -shared -fPIC `python3 -m pybind11 --includes`"
custatevec/../pybind_main.cpp(53): warning #177-D: handler parameter "exp" was declared but never referenced

custatevec/../pybind_main.cpp(61): warning #177-D: handler parameter "exp" was declared but never referenced

custatevec/../pybind_main.cpp(70): warning #177-D: handler parameter "exp" was declared but never referenced

/apps/system/easybuild/software/binutils/2.37-GCCcore-11.2.0/bin/ld.gold: error: cannot find -lcustatevec
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:55: pybind-custatevec] Error 1
make[1]: Leaving directory '/home/usr8221/qsim/pybind_interface'
make: *** [Makefile:48: pybind] Error 2

Here are the steps I'm taking...

Conda environment:

  1. conda create -n env-cuquantum python=3.8 pybind11
  2. conda activate env-cuquantum

Download packages:

  1. wget https://developer.download.nvidia.com/compute/cuquantum/redist/cuquantum/linux-x86_64/cuquantum-linux-x86_64-0.1.0.30-archive.tar.xz
  2. tar -xf cuquantum-linux-x86_64-0.1.0.30-archive.tar.xz
  3. git clone https://github.com/quantumlib/qsim.git

Build packages:

  1. export CUQUANTUM_DIR=/home/usr8221/cuquantum-linux-x86_64-0.1.0.30-archive
  2. export CUQUANTUM_ROOT=/home/usr8221/cuquantum-linux-x86_64-0.1.0.30-archive
  3. export LD_LIBRARY_PATH=${CUQUANTUM_ROOT}/lib:${LD_LIBRARY_PATH}
  4. cd /home/usr8221/qsim
  5. make
  6. pip install .
1tnguyen commented

I was having the same issue. My libcustatevec.so is in $CUQUANTUM_DIR/lib not lib64 as specified in the CMakeLists link_directories
Changing that directory path fixes the build issue.

  • 👍1
leofang commented

Thanks, @1tnguyen, good catch. Yes we also found this issue internally. The cause here is that cuQuantum is provided in two paths: devzone (with path lib) and NVIDIA HPC SDK (with path lib64). We were unaware of this difference at the time of releasing beta 2.

@95-martin-orion @sergeisakov I'd suggest to fix in this way:

link_directories($ENV{CUQUANTUM_DIR}/lib
                 $ENV{CUQUANTUM_DIR}/lib64)

The doc/sample fixes are already in our pipeline but might take a while to finish.

sergeisakov commented

@ochapman-phd, -L${CUQUANTUM_DIR}/lib should be added to line 15 in Makefile.

sergeisakov commented

The reason is that it was lib64 in the first beta version and lib in the second beta version.
#507 should fix this issue.

ochapman-dphil commented

Renaming lib to lib64 fixed issue. Cheers!

basnijholt commented

@ochapman-dphil, sorry to drag up an old issue, however, why did you not install cuQuantum via conda too?