gcc 9.4 compile issue with CUDA
Closed this issue · 6 comments
Hi,
I can compile the serial CPU and MPI version of QUICK, I can run all kinds of other CUDA based software (Ampere architecture). This is under Ubuntu 20.0 and gcc 9.4, the Cuda compilation tool release is v10.1 but the running CUDA version is CUDA Version: 12.2.
When using cmake with "-DCUDA=TRUE", I get a make error: "unsupported GNU version! gcc versions later than 8 are not supported!" Is there a way to change the gcc version using some CMAKE compiler flags, like using an older compiler (gcc-8) or is there another way around without upgrading CUDA.
Thank you.
[ 32%] Linking CXX shared library libquick.so
[ 32%] Built target libquick
[ 32%] Building NVCC (Device) object src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/__/__/cuda/xc_cuda_generated_gpu_getxc.cu.o
In file included from /usr/include/cuda_runtime.h:83,
from <command-line>:
/usr/include/crt/host_config.h:138:2: error: #error -- unsupported GNU version! gcc versions later than 8 are not supported!
138 | #error -- unsupported GNU version! gcc versions later than 8 are not supported!
| ^~~~~
CMake Error at xc_cuda_generated_gpu_getxc.cu.o.RELEASE.cmake:218 (message):
Error generating
/home/user/quick/QUICK-23.08a/builddir/src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/__/__/cuda/./xc_cuda_generated_gpu_getxc.cu.o
make[2]: *** [src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/build.make:919: src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/__/__/cuda/xc_cuda_generated_gpu_getxc.cu.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:602: src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/all] Error 2
make: *** [Makefile:152: all] Error 2
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243
nvidia-smi
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.133 Driver Version: 537.79 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
gcc --version
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
gcc-8 --version
gcc-8 (Ubuntu 8.4.0-3ubuntu2) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
You can specify the path to the compiler when you configure your build with cmake. For example, if you have gcc-8 in your path, you can use
cmake -DCMAKE_C_COMPILER='gcc-8'
See the CMake manual for instance here https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER.html
Because this is not a problem with QUICK itself, I will close this issue.
Thank you @agoetz! I will try that.
Not sure this worked:
cmake .. -DMPI=TRUE -DCUDA=TRUE -DQUICK_USER_ARCH=volta \
-DCMAKE_INSTALL_PREFIX=${QUICK_INSTALL} -DCMAKE_C_COMPILER='gcc-8' -DCOMPILER=GNU
results in
-- Configuring done
You have changed variables that require your cache to be deleted.
Configure will be re-run and you may have to reset some variables.
The following variables have changed:
CMAKE_C_COMPILER= gcc-8
-- **************************************************************************
-- Starting configuration of quick version 22.3...
-- CMake Version: 3.16.3
-- For how to use this build system, please read this wiki:
-- http://ambermd.org/pmwiki/pmwiki.php/Main/CMake
-- For a list of important CMake variables, check here:
-- http://ambermd.org/pmwiki/pmwiki.php/Main/CMake-Common-Options
-- **************************************************************************
--
-- **************************************************************************
-- Error: Please select the compiler you want quick to use by setting the COMPILER CMake vairable to one of the below:
--
Supported quick compilers:
-----------------------------------------------------------------------
COMPILER value | C executable | C++ executable | Fortran executable | tested versions
--------------------------------------------------------------------------------------
GNU | gcc | g++ | gfortran | 4.8.5+
INTEL | icc | icpc | ifort | 19
PGI | pgcc | pgc++ | pgf90 |
CLANG | clang | clang++ | gfortran |
CRAY | cc | CC | ftn |
--------------------------------------------------------------------------------------
AUTO | <uses the default CMake-chosen compilers>
MANUAL | <uses the CC, CXX, and FC environment variables, or the
CMAKE_<LANGUAGE>_COMPILER CMake variables>
--
CMake Error at cmake/AmberCompilerConfig.cmake:48 (message):
Amber cannot build until COMPILER is set properly.
Call Stack (most recent call first):
cmake/AmberBuildSystemInit.cmake:65 (include)
CMakeLists.txt:11 (include)
but changing it to
cmake .. -DMPI=TRUE -DCUDA=TRUE -DQUICK_USER_ARCH=volta \
-DCMAKE_INSTALL_PREFIX=${QUICK_INSTALL} -DCMAKE_C_COMPILER='gcc-8' -DCOMPILER=AUTO
actually works fine for the CMAKE workflow. but then make fails:
$ make
...
[ 29%] Building CXX object src/CMakeFiles/libquick.dir/subs/time.cpp.o
[ 29%] Building Fortran object src/CMakeFiles/libquick.dir/dlfind/dlf_coords.f90.o
[ 29%] Building Fortran object src/CMakeFiles/libquick.dir/dlfind/dlf_hdlc_interface.f90.o
[ 29%] Building Fortran object src/CMakeFiles/libquick.dir/dlfind/dl_find.f90.o
[ 29%] Building Fortran object src/CMakeFiles/libquick.dir/dlfind/dlf_linalg.f90.o
[ 30%] Building Fortran object src/CMakeFiles/libquick.dir/dlfind/dlf_parallel_opt.f90.o
[ 30%] Building Fortran object src/CMakeFiles/libquick.dir/dlfind/dlf_serial.f90.o
[ 30%] Building Fortran object src/CMakeFiles/libquick.dir/dlfind/dlf_svnversion.f90.o
[ 30%] Building Fortran object src/CMakeFiles/libquick.dir/initialize.f90.o
[ 30%] Building Fortran object src/CMakeFiles/libquick.dir/read_job_and_atom.f90.o
[ 30%] Building Fortran object src/CMakeFiles/libquick.dir/getMol.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/quick_one_electron_integral.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/getEnergy.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/ecp.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/nuclear.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/finalize.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/mpi_setup.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/quick_debug.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/calMP2.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/hessian.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/CPHF.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/frequency.f90.o
[ 31%] Building Fortran object src/CMakeFiles/libquick.dir/basis.f90.o
[ 32%] Linking CXX shared library libquick.so
[ 32%] Built target libquick
[ 32%] Building NVCC (Device) object src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/__/__/cuda/xc_cuda_generated_gpu_getxc.cu.o
In file included from /usr/include/cuda_runtime.h:83,
from <command-line>:
/usr/include/crt/host_config.h:138:2: error: #error -- unsupported GNU version! gcc versions later than 8 are not supported!
138 | #error -- unsupported GNU version! gcc versions later than 8 are not supported!
| ^~~~~
CMake Error at xc_cuda_generated_gpu_getxc.cu.o.RELEASE.cmake:218 (message):
Error generating
/home/user/quick/QUICK-23.08a/builddir/src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/__/__/cuda/./xc_cuda_generated_gpu_getxc.cu.o
make[2]: *** [src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/build.make:919: src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/__/__/cuda/xc_cuda_generated_gpu_getxc.cu.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:602: src/libxc/maple2c_device/CMakeFiles/xc_cuda.dir/all] Error 2
make: *** [Makefile:152: all] Error 2
probably a CUDA mismatch between compiler and installed version.
The CUDA compiler picks up the host compiler provided by your OS, gcc 9.4 in your case, this is handled by include file $CUDA_HOME/include/crt/host_config.h. I do not have experience with Ubuntu so cannot give advice on best practices to deal with non-standard compiler installation paths. There are probably ways to override this behavior but that would require changes to the CMake build system.
May I ask why you are using the CUDA toolkit version 10.1 for compilation? GNU 9.x compilers are supported since CUDA 11. You seem to have CUDA 12.2 on your system. For QUICK-23.08a you would have to make a minor edit to quick-cmake/QUICKCudaConfig.cmake in line 88 to allow CUDA versions larger than 12.0. Can you try that?
@agoetz will do, thanks for the additional information. I agree it could be simply a mismatch between the CUDA compiler and CUDA itself. I also tried to compile it on another system and then transfer the binaries, but all the dependent libraries were missing so that was a no go.
Let me know if you can resolve the problem by using CUDA toolkit compilers that support GNU 9 compilers. Anything up to CUDA toolkit 12.2 should work on your system. Good luck.