NERSC Perlmutter Compilation Error
kirstynvalles02 opened this issue · 2 comments
kirstynvalles02 commented
I was following these instructions for installing WarpX on Perlmutter:
https://warpx.readthedocs.io/en/latest/install/hpc/perlmutter.html
any help appreciated. Below is the SLURM file to the compilation and the error output.
#!/bin/bash
#SBATCH --qos=regular
#SBATCH -t 03:00:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --constraint=cpu
#SBATCH -o WarpXinstall.o%j
#SBATCH -e WarpXinstall.e%j
source $HOME/perlmutter_gpu_warpx.profile
cd $HOME/src/warpx
rm -rf build_pm_gpu
cmake -S . -B build_pm_gpu -DWarpX_COMPUTE=CUDA -DWarpX_FFT=ON -DWarpX_HEFFTE=ON -DWarpX_QED_TABLE_GEN=ON -DWarpX_DIMS="1;2;RZ;3"
cmake --build build_pm_gpu -j 16
#cmake -d -j(16)
openPMD build configuration:
library Version: 0.15.2
openPMD Standard: 1.1.0
C++ Compiler: GNU 12.3.0 CrayPrgEnv
/opt/cray/pe/craype/2.7.30/bin/CC
Installation: OFF
Build Type: Release
Library: static
CLI Tools: OFF
Examples: OFF
Testing: OFF
Invasive Tests: OFF
Internal VERIFY: ON
Build Options:
MPI: ON
HDF5: ON
ADIOS1: OFF
ADIOS2: ON
PYTHON: OFF
CUDA_EXAMPLES: OFF
WarpX build configuration:
Version: 24.05 (24.05-21-g67419c6ad3ed)
C++ Compiler: GNU 12.3.0 CrayPrgEnv
/opt/cray/pe/craype/2.7.30/bin/CC
Installation prefix: /usr/local
bin: bin
lib: lib64
include: include
cmake: lib64/cmake/WarpX
Build type: Release
Build options:
APP: ON
ASCENT: OFF
COMPUTE: CUDA
DIMS: 1;2;RZ;3
Embedded Boundary: OFF
IPO/LTO: OFF
LIB: ON (static)
MPI: ON
PARTICLE PRECISION: DOUBLE
PRECISION: DOUBLE
FFT Solvers: ON
PYTHON: OFF
OPENPMD: ON
QED: ON
QED table generation: ON
QED tools: OFF
SENSEI: OFF
CMake Warning:
Manually-specified variables were not used by the project:
WarpX_HEFFTE
/global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedamrex-src/Src/AmrCore/AMReX_InterpFaceRegister.cpp(96): warning #177-D: function "amrex::<unnamed>::IFRTag::box" was declared but never referenced
Box box() const noexcept { return Box(mask); }
^
Remark: The warnings can be suppressed with "-diag-suppress <warning-number>"
/global/homes/c/cvalles/src/warpx/Source/ablastr/fields/PoissonSolver.H(149): warning #811-D: const variable "info" requires an initializer -- class "amrex::LPInfo" has no user-provided default constructor
const amrex::LPInfo info;
^
Remark: The warnings can be suppressed with "-diag-suppress <warning-number>"
/global/homes/c/cvalles/src/warpx/Source/ablastr/fields/PoissonSolver.H(149): warning #811-D: const variable "info" requires an initializer -- class "amrex::LPInfo" has no user-provided default constructor
const amrex::LPInfo info;
^
Remark: The warnings can be suppressed with "-diag-suppress <warning-number>"
/usr/include/c++/12/bits/unique_ptr.h(1065): error: no instance of constructor "blas::Queue::Queue" matches the argument list
argument types are: (const int, int)
{ return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
^
detected during instantiation of "std::__detail::__unique_ptr_t<_Tp> std::make_unique<_Tp,_Args...>(_Args &&...) [with _Tp=blas::Queue, _Args=<const int &, int>]" at line 39 of /global/homes/c/cvalles/src/warpx/Source/FieldSolver/SpectralSolver/SpectralHankelTransform/HankelTransform.cpp
1 error detected in the compilation of "/global/homes/c/cvalles/src/warpx/Source/FieldSolver/SpectralSolver/SpectralHankelTransform/HankelTransform.cpp".
gmake[2]: *** [CMakeFiles/lib_rz.dir/build.make:1448: CMakeFiles/lib_rz.dir/Source/FieldSolver/SpectralSolver/SpectralHankelTransform/HankelTransform.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/Makefile2:1500: CMakeFiles/lib_rz.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
/global/homes/c/cvalles/src/warpx/Source/Particles/PhysicalParticleContainer.cpp(705): warning #811-D: const variable "attr_int" requires an initializer -- class "amrex::Vector<amrex::Vector<int, std::allocator<int>>, std::allocator<amrex::Vector<int, std::allocator<int>>>>" has no user-provided default constructor
const amrex::Vector<amrex::Vector<int>> attr_int;
^
/global/homes/c/cvalles/src/warpx/Source/Particles/LaserParticleContainer.cpp(549): warning #811-D: const variable "attr_int" requires an initializer -- class "amrex::Vector<amrex::Vector<int, std::allocator<int>>, std::allocator<amrex::Vector<int, std::allocator<int>>>>" has no user-provided default constructor
const amrex::Vector<amrex::Vector<int>> attr_int;
^
Remark: The warnings can be suppressed with "-diag-suppress <warning-number>"
/global/homes/c/cvalles/src/warpx/Source/Particles/LaserParticleContainer.cpp(549): warning #811-D: const variable "attr_int" requires an initializer -- class "amrex::Vector<amrex::Vector<int, std::allocator<int>>, std::allocator<amrex::Vector<int, std::allocator<int>>>>" has no user-provided default constructor
const amrex::Vector<amrex::Vector<int>> attr_int;
^
Remark: The warnings can be suppressed with "-diag-suppress <warning-number>"
gmake: *** [Makefile:136: all] Error 2
-- The C compiler identification is GNU 12.3.0
-- The CXX compiler identification is GNU 12.3.0
-- Cray Programming Environment 2.7.30 C
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/cray/pe/craype/2.7.30/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Cray Programming Environment 2.7.30 CXX
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/cray/pe/craype/2.7.30/bin/CC - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found CCache: /usr/bin/ccache
-- Downloading AMReX ...
-- AMReX repository: https://github.com/AMReX-Codes/amrex.git (28b010126a1b39297d8a496ba81f171d8563953b)
-- The CUDA compiler identification is NVIDIA 12.2.91
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /opt/nvidia/hpc_sdk/Linux_x86_64/23.9/cuda/12.2/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CMake version: 3.24.3
-- AMReX installation directory: /usr/local
-- Build type set by user to 'Release'.
-- Building AMReX with AMReX_SPACEDIM = 1;2;3
-- Configuring AMReX with the following options enabled:
-- AMReX_PRECISION = DOUBLE
-- AMReX_GPU_BACKEND = CUDA
-- AMReX_GPU_MAX_THREADS = 256
-- AMReX_MPI
-- AMReX_MPI_THREAD_MULTIPLE
-- AMReX_LINEAR_SOLVERS
-- AMReX_PARTICLES
-- AMReX_PARTICLES_PRECISION = DOUBLE
-- AMReX_PIC
-- AMReX_INLINE_LIMIT = 43210
-- AMReX_TINY_PROFILE
-- Enabled CUDA options:
-- AMReX_CUDA_FASTMATH
-- AMReX_CUDA_MAXREGCOUNT = 255
-- AMReX_CUDA_WARN_CAPTURE_THIS
-- AMReX_CUDA_SHOW_LINENUMBERS
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found MPI_C: /opt/cray/pe/craype/2.7.30/bin/cc (found version "3.1")
-- Found MPI_CXX: /opt/cray/pe/craype/2.7.30/bin/CC (found version "3.1")
-- Found MPI: TRUE (found version "3.1") found components: C CXX
-- Found CUDAToolkit: /opt/nvidia/hpc_sdk/Linux_x86_64/23.9/cuda/12.2/include (found version "12.2.91")
-- AMReX configuration summary:
-- Build type = Release
-- Install directory = /usr/local
-- C++ compiler = /opt/cray/pe/craype/2.7.30/bin/CC
-- CUDA compiler = /opt/nvidia/hpc_sdk/Linux_x86_64/23.9/cuda/12.2/bin/nvcc
-- C++ defines = -DAMREX_SPACEDIM=3
-- C++ flags = -O3 -DNDEBUG -march=znver3 -finline-limit=43210
-- CUDA flags = -O3 -DNDEBUG --expt-relaxed-constexpr --expt-extended-lambda -Xcudafe --diag_suppress=esa_on_defaulted_function_ignored -Xcudafe --diag_suppress=implicit_return_from_non_void_function -maxrregcount=255 -Xcudafe --display_error_number --Wext-lambda-captures-this --use_fast_math --generate-line-info
-- C++ include paths = -I/global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedamrex-src/Src/Base -I/global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedamrex-src/Src/Base/Parser -I/global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedamrex-src/Src/Boundary -I/global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedamrex-src/Src/AmrCore -I/global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedamrex-src/Src/LinearSolvers -I/global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG -I/global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedamrex-src/Src/LinearSolvers/OpenBC -I/global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedamrex-src/Src/Particle -I/opt/nvidia/hpc_sdk/Linux_x86_64/23.9/cuda/12.2/include -I/opt/nvidia/hpc_sdk/Linux_x86_64/23.9/math_libs/include
-- Link line =
-- AMReX: Using version '24.05' (24.05-32-g28b010126a1b)
-- Downloading PICSAR ...
-- PICSAR repository: https://github.com/ECP-WarpX/picsar.git (aa54e985398c1d575abc7e6737cdbc660a13765f)
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5") found components: CXX
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found Boost: /global/common/software/spackecp/perlmutter/e4s-23.05/default/spack/opt/spack/linux-sles15-zen3/gcc-11.2.0/boost-1.82.0-ow5r5qrgslcwu33grygouajmuluzuzv3/lib/cmake/Boost-1.82.0/BoostConfig.cmake (found suitable version "1.82.0", minimum required is "1.66.0")
-- Downloading openPMD-api ...
-- openPMD-api repository: https://github.com/openPMD/openPMD-api.git (0.15.2)
-- Found MPI: TRUE (found version "3.1") found components: CXX
-- Using the single-header code from /global/homes/c/cvalles/src/warpx/build_pm_gpu/_deps/fetchedopenpmd-src/share/openPMD/thirdParty/json/single_include/
-- nlohmann-json: Using INTERNAL version '3.9.1'
-- toml11: Using INTERNAL version '3.7.1'
-- Found HDF5: Included by compiler wrappers (found version "1.12.2") found components: C
-- Found MPI: TRUE (found version "3.1")
-- Found ADIOS2: /global/cfs/cdirs/mp217/cvalles/sw/perlmutter/gpu/adios2-2.8.3/lib64/cmake/adios2/adios2-config.cmake (found suitable version "2.8.3", minimum required is "2.7.0") found components: C CXX MPI
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found Git: /usr/bin/git (found version "2.35.3")
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /global/homes/c/cvalles/src/warpx/build_pm_gpu
[ 0%] Building CXX object CMakeFiles/buildInfoapp_3d.dir/app_3d/AMReX_buildInfo.cpp.o
[ 0%] Building CXX object CMakeFiles/buildInfoapp_rz.dir/app_rz/AMReX_buildInfo.cpp.o
[ 1%] Building CXX object _deps/fetchedopenpmd-build/CMakeFiles/openPMD.dir/src/Datatype.cpp.o
[ 1%] Building CXX object _deps/fetchedopenpmd-build/CMakeFiles/openPMD.dir/src/Iteration.cpp.o
...
[ 94%] Building CUDA object CMakeFiles/lib_2d.dir/Source/Utils/WarpXVersion.cpp.o
[ 94%] Building CUDA object CMakeFiles/lib_2d.dir/Source/Utils/Logo/GetLogo.cpp.o
[ 94%] Building CUDA object CMakeFiles/lib_2d.dir/Source/Utils/Parser/IntervalsParser.cpp.o
[ 94%] Building CUDA object CMakeFiles/lib_2d.dir/Source/Utils/Parser/ParserUtils.cpp.o
[ 94%] Built target lib_1d
[ 94%] Linking CXX static library lib/libwarpx.2d.MPI.CUDA.DP.PDP.OPMD.PSATD.QED.GENQEDTABLES.a
[ 94%] Built target lib_2d
ax3l commented
Thanks for reporting! I updated the script in #4986 , which probably caused this.
The warning
CMake Warning:
Manually-specified variables were not used by the project:WarpX_HEFFTE
will vanish if you update to the latest development of WarpX.
The error
/usr/include/c++/12/bits/unique_ptr.h(1065): error: no instance of constructor "blas::Queue::Queue" matches the argument list
argument types are: (const int, int)
{ return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
^
detected during instantiation of "std::__detail::__unique_ptr_t<_Tp> std::make_unique<_Tp,_Args...>(_Args &&...) [with _Tp=blas::Queue, _Args=<const int &, int>]" at line 39 of /global/homes/c/cvalles/src/warpx/Source/FieldSolver/SpectralSolver/SpectralHankelTransform/HankelTransform.cpp
I have to validate if it also appears on the latest development, maybe WarpX needs an update for the latest BLAS++ APIs... 👍
ax3l commented
Hi @kirstynvalles02, yes this is a breaking change from the latest BLAS++ release and we pushed a fix to #5012 for your consideration.