branch cuda_support meets cuda compiler error on Summit
weilewei opened this issue · 8 comments
weilewei commented
On Summit, if I load cuda@10.1.243, I got a compilator issue, see error 1. If I load cuda/10.1.105, I will get linker issue, see error 2. Can you please give me some suggestions, thanks!
error1
[weile@login5.summit scripts]$ ./build_hpx_apex.sh install RelWithDebInfo
Resetting modules to system default. Reseting $MODULEPATH back to system default. All extra directories will be removed from $MODULEPATH.
Due to MODULEPATH changes, the following have been reloaded:
1) spectrum-mpi/10.3.1.2-20200121
Lmod is automatically replacing "xl/16.1.1-5" with "gcc/8.1.1".
-- The CXX compiler identification is GNU 8.1.1
-- The C compiler identification is GNU 8.1.1
-- Check for working CXX compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-8.1.1/spectrum-mpi-10.3.1.2-20200121-chae23sgwacfeot7vxkpfboz6wao2c33/bin/mpicxx
-- Check for working CXX compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-8.1.1/spectrum-mpi-10.3.1.2-20200121-chae23sgwacfeot7vxkpfboz6wao2c33/bin/mpicxx - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-8.1.1/spectrum-mpi-10.3.1.2-20200121-chae23sgwacfeot7vxkpfboz6wao2c33/bin/mpicc
-- Check for working C compiler: /autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-8.1.1/spectrum-mpi-10.3.1.2-20200121-chae23sgwacfeot7vxkpfboz6wao2c33/bin/mpicc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Using C++17
-- CMake version: 3.17.3
-- HPX version: 1.5.0
-- No vectorization library configured
-- Native TLS is enabled.
-- Nice threadlevel is disabled.
-- Stack traces are enabled.
-- Compiling with the native toolset
-- Looking for __atomic_fetch_add_4 in atomic
-- Looking for __atomic_fetch_add_4 in atomic - found
-- Performing Test HPX_WITH_CXX11_ATOMIC - Success
-- Performing Test HPX_WITH_CXX11_ATOMIC_128BIT - Success
-- Performing Test HPX_WITH_CXX11_STD_QUICK_EXIT - Success
-- Performing Test HPX_WITH_CXX11_SHARED_PTR_LWG3018 - Success
-- Performing Test HPX_WITH_CXX17_FILESYSTEM - Failed
-- Performing Test HPX_WITH_CXX17_FOLD_EXPRESSIONS - Success
-- Performing Test HPX_WITH_CXX17_FALLTHROUGH_ATTRIBUTE - Success
-- Performing Test HPX_WITH_CXX17_HARDWARE_DESTRUCTIVE_INTERFERENCE_SIZE - Failed
-- Performing Test HPX_WITH_CXX17_STRUCTURED_BINDINGS - Success
-- Performing Test HPX_WITH_CXX17_IF_CONSTEXPR - Success
-- Performing Test HPX_WITH_CXX17_INLINE_CONSTEXPR_VALUE - Success
-- Performing Test HPX_WITH_CXX17_ALIGNED_NEW - Success
-- Performing Test HPX_WITH_CXX17_STD_IN_PLACE_TYPE_T - Success
-- Performing Test HPX_WITH_CXX17_STD_VARIANT - Success
-- Performing Test HPX_WITH_CXX17_SHARED_PTR_ARRAY - Success
-- Performing Test HPX_WITH_CXX17_MAYBE_UNUSED - Success
-- Performing Test HPX_WITH_CXX17_DEDUCTION_GUIDES - Success
-- Performing Test HPX_WITH_CXX17_NOEXCEPT_FUNCTIONS_AS_NONTYPE_TEMPLATE_ARGUMENTS - Success
-- Performing Test HPX_WITH_CXX17_STD_TRANSFORM_SCAN_ALGORITHMS - Failed
-- Performing Test HPX_WITH_CXX17_STD_SCAN_ALGORITHMS - Failed
-- Performing Test HPX_WITH_CXX17_NONTYPE_TEMPLATE_PARAMETER_AUTO - Failed
-- Performing Test HPX_WITH_CXX20_COROUTINES - Failed
-- Performing Test HPX_WITH_CXX20_STD_DISABLE_SIZED_SENTINEL_FOR - Failed
-- Performing Test HPX_WITH_CXX20_NO_UNIQUE_ADDRESS_ATTRIBUTE - Failed
-- Performing Test HPX_WITH_BUILTIN_INTEGER_PACK - Success
-- Performing Test HPX_WITH_BUILTIN_MAKE_INTEGER_SEQ - Failed
-- Performing Test HPX_WITH_BUILTIN_TYPE_PACK_ELEMENT - Failed
-- Performing Test HPX_WITH_MM_PREFETCH - Failed
-- Performing Test HPX_WITH_STABLE_INPLACE_MERGE - Success
-- Performing Test HPX_WITH_UNISTD_H - Success
-- Performing Test HPX_WITH_CXX_FLAG_PTHREAD
-- Performing Test HPX_WITH_CXX_FLAG_PTHREAD - Success
-- Performing Test HPX_WITH_CXX_FLAG_WALL
-- Performing Test HPX_WITH_CXX_FLAG_WALL - Success
-- Performing Test HPX_WITH_C_FLAG_WALL
-- Performing Test HPX_WITH_C_FLAG_WALL - Success
-- Performing Test HPX_WITH_Fortran_FLAG_WALL
-- Performing Test HPX_WITH_Fortran_FLAG_WALL - Success
-- Performing Test HPX_WITH_CXX_FLAG_WEXTRA
-- Performing Test HPX_WITH_CXX_FLAG_WEXTRA - Success
-- Performing Test HPX_WITH_C_FLAG_WEXTRA
-- Performing Test HPX_WITH_C_FLAG_WEXTRA - Success
-- Performing Test HPX_WITH_Fortran_FLAG_WEXTRA
-- Performing Test HPX_WITH_Fortran_FLAG_WEXTRA - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_UNUSED_LOCAL_TYPEDEFS
-- Performing Test HPX_WITH_CXX_FLAG_WNO_UNUSED_LOCAL_TYPEDEFS - Success
-- Performing Test HPX_WITH_C_FLAG_WNO_UNUSED_LOCAL_TYPEDEFS
-- Performing Test HPX_WITH_C_FLAG_WNO_UNUSED_LOCAL_TYPEDEFS - Success
-- Performing Test HPX_WITH_Fortran_FLAG_WNO_UNUSED_LOCAL_TYPEDEFS
-- Performing Test HPX_WITH_Fortran_FLAG_WNO_UNUSED_LOCAL_TYPEDEFS - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_STRICT_ALIASING
-- Performing Test HPX_WITH_CXX_FLAG_WNO_STRICT_ALIASING - Success
-- Performing Test HPX_WITH_C_FLAG_WNO_STRICT_ALIASING
-- Performing Test HPX_WITH_C_FLAG_WNO_STRICT_ALIASING - Success
-- Performing Test HPX_WITH_Fortran_FLAG_WNO_STRICT_ALIASING
-- Performing Test HPX_WITH_Fortran_FLAG_WNO_STRICT_ALIASING - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_SIGN_PROMO
-- Performing Test HPX_WITH_CXX_FLAG_WNO_SIGN_PROMO - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_ATTRIBUTES
-- Performing Test HPX_WITH_CXX_FLAG_WNO_ATTRIBUTES - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_CAST_ALIGN
-- Performing Test HPX_WITH_CXX_FLAG_WNO_CAST_ALIGN - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_NOEXCEPT_TYPE
-- Performing Test HPX_WITH_CXX_FLAG_WNO_NOEXCEPT_TYPE - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_UNUSED_PARAMETER
-- Performing Test HPX_WITH_CXX_FLAG_WNO_UNUSED_PARAMETER - Success
-- Performing Test HPX_WITH_CXX_FLAG_WFORMAT_2
-- Performing Test HPX_WITH_CXX_FLAG_WFORMAT_2 - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_FORMAT_NONLITERAL
-- Performing Test HPX_WITH_CXX_FLAG_WNO_FORMAT_NONLITERAL - Success
-- Performing Test HPX_WITH_CXX_FLAG_WINIT_SELF
-- Performing Test HPX_WITH_CXX_FLAG_WINIT_SELF - Success
-- Performing Test HPX_WITH_CXX_FLAG_WDOUBLE_PROMOTION
-- Performing Test HPX_WITH_CXX_FLAG_WDOUBLE_PROMOTION - Success
-- Performing Test HPX_WITH_CXX_FLAG_WCAST_QUAL
-- Performing Test HPX_WITH_CXX_FLAG_WCAST_QUAL - Success
-- Performing Test HPX_WITH_CXX_FLAG_WCAST_ALIGN
-- Performing Test HPX_WITH_CXX_FLAG_WCAST_ALIGN - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_TRAMPOLINES
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_TRAMPOLINES - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_PARENTHESES
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_PARENTHESES - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_REORDER
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_REORDER - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_RETURN_TYPE
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_RETURN_TYPE - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_SEQUENCE_POINT
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_SEQUENCE_POINT - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_UNINITIALIZED
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_UNINITIALIZED - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_FORMAT
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_FORMAT - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_MISSING_BRACES
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_MISSING_BRACES - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_SIGN_COMPARE
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_SIGN_COMPARE - Success
-- Performing Test HPX_WITH_CXX_FLAG_FDIAGNOSTICS_SHOW_OPTION
-- Performing Test HPX_WITH_CXX_FLAG_FDIAGNOSTICS_SHOW_OPTION - Success
-- Performing Test HPX_WITH_C_FLAG_FDIAGNOSTICS_SHOW_OPTION
-- Performing Test HPX_WITH_C_FLAG_FDIAGNOSTICS_SHOW_OPTION - Success
-- Performing Test HPX_WITH_Fortran_FLAG_FDIAGNOSTICS_SHOW_OPTION
-- Performing Test HPX_WITH_Fortran_FLAG_FDIAGNOSTICS_SHOW_OPTION - Success
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_VLA
-- Performing Test HPX_WITH_CXX_FLAG_WERROR_VLA - Success
-- Performing Test HPX_WITH_C_FLAG_WERROR_RETURN_TYPE
-- Performing Test HPX_WITH_C_FLAG_WERROR_RETURN_TYPE - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_UNUSED_BUT_SET_PARAMETER
-- Performing Test HPX_WITH_CXX_FLAG_WNO_UNUSED_BUT_SET_PARAMETER - Success
-- Performing Test HPX_WITH_C_FLAG_WNO_UNUSED_BUT_SET_PARAMETER
-- Performing Test HPX_WITH_C_FLAG_WNO_UNUSED_BUT_SET_PARAMETER - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_UNUSED_BUT_SET_VARIABLE
-- Performing Test HPX_WITH_CXX_FLAG_WNO_UNUSED_BUT_SET_VARIABLE - Success
-- Performing Test HPX_WITH_C_FLAG_WNO_UNUSED_BUT_SET_VARIABLE
-- Performing Test HPX_WITH_C_FLAG_WNO_UNUSED_BUT_SET_VARIABLE - Success
-- Performing Test HPX_WITH_C_FLAG_WERROR_UNINITIALIZED
-- Performing Test HPX_WITH_C_FLAG_WERROR_UNINITIALIZED - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_MAYBE_UNINITIALIZED
-- Performing Test HPX_WITH_CXX_FLAG_WNO_MAYBE_UNINITIALIZED - Success
-- Performing Test HPX_WITH_C_FLAG_WNO_MAYBE_UNINITIALIZED
-- Performing Test HPX_WITH_C_FLAG_WNO_MAYBE_UNINITIALIZED - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_SYNC_NAND
-- Performing Test HPX_WITH_CXX_FLAG_WNO_SYNC_NAND - Success
-- Performing Test HPX_WITH_C_FLAG_WNO_SYNC_NAND
-- Performing Test HPX_WITH_C_FLAG_WNO_SYNC_NAND - Success
-- Performing Test HPX_WITH_CXX_FLAG_WNO_DELETE_NON_VIRTUAL_DTOR
-- Performing Test HPX_WITH_CXX_FLAG_WNO_DELETE_NON_VIRTUAL_DTOR - Success
-- Performing Test WITH_LINKER_FLAG_WL__Z_DEFS
-- Performing Test WITH_LINKER_FLAG_WL__Z_DEFS - Success
-- Performing Test HPX_WITH_CXX_FLAG_FVISIBILITY_HIDDEN
-- Performing Test HPX_WITH_CXX_FLAG_FVISIBILITY_HIDDEN - Success
-- Performing Test HPX_WITH_C_FLAG_FVISIBILITY_HIDDEN
-- Performing Test HPX_WITH_C_FLAG_FVISIBILITY_HIDDEN - Success
-- Performing Test HPX_WITH_Fortran_FLAG_FVISIBILITY_HIDDEN
-- Performing Test HPX_WITH_Fortran_FLAG_FVISIBILITY_HIDDEN - Success
-- Performing Test WITH_LINKER_FLAG_FVISIBILITY_HIDDEN
-- Performing Test WITH_LINKER_FLAG_FVISIBILITY_HIDDEN - Success
-- Architecture detected: ppc64
-- Found Boost: /gpfs/alpine/proj-shared/cph102/weile/dev/install/boost/1.73.0/include (found suitable version "1.73.0", minimum required is "1.61")
-- Found Boost: /gpfs/alpine/proj-shared/cph102/weile/dev/install/boost/1.73.0/include (found suitable version "1.73.0", minimum required is "1.61") found components: filesystem
-- Found Boost: /gpfs/alpine/proj-shared/cph102/weile/dev/install/boost/1.73.0/include (found suitable version "1.73.0", minimum required is "1.61") found components: iostreams regex
-- iostreams
-- Found Boost: /gpfs/alpine/proj-shared/cph102/weile/dev/install/boost/1.73.0/include (found suitable version "1.73.0", minimum required is "1.61") found components: program_options
-- Found Jemalloc: /gpfs/alpine/proj-shared/cph102/weile/dev/install/jemalloc/5.1.0/lib/libjemalloc.so
-- Using JEMALLOC allocator.
-- Found Hwloc: /gpfs/alpine/proj-shared/cph102/weile/dev/install/hwloc/2.2.0/lib/libhwloc.so
-- apex Will build APEX
-- apex APEX source dir is /gpfs/alpine/proj-shared/cph102/weile/dev/src/hpx_apex/hpx/apex/src/apex
-- apex APEX binary dir is /gpfs/alpine/proj-shared/cph102/weile/dev/src/hpx_apex/hpx/build_hpxmaster_apexcuda_RelWithDebInfo/apex/src/apex
System architecture: ppc64le
-- apex Building APEX with /proc/stat sampler
-- apex This is not edison
-- The CUDA compiler identification is unknown
-- Check for working CUDA compiler: /sw/summit/cuda/10.1.243/bin/nvcc
-- Check for working CUDA compiler: /sw/summit/cuda/10.1.243/bin/nvcc - broken
CMake Error at /autofs/nccs-svm1_sw/summit/.swci/0-core/opt/spack/20180914/linux-rhel7-ppc64le/gcc-4.8.5/cmake-3.17.3-ranbt2pk3wzzvd2i7j3ekexaqya3m4f2/share/cmake-3.17/Modules/CMakeTestCUDACompiler.cmake:46 (message):
The CUDA compiler
"/sw/summit/cuda/10.1.243/bin/nvcc"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /gpfs/alpine/proj-shared/cph102/weile/dev/src/hpx_apex/hpx/build_hpxmaster_apexcuda_RelWithDebInfo/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/gmake cmTC_4ce5c/fast && /usr/bin/gmake -f CMakeFiles/cmTC_4ce5c.dir/build.make CMakeFiles/cmTC_4ce5c.dir/build
gmake[1]: Entering directory `/gpfs/alpine/cph102/proj-shared/weile/dev/src/hpx_apex/hpx/build_hpxmaster_apexcuda_RelWithDebInfo/CMakeFiles/CMakeTmp'
Building CUDA object CMakeFiles/cmTC_4ce5c.dir/main.cu.o
/sw/summit/cuda/10.1.243/bin/nvcc -x cu -c /gpfs/alpine/proj-shared/cph102/weile/dev/src/hpx_apex/hpx/build_hpxmaster_apexcuda_RelWithDebInfo/CMakeFiles/CMakeTmp/main.cu -o CMakeFiles/cmTC_4ce5c.dir/main.cu.o
/autofs/nccs-svm1_sw/summit/gcc/8.1.1/include/c++/8.1.1/type_traits(347): error: identifier "__ieee128" is undefined
/autofs/nccs-svm1_sw/summit/gcc/8.1.1/include/c++/8.1.1/bits/std_abs.h(101): error: identifier "__ieee128" is undefined
/autofs/nccs-svm1_sw/summit/gcc/8.1.1/include/c++/8.1.1/bits/std_abs.h(102): error: identifier "__ieee128" is undefined
3 errors detected in the compilation of "/tmp/tmpxft_0000b48c_00000000-6_main.cpp1.ii".
gmake[1]: *** [CMakeFiles/cmTC_4ce5c.dir/main.cu.o] Error 1
gmake[1]: Leaving directory `/gpfs/alpine/cph102/proj-shared/weile/dev/src/hpx_apex/hpx/build_hpxmaster_apexcuda_RelWithDebInfo/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_4ce5c/fast] Error 2
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
apex/cmake/Modules/APEX_SetupCUPTI.cmake:7 (enable_language)
apex/src/apex/CMakeLists.hpx:171 (include)
apex/src/apex/CMakeLists.txt:9 (include)
-- Configuring incomplete, errors occurred!
See also "/gpfs/alpine/proj-shared/cph102/weile/dev/src/hpx_apex/hpx/build_hpxmaster_apexcuda_RelWithDebInfo/CMakeFiles/CMakeOutput.log".
See also "/gpfs/alpine/proj-shared/cph102/weile/dev/src/hpx_apex/hpx/build_hpxmaster_apexcuda_RelWithDebInfo/CMakeFiles/CMakeError.log".
[weile@login5.summit scripts]$
[weile@login5.summit scripts]$
[weile@login5.summit scripts]$ cd -
/gpfs/alpine/proj-shared/cph102/weile/dev/src/hpx_apex/hpx/apex
[weile@login5.summit apex]$ git branch
* cuda_support
develop
error 2:
[ 56%] Linking CXX shared library libhpx_apex.so
CMakeFiles/apex.dir/activity_trace_async.cpp.o: In function `store_profiler_data(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned long, unsigned long)':
activity_trace_async.cpp:(.text+0x1134): undefined reference to `apex::profiler_listener::push_profiler(int, std::shared_ptr<apex::profiler>&)'
/usr/bin/ld: link errors found, deleting executable `libhpx_apex.so.1.5.0'
collect2: error: ld returned 1 exit status
make[2]: *** [apex/src/apex/libhpx_apex.so.1.5.0] Error 1
make[1]: *** [apex/src/apex/CMakeFiles/apex.dir/all] Error 2
make: *** [all] Error 2
weilewei commented
The cmake scripts I used is:
#!/bin/bash -e
GCC_version=8.1.1
CMAKE_version=3.17.3
HWLOC_VER=2.2.0
JEMALLOC_VER=5.1.0
module reset
module unload gcc
module unload boost
module unload spectrum-mpi
module load gcc/$GCC_version
module load cmake/$CMAKE_version
module load spectrum-mpi/10.3.1.2-20200121
#module load cuda/10.1.243
module load cuda/10.1.105
export CC=mpicc
export CXX=mpicxx
COMMAND=$1
BUILD_TYPE=$2
OPTION=$3
HPX_PATH=/gpfs/alpine/proj-shared/cph102/weile/dev/src/hpx_apex/hpx
BUILD_PATH=$HPX_PATH/build_hpxmaster_apexcuda_$BUILD_TYPE
INSTALL_ROOT=/gpfs/alpine/proj-shared/cph102/weile/dev/install
INSTALL_PATH=$INSTALL_ROOT/hpxmaster_apexcuda_$BUILD_TYPE
BOOST_PATH=/gpfs/alpine/proj-shared/cph102/weile/dev/install/boost/1.73.0
HWLOC_PATH=$INSTALL_ROOT/hwloc/$HWLOC_VER
JEMALLOC_PATH=$INSTALL_ROOT/jemalloc/$JEMALLOC_VER
OTF2_PATH=/gpfs/alpine/cph102/proj-shared/weile/dev/src/spack/opt/spack/linux-rhel7-power9le/gcc-8.1.1/otf2-2.2-ihp5wgzz6rojxunxdet45s7zpnj4agt3
#GPERFTOOLS=/autofs/nccs-svm1_sw/summit/.swci/1-compute/opt/spack/20180914/linux-rhel7-ppc64le/gcc-8.1.1/gperftools-2.7-k2nb32b3srou77zbh4ximsbbpaphw5bh/
if [ "$COMMAND" = "clean" ]; then
if [ "$BUILD_TYPE" != "Debug" ] && [ "$BUILD_TYPE" != "Release" ] && [ "$BUILD_TYPE" != "RelWithDebInfo" ]; then
echo "Invalid build type. Please pick one of the following build types:"
echo "$0 distclean [Debug, Release, RelWithDebInfo]"
exit -1
fi
cd $BUILD_PATH
make clean
exit 0
fi
if [ "$COMMAND" = "distclean" ]; then
if [ "$BUILD_TYPE" != "Debug" ] && [ "$BUILD_TYPE" != "Release" ] && [ "$BUILD_TYPE" != "RelWithDebInfo" ]; then
echo "Invalid build type. Please pick one of the following build types:"
echo "$0 distclean [Debug, Release, RelWithDebInfo]"
exit -1
fi
rm -rf $INSTALL_PATH $BUILD_PATH
exit 0
fi
if [ "$COMMAND" = "" ]; then
echo "Build Type Not Specified. Please pick one of the following build types:"
echo "$0 distclean [Debug, Release, RelWithDebInfo]"
exit -1
fi
if [ "$COMMAND" = "build" ] || [ "$COMMAND" = "install" ] ; then
if [ "$BUILD_TYPE" != "Debug" ] && [ "$BUILD_TYPE" != "Release" ] && [ "$BUILD_TYPE" != "RelWithDebInfo" ]; then
echo "Build type '$BUILD_TYPE' is not valid! Please pick one of the following as the build type:"
echo "$0 [Debug, Release, RelWithDebInfo]"
exit -1
fi
if [ ! -d $HPX_PATH ]; then
git --depth=1 https://github.com/STEllAR-GROUP/hpx $HPX_PATH
fi
mkdir -p "$BUILD_PATH"
cd "$BUILD_PATH"
if [ "$OPTION" = "pull" ]; then
git pull
fi
cmake \
-DHPX_WITH_MAX_CPU_COUNT=168 \
-DCMAKE_BUILD_TYPE=$RelWithDelInfo \
-DBOOST_ROOT=$BOOST_PATH \
-DHPX_WITH_CXX17=ON \
-DCMAKE_CXX_FLAGS=$CXXFLAGS \
-DHWLOC_ROOT=$HWLOC_PATH \
-DJEMALLOC_ROOT=$JEMALLOC_PATH \
-DHPX_WITH_MALLOC=JEMALLOC \
-DHPX_WITH_HWLOC=ON \
-DHPX_WITH_THREAD_IDLE_RATES=ON \
-DHPX_WITH_NETWORKING=OFF \
-DHPX_WITH_PARCELPORT_MPI=OFF \
-DHPX_WITH_PARCELPORT_TCP=OFF \
-DCMAKE_INSTALL_PREFIX=$INSTALL_PATH \
-DHPX_WITH_EXAMPLES=ON \
-DHPX_WITH_TOOLS=ON \
-DHPX_WITH_TESTS=ON \
-DHPX_WITH_EXAMPLES=ON \
-DAPEX_ROOT=/gpfs/alpine/proj-shared/cph102/weile/dev/src/hpx_apex/hpx/apex \
-DHPX_WITH_APEX=TRUE \
-DHPX_WITH_APEX_TAG=develop \
-DHPX_WITH_APEX_NO_UPDATE=FALSE \
-DAPEX_WITH_ACTIVEHARMONY=FALSE \
-DAPEX_WITH_OTF2=TRUE \
-DOTF2_ROOT=$OTF2_PATH \
-DAPEX_WITH_PAPI=FALSE \
-DAPEX_WITH_CUDA=TRUE \
-DHPX_WITH_TESTS_BENCHMARKS=OFF \
-DHPX_WITH_TESTS_REGRESSIONS=OFF \
-DHPX_WITH_TESTS_UNIT=OFF \
-DHPX_WITH_CUDA=OFF \
-Wdev \
$HPX_PATH
make -j 15
if [ "$COMMAND" = "install" ]; then
make -j 10 install
fi
fi
khuck commented
It'll take me a while to set up a build environment on summit, but I'll give it a shot on Monday. I had problems with our installation of CUDA 10.1 at UO, so I was testing with 10.2.
weilewei commented
Thanks, let's talk about it tomorrow, as Summit doesn't have 10.2, only cuda/10.1.105, cuda/10.1.168, cuda/10.1.243, and cuda/11.0.1 and cuda/11.0.2
khuck commented
out of curiosity, how did you build boost and hwloc? boost 1.73 is giving me bus errors with both gcc 8 and 7.
weilewei commented
I once followed John (CSCS)'s scripts: https://github.com/biddisco/biddisco.github.io/wiki/summit-(power9) on Summit a while ago, but I think spack also works, something like, spack install boost@1.73.0%gcc@8.1.1 (works for other platforms). I haven't experienced buss errors
khuck commented
I didn't have a problem configuring with either CUDA 10.1.243 or 10.1.105. I did have problems building, though. Which HPX commit are you using?
weilewei commented
I use commit 492c2858c1c94a041259121a8d8c1db5a130287b
khuck commented
I pushed a couple of minor updates - I was able to build on Summit with 10.1.105 - please let me know if it works for you.