Dam break example does not generate output files
Closed this issue · 7 comments
Hi,
My aim is to write a SPH-DEM solver and simulate millions of particles using Cabana, and in the process I am learning to understand the internals and basics of the API.
I am successful at installing the ExaMPM software, however, the DamBreak example does not dump any output files while running.
Here is the output:
/DamBreak 0.05 2 0 0.001 1.0 50 serial
Kokkos::OpenMP::initialize WARNING: OMP_PROC_BIND environment variable not set
In general, for best performance with OpenMP 4.0 or better set OMP_PROC_BIND=spread and OMP_PLACES=threads
For best performance with OpenMP 3.1 set OMP_PROC_BIND=true
For unit testing set OMP_PROC_BIND=false
Time 0.000000 / 1.000000
Time 0.050000 / 1.000000
...
Can someone please help with this issue?
Many thanks.
Sorry for the confusion - see the PR I just opened where it prints a message letting you know that external libraries are needed in Cabana in order to get particle output. Let me know if you have further questions
Thank you for your prompt response. I have added the flags to the cmake argument. I running into trouble getting parallel hdf5 linked while compiling Cabana. I have very little experience with c++ build systems and compilation procedures. I will put forward what I have worked on towards this issue.
>>> adepudinesh@UoSDellLaptop (master) /home/adepudinesh/Kokkos/Cabana/build $
| DELL LAPTOP => cmake -D CMAKE_BUILD_TYPE="Debug" -D CMAKE_PREFIX_PATH=$KOKKOS_INSTALL_DIR -D CMAKE_INSTALL_PREFIX=$CABANA_INSTALL_DIR -D Cabana_REQUIRE_OPENMP=ON -D Cabana_ENABLE_EXAMPLES=ON -D Cabana_ENABLE_PERFORMANCE_TESTING=OFF -D Cabana_ENABLE_CAJITA=ON -D Cabana_REQUIRE_SILO=OFF -D Cabana_REQUIRE_HDF5=ON .. ;
-- Enabled Kokkos devices: OPENMP;SERIAL
-- HDF5 C compiler wrapper is unable to compile a minimal HDF5 program.
-- Cabana Revision = 'd24d188b0c4a33118ac96ddd81d7c6462f921b8a'
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/adepudinesh/Kokkos/Cabana/build
From the seconds line, it says that HDF5 C compiler wrapper is unable to compile a minimal HDF5 program.. I am not sure, why I am getting this error. I have both libhdf5-serial-dev and libhdf5-mpi-dev installed. I went ahead and tried make install, which resulted in the following error,
adepudinesh@UoSDellLaptop (master) /home/adepudinesh/Kokkos/Cabana/build $
| DELL LAPTOP UoS => make install
Consolidate compiler generated dependencies of target HelloWorld
[ 1%] Building CXX object example/core_tutorial/01_hello_world/CMakeFiles/HelloWorld.dir/hello_world.cpp.o
In file included from /home/adepudinesh/Kokkos/Cabana/core/src/Cabana_Core.hpp:53,
from /home/adepudinesh/Kokkos/Cabana/example/core_tutorial/01_hello_world/hello_world.cpp:12:
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp: In function ‘void Cabana::Experimental::HDF5ParticleOutput::Impl::writeFields(Cabana::Experimental::HDF5ParticleOutput::HDF5Config, hid_t, std::size_t, std::size_t, hsize_t, int, const char*, const char*, const SliceType&, typename std::enable_if<(2 == typename SliceType::kokkos_view::traits::dimension::rank), int*>::type)’:
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:275:9: error: there are no arguments to ‘H5Pset_dxpl_mpio’ that depend on a template parameter, so a declaration of ‘H5Pset_dxpl_mpio’ must be available [-fpermissive]
275 | H5Pset_dxpl_mpio( plist_id, H5FD_MPIO_COLLECTIVE );
| ^~~~~~~~~~~~~~~~
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:275:9: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp: In function ‘void Cabana::Experimental::HDF5ParticleOutput::Impl::writeFields(Cabana::Experimental::HDF5ParticleOutput::HDF5Config, hid_t, std::size_t, std::size_t, hsize_t, int, const char*, const char*, const SliceType&, typename std::enable_if<(3 == typename SliceType::kokkos_view::traits::dimension::rank), int*>::type)’:
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:352:9: error: there are no arguments to ‘H5Pset_dxpl_mpio’ that depend on a template parameter, so a declaration of ‘H5Pset_dxpl_mpio’ must be available [-fpermissive]
352 | H5Pset_dxpl_mpio( plist_id, H5FD_MPIO_COLLECTIVE );
| ^~~~~~~~~~~~~~~~
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp: In function ‘void Cabana::Experimental::HDF5ParticleOutput::Impl::writeFields(Cabana::Experimental::HDF5ParticleOutput::HDF5Config, hid_t, std::size_t, std::size_t, hsize_t, int, const char*, const char*, const SliceType&, typename std::enable_if<(4 == typename SliceType::kokkos_view::traits::dimension::rank), int*>::type)’:
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:433:9: error: there are no arguments to ‘H5Pset_dxpl_mpio’ that depend on a template parameter, so a declaration of ‘H5Pset_dxpl_mpio’ must be available [-fpermissive]
433 | H5Pset_dxpl_mpio( plist_id, H5FD_MPIO_COLLECTIVE );
| ^~~~~~~~~~~~~~~~
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp: In function ‘void Cabana::Experimental::HDF5ParticleOutput::writeTimeStep(Cabana::Experimental::HDF5ParticleOutput::HDF5Config, const string&, MPI_Comm, int, double, std::size_t, const CoordSliceType&, FieldSliceTypes&& ...)’:
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:522:5: error: there are no arguments to ‘H5Pset_fapl_mpio’ that depend on a template parameter, so a declaration of ‘H5Pset_fapl_mpio’ must be available [-fpermissive]
522 | H5Pset_fapl_mpio( plist_id, comm, MPI_INFO_NULL );
| ^~~~~~~~~~~~~~~~
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:535:9: error: there are no arguments to ‘H5Pset_all_coll_metadata_ops’ that depend on a template parameter, so a declaration of ‘H5Pset_all_coll_metadata_ops’ must be available [-fpermissive]
535 | H5Pset_all_coll_metadata_ops( plist_id, 1 );
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:536:9: error: there are no arguments to ‘H5Pset_coll_metadata_write’ that depend on a template parameter, so a declaration of ‘H5Pset_coll_metadata_write’ must be available [-fpermissive]
536 | H5Pset_coll_metadata_write( plist_id, 1 );
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:607:9: error: there are no arguments to ‘H5Pset_dxpl_mpio’ that depend on a template parameter, so a declaration of ‘H5Pset_dxpl_mpio’ must be available [-fpermissive]
607 | H5Pset_dxpl_mpio( plist_id, H5FD_MPIO_COLLECTIVE );
| ^~~~~~~~~~~~~~~~
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp: In function ‘void Cabana::Experimental::HDF5ParticleOutput::readTimeStep(Cabana::Experimental::HDF5ParticleOutput::HDF5Config, const string&, MPI_Comm, int, std::size_t, const string&, double&, FieldSliceType&)’:
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:760:5: error: there are no arguments to ‘H5Pset_fapl_mpio’ that depend on a template parameter, so a declaration of ‘H5Pset_fapl_mpio’ must be available [-fpermissive]
760 | H5Pset_fapl_mpio( plist_id, comm, MPI_INFO_NULL );
| ^~~~~~~~~~~~~~~~
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:765:9: error: there are no arguments to ‘H5Pset_all_coll_metadata_ops’ that depend on a template parameter, so a declaration of ‘H5Pset_all_coll_metadata_ops’ must be available [-fpermissive]
765 | H5Pset_all_coll_metadata_ops( plist_id, 1 );
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/adepudinesh/Kokkos/Cabana/core/src/Cabana_HDF5ParticleOutput.hpp:818:9: error: there are no arguments to ‘H5Pset_dxpl_mpio’ that depend on a template parameter, so a declaration of ‘H5Pset_dxpl_mpio’ must be available [-fpermissive]
818 | H5Pset_dxpl_mpio( plist_id, H5FD_MPIO_COLLECTIVE );
| ^~~~~~~~~~~~~~~~
make[2]: *** [example/core_tutorial/01_hello_world/CMakeFiles/HelloWorld.dir/build.make:76: example/core_tutorial/01_hello_world/CMakeFiles/HelloWorld.dir/hello_world.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:722: example/core_tutorial/01_hello_world/CMakeFiles/HelloWorld.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
I have looked this up, and it says I need parallel HDF5 installed. Which I have done using the instructions given https://github.com/HDFGroup/hdf5/blob/develop/release_docs/INSTALL . I have installed parallel hdf5 library locally in downloads folder, as below
----UoS LAPTOP DELL LAPTOP DELL LAPTOP ----------------------------------
adepudinesh@UoSDellLaptop /home/adepudinesh/Downloads/hdf5-1.14.1-2/bin $
| DELL LAPTOP UoS => ls
batch config.guess h5redeploy Makefile.am README.md
bbrelease config.sub h5redeploy.in Makefile.in release
buildhdf5 debug-ohdr h5vers makehelp restore.sh
checkapi depcomp install-sh make_overflow runbkgprog
checkposix format_source iostats make_vers switch_maint_mode
chkcopyright genparser ltmain.sh missing test-driver
cmakehdf5 h5cc make_err output_filter.sh trace
compile h5cc.in Makefile pkgscrpts warnhist
(base)
----UoS LAPTOP DELL LAPTOP DELL LAPTOP ----------------------------------
adepudinesh@UoSDellLaptop /home/adepudinesh/Downloads/hdf5-1.14.1-2/bin $
| DELL LAPTOP UoS => h5cc --showconfig
gcc -L/home/adepudinesh/Downloads/hdf5-1.14.1-2/hdf5/lib /home/adepudinesh/Downloads/hdf5-1.14.1-2/hdf5/lib/libhdf5_hl.a /home/adepudinesh/Downloads/hdf5-1.14.1-2/hdf5/lib/libhdf5.a -lz -ldl -lm -Wl,-rpath -Wl,/home/adepudinesh/Downloads/hdf5-1.14.1-2/hdf5/lib -I/usr/lib/x86_64-linux-gnu/openmpi/include -I/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi
(base)
Further I have added it to my path
# for parallel hdf5 for Cabana
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/adepudinesh/Downloads/hdf5-1.14.1-2/src/.libs
export PATH=/home/adepudinesh/Downloads/hdf5-1.14.1-2/bin:$PATH
export PATH=/home/adepudinesh/Downloads/hdf5-1.14.1-2/src:$PATH
export PATH=/home/adepudinesh/Downloads/hdf5-1.14.1-2/src/.libs:$PATH
export HDF5_ROOT=/home/adepudinesh/Downloads/hdf5-1.14.1-2
Even after this, I still get the same error. I am not sure how to proceed from here. Is this the right approach?
Thank you for your help. :)
I have tried with Silo input output format. I have installed it at /home/adepudinesh/Kokkos/Silo and while running Cabana cmake command, I used the following command
cmake -D CMAKE_BUILD_TYPE="Debug"
-D CMAKE_PREFIX_PATH=$KOKKOS_INSTALL_DIR
-D CMAKE_INSTALL_PREFIX=$CABANA_INSTALL_DIR
-D Cabana_REQUIRE_OPENMP=ON
-D Cabana_ENABLE_EXAMPLES=ON
-D Cabana_ENABLE_TESTING=OFF
-D Cabana_ENABLE_PERFORMANCE_TESTING=OFF
-D Cabana_ENABLE_CAJITA=ON
-D Cabana_REQUIRE_SILO=ON
-D Cabana_REQUIRE_HDF5=OFF
-D CMAKE_PREFIX_PATH=/home/adepudinesh/Kokkos/Silo/include
-D CMAKE_PREFIX_PATH=/home/adepudinesh/Kokkos/Silo/lib .. ;
I figured out that I need to add the paths of the Silo installation to cmake command through, CMAKE_PREFIX_PATH argument. This way I could compile the Cabana library with output functionality.
I will look into hdf5 later, and update.
Thank you.
You mentioned you have both serial and parallel HDF5 installed - if that is the case you will need to directly point to the parallel install location (Cabana still finds the serial version first otherwise). And you can generally pass the top level install location, not the separate include/library directories (although you can set those separately if you really want to): -D CMAKE_PREFIX_PATH="$KOKKOS_INSTALL_DIR;$HDF5_INSTALL_DIR"
Thank you for the instruction. Yes, I have installed them though apt-get, as well as I have built the parallel version from sources. I do not understand much of c++ yet. I will try hdf5 after a few weeks.
I was able to successfully install Cabana with parallel HDF5. However, while I am running exampm, I am running into this cmake error
adepudinesh@UoSDellLaptop (master) /home/adepudinesh/post_doc/softwares/ExaMPM/build $
| DELL LAPTOP UoS => cmake -D CMAKE_PREFIX_PATH="/home/adepudinesh/post_doc/softwares/kokkos/build/install;/home/adepudinesh/post_doc/softwares/Cabana/build/install" ..
-- The CXX compiler identification is GNU 11.3.0
-- The C compiler identification is GNU 11.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Enabled Kokkos devices: OPENMP;SERIAL
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1") found components: CXX
-- Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Found SILO: /home/adepudinesh/post_doc/softwares/Silo/SiloInstall/include
CMake Error at /usr/share/cmake-3.22/Modules/CMakeFindDependencyMacro.cmake:47 (find_package):
find_package called with invalid argument "1.14.1-2"
Call Stack (most recent call first):
/home/adepudinesh/post_doc/softwares/Cabana/build/install/lib/cmake/Cabana/CabanaConfig.cmake:47 (find_dependency)
CMakeLists.txt:11 (find_package)
-- Found CLANG_FORMAT: /usr/bin/clang-format (found suitable version "14.0.0", minimum required is "14")
-- Configuring incomplete, errors occurred!
See also "/home/adepudinesh/post_doc/softwares/ExaMPM/build/CMakeFiles/CMakeOutput.log".
I have install hdf5 with hdf5-1.14.1-2 source files. The line at CabanaConfig.cmake:47 is
find_dependency(HDF5 1.14.1-2 REQUIRED COMPONENTS C)
My hdf5 installation is at
/home/adepudinesh/Downloads/hdf5/install/
Can you please help me with fixing this cmake error?
I have manually removed the extra 1.14.1-2 from find_dependency command, and tried to run cmake again.
This fixed my problem.