olcf/olcf-user-docs

known issue: cuda hooks / pami

Closed this issue · 0 comments

When running simple gpu codes with jsrun that do not also have MPI support, sometimes one might run into a warning such as:

CUDA Hook Library: Failed to find symbol mem_find_dreg_entries, ./a.out: undefined symbol: __PAMI_Invalidate_region

This can be solved in a few ways:

  • use jsrun -E LD_PRELOAD=/opt/ibm/spectrum_mpi/lib/pami_451/libpami.so ...
  • use jsrun --smpiargs="off" ...
  • use jsrun --smpiargs="-disable_gpu_hooks ...

Some more discussion can be found at kokkos/kokkos#1985; reports there say it has been reported to IBM.