cb-geo/mpm

MPI_TAG overflows for large number of domain shared nodes with Halo exchange

kks32 opened this issue · 0 comments

kks32 commented

Describe the bug
Running a large problem where the number of shared nodes exceeds the capacity of MPI_TAG a 16-bit signed integer, we have an overflow problem in the MPI_TAG.

Abort(671723268) on node 3 (rank 3 in comm 0): Fatal error in PMPI_Isend: Invalid tag, error stack:
PMPI_Isend(148): MPI_Isend(buf=0x7fff7328ea78, count=1, MPI_DOUBLE, dest=5, tag=524547, MPI_COMM_WORLD, request=0xc54a70fc) failed
PMPI_Isend(95).: Invalid tag, value is 524547

To Reproduce
Run a problem where the number of domain shared nodes exceed 32,768.

Expected behavior
Pass MPI_TAG without an overflow

Runtime environment (please complete the following information):

  • Frontera

Additional context
How can we alter this prototype code to avoid MPI_TAG overflows? At the moment, the node id or particle id is used as the MPI_TAG.