Do all MPI ranks have to profile the same regions?
adayton1 opened this issue · 2 comments
adayton1 commented
I can't find any documentation on what happens if one MPI rank has a profile region but another does not. Does that create deadlock? If it does not, how do the aggregate reports work (min/max/average across all MPI ranks)?
daboehme commented
Hi @adayton1, there are no issues if there are different profiling regions on different MPI ranks. They will be merged into a unified call tree. The min/max/avg are computed over the ranks where the region exists (so if there is a region "foo" on two out of eight ranks, the average time reported for foo will be the average over those two ranks, not all eight). Hope this clarifies things!