mpi-trace analysis
twang15 opened this issue · 2 comments
I have a MPI program P, instrumented with Caliper.
W/ CALI_CONFIG_PROFILE=mpi-trace, P produces #MPI rank .cali files.
Since the file names are randomized, I want to know how to determine which one should be used in order to determine caliper timing of different hot loops within P, and how to analyze these files?
Thanks a lot!
Hi @twang15 ,
First, make sure to link the target program with libcaliper-mpiwrap.so
in addition to libcaliper.so
when using the mpi-trace profile. Then, the resulting .cali files should contain an mpi.rank attribute with the MPI rank for each process. With this, you can run cali-query on all files, and sort the output by rank, e.g.:
cali-query *.cali -t --sort-by mpi.rank
Alternatively, you can generate a file name based on the MPI rank with CALI_RECORDER_FILENAME
. For example, CALI_RECORDER_FILENAME=%mpi.rank%.cali
will create the files 0.cali, 1.cali, ..., etc. for each MPI rank.
Closing this - question answered, no Caliper action items.