Does cali-query do multi-threaded processing?
Closed this issue · 2 comments
cali-query: processing 1 files using 1 thread.
== CALIPER: Trace: Flushed 13 snapshots.
== CALIPER: Recorder: Wrote 90 records.
Does it mean there is a switch to enable multi-threaded processing of cali-query?
When the caliper output is large, it would be nice to parallelize the processing.
Hi @twang15
cali-query can use multi-threading when processing multiple files (e.g. cali-query <...> 1.cali 2.cali 3.cali ...), but not on a single file. Also, the parallel efficiency of the multi-threaded cali-query is not that great and depends on the query operations that are run. You can set the maximum number of threads to use with --threads, e.g. cali-query --threads 3 will use a maximum of 3 threads. But again, this only works when processing multiple .cali files. With --threads 1, cali-query will always use just one thread.
There is also a scalable MPI-based version of Caliper called mpi-caliquery. This, too, only works with multiple input files. It is intended for querying Caliper output from large-scale MPI runs.
Closing this - explained parallel cali-query in previous comment.