Optimize the multiprocessing strategy when parsing multiple traces
rzdao opened this issue · 0 comments
rzdao commented
🚀 Motivation and context
The motivation for for this issue is that the current multiprocessing strategy not taken into account, which may cause OOM when process pool is too large.
Description
This feature will add the target machine's memory information to determine the size of the process pool. With this additional information, along with the rank number and physical processor number, it's easy to calculate the optimal pool size.
Alternatives
No response
Additional context
No response