Can using "--mm" flag reduce memory usage?
Opened this issue · 2 comments
Dear Developer,
I am wondering whether using --mm could reduce memory usage. If I am not mistaken, --mm alters the way of memory mapping and allows sharing the index with other instances. In this scenario, multiple Centrifuge processes could use the same index, eliminating the need to load the index into memory for each process.
Initially, I ran five Centrifuge programs simultaneously using the command: centrifuge -q -1 $fq1 -2 $fq2 -p 5 -x nt, and the memory usage was up to ~700GB. Then I tried to use the command: centrifuge -q -1 $fq1 -2 $fq2 -x nt --mm, however, the memory usage didn't seem to decrease. So I'd like to ask how can I minimize memory usage while maintaining multiple threads or running multiple programs simultaneously. Thank you so much for your kind help!
What is the index size for the nt?
What is the index size for the nt?
-rw-r--r-- 1 zniu root 63G Mar 27 2018 nt.1.cf
-rw-r--r-- 1 zniu root 38G Mar 27 2018 nt.2.cf
-rw-r--r-- 1 zniu root 1008M Mar 23 2018 nt.3.cf
-rw-r--r-- 1 zniu root 529M Mar 27 2018 nt.4.cf
The total size is ~102 GB