bioinfologics/w2rap-contigger

Step 1 Speed for w2rap contigger

sanjitsbatra opened this issue · 3 comments

sanjitsbatra commented

Hey!

On running a profiler on the w2rap-contigger, one sees that ~98.5% of the time is being spent in the function: PQVecEncoder::init. It is currently taking really long times to load reads into memory. Is there any way you can recommend speeding this up?

Thanks

bjclavijo commented

That is actually to be expected, as that step is doing an RLE encoding of the reads while everything else on step 1 is just bitencoding. This step 1 shouldn't be the most time consuming part of the assembly anyway so I wouldn't particularly worry about that. Can you try running with a smaller dataset and checking if that step is the critical one and is indeed unreasonably slow in your setup?

sanjitsbatra commented

On checking the Top progress of the w2rap-contigger, we see that the CPU usage is not crossing 100% which means that it is using only 1 core. I think this is the root cause of this tremendous slowdown. Could you please suggest why this might be happening and how we might go about fixing this?

Might these flags:
export OMP_PROC_BIND=spread
export MALLOC_PER_THREAD=1

have something to do with this?

bjclavijo commented

This was solved by the user by changing OMP parameters to match his own architecture better.