Recurrently getting "Kmer matching step died"
apcamargo opened this issue · 7 comments
I'm recurrently getting the "Kmer matching step died" error during my assemblies. I wasn't able to pinpoint what may be causing it because I sometimes get it and sometimes don't even when I'm assembling the same data and using the same parameters.
PLASS Version: a983491
Ubuntu 18.04, 184 GB of memory, 96 × Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz
Do you use a grid engine and the job might have timed out? Your log outputs "killed" after the 6th iteration. So the OS or the grid engine might have killed the job.
Another idea: Could you try to run the assembly with less threads?
I think we assume about 4-8GB RAM per thread, at least that are the conditions in which we developed our software.
Thank you for the answers!
I'm not using a grid engine. This execution was performed in an AWS instance.
I'll try using less threads in my next assemblies. In any case, I could finish this assembly by starting it again in the same server (I needed three additional executions after this first one).
@milot-mirdita I reduced the number of cores so that I had ~5GB of memory per thread, but the problem persisted. I got the same error across multiple tries.
Something else that I've noticed: After finishing the assembly of a sample I started assembling the second one. The "generate k-mers list" step was very slow and eventually died. After deleting the PLASS directory and re-compiling it, the "generate k-mers list" of the first step was quick again, until eventually getting slow and dying in the 6th step.
I noticed this behavior several times already. After re-compiling PLASS gets fast again and doesn't die until it has processed several steps.
@apcamargo I really do not know whats the issues. Some times it runs only one iteration and then six. Is it possible to share the data? I would try to reproduce the issues.
Sure @martin-steinegger! I'll email the data to you.