eldariont/svim

MemoryError

Closed this issue · 3 comments

Bobo22331 commented

Hi,
When I run SVIM using the following command

svim alignment --sample darmor --min_mapq 30 --min_sv_size 10 variants/darmor/18 align/darmor/darmor.westar.sorted.REF_chrC08.bam westar.fa

but it occurs MemoryError, and the SVIM often using memory more than 1TB, my each bam file is about 400MB. So, is there any suggestions about how to reduce the memory usage?

Thank you in advance!

eldariont commented

Hi Chipcui,

thanks for reporting this problem. In my experience, SVIM has a low memory usage of up to a few gigabases on a human dataset with good coverage so I am very suprised that it uses more than 1 TB on your dataset.

To investigate the cause of this issue I would need to know a few things:

  • From the command you sent it seems that you are running SVIM on each chromosome individually. Does one of these individual runs consume 1TB or is that the sum of many jobs running at the same time?
  • How large is the reference genome you are using?
  • What coverage does your dataset have?

Best
David

Bobo22331 commented

Hi,

Thank you for your reply!

  • I run SVIM only on one chromosome, and there is only one job running each time.
  • My reference genome is about 1.0 Gb.
  • My bam file is between assembled genomes, so the memory usage related to this?
eldariont commented

Thanks for the information. If you are comparing two genome assemblies I would recommend to use SVIM-asm instead of SVIM because it is optimized for this use-case.

However, I'm not sure that will necessarily fix your memory problem because the core (reading the BAM file and iterating through the alignment records on the search for SV signatures) of both tools is the same.

Can you maybe try out SVIM-asm on your dataset and let me know whether the problem persists?! And in case it does, could you send me your two files so that I could have a look at the problem?

Cheers
David