Memory limit exceeded during vg autoindex for GCSA/LCP indexing
Closed this issue · 0 comments
Hello,
I am encountering an issue when running vg autoindex to construct a graph from a HG002 reference FASTA and VCF file. The command I am using is as follows:
vg autoindex --workflow map --threads 24 --prefix /public1/home/sc30852/HG002/vg/graph --ref-fasta ../../hg002.mat.fasta --vcf ../mat.vcf.gz
Here is part of the log output:
[IndexRegistry]: Checking for phasing in VCF(s).
[IndexRegistry]: Chunking inputs for parallelism.
[IndexRegistry]: Chunking FASTA(s).
[IndexRegistry]: Chunking VCF(s).
[IndexRegistry]: Constructing VG graph from FASTA and VCF input.
[IndexRegistry]: Constructing XG graph from VG graph.
[IndexRegistry]: Pruning complex regions of VG to prepare for GCSA indexing.
[IndexRegistry]: Constructing GCSA/LCP indexes.
PathGraphBuilder::write(): Memory use of file 5 of kmer paths (503.81 GB) exceeds memory limit (503.781 GB).
It seems like the memory consumption during the GCSA indexing step exceeds the available memory (around 504 GB). Do you have any suggestions on how I can reduce memory usage, or is there a way to chunk the input differently to avoid this issue?
Any help would be appreciated!
Thank you!