kundajelab/chrombpnet

Job killed during ChromBPNet training

Opened this issue · 1 comments

Hello?
when I worked on "Training bias-factorized ChromBPNet" section of the ChromBPNet tutorial, I was not able to finish the training using chrombpnet pipeline. I followed the prompt and checked all files were in the right paths.
The command I used:
(chrombp) eunnkyu@Eunnkyu-DeskTop:~/chrombpnet_tutorial$ chrombpnet pipeline \ -ibam ~/chrombpnet_tutorial/data/downloads/merged.bam \ -d "ATAC" \ -g ~/chrombpnet_tutorial/data/downloads/hg38.fa \ -c ~/chrombpnet_tutorial/data/downloads/hg38.chrom.sizes \ -p ~/chrombpnet_tutorial/data/peaks_no_blacklist.bed \ -n ~/chrombpnet_tutorial/data/output_negatives.bed \ -fl ~/chrombpnet_tutorial/data/splits/fold_0.json \ -b ~/chrombpnet_tutorial/bias_model/ENCSR868FGK_bias_fold_0.h5 \ -o ~/chrombpnet_tutorial/chrombpnet_model/

And the result of the command was:

Estimating enzyme shift in input file
Current estimated shift: +0/+0
awk -v OFS="\t" '{if ($6=="+"){print $1,$2+4,$3,$4,$5,$6} else if ($6=="-") {print $1,$2,$3-4,$4,$5,$6}}' | sort -k1,1 | bedtools genomecov -bg -5 -i stdin -g /home/eunnkyu/chrombpnet_tutorial/data/downloads/hg38.chrom.sizes | LC_COLLATE="C" sort -k1,1 -k2,2n
Making BedGraph (Filter chromosomes not in reference fasta)
Making Bigwig
non zero bigwig entries in the given chromosome: 11988527
Killed

When I checked the output folders, they were empty except auxiliary, which contained a bigwig file:

(chrombp) eunnkyu@Eunnkyu-DeskTop:~/chrombpnet_tutorial/chrombpnet_model/auxiliary$ ls -l
total 1212784
-rw-r--r-- 1 eunnkyu eunnkyu 1241886467 Jun 12 10:43 data_unstranded.bw

I downloaded all test data using wget with provided information.
I double checked that I installed all packages listed in requirement.txt for chrombpnet.
Last, I did all preprocessing steps before the training bias-factorized chrombpnet.

Here is my system information:
Microsoft Windows, version 23H2 (OS build 22631,3737)
Ubuntu 22.04.3 LTS
Micromamba 1.5.8

If there are any other information you need to go over the issue, please let me know.

Thank you so much to take your time to go over my issue.

Best,

Eun Sung

The bam files provided with the tutorial are very deeply sequenced, your job is being killed because of lack of memory on the system.