One of the commands exited with non-zero exit code; note that snakemake uses bash strict mode
Closed this issue · 2 comments
M-Asim-Javed commented
Hello,
I run Verkko lastest version v1.4.1 with HiFi and ONT data for the haploid genome (25Mb) but got an error twice. Here are the #SplitOnt.err, #process.err, and end part of my #log: Can you please help in this regard? Thanks!
Command
verkko -d verkko_asm --hifi /home/edelab/pacbio/new_assembly/merged_bam_file/chopper_files/filtered_3kb_pbhifi_merged.fastq.gz --nano /home/edelab/pacbio/new_assembly/merged_bam_file/chopper_files/filtered_3kb_ont_merged.fastq.gz
SplitOnt.err file
./splitONT.sh: 2: set: Illegal option -o pipefail
process.err
Gap insertion pass 1.
inserted 0 gaps
Gap insertion pass 2.
inserted 0 gaps
Gap insertion pass 3.
inserted 0 gaps
Haplogap fixing.
mend >124 >560
mend >180 >688
mend >110 <811
mend <485 >709
mend >289 >739
mend >356 <649
Tip unrolling.
unroll 724
unrolled 1 looptips
Unitig creation.
LogError file
[Tue Oct 24 18:53:12 2023]
checkpoint splitTipsONT:
input: /home/edelab/pacbio/new_assembly/merged_bam_file/chopper_files/filtered_3kb_ont_merged.fastq.gz, 2-processGraph/unitig-unrolled-hifi-resolved.gfa, 3-align/alns-ont.gaf, 3-align/splitONT.finished
output: 3-alignTips/splitONT.finished
log: 3-alignTips/splitONT.err
jobid: 15
reason: Missing output files: 3-alignTips/splitONT.finished; Input files updated by another job: 2-processGraph/unitig-unrolled-hifi-resolved.gfa, 3-align/alns-ont.gaf
resources: tmpdir=/tmp, job_id=1, n_cpus=1, mem_gb=8, time_h=96
DAG of jobs will be updated after completion.
[Tue Oct 24 18:53:12 2023]
Error in rule splitTipsONT:
jobid: 15
input: /home/edelab/pacbio/new_assembly/merged_bam_file/chopper_files/filtered_3kb_ont_merged.fastq.gz, 2-processGraph/unitig-unrolled-hifi-resolved.gfa, 3-align/alns-ont.gaf, 3-align/splitONT.finished
output: 3-alignTips/splitONT.finished
log: 3-alignTips/splitONT.err (check log file(s) for error details)
shell:
cd 3-alignTips
cat > ./splitONT.sh <<EOF
#!/bin/sh
set -e -o pipefail
mkdir -p split
/home/edelab/miniconda3/envs/verkko/lib/verkko/scripts/find_tips.py ../2-processGraph/unitig-unrolled-hifi-resolved.gfa > tips.fasta 2> find_tips.err
awk '/^>/ { print substr(\$1, 2) }' < tips.fasta > tips.ids
cat ../3-align/alns-ont.gaf | /home/edelab/miniconda3/envs/verkko/lib/verkko/scripts/filter_alignments_by_column.py tips.ids 5 false | awk -F "\t" '{ if (\$12 >= 20) print \$1;}' | sort |uniq > tip_reads.ids
for fn in ont001.fasta.gz ont003.fasta.gz ont004.fasta.gz ont002.fasta.gz ; do
echo >> tip_reads.extract extract \$fn tip_reads.ids ../3-align/split/\$fn
done
xargs -L 1 -P 1 < tip_reads.extract \\
/home/edelab/miniconda3/envs/verkko/lib/verkko/scripts/fasta_extract.py
gunzip -c ont001.fasta.gz ont003.fasta.gz ont004.fasta.gz ont002.fasta.gz |awk -v SEQ="" '{ if (match(\$1, ">") && \$1 != NAME) { if (SEQ != "") { print NAME; print SEQ; } SEQ=""; NAME=\$1; } else { SEQ=SEQ""\$1;} } END { print NAME; print SEQ}' | tr -s ACGT > tip_reads_compressed.fasta
rm -f ont001.fasta.gz ont003.fasta.gz ont004.fasta.gz ont002.fasta.gz
cat ../2-processGraph/unitig-unrolled-hifi-resolved.gfa |awk '{ if (match(\$1, "^S")) { print ">"\$2; print \$3 } }' |fold -b > tips.fasta
bases=\`expr 3000000000 / 2 \`
/home/edelab/miniconda3/envs/verkko/lib/verkko/scripts/fasta_partition.py \\
partition split/ont \$bases 150000 0 \\
tip_reads_compressed.fasta \\
&& \\
touch ../3-alignTips/splitONT.finished
EOF
chmod +x ./splitONT.sh
./splitONT.sh > ../3-alignTips/splitONT.err 2>&1
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Exiting because a job execution failed. Look above for error message.
skoren commented
Thanks for the detailed logs, this looks to be the same bash/sh issue as #185/186. Updating the code as suggested there should resolve your issue.
M-Asim-Javed commented
Thank you so much for your help. In my case, I changed the code sh to bash in two files # splitONT.sh in the first step while aligned001.sh in the second step, and ran both files manually. Now worked and got the final assembly.
Thanks again!