One of the commands exited with non-zero exit code; note that snakemake uses bash strict mode

[M-Asim-Javed]

Hello,

I run Verkko lastest version v1.4.1 with HiFi and ONT data for the haploid genome (25Mb) but got an error twice. Here are the #SplitOnt.err, #process.err, and end part of my #log: Can you please help in this regard? Thanks!

Command

verkko -d verkko_asm --hifi /home/edelab/pacbio/new_assembly/merged_bam_file/chopper_files/filtered_3kb_pbhifi_merged.fastq.gz --nano /home/edelab/pacbio/new_assembly/merged_bam_file/chopper_files/filtered_3kb_ont_merged.fastq.gz

SplitOnt.err file

./splitONT.sh: 2: set: Illegal option -o pipefail

process.err

Gap insertion pass 1.
inserted 0 gaps

Gap insertion pass 2.
inserted 0 gaps

Gap insertion pass 3.
inserted 0 gaps

Haplogap fixing.
mend >124 >560
mend >180 >688
mend >110 <811
mend <485 >709
mend >289 >739
mend >356 <649

Tip unrolling.
unroll 724
unrolled 1 looptips

Unitig creation.

LogError file

[Tue Oct 24 18:53:12 2023]
checkpoint splitTipsONT:
    input: /home/edelab/pacbio/new_assembly/merged_bam_file/chopper_files/filtered_3kb_ont_merged.fastq.gz, 2-processGraph/unitig-unrolled-hifi-resolved.gfa, 3-align/alns-ont.gaf, 3-align/splitONT.finished
    output: 3-alignTips/splitONT.finished
    log: 3-alignTips/splitONT.err
    jobid: 15
    reason: Missing output files: 3-alignTips/splitONT.finished; Input files updated by another job: 2-processGraph/unitig-unrolled-hifi-resolved.gfa, 3-align/alns-ont.gaf
    resources: tmpdir=/tmp, job_id=1, n_cpus=1, mem_gb=8, time_h=96
DAG of jobs will be updated after completion.

[Tue Oct 24 18:53:12 2023]
Error in rule splitTipsONT:
    jobid: 15
    input: /home/edelab/pacbio/new_assembly/merged_bam_file/chopper_files/filtered_3kb_ont_merged.fastq.gz, 2-processGraph/unitig-unrolled-hifi-resolved.gfa, 3-align/alns-ont.gaf, 3-align/splitONT.finished
    output: 3-alignTips/splitONT.finished
    log: 3-alignTips/splitONT.err (check log file(s) for error details)
    shell:
        
cd 3-alignTips

cat > ./splitONT.sh <<EOF
#!/bin/sh
set -e -o pipefail

mkdir -p split

 /home/edelab/miniconda3/envs/verkko/lib/verkko/scripts/find_tips.py ../2-processGraph/unitig-unrolled-hifi-resolved.gfa > tips.fasta 2> find_tips.err
awk '/^>/ { print substr(\$1, 2) }' < tips.fasta > tips.ids
cat ../3-align/alns-ont.gaf |  /home/edelab/miniconda3/envs/verkko/lib/verkko/scripts/filter_alignments_by_column.py tips.ids 5 false | awk -F "\t" '{ if (\$12 >= 20) print \$1;}' | sort |uniq > tip_reads.ids

for fn in ont001.fasta.gz ont003.fasta.gz ont004.fasta.gz ont002.fasta.gz ; do
  echo >> tip_reads.extract extract \$fn tip_reads.ids ../3-align/split/\$fn
done

xargs -L 1 -P 1 < tip_reads.extract \\
   /home/edelab/miniconda3/envs/verkko/lib/verkko/scripts/fasta_extract.py

gunzip -c ont001.fasta.gz ont003.fasta.gz ont004.fasta.gz ont002.fasta.gz |awk -v SEQ="" '{ if (match(\$1, ">") && \$1 != NAME) { if (SEQ != "") { print NAME; print SEQ; } SEQ=""; NAME=\$1; } else { SEQ=SEQ""\$1;} } END { print NAME; print SEQ}' | tr -s ACGT > tip_reads_compressed.fasta
rm -f ont001.fasta.gz ont003.fasta.gz ont004.fasta.gz ont002.fasta.gz

cat ../2-processGraph/unitig-unrolled-hifi-resolved.gfa |awk '{ if (match(\$1, "^S")) { print ">"\$2; print \$3 } }' |fold -b > tips.fasta

bases=\`expr 3000000000 / 2 \`
 /home/edelab/miniconda3/envs/verkko/lib/verkko/scripts/fasta_partition.py \\
  partition split/ont \$bases 150000 0 \\
  tip_reads_compressed.fasta \\
&& \\
touch ../3-alignTips/splitONT.finished
EOF

chmod +x ./splitONT.sh

./splitONT.sh > ../3-alignTips/splitONT.err 2>&1
        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Exiting because a job execution failed. Look above for error message.

[skoren]

Thanks for the detailed logs, this looks to be the same bash/sh issue as #185/186. Updating the code as suggested there should resolve your issue.

[M-Asim-Javed]

Thank you so much for your help. In my case, I changed the code sh to bash in two files # splitONT.sh in the first step while aligned001.sh in the second step, and ran both files manually. Now worked and got the final assembly.

Thanks again!