error running Aeron software
sandybioteck opened this issue · 4 comments
Hi,
I am interested in using Aeron software for detecting fusions from nanopore data.
I am trying to run Aeron on the example files which are provided, however, I am getting following error:
SyntaxError:
Not all output, log and benchmark files of rule assign_reads_to_transcripts contain the same wildcards. This is crucial though, in order to avoid that two or more jobs write to the same file.
File "/sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/aeron/snakemake_pipeline/Snakefile", line 99, in
I have downloaded from both the sources (i.e. https://github.com/SchulzLab/Aeron as well as https://bitbucket.org/dilipdurai/aeron) and from both the sources I got the same error.
Can you please help me to run the software successfully?
Thanks,
Sandeep
Hi Sandeep,
There was an issue with the snakemake file. Could you please try again with the newest version from this repo?
Hi,
I am getting following error now while running the software
(I am using following version of dependency software: vg version 1.22, snakemake version 5.2.2, python version 3.6.6)
Please help.
sandeep@Aeron: snakemake --cores=1
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1
Rules claiming more threads will be scaled down.
Job counts:
count jobs
2 align
1 all
1 assign_reads_to_transcripts
1 convert_json
1 find_best_assignments
1 find_unambiguous_assignments
1 generateCountMatrix
2 output_alignment_statistics
1 output_assignment_statistics
2 postprocess
13
[Fri Apr 3 12:53:33 2020]
rule align:
input: input/hg38.gfa, input/SampleDataset.fq
output: output/aln_SampleDataset_hg38_all.gam
log: tmp/aligner_stdout_SampleDataset_hg38.txt, tmp/aligner_stderr_SampleDataset_hg38.txt
jobid: 6
benchmark: benchmark/aln_SampleDataset_hg38_all.txt
wildcards: reads=SampleDataset, graph=hg38
[Fri Apr 3 12:53:49 2020]
Finished job 6.
1 of 13 steps (8%) done
[Fri Apr 3 12:53:49 2020]
rule postprocess:
input: output/aln_SampleDataset_hg38_all.gam, input/SampleDataset.fq
output: output/aln_SampleDataset_hg38_selected.gam, output/aln_SampleDataset_hg38_full_length.gam, tmp/run_SampleDataset_hg38_summary.txt
jobid: 2
benchmark: benchmark/postprocess_SampleDataset_hg38.txt
wildcards: reads=SampleDataset, graph=hg38
[Fri Apr 3 12:53:51 2020]
Finished job 2.
2 of 13 steps (15%) done
[Fri Apr 3 12:53:51 2020]
rule align:
input: input/hg38.gfa, input/ensembl89HomosapiensGRCh38cdnaall.fasta
output: output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_all.gam
log: tmp/aligner_stdout_ensembl89HomosapiensGRCh38cdnaall_hg38.txt, tmp/aligner_stderr_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
jobid: 12
benchmark: benchmark/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_all.txt
wildcards: reads=ensembl89HomosapiensGRCh38cdnaall, graph=hg38
[Fri Apr 3 12:54:17 2020]
Finished job 12.
3 of 13 steps (23%) done
[Fri Apr 3 12:54:17 2020]
rule postprocess:
input: output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_all.gam, input/ensembl89HomosapiensGRCh38cdnaall.fasta
output: output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_selected.gam, output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_full_length.gam, tmp/run_ensembl89HomosapiensGRCh38cdnaall_hg38_summary.txt
jobid: 5
benchmark: benchmark/postprocess_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
wildcards: reads=ensembl89HomosapiensGRCh38cdnaall, graph=hg38
[Fri Apr 3 12:54:20 2020]
Finished job 5.
4 of 13 steps (31%) done
[Fri Apr 3 12:54:20 2020]
rule assign_reads_to_transcripts:
input: output/aln_SampleDataset_hg38_selected.gam, output/aln_ensembl89HomosapiensGRCh38cdnaall_hg38_full_length.gam, input/SampleDataset.fq
output: output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_all.txt
jobid: 11
benchmark: benchmark/assign_reads_to_transcript_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
wildcards: reads=SampleDataset, transcripts=ensembl89HomosapiensGRCh38cdnaall, graph=hg38
[Fri Apr 3 12:54:21 2020]
Finished job 11.
5 of 13 steps (38%) done
[Fri Apr 3 12:54:21 2020]
rule output_alignment_statistics:
input: tmp/run_ensembl89HomosapiensGRCh38cdnaall_hg38_summary.txt, tmp/aligner_stdout_ensembl89HomosapiensGRCh38cdnaall_hg38.txt, tmp/aligner_stderr_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
output: output/alignmentstats_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
jobid: 3
wildcards: reads=ensembl89HomosapiensGRCh38cdnaall, graph=hg38
Job counts:
count jobs
1 output_alignment_statistics
1
[Fri Apr 3 12:54:22 2020]
Finished job 3.
6 of 13 steps (46%) done
[Fri Apr 3 12:54:22 2020]
rule output_alignment_statistics:
input: tmp/run_SampleDataset_hg38_summary.txt, tmp/aligner_stdout_SampleDataset_hg38.txt, tmp/aligner_stderr_SampleDataset_hg38.txt
output: output/alignmentstats_SampleDataset_hg38.txt
jobid: 9
wildcards: reads=SampleDataset, graph=hg38
Job counts:
count jobs
1 output_alignment_statistics
1
[Fri Apr 3 12:54:23 2020]
Finished job 9.
7 of 13 steps (54%) done
[Fri Apr 3 12:54:23 2020]
rule find_unambiguous_assignments:
input: output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_all.txt
output: output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_unambiguous.txt
jobid: 1
benchmark: benchmark/matrix_unambigous_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
wildcards: runid=SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38
[Fri Apr 3 12:54:23 2020]
Finished job 1.
8 of 13 steps (62%) done
[Fri Apr 3 12:54:23 2020]
**rule convert_json:
input: output/aln_SampleDataset_hg38_selected.gam
output: output/aln_SampleDataset_hg38_selected.json
jobid: 7
benchmark: benchmark/convertjson_SampleDataset_hg38.txt
wildcards: reads=SampleDataset, graph=hg38
vg: variation graph tool, version v1.22.0 "Rotella"
usage: vg [options]
main mapping and calling pipeline:
-- construct graph construction
-- index index graphs or alignments for random access or mapping
-- map MEM-based read alignment
-- augment augment a graph from an alignment
-- pack convert alignments to a compact coverage index
-- call call or genotype VCF variants
-- help show all subcommands
For more commands, type vg help.
For technical support, please visit: https://www.biostars.org/t/vg/
**[Fri Apr 3 12:54:24 2020]
Error in rule convert_json:
jobid: 7
output: output/aln_SampleDataset_hg38_selected.json
RuleException:
CalledProcessError in line 89 of /sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/Snakefile:
Command ' set -euo pipefail; ~/bin/vg view -a output/aln_SampleDataset_hg38_selected.gam > output/aln_SampleDataset_hg38_selected.json ' returned non-zero exit status 1.
File "/sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/Snakefile", line 89, in __rule_convert_json
File "/apps/software/standard/core/anaconda/5.2.0-py3.6/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Removing output files of failed job convert_json since they might be corrupted:
output/aln_SampleDataset_hg38_selected.json
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message****
the above problem was solved. However now getting this error (possibly because of python indentation problem, however, I can't find it).
[Fri Apr 3 14:27:01 2020]
rule generateCountMatrix:
input: output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_bestmatch.txt, input/HomosapiensGRCh3889.gtf, output/aln_SampleDataset_hg38_selected.json
output: output/CountMatrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
jobid: 3
benchmark: benchmark/generateCount_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
wildcards: reads=SampleDataset, transcripts=ensembl89HomosapiensGRCh38cdnaall, graph=hg38
Traceback (most recent call last):
File "AeronScripts/ThreePrime.py", line 3, in
from ParseGTF import *
File "/sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/AeronScripts/ParseGTF.py", line 69
def getTranscriptPosition(self, key):
^
TabError: inconsistent use of tabs and spaces in indentation
[Fri Apr 3 14:27:01 2020]
Error in rule generateCountMatrix:
jobid: 3
output: output/CountMatrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
RuleException:
CalledProcessError in line 201 of /sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/Snakefile:
Command ' set -euo pipefail; python AeronScripts/ThreePrime.py -g input/HomosapiensGRCh3889.gtf -m output/matrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38_bestmatch.txt -j output/aln_SampleDataset_hg38_selected.json >> output/CountMatrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt ' returned non-zero exit status 1.
File "/sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/Snakefile", line 201, in __rule_generateCountMatrix
File "/apps/software/standard/core/anaconda/5.2.0-py3.6/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Removing output files of failed job generateCountMatrix since they might be corrupted:
output/CountMatrix_SampleDataset_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /sfs/lustre/bahamut/scratch/ss7mh/softwares/AERON/Aeron/.snakemake/log/2020-04-03T142607.895574.snakemake.log
After correcting the indentation problem in one of the python scripts, the processes successfully completed, however many of the output files are empty
ls -lrt
total 7
-rw-r--r-- 1 ss7mh users 21 Apr 3 15:38 aln_both_hg38_all.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:38 aln_both_hg38_selected.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:38 aln_both_hg38_full_length.gam
-rw-r--r-- 1 ss7mh users 21 Apr 3 15:39 aln_ensembl89HomosapiensGRCh38cdnaall_hg38_all.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 aln_ensembl89HomosapiensGRCh38cdnaall_hg38_selected.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 aln_ensembl89HomosapiensGRCh38cdnaall_hg38_full_length.gam
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 matrix_both_ensembl89HomosapiensGRCh38cdnaall_hg38_all.txt
-rw-r--r-- 1 ss7mh users 1002 Apr 3 15:39 alignmentstats_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 matrix_both_ensembl89HomosapiensGRCh38cdnaall_hg38_bestmatch.txt
-rw-r--r-- 1 ss7mh users 947 Apr 3 15:39 alignmentstats_both_hg38.txt
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 aln_both_hg38_selected.json
-rw-r--r-- 1 ss7mh users 294 Apr 3 15:39 matrixstats_both_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
-rw-r--r-- 1 ss7mh users 0 Apr 3 15:39 matrix_both_ensembl89HomosapiensGRCh38cdnaall_hg38_unambiguous.txt
-rw-r--r-- 1 ss7mh users 17 Apr 3 15:39 CountMatrix_both_ensembl89HomosapiensGRCh38cdnaall_hg38.txt
