aws specific crash with multiqc
cariaso opened this issue · 3 comments
cariaso commented
commandline
nextflow -c nfconf.txt run nf-core/hlatyping -profile awsbatch,test --awsqueue arn:aws:batch:eu-west-1:123456789:job-queue/test123aqueue --awsregion eu-west-1 --awscli /usr/local/bin/aws -bucket-dir s3://test123-nf-logs/hlarun/test1 --outdir s3://test123-nf-runs/hlarun/test1
N E X T F L O W ~ version 20.07.1
Launching `nf-core/hlatyping` [festering_gutenberg] - revision: 6998794795 [master]
BAM file format detected. Initiate remapping to HLA alleles with yara mapper.
----------------------------------------------------
,--./,-.
___ __ __ __ ___ /,-._.--~'
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/hlatyping v1.2.0
----------------------------------------------------
Pipeline Release : master
Run Name : festering_gutenberg
File Type : BAM
Seq Type : dna
Index Location : /Users/cariaso/.nextflow/assets/nf-core/hlatyping/data/indices/yara/hla_reference_dna
IP Solver : glpk
Enumerations : 1
Beta : 0.009
Max Memory : 6 GB
Max CPUs : 2
Max Time : 2d
Input : https://github.com/nf-core/test-datasets/raw/hlatyping/bam/example_pe.bam
Data Type : Paired-End
Output Dir : s3://test123-nf-runs/hlarun/test1
Launch Dir : /Users/cariaso/dev/curious1
Working Dir : /Users/cariaso/dev/curious1/work
Script Dir : /Users/cariaso/.nextflow/assets/nf-core/hlatyping
User : cariaso
Max Resources : 6 GB memory, 2 cpus, 2d time per job
AWS Region : eu-west-1
AWS Queue : arn:aws:batch:eu-west-1:123456789:job-queue/test123aqueue
AWS CLI : /usr/local/bin/aws
Config Profile : awsbatch,test
Config Profile Description: Minimal test dataset to check pipeline function
Config Profile Contact: Alexander Peltzer (@apeltzer)
Config Profile URL: https://aws.amazon.com/batch/
Config Files : /Users/cariaso/.nextflow/assets/nf-core/hlatyping/nextflow.config, /Users/cariaso/dev/curious1/nfconf.txt
----------------------------------------------------
BAM file format detected. Initiate remapping to HLA alleles with yara mapper.
Uploading local `bin` scripts folder to s3://test123-nf-logs/hlarun/test1/tmp/28/5d8fc46240449a98bef8f29904c74c/bin
WARN: Config setting `executor.awscli` has been deprecated -- Use instead `aws.batch.cliPath`
executor > awsbatch (6)
[31/c83122] process > remap_to_hla (1) [100%] 1 of 1 ✔
[4a/231f75] process > make_ot_config [100%] 1 of 1 ✔
[79/7b305c] process > run_optitype (1) [ 0%] 0 of 1
[cb/47df95] process > output_documentation [100%] 1 of 1 ✔
[a9/03ac98] process > get_software_versions [100%] 1 of 1 ✔
executor > awsbatch (6)
[31/c83122] process > remap_to_hla (1) [100%] 1 of 1 ✔
[4a/231f75] process > make_ot_config [100%] 1 of 1 ✔
[79/7b305c] process > run_optitype (1) [ 0%] 0 of 1
[cb/47df95] process > output_documentation [100%] 1 of 1 ✔
[a9/03ac98] process > get_software_versions [100%] 1 of 1 ✔
executor > awsbatch (6)
[31/c83122] process > remap_to_hla (1) [100%] 1 of 1 ✔
[4a/231f75] process > make_ot_config [100%] 1 of 1 ✔
[79/7b305c] process > run_optitype (1) [ 0%] 0 of 1
[cb/47df95] process > output_documentation [100%] 1 of 1 ✔
[a9/03ac98] process > get_software_versions [100%] 1 of 1 ✔
executor > awsbatch (6)
[31/c83122] process > remap_to_hla (1) [100%] 1 of 1 ✔
[4a/231f75] process > make_ot_config [100%] 1 of 1 ✔
[79/7b305c] process > run_optitype (1) [ 0%] 0 of 1
[cb/47df95] process > output_documentation [100%] 1 of 1 ✔
[a9/03ac98] process > get_software_versions [100%] 1 of 1 ✔
executor > awsbatch (6)
[31/c83122] process > remap_to_hla (1) [100%] 1 of 1 ✔
[4a/231f75] process > make_ot_config [100%] 1 of 1 ✔
[79/7b305c] process > run_optitype (1) [ 0%] 0 of 1 ✔
[cb/47df95] process > output_documentation [100%] 1 of 1 ✔
[a9/03ac98] process > get_software_versions [100%] 1 of 1 ✔
executor > awsbatch (6)
[31/c83122] process > remap_to_hla (1) [100%] 1 of 1 ✔
[4a/231f75] process > make_ot_config [100%] 1 of 1 ✔
[79/7b305c] process > run_optitype (1) [100%] 1 of 1 ✔
[cb/47df95] process > output_documentation [100%] 1 of 1 ✔
[a9/03ac98] process > get_software_versions [100%] 1 of 1 ✔
executor > awsbatch (6)
[31/c83122] process > remap_to_hla (1) [100%] 1 of 1 ✔
[4a/231f75] process > make_ot_config [100%] 1 of 1 ✔
[79/7b305c] process > run_optitype (1) [100%] 1 of 1 ✔
[cb/47df95] process > output_documentation [100%] 1 of 1 ✔
[a9/03ac98] process > get_software_versions [100%] 1 of 1 ✔
executor > awsbatch (6)
[31/c83122] process > remap_to_hla (1) [100%] 1 of 1 ✔
[4a/231f75] process > make_ot_config [100%] 1 of 1 ✔
[79/7b305c] process > run_optitype (1) [100%] 1 of 1 ✔
[cb/47df95] process > output_documentation [100%] 1 of 1 ✔
[a9/03ac98] process > get_software_versions [100%] 1 of 1 ✔
[ee/d9ce35] process > multiqc (1) [100%] 1 of 1, failed: 1 ✘
-[nf-core/hlatyping] Pipeline completed with errors-
Error executing process > 'multiqc (1)'
Caused by:
Missing output file(s) `multiqc_plots` expected by process `multiqc (1)`
Command executed:
multiqc -f .
Command exit status:
0
Command output:
(empty)
Command error:
[INFO ] multiqc : This is MultiQC v1.9
[INFO ] multiqc : Template : default
[INFO ] multiqc : Searching : /tmp/nxf.zWcEb6mGfi
[INFO ] custom_content : nf-core-hlatyping-summary: Found 1 sample (html)
[INFO ] custom_content : software_versions: Found 1 sample (html)
[INFO ] multiqc : Compressing plot data
[INFO ] multiqc : Report : multiqc_report.html
[INFO ] multiqc : Data : multiqc_data
[INFO ] multiqc : Plots : multiqc_plots
[INFO ] multiqc : MultiQC complete
Work dir:
s3://test123-nf-logs/hlarun/test1/ee/d9ce3573511c7a3891f1bb332742f3
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
contents of nfconf.txt are just
params {
monochrome_logs = true
}
profiles {
local {
process.executor = 'local'
docker.enabled = true
}
awsbatch {
process {
executor = 'awsbatch'
queue = "arn:aws:batch:eu-west-1:123456789:job-queue/test123aqueue"
}
}
}
szymonwieloch commented
I experience the same issue.
vsmalladi commented
Fixed with new optional mutliqc