.command.sh: line 2: 33 Killed minimap2 -x map-ont -d "reference.fasta.mmi" "reference.fasta"
Closed this issue · 8 comments
socialhang commented
Operating System
CentOS 7
Other Linux
No response
Workflow Version
wf-pore-c v1.0.0-g265a1a4
Workflow Execution
Command line
EPI2ME Version
No response
CLI command run
nextflow_cmd.sh :
#######
fq=$1
/public/home/wangwen_lab/zonghang/environment/nextflow/bin/nextflow run epi2me-labs/wf-pore-c --fastq $fq --ref /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/hifiasm_hic_ul_p_utg.fa --cutter DnpII --paired_end_minimum_distance 100 --paired_end_maximum_distance 200 --hi_c --mcool --paired_end --threads 19
sh nextflow_cmd.sh ul10.fq.gz
Workflow Execution - CLI Execution Profile
standard (default)
What happened?
When "minimap2" made "mmi" file, the pipline broken, and "mmi" file is empty!
Relevant log output
Dec-19 21:19:31.050 [main] DEBUG nextflow.cli.Launcher - $> nextflow run epi2me-labs/wf-pore-c --fastq ../ul10.fq.gz --ref /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/hifiasm_hic_ul_p_utg.fa --cutter DnpII --paired_end_minimum_distance 100 --paired_end_maximum_distance 200 --hi_c --mcool --paired_end --threads 19
Dec-19 21:19:31.738 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 23.10.0
Dec-19 21:19:31.896 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/public/home/wangwen_lab/zonghang/.nextflow/plugins; core-plugins: nf-amazon@2.1.4,nf-azure@1.3.2,nf-cloudcache@0.3.0,nf-codecommit@0.1.5,nf-console@1.0.6,nf-ga4gh@1.1.0,nf-google@1.8.3,nf-tower@1.6.3,nf-wave@1.0.0
Dec-19 21:19:32.073 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Dec-19 21:19:32.101 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Dec-19 21:19:32.128 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Dec-19 21:19:32.297 [main] INFO org.pf4j.AbstractPluginManager - No plugins
Dec-19 21:19:32.366 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /public/home/wangwen_lab/zonghang/.nextflow/scm
Dec-19 21:19:38.919 [main] DEBUG nextflow.scm.AssetManager - Git config: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-pore-c.git
Dec-19 21:19:39.117 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
Dec-19 21:19:39.141 [main] DEBUG nextflow.scm.AssetManager - Git config: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-pore-c.git
Dec-19 21:19:42.684 [main] DEBUG nextflow.scm.AssetManager - WARN: Failed to check remote Git revision
org.eclipse.jgit.api.errors.TransportException: https://github.com/epi2me-labs/wf-pore-c.git: cannot open git-upload-pack
at org.eclipse.jgit.api.LsRemoteCommand.execute(LsRemoteCommand.java:192)
at org.eclipse.jgit.api.LsRemoteCommand.call(LsRemoteCommand.java:131)
at nextflow.scm.AssetManager.getRemoteCommitId(AssetManager.groovy:1003)
at nextflow.scm.AssetManager.checkRemoteStatus0(AssetManager.groovy:1014)
at nextflow.scm.AssetManager.checkRemoteStatus(AssetManager.groovy:1032)
at nextflow.cli.CmdRun.getScriptFile0(CmdRun.groovy:540)
at nextflow.cli.CmdRun.getScriptFile(CmdRun.groovy:462)
at nextflow.cli.CmdRun.run(CmdRun.groovy:317)
at nextflow.cli.Launcher.run(Launcher.groovy:499)
at nextflow.cli.Launcher.main(Launcher.groovy:670)
Caused by: org.eclipse.jgit.errors.TransportException: https://github.com/epi2me-labs/wf-pore-c.git: cannot open git-upload-pack
at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:749)
at org.eclipse.jgit.transport.TransportHttp.openFetch(TransportHttp.java:465)
at org.eclipse.jgit.api.LsRemoteCommand.execute(LsRemoteCommand.java:170)
... 9 common frames omitted
Caused by: java.net.UnknownHostException: github.com
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:567)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
at java.base/java.net.Socket.connect(Socket.java:633)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:299)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:178)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:498)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:603)
at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:266)
at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:380)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:189)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1287)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1128)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:175)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
at org.eclipse.jgit.transport.http.JDKHttpConnection.getResponseCode(JDKHttpConnection.java:85)
at org.eclipse.jgit.util.HttpSupport.response(HttpSupport.java:232)
at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:654)
... 11 common frames omitted
Dec-19 21:19:42.835 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/nextflow.config
Dec-19 21:19:42.843 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/nextflow.config
Dec-19 21:19:42.917 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Dec-19 21:19:44.559 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Dec-19 21:19:44.564 [main] INFO nextflow.cli.CmdRun - Launching `https://github.com/epi2me-labs/wf-pore-c` [berserk_rubens] DSL2 - revision: 265a1a4f88 [master]
Dec-19 21:19:44.566 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Dec-19 21:19:44.566 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Dec-19 21:19:44.581 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /public/home/wangwen_lab/zonghang/.nextflow/secrets/store.json
Dec-19 21:19:44.604 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@60f77af] - activable => nextflow.secret.LocalSecretsProvider@60f77af
Dec-19 21:19:45.024 [main] DEBUG nextflow.Session - Session UUID: deb9d9c7-3135-424c-a79d-964e40ab23a3
Dec-19 21:19:45.026 [main] DEBUG nextflow.Session - Run name: berserk_rubens
Dec-19 21:19:45.029 [main] DEBUG nextflow.Session - Executor pool size: 128
Dec-19 21:19:45.068 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Dec-19 21:19:45.098 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=384; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Dec-19 21:19:45.651 [main] DEBUG nextflow.cli.CmdRun -
Version: 23.10.0 build 5889
Created: 15-10-2023 15:07 UTC (23:07 CDT)
System: Linux 3.10.0-957.el7.x86_64
Runtime: Groovy 3.0.19 on Java HotSpot(TM) 64-Bit Server VM 17.0.2+8-LTS-86
Encoding: UTF-8 (UTF-8)
Process: 70259@node12 [10.10.10.12]
CPUs: 128 - Mem: 251.6 GB (16.6 GB) - Swap: 0 (0)
Dec-19 21:19:45.902 [main] DEBUG nextflow.Session - Work-dir: /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/ul10.fq.gz.tmp/work [nfs]
Dec-19 21:19:46.009 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Dec-19 21:19:46.167 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Dec-19 21:19:46.694 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Dec-19 21:19:46.950 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 129; maxThreads: 1000
Dec-19 21:19:47.832 [main] DEBUG nextflow.Session - Session start
Dec-19 21:19:47.857 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/ul10.fq.gz.tmp/output/execution/trace.txt
Dec-19 21:19:47.948 [main] DEBUG nextflow.Session - Using default localLib path: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/lib
Dec-19 21:19:47.969 [main] DEBUG nextflow.Session - Adding to the classpath library: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/lib
Dec-19 21:19:47.972 [main] DEBUG nextflow.Session - Adding to the classpath library: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/lib/nfcore_external_java_deps.jar
Dec-19 21:19:56.063 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Dec-19 21:20:02.385 [main] INFO nextflow.Nextflow -
|||||||||| _____ ____ ___ ____ __ __ _____ _ _
|||||||||| | ____| _ \_ _|___ \| \/ | ____| | | __ _| |__ ___
||||| | _| | |_) | | __) | |\/| | _| _____| |/ _` | '_ \/ __|
||||| | |___| __/| | / __/| | | | |__|_____| | (_| | |_) \__ \
|||||||||| |_____|_| |___|_____|_| |_|_____| |_|\__,_|_.__/|___/
|||||||||| wf-pore-c v1.0.0-g265a1a4
--------------------------------------------------------------------------------
Core Nextflow options
revision : master
runName : berserk_rubens
containerEngine : docker
container : [withLabel:wfporec:ontresearch/wf-pore-c:latest, withLabel:wf_common:ontresearch/wf-common:sha91452ece4f647f62b32dac3a614635a6f0d7f8b5]
launchDir : /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/ul10.fq.gz.tmp
workDir : /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/ul10.fq.gz.tmp/work
projectDir : /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c
userName : zonghang
profile : standard
configFiles : /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/nextflow.config
Input Options
fastq : ../ul10.fq.gz
ref : /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/hifiasm_hic_ul_p_utg.fa
cutter : DnpII
Output Options
hi_c : true
Advanced Options
threads : 19
4DN files Options
mcool : true
Paired-end BAM Options
paired_end : true
paired_end_minimum_distance: 100
paired_end_maximum_distance: 200
!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-pore-c for your analysis please cite:
* The nf-core framework
https://doi.org/10.1038/s41587-020-0439-x
--------------------------------------------------------------------------------
This is epi2me-labs/wf-pore-c v1.0.0-g265a1a4.
--------------------------------------------------------------------------------
Dec-19 21:20:02.782 [main] INFO nextflow.Nextflow - Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
Dec-19 21:20:03.547 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wf_common` matches labels `ingress,wf_common` for process with name POREC:fastcat
Dec-19 21:20:03.574 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:03.578 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:03.602 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Dec-19 21:20:03.719 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=128; memory=251.6 GB; capacity=128; pollInterval=100ms; dumpInterval=5m
Dec-19 21:20:03.730 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
Dec-19 21:20:04.220 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:index_bam
Dec-19 21:20:04.224 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:04.224 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:04.558 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:prepare_genome:index_ref_mmi
Dec-19 21:20:04.563 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:04.564 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:04.697 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:digest_align_annotate
Dec-19 21:20:04.706 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:04.708 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:04.792 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:haplotag_alignments
Dec-19 21:20:04.805 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:04.807 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:04.895 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:merge_coordsorted_bams
Dec-19 21:20:04.898 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:merge_coordsorted_bams|merge_namesorted_bams` matches process POREC:merge_coordsorted_bams
Dec-19 21:20:04.906 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:04.907 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:04.953 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:merge_namesorted_bams
Dec-19 21:20:04.956 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:merge_coordsorted_bams|merge_namesorted_bams` matches process POREC:merge_namesorted_bams
Dec-19 21:20:04.958 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:04.958 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.120 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:create_restriction_bed
Dec-19 21:20:05.124 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:merge_pairs|merge_pairs_stats|merge_mcools|create_restriction_bed|pair_stats_report` matches process POREC:create_restriction_bed
Dec-19 21:20:05.126 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.127 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.290 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:to_pairs_file
Dec-19 21:20:05.310 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.312 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.352 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:pairsToCooler
Dec-19 21:20:05.356 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.357 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.405 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:merge_mcools
Dec-19 21:20:05.410 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:merge_pairs|merge_pairs_stats|merge_mcools|create_restriction_bed|pair_stats_report` matches process POREC:merge_mcools
Dec-19 21:20:05.415 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.416 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.463 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:merge_pairs
Dec-19 21:20:05.469 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:merge_pairs|merge_pairs_stats|merge_mcools|create_restriction_bed|pair_stats_report` matches process POREC:merge_pairs
Dec-19 21:20:05.478 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.480 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.500 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:merge_pairs_stats
Dec-19 21:20:05.516 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:merge_pairs|merge_pairs_stats|merge_mcools|create_restriction_bed|pair_stats_report` matches process POREC:merge_pairs_stats
Dec-19 21:20:05.526 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.527 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.543 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:pair_stats_report
Dec-19 21:20:05.545 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:merge_pairs|merge_pairs_stats|merge_mcools|create_restriction_bed|pair_stats_report` matches process POREC:pair_stats_report
Dec-19 21:20:05.550 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.550 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.604 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:merge_paired_end_bams
Dec-19 21:20:05.606 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:merge_coordsorted_bams|merge_namesorted_bams` matches process POREC:merge_paired_end_bams
Dec-19 21:20:05.607 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:merge_paired_end_bams` matches process POREC:merge_paired_end_bams
Dec-19 21:20:05.627 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.632 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.649 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:getVersions
Dec-19 21:20:05.677 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.679 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.700 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:getParams
Dec-19 21:20:05.703 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.703 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.745 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:makeReport
Dec-19 21:20:05.747 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:makeReport` matches process POREC:makeReport
Dec-19 21:20:05.748 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.749 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.772 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name POREC:prepare_hic
Dec-19 21:20:05.774 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:prepare_hic` matches process POREC:prepare_hic
Dec-19 21:20:05.776 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.776 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.813 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:wfporec` matches labels `wfporec` for process with name output
Dec-19 21:20:05.816 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Dec-19 21:20:05.816 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Dec-19 21:20:05.835 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: move_or_compress_fq_file, haplotag_alignments, pair_stats_report, haplotagReads, POREC:pairsToCooler, POREC:makeReport, POREC:prepare_genome:index_ref_mmi, POREC:merge_coordsorted_bams, prepare_hic, checkBamHeaders, merge_pairs_stats, POREC:merge_mcools, mergeBams, output, POREC:getVersions, makeReport, merge_coordsorted_bams, POREC:prepare_hic, getVersions, POREC:merge_paired_end_bams, POREC:fastcat, index_ref_mmi, POREC:merge_namesorted_bams, mosdepth_coverage, merge_pairs, bamstats, merge_namesorted_bams, fastcat, pairsToCooler, POREC:to_pairs_file, to_pairs_file, merge_paired_end_bams, getParams, POREC:merge_pairs, index_bam, sortBam, merge_parquets_to_dataset, validate_sample_sheet, POREC:index_bam, POREC:merge_pairs_stats, POREC:getParams, POREC:pair_stats_report, index_ref_fai, digest_align_annotate, create_restriction_bed, publish_artifact, POREC:create_restriction_bed, POREC:haplotag_alignments, merge_mcools, POREC:digest_align_annotate, decompress_ref, catSortBams
Dec-19 21:20:05.841 [main] DEBUG nextflow.Session - Igniting dataflow network (26)
Dec-19 21:20:05.846 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:fastcat
Dec-19 21:20:05.851 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:index_bam
Dec-19 21:20:06.108 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:prepare_genome:index_ref_mmi
Dec-19 21:20:06.146 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:digest_align_annotate
Dec-19 21:20:06.163 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:haplotag_alignments
Dec-19 21:20:06.191 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:merge_coordsorted_bams
Dec-19 21:20:06.213 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:merge_namesorted_bams
Dec-19 21:20:06.248 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:create_restriction_bed
Dec-19 21:20:06.264 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:to_pairs_file
Dec-19 21:20:06.303 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:pairsToCooler
Dec-19 21:20:06.313 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:merge_mcools
Dec-19 21:20:06.339 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:merge_pairs
Dec-19 21:20:06.386 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:merge_pairs_stats
Dec-19 21:20:06.406 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:pair_stats_report
Dec-19 21:20:06.411 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:merge_paired_end_bams
Dec-19 21:20:06.426 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:getVersions
Dec-19 21:20:06.440 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:getParams
Dec-19 21:20:06.452 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:makeReport
Dec-19 21:20:06.489 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > POREC:prepare_hic
Dec-19 21:20:06.528 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > output
Dec-19 21:20:06.573 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
Script_75bdc7148204fc84: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/main.nf
Script_cf4f527e2875872f: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/./subworkflows/local/../../modules/local/common.nf
Script_94fc0468daf7bee0: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/./modules/local/4dn.nf
Script_6f2dfe738e879fd7: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/./subworkflows/local/prepare_genome.nf
Script_f885de694c629aa8: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/./lib/ingress.nf
Script_9757b38690d83d3c: /public/home/wangwen_lab/zonghang/.nextflow/assets/epi2me-labs/wf-pore-c/./modules/local/pore-c.nf
Dec-19 21:20:06.581 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination
Dec-19 21:20:06.582 [main] DEBUG nextflow.Session - Session await
Dec-19 21:20:07.568 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Dec-19 21:20:07.627 [Task submitter] INFO nextflow.Session - [f7/fb9e13] Submitted process > POREC:getParams
Dec-19 21:20:07.698 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Dec-19 21:20:07.703 [Task submitter] INFO nextflow.Session - [dc/0fee17] Submitted process > POREC:fastcat (1)
Dec-19 21:20:07.766 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Dec-19 21:20:07.772 [Task submitter] INFO nextflow.Session - [08/0d9e07] Submitted process > POREC:prepare_genome:index_ref_mmi (1)
Dec-19 21:20:07.869 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Dec-19 21:20:07.878 [Task submitter] INFO nextflow.Session - [f9/bdeefa] Submitted process > POREC:getVersions
Dec-19 21:20:11.735 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: POREC:getParams; status: COMPLETED; exit: 0; error: -; workDir: /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/ul10.fq.gz.tmp/work/f7/fb9e1302a547ddc45f426ccd7df105]
Dec-19 21:20:28.238 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: POREC:getVersions; status: COMPLETED; exit: 0; error: -; workDir: /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/ul10.fq.gz.tmp/work/f9/bdeefad45a3f66b500fc272def7da0]
Dec-19 21:23:24.193 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: POREC:prepare_genome:index_ref_mmi (1); status: COMPLETED; exit: 137; error: -; workDir: /public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/ul10.fq.gz.tmp/work/08/0d9e07be05828139553081644961eb]
Dec-19 21:23:24.268 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
task: name=POREC:prepare_genome:index_ref_mmi (1); work-dir=/public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/ul10.fq.gz.tmp/work/08/0d9e07be05828139553081644961eb
error [nextflow.exception.ProcessFailedException]: Process `POREC:prepare_genome:index_ref_mmi (1)` terminated with an error exit status (137)
Dec-19 21:23:24.580 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'POREC:prepare_genome:index_ref_mmi (1)'
Caused by:
Process `POREC:prepare_genome:index_ref_mmi (1)` terminated with an error exit status (137)
Command executed:
minimap2 -x map-ont -d "reference.fasta.mmi" "reference.fasta"
Command exit status:
137
Command output:
(empty)
Command error:
[M::mm_idx_gen::173.163*1.62] collected minimizers
.command.sh: line 2: 33 Killed minimap2 -x map-ont -d "reference.fasta.mmi" "reference.fasta"
Work dir:
/public/home/wangwen_lab/zonghang/project/new_muxu_project/20.T2T/03.assembly_test/14.porec/02.hifiasm_hifi_ul_hic_utg/ul10.fq.gz.tmp/work/08/0d9e07be05828139553081644961eb
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
Dec-19 21:23:24.664 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `POREC:prepare_genome:index_ref_mmi (1)` terminated with an error exit status (137)
Dec-19 21:23:24.894 [Task monitor] DEBUG nextflow.Session - The following nodes are still active:
[process] POREC:index_bam
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (value) bound ; channel: chunk_size
port 2: (cntrl) - ; channel: $
[process] POREC:digest_align_annotate
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:haplotag_alignments
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:merge_coordsorted_bams
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:merge_namesorted_bams
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:create_restriction_bed
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:to_pairs_file
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:pairsToCooler
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:merge_mcools
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:merge_pairs
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:merge_pairs_stats
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:pair_stats_report
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:merge_paired_end_bams
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] POREC:makeReport
status=ACTIVE
port 0: (value) OPEN ; channel: metadata
port 1: (value) OPEN ; channel: per_read_stats/{?}.gz
port 2: (value) bound ; channel: versions/*
port 3: (value) bound ; channel: params.json
port 4: (cntrl) - ; channel: $
[process] POREC:prepare_hic
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
[process] output
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (cntrl) - ; channel: $
Dec-19 21:23:25.088 [main] DEBUG nextflow.Session - Session await > all processes finished
Dec-19 21:23:25.091 [main] DEBUG nextflow.Session - Session await > all barriers passed
Dec-19 21:23:25.130 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Dec-19 21:23:25.322 [main] WARN n.processor.TaskPollingMonitor - Killing running tasks (1)
Dec-19 21:23:25.492 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=2; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=16.5s; failedDuration=13m 5s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=4; peakCpus=9; peakMemory=22 GB; ]
Dec-19 21:23:25.494 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Dec-19 21:23:25.510 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Dec-19 21:23:27.179 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
Dec-19 21:23:27.565 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Dec-19 21:23:27.677 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- GoodbyeApplication activity log entry
No response
cjw85 commented
Hi again,
Thank you for that. The error status 137 indicates that the process was killed by your OS. The most common cause of this, and what I suspect is happening here is that your computer has run out of memory.
See https://labs.epi2me.io/how-to-exits/ for more information on common exit codes.
socialhang commented
hello, thanks for your responds. I don't think its memory issues. When I use minimap2 without pipline, its running ok.
could you give me more information about this ?
cjw85 commented
There's not much more information I can give. I suggest working with your local IT staff to track the memory usage of the process and understand why the OS is killing the process.
socialhang commented
OK，thanks！And I will try this method below！
sarahjeeeze commented
Hi, were you able to resolve this issue in the end?
gotouerina commented
Config file I used:
process {
withLabel:"wfporec" {
cpus=64
memory = 256.GB
}
}
In addition, I think CPU set in your .nf files are not enough, modify them with higher CPU and RAM would increase performance greatly.
sarahjeeeze commented
Hi, so did you get it to work? We have since modified them
sarahjeeeze commented
closing through lack of response and we have hopefully fixed this in more recent release.