error
Closed this issue · 11 comments
Chris091089 commented
NOTE: Process BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py) terminated with an error exit status (127) -- Execution is retried (1)
ERROR ~ Error executing process > 'BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)'
Caused by:
Process BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py) terminated with an error exit status (127)
biocoder commented
Exit status 127 is generally a $PATH issue. What's the memory of the system where you are running it? Can you also upload the .nextflow.log file from where you started this pipeline? It should contain some additional details
Chris091089 commented
i have around 1TB of memory and change the location of the directory for
the db but have the same problem
El mar, 12 dic 2023 a las 11:41, Kranti Konganti ***@***.***>)
escribió:
… Exit status 127 is generally a $PATH issue. What's the memory of
the system where you are running it? Can you also upload the .nextflow.log
file from where you started this pipeline? It should contain some
additional details
—
Reply to this email directly, view it on GitHub
<#1 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJWAA4BPJYCFWS24LOFKRI3YJCJMFAVCNFSM6AAAAABAR3FXSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSGUYTANJUHA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
biocoder commented
Hello Chris,
Please attach the .nextflow.log file to this issue. It should be in the same directory from where you executed the workflow. Can you paste the command you are trying to execute?
Thanks,
Kranti
Chris091089 commented
what is the line to display the log file?
El mar, 12 dic 2023 a las 16:35, Kranti Konganti ***@***.***>)
escribió:
… Hello Chris,
Please attach the .nextflow.log file to this issue. It should be in the
same directory from where you executed the workflow. Can you paste the
command you are trying to execute?
Thanks,
Kranti
—
Reply to this email directly, view it on GitHub
<#1 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJWAA4DPXWQFXSYNKYKUGXLYJDL45AVCNFSM6AAAAABAR3FXSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSHEZDCOBZG4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
Chris091089 commented
the line I am executing is the following:
cpipes --pipeline bettercallsal_db --pdg_release PDG000000002.2727 --output
/home/chris_franco/bin/bases_de_datos/better_db/
El mar, 12 dic 2023 a las 16:35, Kranti Konganti ***@***.***>)
escribió:
… Hello Chris,
Please attach the .nextflow.log file to this issue. It should be in the
same directory from where you executed the workflow. Can you paste the
command you are trying to execute?
Thanks,
Kranti
—
Reply to this email directly, view it on GitHub
<#1 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJWAA4DPXWQFXSYNKYKUGXLYJDL45AVCNFSM6AAAAABAR3FXSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSHEZDCOBZG4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
biocoder commented
what is the line to display the log file? El mar, 12 dic 2023 a las 16:35, Kranti Konganti @.>) escribió:
…
Hello Chris, Please attach the .nextflow.log file to this issue. It should be in the same directory from where you executed the workflow. Can you paste the command you are trying to execute? Thanks, Kranti — Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJWAA4DPXWQFXSYNKYKUGXLYJDL45AVCNFSM6AAAAABAR3FXSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSHEZDCOBZG4 . You are receiving this because you authored the thread.Message ID: @.>
Just do cat .nextflow.log from where you ran that cpipes ....... command and it should print the output. Paste the output here
Chris091089 commented
cpipes --pipeline bettercallsal_db --pdg_release PDG000000002.2727 --output
/home/chris_franco/bin/bases_de_datos/better_db/
all the best
El mié, 13 dic 2023 a las 13:23, Kranti Konganti ***@***.***>)
escribió:
what is the line to display the log file? El mar, 12 dic 2023 a las 16:35,
Kranti Konganti *@*.
*>) escribió: … <#m_2671920594360010536_> Hello Chris, Please attach the
.nextflow.log file to this issue. It should be in the same directory from
where you executed the workflow. Can you paste the command you are trying
to execute? Thanks, Kranti — Reply to this email directly, view it on
GitHub <#1 (comment)
<#1 (comment)>>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJWAA4DPXWQFXSYNKYKUGXLYJDL45AVCNFSM6AAAAABAR3FXSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSHEZDCOBZG4
<https://github.com/notifications/unsubscribe-auth/AJWAA4DPXWQFXSYNKYKUGXLYJDL45AVCNFSM6AAAAABAR3FXSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSHEZDCOBZG4>
. You are receiving this because you authored the thread.Message ID: @.*>
Just do cat .nextflow.log from where you ran that cpipes ....... command
and it should print the output. Paste the output here
—
Reply to this email directly, view it on GitHub
<#1 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJWAA4ADF73PNVIM6SMM7MTYJH6CNAVCNFSM6AAAAABAR3FXSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJUGU3TCOJTG4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
dic-13 13:44:17.869 [main] DEBUG nextflow.cli.Launcher - $> nextflow /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/cpipes --pipeline bettercallsal_db --pdg_release PDG000000002.2727 --output /home/chris_franco/bin/bases_de_datos/better_db/
dic-13 13:44:17.940 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 23.10.0
dic-13 13:44:17.961 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/chris_franco/.nextflow/plugins; core-plugins: ***@***.******@***.******@***.******@***.******@***.******@***.******@***.******@***.******@***.***
dic-13 13:44:17.969 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
dic-13 13:44:17.970 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
dic-13 13:44:17.972 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
dic-13 13:44:17.981 [main] INFO org.pf4j.AbstractPluginManager - No plugins
dic-13 13:44:17.995 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/nextflow.config
dic-13 13:44:17.996 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/nextflow.config
dic-13 13:44:18.019 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
dic-13 13:44:19.463 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
dic-13 13:44:19.477 [main] INFO nextflow.cli.CmdRun - Launching `/home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/cpipes` [nasty_mclean] DSL2 - revision: 8da4e11078
dic-13 13:44:19.477 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins ***@***.***
dic-13 13:44:19.478 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
dic-13 13:44:19.478 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved ***@***.***
dic-13 13:44:19.478 [main] DEBUG nextflow.plugin.PluginUpdater - Installing plugin nf-amazon version: 2.1.4
dic-13 13:44:19.483 [main] INFO org.pf4j.AbstractPluginManager - Plugin ***@***.***' resolved
dic-13 13:44:19.484 [main] INFO org.pf4j.AbstractPluginManager - Start plugin ***@***.***'
dic-13 13:44:19.502 [main] DEBUG nextflow.plugin.BasePlugin - Plugin started ***@***.***
dic-13 13:44:19.513 [main] DEBUG nextflow.file.FileHelper - > Added 'S3FileSystemProvider' to list of installed providers [s3]
dic-13 13:44:19.523 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/chris_franco/.nextflow/secrets/store.json
dic-13 13:44:19.525 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: ***@***.*** - activable => ***@***.***
dic-13 13:44:19.575 [main] DEBUG nextflow.Session - Session UUID: 5aad7b25-6a0e-4de7-9060-93b0d471cb81
dic-13 13:44:19.575 [main] DEBUG nextflow.Session - Run name: nasty_mclean
dic-13 13:44:19.575 [main] DEBUG nextflow.Session - Executor pool size: 20
dic-13 13:44:19.581 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
dic-13 13:44:19.584 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=60; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
dic-13 13:44:19.600 [main] DEBUG nextflow.cli.CmdRun -
Version: 23.10.0 build 5889
Created: 15-10-2023 15:07 UTC (10:07 CDT)
System: Linux 6.2.0-37-generic
Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 11.0.13+7-b1751.21
Encoding: UTF-8 (UTF-8)
Process: ***@***.*** [127.0.1.1]
CPUs: 20 - Mem: 15.3 GB (4.9 GB) - Swap: 2 GB (2 GB)
dic-13 13:44:19.616 [main] DEBUG nextflow.Session - Work-dir: /home/chris_franco/bin/bases_de_datos/better_db/work [ext2/ext3]
dic-13 13:44:19.639 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[AwsBatchExecutor]
dic-13 13:44:19.648 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
dic-13 13:44:19.687 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
dic-13 13:44:19.695 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 21; maxThreads: 1000
dic-13 13:44:19.917 [main] DEBUG nextflow.Session - Session start
dic-13 13:44:19.925 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /home/chris_franco/bin/bases_de_datos/better_db/CPIPES-bettercallsal_db/nextflow-reports/bettercallsal_db_exec_trace.txt
dic-13 13:44:19.941 [main] DEBUG nextflow.Session - Using default localLib path: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib
dic-13 13:44:19.946 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib
dic-13 13:44:20.124 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
dic-13 13:44:20.355 [main] INFO nextflow.Nextflow - [0;37m[0;37m================================================================================[0m[0;35m
(o)
___ _ __ _ _ __ ___ ___
/ __|| '_ \ | || '_ \ / _ \/ __|
| (__ | |_) || || |_) || __/\__ \
\___|| .__/ |_|| .__/ \___||___/
| | | |
|_| |_|[0m
[0;37m--------------------------------------------------------------------------------[0m
[0;34mA collection of modular pipelines at CFSAN, FDA.[0m
[0;37m--------------------------------------------------------------------------------[0m
Name : [0;35mbettercallsal[0m
Author : [0;36mKranti Konganti[0m
Version : [0;32m0.6.1[0m
Center : [0;32mCFSAN, FDA.[0m
[0;37m================================================================================[0m
dic-13 13:44:21.376 [main] INFO nextflow.Nextflow - [0;37m--------------------------------------------------------------------------------[0m
Summary of the current workflow ([0;35mbettercallsal_db[0m) parameters
[0;37m--------------------------------------------------------------------------------[0m
workflow_conceived_by : Kranti Konganti
workflow_built_by : Kranti Konganti
workflow_version : 0.6.1
pdg_release : PDG000000002.2727
ncbi_asm_suffix : _genomic.fna.gz
wcomp_serocol : false
wcomp_complete_sero : false
wcomp_not_null_serovar : false
wcomp_num : false
wcomp_min_contig_size : false
wcomp_i : false
wsnp_serocol : false
wsnp_complete_sero : true
wsnp_not_null_serovar : false
wsnp_num : false
wsnp_i : 'I 4,[5],12:i:-'
genomes_chunk : 100
mashsketch_run : true
mashsketch_l : true
mashsketch_I : false
mashsketch_C : false
mashsketch_k : 21
mashsketch_s : 1000
mashsketch_i : false
mashsketch_S : false
mashsketch_w : false
mashsketch_r : false
mashsketch_b : false
mashsketch_m : false
mashsketch_c : false
mashsketch_g : false
mashsketch_n : false
mashsketch_a : false
mashsketch_z : false
mashsketch_Z : false
metadata : null
input : null
output : /home/chris_franco/bin/bases_de_datos/
better_db
fq_suffix : .fastq.gz
fq2_suffix : false
fq_strandedness : unstranded
fq_single_end : true
fq_filter_by_len : 4000
fq_filename_delim : _
fq_filename_delim_idx : 1
enable_conda : false
enable_module : true
max_cpus : 10
[0;37m--------------------------------------------------------------------------------[0m
[0;36mN E X T F L O W[0m - [0;35mCPIPES[0m - Runtime metadata
[0;37m--------------------------------------------------------------------------------[0m
nextflow_version : 23.10.0
nextflow_build : 5889
nextflow_timestamp : 15-10-2023 15:07 UTC
workflow_projectDir : /home/chris_franco/.nextflow/assets/CFSAN-
Biostatistics/bettercallsal
workflow_launchDir : /home/chris_franco/bin/bases_de_datos/
better_db
workflow_workDir : /home/chris_franco/bin/bases_de_datos/
better_db/work
workflow_container : [:]
workflow_containerEngine : null
workflow_runName : nasty_mclean
workflow_sessionId : 5aad7b25-6a0e-4de7-9060-93b0d471cb81
workflow_profile : standard
workflow_start : 2023-12-13T13:44:19.666592-06:00
workflow_commandLine : nextflow /home/chris_franco/.nextflow/
assets/CFSAN-Biostatistics/bettercallsal/
cpipes --pipeline bettercallsal_db --
pdg_release PDG000000002.2727 --output /
home/chris_franco/bin/bases_de_datos/
better_db
[0;37m--------------------------------------------------------------------------------[0m
dic-13 13:44:21.458 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA
dic-13 13:44:21.482 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 13:44:21.483 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 13:44:21.487 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
dic-13 13:44:21.494 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=20; memory=15.3 GB; capacity=20; pollInterval=100ms; dumpInterval=5m
dic-13 13:44:21.497 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
dic-13 13:44:21.655 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_micro` matches labels `process_micro` for process with name BETTERCALLSAL_DB:FILTER_PDG_METADATA
dic-13 13:44:21.658 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 13:44:21.658 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 13:44:21.682 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:DB_PER_COMPUTED_SEROTYPE
dic-13 13:44:21.683 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:DB_PER_COMPUTED_SEROTYPE` matches process BETTERCALLSAL_DB:DB_PER_COMPUTED_SEROTYPE
dic-13 13:44:21.684 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 13:44:21.684 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 13:44:21.698 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:DB_PER_SNP_CLUSTER
dic-13 13:44:21.699 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:DB_PER_SNP_CLUSTER` matches process BETTERCALLSAL_DB:DB_PER_SNP_CLUSTER
dic-13 13:44:21.701 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 13:44:21.701 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 13:44:21.763 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:INDEX_METADATA
dic-13 13:44:21.765 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 13:44:21.765 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 13:44:21.826 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_micro` matches labels `process_micro` for process with name BETTERCALLSAL_DB:SCAFFOLD_GENOMES
dic-13 13:44:21.826 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:SCAFFOLD_GENOMES` matches process BETTERCALLSAL_DB:SCAFFOLD_GENOMES
dic-13 13:44:21.828 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 13:44:21.828 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 13:44:21.885 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_micro` matches labels `process_micro` for process with name BETTERCALLSAL_DB:MASH_SKETCH
dic-13 13:44:21.885 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:MASH_SKETCH` matches process BETTERCALLSAL_DB:MASH_SKETCH
dic-13 13:44:21.887 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 13:44:21.887 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 13:44:21.917 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:DUMP_SOFTWARE_VERSIONS
dic-13 13:44:21.918 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 13:44:21.919 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 13:44:21.926 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: FILTER_PDG_METADATA, DB_PER_SNP_CLUSTER, DOWNLOAD_PDG_METADATA, MASH_SKETCH, SCAFFOLD_GENOMES, BETTERCALLSAL_DB:FILTER_PDG_METADATA, DB_PER_COMPUTED_SEROTYPE, BETTERCALLSAL_DB:DB_PER_SNP_CLUSTER, INDEX_METADATA, DUMP_SOFTWARE_VERSIONS, BETTERCALLSAL_DB:SCAFFOLD_GENOMES, BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA, BETTERCALLSAL_DB:DUMP_SOFTWARE_VERSIONS, BETTERCALLSAL_DB:INDEX_METADATA, BETTERCALLSAL_DB:DB_PER_COMPUTED_SEROTYPE, BETTERCALLSAL_DB:MASH_SKETCH
dic-13 13:44:21.928 [main] DEBUG nextflow.Session - Igniting dataflow network (8)
dic-13 13:44:21.929 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA
dic-13 13:44:21.930 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:FILTER_PDG_METADATA
dic-13 13:44:21.930 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:DB_PER_COMPUTED_SEROTYPE
dic-13 13:44:21.931 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:DB_PER_SNP_CLUSTER
dic-13 13:44:21.931 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:INDEX_METADATA
dic-13 13:44:21.931 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:SCAFFOLD_GENOMES
dic-13 13:44:21.932 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:MASH_SKETCH
dic-13 13:44:21.932 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:DUMP_SOFTWARE_VERSIONS
dic-13 13:44:21.935 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
Script_fff62b623fcd7dc4: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/db_per_snp_cluster/main.nf
Script_eef3004b4d6f1c32: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib/help/wcomp.nf
Script_3ed63f4e78e60ed8: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/workflows/bettercallsal_db.nf
Script_2889a423c5adc0ab: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/scaffold_genomes/main.nf
Script_8c2b4a32404b5a23: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib/help/mashsketch.nf
Script_f9f5befbbaea8ea5: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/download_pdg_metadata/main.nf
Script_d2b4283b9bd2da1a: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/custom/dump_software_versions/main.nf
Script_8c429000b98a8196: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/index_metadata/main.nf
Script_1fce575d0308b7da: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib/routines.nf
Script_f182e2f876b08412: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib/help/wsnp.nf
Script_86e020c2868ffad2: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/cpipes
Script_0a8dd90f87b4dbe9: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/mash/sketch/main.nf
Script_de21d171612ea89c: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/filter_pdg_metadata/main.nf
Script_6c655ad2d57a9fb5: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/db_per_computed_serotype/main.nf
dic-13 13:44:21.935 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination
dic-13 13:44:21.935 [main] DEBUG nextflow.Session - Session await
dic-13 13:44:22.226 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
dic-13 13:44:22.231 [Task submitter] INFO nextflow.Session - [3f/b9f46c] Submitted process > BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)
dic-13 13:44:22.292 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py); status: COMPLETED; exit: 127; error: -; workDir: /home/chris_franco/bin/bases_de_datos/better_db/work/3f/b9f46c73b90af3a61d207275ccb617]
dic-13 13:44:22.299 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
task: name=BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py); work-dir=/home/chris_franco/bin/bases_de_datos/better_db/work/3f/b9f46c73b90af3a61d207275ccb617
error [nextflow.exception.ProcessFailedException]: Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (127)
dic-13 13:44:22.314 [Task monitor] INFO nextflow.processor.TaskProcessor - [3f/b9f46c] NOTE: Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (127) -- Execution is retried (1)
dic-13 13:44:22.331 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
dic-13 13:44:22.333 [Task submitter] INFO nextflow.Session - [90/d8c141] Re-submitted process > BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)
dic-13 13:44:22.357 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py); status: COMPLETED; exit: 127; error: -; workDir: /home/chris_franco/bin/bases_de_datos/better_db/work/90/d8c141fd083a6b982ff8f6602754c6]
dic-13 13:44:22.358 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
task: name=BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py); work-dir=/home/chris_franco/bin/bases_de_datos/better_db/work/90/d8c141fd083a6b982ff8f6602754c6
error [nextflow.exception.ProcessFailedException]: Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (127)
dic-13 13:44:22.374 [Task monitor] DEBUG nextflow.processor.TaskRun - Unable to dump output of process 'BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)' -- Cause: java.nio.file.NoSuchFileException: /home/chris_franco/bin/bases_de_datos/better_db/work/90/d8c141fd083a6b982ff8f6602754c6/.command.out
dic-13 13:44:22.376 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)'
Caused by:
Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (127)
Command executed:
dl_pdg_metadata.py \
-rel PDG000000002.2727
cat <<-END_VERSIONS > versions.yml
"BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA":
python: $( python --version | sed 's/Python //g' )
END_VERSIONS
Command exit status:
127
Command output:
(empty)
Command error:
.command.run: línea 237: module: orden no encontrada
Work dir:
/home/chris_franco/bin/bases_de_datos/better_db/work/90/d8c141fd083a6b982ff8f6602754c6
Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
dic-13 13:44:22.384 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (127)
dic-13 13:44:22.402 [Actor Thread 15] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
dic-13 13:44:22.402 [Actor Thread 16] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
dic-13 13:44:22.402 [Actor Thread 10] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
dic-13 13:44:22.409 [Actor Thread 16] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
dic-13 13:44:22.410 [main] DEBUG nextflow.Session - Session await > all processes finished
dic-13 13:44:22.426 [main] DEBUG nextflow.Session - Session await > all barriers passed
dic-13 13:44:22.432 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
dic-13 13:44:22.441 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=2; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=1; abortedCount=0; succeedDuration=0ms; failedDuration=102ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=4; peakMemory=8 GB; ]
dic-13 13:44:22.441 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
dic-13 13:44:22.445 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
dic-13 13:44:23.114 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
dic-13 13:44:23.196 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
dic-13 13:44:23.197 [main] INFO org.pf4j.AbstractPluginManager - Stop plugin ***@***.***'
dic-13 13:44:23.197 [main] DEBUG nextflow.plugin.BasePlugin - Plugin stopped nf-amazon
dic-13 13:44:23.205 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
biocoder commented
Ok. You need to select one of the profiles. If you are running this on a single machine and not a cluster, add -profile stdkondagac. For slurm cluster, you can use one of the pre-defined profiles, -profile konda.
For your single 1TB machine, you can try:
Ex:
cpipes --pipeline bettercallsal_db --pdg_release PDG000000002.2727 --output /home/chris_franco/bin/bases_de_datos/better_db/ -profile stdkondagac -resumeThe -resume option will resume the workflow from last failed step when you re-execute the same command.
Please note that the bettercallsal_db workflow creates at least 500 to 1000 steps, so it may take a really long time on a single machine. You can download one of the pre-created databases for an older PDG release: https://cfsan-pub-xfer.s3.amazonaws.com/Kranti.Konganti/bettercallsal/PDG000000002.2537.tar.bz2, then do tar -jxvf PDG000000002.2537.tar.bz2.
Thanks,
Kranti
biocoder commented
You need to also make sure, you have micromamba installed. Running the workflow via micromamba software provisioning is preferred as it does not require any sudo or admin privileges or any other configurations with respect to the various container providers.
-
To install
micromambafor your system type, please follow these installation steps and make sure that themicromambabinary is made available in your$PATH. -
Just the
curlstep is sufficient to download the binary as far as running the workflows are concerned. -
Once you have finished the installation, it is important that you downgrade
micromambato version1.0.0.
micromamba self-update --version 1.0.0Chris091089 commented
sorry for so many inconveniences but this is the first time I use the
software, now I get the following errors
El mié, 13 dic 2023 a las 14:51, Kranti Konganti ***@***.***>)
escribió:
Ok. You need to select one of the profiles. If you are running this on a
single machine and not a cluster, add -profile stdkondagac. For slurm
cluster, you can use one of the pre-defined profiles, -profile konda.
For your single 1TB machine, you can try:
Ex:
cpipes --pipeline bettercallsal_db --pdg_release PDG000000002.2727 --output /home/chris_franco/bin/bases_de_datos/better_db/ -profile stdkondagac -resume
The -resume option will resume the workflow from last failed step when
you re-execute the same command.
Please note that the bettercallsal_db workflow creates at least 500 to
1000 steps, so it may take a really long time on a single machine. You can
download one of the pre-created databases for an older PDG release:
https://cfsan-pub-xfer.s3.amazonaws.com/Kranti.Konganti/bettercallsal/PDG000000002.2537.tar.bz2,
then do tar -jxvf PDG000000002.2537.tar.bz2.
Thanks,
Kranti
—
Reply to this email directly, view it on GitHub
<#1 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJWAA4CBNAXSCFC2NADGU4TYJIIMFAVCNFSM6AAAAABAR3FXSOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJUGY4DENBTGU>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
dic-13 15:41:54.093 [main] DEBUG nextflow.cli.Launcher - $> nextflow /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/cpipes --pipeline bettercallsal_db --pdg_release PDG000000002.2727 --output /media/chris_franco/exodia/databases/ -profile stdkondagac -resume
dic-13 15:41:54.170 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 23.10.0
dic-13 15:41:54.196 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/chris_franco/.nextflow/plugins; core-plugins: ***@***.******@***.******@***.******@***.******@***.******@***.******@***.******@***.******@***.***
dic-13 15:41:54.223 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
dic-13 15:41:54.224 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
dic-13 15:41:54.227 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
dic-13 15:41:54.236 [main] INFO org.pf4j.AbstractPluginManager - No plugins
dic-13 15:41:54.248 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/nextflow.config
dic-13 15:41:54.250 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/nextflow.config
dic-13 15:41:54.268 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `stdkondagac`
dic-13 15:41:55.555 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [standard, konda, stdkondagac, raven, cingularity, kondagac, cfsanawsbatch, cingularitygac, eprodkonda, stdcingularitygac, eprodcingularity, eprod]
dic-13 15:41:55.561 [main] WARN nextflow.config.ConfigBuilder - It appears you have never run this project before -- Option `-resume` is ignored
dic-13 15:41:55.580 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
dic-13 15:41:55.594 [main] INFO nextflow.cli.CmdRun - Launching `/home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/cpipes` [pedantic_easley] DSL2 - revision: 8da4e11078
dic-13 15:41:55.595 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins ***@***.***
dic-13 15:41:55.595 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
dic-13 15:41:55.595 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved ***@***.***
dic-13 15:41:55.595 [main] DEBUG nextflow.plugin.PluginUpdater - Installing plugin nf-amazon version: 2.1.4
dic-13 15:41:55.601 [main] INFO org.pf4j.AbstractPluginManager - Plugin ***@***.***' resolved
dic-13 15:41:55.601 [main] INFO org.pf4j.AbstractPluginManager - Start plugin ***@***.***'
dic-13 15:41:55.620 [main] DEBUG nextflow.plugin.BasePlugin - Plugin started ***@***.***
dic-13 15:41:55.631 [main] DEBUG nextflow.file.FileHelper - > Added 'S3FileSystemProvider' to list of installed providers [s3]
dic-13 15:41:55.640 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/chris_franco/.nextflow/secrets/store.json
dic-13 15:41:55.643 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: ***@***.*** - activable => ***@***.***
dic-13 15:41:55.692 [main] DEBUG nextflow.Session - Session UUID: 84a94c72-5c23-4ae5-ae74-2d91c2fd32ef
dic-13 15:41:55.692 [main] DEBUG nextflow.Session - Run name: pedantic_easley
dic-13 15:41:55.692 [main] DEBUG nextflow.Session - Executor pool size: 20
dic-13 15:41:55.698 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
dic-13 15:41:55.701 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=60; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
dic-13 15:41:55.734 [main] DEBUG nextflow.cli.CmdRun -
Version: 23.10.0 build 5889
Created: 15-10-2023 15:07 UTC (10:07 CDT)
System: Linux 6.2.0-37-generic
Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 11.0.13+7-b1751.21
Encoding: UTF-8 (UTF-8)
Process: ***@***.*** [127.0.1.1]
CPUs: 20 - Mem: 15.3 GB (4.3 GB) - Swap: 2 GB (2 GB)
dic-13 15:41:55.753 [main] DEBUG nextflow.Session - Work-dir: /media/chris_franco/exodia/databases/work [UNKNOWN (0x7366746e)]
dic-13 15:41:55.804 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[AwsBatchExecutor]
dic-13 15:41:55.822 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
dic-13 15:41:55.890 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
dic-13 15:41:55.943 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 21; maxThreads: 1000
dic-13 15:41:56.537 [main] DEBUG nextflow.Session - Session start
dic-13 15:41:56.545 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /media/chris_franco/exodia/databases/CPIPES-bettercallsal_db/nextflow-reports/bettercallsal_db_exec_trace.txt
dic-13 15:41:56.561 [main] DEBUG nextflow.Session - Using default localLib path: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib
dic-13 15:41:56.566 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib
dic-13 15:41:56.712 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
dic-13 15:41:56.925 [main] INFO nextflow.Nextflow - [0;37m[0;37m================================================================================[0m[0;35m
(o)
___ _ __ _ _ __ ___ ___
/ __|| '_ \ | || '_ \ / _ \/ __|
| (__ | |_) || || |_) || __/\__ \
\___|| .__/ |_|| .__/ \___||___/
| | | |
|_| |_|[0m
[0;37m--------------------------------------------------------------------------------[0m
[0;34mA collection of modular pipelines at CFSAN, FDA.[0m
[0;37m--------------------------------------------------------------------------------[0m
Name : [0;35mbettercallsal[0m
Author : [0;36mKranti Konganti[0m
Version : [0;32m0.6.1[0m
Center : [0;32mCFSAN, FDA.[0m
[0;37m================================================================================[0m
dic-13 15:41:57.983 [main] INFO nextflow.Nextflow - [0;37m--------------------------------------------------------------------------------[0m
Summary of the current workflow ([0;35mbettercallsal_db[0m) parameters
[0;37m--------------------------------------------------------------------------------[0m
workflow_conceived_by : Kranti Konganti
workflow_built_by : Kranti Konganti
workflow_version : 0.6.1
pdg_release : PDG000000002.2727
ncbi_asm_suffix : _genomic.fna.gz
wcomp_serocol : false
wcomp_complete_sero : false
wcomp_not_null_serovar : false
wcomp_num : false
wcomp_min_contig_size : false
wcomp_i : false
wsnp_serocol : false
wsnp_complete_sero : true
wsnp_not_null_serovar : false
wsnp_num : false
wsnp_i : 'I 4,[5],12:i:-'
genomes_chunk : 100
mashsketch_run : true
mashsketch_l : true
mashsketch_I : false
mashsketch_C : false
mashsketch_k : 21
mashsketch_s : 1000
mashsketch_i : false
mashsketch_S : false
mashsketch_w : false
mashsketch_r : false
mashsketch_b : false
mashsketch_m : false
mashsketch_c : false
mashsketch_g : false
mashsketch_n : false
mashsketch_a : false
mashsketch_z : false
mashsketch_Z : false
metadata : null
input : null
output : /media/chris_franco/exodia/databases/
fq_suffix : .fastq.gz
fq2_suffix : false
fq_strandedness : unstranded
fq_single_end : true
fq_filter_by_len : 4000
fq_filename_delim : _
fq_filename_delim_idx : 1
enable_conda : true
enable_module : false
max_cpus : 10
[0;37m--------------------------------------------------------------------------------[0m
[0;36mN E X T F L O W[0m - [0;35mCPIPES[0m - Runtime metadata
[0;37m--------------------------------------------------------------------------------[0m
nextflow_version : 23.10.0
nextflow_build : 5889
nextflow_timestamp : 15-10-2023 15:07 UTC
workflow_projectDir : /home/chris_franco/.nextflow/assets/CFSAN-
Biostatistics/bettercallsal
workflow_launchDir : /media/chris_franco/exodia/databases
workflow_workDir : /media/chris_franco/exodia/databases/work
workflow_container : [:]
workflow_containerEngine : null
workflow_runName : pedantic_easley
workflow_sessionId : 84a94c72-5c23-4ae5-ae74-2d91c2fd32ef
workflow_profile : stdkondagac
workflow_start : 2023-12-13T15:41:55.857716-06:00
workflow_commandLine : nextflow /home/chris_franco/.nextflow/
assets/CFSAN-Biostatistics/bettercallsal/
cpipes --pipeline bettercallsal_db --
pdg_release PDG000000002.2727 --output /
media/chris_franco/exodia/databases/ -
profile stdkondagac -resume
[0;37m--------------------------------------------------------------------------------[0m
dic-13 15:41:58.025 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA
dic-13 15:41:58.040 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 15:41:58.040 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 15:41:58.043 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
dic-13 15:41:58.047 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=20; memory=15.3 GB; capacity=20; pollInterval=100ms; dumpInterval=5m
dic-13 15:41:58.049 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
dic-13 15:41:58.217 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_micro` matches labels `process_micro` for process with name BETTERCALLSAL_DB:FILTER_PDG_METADATA
dic-13 15:41:58.222 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 15:41:58.222 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 15:41:58.262 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:DB_PER_COMPUTED_SEROTYPE
dic-13 15:41:58.262 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:DB_PER_COMPUTED_SEROTYPE` matches process BETTERCALLSAL_DB:DB_PER_COMPUTED_SEROTYPE
dic-13 15:41:58.264 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 15:41:58.264 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 15:41:58.277 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:DB_PER_SNP_CLUSTER
dic-13 15:41:58.277 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:DB_PER_SNP_CLUSTER` matches process BETTERCALLSAL_DB:DB_PER_SNP_CLUSTER
dic-13 15:41:58.279 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 15:41:58.279 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 15:41:58.322 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:INDEX_METADATA
dic-13 15:41:58.323 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 15:41:58.323 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 15:41:58.364 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_micro` matches labels `process_micro` for process with name BETTERCALLSAL_DB:SCAFFOLD_GENOMES
dic-13 15:41:58.364 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:SCAFFOLD_GENOMES` matches process BETTERCALLSAL_DB:SCAFFOLD_GENOMES
dic-13 15:41:58.365 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 15:41:58.365 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 15:41:58.405 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_micro` matches labels `process_micro` for process with name BETTERCALLSAL_DB:MASH_SKETCH
dic-13 15:41:58.405 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withName:MASH_SKETCH` matches process BETTERCALLSAL_DB:MASH_SKETCH
dic-13 15:41:58.407 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 15:41:58.407 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 15:41:58.431 [main] DEBUG nextflow.script.ProcessConfig - Config settings `withLabel:process_pico` matches labels `process_pico` for process with name BETTERCALLSAL_DB:DUMP_SOFTWARE_VERSIONS
dic-13 15:41:58.433 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: local
dic-13 15:41:58.433 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
dic-13 15:41:58.440 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: FILTER_PDG_METADATA, DOWNLOAD_PDG_METADATA, DB_PER_SNP_CLUSTER, MASH_SKETCH, SCAFFOLD_GENOMES, BETTERCALLSAL_DB:FILTER_PDG_METADATA, DB_PER_COMPUTED_SEROTYPE, BETTERCALLSAL_DB:DB_PER_SNP_CLUSTER, INDEX_METADATA, DUMP_SOFTWARE_VERSIONS, BETTERCALLSAL_DB:SCAFFOLD_GENOMES, BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA, BETTERCALLSAL_DB:DUMP_SOFTWARE_VERSIONS, BETTERCALLSAL_DB:INDEX_METADATA, BETTERCALLSAL_DB:DB_PER_COMPUTED_SEROTYPE, BETTERCALLSAL_DB:MASH_SKETCH
dic-13 15:41:58.443 [main] DEBUG nextflow.Session - Igniting dataflow network (8)
dic-13 15:41:58.443 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA
dic-13 15:41:58.444 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:FILTER_PDG_METADATA
dic-13 15:41:58.444 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:DB_PER_COMPUTED_SEROTYPE
dic-13 15:41:58.444 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:DB_PER_SNP_CLUSTER
dic-13 15:41:58.445 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:INDEX_METADATA
dic-13 15:41:58.445 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:SCAFFOLD_GENOMES
dic-13 15:41:58.446 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:MASH_SKETCH
dic-13 15:41:58.446 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > BETTERCALLSAL_DB:DUMP_SOFTWARE_VERSIONS
dic-13 15:41:58.447 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
Script_fff62b623fcd7dc4: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/db_per_snp_cluster/main.nf
Script_eef3004b4d6f1c32: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib/help/wcomp.nf
Script_3ed63f4e78e60ed8: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/workflows/bettercallsal_db.nf
Script_2889a423c5adc0ab: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/scaffold_genomes/main.nf
Script_8c2b4a32404b5a23: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib/help/mashsketch.nf
Script_f9f5befbbaea8ea5: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/download_pdg_metadata/main.nf
Script_d2b4283b9bd2da1a: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/custom/dump_software_versions/main.nf
Script_8c429000b98a8196: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/index_metadata/main.nf
Script_1fce575d0308b7da: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib/routines.nf
Script_f182e2f876b08412: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/lib/help/wsnp.nf
Script_86e020c2868ffad2: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/cpipes
Script_de21d171612ea89c: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/filter_pdg_metadata/main.nf
Script_0a8dd90f87b4dbe9: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/mash/sketch/main.nf
Script_6c655ad2d57a9fb5: /home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/modules/db_per_computed_serotype/main.nf
dic-13 15:41:58.447 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination
dic-13 15:41:58.448 [main] DEBUG nextflow.Session - Session await
dic-13 15:41:58.564 [Actor Thread 20] DEBUG nextflow.conda.CondaCache - micromamba found local env for environment=conda-forge::python=3.10.4; path=/home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/kondagac_cache/env-0b2f8bc5dcb055014216a0e0455a1d48
dic-13 15:41:58.731 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
dic-13 15:41:58.734 [Task submitter] INFO nextflow.Session - [5d/7644fe] Submitted process > BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)
dic-13 15:42:02.192 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py); status: COMPLETED; exit: 1; error: -; workDir: /media/chris_franco/exodia/databases/work/5d/7644fede4c59f4d5e001440fb844d4]
dic-13 15:42:02.199 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
task: name=BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py); work-dir=/media/chris_franco/exodia/databases/work/5d/7644fede4c59f4d5e001440fb844d4
error [nextflow.exception.ProcessFailedException]: Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (1)
dic-13 15:42:02.214 [Task monitor] INFO nextflow.processor.TaskProcessor - [5d/7644fe] NOTE: Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (1) -- Execution is retried (1)
dic-13 15:42:02.231 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
dic-13 15:42:02.232 [Task submitter] INFO nextflow.Session - [38/a6034c] Re-submitted process > BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)
dic-13 15:42:03.416 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py); status: COMPLETED; exit: 1; error: -; workDir: /media/chris_franco/exodia/databases/work/38/a6034cf9b182a86f34603f68991bee]
dic-13 15:42:03.416 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
task: name=BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py); work-dir=/media/chris_franco/exodia/databases/work/38/a6034cf9b182a86f34603f68991bee
error [nextflow.exception.ProcessFailedException]: Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (1)
dic-13 15:42:03.436 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)'
Caused by:
Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (1)
Command executed:
dl_pdg_metadata.py \
-rel PDG000000002.2727
cat <<-END_VERSIONS > versions.yml
"BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA":
python: $( python --version | sed 's/Python //g' )
END_VERSIONS
Command exit status:
1
Command output:
(empty)
Command error:
self._send_request(method, url, body, headers, encode_chunked)
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/http/client.py", line 1332, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/http/client.py", line 1281, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/http/client.py", line 1041, in _send_output
self.send(msg)
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/http/client.py", line 979, in send
self.connect()
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/http/client.py", line 1458, in connect
self.sock = self._context.wrap_socket(self.sock,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/ssl.py", line 1108, in _create
self.do_handshake()
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/ssl.py", line 1379, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1006)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/bin/dl_pdg_metadata.py", line 206, in <module>
main()
File "/home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/bin/dl_pdg_metadata.py", line 170, in main
file, dest_dir = dl_pdg(
^^^^^^^
File "/home/chris_franco/.nextflow/assets/CFSAN-Biostatistics/bettercallsal/bin/dl_pdg_metadata.py", line 54, in dl_pdg
with urlopen(url) as response:
^^^^^^^^^^^^
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/urllib/request.py", line 519, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/urllib/request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/urllib/request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/chris_franco/bin/programas_bioinformaticos/anaconda3/lib/python3.11/urllib/request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1006)>
Work dir:
/media/chris_franco/exodia/databases/work/38/a6034cf9b182a86f34603f68991bee
Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
dic-13 15:42:03.445 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `BETTERCALLSAL_DB:DOWNLOAD_PDG_METADATA (dl_pdg_metadata.py)` terminated with an error exit status (1)
dic-13 15:42:03.462 [Actor Thread 9] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
dic-13 15:42:03.462 [Actor Thread 17] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
dic-13 15:42:03.463 [Actor Thread 14] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
dic-13 15:42:03.469 [Actor Thread 9] DEBUG nextflow.file.SortFileCollector - FileCollector temp dir not removed: null
dic-13 15:42:03.470 [main] DEBUG nextflow.Session - Session await > all processes finished
dic-13 15:42:03.471 [Task monitor] DEBUG nextflow.Session - The following nodes are still active:
[process] BETTERCALLSAL_DB:MASH_SKETCH
[process] BETTERCALLSAL_DB:DUMP_SOFTWARE_VERSIONS
dic-13 15:42:03.474 [main] DEBUG nextflow.Session - Session await > all barriers passed
dic-13 15:42:03.480 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
dic-13 15:42:03.489 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=2; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=1; abortedCount=0; succeedDuration=0ms; failedDuration=11.5s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=4; peakMemory=8 GB; ]
dic-13 15:42:03.489 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
dic-13 15:42:03.492 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
dic-13 15:42:04.141 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
dic-13 15:42:04.226 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
dic-13 15:42:04.227 [main] INFO org.pf4j.AbstractPluginManager - Stop plugin ***@***.***'
dic-13 15:42:04.227 [main] DEBUG nextflow.plugin.BasePlugin - Plugin stopped nf-amazon
dic-13 15:42:04.245 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
biocoder commented
Ok, delete the dl_pdg_metadata.py inside the bin folder and replace it with the script attached in this comment. Try again and let me know if it worked. If it works, then, I am going to patch the release. Also, please note that PDG000000002.2727 does not exist at NCBI anymore.
Try using --pdg_release latest_snps when you call the cpipes .......... command.
Note: Please make sure you rename the extension of the script from .txt to .py after you place it inside the bin folder.