[Bug]: Not sure how to change number of threads executor will use.

[callumparr]

What happened?

A bug happened!

using the threads flag in the command seems not to override the nextflow config and I am unsure where I should change in order to allow more threads to be used.

 nextflow run epi2me-labs/wf-pore-c --ubam /work/callum/apac-seq/wf-pore-c/test_data/bams/barcode01/porec_test.concatemers.bam --ref ${genome} --cutter NlaIII --mcool_resolutions 1000,2000,3000 --pairs --chromunity --paired_end --filter_pairs \
--paired_end_minimum_distance 100 --paired_end_maximum_distance 200 --chromunity --chromunity_merge_distance 5 --mcool --out_dir pore-c_control_barcode01 --coverage -profile singularity --threads 12

I have tried modifying the config file as suggested

limited by config executor cpus

But it does not seem to have an effect and gives the same output.

Operating System

ubuntu 20.04

Workflow Execution

Command line

Workflow Execution - EPI2ME Labs Versions

No response

Workflow Execution - CLI Execution Profile

Singularity

Workflow Version

wf-pore-c v0.0.8-gba50dfd

Relevant log output

WARN: Access to undefined parameter `sample` -- Initialise it to a default value eg. `params.sample = some_value`
Checking ubam input.
[-        ] process > POREC:bamstats                     -
[-        ] process > POREC:index_ubam                   -
[-        ] process > POREC:bamstats                     [  0%] 0 of 1
[-        ] process > POREC:index_ubam                   -
[-        ] process > POREC:bamstats                         [  0%] 0 of 1
[-        ] process > POREC:index_ubam                       -
[71/b584fd] process > POREC:prepare_genome:index_ref_mmi (1) [100%] 1 of 1, failed: 1
[-        ] process > POREC:digest_align_annotate            -
[-        ] process > POREC:haplotag_alignments              -
[-        ] process > POREC:merge_coordsorted_bams           -
[-        ] process > POREC:merge_namesorted_bams            -
[-        ] process > POREC:create_restriction_bed           -
[-        ] process > POREC:mosdepth_coverage                -
[-        ] process > POREC:to_pairs_file                    -
[-        ] process > POREC:pairsToCooler                    -
[-        ] process > POREC:merge_mcools                     -
[-        ] process > POREC:merge_pairs                      -
[-        ] process > POREC:merge_pairs_stats                -
[-        ] process > POREC:pair_stats_report                -
[-        ] process > POREC:merge_parquets_to_dataset        -
[-        ] process > POREC:merge_paired_end_bams            -
[-        ] process > POREC:getVersions                      [  0%] 0 of 1
[-        ] process > POREC:getParams                        [  0%] 0 of 1
[-        ] process > POREC:makeReport                       -
Pulling Singularity image docker://ontresearch/wf-pore-c:shaf398a84ba0723c0e612e43926145008535c345c9 [cache /work/callum/apac-seq/wf-pore-c/work/singularity/ontresearch-wf-pore-c-shaf398a84ba0723c0e612e43926145008535c345c9.img]
WARN: Singularity cache directory has not been defined -- Remote image will be stored in the path: /work/callum/apac-seq/wf-pore-c/work/singularity -- Use the environment variable NXF_SINGULARITY_CACHEDIR to specify a different location
ERROR ~ Error executing process > 'POREC:prepare_genome:index_ref_mmi (1)'

Caused by:
  Process requirement exceeds available CPUs -- req: 24; avail: 4

Command executed:

  minimap2 -x map-ont -d "reference.fasta.mmi" "reference.fasta"

Command exit status:
  -

Command output:
  (empty)

Work dir:
  /work/callum/apac-seq/wf-pore-c/work/71/b584fd91877c2fe7e85ec1989bb111

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details

[cjw85]

Hi Callum,

You need to provide a configuration file to nextflow with the content:

executor {
    $local {
        cpus = 4
        memory = "8 GB"
    }
}

changing the 4 and 8 as appropriate for your machine. You can give this additional config file to nextflow using its -C option.

[callumparr]

Hi Callum,

You need to provide a configuration file to nextflow with the content:

executor {
    $local {
        cpus = 4
        memory = "8 GB"
    }
}

changing the 4 and 8 as appropriate for your machine. You can give this additional config file to nextflow using its -C option.

Great thank you that works! So it should be a separate file and not changing the nextflow.config that comes with wf-pore-c?

[cjw85]

There is never any need to edit the included file. See https://www.nextflow.io/docs/latest/config.html