uclahs-cds/pipeline-call-sSNV

Use `methods.setup_process_afterscript()` for process logs

nwiltsie opened this issue · 0 comments

We should incorporate the new methods.setup_process_afterscript() from uclahs-cds/pipeline-Nextflow-config#64 into this pipeline.

Under the new method, log files are automatically saved to ${params.log_output_dir}/process-log/${task.ext.log_dir}${task.ext.log_dir_suffix}/ (no need to specify a publishDir directive).

${task.ext.log_dir} defaults to ${task.process.replace(':', '/')}.
${task.ext.log_dir_suffix} defaults to a blank string.

The key steps are:

  1. Add methods.setup_process_afterscript() to methods.setup().
  2. For each Nextflow process:
    1. Remove the file ".command.*" output directive.
    2. Add ext log_dir { ... } and ext log_sur_suffix { ... } directives, if appropriate.
    3. Remove the publishDir directive with pattern: ".command.*".

This pipeline does some atypical configuration with params.workflow_log_output_dir, but that can still be handled with ext log_dir:

Current

include { somaticsniper } from './module/somaticsniper' addParams(
workflow_output_dir: "${params.output_dir_base}/SomaticSniper-${params.somaticsniper_version}",
workflow_log_output_dir: "${params.log_output_dir}/process-log/SomaticSniper-${params.somaticsniper_version}",

publishDir path: "${params.workflow_log_output_dir}",
mode: "copy",
pattern: ".command.*",
saveAs: { "${task.process.split(':')[-1]}/log${file(it).getName()}" }

Updated

-    publishDir path: "${params.workflow_log_output_dir}",
-               mode: "copy",
-               pattern: ".command.*",
-               saveAs: { "${task.process.split(':')[-1]}/log${file(it).getName()}" }
+    ext log_dir { "SomaticSniper-${params.somaticsniper_version}/${task.process.split(':')[-1]}" }

See https://github.com/uclahs-cds/pipeline-recalibrate-BAM/pull/76/files for a full example.