No process matching config selector warnings

Problem

You are using a process twice in a pipeline in two different subworkflows. Additionally, both are optional to run, so you wrap them in if statements.

if ( params.run_module ) {
    PROCESS ()
}

You may need to provide each of them different publishDir options so to disambiguate between the two in a configuration file, you differentiate them using a withName selector and give them the fully resolved path name PIPELINE:WORKFLOW:SUBWORKFLOW:PROCESS.

conf/modules.config

process {
    withName: 'PIPELINE:WORKFLOW:SUBWORKFLOW_ONE:PROCESS' {
         publishDir = [
            path: { "${params.outdir}/PROCESS_ONE" },
        ]
    }
    withName: 'PIPELINE:WORKFLOW:SUBWORKFLOW_TWO:PROCESS' {
         publishDir = [
            path: { "${params.outdir}/PROCESS_TWO" },
        ]
    }
}

This should be sufficient to distinguish between the two, right?

But then when you run your pipeline, with the if statement around the module set to false

nextflow run main.nf --run_module false

You suddenly get the following warnings:

WARN: There's no process matching config selector: <PIPELINE>:<WORKFLOW>:<SUBWORKFLOW_ONE>:<PROCESS>
WARN: There's no process matching config selector: <PIPELINE>:<WORKFLOW>:<SUBWORKFLOW_TWO>:<PROCESS>

...huh?

You can see the problem for yourself in an actual pipeline by using GitPod or getting a copy of the project and navigating to examples/process-selector-warn. Where you execute the following command:

nextflow -c conf/problem.config run main.nf --skip_hello

Solution

Apparently, the different ways of specifying a module using the withName selector have different behaviours.

• Only an explicit module name can cope with 'optional' execution and have a selector still picked up, even if it's 'turned off'.
• Fully resolved paths or wildcarded withName selectors cannot be evaluated by Nextflow in this manner, and thus give you warnings like above.

Here, your solutions are:

1. Give each of the two processes a unique name via an alias and use that in the configuration file.

   subworkflows/subworkflow_one.nf

   include { PROCESS as PROCESS_ONE } from 'modules/process_module'
   

   subworkflows/subworkflow_two.nf

   include { PROCESS as PROCESS_TWO } from 'modules/process_module'
   

   conf/modules.config

   process {
       withName: 'PROCESS_ONE' {
           publishDir = [
               path: { "${params.outdir}/PROCESS_ONE" },
           ]
       }
       withName: 'PROCESS_TWO' {
           publishDir = [
               path: { "${params.outdir}/PROCESS_TWO" },
           ]
       }
   }
   

2. Make the withName selector for the module also conditional in the configuration file:

   conf/modules.config

   process {
       if ( param.run_module ) {
           withName: 'PIPELINE:WORKFLOW:SUBWORKFLOW_ONE:PROCESS' {
               publishDir = [
                   path: { "${params.outdir}/PROCESS_ONE" },
               ]
           }
           withName: 'PIPELINE:WORKFLOW:SUBWORKFLOW_TWO:PROCESS' {
               publishDir = [
                   path: { "${params.outdir}/PROCESS_TWO" },
               ]
           }
       }
   }
   

You can see the solution for yourself in an actual pipeline by using GitPod or getting a copy of the project and navigating to examples/process-selector-warn. Where you execute the following command:

nextflow -c conf/solution.config run main.nf --skip_hello