Skip to content

Variable scope

Open in Gitpod

Under most circumstances it is recommended to use local variable scope. In Groovy and thus nextflow, you do this with the def keyword. However, there are some situations where this can be awkward or even surprising.

Conditionals

In the following workflow, the variable message is declared local to the if statement and cannot be accessed outside it. 😕

if.nf
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
#!/usr/bin/env nextflow

nextflow.enable.dsl = 2

/*******************************************************************************
 * Define main workflow
 ******************************************************************************/

workflow {
    if (true) {
        def message = 'Hello... noone?'
    }
    def ch = Channel.of(message)
}

NXF_VER='21.10.6' nextflow run examples/variable-scope/if.nf

No such variable: message

 -- Check script 'examples/variable-scope/problem.nf' at line: 13 or see '.nextflow.log' file for more details

Process blocks

Problem

In my mental model, a nextflow process is very much a logical unit. However, a process consists of up to five blocks and variables local to one block, cannot be used in another. As an example:

process-problem.nf
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#!/usr/bin/env nextflow

nextflow.enable.dsl = 2

/*******************************************************************************
 * Define processes
 ******************************************************************************/

process TOUCH {
    input:
    val(meta)

    output:
    tuple val(meta), path(result)


    script:
    def result = "${meta.id}.txt"
    """
    touch '${result}'
    """
}

/*******************************************************************************
 * Define main workflow
 ******************************************************************************/

workflow {
    TOUCH([id: 'snafu'])
}

NXF_VER='21.10.6' nextflow run examples/variable-scope/process-problem.nf

executor >  local (1)
[28/98c503] process > TOUCH [100%] 1 of 1, failed: 1 ✘
Error executing process > 'TOUCH'

Caused by:
  Missing output file(s) `result` expected by process `TOUCH`

Command executed:

  touch 'snafu.txt'

Command exit status:
  0

Command output:
  (empty)

Solution

It is an error to use the result variable which was declared local to the script block, in any other block. One may want to use a variable defined in the script block in the output block. In that case, you have to remove the def keyword to make it global.

process-solution.nf
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#!/usr/bin/env nextflow

nextflow.enable.dsl = 2

/*******************************************************************************
 * Define processes
 ******************************************************************************/

process TOUCH {
    input:
    val(meta)

    output:
    tuple val(meta), path(result)


    script:
    result = "${meta.id}.txt"
    """
    touch '${result}'
    """
}

/*******************************************************************************
 * Define main workflow
 ******************************************************************************/

workflow {
    TOUCH([id: 'snafu'])
}

NXF_VER='21.10.6' nextflow run examples/variable-scope/process-solution.nf

executor >  local (1)
[f4/263762] process > TOUCH [100%] 1 of 1 ✔

Mixing variables

Additionally, if you use a variable with global scope in the assignment of a variable with local scope, this is also an error.

mixing-problem.nf
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#!/usr/bin/env nextflow

nextflow.enable.dsl = 2

/*******************************************************************************
 * Define processes
 ******************************************************************************/

process ECHO {
    input:
    val(meta)

    output:
    tuple val(meta), path(result)


    script:
    result = "${meta.id}.txt"
    def choice = result.startsWith('snafu') ? 'yes' : 'no'
    """
    echo '${choice}' > '${result}'
    """
}

/*******************************************************************************
 * Define main workflow
 ******************************************************************************/

workflow {
    ECHO([id: 'snafu'])
}

NXF_VER='21.10.6' nextflow run examples/variable-scope/mixing-problem.nf

Script compilation error
- cause: Variable `result` already defined in the process scope @ line 19, column 18.
       def choice = result.startsWith('snafu') ? 'yes' : 'no'
                    ^

That means, any variable that uses a global in its assignment, has to be global also.

mixing-solution.nf
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#!/usr/bin/env nextflow

nextflow.enable.dsl = 2

/*******************************************************************************
 * Define processes
 ******************************************************************************/

process ECHO {
    input:
    val(meta)

    output:
    tuple val(meta), path(result)


    script:
    result = "${meta.id}.txt"
    choice = result.startsWith('snafu') ? 'yes' : 'no'
    """
    echo '${choice}' > '${result}'
    """
}

/*******************************************************************************
 * Define main workflow
 ******************************************************************************/

workflow {
    ECHO([id: 'snafu'])
}

NXF_VER='21.10.6' nextflow run examples/variable-scope/mixing-solution.nf

executor >  local (1)
[c2/70f706] process > ECHO [100%] 1 of 1 ✔

Hope this helps, it can be quite baffling.