Allow to skip publishDir

[Lehmann-Fabian]

If you have a big project such as nf-core it happens that publishDir directives are spread over different config files that you don't wan't to touch. However, a publishDir has to be disabled in the place of creation.
process.publishDir.enable = false is not applied to existing publishDirs.
Therefore, I implemented this small feature.

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	fddc3ef
🔍 Latest deploy log	https://app.netlify.com/sites/nextflow-docs-staging/deploys/64c8b8d053c6f50008f4d323

[pditommaso]

Not sure to understand what's the problem you are trying to solve

[bentsherman]

He wants a global switch to toggle all publishing.

However you could accomplish this with a param like skip_publish that is checked by every publishDir in your pipeline.

[Lehmann-Fabian]

Partially Ben is right.
Let me provide a concrete example.
In nf-core, you always have a big config file with all publishDirs, such as here: modules.config.
Of course, you can introduce a parameter for all publishDirs you want to turn off. But therefore, you have to touch the module.config file.
However, running the official nf-core project directly from Github does not work as the config is pulled as it is.
The idea for all other parameters is to define your config file, overwriting the default configuration. For publishDirs, you can add new ones, but modifying them in a new config file is impossible.

My idea is now that you have your config and you can say something like:

withName: 'GFFREAD' {
        skipPublishDir = true
}

And, of course, you can also globally turn off publishDirs and enable it only for selected processes.

[pditommaso]

I don't think it should be introduced a specific directive for this. I'd be more open to discussing a mechanism to override the settings of publish dir attributes.

For example, having something like

nextflow {
  defaults {
    publishDir = [mode: 'copy', enabled: false]
  }
}

This would set publish dir copy mode and disable it by default.

Would this solve your use case?

[Lehmann-Fabian]

Thanks for getting back to me.
This would be exactly the solution I am looking for.

[pditommaso]

Closing in favour of #4186