Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add wittyer as module #5171

Merged
merged 41 commits into from
Mar 20, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
ff7fea5
feat: add wittyer as module
famosab Mar 18, 2024
03da21d
fix: adjust output and version
famosab Mar 18, 2024
699666c
fix: copy output and script from local
famosab Mar 18, 2024
1efcfe3
feat: fill in in meta file
famosab Mar 18, 2024
9b5c0c7
fix: remove config file input
famosab Mar 18, 2024
b79ee74
fix: run prettier
famosab Mar 18, 2024
cbb2e0b
feat: explain how to create docker image
famosab Mar 18, 2024
c974cbd
fix: README for Docker image
famosab Mar 19, 2024
e0f5f69
fix: modify environment file
famosab Mar 19, 2024
213d6fa
fix: stub and file extensions
famosab Mar 19, 2024
80e352e
fix: remove custom extensions
famosab Mar 19, 2024
7babe7a
feat: add test file - failing
famosab Mar 19, 2024
e702a20
fix: correct input for tests
famosab Mar 19, 2024
522f72e
fix: licence
famosab Mar 19, 2024
7d06402
fix: container name
famosab Mar 19, 2024
fdf993d
fix: finally correct container
famosab Mar 19, 2024
c07bd11
fix syntax errors
famosab Mar 19, 2024
d6e428b
normal test works, stub fails
famosab Mar 19, 2024
3f7e016
fix expected outputs
famosab Mar 19, 2024
bf138d6
fix: mention making image public
famosab Mar 19, 2024
d205ddd
fix add stageAs, use dotnet
famosab Mar 19, 2024
e416458
run prettier
famosab Mar 19, 2024
a6564b6
[automated] Fix linting with Prettier
nf-core-bot Mar 19, 2024
a205762
add comment for run
famosab Mar 19, 2024
ae11718
fix trailing whitespace
famosab Mar 19, 2024
32081bc
add dependencies to fix linting
famosab Mar 19, 2024
1e55171
fix harmonize inputs in meta and main
famosab Mar 19, 2024
1eb7526
fix: run prettier
famosab Mar 19, 2024
e73bde5
Merge branch 'master' into wittyer
famosab Mar 19, 2024
fe25500
[automated] Fix linting with Prettier
nf-core-bot Mar 19, 2024
1903c8d
fix index names
famosab Mar 19, 2024
fb14a9c
Merge branch 'wittyer' of https://github.com/famosab/modules into wit…
famosab Mar 19, 2024
b2a8c10
fix version to stub_version in stub test
famosab Mar 19, 2024
9f908f8
extend tests to capture as much as possible
famosab Mar 19, 2024
4528b41
Merge branch 'master' into wittyer
famosab Mar 19, 2024
fd4c307
test with bed file
famosab Mar 19, 2024
6aac9e8
fix: remove stageAs
famosab Mar 20, 2024
67fa022
fix tests for stageAs removal
famosab Mar 20, 2024
e46dbd3
remove '' in meta file
famosab Mar 20, 2024
682c539
exclude conda test for wittyer
famosab Mar 20, 2024
79aac08
Merge branch 'master' into wittyer
famosab Mar 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions modules/nf-core/wittyer/environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json
name: "wittyer"
channels:
- conda-forge
- bioconda
- defaults
famosab marked this conversation as resolved.
Show resolved Hide resolved
dependencies:
- "YOUR-TOOL-HERE"
94 changes: 94 additions & 0 deletions modules/nf-core/wittyer/main.nf
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
// TODO nf-core: If in doubt look at other nf-core/modules to see how we are doing things! :)
// https://github.com/nf-core/modules/tree/master/modules/nf-core/
// You can also ask for help via your pull request or on the #modules channel on the nf-core Slack workspace:
// https://nf-co.re/join
// TODO nf-core: A module file SHOULD only define input and output files as command-line parameters.
// All other parameters MUST be provided using the "task.ext" directive, see here:
// https://www.nextflow.io/docs/latest/process.html#ext
// where "task.ext" is a string.
// Any parameters that need to be evaluated in the context of a particular sample
// e.g. single-end/paired-end data MUST also be defined and evaluated appropriately.
// TODO nf-core: Software that can be piped together SHOULD be added to separate module files
// unless there is a run-time, storage advantage in implementing in this way
// e.g. it's ok to have a single module for bwa to output BAM instead of SAM:
// bwa mem | samtools view -B -T ref.fasta
// TODO nf-core: Optional inputs are not currently supported by Nextflow. However, using an empty
// list (`[]`) instead of a file can be used to work around this issue.

process WITTYER {
tag "$meta.id"
label 'process_single'

// TODO nf-core: List required Conda package(s).
// Software MUST be pinned to channel (i.e. "bioconda"), version (i.e. "1.10").
// For Conda, the build (i.e. "h9402c20_2") must be EXCLUDED to support installation on different operating systems.
// TODO nf-core: See section in main README for further information regarding finding and adding container addresses to the section below.
// conda "${moduleDir}/environment.yml"
container "nf-core/wittier:0.3.3"
// "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
// 'https://depot.galaxyproject.org/singularity/YOUR-TOOL-HERE':
// 'biocontainers/YOUR-TOOL-HERE' }"

input:
// TODO nf-core: Where applicable all sample-specific information e.g. "id", "single_end", "read_group"
// MUST be provided as an input via a Groovy Map called "meta".
// This information may not be required in some instances e.g. indexing reference genome files:
// https://github.com/nf-core/modules/blob/master/modules/nf-core/bwa/index/main.nf
// TODO nf-core: Where applicable please provide/convert compressed files as input/output
// e.g. "*.fastq.gz" and NOT "*.fastq", "*.bam" and NOT "*.sam" etc.
tuple val(meta), path(query_vcf), path(query_vcf_tbi), path(truth_vcf), path(truth_vcf_tbi), path(bed)

output:
tuple val(meta), path("*.Stats.json") , emit: report
tuple val(meta), path("*eval.vcf.gz") , emit: bench_vcf
tuple val(meta), path("*eval.vcf.gz.tbi") , emit: bench_vcf_tbi
path "versions.yml"

when:
task.ext.when == null || task.ext.when

script:
// Exit if running this module with -profile conda / -profile mamba
if (workflow.profile.tokenize(',').intersect(['conda', 'mamba']).size() >= 1) {
error "WHITTYER module does not support Conda. Please use Docker / Singularity / Podman instead."
}
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
def regions = bed ? "--includeBed=$bed" : ""
"""
mkdir bench
wittyer \\
famosab marked this conversation as resolved.
Show resolved Hide resolved
--truthVcf=${truth_vcf} \\
--inputVcf=${query_vcf} \\
--outputDirectory=bench \\
${regions} \\
${args}

mv bench/Wittyer.Stats.json ${prefix}.Stats.json
mv bench/*.vcf.gz ${prefix}.eval.vcf.gz
mv bench/*.vcf.gz.tbi ${prefix}.eval.vcf.gz.tbi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove any custom extensions that are not file extensions? This should be done to make all filenames as customizable as possible :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think eval was added to show that this vcf is the result of the benchmarking comparison. Should that be captured via prefix then?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that should be added via the prefix. It could also be a good idea to add a simple check to see if the output filename isn't the same as the input files (otherwise Nextflow will give some weird errors) => See https://nf-co.re/docs/contributing/modules#naming-conventions part 6 for an example

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but the name is generated through wittyer, I think it makes sense to keep original extensions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eval is already exist in original output, only prefix is being added here to the name. I would prefer it in this way actually.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it now and added the if statements to make sure we don't run into Nextflow Errors.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I was not adding the extension by myself :D it is the standard output by the tool.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This type of names are important for traceability with standard outputs of the tool. So I would really prefer to keep is as it is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the eval is only added when you move the file from the output directory?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, I am copying the result here. eval is in standard output.


rm -rf bench

cat <<-END_VERSIONS > versions.yml
"${task.process}":
wittyer: \$(wittyer --version |& sed '1!d ; s/witty.er //')
END_VERSIONS
"""

stub:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
// TODO nf-core: A stub section should mimic the execution of the original module as best as possible
// Have a look at the following examples:
// Simple example: https://github.com/nf-core/modules/blob/818474a292b4860ae8ff88e149fbcda68814114d/modules/nf-core/bcftools/annotate/main.nf#L47-L63
// Complex example: https://github.com/nf-core/modules/blob/818474a292b4860ae8ff88e149fbcda68814114d/modules/nf-core/bedtools/split/main.nf#L38-L54
"""
touch ${prefix}.bam
famosab marked this conversation as resolved.
Show resolved Hide resolved

cat <<-END_VERSIONS > versions.yml
"${task.process}":
wittyer: \$(wittyer --version |& sed '1!d ; s/witty.er //')
END_VERSIONS
"""
}
66 changes: 66 additions & 0 deletions modules/nf-core/wittyer/meta.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/meta-schema.json
name: "wittyer"
description: A large variant benchmarking tool analogous to hap.py for small variants.
keywords:
- structural-variants
- benchmarking
- vcf
tools:
- "wittyer":
description: "Illumina tool for large variant benchmarking"
homepage: "https://github.com/Illumina/witty.er"
documentation: "https://github.com/Illumina/witty.er"
tool_dev_url: "https://github.com/Illumina/witty.er"
licence: "['BSD-2']"
famosab marked this conversation as resolved.
Show resolved Hide resolved
input:
- meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1', single_end:false ]`
- query_vcf:
type: file
description: A VCF with called variants to benchmark against the standard
pattern: "*.{vcf,vcf.gz}"
- query_vcf_index:
type: file
description: The index of the called VCF (optional)
pattern: "*.tbi"
- truth_vcf:
type: file
description: A standard VCF to compare against
pattern: "*.{vcf,vcf.gz}"
- truth_vcf_index:
type: file
description: The index of the standard VCF (optional)
pattern: "*.tbi"
- bed:
type: file
description: A BED file specifying regions to be included in the analysis (optional)
pattern: "*.bed"
output:
- meta:
type: map
description: |
Groovy Map containing sample information
e.g. `[ id:'sample1', single_end:false ]`
- versions:
type: file
description: File containing software versions
pattern: "versions.yml"
- report:
type: file
description: Detailed per-sample-pair, per-svtype, per-bin stats
pattern: "*.Stats.json"
- bench_vcf:
type: file
description: Updated query and truth entries merged into one file
pattern: "*eval.vcf.gz"
- bench_vcf_tbi:
type: file
description: Index of merged query and truth entries VCF file
pattern: "*eval.vcf.gz.tbi"
authors:
- "@famosab"
maintainers:
- "@famosab"
73 changes: 73 additions & 0 deletions modules/nf-core/wittyer/tests/main.nf.test
famosab marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
// TODO nf-core: Once you have added the required tests, please run the following command to build this file:
// nf-core modules test wittyer
nextflow_process {

name "Test Process WITTYER"
script "../main.nf"
process "WITTYER"

tag "modules"
tag "modules_nfcore"
tag "wittyer"

// TODO nf-core: Change the test name preferably indicating the test-data and file-format used
test("sarscov2 - bam") {

// TODO nf-core: If you are created a test for a chained module
// (the module requires running more than one process to generate the required output)
// add the 'setup' method here.
// You can find more information about how to use a 'setup' method in the docs (https://nf-co.re/docs/contributing/modules#steps-for-creating-nf-test-for-chained-modules).

when {
process {
"""
// TODO nf-core: define inputs of the process here. Example:

input[0] = [
[ id:'test', single_end:false ], // meta map
file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true)
]
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out).match() }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need more assertions here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, per output

//TODO nf-core: Add all required assertions to verify the test output.
// See https://nf-co.re/docs/contributing/tutorials/nf-test_assertions for more information and examples.
)
}

}

// TODO nf-core: Change the test name preferably indicating the test-data and file-format used but keep the " - stub" suffix.
test("sarscov2 - bam - stub") {

options "-stub"

when {
process {
"""
// TODO nf-core: define inputs of the process here. Example:

input[0] = [
[ id:'test', single_end:false ], // meta map
file(params.test_data['sarscov2']['illumina']['test_paired_end_bam'], checkIfExists: true)
]
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out).match() }
//TODO nf-core: Add all required assertions to verify the test output.
)
}

}

}
2 changes: 2 additions & 0 deletions modules/nf-core/wittyer/tests/tags.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
wittyer:
- "modules/nf-core/wittyer/**"
Loading