Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bump fgbio versions and swap to nftest #5624

Merged
merged 18 commits into from
Jul 2, 2024
Merged

Bump fgbio versions and swap to nftest #5624

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
d6cc2b3
Convert callduplex and callmolecular to nftest and update version
SPPearce May 17, 2024
01bd4c3
Convert zipperbams
SPPearce May 21, 2024
3fc5a08
Convert groupreadsbyumi
SPPearce May 21, 2024
5ec0a95
Convert sortbam and add stubs
SPPearce May 21, 2024
3d38cc5
Merge branch 'master' into fgbio_nftest
SPPearce May 21, 2024
0bfe648
Convert filterconsensusreads
SPPearce May 21, 2024
95ed61f
Update snapshot
SPPearce May 21, 2024
b1e1beb
Merge branch 'master' into fgbio_nftest
SPPearce May 21, 2024
7b18f7b
Merge branch 'master' into fgbio_nftest
SPPearce May 21, 2024
c3b5a1a
Update modules/nf-core/fgbio/callmolecularconsensusreads/main.nf
SPPearce May 21, 2024
49922ba
Add collision checks, swap test data paths
SPPearce May 21, 2024
abb2311
Merge branch 'master' into fgbio_nftest
SPPearce May 21, 2024
8d1dff3
Fix conda version and linting
SPPearce May 21, 2024
559fb11
Apply suggestions from code review
SPPearce May 22, 2024
b0589f1
Merge branch 'master' into fgbio_nftest
maxulysse Jun 17, 2024
ceb8df1
Update modules/nf-core/fgbio/callmolecularconsensusreads/meta.yml
SPPearce Jun 29, 2024
4a99079
Update duplex snap
SPPearce Jul 2, 2024
44458bc
Merge branch 'master' into fgbio_nftest
SPPearce Jul 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion modules/nf-core/fgbio/callduplexconsensusreads/environment.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ channels:
- bioconda
- defaults
dependencies:
- bioconda::fgbio=2.0.2
- bioconda::fgbio=2.2.1
30 changes: 22 additions & 8 deletions modules/nf-core/fgbio/callduplexconsensusreads/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,24 @@ process FGBIO_CALLDUPLEXCONSENSUSREADS {

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/fgbio:2.0.2--hdfd78af_0' :
'biocontainers/fgbio:2.0.2--hdfd78af_0' }"
'https://depot.galaxyproject.org/singularity/fgbio:2.2.1--hdfd78af_0' :
'biocontainers/fgbio:2.2.1--hdfd78af_0' }"

input:
tuple val(meta), path(bam)
// please note:
// --min-reads is a required argument with no default
// --min-input-base-quality is a required argument with no default
// make sure they are specified via ext.args in your config
val min_reads
val min_baseq

output:
tuple val(meta), path("${prefix}.bam"), emit: bam
path "versions.yml" , emit: versions
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
prefix = task.ext.prefix ?: "${meta.id}_consensus"
prefix = task.ext.prefix ?: "${meta.id}_consensus_unmapped"
SPPearce marked this conversation as resolved.
Show resolved Hide resolved

def mem_gb = 8
if (!task.memory) {
Expand All @@ -35,6 +33,7 @@ process FGBIO_CALLDUPLEXCONSENSUSREADS {
mem_gb = task.memory.giga - 1
}
}
if ("$bam" == "${prefix}.bam") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!"

"""
fgbio \\
Expand All @@ -45,6 +44,8 @@ process FGBIO_CALLDUPLEXCONSENSUSREADS {
CallDuplexConsensusReads \\
--input $bam \\
--output ${prefix}.bam \\
--min-reads ${min_reads} \\
--min-input-base-quality ${min_baseq} \\
--threads ${task.cpus} \\
$args

Expand All @@ -53,4 +54,17 @@ process FGBIO_CALLDUPLEXCONSENSUSREADS {
fgbio: \$( echo \$(fgbio --version 2>&1 | tr -d '[:cntrl:]' ) | sed -e 's/^.*Version: //;s/\\[.*\$//')
END_VERSIONS
"""

stub:
prefix = task.ext.prefix ?: "${meta.id}_consensus"
SPPearce marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
prefix = task.ext.prefix ?: "${meta.id}_consensus"
prefix = task.ext.prefix ?: "${meta.id} _consensus_unmapped"

if ("$bam" == "${prefix}.bam") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!"
"""
touch ${prefix}.bam

cat <<-END_VERSIONS > versions.yml
"${task.process}":
fgbio: \$( echo \$(fgbio --version 2>&1 | tr -d '[:cntrl:]' ) | sed -e 's/^.*Version: //;s/\\[.*\$//')
END_VERSIONS
"""

}
13 changes: 9 additions & 4 deletions modules/nf-core/fgbio/callduplexconsensusreads/meta.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,23 @@ tools:
homepage: http://fulcrumgenomics.github.io/fgbio/
documentation: http://fulcrumgenomics.github.io/fgbio/tools/latest/CallDuplexConsensusReads.html
tool_dev_url: https://github.com/fulcrumgenomics/fgbio
licence: "['MIT']"
licence: ["MIT"]
input:
# Only when we have meta
- meta:
type: map
description: |
Groovy Map containing sample information
e.g. [ id:'test', single_end:false ]
- bam:
- grouped_bam:
SPPearce marked this conversation as resolved.
Show resolved Hide resolved
SPPearce marked this conversation as resolved.
Show resolved Hide resolved
type: file
description: BAM/SAM file
description: BAM/SAM file, grouped by UMI
pattern: "*.{bam,sam}"
- min_reads:
type: integer
description: Minimum number of original reads to build each consensus read.
SPPearce marked this conversation as resolved.
Show resolved Hide resolved
- min_baseq:
type: integer
description: Minimum base quality score
SPPearce marked this conversation as resolved.
Show resolved Hide resolved
output:
- meta:
type: map
Expand Down
62 changes: 62 additions & 0 deletions modules/nf-core/fgbio/callduplexconsensusreads/tests/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
nextflow_process {

name "Test Process FGBIO_CALLDUPLEXCONSENSUSREADS"
script "../main.nf"
process "FGBIO_CALLDUPLEXCONSENSUSREADS"

tag "modules"
tag "modules_nfcore"
tag "fgbio"
tag "fgbio/callduplexconsensusreads"

test("homo_sapiens - bam") {

when {
process {
"""
input[0] = [
[ id:'test', single_end:false ], // meta map
file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/umi/test.paired_end.duplex_umi_grouped.bam', checkIfExists: true)
]
input[1] = 3
input[2] = 20
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out).match() }
)
}

}

test("homo_sapiens - stub") {

options "-stub"

when {
process {
"""
input[0] = [
[ id:'test', single_end:false ], // meta map
file(params.modules_testdata_base_path + 'genomics/homo_sapiens/illumina/bam/umi/test.paired_end.duplex_umi_grouped.bam', checkIfExists: true)
]
input[1] = 3
input[2] = 20
"""
}
}

then {
assertAll(
{ assert process.success },
{ assert snapshot(process.out).match() }
)
}

}

}
72 changes: 72 additions & 0 deletions modules/nf-core/fgbio/callduplexconsensusreads/tests/main.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
{
"homo_sapiens - stub": {
"content": [
{
"0": [
[
{
"id": "test",
"single_end": false
},
"test_consensus.bam:md5,d41d8cd98f00b204e9800998ecf8427e"
]
],
"1": [
"versions.yml:md5,7277dba1bc055b578eb6d8d6af43b128"
],
"bam": [
[
{
"id": "test",
"single_end": false
},
"test_consensus.bam:md5,d41d8cd98f00b204e9800998ecf8427e"
]
],
"versions": [
"versions.yml:md5,7277dba1bc055b578eb6d8d6af43b128"
]
}
],
"meta": {
"nf-test": "0.8.4",
"nextflow": "23.10.1"
},
"timestamp": "2024-05-17T06:05:42.765492882"
},
"homo_sapiens - bam": {
"content": [
{
"0": [
[
{
"id": "test",
"single_end": false
},
"test_consensus_unmapped.bam:md5,4f0e87feb7601d06617c9f29d7aec352"
]
],
"1": [
"versions.yml:md5,7277dba1bc055b578eb6d8d6af43b128"
],
"bam": [
[
{
"id": "test",
"single_end": false
},
"test_consensus_unmapped.bam:md5,4f0e87feb7601d06617c9f29d7aec352"
]
],
"versions": [
"versions.yml:md5,7277dba1bc055b578eb6d8d6af43b128"
]
}
],
"meta": {
"nf-test": "0.8.4",
"nextflow": "23.10.1"
},
"timestamp": "2024-05-17T06:05:28.894178772"
}
}
2 changes: 2 additions & 0 deletions modules/nf-core/fgbio/callduplexconsensusreads/tests/tags.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
fgbio/callduplexconsensusreads:
- "modules/nf-core/fgbio/callduplexconsensusreads/**"
2 changes: 1 addition & 1 deletion modules/nf-core/fgbio/callmolecularconsensusreads/environment.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ channels:
- bioconda
- defaults
dependencies:
- bioconda::fgbio=2.0.2
- bioconda::fgbio=2.2.1
37 changes: 32 additions & 5 deletions modules/nf-core/fgbio/callmolecularconsensusreads/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ process FGBIO_CALLMOLECULARCONSENSUSREADS {

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/fgbio:2.0.2--hdfd78af_0' :
'biocontainers/fgbio:2.0.2--hdfd78af_0' }"
'https://depot.galaxyproject.org/singularity/fgbio:2.2.1--hdfd78af_0' :
'biocontainers/fgbio:2.2.1--hdfd78af_0' }"

input:
tuple val(meta), path(bam)
val min_reads
val min_baseq

output:
tuple val(meta), path("*.bam"), emit: bam
Expand All @@ -19,19 +21,44 @@ process FGBIO_CALLMOLECULARCONSENSUSREADS {

script:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
def prefix = task.ext.prefix ?: "${meta.id}_consensus_unmapped"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you add this custom suffix here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs some kind of name renaming to prevent it from colliding with the input file. Some of the subtools already done this, some don't. Although this one is now setting a double prefix in the output file, so that needs fixing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can add a check for that (a lot of modules do this already) => https://nf-co.re/docs/guidelines/components/modules#command-file-output-naming

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it needs the checks whatever.
I think it is better to have a default prefix that doesn't require you to add one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not one to argue with that, but I don't think diverting from the guidelines is a good idea, they're here for a reason. You can maybe bring this up on Slack?

SPPearce marked this conversation as resolved.
Show resolved Hide resolved
def mem_gb = 8
if (!task.memory) {
log.info '[fgbio CallMolecularConsensusReads] Available memory not known - defaulting to 8GB. Specify process memory requirements to change this.'
} else {
mem_gb = task.memory.giga
}
if ("$bam" == "${prefix}.bam") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!"
"""
fgbio \\
-Xmx${mem_gb}g \\
--tmp-dir=. \\
--async-io=true \\
--compression=1 \\
CallMolecularConsensusReads \\
--input $bam \\
--output ${prefix}.bam \\
--min-reads ${min_reads} \\
--min-input-base-quality ${min_baseq} \\
--threads ${task.cpus} \\
$args \\
--output ${prefix}.bam
$args;

cat <<-END_VERSIONS > versions.yml
"${task.process}":
fgbio: \$( echo \$(fgbio --version 2>&1 | tr -d '[:cntrl:]' ) | sed -e 's/^.*Version: //;s/\\[.*\$//')
END_VERSIONS
"""

stub:
prefix = task.ext.prefix ?: "${meta.id}_consensus_unmapped"
SPPearce marked this conversation as resolved.
Show resolved Hide resolved
if ("$bam" == "${prefix}.bam") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!"
"""
touch ${prefix}.bam

cat <<-END_VERSIONS > versions.yml
"${task.process}":
fgbio: \$( echo \$(fgbio --version 2>&1 | tr -d '[:cntrl:]' ) | sed -e 's/^.*Version: //;s/\\[.*\$//')
END_VERSIONS
"""

}
11 changes: 8 additions & 3 deletions modules/nf-core/fgbio/callmolecularconsensusreads/meta.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ keywords:
- UMIs
- consensus sequence
- bam
- sam
tools:
- fgbio:
description: Tools for working with genomic and high throughput sequencing data.
Expand All @@ -17,11 +16,17 @@ input:
description: |
Groovy Map containing sample information
e.g. [ id:'test', single_end:false, collapse:false ]
- bam:
- grouped_bam:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto, does the main.nf need to be updated?

type: file
description: |
The input SAM or BAM file.
The input SAM or BAM file, grouped by UMIs
pattern: "*.{bam,sam}"
- min_reads:
type: integer
description: Minimum number of original reads to build each consensus read.
- min_baseq:
type: integer
description: Minimum base quality score
SPPearce marked this conversation as resolved.
Show resolved Hide resolved
output:
- meta:
type: map
Expand Down
Loading
Loading