Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update isoseq modules and migrate to nf-test #4753

Merged
merged 45 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
8b8e2d2
isoseq3/cluster: migrate to nf-test
sguizard Jan 17, 2024
dc845c7
isoseq3/cluster Fix typos
sguizard Jan 17, 2024
f5b310d
isoseq3/refine update to version 4.0.0 and migrate to nf-test
sguizard Jan 17, 2024
8f03bdd
isoseq3/refine Fix linting
sguizard Jan 17, 2024
61a606a
isoseq3 /refine Update script path
sguizard Jan 17, 2024
23a68ee
pbccs migrate to nf-test
sguizard Jan 17, 2024
2b2bb3c
pbbam/pbmerge migrate to nf-test
sguizard Jan 17, 2024
5501384
lima update to 2.9.0 and migrate to nf-test
sguizard Jan 17, 2024
3b3ef76
Remove modules from pytest_modules.yml
sguizard Jan 17, 2024
65cbbe9
Merge branch 'master' into update_isoseq_modules
sguizard Jan 18, 2024
4ae280c
Merge branch 'master' into update_isoseq_modules
sguizard Jan 18, 2024
8f92251
Rename isoseq3 to isoseq
sguizard Jan 23, 2024
8caa425
Merge branch 'master' into update_isoseq_modules
sguizard Jan 23, 2024
b9b73a3
Update isoseq/cluster and isoseq/refine snap
sguizard Jan 23, 2024
98787cf
Update pbbam/pbmerge snap
sguizard Jan 23, 2024
096714e
Revert pbbam/pbmerge snap
sguizard Jan 23, 2024
e749c67
Merge branch 'master' into update_isoseq_modules
famosab Mar 21, 2024
81c506f
Merge branch 'master' into update_isoseq_modules
sguizard May 8, 2024
dace8e0
Update test dataset paths
sguizard May 8, 2024
a430048
Update test names
sguizard May 8, 2024
239112a
Add stub
sguizard May 10, 2024
42b027a
Add stub tests
sguizard May 10, 2024
5e7eb65
Apply Maxime's patch
sguizard May 10, 2024
ab56f17
Merge branch 'master' into update_isoseq_modules
sguizard May 13, 2024
4eaa459
Fix tests
sguizard May 13, 2024
769cfc4
Merge branch 'debug' into update_isoseq_modules
sguizard May 13, 2024
8537273
Remove PublishDir in config
sguizard May 13, 2024
5e50f5b
update snapshot
sguizard May 13, 2024
39df9dc
Fix test
sguizard May 13, 2024
527e809
Update test
sguizard May 13, 2024
f9af41b
Fix pbccs
sguizard May 13, 2024
369d9e2
Merge branch 'master' into update_isoseq_modules
sguizard May 13, 2024
256c411
Clean pbmerge config
sguizard May 13, 2024
07c615f
Fix isoseq/refine
sguizard May 14, 2024
f7e697d
Add version test to isoseq refine
sguizard May 14, 2024
3ab3572
Merge branch 'master' into update_isoseq_modules
sguizard May 14, 2024
818308a
Fix pbmerge versions test
sguizard May 14, 2024
07f2b93
Merge branch 'master' into update_isoseq_modules
sguizard May 14, 2024
46ed5ce
removed pbbam module
sguizard May 15, 2024
0db4e6d
Merge branch 'master' into update_isoseq_modules
sguizard May 15, 2024
c6adeb6
Rollback pbmerge to original state
sguizard May 15, 2024
76d760b
remove accidentally added vscode config
sguizard May 15, 2024
867d1db
Update tests/config/pytest_modules.yml
SPPearce May 15, 2024
de5eb44
fix pytest_modules.yml
sguizard May 15, 2024
16aae9c
prettier
sguizard May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion modules/nf-core/isoseq3/cluster/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ channels:
- bioconda
- defaults
dependencies:
- bioconda::isoseq3=3.8.1
- bioconda::isoseq3=4.0.0
8 changes: 4 additions & 4 deletions modules/nf-core/isoseq3/cluster/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ process ISOSEQ3_CLUSTER {

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/isoseq3:3.8.1--h9ee0642_0' :
'biocontainers/isoseq3:3.8.1--h9ee0642_0' }"
'https://depot.galaxyproject.org/singularity/isoseq3:4.0.0--h9ee0642_0' :
'biocontainers/isoseq3:4.0.0--h9ee0642_0' }"
sguizard marked this conversation as resolved.
Show resolved Hide resolved

input:
tuple val(meta), path(bam)
Expand All @@ -31,15 +31,15 @@ process ISOSEQ3_CLUSTER {
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
"""
isoseq3 \\
isoseq \\
cluster \\
$bam \\
${prefix}.transcripts.bam \\
$args

cat <<-END_VERSIONS > versions.yml
"${task.process}":
isoseq3: \$( isoseq3 cluster --version | head -n 1 | sed 's/isoseq cluster //g' | sed 's/ (.*//g' )
isoseq3: \$( isoseq cluster --version | head -n 1 | sed 's/isoseq cluster //g' | sed 's/ (.*//g' )
END_VERSIONS
"""
}
5 changes: 4 additions & 1 deletion modules/nf-core/isoseq3/cluster/meta.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ name: isoseq3_cluster
description: IsoSeq3 - Cluster - Cluster trimmed consensus sequences
keywords:
- cluster
- HiFi
- isoseq
- Pacbio
tools:
- isoseq3:
description: IsoSeq3 - Cluster - Cluster trimmed consensus sequences
Expand All @@ -25,7 +28,7 @@ output:
description: |
Groovy Map containing sample information
e.g. [ id:'test', single_end:false ]
- version:
- versions:
type: file
description: File containing software version
pattern: "versions.yml"
Expand Down
46 changes: 46 additions & 0 deletions modules/nf-core/isoseq3/cluster/tests/main.nf.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
nextflow_process {

name "Test Process ISOSEQ3_CLUSTER"
script "../main.nf"
config "./nextflow.config"
process "ISOSEQ3_CLUSTER"

tag "modules"
tag "modules_nfcore"
tag "isoseq3"
tag "isoseq3/cluster"

test("Main test") {

when {
process {
"""
input[0] = [
[ id:'test' ], // meta map
file(params.test_data['homo_sapiens']['pacbio']['refine'], checkIfExists: true),
]
"""
}
}

then {
assertAll {
assert process.success
assert snapshot(process.out.bam).match("bam")
assert snapshot(process.out.pbi).match("pbi")
assert snapshot(process.out.cluster).match("cluster")
assert snapshot(process.out.cluster_report).match("cluster_report")
assert snapshot(process.out.hq_bam).match("hq_bam")
assert snapshot(process.out.hq_pbi).match("hq_pbi")
assert snapshot(process.out.lq_bam).match("lq_bam")
assert snapshot(process.out.lq_pbi).match("lq_pbi")
assert snapshot(process.out.singletons_bam).match("singletons_bam")
assert snapshot(process.out.singletons_pbi).match("singletons_pbi")
assert snapshot(process.out.versions).match("versions")
}

}

}

}
140 changes: 140 additions & 0 deletions modules/nf-core/isoseq3/cluster/tests/main.nf.test.snap
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
{
"cluster": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.cluster:md5,e25f5e4179bc8b6accec024008001c52"
]
]
],
"timestamp": "2024-01-16T17:51:37.025827"
},
"singletons_bam": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.singletons.bam:md5,f8bcaa7d071be24c199d4243e77a65a6"
]
]
],
"timestamp": "2024-01-17T08:55:36.093361"
},
"lq_bam": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.lq.bam:md5,f4948859ae09db09221ef7c08b9dc1ea"
]
]
],
"timestamp": "2024-01-17T08:55:36.011969"
},
"hq_bam": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.hq.bam:md5,0ad13a6f56aa1800cc553ee3e8f52e59"
]
]
],
"timestamp": "2024-01-17T08:55:35.949396"
},
"versions": {
"content": [
[
"versions.yml:md5,a10cb8125ff5dc181f08493e68900b4d"
]
],
"timestamp": "2024-01-16T17:51:37.300931"
},
"pbi": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.bam.pbi:md5,0f9569d3132bc58a3b6827364215d2a8"
]
]
],
"timestamp": "2024-01-17T08:55:35.917491"
},
"singletons_pbi": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.singletons.bam.pbi:md5,f63eed07e30bc66a72c6ebd24d0116e0"
]
]
],
"timestamp": "2024-01-17T08:55:36.151147"
},
"cluster_report": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.cluster_report.csv:md5,5437fc2d59122b28d81c3f1215f6ff17"
]
]
],
"timestamp": "2024-01-16T17:51:37.069648"
},
"hq_pbi": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.hq.bam.pbi:md5,4eecd253bb8cb0dbdc039ec80bb88ba0"
]
]
],
"timestamp": "2024-01-17T08:55:35.977037"
},
"lq_pbi": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.lq.bam.pbi:md5,d1f96a5748c71d9eb1b69c620412c863"
]
]
],
"timestamp": "2024-01-17T08:55:36.032895"
},
"bam": {
"content": [
[
[
{
"id": "test"
},
"test.transcripts.bam:md5,540c126bca7bd75f7a9084d3e2e642f7"
]
]
],
"timestamp": "2024-01-17T08:55:35.880903"
}
}
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
process {

publishDir = { "${params.outdir}/${task.process.tokenize(':')[-1].tokenize('_')[0].toLowerCase()}" }

withName: PBCCS {
ext.args = '--min-rq 0.9'
}
ext.args = '--singletons --use-qvs --verbose'

}
2 changes: 2 additions & 0 deletions modules/nf-core/isoseq3/cluster/tests/tags.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
isoseq3/cluster:
- modules/nf-core/isoseq3/cluster/**
2 changes: 1 addition & 1 deletion modules/nf-core/isoseq3/refine/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ channels:
- bioconda
- defaults
dependencies:
- bioconda::isoseq3=3.8.2
- bioconda::isoseq3=4.0.0
20 changes: 10 additions & 10 deletions modules/nf-core/isoseq3/refine/main.nf
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,20 @@ process ISOSEQ3_REFINE {

conda "${moduleDir}/environment.yml"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/isoseq3:3.8.2--h9ee0642_0' :
'biocontainers/isoseq3:3.8.2--h9ee0642_0' }"
'https://depot.galaxyproject.org/singularity/isoseq3:4.0.0--h9ee0642_0' :
'biocontainers/isoseq3:4.0.0--h9ee0642_0' }"

input:
tuple val(meta), path(bam)
path primers

output:
tuple val(meta), path("*.bam") , emit: bam
tuple val(meta), path("*.bam.pbi") , emit: pbi
tuple val(meta), path("*.consensusreadset.xml") , emit: consensusreadset
tuple val(meta), path("*.filter_summary.report.json") , emit: summary
tuple val(meta), path("*.report.csv") , emit: report
path "versions.yml" , emit: versions
tuple val(meta), path("*.bam") , emit: bam
tuple val(meta), path("*.bam.pbi") , emit: pbi
tuple val(meta), path("*.consensusreadset.xml") , emit: consensusreadset
tuple val(meta), path("*.filter_summary.report.json"), emit: summary
tuple val(meta), path("*.report.csv") , emit: report
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when
Expand All @@ -26,7 +26,7 @@ process ISOSEQ3_REFINE {
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${meta.id}"
"""
isoseq3 \\
isoseq \\
refine \\
-j $task.cpus \\
$args \\
Expand All @@ -36,7 +36,7 @@ process ISOSEQ3_REFINE {

cat <<-END_VERSIONS > versions.yml
"${task.process}":
isoseq3: \$( isoseq3 refine --version | head -n 1 | sed 's/isoseq refine //' | sed 's/ (commit.\\+//' )
isoseq3: \$( isoseq refine --version | head -n 1 | sed 's/isoseq refine //' | sed 's/ (commit.\\+//' )
END_VERSIONS
"""
}
37 changes: 37 additions & 0 deletions modules/nf-core/isoseq3/refine/tests/main.nf.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
nextflow_process {

name "Test Process ISOSEQ3_REFINE"
script "../main.nf"
config "./nextflow.config"
process "ISOSEQ3_REFINE"

tag "modules"
tag "modules_nfcore"
tag "isoseq3"
tag "isoseq3/refine"

test("Main test") {

when {
process {
"""
input[0] = [
[ id:'test' ], // meta map
file(params.test_data['homo_sapiens']['pacbio']['lima'], checkIfExists: true),
]
input[1] = file(params.test_data['homo_sapiens']['pacbio']['primers'], checkIfExists: true)
"""
}
}

then {
assert process.success
assert snapshot(process.out.bam).match("bam")
assert snapshot(process.out.pbi).match("pbi")
assert snapshot(process.out.report).match("report")
assert snapshot(process.out.versions).match("versions")
}

}

}
49 changes: 49 additions & 0 deletions modules/nf-core/isoseq3/refine/tests/main.nf.test.snap
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{
"versions": {
"content": [
[
"versions.yml:md5,6cc3190a74e747de318b9b1227379bee"
]
],
"timestamp": "2024-01-17T10:06:26.844337"
},
"pbi": {
"content": [
[
[
{
"id": "test"
},
"test.refine.bam.pbi:md5,fcd3c9b9c7efea71732591ff1271a3b5"
]
]
],
"timestamp": "2024-01-17T10:06:26.482922"
},
"report": {
"content": [
[
[
{
"id": "test"
},
"test.refine.report.csv:md5,d42a139e5d9b08396bdb087c01243ea9"
]
]
],
"timestamp": "2024-01-17T10:06:26.665805"
},
"bam": {
"content": [
[
[
{
"id": "test"
},
"test.refine.bam:md5,4a666931c2c8843d8a8f0d901e1ba15f"
]
]
],
"timestamp": "2024-01-17T10:06:26.286284"
}
}
Loading