Bump fgbio versions and swap to nftest #5624

SPPearce · 2024-05-17T06:14:26Z

Starting the process of updating fgbio versions.
This is porting the modules from fastquorum, updating them to the most recent fgbio version and swapping to nf-test

modules/nf-core/fgbio/callduplexconsensusreads/tests/main.nf.test

nvnieuwk · 2024-05-21T08:40:58Z

modules/nf-core/fgbio/callmolecularconsensusreads/main.nf

@@ -19,19 +21,42 @@ process FGBIO_CALLMOLECULARCONSENSUSREADS {

    script:
    def args = task.ext.args ?: ''
-    def prefix = task.ext.prefix ?: "${meta.id}"
+    def prefix = task.ext.prefix ?: "${meta.id}_consensus_unmapped"


Why did you add this custom suffix here?

It needs some kind of name renaming to prevent it from colliding with the input file. Some of the subtools already done this, some don't. Although this one is now setting a double prefix in the output file, so that needs fixing.

You can add a check for that (a lot of modules do this already) => https://nf-co.re/docs/guidelines/components/modules#command-file-output-naming

Yes, it needs the checks whatever.
I think it is better to have a default prefix that doesn't require you to add one.

I'm not one to argue with that, but I don't think diverting from the guidelines is a good idea, they're here for a reason. You can maybe bring this up on Slack?

modules/nf-core/fgbio/groupreadsbyumi/main.nf

nvnieuwk

I left some comments from a quick look at it, I'll look at it a bit more thoroughly if you fixed these comments :p

modules/nf-core/fgbio/callmolecularconsensusreads/main.nf

nvnieuwk

Hi some suggestions left (I was unable to provide suggestions for fgbio/fastqtobam, but can you fix that one too?). Feel free to merge once you've addressed those :)

modules/nf-core/fgbio/callmolecularconsensusreads/main.nf

modules/nf-core/fgbio/groupreadsbyumi/main.nf

modules/nf-core/fgbio/sortbam/main.nf

modules/nf-core/fgbio/zipperbams/main.nf

modules/nf-core/fgbio/callduplexconsensusreads/main.nf

SPPearce · 2024-05-21T17:58:23Z

As is currently implemented here, I am keen to include both the check that the input and output files have different names, but also set a default value on the prefix.

nh13

Some minor suggestions, that I don't need to re-review. Thank-you for doing this!

modules/nf-core/fgbio/callduplexconsensusreads/main.nf

nh13 · 2024-05-22T05:32:22Z

modules/nf-core/fgbio/callduplexconsensusreads/main.nf

@@ -53,4 +54,17 @@ process FGBIO_CALLDUPLEXCONSENSUSREADS {
        fgbio: \$( echo \$(fgbio --version 2>&1 | tr -d '[:cntrl:]' ) | sed -e 's/^.*Version: //;s/\\[.*\$//')
    END_VERSIONS
    """
+
+    stub:
+    prefix = task.ext.prefix ?: "${meta.id}_consensus"


Suggested change

prefix = task.ext.prefix ?: "${meta.id}_consensus"

prefix = task.ext.prefix ?: "${meta.id} _consensus_unmapped"

modules/nf-core/fgbio/callduplexconsensusreads/meta.yml

modules/nf-core/fgbio/callmolecularconsensusreads/meta.yml

modules/nf-core/fgbio/groupreadsbyumi/meta.yml

modules/nf-core/fgbio/zipperbams/meta.yml

modules/nf-core/fgbio/callduplexconsensusreads/meta.yml

Co-authored-by: Nils Homer <[email protected]>

* Convert callduplex and callmolecular to nftest and update version * Convert zipperbams * Convert groupreadsbyumi * Convert sortbam and add stubs * Convert filterconsensusreads * Update snapshot * Update modules/nf-core/fgbio/callmolecularconsensusreads/main.nf * Add collision checks, swap test data paths * Fix conda version and linting * Apply suggestions from code review Co-authored-by: Nils Homer <[email protected]> * Update modules/nf-core/fgbio/callmolecularconsensusreads/meta.yml Co-authored-by: Nils Homer <[email protected]> * Update duplex snap --------- Co-authored-by: Nils Homer <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]>

@SPPearce

* Update Abricate to include (missing) additional optional input channel and add stub (#5849) * Add missing optional input channel to abricate * Update abricate module to include missing optional input path * Update modules/nf-core/abricate/run/tests/main.nf.test Co-authored-by: Carson J Miller <[email protected]> * Fix linting * Update modules/nf-core/abricate/run/main.nf * Get downstream module test --------- Co-authored-by: Carson J Miller <[email protected]> * Fix custom/catadditionalfasta (#5851) * Update environment.yml * Update main.nf * update to samtools 1.20 (#5852) * update to samtools 1.20 * keep samtools 1.19.2 for now * update snapshot * [automated] Fix linting with Prettier * keep 1.20 afterall * update snapshot --------- Co-authored-by: nf-core-bot <[email protected]> * Update input params for bwameme (#5853) * update recipe * update rthreads * update tests * Update meta.yml * chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.4.10 (#5857) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * `--output-unselected` option in `samtools view` (#2674) * add unoutput option * trying to fix emit * CI fixed, tests still incorrect * Fixed the checksums * Fixed the syntax * Call this "unselected" rather than "unoutput" * Alignment rocks * bugfix: the variable needs to be visible by the output declaration * bugfix: this was not the regular file name * Updated the test case * fixup! Alignment rocks * Updated the meta.yml --------- Co-authored-by: Matthieu Muffato <[email protected]> * stubs and test for rnaseq modules - part I (#5854) * stubs and test for bbmap_bbsplit * add test for index * add stubs for custom/catadditionalfasta * add stubs for fq/subsample * add stubs for hisat2/extractsplicesites * update fastq_align_hisat2 snapshot * add stubs for hisat2/build * add stubs for hisat2/align * more stubs bbmap/bbsplit * add stubs for preseq/lcextrap * more stubs for fq/subsample * proper stubs for hisat2/align * remove old snaps * Remove CUTUPFASTA from pytests * Revert "Remove CUTUPFASTA from pytests" This reverts commit 0fa8c1b. * Bump concoct/cutupfasta version, add stub, add nf-test (#5864) * Bump concoct/cutupfasta version, add stub, add nf-test * Fix linting * Adding meta to multivcfanalyzer, updating test to nf-test (#5773) * Adding meta to multivcfanalyzer, updating test to nf-test * Apply suggestions from code review Co-authored-by: James A. Fellows Yates <[email protected]> * fix meta.yml * Add sort to vcfs * Add tab to tags * add checks for files due to failing md5 sums across docker,conda,singularity --------- Co-authored-by: James A. Fellows Yates <[email protected]> Co-authored-by: Simon Pearce <[email protected]> * Concoct/concoctcoveragetable modules updates (#5868) * Bump concoct/cutupfasta version, add stub, add nf-test * Bump conda build, add stub and nf-test for CONCOT/concoctcoveragetable * Remove pytest entry --------- Co-authored-by: Aida Andrades Valtueña <[email protected]> * Added stub logic and test to fastq_fastqc_umitools_fastp (#5858) * Added stub logic and test to fastq_fastqc_umitools_fastp * Now emitting trimmed reads * Adding input value-channel for controlling spring-decompress cmd and output (#5850) * Enable single fastq.gz-output from spring/decompress by setting meta.single_end or meta.one_strand * Controlling output with value-channel write_one_fastq_gz * Update modules/nf-core/spring/decompress/meta.yml Co-authored-by: Maxime U Garcia <[email protected]> * nf-test for spring/decompress * Trying to please linter * Trying to please linter --------- Co-authored-by: Simon Pearce <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]> * Update stranger (#5717) * everything but snapshot * fix gz snap * update testdata paths * Update main.nf.test * update test --------- Co-authored-by: Simon Pearce <[email protected]> * mapAD: fix `versions.yml` emission (#5874) mapAD: fix `version.yml` emission * add stub and stub test to chromap, change test names (#5872) * add stub and stub test to chromap, change test names * add stub and stub test to chromap, change test names * change setup * move setup out of test * move * produce stub files (#5875) * chore(deps): update pre-commit hook python-jsonschema/check-jsonschema to v0.28.6 (#5878) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * update peddy tests (#5880) update peddy * Update main.nf (#5870) * Update main.nf VEP html report is missing when publishing results because by default vep html file is `_summary.html` instead of `.summary.html`. This changes just adjust the module to the vep defaults to avoid the issue. * Updated html pattern in VEP to be more generic * bump MultiQC version (#5884) bump MQC * Bump shinyngs modules (#5881) * Bump shinyngs modules * Bump versions for snapshot * Update CONCOCT module(s): conda build bump, stub, and nf-test (#5869) * Version update, stub, nf-test for concoct/concoct * Bump version, add nf-test, stub to concoct/mergecutupclustering * nf-test, stub, and version conda build version bump for concoct/extrafastabins * Remove a superfluous config and specify a used one * Update modules/nf-core/concoct/extractfastabins/main.nf Co-authored-by: Simon Pearce <[email protected]> * Remove superfluous config file * Clean up code * Repair all stub tests that were empty --------- Co-authored-by: Simon Pearce <[email protected]> * `fasta_binning_concoct` subworkflow: do not fail on join (#5824) * fix: do not fail on mismatch for samples not binned with concoct * Migrate to pytest * Improved Harshil alignment --------- Co-authored-by: James A. Fellows Yates <[email protected]> * new module: datavzrd (#5883) * Generated structure for new module * Implemented module * Added semi-functional tests * Cleaned tests - test don't work yet * fix main, start nf-test * add new test data * fix linting * try to assert folders * fix tests * change output to prefix * change to prefix * update to prefix --------- Co-authored-by: vickylaram <[email protected]> * samtools/reheader: Fix bug causing output name to be incorrectly evaluated (#5887) Fix bug causing output name to be incorrectly evaluated * nf-test for snpsites (#5284) * nf-test for snpsites * remove old pytest * Delete modules/nf-core/snpsites/tests/nextflow.config * Update modules/nf-core/snpsites/tests/main.nf.test Co-authored-by: Joon Klaps <[email protected]> * Update assert Co-authored-by: Joon Klaps <[email protected]> * renamed test Co-authored-by: Joon Klaps <[email protected]> * reverted the input * updated input...again Co-authored-by: Joon Klaps <[email protected]> * fixed typo Co-authored-by: Joon Klaps <[email protected]> * Regenerate snapshot and fix linting --------- Co-authored-by: Joon Klaps <[email protected]> Co-authored-by: Simon Pearce <[email protected]> * Added prefix to gunzip and while keeping the default name as `archive.toString() - '.gz'` (#5879) * Added prefix to gunzip * Fixed linting issues in fargene and leviosam2/lift * Removed unstable liftoff/conda * add dedicated output channel for trimmomatic stderr log (needed for multiqc) (#5501) * created new output channel capturing stderr of trimmomatic * updated output channel for trimmomatic * Add stub test and update snapshots * Add stub and update tests * Fix linting and add tee --------- Co-authored-by: John Palmer <[email protected]> Co-authored-by: Simon Pearce <[email protected]> Co-authored-by: James A. Fellows Yates <[email protected]> * `ariba/getref` port to nf-test, add meta to tuple and stub (#5877) * port to nf-test, add meta to tuple and stub * modify meta and fix tags * change from each to tuple * Pass annotation files to last-dotplot (#5885) * Pass annotation files to last-dotplot The annotation of the target (plotted horizontally) genome is passed as a new channel. The annotation of the query (plotted vertically) genome is passed as a new component of the first channel. This is because this module typically loops on a list of alignments of various query genomes (from samplesheet) to a single target genome. The annotation files are called `annot_a` and `annot_b` because they are passed to the `-a` and `-b` arguments of `last-dotplot`. * Put arguments on separate lines. * Add meta map and reorder channels. * Test with dummy annotation files. Thanks @SPPearce for the hint. * Add meta to bowtie build (#3848) * Add meta to bowtie build * Swap fastq ngscheckmate tests * Swap to nf-test * Remove spurious changes to bowtie2 * Actually revert bowtie2 * Add the rest of bowtie2 * Again * Revert bowtie2 changes * Swap to meta.id as prefix * Swap bowtie align to nftest * Update tests * Use nft-bam * Update modules/nf-core/bowtie/build/tests/main.nf.test Co-authored-by: Mahesh Binzer-Panchal <[email protected]> * Swap to testdata path * Update modules/nf-core/ngscheckmate/patterngenerator/main.nf * Swap index finding to $ * Fix vafncm test * Update meta * Remove bam header md5sum * Update fastq test * Update paired bam test * Delete modules/nf-core/bowtie/align/tests/nextflow.config --------- Co-authored-by: Mahesh Binzer-Panchal <[email protected]> * nf-test for pilon (#5293) * nf-test for pilon * linting for license * Add stub and extra test * Swap to testdata_base_path * Add missing END_VERSIONS and alignment * Add optional outputs to stub --------- Co-authored-by: Simon Pearce <[email protected]> * Bump phantompeakqualtools 1.2.2 (#5889) * Add stub section * Update yml * Add nf-test * Get rid of pytest * Make lint happy * Update tests * Update tests to make lint happy * update tests * Make lint happy * Using the same version as CI of nf-test locally might be a good idea * Use wave images * Make lint happy * Simplify tests --------- Co-authored-by: Simon Pearce <[email protected]> * Added variance filtering to custom/matrixfilter (#5358) * Added variant filtering to matrix filter * Corrected indentation * Simplified function in single conditional block * fixed line indentation * Fixed whitespace * Fixed logic * line wrapping change * added whitespace around comments * [automated] Fix linting with Prettier * remove trailing whitespace * Update matrixfilter.R * Remove whitespace --------- Co-authored-by: Jonathan Manning <[email protected]> Co-authored-by: nf-core-bot <[email protected]> Co-authored-by: Jonathan Manning <[email protected]> Co-authored-by: Simon Pearce <[email protected]> * Update md5sum module to be able to take multiple files (#3610) * Fix version string spurious text in singularity tests * Add config for subworkflow * Update md5sum module to work with multiple files * linting * Fix stub * Add option to generate single file * Update modules/nf-core/md5sum/main.nf Co-authored-by: Matthieu Muffato <[email protected]> * Simplify versions processing --------- Co-authored-by: Matthieu Muffato <[email protected]> * Add share to find path (#5899) * build: Bump deepvariant to 1.6.1 (#5801) * build: Bump deepvariant to 1.6.1 * interim dir not allowed to be a dot in 1.6 * Update modules/nf-core/deepvariant/main.nf Co-authored-by: Maxime U Garcia <[email protected]> * hack: Hard code version google/deepvariant#830 --------- Co-authored-by: Felix Lenner <[email protected]> Co-authored-by: Felix Lenner <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]> * bump somalier/extract and somalier/relate version (#5894) * bump version * nf-test migratino * revert ancestry * fix linting * Update snap * fix unicode space --------- Co-authored-by: Simon Pearce <[email protected]> * Added evigene/tr2aacds (#5898) * Added evigene/tr2aacds * Sorted file list and added activation script * Fixed issues raised by feedback * correct president stub (#5904) * correct president stub * Large update to module * Update prefix and meta --------- Co-authored-by: Simon Pearce <[email protected]> * Fix EClint errors (#5907) Fix EClint * New module: nanoq (#5896) * Add the new module nanoq * Update modules/nf-core/nanoq/main.nf Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/nanoq/main.nf Co-authored-by: Simon Pearce <[email protected]> * define the output_format * Update modules/nf-core/nanoq/main.nf Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/nanoq/main.nf Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/nanoq/main.nf Co-authored-by: Simon Pearce <[email protected]> * update the stub run with gzip * Update modules/nf-core/nanoq/main.nf Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/nanoq/main.nf Co-authored-by: Simon Pearce <[email protected]> --------- Co-authored-by: Simon Pearce <[email protected]> * Foldseek: createdb and easysearch modules version bump + stub nf-tests added (#5906) * foldseek/createdb version bump, added stub nf-test * foldseek/easysearch version bump, stub nf-test added * test.snap for createdb added * sourmash/compare stub, nf-test, snapshot (#4665) * sourmash/compare stub, nf-test, snapshot * Add sort * Add a sort * Update snap * Try sort inside the module * Add meta to output and change formatting on test --------- Co-authored-by: Simon Pearce <[email protected]> * adding chewbbaca/createschema module (#4512) * adding chewbbaca/createschema module * updated for handling a mix of uncompressed and compressed files * update the formatting * update the formatting * updated module to emit meta * updated test.yml * update pytest * clean the meta.yml * clean the meta.yml * clean the meta.yml * replaced conda package with environment.yml file * py-test to nf-test migration * add stub and update tests, snapshot * updated stub * update stub test * update stub test * bump the version * update * updating container version * updated test snapshot * updated nf-test snapshot * added libmamba to environment file * Fix tests and linting * Update modules/nf-core/chewbbaca/createschema/main.nf Co-authored-by: Jose Espinosa-Carrasco <[email protected]> * Apply suggestions from code review Co-authored-by: Jose Espinosa-Carrasco <[email protected]> --------- Co-authored-by: Gisela Gabernet <[email protected]> Co-authored-by: Sateesh <[email protected]> Co-authored-by: Simon Pearce <[email protected]> Co-authored-by: Jose Espinosa-Carrasco <[email protected]> * stubs and test for rnaseq modules - part II (#5861) * add stubs for rsem * add stubs for rseqc * add stubs for subread/featurecounts * add stubs for trimgalore * add stubs for ucsc/bedclip * add stubs and more tests for umitools/extract * update snap * update snapshot for umitools/extract * fix umitools/extract tests * Apply suggestions from code review Co-authored-by: Simon Pearce <[email protected]> --------- Co-authored-by: Simon Pearce <[email protected]> * affy/justrma add stub, nf-test and snapshot (#4651) * affy/justrma add stub, nf-test and snapshot * Update snapshot and environment * Update snap * Update modules/nf-core/affy/justrma/main.nf Co-authored-by: James A. Fellows Yates <[email protected]> * Swap test data file paths, fix config, update snaps and stub --------- Co-authored-by: Famke Bäuerle <[email protected]> Co-authored-by: Simon Pearce <[email protected]> Co-authored-by: James A. Fellows Yates <[email protected]> * Bump fgbio versions and swap to nftest (#5624) * Convert callduplex and callmolecular to nftest and update version * Convert zipperbams * Convert groupreadsbyumi * Convert sortbam and add stubs * Convert filterconsensusreads * Update snapshot * Update modules/nf-core/fgbio/callmolecularconsensusreads/main.nf * Add collision checks, swap test data paths * Fix conda version and linting * Apply suggestions from code review Co-authored-by: Nils Homer <[email protected]> * Update modules/nf-core/fgbio/callmolecularconsensusreads/meta.yml Co-authored-by: Nils Homer <[email protected]> * Update duplex snap --------- Co-authored-by: Nils Homer <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]> * Added pbtk/pbindex (#5901) * Added pbtk/pbindex * Removed prefix * Added agat/spfilterfeaturefromkilllist (#5771) * Added agat/spfilterfeaturefromkilllist * Added AGAT config * Included test with agat config * bump stranger to 0.9.1 (#5910) bump stranger * Update nft-bam to v0.3.0 (#5912) * Add stubs for rnaseq subworkflows - part I (#5911) * fix stub and add tests + single snapshot * add stub * fix stub * snapshot versions + better snap for stub * add versions to snapshot + add tests for stub * Add stub tests + single snapshot * Add stub tests + single snapshot * maxime sorted * Add stub tests + single snapshot * Add stub tests * Add stub tests + single snapshot * code polish + update snap * update snapshot using nft-bam * Add stubs for rnaseq subworkflows - part II (#5913) * Add stub tests + single snapshot * Add stub tests + single snapshot * fix stub and update tests for kallisto modules * Add stub tests + single snapshot * Add stub tests + single snapshot * bcftools isec input can be bcf or bcf.gz as well as vcf.gz (#5685) * bcftools isec input can be bcf or bcf.gz as well as vcf.gz * Add stub and stub test * Update tests * Update meta.yml * Add stub to name of stub test * Update meta * Update tests Changes: - exclude .tbi files from snapshot due to failing conda tests - update main.nf.test.snap - specificy path type of the output * Update tests again Changes: - include versions - sort file name list for consistent order - update snapshot --------- Co-authored-by: Simon Pearce <[email protected]> Co-authored-by: Theresa Störiko <[email protected]> Co-authored-by: Theresa Störiko <[email protected]> * Add stubs for rnaseq subworkflows - part III (#5921) * do not snapshot input * do not snapshot input * Add stubs for rnaseq subworkflows - part IV (#5922) do not snapshot input with stub * mulled biocontainer for vardict-java and htslib (#5920) * vardict-java bgzip mulled container * update bioconda env * code polishing Co-authored-by: Nicolas Vannieuwkerke <[email protected]> * Update modules/nf-core/vardictjava/main.nf Co-authored-by: Nicolas Vannieuwkerke <[email protected]> * args3 for bgzip * args3 * Update snap * removed whitespace --------- Co-authored-by: Nicolas Vannieuwkerke <[email protected]> Co-authored-by: Simon Pearce <[email protected]> * Add same file clash check to bioawk (#5926) * Update pre-commit hook astral-sh/ruff-pre-commit to v0.5.1 (#5893) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update actions/upload-artifact digest to 0b2256b (#5927) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * dysgu==1.6.4_version update (#5928) * all_files_update * Update environment.yml * update_meta.yml * input==input_bam * update_snapshot * meta.yml,tags.yml update * Update modules/nf-core/dysgu/meta.yml Co-authored-by: Famke Bäuerle <[email protected]> * Update modules/nf-core/dysgu/meta.yml Co-authored-by: Famke Bäuerle <[email protected]> * Update modules/nf-core/dysgu/meta.yml Co-authored-by: Famke Bäuerle <[email protected]> * Update modules/nf-core/dysgu/meta.yml Co-authored-by: Famke Bäuerle <[email protected]> * Update modules/nf-core/dysgu/meta.yml Co-authored-by: Famke Bäuerle <[email protected]> * update_author --------- Co-authored-by: Famke Bäuerle <[email protected]> * ADD argNorm module (#5905) * ADD argnorm module * ENH correct pattern for 'hamronized' input * ENH specify hamronization command in script * ENH don't pass 'hamronized' as val * Fixed trailing whitespace * "single_end" is not relevant in this module * alignment * "db_args" to avoid any confusion with the input channel * Added a test to check $tool is provided * Implemented the stub mode * Added a test for the missing database * Reordered the arguments to match the script --------- Co-authored-by: Matthieu Muffato <[email protected]> * Sentieon license server - testing (#5856) * test it out * ci: Add license_message script nf-core/sarek#1380 https://github.com/DonFreed/docker-actions-test?tab=readme-ov-file * test: Add tests for sentieon bwamem Because the pytest-workflow tests are taking so long * Add scratch from meeting * Clean up sentieon secrets * ci: Fix Nextflow secrets setup * fix(sentieon): Remove encryption key because it won't get used * fix # * ci: Fix when to setup secrets * feat: Add sentieon auth mech secret * chore(sentieon): Remove bwamem pytest-workflow tests * fix: Spike in with environment variables * fix: Add SENTIEON_AUTH_DATA in env file The reason for this is because when we require it in the module it fails. That doesn't make sense for normal Sentieon users. They don't care, this is more of a GitHub actions thing. * docs(sentieon): Make this a README * test: Add local testing setup * test: Remove publishDir * style: Remove sentieon/bwamem * fix: Change comments to prints for debugging and better reporting * fix: Add a quote? * chore: Remove unnecessary variable * debug: Print variables * Fix typo Co-authored-by: Anders Sune Pedersen <[email protected]> * chore: Clean up code * docs: Write up all the learnings from Sentieon * style: Run prettier * style: Add tags for nf-core lint * docs: Clean up notes Co-authored-by: maxulysse <[email protected]> * chore: Add @DonFreed as maintainer --------- Co-authored-by: Edmund Miller <[email protected]> Co-authored-by: Edmund Miller <[email protected]> Co-authored-by: Anders Sune Pedersen <[email protected]> Co-authored-by: maxulysse <[email protected]> * Fix Sentieon CI (#5934) * ci: Remove setting secrets in pytest-workflow * ci: Add a longer note about conditionals https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#using-secrets-in-a-workflow Secrets cannot be directly referenced in if: conditionals. Instead, consider setting secrets as job-level environment variables, then referencing the environment variables to conditionally run steps in the job. For more information, see "Contexts" and jobs.<job_id>.steps[*].if. * ci: Check for secrets in env Because if this is coming from a fork it will fail * disable conda setup when not using a conda profile (#5935) * stubs and test for rnaseq modules - part III (#5925) * improve bbmap_split stub * update stub tests * improve stub * single snapshot * fix stub + add stub tests * fix stub + add stub tests * fix stub * restore tags * fix stub + better stub tests * fix snapshot * NARFMAP segfault (#4895) * test(narfmap): Add segfault test * fix(narfmap): Clean up log error * test(narfmap): Should segfault * chore(narfmap): Bump to 1.4.2 * test(narfmap): Mark test as passing bioinformaticsorphanage/NARFMAP#10 * style: Add extra tag for nf-core/lint * test: Update all the snapshot declarations * chore: Add a TODO for poddar's test * narfmap cleanup (#5937) * chore: Add Note about issue * test: Remove bam name Co-authored-by: maxulysse <[email protected]> --------- Co-authored-by: maxulysse <[email protected]> * Add CRAM/index support to NARFMAP (#5946) * feat: Add cram/index support to narfmap * chore: Remove singularity image comment * test: Update tests for CRAM * Update actions/setup-node digest to 1e60f62 (#5947) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Fix bcftools plugin split (#5942) * bump bclconvert (#5952) * bump bclconvert * fix snapshots * bump multiqc (#5950) * bump multiqc * update snapshot * chore: bump picard (#5948) * bump picard * fix hsmetrics test * fix markduplicates * fix extractfingerprints * fix addorreplacereadgroups * fix crosscheckfingerprints * Add prefix to output file for barrnap (#5919) * Add prefix to output file * Update barrnap to nf-test * Update tags.yml * Add missing bracket and run test * Correct format of license * Update modules/nf-core/barrnap/tests/main.nf.test Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/barrnap/tests/main.nf.test Co-authored-by: Simon Pearce <[email protected]> * Review suggestions * Prettier * Update meta.yml * Update tests --------- Co-authored-by: Simon Pearce <[email protected]> * chore(deps): update actions/setup-python digest to 39cd149 (#5961) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * LAST alignment statistics for MultiQC (#5902) * last train modified to output multiqc * Report alignment length and percent similarity, for MultiQC. * Update snapshot file. * Document the new channels * Fix indentation * Update test results. * Give a longer suffix for better MultiQC search patterns. --------- Co-authored-by: Mohammed Mahdi <[email protected]> * Update lofreq somatic and callparallel (#5931) * update lofreq callparallel and somatic ssh * [automated] Fix linting with Prettier * suggestions * remove files tests * label lofreq somatic * Update modules/nf-core/lofreq/callparallel/main.nf Co-authored-by: Famke Bäuerle <[email protected]> * port lofreq/callparallel to nf-test * Swap to nftest * update paths * add meta * Update main.nf * modify main * update paths * fix small errors * Update modules/nf-core/lofreq/somatic/tests/main.nf.test Co-authored-by: Simon Pearce <[email protected]> * Update main.nf.test.snap --------- Co-authored-by: AitorOP <[email protected]> Co-authored-by: nf-core-bot <[email protected]> Co-authored-by: Famke Bäuerle <[email protected]> Co-authored-by: Francisco Martínez <[email protected]> Co-authored-by: Famke Bäuerle <[email protected]> Co-authored-by: Simon Pearce <[email protected]> * Update dupradar.r according to new MultiQC (>=v1.22) config values (#5943) Update dupradar.r according to new MultiQC config values. Co-authored-by: Matthias Zepper <[email protected]> * Update the module: nanoq (#5964) * update nanoq module * update nf-test * Update Glimpse2 phase (#5944) * Update Glimpse2 phase * Fix label * Update snapshot * Update modules/nf-core/glimpse2/phase/main.nf Co-authored-by: Maxime U Garcia <[email protected]> * Update meta.yml --------- Co-authored-by: LouisLeNezet <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]> * Update dupradar test snapshot (#5966) * Update dupradar test snapshot * Dummy commit to try and trigger nf-test in ci * Revert "Dummy commit to try and trigger nf-test in ci" This reverts commit 130f998. * stubs and test for rnaseq modules - part IV (#5939) * fix stubs and add stub tests * fix linting * fix some linting * fix linting * better comment * change container * update snapshots * improve untar stubs * update snapshots * update snapshots * update env * better * use seqera containers * quay.io * proper usage * update snapshots * keep ubuntu for now * update tests and snap * update test and snapshot * update test and snapshot * Add new input to multiqc module for use with --rename_samples (#5973) * Add new input to multiqc module for use with --rename_samples * Update tests * update meta.yml * Add provision for sample_names * port `gridss/gridss` to nf-test (#5933) * port * remove pytest files * fix meta * Update meta.yml * fix indentation * update test with bam plugin * fix indents, try other test * remove --assembly flag for now --------- Co-authored-by: Maxime U Garcia <[email protected]> * by remove the 2>&1 to avoid the warning message. (#5974) * chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.5.2 (#5975) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Handle empty files (#5720) * fix: emit all fastqs, remove global log file * fix: remove rg parseing * fix(subworkflow/bcl_demultiplex): remove rg parsing, output empty fastqs, remove log file * test: update snaps * lint: traill\ing whitespace * feat: include read group parsing and empty file check in single closure * test: update snapshot * fix: lint final new line * fix: lint trailing whitespace * fix: add branch and emit empty fastq channel * tests: update snaps * lint: trailing whitespace * fix: filter bool and add test * test: update snaps * test: assert empty file exist * test: assert empty file exist * test: check file exists, update snap * Update test.yml * Update test.yml * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * ci: debug nf-test failure * lint: fix lint --------- Co-authored-by: [email protected] <--unset> Co-authored-by: Simon Pearce <[email protected]> * new module: custom_gtffilter (#5965) * Add basic gtffilter implementation * Add gtffilter stub * Add gtffilter meta map * Update gtffilter meta.yml * Fix gtffilter formatting * Add first gtffilter tests * Fix problems in gtffilter module defintion * Add gtffilter nf-test snapshot * Add support for gzipped inputs to gtffilter * Add gtffilter gzip stub * Update nf-test definition for gtffilter * Add meta2 to fasta input Co-authored-by: Nicolas Vannieuwkerke <[email protected]> * Add fasta meta map to gtffilter test definitions * Add gtffilter MIT license text --------- Co-authored-by: Nicolas Vannieuwkerke <[email protected]> * Fix gtffilter output definition (#5977) * Added agat/spmergeannotations (#5772) * Added agat/spmergeannotations * Added tests with config * Rename preprocess rnaseq (#5959) * Edit content with subworkflow name * Rename subworkflow folder * Readd subworkflow with old name, with deprecation * remove rogue word * Appease eclint * Remove test suite for deprecated subworkflow * Using Seqera containers for Sentieon to avoid LD_LIBRARY errors (#5823) * Using seqera containers for sentieon * update containers for sentieon modules with quay.io registry --------- Co-authored-by: Simon Pearce <[email protected]> Co-authored-by: Adam Talbot <[email protected]> Co-authored-by: maxulysse <[email protected]> * Swap unzip to nf-test (continuation of 5357) (#5897) * unzip stub, nf-test, snap * Revert "unzip stub, nf-test, snap" This reverts commit dec35aa. * Reapply "unzip stub, nf-test, snap" This reverts commit e0d708e. * added minimal required tests for process and stub * added snapshot * simplify snapshot input in stub test Co-authored-by: Matthias Hörtenhuber <[email protected]> * remove unneeded stub files * migrate pytest for unzipfiles * add snapshot file for script block * added stub * added stub and stub test * updated stub snapshot for unzip * remove unneeded test config * Update snaps and malt * Update malt for linting * Fix linting for maltextract * Swap test-data paths for malt modules * Update modules/nf-core/maltextract/tests/main.nf.test --------- Co-authored-by: Sateesh <[email protected]> Co-authored-by: Jenny Leopoldina Smith <[email protected]> Co-authored-by: Matthias Hörtenhuber <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]> * Improve strandedness derivation in rnaseq preprocessing swf (#5982) * Improve strandedness devivation * Add missing params to nf-test * RNAseq preprocessing: rrna sequences make more sense as a channel of FASTAs, port tests (#5988) * rrna sequences make more sense as a channel of FASTAs * Also update tests * Add function test to swf * Fix up subworkflow function testing * [Upgrade] Normalize input of lofreq-viterbi to match other lofreq modules (#5989) * add meta2 to input fasta channel * update snapshot; nothing new but timestamp * Update modules/nf-core/lofreq/viterbi/tests/main.nf.test Co-authored-by: Felix Lenner <[email protected]> * Update modules/nf-core/lofreq/viterbi/tests/main.nf.test Co-authored-by: Felix Lenner <[email protected]> * Update modules/nf-core/lofreq/viterbi/tests/main.nf.test Co-authored-by: Felix Lenner <[email protected]> --------- Co-authored-by: Felix Lenner <[email protected]> * Add module fgbio/collectduplexseqmetrics (#5960) * Initial commit * Rerun nf-test * Change assertion for duplex_qc output * Add ggplot2 to version.yml * Update snapshot * Correct conda packages version * Add Seqfu/derep (#5958) * add-usearch-unoise3 * add seqfu/derep * Delete modules/nf-core/usearch/unoise3/environment.yml * Delete modules/nf-core/usearch/unoise3/main.nf * remove files * update metadata * Update modules/nf-core/seqfu/derep/main.nf Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/seqfu/derep/main.nf Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/seqfu/derep/main.nf Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/seqfu/derep/main.nf Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/seqfu/derep/main.nf Co-authored-by: Simon Pearce <[email protected]> * Update modules/nf-core/seqfu/derep/tests/main.nf.test Co-authored-by: Simon Pearce <[email protected]> * Fix dataset in -stub * Clarify description * Update modules/nf-core/seqfu/derep/meta.yml Co-authored-by: Simon Pearce <[email protected]> * check for pre-existing output file && update test * Update main.nf --------- Co-authored-by: Simon Pearce <[email protected]> * modify valCutoff funtion in propr.R (#5995) Co-authored-by: Cristina Araiz <[email protected]> * liftoff: Re-added conda environment (#5981) * liftoff: Readded conda environment * Added liftoff to env file * Cleaned up env file * chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.5.3 (#5996) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update: `CIRCULARMAPPER_CIRCULARGENERATOR` (#5997) * Fix output naming, add req argument to input * add new input to nf-test * update nf-test due to changes in upstream module inputs * update snapshots * add metas to other input channels * update tests according to changes in module inputs. * Apply suggestions from code review Co-authored-by: James A. Fellows Yates <[email protected]> * rename elongation factor for consistency with other CM module --------- Co-authored-by: James A. Fellows Yates <[email protected]> * Update: `CIRCULARMAPPER_REALIGNSAMFILE` (#5998) * Fix output naming, add req argument to input * add new input to nf-test * update nf-test due to changes in upstream module inputs * update snapshots * add metas to other input channels * update tests according to changes in module inputs. * add meta to elongation factor channel for consistency with other CM module * update nf-test to include new meta * rename metas * Add stub and test to lofreq/indelqual (#5941) * Add stub and test to lofreq/indelqual * Add author * new test setup using --migrate-pytest and PR feedback * Update main.nf.test * Update meta.yml * Update main.nf * add snap; del pytest * Update meta.yml * Update main.nf.test * Update meta.yml * update test * minor fix in stub test * Update tabix and add threads (#5999) * Update tabix and add threads * fix linting * Remove problematic ifEmpty() from rnaseq preprocessing (#6000) * Refactor Sentieon (#5945) * refactor: Clean up the logic and let Sentieon figure it out * refactor: Remove Sentieon license server IP check * fix: Add a -n * Try accessing secrets in the module? * fix: Use Nextflow for if statement * update snapshot * update snapshot * extend logic to all main.nf * ci: Add Sentieon logic to pytest-workflow * test(sentieon): Add env section to every config * Revert "ci: Add Sentieon logic to pytest-workflow" This reverts commit 76884d2. * test: Remove pytest_modules for sentieon --------- Co-authored-by: maxulysse <[email protected]> * ltrharvest: Added test to verify graceful handling of no ltr input (#5978) * ltrharvest: Added test to verify graceful handling of no ltr input * Updated test data paths * Fixed test data path * Removed unstable items from snapshot * Updated snapshotting logic * ltrfinder: Added test to verify graceful handling of no ltr input (#5979) * ltrretriever/ltrretriever: Added test to verify graceful handling of … (#5980) * ltrretriever/ltrretriever: Added test to verify graceful handling of no ltr input * Removed unstable items from snapshot * update nf-test t0 0.9.0 (#6010) * Fix keyword meta (#6011) * Updated mkfastq module (#5984) * Updated mkfastq module * Combined input channels * Updated main.nf.test and snap --------- Co-authored-by: zxBIB Schcolnicov <[email protected]> Co-authored-by: Anabella Trigila <[email protected]> * bulk update of meta.yml modules structure * update meta.yml files automatically with manual curation for missing information --------- Co-authored-by: James A. Fellows Yates <[email protected]> Co-authored-by: Carson J Miller <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]> Co-authored-by: nf-core-bot <[email protected]> Co-authored-by: Ramprasad Neethiraj <[email protected]> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Priyanka Surana <[email protected]> Co-authored-by: Matthieu Muffato <[email protected]> Co-authored-by: Aida Andrades Valtueña <[email protected]> Co-authored-by: Simon Pearce <[email protected]> Co-authored-by: Usman Rashid <[email protected]> Co-authored-by: Anders Sune Pedersen <[email protected]> Co-authored-by: Maxime U Garcia <[email protected]> Co-authored-by: Felix Lenner <[email protected]> Co-authored-by: Christian Heide <[email protected]> Co-authored-by: Famke Bäuerle <[email protected]> Co-authored-by: Raquel Manzano <[email protected]> Co-authored-by: Matthias De Smet <[email protected]> Co-authored-by: Jonathan Manning <[email protected]> Co-authored-by: Maxime Borry <[email protected]> Co-authored-by: vickylaram <[email protected]> Co-authored-by: Tyler Chafin <[email protected]> Co-authored-by: Thanh Lee <[email protected]> Co-authored-by: Joon Klaps <[email protected]> Co-authored-by: John Palmer <[email protected]> Co-authored-by: John Palmer <[email protected]> Co-authored-by: Charles Plessy <[email protected]> Co-authored-by: Mahesh Binzer-Panchal <[email protected]> Co-authored-by: Jose Espinosa-Carrasco <[email protected]> Co-authored-by: thomgiles <[email protected]> Co-authored-by: Jonathan Manning <[email protected]> Co-authored-by: Matthieu Muffato <[email protected]> Co-authored-by: Edmund Miller <[email protected]> Co-authored-by: Felix Lenner <[email protected]> Co-authored-by: paulwolk <[email protected]> Co-authored-by: Lili Andersson-Li <[email protected]> Co-authored-by: Evangelos Karatzas <[email protected]> Co-authored-by: Sateesh_Peri <[email protected]> Co-authored-by: Zohaib Anwar <[email protected]> Co-authored-by: Gisela Gabernet <[email protected]> Co-authored-by: Sateesh <[email protected]> Co-authored-by: Nils Homer <[email protected]> Co-authored-by: Nicolas Vannieuwkerke <[email protected]> Co-authored-by: RuthEberhardt <[email protected]> Co-authored-by: Theresa Störiko <[email protected]> Co-authored-by: Theresa Störiko <[email protected]> Co-authored-by: eolaniru <[email protected]> Co-authored-by: poddarharsh15 <[email protected]> Co-authored-by: Vedanth Ramji <[email protected]> Co-authored-by: Edmund Miller <[email protected]> Co-authored-by: maxulysse <[email protected]> Co-authored-by: Sofia Stamouli <[email protected]> Co-authored-by: Mohammed Mahdi <[email protected]> Co-authored-by: AitorPeseta <[email protected]> Co-authored-by: AitorOP <[email protected]> Co-authored-by: Francisco Martínez <[email protected]> Co-authored-by: Famke Bäuerle <[email protected]> Co-authored-by: Matthias Zepper <[email protected]> Co-authored-by: Matthias Zepper <[email protected]> Co-authored-by: Louis LE NEZET <[email protected]> Co-authored-by: LouisLeNezet <[email protected]> Co-authored-by: JIANHONG OU <[email protected]> Co-authored-by: Kyle Hazen <[email protected]> Co-authored-by: Nico Trummer <[email protected]> Co-authored-by: Adam Talbot <[email protected]> Co-authored-by: Jenny Leopoldina Smith <[email protected]> Co-authored-by: Matthias Hörtenhuber <[email protected]> Co-authored-by: Thomas Krannich <[email protected]> Co-authored-by: Georgia Kesisoglou <[email protected]> Co-authored-by: Andrea Telatin <[email protected]> Co-authored-by: caraiz2001 <[email protected]> Co-authored-by: Cristina Araiz <[email protected]> Co-authored-by: Thiseas C. Lamnidis <[email protected]> Co-authored-by: Nicolás Schcolnicov <[email protected]> Co-authored-by: zxBIB Schcolnicov <[email protected]> Co-authored-by: Anabella Trigila <[email protected]>

Convert callduplex and callmolecular to nftest and update version

d6cc2b3

SPPearce requested review from sruthipsuresh, lescai and a team as code owners May 17, 2024 06:14

SPPearce requested review from louperelo and nh13 and removed request for a team May 17, 2024 06:14

Convert zipperbams

01bd4c3

SPPearce mentioned this pull request May 21, 2024

Release 1.0.0 nf-core/fastquorum#42

Merged

10 tasks

SPPearce added 6 commits May 21, 2024 05:31

Convert groupreadsbyumi

3fc5a08

Convert sortbam and add stubs

5ec0a95

Merge branch 'master' into fgbio_nftest

3d38cc5

Convert filterconsensusreads

0bfe648

Update snapshot

95ed61f

Merge branch 'master' into fgbio_nftest

b1e1beb

SPPearce added the Ready for Review label May 21, 2024

Merge branch 'master' into fgbio_nftest

7b18f7b

nvnieuwk reviewed May 21, 2024

View reviewed changes

modules/nf-core/fgbio/callduplexconsensusreads/tests/main.nf.test Outdated Show resolved Hide resolved

nvnieuwk reviewed May 21, 2024

View reviewed changes

modules/nf-core/fgbio/groupreadsbyumi/main.nf Show resolved Hide resolved

nvnieuwk reviewed May 21, 2024

View reviewed changes

SPPearce commented May 21, 2024

View reviewed changes

modules/nf-core/fgbio/callmolecularconsensusreads/main.nf Outdated Show resolved Hide resolved

SPPearce added 2 commits May 21, 2024 09:57

Update modules/nf-core/fgbio/callmolecularconsensusreads/main.nf

c3b5a1a

Add collision checks, swap test data paths

49922ba

SPPearce requested a review from matthdsm as a code owner May 21, 2024 11:09

Merge branch 'master' into fgbio_nftest

abb2311

nvnieuwk approved these changes May 21, 2024

View reviewed changes

Fix conda version and linting

8d1dff3

nh13 approved these changes May 22, 2024

View reviewed changes

SPPearce commented May 22, 2024

View reviewed changes

modules/nf-core/fgbio/callduplexconsensusreads/meta.yml Outdated Show resolved Hide resolved

Apply suggestions from code review

559fb11

Co-authored-by: Nils Homer <[email protected]>

nh13 mentioned this pull request May 23, 2024

Use updated fgbio modules in nf-core/modules nf-core/fastquorum#45

Open

maxulysse and others added 4 commits June 17, 2024 11:19

Merge branch 'master' into fgbio_nftest

b0589f1

Update modules/nf-core/fgbio/callmolecularconsensusreads/meta.yml

ceb8df1

Co-authored-by: Nils Homer <[email protected]>

Update duplex snap

4a99079

Merge branch 'master' into fgbio_nftest

44458bc

SPPearce enabled auto-merge July 2, 2024 18:05

SPPearce added this pull request to the merge queue Jul 2, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 2, 2024

SPPearce added this pull request to the merge queue Jul 2, 2024

Merged via the queue into master with commit 2fc7438 Jul 2, 2024
36 checks passed

SPPearce deleted the fgbio_nftest branch July 2, 2024 19:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump fgbio versions and swap to nftest #5624

Bump fgbio versions and swap to nftest #5624

SPPearce commented May 17, 2024 •

edited

Loading

nvnieuwk May 21, 2024

SPPearce May 21, 2024

nvnieuwk May 21, 2024

SPPearce May 21, 2024

nvnieuwk May 21, 2024

nvnieuwk left a comment

nvnieuwk left a comment

SPPearce commented May 21, 2024

nh13 left a comment

nh13 May 22, 2024

	prefix = task.ext.prefix ?: "${meta.id}_consensus"
	prefix = task.ext.prefix ?: "${meta.id} _consensus_unmapped"

Bump fgbio versions and swap to nftest #5624

Bump fgbio versions and swap to nftest #5624

Conversation

SPPearce commented May 17, 2024 • edited Loading

nvnieuwk May 21, 2024

Choose a reason for hiding this comment

SPPearce May 21, 2024

Choose a reason for hiding this comment

nvnieuwk May 21, 2024

Choose a reason for hiding this comment

SPPearce May 21, 2024

Choose a reason for hiding this comment

nvnieuwk May 21, 2024

Choose a reason for hiding this comment

nvnieuwk left a comment

Choose a reason for hiding this comment

nvnieuwk left a comment

Choose a reason for hiding this comment

SPPearce commented May 21, 2024

nh13 left a comment

Choose a reason for hiding this comment

nh13 May 22, 2024

Choose a reason for hiding this comment

SPPearce commented May 17, 2024 •

edited

Loading