Add validation for fasta input and tests #20

JoseEspinosa · 2024-05-08T16:30:03Z

Validate fasta input.
Branching samplesheet into samplesheet_query and samplesheet_fasta and refactor code accordingly.
Add test for fasta, run with -profile test_fasta.
Run test_fasta on CI.
Other minor fixes.

Closes #17

PR checklist

This comment contains a description of changes (with reason).
If you've fixed a bug or added code that should be tested, add tests!
Make sure your code lints (nf-core lint).
Ensure the test suite passes (nf-test test main.nf.test -profile test,docker).
Usage Documentation in docs/usage.md is updated.
README.md is updated (including new tool citations and authors/contributors).

github-actions · 2024-05-08T16:31:35Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit cc50269

+| ✅ 216 tests passed       |+
!| ❗  13 tests had warnings |!

❗ Test warnings:

files_exist - File not found: conf/igenomes.config
readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
pipeline_todos - TODO string in base.config: Check the defaults for all processes
pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
pipeline_todos - TODO string in ci.yml: You can customise CI pipeline run tests as required
pipeline_todos - TODO string in ci.yml: You can customise CI pipeline run tests as required
pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-reportho_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-reportho_logo_light.png
files_exist - File found: docs/images/nf-core-reportho_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-reportho_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowReportho.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 1.0dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.use_all= false
nextflow_config - Config default value correct: params.local_databases= false
nextflow_config - Config default value correct: params.skip_oma= false
nextflow_config - Config default value correct: params.skip_panther= false
nextflow_config - Config default value correct: params.skip_orthoinspector= false
nextflow_config - Config default value correct: params.orthoinspector_version= Eukaryota2023
nextflow_config - Config default value correct: params.skip_eggnog= false
nextflow_config - Config default value correct: params.use_centroid= false
nextflow_config - Config default value correct: params.min_score= 2.0
nextflow_config - Config default value correct: params.skip_orthoplots= false
nextflow_config - Config default value correct: params.skip_downstream= false
nextflow_config - Config default value correct: params.skip_report= false
nextflow_config - Config default value correct: params.use_structures= false
nextflow_config - Config default value correct: params.skip_iqtree= false
nextflow_config - Config default value correct: params.skip_fastme= false
nextflow_config - Config default value correct: params.iqtree_bootstrap= 1000
nextflow_config - Config default value correct: params.fastme_bootstrap= 100
nextflow_config - Config default value correct: params.skip_treeplots= false
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.max_cpus= 16
nextflow_config - Config default value correct: params.max_memory= 128.GB
nextflow_config - Config default value correct: params.max_time= 240.h
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-reportho_logo_light.png matches the template
files_unchanged - docs/images/nf-core-reportho_logo_light.png matches the template
files_unchanged - docs/images/nf-core-reportho_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (182 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
modules_config - conf/modules.config found and not ignored.
modules_config - IDENTIFY_SEQ_ONLINE found in conf/modules.config and Nextflow scripts.
modules_config - FETCH_OMA_GROUP_LOCAL found in conf/modules.config and Nextflow scripts.
modules_config - FETCH_PANTHER_GROUP_LOCAL found in conf/modules.config and Nextflow scripts.
modules_config - FETCH_INSPECTOR_GROUP_ONLINE found in conf/modules.config and Nextflow scripts.
modules_config - FETCH_EGGNOG_GROUP_LOCAL found in conf/modules.config and Nextflow scripts.
modules_config - MERGE_CSV found in conf/modules.config and Nextflow scripts.
modules_config - MAKE_SCORE_TABLE found in conf/modules.config and Nextflow scripts.
modules_config - FILTER_HITS found in conf/modules.config and Nextflow scripts.
modules_config - PLOT_ORTHOLOGS found in conf/modules.config and Nextflow scripts.
modules_config - MAKE_HITS_TABLE found in conf/modules.config and Nextflow scripts.
modules_config - MERGE_HITS found in conf/modules.config and Nextflow scripts.
modules_config - MAKE_STATS found in conf/modules.config and Nextflow scripts.
modules_config - STATS2CSV found in conf/modules.config and Nextflow scripts.
modules_config - MERGE_STATS found in conf/modules.config and Nextflow scripts.
modules_config - FETCH_SEQUENCES_ONLINE found in conf/modules.config and Nextflow scripts.
modules_config - FETCH_AFDB_STRUCTURES found in conf/modules.config and Nextflow scripts.
modules_config - FILTER_FASTA found in conf/modules.config and Nextflow scripts.
modules_config - CREATE_TCOFFEETEMPLATE found in conf/modules.config and Nextflow scripts.
modules_config - TCOFFEE_ALIGN found in conf/modules.config and Nextflow scripts.
modules_config - CONVERT_PHYLIP found in conf/modules.config and Nextflow scripts.
modules_config - IQTREE found in conf/modules.config and Nextflow scripts.
modules_config - PLOT_IQTREE found in conf/modules.config and Nextflow scripts.
modules_config - FASTME found in conf/modules.config and Nextflow scripts.
modules_config - PLOT_FASTME found in conf/modules.config and Nextflow scripts.
modules_config - CONVERT_FASTA found in conf/modules.config and Nextflow scripts.
modules_config - DUMP_PARAMS found in conf/modules.config and Nextflow scripts.
modules_config - MAKE_REPORT found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 2.14.1

Run details

nf-core/tools version 2.14.1
Run at 2024-05-13 09:57:38

bin/fetch_oma_by_sequence.py

itrujnara

The code looks good, hopefully it is tested as I have very little knowledge of nf-schema. Should we not update the readme and usage to reflect the changes in the input format?

…to updates

itrujnara

Looks good, just some very minor comments

README.md

itrujnara · 2024-05-13T08:37:59Z

docs/usage.md

+| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `id`    | User-defined identifier. It is used to identify output files for the protein. Can be anything descriptive, as long as it does not contain spaces. |
+| `query` | The query of the user-specified type. It should be a valid Uniprot accession.                                                                     |
+| `fasta` | It should be a valid path to a FASTA file.                                                                                                        |

 An [example samplesheet](../assets/samplesheet.csv) has been provided with the pipeline.


Do we want an example for FASTA samplesheet as well?

It is changed in line 33, or are you referring to something else?

Line 43 mentions a separate samplesheet file, I wonder if we need a similar one for FASTA. Just a question to consider, not necessarily demanding it.

Done in 195ab8d

Co-authored-by: Igor Trujnara <[email protected]>

itrujnara

Final tiny tweak

docs/usage.md

itrujnara

LGTM

JoseEspinosa added 6 commits May 8, 2024 18:22

Add validation for fasta files

4aca65a

Fix format

be8b290

Add test_fasta profile

0536976

Avoid error if dict key not set

fe1ca80

Get rid of parameter from module

38bd3ee

Merge remote-tracking branch 'upstream/dev' into updates

a47e239

JoseEspinosa added 2 commits May 8, 2024 18:35

Make lint happy

7c48293

Merge branch 'dev' into updates

cf6688b

JoseEspinosa commented May 9, 2024

View reviewed changes

bin/fetch_oma_by_sequence.py Outdated Show resolved Hide resolved

Update bin/fetch_oma_by_sequence.py

ca19068

JoseEspinosa commented May 9, 2024

View reviewed changes

bin/fetch_oma_by_sequence.py Outdated Show resolved Hide resolved

Update bin/fetch_oma_by_sequence.py

2f2eaf0

itrujnara reviewed May 9, 2024

View reviewed changes

JoseEspinosa added 14 commits May 9, 2024 17:59

Merge remote-tracking branch 'upstream/dev' into updates

0f1ef99

Merge branch 'updates' of https://github.com/JoseEspinosa/reportho in…

36d3769

…to updates

Branch depending on whether uniprot_id or fasta provided

15aa8f6

Update tests

bb04c56

Get rid of leftovers of the uniprot_query parameter

a376ae3

Update docs

f1f25bd

Do not use set for ch_versions

b899246

Add test_fasta to CI

3f472ad

Make nf-core lint happy

72f80aa

Make prettier happy

3490c4c

Make nf-core lint happy (bug in tools until fixed)

8fe4f82

Revert changes in .nf-core.yml

b13725c

Merge remote-tracking branch 'upstream/dev' into updates

320ec61

Assign ch_versions

6d270ea

itrujnara reviewed May 13, 2024

View reviewed changes

Fix tyop

b849f92

Co-authored-by: Igor Trujnara <[email protected]>

Add samplesheet_fasta in assets

195ab8d

itrujnara reviewed May 13, 2024

View reviewed changes

docs/usage.md Outdated Show resolved Hide resolved

JoseEspinosa and others added 2 commits May 13, 2024 11:56

Rename samplesheet_fasta example

f276a8e

Update docs/usage.md

cc50269

itrujnara approved these changes May 13, 2024

View reviewed changes

JoseEspinosa merged commit 8ba2862 into nf-core:dev May 13, 2024
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add validation for fasta input and tests #20

Add validation for fasta input and tests #20

JoseEspinosa commented May 8, 2024 •

edited

Loading

github-actions bot commented May 8, 2024 •

edited

Loading

❗ Test warnings:

✅ Tests passed:

Run details

itrujnara left a comment

itrujnara left a comment

itrujnara May 13, 2024

JoseEspinosa May 13, 2024

itrujnara May 13, 2024

JoseEspinosa May 13, 2024

itrujnara left a comment

itrujnara left a comment

Add validation for fasta input and tests #20

Add validation for fasta input and tests #20

Conversation

JoseEspinosa commented May 8, 2024 • edited Loading

PR checklist

github-actions bot commented May 8, 2024 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

✅ Tests passed:

Run details

itrujnara left a comment

Choose a reason for hiding this comment

itrujnara left a comment

Choose a reason for hiding this comment

itrujnara May 13, 2024

Choose a reason for hiding this comment

JoseEspinosa May 13, 2024

Choose a reason for hiding this comment

itrujnara May 13, 2024

Choose a reason for hiding this comment

JoseEspinosa May 13, 2024

Choose a reason for hiding this comment

itrujnara left a comment

Choose a reason for hiding this comment

itrujnara left a comment

Choose a reason for hiding this comment

JoseEspinosa commented May 8, 2024 •

edited

Loading

github-actions bot commented May 8, 2024 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️