Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add dsl2 module for deepvariant #394

Merged
merged 8 commits into from
Jul 15, 2021
Merged

Add dsl2 module for deepvariant #394

merged 8 commits into from
Jul 15, 2021

Conversation

abhi18av
Copy link
Member

@abhi18av abhi18av commented Jul 6, 2021 •
edited
Loading

Hi team,

I'm initiating the draft PR for the integration of deepvariant tool, we had a discussion a while back in the nf-core's #deepvariant channel regarding this but for I couldn't find the time to take it forward then. Now, I find myself with some time to finally see this through.

I'll update the checklist once I have finalized the code and it's ready for a wider/final review. In the meantime, I've marked this PR as a draft to have a placeholder for early discussions

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
    • If you've added a new tool - add to the software_versions process and a regex to scrape_software_versions.py
    • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
    • If necessary, also make a PR on the nf-core/sarek branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint .).
  • Ensure the test suite passes (nextflow run . -profile test,docker).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@abhi18av abhi18av self-assigned this Jul 6, 2021
@github-actions
Copy link

github-actions bot commented Jul 6, 2021 •
edited
Loading

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit f038a67

+| ✅ 135 tests passed       |+
#| ❔  11 tests were ignored |#
!| ❗  84 tests had warnings |!

❗ Test warnings:

  • files_exist - File not found: environment.yml
  • files_exist - File not found: Dockerfile
  • nextflow_config - Config variable not found: process.container
  • params_used - Config variable not found in main.nf: params.input
  • params_used - Config variable not found in main.nf: params.step
  • params_used - Config variable not found in main.nf: params.genome
  • params_used - Config variable not found in main.nf: params.genomes_base
  • params_used - Config variable not found in main.nf: params.save_reference
  • params_used - Config variable not found in main.nf: params.help
  • params_used - Config variable not found in main.nf: params.no_intervals
  • params_used - Config variable not found in main.nf: params.nucleotides_per_second
  • params_used - Config variable not found in main.nf: params.sentieon
  • params_used - Config variable not found in main.nf: params.skip_qc
  • params_used - Config variable not found in main.nf: params.target_bed
  • params_used - Config variable not found in main.nf: params.tools
  • params_used - Config variable not found in main.nf: params.trim_fastq
  • params_used - Config variable not found in main.nf: params.clip_r1
  • params_used - Config variable not found in main.nf: params.clip_r2
  • params_used - Config variable not found in main.nf: params.three_prime_clip_r1
  • params_used - Config variable not found in main.nf: params.three_prime_clip_r2
  • params_used - Config variable not found in main.nf: params.trim_nextseq
  • params_used - Config variable not found in main.nf: params.save_trimmed
  • params_used - Config variable not found in main.nf: params.split_fastq
  • params_used - Config variable not found in main.nf: params.aligner
  • params_used - Config variable not found in main.nf: params.markdup_java_options
  • params_used - Config variable not found in main.nf: params.use_gatk_spark
  • params_used - Config variable not found in main.nf: params.save_bam_mapped
  • params_used - Config variable not found in main.nf: params.skip_markduplicates
  • params_used - Config variable not found in main.nf: params.ascat_ploidy
  • params_used - Config variable not found in main.nf: params.ascat_purity
  • params_used - Config variable not found in main.nf: params.cf_coeff
  • params_used - Config variable not found in main.nf: params.cf_contamination
  • params_used - Config variable not found in main.nf: params.cf_contamination_adjustment
  • params_used - Config variable not found in main.nf: params.cf_ploidy
  • params_used - Config variable not found in main.nf: params.cf_window
  • params_used - Config variable not found in main.nf: params.generate_gvcf
  • params_used - Config variable not found in main.nf: params.no_strelka_bp
  • params_used - Config variable not found in main.nf: params.pon
  • params_used - Config variable not found in main.nf: params.pon_index
  • params_used - Config variable not found in main.nf: params.ignore_soft_clipped_bases
  • params_used - Config variable not found in main.nf: params.umi
  • params_used - Config variable not found in main.nf: params.read_structure1
  • params_used - Config variable not found in main.nf: params.read_structure2
  • params_used - Config variable not found in main.nf: params.annotate_tools
  • params_used - Config variable not found in main.nf: params.annotation_cache
  • params_used - Config variable not found in main.nf: params.cadd_cache
  • params_used - Config variable not found in main.nf: params.cadd_indels
  • params_used - Config variable not found in main.nf: params.cadd_indels_tbi
  • params_used - Config variable not found in main.nf: params.cadd_wg_snvs
  • params_used - Config variable not found in main.nf: params.cadd_wg_snvs_tbi
  • params_used - Config variable not found in main.nf: params.genesplicer
  • params_used - Config variable not found in main.nf: params.snpeff_cache
  • params_used - Config variable not found in main.nf: params.config_profile_contact
  • params_used - Config variable not found in main.nf: params.config_profile_description
  • params_used - Config variable not found in main.nf: params.config_profile_url
  • params_used - Config variable not found in main.nf: params.outdir
  • params_used - Config variable not found in main.nf: params.publish_dir_mode
  • params_used - Config variable not found in main.nf: params.sequencing_center
  • params_used - Config variable not found in main.nf: params.multiqc_config
  • params_used - Config variable not found in main.nf: params.monochrome_logs
  • params_used - Config variable not found in main.nf: params.email
  • params_used - Config variable not found in main.nf: params.email_on_fail
  • params_used - Config variable not found in main.nf: params.plaintext_email
  • params_used - Config variable not found in main.nf: params.max_multiqc_email_size
  • params_used - Config variable not found in main.nf: params.hostnames
  • params_used - Config variable not found in main.nf: params.validate_params
  • params_used - Config variable not found in main.nf: params.tracedir
  • params_used - Config variable not found in main.nf: params.enable_conda
  • params_used - Config variable not found in main.nf: params.pull_docker_container
  • params_used - Config variable not found in main.nf: params.cpus
  • params_used - Config variable not found in main.nf: params.max_cpus
  • params_used - Config variable not found in main.nf: params.max_memory
  • params_used - Config variable not found in main.nf: params.max_time
  • actions_awsfulltest - .github/workflows/awsfulltest.yml should test full datasets, not -profile test
  • readme - README did not have a Nextflow minimum version badge.
  • pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
  • pipeline_todos - TODO string in awstest.yml: You can customise CI pipeline run tests as required
  • pipeline_todos - TODO string in main.nf: It MUST be possible to pass additional parameters to the tool as a command-line string via the "$ioptions.args" variable
  • pipeline_todos - TODO string in main.nf: If the tool supports multi-threading then you MUST provide the appropriate parameter
  • pipeline_todos - TODO string in main.nf: A module file SHOULD only define input and output files as command-line parameters.
  • pipeline_todos - TODO string in main.nf: Where applicable all sample-specific information e.g. "id", "single_end", "read_group"
  • pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
  • pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
  • schema_description - No description provided in schema for parameter: cpus

❔ Tests ignored:

  • files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.md
  • files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/feature_request.md
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/nf-core-sarek_logo.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo.png
  • files_unchanged - File ignored due to lint config: lib/NfcoreSchema.groovy
  • files_unchanged - File ignored due to lint config: .gitignore or foo
  • files_unchanged - File does not exist: .github/workflows/push_dockerhub_dev.yml
  • files_unchanged - File does not exist: .github/workflows/push_dockerhub_release.yml
  • conda_env_yaml - No environment.yml file found - skipping conda_env_yaml test
  • conda_dockerfile - No environment.yml / Dockerfile file found - skipping conda_dockerfile test

✅ Tests passed:

Run details

  • nf-core/tools version 1.14
  • Run at 2021-07-07 08:22:38

This was referenced Jul 6, 2021
Open
Closed
abhi18av
abhi18av commented Jul 7, 2021
View reviewed changes
Copy link
Member Author

@abhi18av abhi18av left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added some comments to confirm certain design decisions and request feedback.

I'm a bit confused about how to use the options.args to pass arbitrary args to the workflow, could you please point me to the right direction?

Also, is there a way to skip triggering the CI for these interim changes?

subworkflows/local/germline_variant_calling.nf Show resolved Hide resolved
subworkflows/local/germline_variant_calling.nf Show resolved Hide resolved
modules/local/deepvariant/functions.nf Show resolved Hide resolved
@maxulysse
Copy link
Member

I've added some comments to confirm certain design decisions and request feedback.

I'm a bit confused about how to use the options.args to pass arbitrary args to the workflow, could you please point me to the right direction?

You need to add the args (args2 or even args3) to https://github.com/nf-core/sarek/blob/dsl2/conf/modules.config

Also, is there a way to skip triggering the CI for these interim changes?

Add skip ci to your commit:

https://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/

@abhi18av abhi18av marked this pull request as ready for review July 12, 2021 11:26
@abhi18av
Copy link
Member Author

abhi18av commented Jul 12, 2021 •
edited
Loading

Hi @maxulysse , I've now opened up the PR for review

  • implemented the basic module
  • intergrated the settings into modules.config
  • integrated the module into germline_variant_caller.nf subworkflow

Questions:

  1. Regarding the testing, how should I test the variant calling steps? nextflow run . -profile test,docker doesn't seem to invoke the variant calling steps.

  2. Would you like to see anyone of these outputs to be included in the output channels?

output.g.vcf.gz.tbi
output.vcf.gz.tbi
output.visual_report.html

Please let me know what further changes are necessary before this PR could be mergeable - apart from the things mentioned in the PR template checklist.

@abhi18av @maxulysse
Accomodate the code review
b446514 
Co-authored-by: Maxime U. Garcia <[email protected]>
@maxulysse maxulysse merged commit 18d5b61 into nf-core:dsl2 Jul 15, 2021
@abhi18av
Copy link
Member Author

Thanks @maxulysse ! 😍

@abhi18av abhi18av deleted the abhinav/deepvariant branch July 15, 2021 08:24
@abhi18av abhi18av mentioned this pull request Jul 22, 2021
Closed
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maxulysse maxulysse maxulysse approved these changes

Assignees

@abhi18av abhi18av

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@abhi18av @maxulysse