Azurized wdls (#1239)

* small change

* azurize optimus  (#1228)

* add logic to choose which docker

* fix param_meta and import

* add cloud provider to checkinput

* handle hard coded white list paths in CheckInputs.wdl

* last few dockers

* last few dockers

* last few dockers

* change error msg

* use ubuntu image

* use ubuntu image

* change whitelists

* point to azure public whitelists

* add sas token

* echo whitelist

* echo whitelist

* testing for coa

* testing for coa

* change back to terra buckets for whitelists

* change whitelists to point at public azure bucket

* files to strings

* print statemtns to checkinputs

* string to files

* change to terra bucket paths

* strings not files

* append sas token

* append sas token

* append sas  and use strings

* back to bucket urls

* back to bucket urls

* use google cloud urls

* using public urls

* trying to export sas_token

* trying to export sas_token

* trying to export sas_token

* terra on gcp

* update azure whitelist files

* changelogs

* changelogs

* changelogs

* changelogs

* fix some inputs

* fix some inputs

* fix some inputs

* fix some inputs

* update optimus dockers

* warp_tools_docker_path for staralign

* stop using ice lake as default

* update pipeline docs

* 2 threads

* counting mode

* changelogs

---------

Co-authored-by: phendriksen100 <[email protected]>
Co-authored-by: kayleemathews <[email protected]>

* Ph pd 2514 multiome on terra (#1237)

* ph logic to pass in docker images based on cloud provider

* determine which whitelist files to use

* update tests

* add parameter metadata

* add error handling in atac

* fix comment

* PR comments

* update image and add utils

* add import

---------

Co-authored-by: phendriksen100 <[email protected]>

* lost the docker_path in joinmultiomebarcode task

* update TestMultiome.wdl

* update cutadapt wdl

* update cutadapt wdl

* try to fix changelog

* try to fix changelog

* remove cloud provider

* try adding sas token to azure public bucket

* try adding sas token to azure public bucket

* try files not strings

* try files not strings

* remove cromwell root

* update docs

* quote whitelist

* quote whitelist

* dirs

* dirs

* dirs

* add quotes to whitelist

* mkdir cromwell_root

* try vm family

* try vm family

* Standard_M128s

* try not to use cromwell root

* try not to use cromwell root

* try using logic

* try using logic

* update snapatac2 docker

* remove mkdir cromwell root

* snap dpcker

* fix snap dpcker

* fix starsolo fastq for other flavors of optimus

* merge conflicts

* merge conflicts

* fix StarAlign.wdl

* put whitelists in quotes

* PD-2516: Update Paired-Tag to run in Azure and GCP (#1212)

* PD-2516: Update PairedTag to run in Azure and GCP

* json formatting

* update file location for GCP vs. Azure and documentation accordingly

* merge conflicts after rebase, update pipeline cahngelog and readme version

* more fixes after rebase

* more fixes after rebase

* more fixes after rebase

* fix readme

* adding sas tokens

* fixing womtools error

* update pipeline change logs and versions

---------

Co-authored-by: npetrill <[email protected]>
Co-authored-by: Nikelle Petrillo <[email protected]>

* Fk pd 2513 reblock gvcf (#1261)

* added azure docker and updated ReblockGVCF to support gcp and azure

* updated verification test wdl

* updated UltimaGenomicsWholeGenomeGermline wdl and changelog

* Updated Changelog on BroadInternalUltimaGenomics

* updated tasks to use the new docker

* updated additional changelogs

* changed path for import utils.wdl

* update other wdls because of qc wdl change

* update other wdls because of qc wdl change

* update test wdls

* still need to update many input jsons

* Update ExomeGermlineSingleSample.changelog.md

* Update UltimaGenomicsWholeGenomeGermline.changelog.md

---------

Co-authored-by: npetrill <[email protected]>
Co-authored-by: Nikelle Petrillo <[email protected]>

* edited wdl and changelog for ToA support

* added cloud provider input to test inputs json

* added util class to log error if unsupported cloud provider used for cloud_provider input

* added cloud provider parameter to test json

* refactored task inputs

* change to inputs

* change to docker path generation

* changes made to test wdl

* updated reference to docker images for consistency with other azurized wdls

* fix to docker image reference

* fix to docker prefix call

* updated var name

* replace hard-coded cromwell_root with variable based on cloud env

* wrap param name in quotations

* provide absolute path

* write out files using ls

* add print for working dir

* provide absolute paths for demultiplexing

* testing

* more testing

* more testing

* put things back

* add lots more logging

* only run cutadapt

* only run cutadapt

* write files to current working dir

* write files to current working dir

* write files to current working dir

* add some of demultiplexing steps back in

* typo

* typo

* typo

* remove print

* uncomment rest of workflow

* add working dir to batch subdir

* ls cromwell root batch

* ls directories to find batch dir

* update changelogs

* set batch dir

* remove extra leading slash

* fixing what i messed up in resvoling conflicts

* fix batch dir

* need to loop through array

* add lots of logging to batch logic

* add lots of logging to batch logic

* change output dir of bams

* fix path of fastq

* fix path of fastq

* fix fastq remove path

* add more echo statements

* add more echo statements again

* trigger update of wdl in workpace

* remove some logging

* fix logging error syntax

* add missing quote

* change working dir to batch dir

* add ls

* version change

* fix paths

* fix batch dir

* fix batch dir

* fix directories

* moving around inputs

* remove working dir from name of tar file

* take basenames

* recursively ls the root to find the batch dirs

* add echo statement

* added docker parameter for DragenTasks.CalibrateDragstrModel supporting azure

* added docker parameter for Utils.ScatterIntervalList to support azure

* added docker parameter for Calling.HaplotypeCaller_GATK4_VCF to support azure

* added docker parameter for Calling.HaplotypeCaller_GATK4_VCF to support azure

* added docker parameter to support azure

* add echo statement

* fix batch dir

* fix batch dir

* fix path to hisat index files

* add pipline_inputs subdir to location of index files

* add pipline_inputs subdir to location of index files again

* fix tar command

* some clean up, remove pipeline inputs subdir, edit single end task

* copy fa file again

* rename some inputs for paired tag demultiplexing

* rename some intermediate inputs for paired tag demultiplexing

* list cromwell root

* fix batch dir

* add conditional for input and output bams

* fix logic for cloud provider vs. crowell root dir

* add ls

* fix batch dir in gcp

* fix sytaxt error

* ugh remove echo statements referencing variable that doesnt exist anymore

* attempt to fix syntax error

* add missing forward slash

* fix directories, add ls commands

* add more ls commands

* add more ls commands

* add more ls commands again

* fix fasta file path

* changing docker inputs to allow for tests to pass against pipelines that have not yet been azurized

* add more logging, put in cromwell root for move rather than local dir

* fix to docker task inputs to allow default values

* add more logging, add more ls, try tarring files in current dir rather than previous subdir

* use correct syntax for verbose untarring

* fix to docker version

* set correct base dirs that are nested

* change to how cromwell root dir is set

* correction to docker image being used

* add debugging

* formatting fix

* add logic for base directory based on gcp vs azure

* added notes to affected changelogs

* updated changelog for WGS Single sample pipeline

* updated pipeline version to match changelong

* updated pipeline version to match changelong

* updated pipeline version to match changelong

* updated changelog and pipeline version

* updated changelog and pipeline version

* updated changelog and pipeline version

* updated changelog and pipeline version

* reverting snm3c changes

* added notes for SlideSeq and changelog for updated tasks

* remove logging and debugging statements

* just testing

* add input to test

* changelogs

* changelogs

* changelogs

* Np make vm size an input to multiome (#1289)

* add vm size as input to Multiome.wdl

* add vm size as input to Multiome.wdl

* add new input to overviews

* add new input to overviews

* add new input to overviews

* add new input to overviews

* more disk and mem

* Azurize Cell Bender in Multiome (#1299)

* update docker for Summary_PerCellOutput

* changelogs

* changelogs

* update dockers

* more changelogs ugh

* more changelogs ugh

* more changelogs ugh

* made minor updates, not patches

* pointing to pinned version of dockers or the sha

* extra space

* Apply suggestions from code review

Co-authored-by: ekiernan <[email protected]>

* changelog

* doc reformatting

* Update pipelines/broad/arrays/imputation/Imputation.changelog.md

* Update pipelines/skylab/slideseq/SlideSeq.changelog.md

Co-authored-by: ekiernan <[email protected]>

---------

Co-authored-by: phendriksen100 <[email protected]>
Co-authored-by: kayleemathews <[email protected]>
Co-authored-by: Nareh Sahakian <[email protected]>
Co-authored-by: Farzaneh Khajouei <[email protected]>
Co-authored-by: John Scira <[email protected]>
Co-authored-by: John-Scira <[email protected]>
Co-authored-by: Nareh Sahakian <[email protected]>
Co-authored-by: aawdeh <[email protected]>
Co-authored-by: ekiernan <[email protected]>
Co-authored-by: ekiernan <[email protected]>

pipelines/broad/arrays/imputation/Imputation.changelog.md

@@ -1,3 +1,9 @@
+# 1.1.13
+2024-05-21 (Date of Last Commit)
+
+* Updated GermlineVariantDiscovery, BamProcessing, DragenTasks, Qc, and Utilities tasks to allow multi-cloud dockers. This change does not affect this pipeline.
+
+
 # 1.1.12
 2023-12-18 (Date of Last Commit)
 

pipelines/broad/arrays/imputation/Imputation.wdl

@@ -6,7 +6,7 @@ import "../../../../tasks/broad/Utilities.wdl" as utils
 
 workflow Imputation {
 
-  String pipeline_version = "1.1.12"
+  String pipeline_version = "1.1.13"
 
   input {
     Int chunkLength = 25000000

pipelines/broad/arrays/single_sample/Arrays.changelog.md

@@ -1,3 +1,7 @@
+# 2.6.25
+2024-07-09
+* Updated tasks GermlineVariantDiscovery.wdl and QC.wdl to allow multi-cloud dockers; this does not affect this pipeline.
+
 # 2.6.24
 2024-07-01 (Date of Last Commit)
 

pipelines/broad/arrays/single_sample/Arrays.wdl

@@ -23,7 +23,7 @@ import "../../../../tasks/broad/Utilities.wdl" as utils
 
 workflow Arrays {
 
-  String pipeline_version = "2.6.24"
+  String pipeline_version = "2.6.25"
 
   input {
     String chip_well_barcode

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/ReblockGVCF.changelog.md

@@ -1,3 +1,8 @@
+# 2.2.0
+2024-07-09 (Date of Last Commit)
+
+* Updated ReblockGVCF.wdl to run in Azure. cloud_provider is a new, required input. 
+
 # 2.1.13
 2024-07-01 (Date of Last Commit)
 

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/ReblockGVCF.exome.inputs.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/ExomeGermlineSingleSample/truth/plumbing/master/RP-929.NA12878/NA12878_PLUMBING.rb.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider":"gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/ReblockGVCF.wdl

@@ -2,10 +2,11 @@ version 1.0
 
 import "../../../../../../tasks/broad/GermlineVariantDiscovery.wdl" as Calling
 import "../../../../../../tasks/broad/Qc.wdl" as QC
+import "../../../../../../tasks/broad/Utilities.wdl" as utils
 
 workflow ReblockGVCF {
 
-  String pipeline_version = "2.1.13"
+  String pipeline_version = "2.2.0"
 
 
   input {
@@ -20,9 +21,22 @@ workflow ReblockGVCF {
     String? annotations_to_remove_command
     Boolean? move_filters_to_genotypes
     String gvcf_file_extension = ".g.vcf.gz"
+    String cloud_provider
   }
 
   String gvcf_basename = basename(gvcf, gvcf_file_extension)
+  # docker images
+  String gatk_docker_gcp = "us.gcr.io/broad-gatk/gatk:4.5.0.0"
+  String gatk_docker_azure = "dsppipelinedev.azurecr.io/gatk_reduced_layers:1.0.0"
+  String gatk_docker = if cloud_provider == "gcp" then gatk_docker_gcp else gatk_docker_azure
+
+  # make sure either gcp or azr is supplied as cloud_provider input
+  if ((cloud_provider != "gcp") && (cloud_provider != "azure")) {
+    call utils.ErrorWithMessage as ErrorMessageIncorrectInput {
+      input:
+        message = "cloud_provider must be supplied with either 'gcp' or 'azure'."
+    }
+  }
 
   call Calling.Reblock as Reblock {
     input:
@@ -35,7 +49,8 @@ workflow ReblockGVCF {
       annotations_to_keep_command = annotations_to_keep_command,
       annotations_to_remove_command = annotations_to_remove_command,
       move_filters_to_genotypes = move_filters_to_genotypes,
-      output_vcf_filename = gvcf_basename + ".rb.g.vcf.gz"
+      output_vcf_filename = gvcf_basename + ".rb.g.vcf.gz",
+      docker_path = gatk_docker
   }
 
     # Validate the (g)VCF output of HaplotypeCaller
@@ -51,7 +66,7 @@ workflow ReblockGVCF {
         calling_intervals_defined = defined(calling_interval_list),
         is_gvcf = true,
         extra_args = "--no-overlaps",
-        gatk_docker = "us.gcr.io/broad-gatk/gatk:4.5.0.0"
+        docker_path = gatk_docker
     }
 
   output {

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Plumbing/G96830.NA12878.index.json

@@ -3,6 +3,7 @@
     "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/wgs/plumbing/input/G96830.NA12878/index_in_different_location/NA12878_PLUMBING.g.vcf.gz.tbi",
     "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
     "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-    "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+    "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+    "ReblockGVCF.cloud_provider": "gcp"
   }
 

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Plumbing/G96830.NA12878.json

@@ -4,5 +4,6 @@
   "ReblockGVCF.calling_interval_list": "gs://gcp-public-data--broad-references/hg38/v0/wgs_calling_regions.hg38.interval_list",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Plumbing/NA12878.ultima.json

@@ -6,5 +6,6 @@
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
   "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
   "ReblockGVCF.tree_score_cutoff": 0.2,
-  "ReblockGVCF.annotations_to_keep_command": "--annotations-to-keep TREE_SCORE --annotations-to-keep ASSEMBLED_HAPS --annotations-to-keep FILTERED_HAPS"
+  "ReblockGVCF.annotations_to_keep_command": "--annotations-to-keep TREE_SCORE --annotations-to-keep ASSEMBLED_HAPS --annotations-to-keep FILTERED_HAPS",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Plumbing/RP-929.NA12878.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/plumbing/input/RP-929.NA12878/NA12878_PLUMBING.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/C1963.CHMI_CHMI3_Nex1.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/C1963.CHMI_CHMI3_Nex1/CHMI_CHMI3_Nex1.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/C862.NA19238.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/C862.NA19238/NA19238.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/D5327.NA12878.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/D5327.NA12878/NA12878.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/D5327.NA12891.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/D5327.NA12891/NA12891.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/G94794.CHMI_CHMI3_WGS2.json

@@ -4,5 +4,6 @@
   "ReblockGVCF.calling_interval_list": "gs://gcp-public-data--broad-references/hg38/v0/wgs_calling_regions.hg38.interval_list",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/G94982.NA12878.json

@@ -4,5 +4,6 @@
   "ReblockGVCF.calling_interval_list": "gs://gcp-public-data--broad-references/hg38/v0/wgs_calling_regions.hg38.interval_list",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/G94982.NA12891.json

@@ -4,5 +4,6 @@
   "ReblockGVCF.calling_interval_list": "gs://gcp-public-data--broad-references/hg38/v0/wgs_calling_regions.hg38.interval_list",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/G94982.NA12892.json

@@ -4,5 +4,6 @@
   "ReblockGVCF.calling_interval_list": "gs://gcp-public-data--broad-references/hg38/v0/wgs_calling_regions.hg38.interval_list",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/G96830.NA12878.json

@@ -4,5 +4,6 @@
   "ReblockGVCF.calling_interval_list": "gs://gcp-public-data--broad-references/hg38/v0/wgs_calling_regions.hg38.interval_list",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/NA12878.bge.json

@@ -7,5 +7,6 @@
   "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
   "ReblockGVCF.annotations_to_remove_command": "--format-annotations-to-remove PRI",
   "ReblockGVCF.move_filters_to_genotypes": true,
-  "ReblockGVCF.gvcf_file_extension": ".gvcf.gz"
+  "ReblockGVCF.gvcf_file_extension": ".gvcf.gz",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/NA12878.ultima.json

@@ -6,5 +6,6 @@
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
   "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
   "ReblockGVCF.tree_score_cutoff": 0.2,
-  "ReblockGVCF.annotations_to_keep_command": "--annotations-to-keep TREE_SCORE --annotations-to-keep ASSEMBLED_HAPS --annotations-to-keep FILTERED_HAPS"
+  "ReblockGVCF.annotations_to_keep_command": "--annotations-to-keep TREE_SCORE --annotations-to-keep ASSEMBLED_HAPS --annotations-to-keep FILTERED_HAPS",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/RP-1535.NA17-308.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/RP-1535.NA17-308/NA17-308.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/Scientific/RP-518.NA12878.json

@@ -4,5 +4,6 @@
   "ReblockGVCF.calling_interval_list": "gs://gcp-public-data--broad-references/hg38/v0/wgs_calling_regions.hg38.interval_list",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/exome/Plumbing/RP-929.NA12878.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/plumbing/input/RP-929.NA12878/NA12878_PLUMBING.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/exome/Scientific/C1963.CHMI_CHMI3_Nex1.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/C1963.CHMI_CHMI3_Nex1/CHMI_CHMI3_Nex1.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/exome/Scientific/C862.NA19238.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/C862.NA19238/NA19238.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/exome/Scientific/D5327.NA12878.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/D5327.NA12878/NA12878.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/exome/Scientific/D5327.NA12891.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/D5327.NA12891/NA12891.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/exome/Scientific/D5327.NA12892.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/D5327.NA12892/NA12892.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/exome/Scientific/RP-1535.NA17-308.json

@@ -3,5 +3,6 @@
   "ReblockGVCF.gvcf_index": "gs://broad-gotc-test-storage/reblock_gvcf/exome/scientific/input/RP-1535.NA17-308/NA17-308.g.vcf.gz.tbi",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/wgs/Plumbing/G96830.NA12878.json

@@ -4,5 +4,6 @@
   "ReblockGVCF.calling_interval_list": "gs://gcp-public-data--broad-references/hg38/v0/wgs_calling_regions.hg38.interval_list",
   "ReblockGVCF.ref_fasta": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta",
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
-  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict"
+  "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
+  "ReblockGVCF.cloud_provider": "gcp"
 }

pipelines/broad/dna_seq/germline/joint_genotyping/reblocking/test_inputs/wgs/Plumbing/NA12878.ultima.json

@@ -6,5 +6,6 @@
   "ReblockGVCF.ref_fasta_index": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.fai",
   "ReblockGVCF.ref_dict": "gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict",
   "ReblockGVCF.tree_score_cutoff": 0.2,
-  "ReblockGVCF.annotations_to_keep_command": "--annotations-to-keep TREE_SCORE --annotations-to-keep ASSEMBLED_HAPS --annotations-to-keep FILTERED_HAPS"
+  "ReblockGVCF.annotations_to_keep_command": "--annotations-to-keep TREE_SCORE --annotations-to-keep ASSEMBLED_HAPS --annotations-to-keep FILTERED_HAPS",
+  "ReblockGVCF.cloud_provider": "gcp"
 }