Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ability to copy interval list files to gs directory [VS-191] #7467

Merged
merged 7 commits into from
Sep 15, 2021
Merged

Add ability to copy interval list files to gs directory [VS-191] #7467

Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
c81bf83
first stab
rsasch Sep 13, 2021
fa5ebf5
Merge branch 'ah_var_store' into rsa_add_interval_list_to_wf_outputs
rsasch Sep 13, 2021
e0ce5a1
dockstore integration
rsasch Sep 14, 2021
86ed587
this OUTPUT_GCS_DIR behaves differently I guess
rsasch Sep 14, 2021
8c66078
added -m option and removed branch from .dockstore.yaml
rsasch Sep 14, 2021
94ff71d
oops typo
rsasch Sep 14, 2021
c73cfd8
Merge branch 'ah_var_store' into rsa_add_interval_list_to_wf_outputs
rsasch Sep 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 18 additions & 4 deletions scripts/variantstore/wdl/GvsExtractCallset.wdl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,9 @@ workflow GvsExtractCallset {
ref_fasta = reference,
ref_fai = reference_index,
ref_dict = reference_dict,
scatter_count = scatter_count
scatter_count = scatter_count,
output_gcs_dir = output_gcs_dir,
service_account_json_path = service_account_json_path
}

call GetBQTableLastModifiedDatetime as fq_cohort_extract_table_datetime {
Expand Down Expand Up @@ -203,9 +205,6 @@ task ExtractTask {
OUTPUT_FILE_INDEX_BYTES="$(du -b ~{output_file}.tbi | cut -f1)"
echo ${OUTPUT_FILE_INDEX_BYTES} > vcf_index_bytes.txt

# Drop trailing slash if one exists
OUTPUT_GCS_DIR="$(echo ~{output_gcs_dir} | sed 's/\/$//')"

if [ -n "${OUTPUT_GCS_DIR}" ]; then
gsutil cp ~{output_file} ${OUTPUT_GCS_DIR}/
gsutil cp ~{output_file}.tbi ${OUTPUT_GCS_DIR}/
Expand Down Expand Up @@ -251,10 +250,14 @@ task ExtractTask {
File ref_dict
Int scatter_count
String? split_intervals_extra_args
String? output_gcs_dir

File? gatk_override
String? service_account_json_path
}

String has_service_account_file = if (defined(service_account_json_path)) then 'true' else 'false'

parameter_meta {
intervals: {
localization_optional: true
Expand Down Expand Up @@ -283,6 +286,17 @@ task ExtractTask {
-O interval-files \
~{split_intervals_extra_args}
cp interval-files/*.interval_list .

# Drop trailing slash if one exists
OUTPUT_GCS_DIR=$(echo ~{output_gcs_dir} | sed 's/\/$//')

if [ -n "$OUTPUT_GCS_DIR" ]; then
if [ ~{has_service_account_file} = 'true' ]; then
gsutil cp ~{service_account_json_path} local.service_account.json
gcloud auth activate-service-account --key-file=local.service_account.json
fi
gsutil -m cp *.interval_list $OUTPUT_GCS_DIR/
Copy link
Contributor

@RoriCremer RoriCremer Sep 14, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any organizational benefit to putting them into their own interval list specific directory? like the interval-files/ one gatk creates?

fi
}

runtime {
Expand Down