Standardized use of mem parameters in CNV WDLs. #4193
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,11 +8,14 @@ task PreprocessIntervals { | |
| File? gatk4_jar_override | ||
|
|
||
| # Runtime parameters | ||
| Int? mem_gb | ||
| String gatk_docker | ||
| Int? preemptible_attempts | ||
| Int? disk_space_gb | ||
|
|
||
| Int machine_mem_mb = select_first([mem_gb, 2]) * 1000 | ||
| Int command_mem_mb = machine_mem_mb - 500 | ||
|
|
||
| # Determine output filename | ||
| String filename = select_first([intervals, "wgs"]) | ||
| String base_filename = basename(filename, ".interval_list") | ||
|
|
@@ -21,7 +24,7 @@ task PreprocessIntervals { | |
| set -e | ||
| export GATK_LOCAL_JAR=${default="/root/gatk.jar" gatk4_jar_override} | ||
|
|
||
| gatk --java-options "-Xmx${command_mem_mb}m" PreprocessIntervals \ | ||
| ${"-L " + intervals} \ | ||
| --sequence-dictionary ${ref_fasta_dict} \ | ||
| --reference ${ref_fasta} \ | ||
|
|
@@ -33,7 +36,7 @@ task PreprocessIntervals { | |
|
|
||
| runtime { | ||
| docker: "${gatk_docker}" | ||
| memory: machine_mem_mb + " MB" | ||
| disks: "local-disk " + select_first([disk_space_gb, 40]) + " HDD" | ||
| preemptible: select_first([preemptible_attempts, 5]) | ||
| } | ||
|
|
@@ -51,16 +54,19 @@ task AnnotateIntervals { | |
| File? gatk4_jar_override | ||
|
|
||
| # Runtime parameters | ||
| Int? mem_gb | ||
| String gatk_docker | ||
| Int? preemptible_attempts | ||
| Int? disk_space_gb | ||
|
|
||
| Int machine_mem_mb = select_first([mem_gb, 2]) * 1000 | ||
| Int command_mem_mb = machine_mem_mb - 500 | ||
|
|
||
| command <<< | ||
| set -e | ||
| export GATK_LOCAL_JAR=${default="/root/gatk.jar" gatk4_jar_override} | ||
|
|
||
| gatk --java-options "-Xmx${command_mem_mb}m" AnnotateIntervals \ | ||
| -L ${intervals} \ | ||
| --reference ${ref_fasta} \ | ||
| --interval-merging-rule OVERLAPPING_ONLY \ | ||
|
|
@@ -69,7 +75,7 @@ task AnnotateIntervals { | |
|
|
||
| runtime { | ||
| docker: "${gatk_docker}" | ||
| memory: machine_mem_mb + " MB" | ||
| disks: "local-disk " + select_first([disk_space_gb, ceil(size(ref_fasta, "GB")) + 50]) + " HDD" | ||
| preemptible: select_first([preemptible_attempts, 5]) | ||
| } | ||
|
|
@@ -87,11 +93,14 @@ task CollectCounts { | |
| File? gatk4_jar_override | ||
|
|
||
| # Runtime parameters | ||
| Int? mem_gb | ||
| String gatk_docker | ||
| Int? preemptible_attempts | ||
| Int? disk_space_gb | ||
|
|
||
| Int machine_mem_mb = select_first([mem_gb, 7]) * 1000 | ||
| Int command_mem_mb = machine_mem_mb - 1000 | ||
|
|
||
| # Sample name is derived from the bam filename | ||
| String base_filename = basename(bam, ".bam") | ||
| String counts_filename = if !defined(format) then "${base_filename}.counts.hdf5" else "${base_filename}.counts.tsv" | ||
|
|
@@ -100,7 +109,7 @@ task CollectCounts { | |
| set -e | ||
| export GATK_LOCAL_JAR=${default="/root/gatk.jar" gatk4_jar_override} | ||
|
|
||
| gatk --java-options "-Xmx${command_mem_mb}m" CollectFragmentCounts \ | ||
| --input ${bam} \ | ||
| -L ${intervals} \ | ||
| --format ${default="HDF5" format} \ | ||
|
|
@@ -110,7 +119,7 @@ task CollectCounts { | |
|
|
||
| runtime { | ||
| docker: "${gatk_docker}" | ||
| memory: machine_mem_mb + " MB" | ||
| disks: "local-disk " + select_first([disk_space_gb, ceil(size(bam, "GB")) + 50]) + " HDD" | ||
| preemptible: select_first([preemptible_attempts, 5]) | ||
| } | ||
|
|
@@ -132,14 +141,13 @@ task CollectAllelicCounts { | |
| File? gatk4_jar_override | ||
|
|
||
| # Runtime parameters | ||
| Int? mem_gb | ||
| String gatk_docker | ||
| Int? preemptible_attempts | ||
| Int? disk_space_gb | ||
|
|
||
|
There was a problem hiding this comment. Somewhere it should made clear that the user provided There was a problem hiding this comment. Any objection to simply using There was a problem hiding this comment. @samuelklee Fine by me... There was a problem hiding this comment.
That is sufficiently self-documenting for my tastes. |
||
| Int machine_mem_mb = select_first([mem_gb, 13]) * 1000 | ||
| Int command_mem_mb = machine_mem_mb - 1000 | ||
|
|
||
| # Sample name is derived from the bam filename | ||
| String base_filename = basename(bam, ".bam") | ||
|
|
@@ -150,7 +158,7 @@ task CollectAllelicCounts { | |
| set -e | ||
| export GATK_LOCAL_JAR=${default="/root/gatk.jar" gatk4_jar_override} | ||
|
|
||
| gatk --java-options "-Xmx${command_mem_mb}m" CollectAllelicCounts \ | ||
| -L ${common_sites} \ | ||
| --input ${bam} \ | ||
| --reference ${ref_fasta} \ | ||
|
|
@@ -160,7 +168,7 @@ task CollectAllelicCounts { | |
|
|
||
| runtime { | ||
| docker: "${gatk_docker}" | ||
| memory: machine_mem_mb + " MB" | ||
| disks: "local-disk " + select_first([disk_space_gb, ceil(size(bam, "GB")) + 50]) + " HDD" | ||
| preemptible: select_first([preemptible_attempts, 5]) | ||
| } | ||
|
|
@@ -176,11 +184,13 @@ task ScatterIntervals { | |
| Int num_intervals_per_scatter | ||
|
|
||
| # Runtime parameters | ||
| Int? mem_gb | ||
| String gatk_docker | ||
| Int? preemptible_attempts | ||
| Int? disk_space_gb | ||
|
|
||
| Int machine_mem_mb = select_first([mem_gb, 2]) * 1000 | ||
|
|
||
| String base_filename = basename(interval_list, ".interval_list") | ||
|
|
||
| command <<< | ||
|
|
@@ -194,7 +204,7 @@ task ScatterIntervals { | |
|
|
||
| runtime { | ||
| docker: "${gatk_docker}" | ||
| memory: machine_mem_mb + " MB" | ||
| disks: "local-disk " + select_first([disk_space_gb, 40]) + " HDD" | ||
| preemptible: select_first([preemptible_attempts, 5]) | ||
| } | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As per @LeeTL1220's template, we should be putting in values for
cpueven in the single threaded case.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't check to make sure all of the tasks adhered to the template just yet, since I'm assuming that it might still change. (Actually, I don't think any of the somatic CNV tasks specify cpu.) I'll focus on the mem changes in this PR and overhaul the rest later (perhaps once tasks are automatically generated), if you don't mind!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@samuelklee Fine by me.