Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNVQ recalibration for flow based reads #8697

Merged
merged 56 commits into from
Apr 4, 2024
Merged

SNVQ recalibration for flow based reads #8697

Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
0ce6afe
ApplySNVQR tool squashed
dror27 Nov 14, 2023
bf5d2d8
[BIOIN-1385] FlowBasedRead now can estimate filling_value from the read
ilyasoifer Dec 27, 2023
876a4e6
[BIOIN-1385] Removed probabilityRatioThreshold
ilyasoifer Dec 27, 2023
86d3ae4
[BIOIN-1385] Removed mistaken usage by fillingValue
ilyasoifer Dec 28, 2023
c5e2ad8
[BIOIN-1385] Fixed bug in t0 parsing
ilyasoifer Dec 30, 2023
fcbbd3c
[BIOIN-1385] Reverted the aggressive change in filling probability in…
ilyasoifer Jan 1, 2024
ba29b3c
[BIOIN-1385] Minor update of poorly_modelded_reads
ilyasoifer Jan 1, 2024
ac66953
[BIOIN-1385] FlowBasedRead class cleanup
ilyasoifer Jan 2, 2024
8d9782f
[BIOIN-1385] Updated ground truth scorer for the case when the fillin…
ilyasoifer Jan 2, 2024
a032030
Fixed typing error + missed file
ilyasoifer Jan 3, 2024
626a458
ceil -> round
ilyasoifer Jan 3, 2024
b4d8341
Set phred score for bins with no zeros to maximum rather than zero
ilyasoifer Jan 5, 2024
2d69e85
Updated tests
ilyasoifer Jan 5, 2024
92aab19
Updated tests
ilyasoifer Jan 5, 2024
c564dc6
Updated tests
ilyasoifer Jan 5, 2024
51ebdb5
-10 qual fix
dror27 Jan 18, 2024
a7947b9
Guard against negative qualities
dror27 Jan 21, 2024
5e6da62
(not final) reuse AddFlowBaseQuality code to accelerate
dror27 Jan 22, 2024
fcb1f83
--debug-use-add-flow-base-quality-algorithm flasg
dror27 Jan 23, 2024
8f17ea5
use of add_base_q now default
dror27 Jan 23, 2024
8634c11
ApplySNVQR test data rebuilt
dror27 Jan 23, 2024
9ba77d3
add_base_q integration in progress ... missing mapping function
dror27 Jan 23, 2024
44b6763
allow unsorted/unmapped input to ApplySNVQR when no model specified
dror27 Jan 24, 2024
2f478de
SNVQ of called base adjusted to sum to 1
dror27 Jan 24, 2024
13aa6ee
update called base snvq
dror27 Jan 25, 2024
5850557
--limit-phred-score replaces --limit-score
dror27 Jan 28, 2024
d6707d2
flow flags passed to add base quality
dror27 Jan 29, 2024
d0e4a0f
BQ adjusted
dror27 Jan 29, 2024
f79da59
Fixed boundary case
ilyasoifer Jan 31, 2024
b2337dd
Snvq modes added. see --snvq-mode param
dror27 Feb 4, 2024
78818b4
SNVq corrected (all 80 issue)
dror27 Feb 5, 2024
f081cb5
Add missing X_ADJACENT_REF_DIFF
dror27 Feb 19, 2024
1a8b734
Tool renamed to AddFlowSNVQuality and model/conf removed
dror27 Feb 26, 2024
1461c82
further cleanup tests
dror27 Feb 26, 2024
1170d4b
FlowFeatureMapper changes removed
dror27 Feb 26, 2024
7da3aa6
expected results adjusted
dror27 Feb 26, 2024
2b25b21
snvq-mode removed. Geometric mode is one used
dror27 Mar 3, 2024
3cb409c
some comments addressed
dror27 Mar 3, 2024
94d4fe7
Update src/main/java/org/broadinstitute/hellbender/tools/walkers/feat…
dror27 Mar 3, 2024
4afc042
Update src/main/java/org/broadinstitute/hellbender/tools/walkers/feat…
dror27 Mar 3, 2024
4f8320e
0.0001 to keep consistent with GATK
dror27 Mar 3, 2024
9c17210
Merge branch 'ultima.snv-qual.develop.rebase' of github.com:Ultimagen…
dror27 Mar 3, 2024
5577f79
includeDupReads removed
dror27 Mar 3, 2024
ef7ea5e
additional updates
dror27 Mar 3, 2024
eacc100
DEFAULT_FILLING_VALUE restored to 0.001
dror27 Mar 11, 2024
a2b7f54
snvq-mode restored
dror27 Mar 11, 2024
06284b0
AddFlowSNVQuality integeration test expected data adjusted
Mar 16, 2024
4ee1255
fix comment spec on expected output
dror27 Mar 16, 2024
bc0ff83
some of the comments addressed
dror27 Mar 18, 2024
ff06e36
additional comments addressed
dror27 Mar 21, 2024
403bd29
additional comments addressed
dror27 Mar 25, 2024
a6afd8e
Clarified function
ilyasoifer Mar 28, 2024
516c152
limit scores precision to regain platform independence
dror27 Mar 31, 2024
4ee5b6d
documented generateHmerBaseErrorProbabilities
dror27 Mar 31, 2024
d9c090b
AddFlowSNVQuality now overrides read base quality by default
dror27 Mar 31, 2024
0c73917
additional doc comments addressed
dror27 Apr 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,12 @@ dependencies {
implementation('net.grey-panther:natural-comparator:1.1')
implementation('com.fasterxml.jackson.module:jackson-module-scala_2.12:2.9.8')

// https://mvnrepository.com/artifact/ml.dmlc/xgboost-jvm
//implementation group: 'ml.dmlc', name: 'xgboost-jvm_2.12', version: '2.0.2', ext: 'pom'
//implementation 'ml.dmlc:xgboost-jvm_2.12:2.0.2'
implementation("ml.dmlc:xgboost4j_2.12:2.0.1")
implementation("ml.dmlc:xgboost4j-spark_2.12:2.0.1")

testUtilsImplementation sourceSets.main.output
testUtilsImplementation 'org.testng:testng:' + testNGVersion
testUtilsImplementation 'org.apache.hadoop:hadoop-minicluster:' + hadoopVersion
Expand Down
8 changes: 1 addition & 7 deletions src/main/java/org/broadinstitute/hellbender/tools/FlowBasedArgumentCollection.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ public class FlowBasedArgumentCollection implements Serializable {
private static final long serialVersionUID = 0;

public static final String FLOW_USE_T0_TAG = "flow-use-t0-tag";
public static final String PROBABILITY_RATIO_THRESHOLD_LONG_NAME = "flow-probability-threshold";
public static final String REMOVE_LONGER_THAN_ONE_INDELS_LONG_NAME = "flow-remove-non-single-base-pair-indels";
public static final String REMOVE_ONE_TO_ZERO_PROBS_LONG_NAME = "flow-remove-one-zero-probs";
public static final String NUMBER_OF_POSSIBLE_PROBS_LONG_NAME = "flow-quantization-bins";
Expand All @@ -27,8 +26,7 @@ public class FlowBasedArgumentCollection implements Serializable {



private static final double DEFAULT_RATIO_THRESHOLD = 0.003;
private static final double DEFAULT_FILLING_VALUE = 0.001;
public static final double DEFAULT_FILLING_VALUE = 0.001;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not 0.0001?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want to keep it 0.001 to be consistent with the GATK

private static final boolean DEFAULT_REMOVE_LONGER_INDELS = false;
private static final boolean DEFAULT_REMOVE_ONE_TO_ZERO = false;
private static final boolean DEFAULT_SYMMETRIC_INDELS = false;
Expand All @@ -45,10 +43,6 @@ public class FlowBasedArgumentCollection implements Serializable {
@Argument(fullName = FLOW_USE_T0_TAG, doc = "Use t0 tag if exists in the read to create flow matrix", optional = true)
public boolean useT0Tag = DEFAULT_FLOW_USE_T0_TAG;

@Advanced
@Argument(fullName = PROBABILITY_RATIO_THRESHOLD_LONG_NAME, doc = "Lowest probability ratio to be used as an option", optional = true)
public double probabilityRatioThreshold = DEFAULT_RATIO_THRESHOLD;

@Advanced
@Argument(fullName = REMOVE_LONGER_THAN_ONE_INDELS_LONG_NAME, doc = "Should the probabilities of more then 1 indel be used", optional = true)
public boolean removeLongerThanOneIndels = DEFAULT_REMOVE_LONGER_INDELS;
Expand Down
Loading
Loading