reset realtime task state #201
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Yaliang Wu <[email protected]>
ohltyler
reviewed
Sep 2, 2021
kaituo
reviewed
Sep 2, 2021
Codecov Report
@@ Coverage Diff @@
## main #201 +/- ##
============================================
+ Coverage 73.48% 73.71% +0.23%
- Complexity 3410 3444 +34
============================================
Files 276 276
Lines 15327 15383 +56
Branches 1560 1567 +7
============================================
+ Hits 11263 11340 +77
+ Misses 3341 3322 -19
+ Partials 723 721 -2
Flags with carried forward coverage won't be shown. Click here to find out more.
|
ohltyler
previously approved these changes
Sep 3, 2021
ylwu-amzn
force-pushed
the
reset_realtime_task_state
branch
from
d84f4c8 to
96b914a
Compare
kaituo
reviewed
Sep 3, 2021
kaituo
reviewed
Sep 3, 2021
| @@ -89,7 +89,7 @@ | |||
| private Map<String, String> detectorTasks; | |||
|
|
|||
| // Use this field to cache all HC tasks. Key is detector id | |||
| private Map<String, ADHCBatchTaskCache> hcTaskCaches; | |||
| private Map<String, ADHCBatchTaskCache> hcBatchTaskCaches; | |||
There was a problem hiding this comment.
What are the differences between hcBatchTaskCaches and batchTaskCaches? Do they have overlap contents? Is it that hcBatchTaskCaches have detector level task cache, while batchTaskCaches has entity level task cache (if it is a HC detector) and detector level task cache (if it is single-stream detector)? if yes, then I am more confused. Why cannot we combine these two maps?
There was a problem hiding this comment.
Your understanding is correct. hcBatchTaskCaches: cache HC detector level information on coordinating node, will add more comments. I think the comment for batchTaskCaches is enough, you can read and suggest more if the comment is still not clear.
They don't have overlap contents. Combine these two maps will mix the coordinating node and worker node cache together. I don't prefer that way now. If no big concern or cons , how about we keep current design now?
kaituo
approved these changes
Sep 3, 2021
ohltyler
approved these changes
Sep 3, 2021
ohltyler
pushed a commit
to ohltyler/anomaly-detection-2
that referenced
this pull request
Sep 7, 2021
* reset realtime task state Signed-off-by: Yaliang Wu <[email protected]> * try to delete AD tasks and job when detector not found * address comments * add more comments * fix bugs; tune max running entities setting; add more UT * rename variable * add more comment
ohltyler
pushed a commit
to ohltyler/anomaly-detection-2
that referenced
this pull request
Sep 7, 2021
* reset realtime task state Signed-off-by: Yaliang Wu <[email protected]> * try to delete AD tasks and job when detector not found * address comments * add more comments * fix bugs; tune max running entities setting; add more UT * rename variable * add more comment
This was referenced Sep 7, 2021
ohltyler
pushed a commit
to ohltyler/anomaly-detection-2
that referenced
this pull request
Sep 14, 2021
* reset realtime task state Signed-off-by: Yaliang Wu <[email protected]> * try to delete AD tasks and job when detector not found * address comments * add more comments * fix bugs; tune max running entities setting; add more UT * rename variable * add more comment
1 task
ohltyler
pushed a commit
that referenced
this pull request
Sep 23, 2021
* reset realtime task state Signed-off-by: Yaliang Wu <[email protected]> * try to delete AD tasks and job when detector not found * address comments * add more comments * fix bugs; tune max running entities setting; add more UT * rename variable * add more comment
ohltyler
pushed a commit
that referenced
this pull request
Sep 23, 2021
* reset realtime task state Signed-off-by: Yaliang Wu <[email protected]> * try to delete AD tasks and job when detector not found * address comments * add more comments * fix bugs; tune max running entities setting; add more UT * rename variable * add more comment
kaituo
added a commit
to kaituo/anomaly-detection-1
that referenced
this pull request
Sep 25, 2021
This PR contains: PRs (all approved) related to integrating ThresholdedRandomCutForest: opensearch-project#221 opensearch-project#222 opensearch-project#223 opensearch-project#224 opensearch-project#226 opensearch-project#227 opensearch-project#228 rebased PR: opensearch-project#201 opensearch-project#202 opensearch-project#216
1 task
kaituo
added a commit
to kaituo/anomaly-detection-1
that referenced
this pull request
Sep 25, 2021
This PR contains: PRs (all approved) related to integrating ThresholdedRandomCutForest: opensearch-project#221 opensearch-project#222 opensearch-project#223 opensearch-project#224 opensearch-project#226 opensearch-project#227 opensearch-project#228 rebased PR: opensearch-project#201 opensearch-project#202 opensearch-project#216
kaituo
added a commit
that referenced
this pull request
Sep 27, 2021
* [1.1] Bump OpenSearch core to 1.1 in CI (#212) Signed-off-by: Tyler Ohlsen <[email protected]> * add thresholded rcf (#215) Signed-off-by: lai <[email protected]> * Integrating with ThresholdedRandomCutForest This PR contains: PRs (all approved) related to integrating ThresholdedRandomCutForest: #221 #222 #223 #224 #226 #227 #228 rebased PR: #201 #202 #216 * pass the correct shingleSize to ThresholdedRandomCutForest Previously, I used shingleSize 1 for externally shingled ThresholdedRandomCutForest because of the double multiplication with shingle size in RCF. Now RCF has fixed the issue. This commits adds new RCF libraries from aws/random-cut-forest-by-aws#278 and passes the correct shingleSize to ThresholdedRandomCutForest. This commits adds new RCF libraries from aws/random-cut-forest-by-aws#278 and passes the correct shingleSize to ThresholdedRandomCutForest. Co-authored-by: Tyler Ohlsen <[email protected]> Co-authored-by: Lai <[email protected]>
hamersu9t
added a commit
to hamersu9t/anomaly-detection
that referenced
this pull request
Aug 10, 2024
* [1.1] Bump OpenSearch core to 1.1 in CI (#212) Signed-off-by: Tyler Ohlsen <[email protected]> * add thresholded rcf (#215) Signed-off-by: lai <[email protected]> * Integrating with ThresholdedRandomCutForest This PR contains: PRs (all approved) related to integrating ThresholdedRandomCutForest: opensearch-project/anomaly-detection#221 opensearch-project/anomaly-detection#222 opensearch-project/anomaly-detection#223 opensearch-project/anomaly-detection#224 opensearch-project/anomaly-detection#226 opensearch-project/anomaly-detection#227 opensearch-project/anomaly-detection#228 rebased PR: opensearch-project/anomaly-detection#201 opensearch-project/anomaly-detection#202 opensearch-project/anomaly-detection#216 * pass the correct shingleSize to ThresholdedRandomCutForest Previously, I used shingleSize 1 for externally shingled ThresholdedRandomCutForest because of the double multiplication with shingle size in RCF. Now RCF has fixed the issue. This commits adds new RCF libraries from aws/random-cut-forest-by-aws#278 and passes the correct shingleSize to ThresholdedRandomCutForest. This commits adds new RCF libraries from aws/random-cut-forest-by-aws#278 and passes the correct shingleSize to ThresholdedRandomCutForest. Co-authored-by: Tyler Ohlsen <[email protected]> Co-authored-by: Lai <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Yaliang Wu [email protected]
Description
Reset realtime task state when get latest detector task. If job is disabled, we should set realtime task state as stopped.
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.