[Backport-2.x][Tiered Caching] Indices Request cache stalekey management (#12625) #12707
Conversation
…h-project#12625) * Introduce IndicesRequestCacheCleanupManager Signed-off-by: Kiran Prakash <[email protected]> * using cleanup mgr to enqueue cleanups Signed-off-by: Kiran Prakash <[email protected]> * readability improvements Signed-off-by: Kiran Prakash <[email protected]> * update updateCleanupKeyToCountMap on new cache entry Signed-off-by: Kiran Prakash <[email protected]> * create IndicesRequestCacheCleanupManager & settings and validators Signed-off-by: Kiran Prakash <[email protected]> * Add IRC specific cache cleaner and remove from IndicesService Signed-off-by: Kiran Prakash <[email protected]> * Unit Tests Signed-off-by: Kiran Prakash <[email protected]> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <[email protected]> * move cachecleaner inside mgr Signed-off-by: Kiran Prakash <[email protected]> * remove processCleanupKeys Signed-off-by: Kiran Prakash <[email protected]> * minor cleanups Signed-off-by: Kiran Prakash <[email protected]> * add updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <[email protected]> * remove locks and make all methods synchronized Signed-off-by: Kiran Prakash <[email protected]> * spotless Signed-off-by: Kiran Prakash <[email protected]> * updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <[email protected]> * Testing Signed-off-by: Kiran Prakash <[email protected]> * add Reschedule back to indices service Signed-off-by: Kiran Prakash <[email protected]> * rename updateStaleKeysCount to incrementStaleKeysCount Signed-off-by: Kiran Prakash <[email protected]> * rename getStaleKeysCountForTesting to getStaleKeysCount Signed-off-by: Kiran Prakash <[email protected]> * rename threshold to stalenessThreshold Signed-off-by: Kiran Prakash <[email protected]> * check for cleanupKey.entity == null Signed-off-by: Kiran Prakash <[email protected]> * use computeIfPresent with keycountmap Signed-off-by: Kiran Prakash <[email protected]> * log both staleKeysInCache & Staleness in debug logs Signed-off-by: Kiran Prakash <[email protected]> * Use HashMap instead of ConcurrentMap Signed-off-by: Kiran Prakash <[email protected]> * Address b/w compatibility Signed-off-by: Kiran Prakash <[email protected]> * remove synchronized for updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <[email protected]> * make cleanCache synchronized Signed-off-by: Kiran Prakash <[email protected]> * remove shouldRemoveKey Signed-off-by: Kiran Prakash <[email protected]> * spotlessApply Signed-off-by: Kiran Prakash <[email protected]> --------- Signed-off-by: Kiran Prakash <[email protected]> (cherry picked from commit b4da802)
kiranprakash154
requested review from
abbashus,
adnapibar,
anasalkouz,
andrross,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
dreamer-89,
gbbafna,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
ryanbogan,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami,
tlfeng and
VachaShah
as code owners
kiranprakash154
changed the title
Compatibility status:Checks if related components are compatible with change ca503bc Incompatible componentsSkipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/sql.git] |
|
❌ Gradle check result for 6697fec: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Kiran Prakash <[email protected]>
|
❌ Gradle check result for 4e937ca: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 4e937ca: null Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
This was due to flaky test that has issue created - #10704 |
This was due to flaky test that has issue created - #12236 |
Signed-off-by: Kiran Prakash <[email protected]>
|
❌ Gradle check result for 9d4f011: Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
|
❌ Gradle check result for ca503bc: Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 9708f79: Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 2.x #12707 +/- ##
============================================
- Coverage 71.28% 71.22% -0.07%
- Complexity 60145 60173 +28
============================================
Files 4957 4963 +6
Lines 282799 283268 +469
Branches 41409 41466 +57
============================================
+ Hits 201591 201752 +161
- Misses 64189 64427 +238
- Partials 17019 17089 +70 ☔ View full report in Codecov by Sentry. |
Description
Backports #12625
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.