[Tiered Caching] Indices Request cache stalekey management (opensearc…

…h-project#12625) * Introduce IndicesRequestCacheCleanupManager Signed-off-by: Kiran Prakash <[email protected]> * using cleanup mgr to enqueue cleanups Signed-off-by: Kiran Prakash <[email protected]> * readability improvements Signed-off-by: Kiran Prakash <[email protected]> * update updateCleanupKeyToCountMap on new cache entry Signed-off-by: Kiran Prakash <[email protected]> * create IndicesRequestCacheCleanupManager & settings and validators Signed-off-by: Kiran Prakash <[email protected]> * Add IRC specific cache cleaner and remove from IndicesService Signed-off-by: Kiran Prakash <[email protected]> * Unit Tests Signed-off-by: Kiran Prakash <[email protected]> * Update CHANGELOG.md Signed-off-by: Kiran Prakash <[email protected]> * move cachecleaner inside mgr Signed-off-by: Kiran Prakash <[email protected]> * remove processCleanupKeys Signed-off-by: Kiran Prakash <[email protected]> * minor cleanups Signed-off-by: Kiran Prakash <[email protected]> * add updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <[email protected]> * remove locks and make all methods synchronized Signed-off-by: Kiran Prakash <[email protected]> * spotless Signed-off-by: Kiran Prakash <[email protected]> * updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <[email protected]> * Testing Signed-off-by: Kiran Prakash <[email protected]> * add Reschedule back to indices service Signed-off-by: Kiran Prakash <[email protected]> * rename updateStaleKeysCount to incrementStaleKeysCount Signed-off-by: Kiran Prakash <[email protected]> * rename getStaleKeysCountForTesting to getStaleKeysCount Signed-off-by: Kiran Prakash <[email protected]> * rename threshold to stalenessThreshold Signed-off-by: Kiran Prakash <[email protected]> * check for cleanupKey.entity == null Signed-off-by: Kiran Prakash <[email protected]> * use computeIfPresent with keycountmap Signed-off-by: Kiran Prakash <[email protected]> * log both staleKeysInCache & Staleness in debug logs Signed-off-by: Kiran Prakash <[email protected]> * Use HashMap instead of ConcurrentMap Signed-off-by: Kiran Prakash <[email protected]> * Address b/w compatibility Signed-off-by: Kiran Prakash <[email protected]> * remove synchronized for updateCleanupKeyToCountMapOnCacheEviction Signed-off-by: Kiran Prakash <[email protected]> * make cleanCache synchronized Signed-off-by: Kiran Prakash <[email protected]> * remove shouldRemoveKey Signed-off-by: Kiran Prakash <[email protected]> * spotlessApply Signed-off-by: Kiran Prakash <[email protected]> --------- Signed-off-by: Kiran Prakash <[email protected]>
amkhar · Mar 17, 2024 · b4da802 · b4da802
1 parent 48004ee
commit b4da802
Show file tree

Hide file tree

Showing 4 changed files with 800 additions and 66 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -115,6 +115,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - [Metrics Framework] Adds support for asynchronous gauge metric type. ([#12642](https://github.com/opensearch-project/OpenSearch/issues/12642))
 - Make search query counters dynamic to support all query types ([#12601](https://github.com/opensearch-project/OpenSearch/pull/12601))
 - [Tiered caching] Add policies controlling which values can enter pluggable caches [EXPERIMENTAL] ([#12542](https://github.com/opensearch-project/OpenSearch/pull/12542))
+- [Tiered caching] Add Stale keys Management and CacheCleaner to IndicesRequestCache ([#12625](https://github.com/opensearch-project/OpenSearch/pull/12625))
 
 ### Dependencies
 - Bump `peter-evans/find-comment` from 2 to 3 ([#12288](https://github.com/opensearch-project/OpenSearch/pull/12288))
@@ -159,7 +160,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Fix `terms` query on `float` field when `doc_values` are turned off by reverting back to `FloatPoint` from `FloatField` ([#12499](https://github.com/opensearch-project/OpenSearch/pull/12499))
 - Fix get task API does not refresh resource stats ([#11531](https://github.com/opensearch-project/OpenSearch/pull/11531))
 - onShardResult and onShardFailure are executed on one shard causes opensearch jvm crashed ([#12158](https://github.com/opensearch-project/OpenSearch/pull/12158))
-- Avoid overflow when sorting missing last on `epoch_millis` datetime field ([#12676](https://github.com/opensearch-project/OpenSearch/pull/12676))
 
 ### Security