-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid negative memory result in IndicesQueryCache stats calculation #6917
Conversation
Gradle Check (Jenkins) Run Completed with:
|
Hmm, seems I made too strong of an assumption on tests:
Will fix. |
Fun thing about that test failure is it let me figure out the root cause of the original bug! // This cache stores two things: filters, and doc id sets. At this time
// we only know that there are no more doc id sets, but we still track
// recently used queries, which we want to reclaim. |
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Daniel Widdis <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more @@ Coverage Diff @@
## main #6917 +/- ##
============================================
+ Coverage 70.66% 70.83% +0.17%
- Complexity 59231 59315 +84
============================================
Files 4812 4812
Lines 283761 283763 +2
Branches 40917 40918 +1
============================================
+ Hits 200519 201011 +492
+ Misses 66784 66240 -544
- Partials 16458 16512 +54
... and 471 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Signed-off-by: Daniel Widdis <[email protected]>
Gradle Check (Jenkins) Run Completed with:
|
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-1.x 1.x
# Navigate to the new working tree
pushd ../.worktrees/backport-1.x
# Create a new branch
git switch --create backport/backport-6917-to-1.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c0a8cf8823341cf92b932fe3d12909d427090a2f
# Push it to GitHub
git push --set-upstream origin backport/backport-6917-to-1.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-1.x Then, create a pull request where the |
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-6917-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c0a8cf8823341cf92b932fe3d12909d427090a2f
# Push it to GitHub
git push --set-upstream origin backport/backport-6917-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x Then, create a pull request where the |
(opensearch-project#6917) * Avoid negative memory result in IndicesQueryCache stats calculation * Cache stores recent queries so isn't empty even with no doc id sets Signed-off-by: Daniel Widdis <[email protected]> * Add test which actually fails without the bug fix Signed-off-by: Daniel Widdis <[email protected]> --------- Signed-off-by: Daniel Widdis <[email protected]>
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-1.3 1.3
# Navigate to the new working tree
pushd ../.worktrees/backport-1.3
# Create a new branch
git switch --create backport/backport-6917-to-1.3
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c0a8cf8823341cf92b932fe3d12909d427090a2f
# Push it to GitHub
git push --set-upstream origin backport/backport-6917-to-1.3
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-1.3 Then, create a pull request where the |
…pensearch-project#6917) * Avoid negative memory result in IndicesQueryCache stats calculation Signed-off-by: Daniel Widdis <[email protected]> * Cache stores recent queries so isn't empty even with no doc id sets Signed-off-by: Daniel Widdis <[email protected]> * Add test which actually fails without the bug fix Signed-off-by: Daniel Widdis <[email protected]> --------- Signed-off-by: Daniel Widdis <[email protected]> (cherry picked from commit c0a8cf8)
(opensearch-project#6917) * Avoid negative memory result in IndicesQueryCache stats calculation Signed-off-by: Daniel Widdis <[email protected]> * Cache stores recent queries so isn't empty even with no doc id sets Signed-off-by: Daniel Widdis <[email protected]> * Add test which actually fails without the bug fix Signed-off-by: Daniel Widdis <[email protected]> --------- Signed-off-by: Daniel Widdis <[email protected]> (cherry picked from commit c0a8cf8)
…pensearch-project#6917) * Avoid negative memory result in IndicesQueryCache stats calculation Signed-off-by: Daniel Widdis <[email protected]> * Cache stores recent queries so isn't empty even with no doc id sets Signed-off-by: Daniel Widdis <[email protected]> * Add test which actually fails without the bug fix Signed-off-by: Daniel Widdis <[email protected]> --------- Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Valentin Mitrofanov <[email protected]>
…6966) (#6917) * Avoid negative memory result in IndicesQueryCache stats calculation Signed-off-by: Daniel Widdis <[email protected]> * Cache stores recent queries so isn't empty even with no doc id sets Signed-off-by: Daniel Widdis <[email protected]> * Add test which actually fails without the bug fix Signed-off-by: Daniel Widdis <[email protected]> --------- Signed-off-by: Daniel Widdis <[email protected]> (cherry picked from commit c0a8cf8)
…6965) (#6917) * Avoid negative memory result in IndicesQueryCache stats calculation * Cache stores recent queries so isn't empty even with no doc id sets * Add test which actually fails without the bug fix --------- Signed-off-by: Daniel Widdis <[email protected]>
Description
In the edge case where shardStats is an empty map (caused by removing the last document from an index through a means other than clearing the index), the stats map is also empty. However this case also leads to divide-by-zero calculation which gives a weight of
Double.POSITIVE_INFINITY
. This rounds toLong.MAX_VALUE
for additionalRamBytesUsed for the shard.Then later in calculating total memory, the max long value is added to other fields, resulting in overflow.
Issues Resolved
Fixes #4474
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.