Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Self-organizing hash table to improve the performance of bucket aggregations (#7652) #8337

Merged

Conversation

ketanv3
Copy link
Contributor

@ketanv3 ketanv3 commented Jun 29, 2023

Description

Backport #7652 to 2.x.

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@ketanv3 ketanv3 force-pushed the backport/backport-7652-to-2.x branch from 5c8024a to bfdff4a Compare June 29, 2023 05:31
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@ketanv3 ketanv3 force-pushed the backport/backport-7652-to-2.x branch from bfdff4a to db2c6ee Compare June 29, 2023 08:29
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@ketanv3
Copy link
Contributor Author

ketanv3 commented Jun 29, 2023

The failure in SegmentReplicationUsingRemoteStoreIT.testPressureServiceStats is unrelated to this change. It seems to be a trending failure lately (#8316, #8343 to name a few).

…gations (opensearch-project#7652)

* Add self-organizing hash table to improve the performance of bucket aggregations

Signed-off-by: Ketan Verma <[email protected]>

* Updated approach: PSL, fingerprint and recency information are embedded in the hash table itself

Signed-off-by: Ketan Verma <[email protected]>

* Updated tests and added microbenchmarks

Signed-off-by: Ketan Verma <[email protected]>

* Renamed FastLongHash to ReorganizingLongHash and updated the default initial capacity

Signed-off-by: Ketan Verma <[email protected]>

---------

Signed-off-by: Ketan Verma <[email protected]>
@ketanv3 ketanv3 force-pushed the backport/backport-7652-to-2.x branch from db2c6ee to 6cef76c Compare June 29, 2023 13:09
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.search.SearchWeightedRoutingIT.testStrictWeightedRoutingWithCustomString

@codecov
Copy link

codecov bot commented Jun 29, 2023

Codecov Report

Merging #8337 (6cef76c) into 2.x (e5e006e) will decrease coverage by 0.08%.
The diff coverage is 43.66%.

@@             Coverage Diff              @@
##                2.x    #8337      +/-   ##
============================================
- Coverage     70.63%   70.55%   -0.08%     
+ Complexity    56914    56909       -5     
============================================
  Files          4714     4716       +2     
  Lines        269643   269784     +141     
  Branches      39857    39878      +21     
============================================
- Hits         190456   190355     -101     
- Misses        62784    63056     +272     
+ Partials      16403    16373      -30     
Impacted Files Coverage Δ
.../org/opensearch/common/util/LongHashBenchmark.java 0.00% <0.00%> (ø)
...g/opensearch/common/util/ReorganizingLongHash.java 95.31% <95.31%> (ø)
...aggregations/bucket/terms/LongKeyedBucketOrds.java 82.75% <100.00%> (ø)

... and 456 files with indirect coverage changes

@ketanv3
Copy link
Contributor Author

ketanv3 commented Jun 29, 2023

@dblock please help with the backport :)

Copy link
Member

@shwetathareja shwetathareja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, it is backport PR only.

@shwetathareja shwetathareja merged commit c716544 into opensearch-project:2.x Jul 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants