-
Notifications
You must be signed in to change notification settings - Fork 24.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Transform] Transform optmize date histogram #54068
[Transform] Transform optmize date histogram #54068
Conversation
Pinging @elastic/ml-core (:ml/Transform) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The optimization makes sense. If we are pivoting with a date_histogram, we shouldn't run through the entire index to update the destination.
The state machine changes were difficult to parse through. Will go through them again once this is no longer a draft.
...n/java/org/elasticsearch/xpack/core/transform/transforms/pivot/DateHistogramGroupSource.java
Outdated
Show resolved
Hide resolved
...n/java/org/elasticsearch/xpack/core/transform/transforms/pivot/DateHistogramGroupSource.java
Show resolved
Hide resolved
...n/java/org/elasticsearch/xpack/core/transform/transforms/pivot/DateHistogramGroupSource.java
Outdated
Show resolved
Hide resolved
...plugin/transform/src/main/java/org/elasticsearch/xpack/transform/transforms/pivot/Pivot.java
Outdated
Show resolved
Hide resolved
09085df
to
9c2ed35
Compare
recent buckets according to sync configuration
dbfd27c
to
80d9413
Compare
run elasticsearch-ci/2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another pair of 👀 would be good
...n/java/org/elasticsearch/xpack/core/transform/transforms/pivot/DateHistogramGroupSource.java
Outdated
Show resolved
Hide resolved
...plugin/transform/src/main/java/org/elasticsearch/xpack/transform/transforms/pivot/Pivot.java
Outdated
Show resolved
Hide resolved
…transform/transforms/pivot/Pivot.java Co-Authored-By: Benjamin Trent <[email protected]>
…transform/transforms/pivot/DateHistogramGroupSource.java Co-Authored-By: Benjamin Trent <[email protected]>
optimize transform for group_by on date_histogram by injecting an additional range query. This limits the number of search and index requests and avoids unnecessary updates. Only recent buckets get re-written. fixes #54254
optimize transform for group_by on date_histogram by injecting an additional range query. This limits the number of search and index requests and avoids unnecessary updates. Only recent buckets get re-written. fixes #54254
optimize transform for group_by on date_histogram by injecting an additional range query. This limits the number of search and index requests and avoids unnecessary updates. Only recent buckets get re-written.
fixes #54254