You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge?
During review of #6932 there were several suggstions made about how we could improve the performance of streaming / bounded grouping (when the data is sorted by some/all of the group keys)
It occurs to me that a potentially faster way to detect the group boundaries would be to use the inequality kernel offset by one w.r.t to each other, ORing the results together, and then iterating over the set bits.
There would be some subtlety to handle nulls correctly, but it would likely be significantly faster and would not require converting to the row format
We could likely do something similar for window functions if we aren't already
@alamb I think this issue can be closed. Suggestion 1 is done, suggestion 2 is no longer relevant as there are no hashes computed for GroupOrderingFull
Is your feature request related to a problem or challenge?
During review of #6932 there were several suggstions made about how we could improve the performance of streaming / bounded grouping (when the data is sorted by some/all of the group keys)
This ticket tracks those improvements
Describe the solution you'd like
Suggestion 1
@tustvold suggests on #6932 (comment)
Note that IOx does something similar to compute partition boundaries here: https://github.com/influxdata/influxdb_iox/blob/d3b3805e5fa87214ea934427df3967a57a14669c/iox_query/src/provider/deduplicate/algo.rs#L101-L207
Suggestion 2
From #6932 (comment)
This could likely go quite a bit faster as hashing is a significant part of the time
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: