-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rewrite array_append/array_prepend
to remove deplicate codes
#8108
Conversation
@alamb @jayzhan211 PTAL ^ . ^ |
Signed-off-by: veeupup <[email protected]>
array_append/array_prepend
to remove deplicate codes
Thanks @Veeupup -- I'll check it out shortly |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @Veeupup ! Could you also move the related tests to sqllogictest?
My expected way for array_append/array_prepend is building array via MutableArrayData, so I need to think about whether row converter is the better choice or not. |
I wrote another function with MutableArrayData and it is a lot faster. 30000 micro second vs 300 micro second. @Veeupup Can you also try with MutableArrayData and compare it yourself, if you also got the same result, we can go with MutableArrayData approach. |
Sure! I'll make it later |
Appreciate your advice! I'll try this way and benchmark it then |
hi @Weijun-H ! it seems that I do not need to do anything? (Or just remove ut I have checked the |
Signed-off-by: veeupup <[email protected]>
Signed-off-by: veeupup <[email protected]>
@jayzhan211 I have implemented the array_append in MutableArray and RowConverter and it shows that Mutable performs better too. Here are the benchmarks for sqllogictests:
|
Very nice! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @Veeupup and @jayzhan211 -- this looks a lot nicer to me. I think there is even more performance to be gained here by avoiding concat
(and using ArrayData to build up the final output) but this is a lot better than what we have currently.
Thanks a lot 🙏
Which issue does this PR close?
One issue from #7988
array_append
&&array_prepend
.Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?