-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
reuse datafusion physical planner in ballista building from protobuf #532
Conversation
Codecov Report
@@ Coverage Diff @@
## master #532 +/- ##
==========================================
+ Coverage 76.04% 76.17% +0.12%
==========================================
Files 157 156 -1
Lines 27098 27017 -81
==========================================
- Hits 20608 20580 -28
+ Misses 6490 6437 -53
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like a nice change to me -- but I think someone more familiar with Ballista to review it too -- @andygrove / @edrevo can you suggest someone?
6ff2361
to
d0f2804
Compare
I will have time tomorrow to review this and other pending ballista PRs
…On Thu, Jun 10, 2021, 9:47 AM QP Hou ***@***.***> wrote:
***@***.**** approved this pull request.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#532 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAHEBRANXOWJAIYJNOMKQTTTSDNCVANCNFSM46NLUHWQ>
.
|
_partition_by: &[Arc<dyn PhysicalExpr>], | ||
_order_by: &[PhysicalSortExpr], | ||
_window_frame: WindowFrame, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: once this is merged, it would be cool to add a link to this code in the respective issues that are tracking these missing features to make it easier for new contributors to start contributing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just realized that this PR changes the behavior when there is a non-empty partition_by/order_by/window_frame: before (at least in ballista) it would error, whereas now it is silently ignored. Maybe it is worth erroring if they aren't empty to make it explicit that there is no support?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it'll be further developed in #520
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but that's a good point let me add guard here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is fixed now
d0f2804
to
45b87f9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. I pulled the branch locally and ran the integration tests.
Which issue does this PR close?
Closes #.
Rationale for this change
reuse datafusion physical planner in ballista building from protobuf.
so far there are duplicated code within ballista and datafusion planner because the latter needs to handle alias but for ballista the expr name is flattened in protobuf, resulting two similar codepath in building the aggregation exec.
this tries to unify both and reuse code.
What changes are included in this PR?
Are there any user-facing changes?