Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Performance improvement of GpuBroadcastNestedLoopJoinExec #1657

Closed
sperlingxx opened this issue Feb 3, 2021 · 2 comments
Closed

[FEA] Performance improvement of GpuBroadcastNestedLoopJoinExec #1657

sperlingxx opened this issue Feb 3, 2021 · 2 comments
Labels
feature request New feature or request

Comments

@sperlingxx
Copy link
Collaborator

sperlingxx commented Feb 3, 2021

Is your feature request related to a problem? Please describe.
The performance of GpuBroadcastNestedLoopJoinExec is not ideal. If I am not wrong, the major bottleneck is innerLikeJoin, which will firstly materialize the full Cartesian Product's result of two input tables. Then, filter the product table with bound conditions. For spark runtime, above two steps are chained in streaming style, avoiding huge cost of materializing the full product table.

Describe the solution you'd like
A naive solution is to split streaming table into several sub-tables. Then, conduct innerLikeJoin on each of them. Finally, concatenate sub-results to get the final result.

@sperlingxx sperlingxx added feature request New feature or request ? - Needs Triage Need team to review and classify labels Feb 3, 2021
@revans2
Copy link
Collaborator

revans2 commented Feb 3, 2021

This is related to #302

I'm not sure if we need both of them.

@sperlingxx
Copy link
Collaborator Author

@revans2 Sorry, I didn't realize we've already had an issue. I closed this one.

@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Feb 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants