-
Notifications
You must be signed in to change notification settings - Fork 28.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-8841] [SQL] Fix partition pruning percentage log message #7227
Conversation
Can one of the admins verify this patch? |
Jenkins, ok to test |
lgtm |
Jenkins, test this please. (Did something go wrong with Jenkins?) |
cc @liancheng |
ok to test |
Merged build triggered. |
@srlindemann Thanks for fixing this! Could you please file a JIRA and change the PR title to "[SPARK-xxxx] [SQL] Fix partition pruning percentage log message"? |
Is this even worth a JIRA? there is virtually no difference between the description and fix |
Merged build started. |
Test build #36567 has started for PR 7227 at commit |
Test build #36567 has finished for PR 7227 at commit
|
Merged build finished. Test PASSed. |
I saw on one of the contrib pages that trivial changes might not have a
|
@srowen Hm, yeah, now I'd agree that this might not worth a JIRA. Anyway, filing one to track it does no harm :) @srlindemann Thanks! Merging to master. |
When pruning partitions for a query plan, a message is logged indicating what how many partitions were selected based on predicate criteria, and what percent were pruned.
The current release erroneously uses
1 - total/selected
to compute this quantity, leading to nonsense messages like "pruned -1000% partitions". The fix is simple and obvious.