-
Notifications
You must be signed in to change notification settings - Fork 28.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-11677][SQL] ORC filter tests all pass if filters are actually not pushed down. #9687
Conversation
Test build #45845 has finished for PR 9687 at commit
|
Test build #45965 has finished for PR 9687 at commit
|
Several things to mention. Firstly, I wonder if it is okay to put Secondly, I originally wanted to add |
cc @liancheng |
In this commit, I renamed the function |
Test build #47395 has finished for PR 9687 at commit
|
The function I mentioned is moved to |
In the commits above, I removed the |
retest this please |
Test build #47785 has finished for PR 9687 at commit
|
Thanks, merging to master. |
Currently ORC filters are not tested properly. All the tests pass even if the filters are not pushed down or disabled. In this PR, I add some logics for this.
Since ORC does not filter record by record fully, this checks the count of the result and if it contains the expected values.