-
Notifications
You must be signed in to change notification settings - Fork 902
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Fix orc reader RLEv2 reader #5473
Conversation
Patch bit width is only allowed to be from a fixed set of values. Patch width is to be selected as the smallest value from the set that fit the required patch size (pw + pgw)
Fixes the narrowing conversion in bytestream reading in patched RLE
Please update the changelog in order to start CI tests. View the gpuCI docs here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
python approval
Codecov Report
@@ Coverage Diff @@
## branch-0.15 #5473 +/- ##
===============================================
+ Coverage 88.65% 88.77% +0.12%
===============================================
Files 57 57
Lines 10945 11003 +58
===============================================
+ Hits 9703 9768 +65
+ Misses 1242 1235 -7
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At line 709, l += (pll * (pgw + pw) + 7) >> 3;
also needs to be modified to use ClosestFixedBitsMap[]
instead of pw + pgw
.
Also, we should probably use ClosestFixedBitsMap[min(pw + pgw, 64u)]
when indexing to prevent crashing on bad data.
Great job and thanks for the fix, @devavret |
Fixes #5440
Fixes #5324
Opened ORC-642 to track issue in apache's docs.