-
Notifications
You must be signed in to change notification settings - Fork 784
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add page index reader test for all types and support empty index. #2012
Conversation
Codecov Report
@@ Coverage Diff @@
## master #2012 +/- ##
==========================================
- Coverage 83.58% 83.55% -0.04%
==========================================
Files 222 222
Lines 57529 58059 +530
==========================================
+ Hits 48088 48510 +422
- Misses 9441 9549 +108
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking really nice 👍
Only some minor nits and then this can go in
Co-authored-by: Raphael Taylor-Davies <[email protected]>
@tustvold do you know this error in IT
|
See #1931 |
parquet/src/file/page_index/index.rs
Outdated
/// Sometimes reading page index from parquet file | ||
/// will only return pageLocations without min_max index, | ||
/// Use `EMPTY_ARRAY` representing None will be more convenient. | ||
EMPTY_ARRAY, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I would prefer None to indicate absence of index information. EMPTY_ARRAY makes me think of an empty byte array or something
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
make sense.
Co-authored-by: Raphael Taylor-Davies <[email protected]>
Which issue does this PR close?
Closes #2010.
Rationale for this change
After apache/parquet-testing#25 merged, there will be standard page index test file in parquet-testing.All types support page index in parquet-format has add test in this pr.
Another change:
I found there is a situation one col has pageLocation but without min_max index(😂 i missed it). So add EMPTY_ARRAY in enum Index.
What changes are included in this PR?
Are there any user-facing changes?