-
Notifications
You must be signed in to change notification settings - Fork 222
Made GroupFilter
optional in parquet'sRecordReader
and added method to set it.
#386
Conversation
400410a
to
38afde4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we also expose RecordReader.metadata
so it can be used to build the row group filter?
Codecov Report
@@ Coverage Diff @@
## main #386 +/- ##
==========================================
+ Coverage 81.12% 81.24% +0.11%
==========================================
Files 331 331
Lines 21957 22215 +258
==========================================
+ Hits 17813 18048 +235
- Misses 4144 4167 +23
Continue to review full report at Codecov.
|
done 👍 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
RecordReader
and added method to set it.GroupFilter
optional in parquet'sRecordReader
and added method to set it.
I am using git commit in my local arrow2 migration branch at the moment, but good to know we could release at a much faster cadence :D |
This is more expressive since filtering is optional.
Backward incompatible changes
The function
RecordReader::try_new
now expectsOption<GroupFilter>
instead ofGroupFilter
. If you were not using filtering, migrate to this by passingNone
. If you were filtering, passSome(...)
instead.