Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parquet Filtering #687

Open
norberttech opened this issue Oct 31, 2023 · 0 comments
Open

Parquet Filtering #687

norberttech opened this issue Oct 31, 2023 · 0 comments

Comments

@norberttech
Copy link
Member

Parquet comes with very handy mechanism called "Column Statistics" which says for example what are the min/max values, total number of null values etc.

By reading those statistics we won't need to iterate through the entire parquet file when for example we are looking for a data from a specific time range or value range.

@norberttech norberttech converted this from a draft issue Oct 31, 2023
@norberttech norberttech added this to the 0.6.0 milestone Nov 6, 2023
@norberttech norberttech modified the milestones: 0.6.0, 0.7.0 Jan 27, 2024
@norberttech norberttech removed this from the 0.7.0 milestone Mar 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Development

No branches or pull requests

1 participant