Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Push additional parquet filtering into the parquet scan [EPIC] #3147

Closed
4 of 5 tasks
alamb opened this issue Aug 15, 2022 · 1 comment
Closed
4 of 5 tasks

Push additional parquet filtering into the parquet scan [EPIC] #3147

alamb opened this issue Aug 15, 2022 · 1 comment
Labels
enhancement New feature or request

Comments

@alamb
Copy link
Contributor

alamb commented Aug 15, 2022

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The more filtering that can be pushed to the parquet reading, the faster a query will run in general as less work is needed to decode and process data that will eventually be filtered from the plan

There are several ongoing workstreams that will eventually lead to pushing down substantial additional filtering into the parquet scan that should substantially increase performance for datafusion. I wanted to capture them here to provide more visibility

cc @Ted-Jiang @tustvold @thinkharderdev

Describe the solution you'd like
Here are some of the tasks I have collected. There are likely more -- please add them (either directly or via comments)

@alamb alamb added the enhancement New feature or request label Aug 15, 2022
@alamb
Copy link
Contributor Author

alamb commented Oct 11, 2022

Duplicate of #3462

@alamb alamb marked this as a duplicate of #3462 Oct 11, 2022
@alamb alamb closed this as completed Oct 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant