Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handle nulls in file-level stats #1520

Merged
merged 2 commits into from
Jul 9, 2023

Conversation

wjones127
Copy link
Collaborator

Description

Fixes issue where predicate pushdown isn't working for null values. This adds tests for both columns and partition columns.

Related Issue(s)

Documentation

@github-actions github-actions bot added the binding/python Issues for the Python package label Jul 8, 2023
@wjones127
Copy link
Collaborator Author

Needed to upgrade the minimal PyArrow version, I think due to https://issues.apache.org/jira/browse/ARROW-12659

@wjones127 wjones127 marked this pull request as ready for review July 9, 2023 01:02
Copy link
Collaborator

@roeap roeap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@wjones127 wjones127 merged commit 155be62 into delta-io:main Jul 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/python Issues for the Python package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pyarrow is_null filter not working as expected after loading using deltalake
2 participants