Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve filter predicates with Utf8View literals #10998

Closed
Tracked by #10918
alamb opened this issue Jun 19, 2024 · 2 comments
Closed
Tracked by #10918

Improve filter predicates with Utf8View literals #10998

alamb opened this issue Jun 19, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@alamb
Copy link
Contributor

alamb commented Jun 19, 2024

Part of #10918

Is your feature request related to a problem or challenge?

DataFusion's predicate creation doesn't work as ideally for Utf8View constants

Specifically, I added a test i #10997

explain SELECT column1_utf8 from test where column1_utf8 = arrow_cast('Andrew', 'Utf8View');
logical_plan
01)Filter: CAST(test.column1_utf8 AS Utf8View) = Utf8View("Andrew")
02)--TableScan: test projection=[column1_utf8]

This plan is non ideal as it casts the column rather than the constant

Describe the solution you'd like

Instead of

CAST(test.column1_utf8 AS Utf8View) = Utf8View("Andrew")

the filter should be

test.column1_utf8 =  Utf8("Andrew")

So the expected plan should look like

explain SELECT column1_utf8 from test where column1_utf8 = arrow_cast('Andrew', 'Utf8View');
logical_plan
01)Filter: test.column1_utf8 = Utf8("Andrew")
02)--TableScan: test projection=[column1_utf8]

Describe alternatives you've considered

No response

Additional context

No response

@doki23
Copy link
Contributor

doki23 commented Jun 27, 2024

I wonder why #11043 isn't able to close this issue.

@alamb
Copy link
Contributor Author

alamb commented Jun 27, 2024

I wonder why #11043 isn't able to close this issue.

I agree -- #11043 should have closed it -- thank you @doki23

@alamb alamb closed this as completed Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants