Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESQL: Comparing a large long to an int field fails when pushed to source #99960

Closed
alex-spies opened this issue Sep 27, 2023 · 5 comments · Fixed by #99961
Closed

ESQL: Comparing a large long to an int field fails when pushed to source #99960

alex-spies opened this issue Sep 27, 2023 · 5 comments · Fixed by #99961
Assignees
Labels
:Analytics/ES|QL AKA ESQL >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Milestone

Comments

@alex-spies
Copy link
Contributor

alex-spies commented Sep 27, 2023

Elasticsearch Version

main@cc42ff62b30b266dd715ae6ab203b790dfc8fe23

Installed Plugins

No response

Java Version

openjdk version "17.0.8" 2023-07-18

OS Version

Ubuntu 23.04

Problem Description

Running from employees | where emp_no == 1000000000000 will push the filter to Lucene, which fails with "reason": "failed to create query: Value [1000000000000] is out of range for an integer"

This succeeds if the filter is not pushed down (with expected no results): from employees | eval x = emp_no == 1000000000000 | where x.

Steps to Reproduce

Run the query from above for an index employees with an Integer field emp_no.

Logs (if relevant)

No response

@alex-spies alex-spies self-assigned this Sep 27, 2023
@elasticsearchmachine elasticsearchmachine added the Team:QL (Deprecated) Meta label for query languages team label Sep 27, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-ql (Team:QL)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/elasticsearch-esql (:Query Languages/ES|QL)

@alex-spies
Copy link
Contributor Author

#99961 fixes this by preventing filter pushdown for filters unsupported in Lucene.

I will iterate on this to find similar unsupported filter pushdowns (and need to add more tests).

@alex-spies
Copy link
Contributor Author

There is an analogous issue for floating point data types. This is harder to solve, unfortunately, so I created a dedicated issue for it: #100130

alex-spies added a commit to alex-spies/elasticsearch that referenced this issue Oct 2, 2023
Prevent pushing down filters for binary comparisons that would
implicitly compare a byte/short/int with an out of range value. This
leads to exceptions thrown by Lucene - instead, evaluate the filter in
ESQL only.

This does not cover the same problem for half_float/scaled_float/float,
see elastic#100130.

Closes elastic#99960.
@costin costin added this to the 8.12 milestone Nov 16, 2023
@alex-spies alex-spies removed this from the 8.12 milestone Nov 21, 2023
@alex-spies alex-spies added this to the 8.13 milestone Dec 5, 2023
@wchaparro wchaparro added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jan 2, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

@elasticsearchmachine elasticsearchmachine removed the Team:QL (Deprecated) Meta label for query languages team label Jan 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants