-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve the DruidRexExecutor w.r.t handling of numeric arrays #11968
Improve the DruidRexExecutor w.r.t handling of numeric arrays #11968
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 could you add a test for this in https://github.com/apache/druid/blob/master/sql/src/test/java/org/apache/druid/sql/calcite/planner/DruidRexExecutorTest.java
Added the testcases, and fixed the older incorrect ones which were known. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
.map(BigDecimal::valueOf) | ||
.collect(Collectors.toList()); | ||
literal = rexBuilder.makeLiteral(resultAsBigDecimalList, constExp.getType(), true); | ||
} else { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would be better if you can add a check for double type here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An else case is required to stop the "Variable might not have been initialized" exception from cropping up, and I didn't want to provide a failsafe else block. In case when it's not an array (https://github.com/LakshSingla/druid/blob/4dcdb2d68563302a9b4d387cdbb220ec4f3a59b3/sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidRexExecutor.java#L130), it was using the else
case as well, that's why I omitted the else if
.
One approach can be to initialize the literal
to null and then including your suggestion. Let me know if that seems better, and I can add that in a separate commit.
Description
Issue:
DruidRexExecutor
while reducing Arrays, specially numeric arrays, doesn't convert the value from ExprResult's type to BigDecimal, which causesmakeLiteral
to cast the values. Also, if NaN or Infinite values are present in the array, the error is a genericNumberFormatException
. For example:SELECT ARRAY[1.11, 2.22]
returns [1, 2]SELECT SQRT(-1)
throws a genericNumberFormatException
instead ofIAE
This PR introduces change to cast the numeric values to
BigDecimal
since Calcite's library understands that easily, and doesn't perform casts.Key changed/added classes in this PR
DruidRexExecutor#reduce()
This PR has: