-
Notifications
You must be signed in to change notification settings - Fork 236
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GpuAdd supports ANSI mode. #3537
Conversation
And let GPU Multiply fall back to CPU when ansi mode is enabled. Signed-off-by: Firestarman <[email protected]>
build |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a few nits that can be handled by follow on issues.
ShortGen(min_val = 1, max_val = 100, special_cases=[]), | ||
IntegerGen(min_val = 1, max_val = 1000, special_cases=[]), | ||
LongGen(min_val = 1, max_val = 3000, special_cases=[]), | ||
float_gen, double_gen, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I think we can have a follow on issue to re-enable float, double, and decimal processing in ANSI mode. Each of them handle overflows already in different ways that should be okay.
@@ -1380,7 +1380,7 @@ object GpuOverrides extends Logging { | |||
ExprChecks.mathUnary, | |||
(a, conf, p, r) => new UnaryExprMeta[Log1p](a, conf, p, r) { | |||
override def convertToGpu(child: Expression): GpuExpression = | |||
GpuLog(GpuAdd(child, GpuLiteral(1d, DataTypes.DoubleType))) | |||
GpuLog(GpuAdd(child, GpuLiteral(1d, DataTypes.DoubleType), SQLConf.get.ansiEnabled)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: This is a little misleading, because there is no fail on error overflow processing on doubles, and doubles are the only thing supported by Log1p
|
||
override def tagSelfForAst(): Unit = { | ||
super.tagSelfForAst() | ||
if (ansiEnabled) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: It would be good to check the data type, because double and float don't have special overflow processing and should be able to work in AST mode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small capitalization nits but otherwise lgtm.
override def tagSelfForAst(): Unit = { | ||
super.tagSelfForAst() | ||
if (ansiEnabled) { | ||
willNotWorkInAst("AST Addition does not support ansi mode.") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
willNotWorkInAst("AST Addition does not support ansi mode.") | |
willNotWorkInAst("AST Addition does not support ANSI mode.") |
@@ -62,6 +63,26 @@ def test_multiplication(data_gen): | |||
f.col('a') * f.col('b')), | |||
conf=allow_negative_scale_of_decimal_conf) | |||
|
|||
# No overflow gens here because we just focus on verifying the fallback to CPU when | |||
# enabling ansi mode. But overflows will fail the tests because CPU runs raise |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# enabling ansi mode. But overflows will fail the tests because CPU runs raise | |
# enabling ANSI mode. But overflows will fail the tests because CPU runs raise |
@@ -1709,6 +1718,10 @@ object GpuOverrides extends Logging { | |||
} | |||
case _ => // NOOP | |||
} | |||
|
|||
if (SQLConf.get.ansiEnabled) { | |||
willNotWorkOnGpu("GPU Multiplication does not support ansi mode") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
willNotWorkOnGpu("GPU Multiplication does not support ansi mode") | |
willNotWorkOnGpu("GPU Multiplication does not support ANSI mode") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll do this in a follow on issue as @firestarman is now out on vacation.
And let Multiply fall back to CPU when ANSI mode is enabled.
This closes #3472
Signed-off-by: Firestarman [email protected]