Move code to fold Stable functions like now() from Simplifier to ConstEvaluator
#1176
Conversation
|
|
||
| match plan { | ||
| LogicalPlan::Filter { predicate, input } => Ok(LogicalPlan::Filter { |
There was a problem hiding this comment.
There is no reason to special case LogicalPlan::Filter as the predicate is handled by LogicalPlan::expressions -- and if you look carefully this doesn't call rewrite using the const_evaluator (I totally missed this in #1153 ) but found it while updating tests in this PR
| @@ -228,15 +220,6 @@ impl<'a> ExprRewriter for Simplifier<'a> { | |||
| Expr::Not(inner) | |||
| } | |||
| } | |||
| // convert now() --> the time in `ExecutionProps` | |||
There was a problem hiding this comment.
The point of this PR is to remove this code (it is now handled by ConstEvaluator)
| @@ -753,27 +732,6 @@ mod tests { | |||
| } | |||
| } | |||
|
|
|||
| #[test] | |||
There was a problem hiding this comment.
covered in the tests for ConstEvaluator
| let expected = "Filter: TimestampNanosecond(1599566400000000000) < CAST(totimestamp(Utf8(\"2020-09-08T12:05:00+00:00\")) AS Int64) + Int32(50000)\ | ||
| // Note that constant folder runs and folds the entire | ||
| // expression down to a single constant (true) | ||
| let expected = "Filter: Boolean(true)\ |
There was a problem hiding this comment.
The filter expression has been totally simplified 🎉
| @@ -838,17 +796,16 @@ mod tests { | |||
| // now() < cast(to_timestamp(...) as int) + 5000000000 | |||
| let plan = LogicalPlanBuilder::from(table_scan) | |||
| .filter( | |||
| now_expr() | |||
| cast_to_int64_expr(now_expr()) | |||
There was a problem hiding this comment.
It will be pretty awesome when we get #194 so casting to do timestamp arithmetic is no longer needed
| // To evaluate stable functions, need ExecutionProps, see | ||
| // Simplifier for code that does that. | ||
| Volatility::Stable => false, | ||
| // Values for functions such as now() are taken from ExecutionProps |
There was a problem hiding this comment.
Here is the actual change that allows the const evaluator to replace now() with a constant.
alamb
changed the title
now() from Simplifier to ConstEvaluatornow() from Simplifier to ConstEvaluator
alamb
force-pushed
the
alamb/move_scalar
branch
from
75d0b21
to
8c1d856
Compare
alamb
changed the title
now() from Simplifier to ConstEvaluatornow() from Simplifier to ConstEvaluator
|
This one is now ready for review @rdettai @houqp and @Dandandan |
Which issue does this PR close?
Resolves #1175
Rationale for this change
This is a follow on suggestion from @rdettai on #1153 https://github.com/apache/arrow-datafusion/pull/1153/files#r735437280
Namely the substitution of
now()for the current value is better described as "constant evaluation" rather than "algebraic simplification" so it should be done in theConstEvaluatorcode.It also has the very nice property that expressions that include
now()can also be more completely evaluated (will comment inline)What changes are included in this PR?
now()fromSimplifier()toConstEvaluatorAre there any user-facing changes?
MOAR constant folding!