feat: support vectorized unary operators #5130
Conversation
onelson
force-pushed
the
onelson/feat/vectorized-unary
branch
from
b8ceb84
to
c48e0ff
Compare
| @@ -644,15 +644,91 @@ func (e *unaryEvaluator) Type() semantic.MonoType { | |||
| return e.t | |||
| } | |||
|
|
|||
| func doUnary(mt semantic.MonoType, op ast.OperatorKind, v values.Value) (values.Value, error) { | |||
There was a problem hiding this comment.
Lightly modified - this was formerly a closure defined inside the unary evaluator. Extracted so it could be shared with the vector version for the vec repeat case.
|
|
||
| /// Enables calls to map to be vectorized when the function contains | ||
| /// unary operators like: add, sub exists, not. | ||
| VectorizedUnaryOps, |
There was a problem hiding this comment.
It might be good to start generating Rust code for features at some point. (just an observation)
There was a problem hiding this comment.
Building the enum from the yaml file? Yeah, that's an interesting idea. It could be something we do via build.rs
| b.Resize(n) | ||
| for i := 0; i < n; i++ { | ||
| if v.IsValid(i) { | ||
| b.Append(-v.Value(i)) |
There was a problem hiding this comment.
I was curious what Go does if you try to negate math.MinInt64:
https://go.dev/play/p/h-QiLnQU8nz
I guess we inherit this behavior...
| switch op { | ||
| case ast.AdditionOperator: | ||
| // Do nothing. | ||
| return v, nil |
There was a problem hiding this comment.
I always wonder why this operator exists... at least the implementation is efficient...
There was a problem hiding this comment.
Yeah, I mean... 🤷
I'm just trying to stay faithful to the source material 🤣
| expect_test::expect![[r##" | ||
| (r) => { | ||
| return {r:{#C with _value: v[#B]} with add: +r:{#C with _value: v[#B]}._value:v[#B]:v[#B], sub: -r:{#C with _value: v[#B]}._value:v[#B]:v[#B]}:{#C with add: v[#B], sub: v[#B], _value: v[#B]} | ||
| }:(r: {#C with _value: v[#B]}) => {#C with add: v[#B], sub: v[#B], _value: v[#B]}"##]].assert_eq(&crate::semantic::formatter::format_node( |
There was a problem hiding this comment.
I'm curious how you like this way of verifying vectorized code, or if you think there's a better way.
There was a problem hiding this comment.
I have a very hard time reading the string here, so I'm glad it's generated for me. I doubt I could write this by hand correctly without much trial and error.
For me, the exercise mostly comes down to making sure I see a v[] around either a scalar type or a type var in the right field positions. Other than that, the test gives me pass/fail on whether or not the function expression can be vectorized.
Seems sufficient.
Fixes #5045
Adds support for vectorized unary operators:
add,sub,exists,not.Differences in the supported
Valuetypes between the regular unary and the vectorized fall along the lines of what can be represented as column data (which is the gateway to vectorized map) and what can be freely stored in arrow arrays.The notable omission here is Duration, which is commonly used with unary sub, ex:
-1d. Since this type cannot appear in tables, it cannot be mapped over using vectorization.The new code path is guarded by a feature flag:
vectorizedUnaryOps.Checklist
Dear Author 👋, the following checks should be completed (or explicitly dismissed) before merging.
experimental/docs/Spec.mdhas been updatedDear Reviewer(s) 👋, you are responsible (among others) for ensuring the completeness and quality of the above before approval.