Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-15582: [C++] Add support for registering tricky functions with the Substrait consumer (or add a bunch of substrait meta functions) #13285

Draft
wants to merge 17 commits into
base: main
Choose a base branch
from
Draft

ARROW-15582: [C++] Add support for registering tricky functions with the Substrait consumer (or add a bunch of substrait meta functions) #13285

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
7830d16
feat: compute functions mapping for substrait
sanjibansg May 30, 2022
91740f8
feat: using function mappings in extension set
sanjibansg Jun 1, 2022
5cbdcf5
feat: variadic boolean function mapping
sanjibansg Jun 2, 2022
ffc77d9
remove: enum for all function mappings
sanjibansg Jun 2, 2022
754b3a3
feat: Get functions to return status invalid if function not found
sanjibansg Jun 2, 2022
23f37f5
feat: registration api to return invalid status if function already r…
sanjibansg Jun 2, 2022
b28e5b1
feat: helpers for conversion functions
sanjibansg Jun 3, 2022
3b939c1
feat: support for subtract and multiply functions
sanjibansg Jun 3, 2022
76891e6
feat: mapped modulus function
sanjibansg Jun 3, 2022
99cab94
feat: remaining comparison function mapping
sanjibansg Jun 3, 2022
a27e03f
feat: mappings for string functions
sanjibansg Jun 5, 2022
93c16b5
feat: mapping for cast function
sanjibansg Jun 5, 2022
7e27a7f
feat: mapping for datetime function
sanjibansg Jun 9, 2022
d0f8ff3
feat: mappings for divide functions
sanjibansg Jun 9, 2022
a716c30
feat: mappings for sum & avg aggregate functions
sanjibansg Jun 13, 2022
56cbfff
feat: register functions to use function mappings
sanjibansg Jun 14, 2022
3f39eae
review changes
sanjibansg Jun 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 11 additions & 14 deletions cpp/src/arrow/engine/substrait/expression_internal.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
#include "arrow/util/make_unique.h"
#include "arrow/visit_scalar_inline.h"


namespace arrow {

using internal::checked_cast;
Expand Down Expand Up @@ -159,21 +160,17 @@ Result<compute::Expression> FromProto(const substrait::Expression& expr,

ARROW_ASSIGN_OR_RAISE(auto decoded_function,
ext_set.DecodeFunction(scalar_fn.function_reference()));
ARROW_ASSIGN_OR_RAISE(auto arrow_function, ext_set.GetFunctionMap().GetArrowFromSubstrait(decoded_function.name.to_string()));
return arrow_function(scalar_fn);
sanjibansg marked this conversation as resolved.
Show resolved Hide resolved
}

std::vector<compute::Expression> arguments(scalar_fn.args_size());
for (int i = 0; i < scalar_fn.args_size(); ++i) {
ARROW_ASSIGN_OR_RAISE(arguments[i], FromProto(scalar_fn.args(i), ext_set));
}

auto func_name = decoded_function.name.to_string();
if (func_name != "cast") {
return compute::call(func_name, std::move(arguments));
} else {
ARROW_ASSIGN_OR_RAISE(auto output_type_desc,
FromProto(scalar_fn.output_type(), ext_set));
auto cast_options = compute::CastOptions::Safe(std::move(output_type_desc.first));
return compute::call(func_name, std::move(arguments), std::move(cast_options));
}
case substrait::Expression::kEnum: {
auto enum_expr = expr.enum_();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this convert to the string value of the enum? Can you add a small comment here explaining that.

if(enum_expr.has_specified()){
return compute::literal(std::move(enum_expr.specified()));
} else {
return Status::Invalid("Substrait Enum value not specified");
}
}

default:
Expand Down
Loading