move the Translate, SubstrIndex, FindInSet functions to datafusion-functions #9864
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ailable feature in DataFusion and building with nightly may not be a good recommendation when getting started.
…_expressions feature flag, move char_length function
# Conflicts: # datafusion/expr/src/built_in_function.rs # datafusion/functions/src/unicode/mod.rs # datafusion/physical-expr/src/unicode_expressions.rs
…n-functions crate
# Conflicts: # datafusion/expr/src/built_in_function.rs # datafusion/expr/src/expr_fn.rs # datafusion/functions/src/unicode/left.rs # datafusion/functions/src/unicode/lpad.rs # datafusion/functions/src/unicode/mod.rs # datafusion/functions/src/unicode/reverse.rs # datafusion/functions/src/unicode/right.rs # datafusion/functions/src/unicode/rpad.rs # datafusion/functions/src/unicode/substr.rs # datafusion/physical-expr/src/functions.rs # datafusion/physical-expr/src/unicode_expressions.rs # datafusion/proto/src/generated/pbjson.rs # datafusion/proto/src/generated/prost.rs # datafusion/proto/src/logical_plan/from_proto.rs # datafusion/proto/src/logical_plan/to_proto.rs
github-actions
bot
added
logical-expr
alamb
approved these changes
Mar 30, 2024
| @@ -67,8 +67,6 @@ regex_expressions = [ | |||
| ] | |||
| serde = ["arrow-schema/serde"] | |||
| unicode_expressions = [ | |||
| "datafusion-physical-expr/unicode_expressions", | |||
| return Some(String::new()); | ||
| } | ||
|
|
||
| let splitted: Box<dyn Iterator<Item = _>> = if n > 0 { |
There was a problem hiding this comment.
we can probably make this significantly faster by avoiding the Box::new() here and the call to to_owned() below. I realize this is not something changed in this PR
There was a problem hiding this comment.
I'll create a separate benchmark for this.
|
Thanks again @Omega359 |
Lordworms
pushed a commit
to Lordworms/arrow-datafusion
that referenced
this pull request
Apr 1, 2024
…nctions (apache#9864) * Fix to_timestamp benchmark * Remove reference to simd and nightly build as simd is no longer an available feature in DataFusion and building with nightly may not be a good recommendation when getting started. * Fixed missing trim() function. * Create unicode module in datafusion/functions/src/unicode and unicode_expressions feature flag, move char_length function * move Left, Lpad, Reverse, Right, Rpad functions to datafusion_functions * move strpos, substr functions to datafusion_functions * move the Translate, SubstrIndex, FindInSet functions to new datafusion-functions crate * Test code cleanup * unicode_expressions Cargo.toml updates. --------- Co-authored-by: Andrew Lamb <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #9842
Rationale for this change
As part of #9285 the unicode functions should be migrated to the new datafusion-functions crate in the new structure
What changes are included in this PR?
Code, tests
Are these changes tested?
Yes
Are there any user-facing changes?
No,