-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Register get_field by default #11959
Conversation
This makes sense in principle, as all other core udfs are register by default in the context. It also has a practical use, which is executing logical plans that have field access already de-sugared into get_field invocations.
I think the reason that datafusion/datafusion/functions-nested/src/planner.rs Lines 145 to 147 in 7a1a23d
|
That's right, this is not for end-users writing SQL queries. My use case is deserializing and executing logical plans that have |
This came up in the ASF slack channel too: https://the-asf.slack.com/archives/C04RJ0C85UZ/p1723558402071579 Thus that is two users so I think this is a good idea to add |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @leoyvens and @jayzhan211
This change makes sense to me given the usecase.
@@ -94,6 +94,7 @@ pub fn functions() -> Vec<Arc<ScalarUDF>> { | |||
nvl2(), | |||
arrow_typeof(), | |||
named_struct(), | |||
get_field(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might help to add the context of why this function is exported. Perhaps something like
get_field(), | |
// Note most users invoke `get_field` indirectly via field access syntax | |
// like `my_struct_col['field_name']` results in a call to `get_field(my_struct_col, "field_name")` | |
// It is also exposed directly for use cases such as serializing / deserializing plans with | |
// the field access desugared to calls to `get_field` | |
get_field(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added in follow on PR: #11996
This makes sense in principle, as all other core udfs are register by default in the context.
It also has a practical use, which is executing logical plans that have field access already de-sugared into get_field invocations.
Are these changes tested?
Yes
Are there any user-facing changes?
This is technically user-facing, though there aren't a lot of use cases for actually calling
get_field
.