Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transform] Add data frame scripted fields for group_by fields #43152

Closed
blaklaybul opened this issue Jun 12, 2019 · 1 comment · Fixed by #53167
Closed

[Transform] Add data frame scripted fields for group_by fields #43152

blaklaybul opened this issue Jun 12, 2019 · 1 comment · Fixed by #53167

Comments

@blaklaybul
Copy link

blaklaybul commented Jun 12, 2019

It would be useful to be able to create a group by field upon data frame creation with the use of a painless script, as in the following example:

"pivot": {
   "group_by": {
      "foo_bar": {
         "terms": {
            "script": {
               "source": "ctx._source.foo +'-' + ctx._source.bar",
               "lang": "painless"
                       } 
                    }
                }
            }
         }

This would enable multindex-style groupby operations, and having the individual fields acted on by the script available in the data frame would allow for multindex-style querying as well.

e.g. (pandas):

gb = df.groupby(['foo', 'bar']).agg({'baz': 'mean'})
gb.loc['foo_1'] -> returns all rows indexed by 'foo_1' value of 'foo'
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core

@sophiec20 sophiec20 changed the title Scripted Fields for Group By Fields [ML] Scripted Fields for Group By Fields Jun 13, 2019
@sophiec20 sophiec20 changed the title [ML] Scripted Fields for Group By Fields [ML] Add data frame scripted fields for group_by fields Jun 13, 2019
hendrikmuhs pushed a commit that referenced this issue Mar 9, 2020
add the possibility to base the group_by on the output of a script.

closes #43152
@hendrikmuhs hendrikmuhs changed the title [ML] Add data frame scripted fields for group_by fields [Transform] Add data frame scripted fields for group_by fields Mar 9, 2020
hendrikmuhs pushed a commit to hendrikmuhs/elasticsearch that referenced this issue Mar 10, 2020
add the possibility to base the group_by on the output of a script.

closes elastic#43152
hendrikmuhs pushed a commit that referenced this issue Mar 10, 2020
add the possibility to base the group_by on the output of a script.

closes #43152
backport #53167
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants