Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ES|QL] Improve STATS command summary extraction #199796

Merged
merged 1 commit into from
Nov 13, 2024

Conversation

vadimkibana
Copy link
Contributor

@vadimkibana vadimkibana commented Nov 12, 2024

Summary

Partially addresses #191812

  • Correctly extracts summary of fields from the BY clause of STATS command.
  • The .summarize() command now returns newFields and usedFields properties. The newFields is a list of newly created fields by the STATS command. The usedFields is a list of all fields which were used by the STATS command.
  • Improves parameter node handling.

Example

Extract all "new" and "used" fields from all STATS commands:

const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b), agg(c) BY d');
const summary = mutate.commands.stats.summarize(query);

console.log(summary.newFields);     // [ 'a', '`agg(c)`' ]
console.log(summary.usedFields);    // [ 'b', 'c', 'd' ]

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@vadimkibana vadimkibana added review release_note:skip Skip the PR/issue when compiling release notes v9.0.0 backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Feature:ES|QL ES|QL related features in Kibana Team:ESQL ES|QL related features in Kibana v8.17.0 labels Nov 12, 2024
@vadimkibana vadimkibana requested a review from a team as a code owner November 12, 2024 13:59
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-esql (Team:ESQL)

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
kbnUiSharedDeps-srcJs 3.4MB 3.4MB +185.0B

const summary = mutate.commands.stats.summarize(query);

console.log(summary.newFields); // [ 'a', '`agg(c)`' ]
console.log(summary.usedFields); // [ 'b', 'c', 'd' ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😍

Copy link
Contributor

@stratoula stratoula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really awesome Vadim! LGTM

@vadimkibana vadimkibana merged commit d276b48 into elastic:main Nov 13, 2024
36 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11813707821

@vadimkibana
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

vadimkibana added a commit to vadimkibana/kibana that referenced this pull request Nov 13, 2024
## Summary

Partially addresses elastic#191812

- Correctly extracts summary from of fields from the `BY` clause of
`STATS` command.
- The `.summarize()` command now returns `newFields` and `usedFields`
properties. The `newFields` is a list of newly created fields by the
`STATS` command. The `usedFields` is a list of all fields which were
used by the `STATS` command.
- Improves parameter node handling.

### Example

Extract all "new" and "used" fields from all `STATS` commands:

```ts
const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b), agg(c) BY d');
const summary = mutate.commands.stats.summarize(query);

console.log(summary.newFields);     // [ 'a', '`agg(c)`' ]
console.log(summary.usedFields);    // [ 'b', 'c', 'd' ]
```

### Checklist

Delete any items that are not applicable to this PR.

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

### For maintainers

- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)

(cherry picked from commit d276b48)

# Conflicts:
#	packages/kbn-esql-ast/src/mutate/commands/stats/index.test.ts
#	packages/kbn-esql-ast/src/mutate/commands/stats/index.ts
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 15, 2024
@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

1 similar comment
@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

@stratoula stratoula removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 18, 2024
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Nov 18, 2024
## Summary

Partially addresses elastic#191812

- Correctly extracts summary from of fields from the `BY` clause of
`STATS` command.
- The `.summarize()` command now returns `newFields` and `usedFields`
properties. The `newFields` is a list of newly created fields by the
`STATS` command. The `usedFields` is a list of all fields which were
used by the `STATS` command.
- Improves parameter node handling.


### Example

Extract all "new" and "used" fields from all `STATS` commands:

```ts
const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b), agg(c) BY d');
const summary = mutate.commands.stats.summarize(query);

console.log(summary.newFields);     // [ 'a', '`agg(c)`' ]
console.log(summary.usedFields);    // [ 'b', 'c', 'd' ]
```


### Checklist

Delete any items that are not applicable to this PR.

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios


### For maintainers

- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
vadimkibana added a commit that referenced this pull request Nov 18, 2024
…199970)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ES|QL] Improve `STATS` command summary extraction
(#199796)](#199796)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Vadim
Kibana","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-13T08:48:05Z","message":"[ES|QL]
Improve `STATS` command summary extraction (#199796)\n\n##
Summary\r\n\r\nPartially addresses
https://github.com/elastic/kibana/issues/191812\r\n\r\n- Correctly
extracts summary from of fields from the `BY` clause of\r\n`STATS`
command.\r\n- The `.summarize()` command now returns `newFields` and
`usedFields`\r\nproperties. The `newFields` is a list of newly created
fields by the\r\n`STATS` command. The `usedFields` is a list of all
fields which were\r\nused by the `STATS` command.\r\n- Improves
parameter node handling.\r\n\r\n\r\n### Example\r\n\r\nExtract all
\"new\" and \"used\" fields from all `STATS`
commands:\r\n\r\n```ts\r\nconst query = EsqlQuery.fromSrc('FROM index |
STATS a = max(b), agg(c) BY d');\r\nconst summary =
mutate.commands.stats.summarize(query);\r\n\r\nconsole.log(summary.newFields);
// [ 'a', '`agg(c)`' ]\r\nconsole.log(summary.usedFields); // [ 'b',
'c', 'd' ]\r\n```\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that
are not applicable to this PR.\r\n\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n\r\n### For
maintainers\r\n\r\n- [x] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)","sha":"d276b4899533b0bde2038c7fedfed7c30a74a80b","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["review","release_note:skip","v9.0.0","backport:prev-minor","Feature:ES|QL","Team:ESQL","v8.17.0"],"number":199796,"url":"https://github.com/elastic/kibana/pull/199796","mergeCommit":{"message":"[ES|QL]
Improve `STATS` command summary extraction (#199796)\n\n##
Summary\r\n\r\nPartially addresses
https://github.com/elastic/kibana/issues/191812\r\n\r\n- Correctly
extracts summary from of fields from the `BY` clause of\r\n`STATS`
command.\r\n- The `.summarize()` command now returns `newFields` and
`usedFields`\r\nproperties. The `newFields` is a list of newly created
fields by the\r\n`STATS` command. The `usedFields` is a list of all
fields which were\r\nused by the `STATS` command.\r\n- Improves
parameter node handling.\r\n\r\n\r\n### Example\r\n\r\nExtract all
\"new\" and \"used\" fields from all `STATS`
commands:\r\n\r\n```ts\r\nconst query = EsqlQuery.fromSrc('FROM index |
STATS a = max(b), agg(c) BY d');\r\nconst summary =
mutate.commands.stats.summarize(query);\r\n\r\nconsole.log(summary.newFields);
// [ 'a', '`agg(c)`' ]\r\nconsole.log(summary.usedFields); // [ 'b',
'c', 'd' ]\r\n```\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that
are not applicable to this PR.\r\n\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n\r\n### For
maintainers\r\n\r\n- [x] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)","sha":"d276b4899533b0bde2038c7fedfed7c30a74a80b"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/199796","number":199796,"mergeCommit":{"message":"[ES|QL]
Improve `STATS` command summary extraction (#199796)\n\n##
Summary\r\n\r\nPartially addresses
https://github.com/elastic/kibana/issues/191812\r\n\r\n- Correctly
extracts summary from of fields from the `BY` clause of\r\n`STATS`
command.\r\n- The `.summarize()` command now returns `newFields` and
`usedFields`\r\nproperties. The `newFields` is a list of newly created
fields by the\r\n`STATS` command. The `usedFields` is a list of all
fields which were\r\nused by the `STATS` command.\r\n- Improves
parameter node handling.\r\n\r\n\r\n### Example\r\n\r\nExtract all
\"new\" and \"used\" fields from all `STATS`
commands:\r\n\r\n```ts\r\nconst query = EsqlQuery.fromSrc('FROM index |
STATS a = max(b), agg(c) BY d');\r\nconst summary =
mutate.commands.stats.summarize(query);\r\n\r\nconsole.log(summary.newFields);
// [ 'a', '`agg(c)`' ]\r\nconsole.log(summary.usedFields); // [ 'b',
'c', 'd' ]\r\n```\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that
are not applicable to this PR.\r\n\r\n-
[x]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n\r\n### For
maintainers\r\n\r\n- [x] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)","sha":"d276b4899533b0bde2038c7fedfed7c30a74a80b"}},{"branch":"8.x","label":"v8.17.0","labelRegex":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: streamich <[email protected]>
Co-authored-by: Stratoula Kalafateli <[email protected]>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Nov 18, 2024
## Summary

Partially addresses elastic#191812

- Correctly extracts summary from of fields from the `BY` clause of
`STATS` command.
- The `.summarize()` command now returns `newFields` and `usedFields`
properties. The `newFields` is a list of newly created fields by the
`STATS` command. The `usedFields` is a list of all fields which were
used by the `STATS` command.
- Improves parameter node handling.


### Example

Extract all "new" and "used" fields from all `STATS` commands:

```ts
const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b), agg(c) BY d');
const summary = mutate.commands.stats.summarize(query);

console.log(summary.newFields);     // [ 'a', '`agg(c)`' ]
console.log(summary.usedFields);    // [ 'b', 'c', 'd' ]
```


### Checklist

Delete any items that are not applicable to this PR.

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios


### For maintainers

- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Feature:ES|QL ES|QL related features in Kibana release_note:skip Skip the PR/issue when compiling release notes review Team:ESQL ES|QL related features in Kibana v8.17.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants