Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESQL: Plumb a way to run phased plans #110445

Merged
merged 3 commits into from
Jul 5, 2024
Merged

Conversation

nik9000
Copy link
Member

@nik9000 nik9000 commented Jul 3, 2024

INLINESTATS is going to run two ESQL commands - one to get the STATS and one to join the stats results to the output. This plumbs a way for EsqlSession#execute to run multiple dips into the compute engine using a BiConsumer<PhysicalPlan, ActionListener<Result>> runPhase. For now, we just plug that right into the output to keep things working as they are now. But soon, so soon, we'll plug in a second phase.

INLINESTATS is going to run two ESQL commands - one to get the STATS and
one to join the stats results to the output. This plumbs a way for
`EsqlSession#execute` to run multiple dips into the compute engine using
a `BiConsumer<PhysicalPlan, ActionListener<Result>> runPhase`. For now,
we just plug that right into the output to keep things working as they
are now. But soon, so soon, we'll plug in a second phase.
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine elasticsearchmachine added Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.16.0 labels Jul 3, 2024
@nik9000 nik9000 mentioned this pull request Jul 3, 2024
Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks Nik!

Copy link
Contributor

@astefan astefan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@alex-spies alex-spies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!

import org.elasticsearch.xpack.esql.core.expression.Attribute;

import java.util.List;

public record Result(List<Attribute> columns, List<List<Object>> values) {}
public record Result(List<Attribute> layout, List<Page> pages, List<DriverProfile> profiles) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super nit: there's also a datatype Layout which is not quite just a list of attributes; we sometimes also use fields (schema would also be appropriate but isn't really used in our code base)

Copy link
Contributor

@ChrisHegarty ChrisHegarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noice! LGTM

@nik9000 nik9000 added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Jul 5, 2024
@nik9000
Copy link
Member Author

nik9000 commented Jul 5, 2024

Thanks friends!

@elasticsearchmachine elasticsearchmachine merged commit df24e4f into elastic:main Jul 5, 2024
15 checks passed
@nik9000 nik9000 deleted the phased branch July 5, 2024 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.16.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants