Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ESQL] copy ql disjunction rule into esql #106499

Merged

Conversation

not-napoleon
Copy link
Member

Similar to #105711, this migrates a QL optimization that depends on specific class types into ESQL to refer to the ESQL version of those classes. This is necessary for #105217, which changes the class hierarchy of the binary comparisons.

In this case, I didn't find any tests that exercise this optimization directly, although a failure in PhysicalPlanOptimizerTests alerted me to the problem. To that end, I've added three tests to directly cover this optimization.

From the look of it, there's several more logical planner level optimizations that rely on the exact type of the comparison classes, so I'm going to migrate those as well. For ease of review, I figured I'd do those in separate PRs.

@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Mar 19, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

List<Expression> ors = new LinkedList<>();

for (Expression exp : exps) {
if (exp instanceof Equals eq) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, the key change here is that this now refers to esql's Equals, not the ql base Equals.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could minimize the change by adding a templating method in the basic QL rule and overriding this in ESQL:

for (Expression exp : exps) {
   if (isEquals(exp)) {
     ....
   }
}

public boolean isEquals(Expression exp) {
        return exp instanceof QlEquals;
}

}

// rule in ESQL
public CombineDisjunctionsIn... extends ql.CombineDisjunctions {
      @Override
      protected boolean isEquals(Expression ex) {
             return ex instanceof EsqlEquals;
      }
}

@@ -1138,6 +1145,57 @@ protected In createIn(Expression key, List<Expression> values, ZoneId zoneId) {
protected Equals createEquals(Expression k, Set<Expression> v, ZoneId finalZoneId) {
return new Equals(k.source(), k, v.iterator().next(), finalZoneId);
}

@Override
protected Expression rule(Or or) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method copied in from OptimizerRules

@not-napoleon
Copy link
Member Author

Looking at it, there's a lot of tests in OptimizerRulesTests that we should port to ES|QL. Notably, those tests should have caught the error that prompted this (from #105217), but didn't because they don't actually exercise the ESQL classes. Porting those tests seems like a good idea, and I'm open to doing it in this PR or opening an issue to do it as a follow up.

Copy link
Member

@costin costin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the change is small, this could be simplified by adding an overriding templating method.

@not-napoleon not-napoleon requested a review from costin March 19, 2024 22:45
* This rule does NOT check for type compatibility as that phase has been
* already be verified in the analyzer.
*/
public static class CombineDisjunctionsToIn extends OptimizerRules.CombineDisjunctionsToIn {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to OptimizerRules (ESQL version) to mirror the QL class hierarchy

}

@Override
protected Expression rule(Or or) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rule logic copied from QL version, but now pointing to ESQL versions of the classes

@@ -3441,15 +3475,6 @@ private void assertNullLiteral(Expression expression) {
assertNull(expression.fold());
}

// TODO: move these from org.elasticsearch.xpack.ql.optimizer.OptimizerRulesTests to org.elasticsearch.xpack.ql.TestUtils
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Member

@costin costin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Porting those tests seems like a good idea, and I'm open to doing it in this PR or opening an issue to do it as a follow up.

👍 whatever you prefer.

@not-napoleon
Copy link
Member Author

Actually, I did port the tests when I did the class hierarchy revision you asked for. They're in the ESQL version of OptimizerRuleTests, which mirrors where they are in QL.

@not-napoleon not-napoleon merged commit f617694 into elastic:main Mar 21, 2024
14 checks passed
@lkts lkts mentioned this pull request Mar 21, 2024
not-napoleon added a commit that referenced this pull request Mar 22, 2024
Relates to #105217

This copies the PropagateEquals logical optimization into ESQL, following the pattern established in #106499. I've copied the optimization rule into the ESQL version of OptimizerRules, and the tests into OpitmizerRulesTests, and changed the imports &c to point to the appropriate ESQL classes instead of their QL counterparts.

I expect to have several more PRs following this pattern, for the remaining logical optimizations that touch the binary comparison logic. I'm intending to make separate PRs for each, in the interest of making them easier to review.
not-napoleon added a commit that referenced this pull request Mar 25, 2024
Relates to #105217

This copies the BooleanFunctionEqualsElimination logical optimization into ESQL, following the pattern established in #106499. I've copied the optimization rule into the ESQL version of OptimizerRules, and the tests into OpitmizerRulesTests, and changed the imports &c to point to the appropriate ESQL classes instead of their QL counterparts.

I only saw two tests for this one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.14.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants