[KQL Query] Create the ANTLR parser #114927

afoucret · 2024-10-16T16:20:30Z

Work done in the PR

Created the new kql plugin into x-pack
ANTLR parser:
- configure gradle to regenerate the parser from the grammar file by using the :x-pack:plugin:kql:regen task
- initial version of the Kql grammar
- generated parser code

elasticsearchmachine · 2024-10-16T16:20:54Z

Pinging @elastic/es-search (Team:Search)

afoucret · 2024-10-16T16:22:21Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlParser.java

+public class KqlParser {
+    private static final Logger log = LogManager.getLogger(KqlParser.class);
+
+    public QueryBuilder parseKqlQuery(String kqlQuery, SearchExecutionContext searchExecutionContext) {


ℹ️ This is the main entry point of the parser that we will use to implement the KqlQueryBuilder.

afoucret · 2024-10-16T16:24:27Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlAstBuilder.java

+import org.elasticsearch.index.query.QueryBuilder;
+import org.elasticsearch.index.query.SearchExecutionContext;
+
+class KqlAstBuilder extends KqlBaseBaseVisitor<QueryBuilder> {


ℹ️ The KqlAstBuilder will transform the AST into a query builder.

The actual implementation will come in a later PR (the current behavior is to always return a match_all query).

afoucret · 2024-10-16T16:25:54Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/ParsingException.java

+
+import static org.elasticsearch.common.logging.LoggerMessageFormat.format;
+
+public class ParsingException extends ElasticsearchException {


ℹ️ Using our own exception type for KQL parsing errors.

Should this be named something explicit like KqlParsingException?

afoucret · 2024-10-16T16:27:19Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlBaseListener.java

@@ -0,0 +1,206 @@
+// ANTLR GENERATED CODE: DO NOT EDIT


⚠️ Generated code

[ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseListener.java:1: Duplicate header javadocs are forbidden [MultipleHeaderJavadoc]

afoucret · 2024-10-16T16:27:29Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlBaseVisitor.java

@@ -0,0 +1,133 @@
+// ANTLR GENERATED CODE: DO NOT EDIT


⚠️ Generated code

afoucret · 2024-10-16T16:27:44Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlBaseLexer.java

@@ -0,0 +1,245 @@
+// ANTLR GENERATED CODE: DO NOT EDIT


⚠️ Generated code

afoucret · 2024-10-16T16:27:49Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlBaseLexer.interp

@@ -0,0 +1,71 @@
+token literal names:


⚠️ Generated code

afoucret · 2024-10-16T16:27:51Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlBaseBaseVisitor.java

@@ -0,0 +1,149 @@
+// ANTLR GENERATED CODE: DO NOT EDIT


⚠️ Generated code

afoucret · 2024-10-16T16:27:54Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlBaseBaseListener.java

@@ -0,0 +1,264 @@
+// ANTLR GENERATED CODE: DO NOT EDIT


⚠️ Generated code

this causes issues it seems in the windows builds.... not sure why.

[ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseBaseListener.java:1: Duplicate header javadocs are forbidden [MultipleHeaderJavadoc]

afoucret · 2024-10-16T16:27:57Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlBase.interp

@@ -0,0 +1,52 @@
+token literal names:


⚠️ Generated code

afoucret · 2024-10-16T16:28:02Z

x-pack/plugin/kql/src/main/antlr/KqlBaseLexer.tokens

@@ -0,0 +1,21 @@
+DEFAULT_SKIP=1


⚠️ Generated code

afoucret · 2024-10-16T16:28:07Z

x-pack/plugin/kql/src/main/antlr/KqlBase.tokens

@@ -0,0 +1,21 @@
+DEFAULT_SKIP=1


⚠️ Generated code

x-pack/plugin/kql/src/test/java/org/elasticsearch/xpack/kql/parser/KqlParserTests.java

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/KqlPlugin.java

x-pack/plugin/kql/src/test/java/org/elasticsearch/xpack/kql/parser/KqlParserTests.java

elasticsearchmachine · 2024-10-17T08:23:01Z

Pinging @elastic/es-search-relevance (Team:Search Relevance)

carlosdelest

A good first step for KQL!

elasticsearchmachine · 2024-10-18T07:59:43Z

💚 Backport successful

Status	Branch	Result
✅	8.x

javanna · 2024-10-21T07:01:13Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlBaseLexer.java

+import org.antlr.v4.runtime.*;
+import org.antlr.v4.runtime.atn.*;
+import org.antlr.v4.runtime.dfa.DFA;
+import org.antlr.v4.runtime.misc.*;


this imports don't follow the checkstyle rules of the repo

[ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseLexer.java:13:8: Unused import - org.antlr.v4.runtime.Token. [UnusedImports] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseLexer.java:14:8: Unused import - org.antlr.v4.runtime.TokenStream. [UnusedImports] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseLexer.java:15:28: Using the '.*' form of import should be avoided - org.antlr.v4.runtime.*. [AvoidStarImport] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseLexer.java:16:32: Using the '.*' form of import should be avoided - org.antlr.v4.runtime.atn.*. [AvoidStarImport] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseLexer.java:18:33: Using the '.*' form of import should be avoided - org.antlr.v4.runtime.misc.*. [AvoidStarImport]

javanna · 2024-10-21T07:02:03Z

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/parser/KqlBaseParser.java

+import org.antlr.v4.runtime.tree.*;
+import java.util.List;
+import java.util.Iterator;
+import java.util.ArrayList;


[ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseParser.java:1: Duplicate header javadocs are forbidden [MultipleHeaderJavadoc] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseParser.java:11:32: Using the '.*' form of import should be avoided - org.antlr.v4.runtime.atn.*. [AvoidStarImport] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseParser.java:13:28: Using the '.*' form of import should be avoided - org.antlr.v4.runtime.*. [AvoidStarImport] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseParser.java:14:33: Using the '.*' form of import should be avoided - org.antlr.v4.runtime.misc.*. [AvoidStarImport] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseParser.java:15:33: Using the '.*' form of import should be avoided - org.antlr.v4.runtime.tree.*. [AvoidStarImport] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseParser.java:17:8: Unused import - java.util.Iterator. [UnusedImports] [ant:checkstyle] [ERROR] C:\bk\x-pack\plugin\kql\src\main\java\org\elasticsearch\xpack\kql\parser\KqlBaseParser.java:18:8: Unused import - java.util.ArrayList. [UnusedImports]

javanna · 2024-10-21T07:26:43Z

I was looking into the windows build failures (https://buildkite.com/elastic/elasticsearch-periodic-platform-support/builds/4478), and I think that perhaps the exclusions rules in the build.gradle file don't work properly on windows. I also noticed that in other places we have separate src folders for generated files, as opposed to having them together with other source files like we did here. That may simplify how we express the exclusions.

afoucret added 9 commits October 16, 2024 11:41

Create the KQL plugin and configure antlr.

5debc71

Add Antlr licenses files.

d00b7db

Small dependencies changes.

13152e2

Adding the KQL antlr grammar

7421ea4

Regenerate the Kql base parser from the grammar.

460220c

Remove useless license files.

e712d9d

Lint for the grammar.

ec22015

Create the KqlParser.

eb12243

Adding some very basic tests for the KqlParser

c46ea5c

afoucret added >non-issue :Search/Search Search-related issues that do not fall into other categories v9.0.0 v8.17.0 labels Oct 16, 2024

afoucret requested review from carlosdelest, ioanatia and a team October 16, 2024 16:20

afoucret requested a review from a team as a code owner October 16, 2024 16:20

elasticsearchmachine added the Team:Search Meta label for search team label Oct 16, 2024

afoucret commented Oct 16, 2024

View reviewed changes

Lint

aa02224

afoucret requested a review from ChrisHegarty October 16, 2024 16:31

carlosdelest reviewed Oct 16, 2024

View reviewed changes

x-pack/plugin/kql/src/main/java/org/elasticsearch/xpack/kql/KqlPlugin.java Show resolved Hide resolved

x-pack/plugin/kql/src/test/java/org/elasticsearch/xpack/kql/parser/KqlParserTests.java Outdated Show resolved Hide resolved

afoucret and others added 2 commits October 16, 2024 20:23

Merge branch 'elastic:main' into create-kql-plugin

1742ddc

Fix boolean operators priority in the grammar.

3db7002

javanna added :Search Relevance/Search Catch all for Search Relevance and removed :Search/Search Search-related issues that do not fall into other categories labels Oct 17, 2024

elasticsearchmachine added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Oct 17, 2024

elasticsearchmachine removed the Team:Search Meta label for search team label Oct 17, 2024

Merge branch 'elastic:main' into create-kql-plugin

64a87e5

carlosdelest approved these changes Oct 17, 2024

View reviewed changes

afoucret added 3 commits October 17, 2024 16:05

Rename ParsingException into KqlParsingException

cc7434a

Grammar indent.

a2b78c3

Adding some grammar tests.

43d85c8

afoucret added the auto-backport Automatically create backport pull requests when merged label Oct 17, 2024

afoucret and others added 2 commits October 17, 2024 17:33

Fixed failed test.

54c060d

Merge branch 'elastic:main' into create-kql-plugin

a09a83f

ChrisHegarty approved these changes Oct 17, 2024

View reviewed changes

afoucret merged commit 6f60880 into elastic:main Oct 18, 2024
16 checks passed

afoucret mentioned this pull request Oct 18, 2024

[8.x] [KQL Query] Create the ANTLR parser (#114927) #115077

Merged

afoucret added a commit to afoucret/elasticsearch that referenced this pull request Oct 18, 2024

[KQL Query] Create the ANTLR parser (elastic#114927)

3e4281d

elasticsearchmachine pushed a commit that referenced this pull request Oct 18, 2024

[KQL Query] Create the ANTLR parser (#114927) (#115077)

62acedd

lkts pushed a commit to lkts/elasticsearch that referenced this pull request Oct 18, 2024

[KQL Query] Create the ANTLR parser (elastic#114927)

1598e19

javanna reviewed Oct 21, 2024

View reviewed changes

georgewallace pushed a commit to georgewallace/elasticsearch that referenced this pull request Oct 25, 2024

[KQL Query] Create the ANTLR parser (elastic#114927)

62b5014

jfreden pushed a commit to jfreden/elasticsearch that referenced this pull request Nov 4, 2024

[KQL Query] Create the ANTLR parser (elastic#114927)

c651d4c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[KQL Query] Create the ANTLR parser #114927

[KQL Query] Create the ANTLR parser #114927

afoucret commented Oct 16, 2024

elasticsearchmachine commented Oct 16, 2024

afoucret Oct 16, 2024

afoucret Oct 16, 2024

afoucret Oct 16, 2024

kderusso Oct 16, 2024

afoucret Oct 17, 2024

afoucret Oct 16, 2024

javanna Oct 21, 2024

afoucret Oct 16, 2024

afoucret Oct 16, 2024

afoucret Oct 16, 2024

afoucret Oct 16, 2024

afoucret Oct 16, 2024

javanna Oct 21, 2024

afoucret Oct 16, 2024

afoucret Oct 16, 2024

afoucret Oct 16, 2024

elasticsearchmachine commented Oct 17, 2024

carlosdelest left a comment

elasticsearchmachine commented Oct 18, 2024

javanna Oct 21, 2024

javanna Oct 21, 2024

javanna commented Oct 21, 2024 •

edited

Loading


		import static org.elasticsearch.common.logging.LoggerMessageFormat.format;

		public class ParsingException extends ElasticsearchException {

[KQL Query] Create the ANTLR parser #114927

[KQL Query] Create the ANTLR parser #114927

Conversation

afoucret commented Oct 16, 2024

elasticsearchmachine commented Oct 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticsearchmachine commented Oct 17, 2024

carlosdelest left a comment

Choose a reason for hiding this comment

elasticsearchmachine commented Oct 18, 2024

💚 Backport successful

Choose a reason for hiding this comment

Choose a reason for hiding this comment

javanna commented Oct 21, 2024 • edited Loading

javanna commented Oct 21, 2024 •

edited

Loading