Skip to content

Commit

Permalink
[Auto Suggest] DQL autosuggest with ANTLR (#7391) (#7590)
Browse files Browse the repository at this point in the history 
* Antlr autocomplete (#7159)

* dql grammar with rudamentary testing parser



* show suggestion of fields depending on current index pattern



* basic code completion with fields populated



* updated grammar and generated for better group handling



* add ignored tokens



* remove console logs



---------



* dql Antlr autocomplete (#7160)

* re-add provider for sql



* added temporary fix for language providor to appear for more than one language



---------



* remove EOF in parser to fix suggestions



* use custom version of cursor token index for dql



* implemented value suggestions based on field



* set param type



* update grouping grammar



* fix grammar for dots in field and value term search with spaces



* value suggestions match field to avoid failing api call and to find assc keyword field



* update value suggestions from partially formed value



* refactor value suggestions and change fieldval listener to visitor



* implement value suggestions within phrases



* make grammar more readable



* rename grammar parser rules



* bring back minimal autocomplete optimized grammar



* enable partially complete value suggestion for value groups



* remove number as lexer rule



* fix cursor import and clean up



* fix completion item range to be current word



* update cursor to use monaco position



* cursor index to use position directly



* move language registration into render function to handle new languages



* include auto closing quotes and parenthesis for dql



* rename generated file



* Changeset file for PR #7391 created/updated

* add license and fix linting



* modify grammar



* add tests for fields and keywords



* move dql test constants to separate file



* pass core setup from autocomplete constructor to query sugg provider and utilize selectionEnd if no position



* update an import



* use updated dataset for index pattern



* remove console log



* [Auto Suggest] DQL Updates (#7498)

* update code completion to not return for visualize



* update types to match completionitemkind



---------



* update saved object test flyout snap



* update tests to match changes



* updated editor



* insert colon and space after selecting suggested field



* fix suggestions after colon without space



* update test



* typing update



* updated dataset name



* remove connection service based lines and rely on service



* add debouncer for index pattern calls



* use index pattern service rather than debounce



* update mock index for test



---------



(cherry picked from commit 2b1d01f)

Signed-off-by: Paul Sebastian <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
  • Loading branch information
@opensearch-trigger-bot @github-actions @opensearch-changeset-bot
3 people authored Jul 30, 2024
1 parent 2fb46db commit b423572
Show file tree
Hide file tree
Showing 26 changed files with 3,195 additions and 58 deletions.
2 changes: 2 additions & 0 deletions changelogs/fragments/7391.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
feat:
- DQL Autocomplete ([#7391](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7391))
56 changes: 56 additions & 0 deletions src/plugins/data/public/antlr/dql/.generated/DQLLexer.interp
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
token literal names:
null
null
null
null
'>'
'<'
'>='
'<='
':'
'('
')'
null
null
null

token symbolic names:
null
OR
AND
NOT
GT
LT
GE
LE
EQ
LPAREN
RPAREN
PHRASE
ID
WS

rule names:
OR
AND
NOT
GT
LT
GE
LE
EQ
LPAREN
RPAREN
PHRASE
ID
WS

channel names:
DEFAULT_TOKEN_CHANNEL
HIDDEN

mode names:
DEFAULT_MODE

atn:
[4, 0, 13, 78, 6, -1, 2, 0, 7, 0, 2, 1, 7, 1, 2, 2, 7, 2, 2, 3, 7, 3, 2, 4, 7, 4, 2, 5, 7, 5, 2, 6, 7, 6, 2, 7, 7, 7, 2, 8, 7, 8, 2, 9, 7, 9, 2, 10, 7, 10, 2, 11, 7, 11, 2, 12, 7, 12, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 1, 2, 1, 2, 1, 3, 1, 3, 1, 4, 1, 4, 1, 5, 1, 5, 1, 5, 1, 6, 1, 6, 1, 6, 1, 7, 1, 7, 1, 8, 1, 8, 1, 9, 1, 9, 1, 10, 1, 10, 5, 10, 57, 8, 10, 10, 10, 12, 10, 60, 9, 10, 1, 10, 3, 10, 63, 8, 10, 1, 11, 1, 11, 5, 11, 67, 8, 11, 10, 11, 12, 11, 70, 9, 11, 1, 12, 4, 12, 73, 8, 12, 11, 12, 12, 12, 74, 1, 12, 1, 12, 0, 0, 13, 1, 1, 3, 2, 5, 3, 7, 4, 9, 5, 11, 6, 13, 7, 15, 8, 17, 9, 19, 10, 21, 11, 23, 12, 25, 13, 1, 0, 10, 2, 0, 79, 79, 111, 111, 2, 0, 82, 82, 114, 114, 2, 0, 65, 65, 97, 97, 2, 0, 78, 78, 110, 110, 2, 0, 68, 68, 100, 100, 2, 0, 84, 84, 116, 116, 2, 0, 34, 34, 92, 92, 5, 0, 42, 42, 48, 57, 65, 90, 95, 95, 97, 122, 6, 0, 42, 42, 46, 46, 48, 57, 65, 90, 95, 95, 97, 122, 3, 0, 9, 10, 13, 13, 32, 32, 81, 0, 1, 1, 0, 0, 0, 0, 3, 1, 0, 0, 0, 0, 5, 1, 0, 0, 0, 0, 7, 1, 0, 0, 0, 0, 9, 1, 0, 0, 0, 0, 11, 1, 0, 0, 0, 0, 13, 1, 0, 0, 0, 0, 15, 1, 0, 0, 0, 0, 17, 1, 0, 0, 0, 0, 19, 1, 0, 0, 0, 0, 21, 1, 0, 0, 0, 0, 23, 1, 0, 0, 0, 0, 25, 1, 0, 0, 0, 1, 27, 1, 0, 0, 0, 3, 30, 1, 0, 0, 0, 5, 34, 1, 0, 0, 0, 7, 38, 1, 0, 0, 0, 9, 40, 1, 0, 0, 0, 11, 42, 1, 0, 0, 0, 13, 45, 1, 0, 0, 0, 15, 48, 1, 0, 0, 0, 17, 50, 1, 0, 0, 0, 19, 52, 1, 0, 0, 0, 21, 54, 1, 0, 0, 0, 23, 64, 1, 0, 0, 0, 25, 72, 1, 0, 0, 0, 27, 28, 7, 0, 0, 0, 28, 29, 7, 1, 0, 0, 29, 2, 1, 0, 0, 0, 30, 31, 7, 2, 0, 0, 31, 32, 7, 3, 0, 0, 32, 33, 7, 4, 0, 0, 33, 4, 1, 0, 0, 0, 34, 35, 7, 3, 0, 0, 35, 36, 7, 0, 0, 0, 36, 37, 7, 5, 0, 0, 37, 6, 1, 0, 0, 0, 38, 39, 5, 62, 0, 0, 39, 8, 1, 0, 0, 0, 40, 41, 5, 60, 0, 0, 41, 10, 1, 0, 0, 0, 42, 43, 5, 62, 0, 0, 43, 44, 5, 61, 0, 0, 44, 12, 1, 0, 0, 0, 45, 46, 5, 60, 0, 0, 46, 47, 5, 61, 0, 0, 47, 14, 1, 0, 0, 0, 48, 49, 5, 58, 0, 0, 49, 16, 1, 0, 0, 0, 50, 51, 5, 40, 0, 0, 51, 18, 1, 0, 0, 0, 52, 53, 5, 41, 0, 0, 53, 20, 1, 0, 0, 0, 54, 58, 5, 34, 0, 0, 55, 57, 8, 6, 0, 0, 56, 55, 1, 0, 0, 0, 57, 60, 1, 0, 0, 0, 58, 56, 1, 0, 0, 0, 58, 59, 1, 0, 0, 0, 59, 62, 1, 0, 0, 0, 60, 58, 1, 0, 0, 0, 61, 63, 5, 34, 0, 0, 62, 61, 1, 0, 0, 0, 62, 63, 1, 0, 0, 0, 63, 22, 1, 0, 0, 0, 64, 68, 7, 7, 0, 0, 65, 67, 7, 8, 0, 0, 66, 65, 1, 0, 0, 0, 67, 70, 1, 0, 0, 0, 68, 66, 1, 0, 0, 0, 68, 69, 1, 0, 0, 0, 69, 24, 1, 0, 0, 0, 70, 68, 1, 0, 0, 0, 71, 73, 7, 9, 0, 0, 72, 71, 1, 0, 0, 0, 73, 74, 1, 0, 0, 0, 74, 72, 1, 0, 0, 0, 74, 75, 1, 0, 0, 0, 75, 76, 1, 0, 0, 0, 76, 77, 6, 12, 0, 0, 77, 26, 1, 0, 0, 0, 5, 0, 58, 62, 68, 74, 1, 0, 1, 0]
20 changes: 20 additions & 0 deletions src/plugins/data/public/antlr/dql/.generated/DQLLexer.tokens
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
OR=1
AND=2
NOT=3
GT=4
LT=5
GE=6
LE=7
EQ=8
LPAREN=9
RPAREN=10
PHRASE=11
ID=12
WS=13
'>'=4
'<'=5
'>='=6
'<='=7
':'=8
'('=9
')'=10
112 changes: 112 additions & 0 deletions src/plugins/data/public/antlr/dql/.generated/DQLLexer.ts
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
// Generated from grammar/DQLLexer.g4 by ANTLR 4.13.1

import * as antlr from "antlr4ng";
import { Token } from "antlr4ng";


export class DQLLexer extends antlr.Lexer {
public static readonly OR = 1;
public static readonly AND = 2;
public static readonly NOT = 3;
public static readonly GT = 4;
public static readonly LT = 5;
public static readonly GE = 6;
public static readonly LE = 7;
public static readonly EQ = 8;
public static readonly LPAREN = 9;
public static readonly RPAREN = 10;
public static readonly PHRASE = 11;
public static readonly ID = 12;
public static readonly WS = 13;

public static readonly channelNames = [
"DEFAULT_TOKEN_CHANNEL", "HIDDEN"
];

public static readonly literalNames = [
null, null, null, null, "'>'", "'<'", "'>='", "'<='", "':'", "'('",
"')'"
];

public static readonly symbolicNames = [
null, "OR", "AND", "NOT", "GT", "LT", "GE", "LE", "EQ", "LPAREN",
"RPAREN", "PHRASE", "ID", "WS"
];

public static readonly modeNames = [
"DEFAULT_MODE",
];

public static readonly ruleNames = [
"OR", "AND", "NOT", "GT", "LT", "GE", "LE", "EQ", "LPAREN", "RPAREN",
"PHRASE", "ID", "WS",
];


public constructor(input: antlr.CharStream) {
super(input);
this.interpreter = new antlr.LexerATNSimulator(this, DQLLexer._ATN, DQLLexer.decisionsToDFA, new antlr.PredictionContextCache());
}

public get grammarFileName(): string { return "DQLLexer.g4"; }

public get literalNames(): (string | null)[] { return DQLLexer.literalNames; }
public get symbolicNames(): (string | null)[] { return DQLLexer.symbolicNames; }
public get ruleNames(): string[] { return DQLLexer.ruleNames; }

public get serializedATN(): number[] { return DQLLexer._serializedATN; }

public get channelNames(): string[] { return DQLLexer.channelNames; }

public get modeNames(): string[] { return DQLLexer.modeNames; }

public static readonly _serializedATN: number[] = [
4,0,13,78,6,-1,2,0,7,0,2,1,7,1,2,2,7,2,2,3,7,3,2,4,7,4,2,5,7,5,2,
6,7,6,2,7,7,7,2,8,7,8,2,9,7,9,2,10,7,10,2,11,7,11,2,12,7,12,1,0,
1,0,1,0,1,1,1,1,1,1,1,1,1,2,1,2,1,2,1,2,1,3,1,3,1,4,1,4,1,5,1,5,
1,5,1,6,1,6,1,6,1,7,1,7,1,8,1,8,1,9,1,9,1,10,1,10,5,10,57,8,10,10,
10,12,10,60,9,10,1,10,3,10,63,8,10,1,11,1,11,5,11,67,8,11,10,11,
12,11,70,9,11,1,12,4,12,73,8,12,11,12,12,12,74,1,12,1,12,0,0,13,
1,1,3,2,5,3,7,4,9,5,11,6,13,7,15,8,17,9,19,10,21,11,23,12,25,13,
1,0,10,2,0,79,79,111,111,2,0,82,82,114,114,2,0,65,65,97,97,2,0,78,
78,110,110,2,0,68,68,100,100,2,0,84,84,116,116,2,0,34,34,92,92,5,
0,42,42,48,57,65,90,95,95,97,122,6,0,42,42,46,46,48,57,65,90,95,
95,97,122,3,0,9,10,13,13,32,32,81,0,1,1,0,0,0,0,3,1,0,0,0,0,5,1,
0,0,0,0,7,1,0,0,0,0,9,1,0,0,0,0,11,1,0,0,0,0,13,1,0,0,0,0,15,1,0,
0,0,0,17,1,0,0,0,0,19,1,0,0,0,0,21,1,0,0,0,0,23,1,0,0,0,0,25,1,0,
0,0,1,27,1,0,0,0,3,30,1,0,0,0,5,34,1,0,0,0,7,38,1,0,0,0,9,40,1,0,
0,0,11,42,1,0,0,0,13,45,1,0,0,0,15,48,1,0,0,0,17,50,1,0,0,0,19,52,
1,0,0,0,21,54,1,0,0,0,23,64,1,0,0,0,25,72,1,0,0,0,27,28,7,0,0,0,
28,29,7,1,0,0,29,2,1,0,0,0,30,31,7,2,0,0,31,32,7,3,0,0,32,33,7,4,
0,0,33,4,1,0,0,0,34,35,7,3,0,0,35,36,7,0,0,0,36,37,7,5,0,0,37,6,
1,0,0,0,38,39,5,62,0,0,39,8,1,0,0,0,40,41,5,60,0,0,41,10,1,0,0,0,
42,43,5,62,0,0,43,44,5,61,0,0,44,12,1,0,0,0,45,46,5,60,0,0,46,47,
5,61,0,0,47,14,1,0,0,0,48,49,5,58,0,0,49,16,1,0,0,0,50,51,5,40,0,
0,51,18,1,0,0,0,52,53,5,41,0,0,53,20,1,0,0,0,54,58,5,34,0,0,55,57,
8,6,0,0,56,55,1,0,0,0,57,60,1,0,0,0,58,56,1,0,0,0,58,59,1,0,0,0,
59,62,1,0,0,0,60,58,1,0,0,0,61,63,5,34,0,0,62,61,1,0,0,0,62,63,1,
0,0,0,63,22,1,0,0,0,64,68,7,7,0,0,65,67,7,8,0,0,66,65,1,0,0,0,67,
70,1,0,0,0,68,66,1,0,0,0,68,69,1,0,0,0,69,24,1,0,0,0,70,68,1,0,0,
0,71,73,7,9,0,0,72,71,1,0,0,0,73,74,1,0,0,0,74,72,1,0,0,0,74,75,
1,0,0,0,75,76,1,0,0,0,76,77,6,12,0,0,77,26,1,0,0,0,5,0,58,62,68,
74,1,0,1,0
];

private static __ATN: antlr.ATN;
public static get _ATN(): antlr.ATN {
if (!DQLLexer.__ATN) {
DQLLexer.__ATN = new antlr.ATNDeserializer().deserialize(DQLLexer._serializedATN);
}

return DQLLexer.__ATN;
}


private static readonly vocabulary = new antlr.Vocabulary(DQLLexer.literalNames, DQLLexer.symbolicNames, []);

public override get vocabulary(): antlr.Vocabulary {
return DQLLexer.vocabulary;
}

private static readonly decisionsToDFA = DQLLexer._ATN.decisionToState.map( (ds: antlr.DecisionState, index: number) => new antlr.DFA(ds, index) );
}
50 changes: 50 additions & 0 deletions src/plugins/data/public/antlr/dql/.generated/DQLParser.interp
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
token literal names:
null
null
null
null
'>'
'<'
'>='
'<='
':'
'('
')'
null
null
null

token symbolic names:
null
OR
AND
NOT
GT
LT
GE
LE
EQ
LPAREN
RPAREN
PHRASE
ID
WS

rule names:
query
operatorExpression
booleanOperator
notExpression
primaryExpression
comparisonExpression
keyValueExpression
tokenSearch
groupExpression
groupContent
field
value
comparisonOperator


atn:
[4, 1, 13, 100, 2, 0, 7, 0, 2, 1, 7, 1, 2, 2, 7, 2, 2, 3, 7, 3, 2, 4, 7, 4, 2, 5, 7, 5, 2, 6, 7, 6, 2, 7, 7, 7, 2, 8, 7, 8, 2, 9, 7, 9, 2, 10, 7, 10, 2, 11, 7, 11, 2, 12, 7, 12, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 5, 1, 33, 8, 1, 10, 1, 12, 1, 36, 9, 1, 1, 2, 1, 2, 1, 3, 3, 3, 41, 8, 3, 1, 3, 1, 3, 1, 4, 1, 4, 1, 4, 1, 4, 1, 4, 1, 4, 1, 4, 3, 4, 52, 8, 4, 1, 5, 1, 5, 1, 5, 1, 5, 1, 6, 1, 6, 1, 6, 1, 6, 3, 6, 62, 8, 6, 1, 7, 1, 7, 5, 7, 66, 8, 7, 10, 7, 12, 7, 69, 9, 7, 1, 8, 1, 8, 3, 8, 73, 8, 8, 1, 8, 1, 8, 1, 8, 3, 8, 78, 8, 8, 1, 8, 5, 8, 81, 8, 8, 10, 8, 12, 8, 84, 9, 8, 1, 8, 1, 8, 1, 9, 1, 9, 3, 9, 90, 8, 9, 1, 10, 1, 10, 1, 11, 1, 11, 3, 11, 96, 8, 11, 1, 12, 1, 12, 1, 12, 0, 0, 13, 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 0, 2, 1, 0, 1, 2, 1, 0, 4, 7, 98, 0, 26, 1, 0, 0, 0, 2, 28, 1, 0, 0, 0, 4, 37, 1, 0, 0, 0, 6, 40, 1, 0, 0, 0, 8, 51, 1, 0, 0, 0, 10, 53, 1, 0, 0, 0, 12, 57, 1, 0, 0, 0, 14, 63, 1, 0, 0, 0, 16, 70, 1, 0, 0, 0, 18, 89, 1, 0, 0, 0, 20, 91, 1, 0, 0, 0, 22, 95, 1, 0, 0, 0, 24, 97, 1, 0, 0, 0, 26, 27, 3, 2, 1, 0, 27, 1, 1, 0, 0, 0, 28, 34, 3, 6, 3, 0, 29, 30, 3, 4, 2, 0, 30, 31, 3, 6, 3, 0, 31, 33, 1, 0, 0, 0, 32, 29, 1, 0, 0, 0, 33, 36, 1, 0, 0, 0, 34, 32, 1, 0, 0, 0, 34, 35, 1, 0, 0, 0, 35, 3, 1, 0, 0, 0, 36, 34, 1, 0, 0, 0, 37, 38, 7, 0, 0, 0, 38, 5, 1, 0, 0, 0, 39, 41, 5, 3, 0, 0, 40, 39, 1, 0, 0, 0, 40, 41, 1, 0, 0, 0, 41, 42, 1, 0, 0, 0, 42, 43, 3, 8, 4, 0, 43, 7, 1, 0, 0, 0, 44, 45, 5, 9, 0, 0, 45, 46, 3, 0, 0, 0, 46, 47, 5, 10, 0, 0, 47, 52, 1, 0, 0, 0, 48, 52, 3, 10, 5, 0, 49, 52, 3, 12, 6, 0, 50, 52, 3, 14, 7, 0, 51, 44, 1, 0, 0, 0, 51, 48, 1, 0, 0, 0, 51, 49, 1, 0, 0, 0, 51, 50, 1, 0, 0, 0, 52, 9, 1, 0, 0, 0, 53, 54, 3, 20, 10, 0, 54, 55, 3, 24, 12, 0, 55, 56, 3, 22, 11, 0, 56, 11, 1, 0, 0, 0, 57, 58, 3, 20, 10, 0, 58, 61, 5, 8, 0, 0, 59, 62, 3, 22, 11, 0, 60, 62, 3, 16, 8, 0, 61, 59, 1, 0, 0, 0, 61, 60, 1, 0, 0, 0, 62, 13, 1, 0, 0, 0, 63, 67, 5, 12, 0, 0, 64, 66, 5, 12, 0, 0, 65, 64, 1, 0, 0, 0, 66, 69, 1, 0, 0, 0, 67, 65, 1, 0, 0, 0, 67, 68, 1, 0, 0, 0, 68, 15, 1, 0, 0, 0, 69, 67, 1, 0, 0, 0, 70, 72, 5, 9, 0, 0, 71, 73, 5, 3, 0, 0, 72, 71, 1, 0, 0, 0, 72, 73, 1, 0, 0, 0, 73, 74, 1, 0, 0, 0, 74, 82, 3, 18, 9, 0, 75, 77, 7, 0, 0, 0, 76, 78, 5, 3, 0, 0, 77, 76, 1, 0, 0, 0, 77, 78, 1, 0, 0, 0, 78, 79, 1, 0, 0, 0, 79, 81, 3, 18, 9, 0, 80, 75, 1, 0, 0, 0, 81, 84, 1, 0, 0, 0, 82, 80, 1, 0, 0, 0, 82, 83, 1, 0, 0, 0, 83, 85, 1, 0, 0, 0, 84, 82, 1, 0, 0, 0, 85, 86, 5, 10, 0, 0, 86, 17, 1, 0, 0, 0, 87, 90, 3, 16, 8, 0, 88, 90, 3, 22, 11, 0, 89, 87, 1, 0, 0, 0, 89, 88, 1, 0, 0, 0, 90, 19, 1, 0, 0, 0, 91, 92, 5, 12, 0, 0, 92, 21, 1, 0, 0, 0, 93, 96, 5, 11, 0, 0, 94, 96, 3, 14, 7, 0, 95, 93, 1, 0, 0, 0, 95, 94, 1, 0, 0, 0, 96, 23, 1, 0, 0, 0, 97, 98, 7, 1, 0, 0, 98, 25, 1, 0, 0, 0, 10, 34, 40, 51, 61, 67, 72, 77, 82, 89, 95]
20 changes: 20 additions & 0 deletions src/plugins/data/public/antlr/dql/.generated/DQLParser.tokens
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
OR=1
AND=2
NOT=3
GT=4
LT=5
GE=6
LE=7
EQ=8
LPAREN=9
RPAREN=10
PHRASE=11
ID=12
WS=13
'>'=4
'<'=5
'>='=6
'<='=7
':'=8
'('=9
')'=10
Loading

0 comments on commit b423572

Please sign in to comment.