[2.x][Manual Backport ] Back port main to 2.x #7477

LDrago27 · 2024-07-25T02:13:45Z

Description

This PR aims at backporting following PR from main branch to 2.x

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Miki <[email protected]> (cherry picked from commit c1bae75)

…ect#7336) * cherry pick Signed-off-by: Eric <[email protected]> * SQL keyword suggestion Signed-off-by: Eric <[email protected]> * column and values suggestions Signed-off-by: Eric <[email protected]> * adjust values suggestions and ignored token Signed-off-by: Eric <[email protected]> * suggestion type casting Signed-off-by: Eric <[email protected]> * ignore functions Signed-off-by: Eric <[email protected]> * case insensitive Signed-off-by: Eric <[email protected]> * change to use grammar for insensitive case Signed-off-by: Eric <[email protected]> * fix multi-line issue Signed-off-by: Eric <[email protected]> * code cleanup Signed-off-by: Eric <[email protected]> * move ppl out of scope Signed-off-by: Eric <[email protected]> * rename folder and add ignoring rules Signed-off-by: Eric <[email protected]> * resolve type issue Signed-off-by: Eric <[email protected]> * fix ppl suggestion provider issue Signed-off-by: Eric <[email protected]> * remove function suggestion Signed-off-by: Eric <[email protected]> * some code clean up and adding tests Signed-off-by: Eric <[email protected]> * cursor tests Signed-off-by: Eric <[email protected]> * remove testing setup Signed-off-by: Eric <[email protected]> * add changelog Signed-off-by: Eric <[email protected]> * update yarn file Signed-off-by: Eric <[email protected]> * add missing testing library Signed-off-by: Eric <[email protected]> * MDS integration Signed-off-by: Eric <[email protected]> * minor interface change and disable word based suggestion Signed-off-by: Eric <[email protected]> * update ID_LITERAL and recompile grammar Signed-off-by: Eric <[email protected]> * column suggest for agg function Signed-off-by: Eric <[email protected]> * revert version Signed-off-by: Eric <[email protected]> * opensearch sql syntax highlighting Signed-off-by: Eric <[email protected]> * add utility tests Signed-off-by: Eric <[email protected]> * symbol table test Signed-off-by: Eric <[email protected]> * error listener test Signed-off-by: Eric <[email protected]> * parse test Signed-off-by: Eric <[email protected]> * add suggestion provider to single line query editor Signed-off-by: Eric <[email protected]> * remove one test Signed-off-by: Eric <[email protected]> * add connection service type Signed-off-by: Eric <[email protected]> --------- Signed-off-by: Eric <[email protected]> (cherry picked from commit 9348bd4)

This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. Signed-off-by: Suchit Sahoo <[email protected]> (cherry picked from commit e46e54a)

…t and other similar cases. (opensearch-project#7401) * feat: add scoped breadcrumbs Signed-off-by: SuZhou-Joe <[email protected]> * Changeset file for PR opensearch-project#7401 created/updated * feat: move data source management register logic up a little bit Signed-off-by: SuZhou-Joe <[email protected]> * feat: add unit test Signed-off-by: SuZhou-Joe <[email protected]> * feat: revert the home related change Signed-off-by: SuZhou-Joe <[email protected]> * fix: overview error in all use case when workspace is enabled Signed-off-by: SuZhou-Joe <[email protected]> * fix: update snapshot Signed-off-by: SuZhou-Joe <[email protected]> * Changeset file for PR opensearch-project#7401 created/updated * fix: hide nav groups that should be displayed Signed-off-by: SuZhou-Joe <[email protected]> * feat: hide expand icon in left navigation Signed-off-by: SuZhou-Joe <[email protected]> * feat: update Signed-off-by: SuZhou-Joe <[email protected]> --------- Signed-off-by: SuZhou-Joe <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: ZilongX <[email protected]> (cherry picked from commit 3f78c94)

…ugin (opensearch-project#7440) * Move data source manageable feature flag to DSM plugin Signed-off-by: yubonluo <[email protected]> * Changeset file for PR opensearch-project#7440 created/updated --------- Signed-off-by: yubonluo <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit dfcd2e1)

* update using query manager Signed-off-by: Kawika Avilla <[email protected]> * aggs not working Signed-off-by: Kawika Avilla <[email protected]> * almost there Signed-off-by: Kawika Avilla <[email protected]> * stablish Signed-off-by: Kawika Avilla <[email protected]> * thanks ashwin Signed-off-by: Kawika Avilla <[email protected]> * update ref name Signed-off-by: Kawika Avilla <[email protected]> * fix timefields Signed-off-by: Kawika Avilla <[email protected]> * clean up some console logs Signed-off-by: Kawika Avilla <[email protected]> * go safer route of setting language Signed-off-by: Kawika Avilla <[email protected]> * its working again Signed-off-by: Kawika Avilla <[email protected]> * fix names Signed-off-by: Kawika Avilla <[email protected]> * restore code editor and indices Signed-off-by: Kawika Avilla <[email protected]> * sql df Signed-off-by: Kawika Avilla <[email protected]> * fix external datasources again Signed-off-by: Kawika Avilla <[email protected]> --------- Signed-off-by: Kawika Avilla <[email protected]> (cherry picked from commit 6a079d3)

* Fix add filter for scripted field Signed-off-by: abbyhu2000 <[email protected]> * add unit test Signed-off-by: abbyhu2000 <[email protected]> * change from nested ternary to if else Signed-off-by: abbyhu2000 <[email protected]> * add more unit tests Signed-off-by: abbyhu2000 <[email protected]> * change unit tests Signed-off-by: abbyhu2000 <[email protected]> * fix typo Signed-off-by: abbyhu2000 <[email protected]> --------- Signed-off-by: abbyhu2000 <[email protected]> (cherry picked from commit 7c1f8fc)

…pensearch-project#7435) * [Bug][Workspace] Add permission validate at workspace detail page Signed-off-by: yubonluo <[email protected]> * Changeset file for PR opensearch-project#7435 created/updated --------- Signed-off-by: yubonluo <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit 45d34b5)

github-actions · 2024-07-25T02:14:16Z

❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

codecov · 2024-07-25T02:30:41Z

Codecov Report

Attention: Patch coverage is 47.66839% with 202 lines in your changes missing coverage. Please review.

Project coverage is 64.22%. Comparing base (cced9fd) to head (69ace22).
Report is 6 commits behind head on 2.x.

Files	Patch %	Lines
...ntlr/opensearch_sql/opensearch_sql_autocomplete.ts	25.97%	56 Missing and 1 partial ⚠️
...ata/public/antlr/opensearch_sql/code_completion.ts	7.14%	52 Missing ⚠️
.../plugins/data/public/antlr/opensearch_sql/table.ts	52.94%	34 Missing and 6 partials ⚠️
src/plugins/data/public/antlr/shared/cursor.ts	41.02%	19 Missing and 4 partials ⚠️
...c/ui/filter_bar/filter_editor/value_input_type.tsx	14.28%	6 Missing ⚠️
src/plugins/data/public/antlr/shared/utils.ts	82.14%	1 Missing and 4 partials ⚠️
.../plugins/data/public/antlr/opensearch_sql/parse.ts	85.00%	3 Missing ⚠️
.../data_explorer/public/components/app_container.tsx	62.50%	1 Missing and 2 partials ⚠️
src/plugins/data/public/plugin.ts	0.00%	2 Missing ⚠️
...ic/react_router_navigate/react_router_navigate.tsx	60.00%	1 Missing and 1 partial ⚠️
... and 8 more

Additional details and impacted files

@@            Coverage Diff             @@
##              2.x    #7477      +/-   ##
==========================================
- Coverage   67.90%   64.22%   -3.68%     
==========================================
  Files        3600     3615      +15     
  Lines       72622    78719    +6097     
  Branches    11904    12436     +532     
==========================================
+ Hits        49312    50558    +1246     
- Misses      20300    25112    +4812     
- Partials     3010     3049      +39

Flag	Coverage Δ
Linux_1	`30.92% <12.36%> (-2.03%)`	⬇️
Linux_2	`55.71% <87.50%> (+<0.01%)`	⬆️
Linux_3	`40.69% <44.11%> (-3.03%)`	⬇️
Linux_4	`31.91% <12.32%> (-2.56%)`	⬇️
Windows_1	`30.94% <12.36%> (-2.03%)`	⬇️
Windows_2	`55.66% <87.50%> (+<0.01%)`	⬆️
Windows_3	`40.70% <44.11%> (-3.03%)`	⬇️
Windows_4	`31.91% <12.32%> (-2.56%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

SuZhou-Joe

LGTM

yubonluo · 2024-07-25T02:41:56Z

LGTM

…pensearch-project#7479) This reverts commit 6a079d3.

* Bump OUI to 1.8.0 (#7363) Signed-off-by: Miki <[email protected]> (cherry picked from commit c1bae75) * [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336) * cherry pick Signed-off-by: Eric <[email protected]> * SQL keyword suggestion Signed-off-by: Eric <[email protected]> * column and values suggestions Signed-off-by: Eric <[email protected]> * adjust values suggestions and ignored token Signed-off-by: Eric <[email protected]> * suggestion type casting Signed-off-by: Eric <[email protected]> * ignore functions Signed-off-by: Eric <[email protected]> * case insensitive Signed-off-by: Eric <[email protected]> * change to use grammar for insensitive case Signed-off-by: Eric <[email protected]> * fix multi-line issue Signed-off-by: Eric <[email protected]> * code cleanup Signed-off-by: Eric <[email protected]> * move ppl out of scope Signed-off-by: Eric <[email protected]> * rename folder and add ignoring rules Signed-off-by: Eric <[email protected]> * resolve type issue Signed-off-by: Eric <[email protected]> * fix ppl suggestion provider issue Signed-off-by: Eric <[email protected]> * remove function suggestion Signed-off-by: Eric <[email protected]> * some code clean up and adding tests Signed-off-by: Eric <[email protected]> * cursor tests Signed-off-by: Eric <[email protected]> * remove testing setup Signed-off-by: Eric <[email protected]> * add changelog Signed-off-by: Eric <[email protected]> * update yarn file Signed-off-by: Eric <[email protected]> * add missing testing library Signed-off-by: Eric <[email protected]> * MDS integration Signed-off-by: Eric <[email protected]> * minor interface change and disable word based suggestion Signed-off-by: Eric <[email protected]> * update ID_LITERAL and recompile grammar Signed-off-by: Eric <[email protected]> * column suggest for agg function Signed-off-by: Eric <[email protected]> * revert version Signed-off-by: Eric <[email protected]> * opensearch sql syntax highlighting Signed-off-by: Eric <[email protected]> * add utility tests Signed-off-by: Eric <[email protected]> * symbol table test Signed-off-by: Eric <[email protected]> * error listener test Signed-off-by: Eric <[email protected]> * parse test Signed-off-by: Eric <[email protected]> * add suggestion provider to single line query editor Signed-off-by: Eric <[email protected]> * remove one test Signed-off-by: Eric <[email protected]> * add connection service type Signed-off-by: Eric <[email protected]> --------- Signed-off-by: Eric <[email protected]> (cherry picked from commit 9348bd4) * Move TopNavLinks to new Nav Bar Discover (#7326) This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. Signed-off-by: Suchit Sahoo <[email protected]> (cherry picked from commit e46e54a) * [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401) * feat: add scoped breadcrumbs Signed-off-by: SuZhou-Joe <[email protected]> * Changeset file for PR #7401 created/updated * feat: move data source management register logic up a little bit Signed-off-by: SuZhou-Joe <[email protected]> * feat: add unit test Signed-off-by: SuZhou-Joe <[email protected]> * feat: revert the home related change Signed-off-by: SuZhou-Joe <[email protected]> * fix: overview error in all use case when workspace is enabled Signed-off-by: SuZhou-Joe <[email protected]> * fix: update snapshot Signed-off-by: SuZhou-Joe <[email protected]> * Changeset file for PR #7401 created/updated * fix: hide nav groups that should be displayed Signed-off-by: SuZhou-Joe <[email protected]> * feat: hide expand icon in left navigation Signed-off-by: SuZhou-Joe <[email protected]> * feat: update Signed-off-by: SuZhou-Joe <[email protected]> --------- Signed-off-by: SuZhou-Joe <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: ZilongX <[email protected]> (cherry picked from commit 3f78c94) * [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440) * Move data source manageable feature flag to DSM plugin Signed-off-by: yubonluo <[email protected]> * Changeset file for PR #7440 created/updated --------- Signed-off-by: yubonluo <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit dfcd2e1) * [Discover-next] data set picker (#7426) * update using query manager Signed-off-by: Kawika Avilla <[email protected]> * aggs not working Signed-off-by: Kawika Avilla <[email protected]> * almost there Signed-off-by: Kawika Avilla <[email protected]> * stablish Signed-off-by: Kawika Avilla <[email protected]> * thanks ashwin Signed-off-by: Kawika Avilla <[email protected]> * update ref name Signed-off-by: Kawika Avilla <[email protected]> * fix timefields Signed-off-by: Kawika Avilla <[email protected]> * clean up some console logs Signed-off-by: Kawika Avilla <[email protected]> * go safer route of setting language Signed-off-by: Kawika Avilla <[email protected]> * its working again Signed-off-by: Kawika Avilla <[email protected]> * fix names Signed-off-by: Kawika Avilla <[email protected]> * restore code editor and indices Signed-off-by: Kawika Avilla <[email protected]> * sql df Signed-off-by: Kawika Avilla <[email protected]> * fix external datasources again Signed-off-by: Kawika Avilla <[email protected]> --------- Signed-off-by: Kawika Avilla <[email protected]> (cherry picked from commit 6a079d3) * Fix add filter for numeric scripted field (#7022) * Fix add filter for scripted field Signed-off-by: abbyhu2000 <[email protected]> * add unit test Signed-off-by: abbyhu2000 <[email protected]> * change from nested ternary to if else Signed-off-by: abbyhu2000 <[email protected]> * add more unit tests Signed-off-by: abbyhu2000 <[email protected]> * change unit tests Signed-off-by: abbyhu2000 <[email protected]> * fix typo Signed-off-by: abbyhu2000 <[email protected]> --------- Signed-off-by: abbyhu2000 <[email protected]> (cherry picked from commit 7c1f8fc) * [Bug][Workspace] Add permission validation at workspace detail page (#7435) * [Bug][Workspace] Add permission validate at workspace detail page Signed-off-by: yubonluo <[email protected]> * Changeset file for PR #7435 created/updated --------- Signed-off-by: yubonluo <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit 45d34b5) * Revert "[Discover-next] data set picker (#7426)" (#7479) This reverts commit 6a079d3. --------- Co-authored-by: Miki <[email protected]> Co-authored-by: Eric Wei <[email protected]> Co-authored-by: SuZhou-Joe <[email protected]> Co-authored-by: yuboluo <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Qingyang(Abby) Hu <[email protected]> Co-authored-by: Sean Li <[email protected]> (cherry picked from commit 0515fb2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Bump OUI to 1.8.0 (#7363) (cherry picked from commit c1bae75) * [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336) * cherry pick * SQL keyword suggestion * column and values suggestions * adjust values suggestions and ignored token * suggestion type casting * ignore functions * case insensitive * change to use grammar for insensitive case * fix multi-line issue * code cleanup * move ppl out of scope * rename folder and add ignoring rules * resolve type issue * fix ppl suggestion provider issue * remove function suggestion * some code clean up and adding tests * cursor tests * remove testing setup * add changelog * update yarn file * add missing testing library * MDS integration * minor interface change and disable word based suggestion * update ID_LITERAL and recompile grammar * column suggest for agg function * revert version * opensearch sql syntax highlighting * add utility tests * symbol table test * error listener test * parse test * add suggestion provider to single line query editor * remove one test * add connection service type --------- (cherry picked from commit 9348bd4) * Move TopNavLinks to new Nav Bar Discover (#7326) This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. (cherry picked from commit e46e54a) * [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401) * feat: add scoped breadcrumbs * Changeset file for PR #7401 created/updated * feat: move data source management register logic up a little bit * feat: add unit test * feat: revert the home related change * fix: overview error in all use case when workspace is enabled * fix: update snapshot * Changeset file for PR #7401 created/updated * fix: hide nav groups that should be displayed * feat: hide expand icon in left navigation * feat: update --------- (cherry picked from commit 3f78c94) * [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440) * Move data source manageable feature flag to DSM plugin * Changeset file for PR #7440 created/updated --------- (cherry picked from commit dfcd2e1) * [Discover-next] data set picker (#7426) * update using query manager * aggs not working * almost there * stablish * thanks ashwin * update ref name * fix timefields * clean up some console logs * go safer route of setting language * its working again * fix names * restore code editor and indices * sql df * fix external datasources again --------- (cherry picked from commit 6a079d3) * Fix add filter for numeric scripted field (#7022) * Fix add filter for scripted field * add unit test * change from nested ternary to if else * add more unit tests * change unit tests * fix typo --------- (cherry picked from commit 7c1f8fc) * [Bug][Workspace] Add permission validation at workspace detail page (#7435) * [Bug][Workspace] Add permission validate at workspace detail page * Changeset file for PR #7435 created/updated --------- (cherry picked from commit 45d34b5) * Revert "[Discover-next] data set picker (#7426)" (#7479) This reverts commit 6a079d3. --------- (cherry picked from commit 0515fb2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Miki <[email protected]> Co-authored-by: Eric Wei <[email protected]> Co-authored-by: SuZhou-Joe <[email protected]> Co-authored-by: yuboluo <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Qingyang(Abby) Hu <[email protected]> Co-authored-by: Sean Li <[email protected]>

AMoo-Miki and others added 8 commits July 25, 2024 01:41

Bump OUI to 1.8.0 (opensearch-project#7363)

0203b20

Signed-off-by: Miki <[email protected]> (cherry picked from commit c1bae75)

LDrago27 requested review from ananzh, kavilla, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, zhongnansu, manasvinibs, ZilongX, Flyingliuhub, curq, bandinib-amzn, SuZhou-Joe, ruanyl, BionIT, xinruiba, zhyuanqi, mengweieric, virajsanghvi and sejli as code owners July 25, 2024 02:13

LDrago27 added the v2.16.0 label Jul 25, 2024

github-actions bot added the failed changeset label Jul 25, 2024

ananzh previously approved these changes Jul 25, 2024

View reviewed changes

SuZhou-Joe previously approved these changes Jul 25, 2024

View reviewed changes

This was referenced Jul 25, 2024

[Backport 2.x] Fix add filter for numeric scripted field #7462

Closed

[2.x][Manual Backport] Move TopNavLinks to new Nav Bar Discover (#7326) #7455

Closed

Revert "[Discover-next] data set picker (opensearch-project#7426)" (o…

69ace22

…pensearch-project#7479) This reverts commit 6a079d3.

LDrago27 dismissed stale reviews from SuZhou-Joe and ananzh via 69ace22 July 25, 2024 17:37

ananzh approved these changes Jul 25, 2024

View reviewed changes

ashwin-pc approved these changes Jul 25, 2024

View reviewed changes

LDrago27 merged commit 0515fb2 into opensearch-project:2.x Jul 25, 2024
61 of 67 checks passed

LDrago27 added the backport 2.16 label Jul 25, 2024

opensearch-trigger-bot bot mentioned this pull request Jul 25, 2024

[Backport 2.16] [2.x][Manual Backport ] Back port main to 2.x #7487

Merged

LDrago27 mentioned this pull request Sep 10, 2024

Move TopNavLinks to new Nav Bar Discover #7326

Merged

7 tasks

AMoo-Miki mentioned this pull request Sep 13, 2024

[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[2.x][Manual Backport ] Back port main to 2.x #7477

[2.x][Manual Backport ] Back port main to 2.x #7477

LDrago27 commented Jul 25, 2024

github-actions bot commented Jul 25, 2024

codecov bot commented Jul 25, 2024 •

edited

Loading

SuZhou-Joe left a comment

yubonluo commented Jul 25, 2024

[2.x][Manual Backport ] Back port main to 2.x #7477

[2.x][Manual Backport ] Back port main to 2.x #7477

Conversation

LDrago27 commented Jul 25, 2024

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

github-actions bot commented Jul 25, 2024

❌ Empty Changelog Section

codecov bot commented Jul 25, 2024 • edited Loading

Codecov Report

SuZhou-Joe left a comment

Choose a reason for hiding this comment

yubonluo commented Jul 25, 2024

codecov bot commented Jul 25, 2024 •

edited

Loading