Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Backport 2.x] Refactor SparkQueryDispatcher (#2636) #2668

Closed
wants to merge 299 commits into from
Closed

[Backport 2.x] Refactor SparkQueryDispatcher (#2636) #2668

wants to merge 299 commits into from

Conversation

ykmr1224
Copy link
Collaborator

Description

Issues Resolved

N/A

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

opensearch-trigger-bot bot and others added 30 commits February 16, 2023 09:07
@opensearch-trigger-bot @GabeFernandez310
Added Support For More Formats For Get_Format (opensearch-project#1343)…
4afe013 
… (opensearch-project#1349)

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit 672c72f)

Co-authored-by: GabeFernandez310 <[email protected]>
@opensearch-trigger-bot @GabeFernandez310
Add WeekOfYear Function To OpenSearch (opensearch-project#1345) (open…
b7a4deb 
…search-project#1351)

Added WeekOfYear As An Alias To Week Function

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit 114e02a)

Co-authored-by: GabeFernandez310 <[email protected]>
@opensearch-trigger-bot @GabeFernandez310
Add last_day Function To OpenSearch SQL Plugin (opensearch-project#…
cc2a768 
…1344) (opensearch-project#1350)

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit d9114f5)

Co-authored-by: GabeFernandez310 <[email protected]>
@opensearch-trigger-bot @dai-chen
Fix index not found reported as server error bug (opensearch-project#…
ff0747d 
…1353) (opensearch-project#1355)

* Capture IndexNotFoundException and re-throw directly

Signed-off-by: Chen Dai <[email protected]>

* Add IT

Signed-off-by: Chen Dai <[email protected]>

---------

Signed-off-by: Chen Dai <[email protected]>
(cherry picked from commit 76a8d29)

Co-authored-by: Chen Dai <[email protected]>
@GumpacG
Modified returning NaN to NULL (opensearch-project#1347)
a32bcbe 
Signed-off-by: Guian Gumpac <[email protected]>
(cherry picked from commit de40f42)
@opensearch-trigger-bot @rupal-bq
Add release notes for 2.6.0 (opensearch-project#1367) (opensearch-pro…
7377bed 
…ject#1369)

* Add release notes for 2.6.0

Signed-off-by: Rupal Mahajan <[email protected]>

* Remove incorrect links

Signed-off-by: Rupal Mahajan <[email protected]>

---------

Signed-off-by: Rupal Mahajan <[email protected]>
(cherry picked from commit 73305e9)

Co-authored-by: Rupal Mahajan <[email protected]>
@vmmusings
Integ Test Refactoring (opensearch-project#1383)
8f6793b 
* Integ Test Refactoring

Signed-off-by: Vamsi Manohar <[email protected]>
@vmmusings
Refactor AWSSigV4 auth to support different AWSCredentialProviders (o…
8583fd1 
…pensearch-project#1389)

* Refactor AWSSigV4 auth to support different AWSCredentialProviders

Signed-off-by: Vamsi Manohar <[email protected]>

* Added unit tests for sts assume role credentials provider

Signed-off-by: Vamsi Manohar <[email protected]>

---------

Signed-off-by: Vamsi Manohar <[email protected]>
@opensearch-trigger-bot @dai-chen
Exclude .opensearch index when cleanup and ignore exception in case (o…
4749176 
…pensearch-project#1380) (opensearch-project#1381)

Signed-off-by: Chen Dai <[email protected]>
(cherry picked from commit 71d8604)

Co-authored-by: Chen Dai <[email protected]>
@opensearch-trigger-bot @opensearch-ci-bot
Increment version to 2.7.0-SNAPSHOT (opensearch-project#1368)
74b802e 
Signed-off-by: opensearch-ci-bot <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
@opensearch-trigger-bot
Update usage of Strings.toString (opensearch-project#1309) (opensearc…
652e771 
…h-project#1404)

Signed-off-by: MaxKsyunz <[email protected]>
(cherry picked from commit 6108ca1)

Co-authored-by: Max Ksyunz <[email protected]>
@penghuo
Deprecated Spring IoC and using Guice instead (opensearch-project#1177)…
fcce603 
… (opensearch-project#1410)

Signed-off-by: Peng Huo <[email protected]>
(cherry picked from commit 0a02009)
@opensearch-trigger-bot @vmmusings
[main]Changes in DataSourceService and DataSourceMetadataStorage inte…
11d0d6d 
…rface (opensearch-project#1408) (opensearch-project#1414)

* Added new interface methods in DataSourceService and DataSourceMetadataStorage

Signed-off-by: Vamsi Manohar <[email protected]>

Signed-off-by: Vamsi Manohar <[email protected]>
(cherry picked from commit 98fb969)

Co-authored-by: vamsi-amazon <[email protected]>
@Yury-Fridlyand
Bump backport version. (opensearch-project#1009)
81e5569 
Signed-off-by: Yury-Fridlyand <[email protected]>
@opensearch-trigger-bot @penghuo
Resolve table function based StorageEngine provided function resolver (…
0b6e581 
…opensearch-project#1354) (opensearch-project#1424)

Signed-off-by: Peng Huo <[email protected]>
(cherry picked from commit bcfda37)

Co-authored-by: Peng Huo <[email protected]>
@opensearch-trigger-bot @matthewryanwells
Added SINH function to V2 engine (opensearch-project#242) (opensearch…
00ff7eb 
…-project#1434) (opensearch-project#1437)

* Updated SINH function to V2 engine, added documentation, unit/IT tests

Signed-off-by: Matthew Wells <[email protected]>
(cherry picked from commit c9f96fb)

Co-authored-by: Matthew Wells <[email protected]>
@opensearch-trigger-bot @matthewryanwells
Added RINT function to V2 engine (opensearch-project#240) (opensearch…
014de25 
…-project#1433) (opensearch-project#1439)

Added RINT to V2 engine, updated documentation, added unit and IT tests

Signed-off-by: Matthew Wells <[email protected]>
(cherry picked from commit 7220cfe)

Co-authored-by: Matthew Wells <[email protected]>
@opensearch-trigger-bot @GabeFernandez310
Add sec_to_time function to OpenSearch (opensearch-project#1378) (o…
56c68b6 
…pensearch-project#1438)

* Added Tests

* Added Implementation And Documentation

* Added Integration Test And Fixed Checkstyle

* Addressed PR Comments

* Fixed Checkstyle

* Added Tests And Modified Docs

* Fixed formatNanos Helper Function

* Temporarily Removed Failing Tests

* Fixed Nanoseconds Float/Double Imprecision Issues

* Fixed Checkstyle

* Added Tests

* Altered Implementation To Use BigDecimal

---------

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit aa7f90e)

Co-authored-by: GabeFernandez310 <[email protected]>
@opensearch-trigger-bot @GabeFernandez310
Add WEEKDAY Function to SQL Plugin (opensearch-project#1418) (opens…
7122a55 
…earch-project#1440)

* Added Tests

* Added Implementation

* Added Documentation

* Fixed IT Test

* Adjusted IT test

* Fixed Implementation for Time And Unit Tests

* Addressed PR Comments

* Cleaned WeekdayTest File

* Fixed Docs

* Fixed Doctest

* Removed Unneeded Function

---------

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit 4f9f5ca)

Co-authored-by: GabeFernandez310 <[email protected]>
@GabeFernandez310
Add YEARWEEK Function To OpenSearch SQL (opensearch-project#1417) (o…
b0fdab9 
…pensearch-project#1445)

* Add `YEARWEEK` Function To OpenSearch SQL

* Added Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Documentation

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Addressed PR Comments

Signed-off-by: GabeFernandez310 <[email protected]>

* Cleaned YearweekTest File

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Tests and Fixed Docs

Signed-off-by: GabeFernandez310 <[email protected]>

* Added And Modified Unit Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Jacoco In Core

Signed-off-by: GabeFernandez310 <[email protected]>

* Reworked convertWeekModeFromMySqlToJava Function

Signed-off-by: GabeFernandez310 <[email protected]>

* Refactored To Remove Extra Function

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Magic Number

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit 03e4f97)
@GabeFernandez310
Add EXTRACT Function To OpenSearch SQL Plugin (opensearch-project#1421
29f005f 
) (opensearch-project#1443)

* Add `EXTRACT` Function To OpenSearch SQL Plugin

* Added Basic Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Lexer And Parser Language

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Implementation And Fixed Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Documentation

Signed-off-by: GabeFernandez310 <[email protected]>

* Modified Implementation And Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Checkstyle Issues

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Javadoc comments and Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Changed Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Changed A Test

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Integration Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Refactored Extract Tests Into New File

Signed-off-by: GabeFernandez310 <[email protected]>

* Refactored Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Renamed Rule In Parser

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed AstExpressionBuilder

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>

* Removed Unneeded Code And Added Parser Test

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit 87018c6)
@GabeFernandez310
Add STR_TO_DATE Function To The SQL Plugin (opensearch-project#1420) (
5f1747d 
opensearch-project#1444)

* Add `STR_TO_DATE` Function To The SQL Plugin.

* Added Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Empty Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Partially Working Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Cleaned Up Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Modified Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Modified IT Test

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Documentation

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Unit Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Altered Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Reworked Implementation To Always Return Datetime

Signed-off-by: GabeFernandez310 <[email protected]>

* Addressed PR Comments

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Some Checkstyle Issues

Signed-off-by: GabeFernandez310 <[email protected]>

* Cleaned Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Supported Function Properties

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Checkstyle

Signed-off-by: GabeFernandez310 <[email protected]>

* Removed Unneeded Function

Signed-off-by: GabeFernandez310 <[email protected]>

* Cleaned Implementation and Added IT Test

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Code Coverage and Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>

* Cleaned Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit 40336d4)
@GabeFernandez310
Add The TO_SECONDS Function To The SQL Plugin (opensearch-project#1419
5d01207 
) (opensearch-project#1447)

* Add The `TO_SECONDS` Function To The SQL Plugin (opensearch-project#232)

* Added Basic Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Added IT Test

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Changed Integration Tests

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Test For Time Type

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Documentation

Signed-off-by: GabeFernandez310 <[email protected]>

* Addressed PR Comments

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Docs and Implementation

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Checkstyle

Signed-off-by: GabeFernandez310 <[email protected]>

* Changed DateTimeFormatter Priority

Signed-off-by: GabeFernandez310 <[email protected]>

* Added More Formatters

Signed-off-by: GabeFernandez310 <[email protected]>

* Updated Docs

Signed-off-by: GabeFernandez310 <[email protected]>

* Reworked Implementation For Formatters

Signed-off-by: GabeFernandez310 <[email protected]>

* Cleanup

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Test

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Implementation And Code Coverage

Signed-off-by: GabeFernandez310 <[email protected]>

* Changed getFormatter Function

Signed-off-by: GabeFernandez310 <[email protected]>

* Added Comments

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>

* Removed Unneeded Code

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit d38a6ec)
@opensearch-trigger-bot @matthewryanwells
Added Arithmetic functions to V2 engine (opensearch-project#1416) (op…
c286846 
…ensearch-project#1448)

* Updated Arithmetic functions from old engine to new engine (opensearch-project#235)

* Updated ADD, SUBTRACT, MULTIPLY, DIVIDE, MODULUS to V2 engine

Signed-off-by: Matthew Wells <[email protected]>
(cherry picked from commit bc39346)

Co-authored-by: Matthew Wells <[email protected]>
@opensearch-trigger-bot @matthewryanwells
Added SIGNUM function to V2 engine (opensearch-project#1429) (opensea…
3ea6c5c 
…rch-project#1442)

* Added SIGNUM function to V2 engine (opensearch-project#241)

* Added SIGNUM function to V2 engine, updated documentation, added unit/IT tests

Signed-off-by: Matthew Wells <[email protected]>

* fixed merge mistake

Signed-off-by: Matthew Wells <[email protected]>

* fixed merging error

Signed-off-by: Matthew Wells <[email protected]>

---------

Signed-off-by: Matthew Wells <[email protected]>
(cherry picked from commit 6496692)

Co-authored-by: Matthew Wells <[email protected]>
@opensearch-trigger-bot @GabeFernandez310
Add TIMESTAMPADD Function To OpenSearch SQL Plugin (opensearch-proj…
305ba24 
…ect#1451) (opensearch-project#1453)

* Add `TIMESTAMPADD` Function To OpenSearch SQL Plugin

Added Testing And Implementation for TIMESTAMPADD

Signed-off-by: GabeFernandez310 <[email protected]>

* Fixed Formatting

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit 7b83e50)

Co-authored-by: GabeFernandez310 <[email protected]>
@opensearch-trigger-bot @Yury-Fridlyand
Rework on OpenSearchDataType: parse, store and use mapping informat…
5cfa702 
…ion (opensearch-project#1314) (opensearch-project#1455)

* Rework on `OpenSearchDataType`: parse, store and use mapping information (opensearch-project#180)

Rework on `OpenSearchDataType`:
* Add data types for those classes are not defined in `ExprCoreType`.
* Address Bit-Quill#180 (comment)
* Remove `TextKeywordValue`.
* Add changes according to the PR review. Bit-Quill#180
* Update `IndexMapping::parseMapping` function.
* Add `OpenSearchDataType::resolve` function.
* Add new constructor for `OpenSearchTextType`.
* Make `fields` and `properties` in `OpenSearchDataType` readonly. Update tests and mapping parser.
* Move `getFields` from `OpenSearchDataType` to `OpenSearchTextType`. Update tests.
* Rewrite `traverseAndFlatten` according to Bit-Quill#180 (comment)
* Minor comment fix.
* A fix to avoid breaking changes.
* `typeName` and `legacyTypeName` to return different type names.
* Change `typeof` function and corresponding tests.
* Move `convertTextToKeyword` from `ScriptUtils` to `OpenSearchTextType`. Update tests.
* Update UT for `typeof` function.
* Make all instances of `OpenSearchDataType` and of derived types singletones as much as possible.
* Make string representations of all `ExprType`s uppercase.
* Remove functions from `IndexMapping` used in tests only.

Signed-off-by: Yury-Fridlyand <[email protected]>
Signed-off-by: Yury-Fridlyand <[email protected]>
(cherry picked from commit d44cd39)

Co-authored-by: Yury-Fridlyand <[email protected]>
@opensearch-trigger-bot @matthewryanwells
Added missing integration tests for the pow and power functions (open…
49a82ae 
…search-project#247) (opensearch-project#1457) (opensearch-project#1459)

* Changed pow and power to return null when an invalid response would be returned and added missing integration tests for the pow and power function.

Signed-off-by: Matthew Wells <[email protected]>
(cherry picked from commit a35f963)

Co-authored-by: Matthew Wells <[email protected]>
@opensearch-trigger-bot @GabeFernandez310
Add Timestampdiff Function To OpenSearch SQL (opensearch-project#1467…
20964e1 
…) (opensearch-project#1472)

* Add `Timestampdiff` Function To OpenSearch SQL

Added Tests And Implementation For Timestampdiff

---------

Signed-off-by: GabeFernandez310 <[email protected]>

* Removed Import

Signed-off-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: GabeFernandez310 <[email protected]>
(cherry picked from commit 31148da)

Co-authored-by: GabeFernandez310 <[email protected]>
@vmmusings
Create datasource API (opensearch-project#1458) (opensearch-project#1479
1aa1861 
)

Signed-off-by: vamsi-amazon <[email protected]>
vmmusings and others added 28 commits March 18, 2024 17:45
@vmmusings
Handle ALTER Index Queries in SQL Plugin (opensearch-project#2554) (o…
8e25dd9 
…pensearch-project#2560)

Signed-off-by: Vamsi Manohar <[email protected]>
(cherry picked from commit d11a268)
@opensearch-trigger-bot @github-actions
Implement vacuum index operation (opensearch-project#2557) (opensearc…
68622f8 
…h-project#2562)

* Add vacuum operation and IT



* Add index state doc delete and more IT



* Refactor IT



* Fix bytebuddy version conflict



* Fix broken IT



* Fix broken IT



* Fix jacoco failure with new IT



* Fix code format



* Fix jacoco test coverage



---------


(cherry picked from commit 8374cb6)

Signed-off-by: Chen Dai <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Change async query default setting (opensearch-project#2561) (opensea…
11195ee 
…rch-project#2563)

* Change aysnc_query default setting



* fix doctest



---------


(cherry picked from commit b375a28)

Signed-off-by: Peng Huo <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Percent encode opensearch index name (opensearch-project#2564) (opens…
59875d8 
…earch-project#2566)

* percent encode opensearch index name



* spec test vacuum



* spotlessApply



---------


(cherry picked from commit e17962f)

Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Refactor query param (opensearch-project#2519) (opensearch-project#2555)
eb5aaf7 
* Refactor query param



* Reduce scope of changes



---------


(cherry picked from commit ee2dbd5)

Signed-off-by: Louis Chu <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
FlintStreamingJobCleanerTask Implementation (opensearch-project#2559) (…
b4c5415 
…opensearch-project#2567)

(cherry picked from commit b3fc1ca)

Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Wrap the query with double quotes (opensearch-project#2565) (opensear…
cf201df 
…ch-project#2569)

(cherry picked from commit 405068b)

Signed-off-by: Louis Chu <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
FlintStreamingJobCleanerTask Implementation (opensearch-project#2574) (…
a436fbb 
…opensearch-project#2575)

(cherry picked from commit 284a0be)

Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Upgrade opensearch-spark jars to 0.3.0 (opensearch-project#2568) (ope…
811ff7f 
…nsearch-project#2571)

(cherry picked from commit 4dc83b7)

Signed-off-by: Louis Chu <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Added release notes for 2.13.0.0 (opensearch-project#2578) (opensearc…
9e2915b 
…h-project#2580)

(cherry picked from commit a67de21)

Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Fix for dataSources Integ test issue (opensearch-project#2578) (opens…
5fa8c3c 
…earch-project#2582) (opensearch-project#2583)

(cherry picked from commit 85dae6f)

Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Handle EMR Exceptions in FlintCancelJob Operation (opensearch-project…
8875a03 
…#2577) (opensearch-project#2589)

(cherry picked from commit bfcaedf)

Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@seankao-az
Bug Fix: Escape query in spark submit parameter (opensearch-project#2587
4d54078 
) (opensearch-project#2592)

* escape query in spark submit parameter

Signed-off-by: Sean Kao <[email protected]>

* spotless

Signed-off-by: Sean Kao <[email protected]>

* test case for special character

Signed-off-by: Sean Kao <[email protected]>

---------

Signed-off-by: Sean Kao <[email protected]>
(cherry picked from commit 2bcf0b8)
@opensearch-trigger-bot @github-actions
Removing old datasources model test (opensearch-project#2594) (opense…
9a1d735 
…arch-project#2595)

(cherry picked from commit e153609)

Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Change vacuum statement semantic (opensearch-project#2606) (opensearc…
f414b06 
…h-project#2607)

(cherry picked from commit 36b423c)

Signed-off-by: Chen Dai <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Refactoring of SparkQueryDispatcher by removing unnecessary class (op…
8a8b5f4 
…ensearch-project#2615) (opensearch-project#2618)

(cherry picked from commit 204c7da)

Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Update v2.13.0 release notes (opensearch-project#2609) (opensearch-pr…
4da8eae 
…oject#2613)

(cherry picked from commit 2649200)

Signed-off-by: Rupal Mahajan <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Handle EMRS exception as 400 (opensearch-project#2612) (opensearch-pr…
92fb88e 
…oject#2627)

(cherry picked from commit 0d8341f)

Signed-off-by: Louis Chu <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Fix pagination for many columns (opensearch-project#2440) (opensearch…
1789523 
…-project#2441) (opensearch-project#2629)

(cherry picked from commit 3f53904)

Signed-off-by: Andreas Kulicke <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Add iceberg support to EMR serverless jobs. (opensearch-project#2602) (…
c5dfbf2 
…opensearch-project#2608)

(cherry picked from commit 39c0222)

Signed-off-by: Adi Suresh <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Fix semicolon parsing for async query (opensearch-project#2631) (open…
3b65c10 
…search-project#2633)

* update sql grammar files from upstream



* resolve semicolon cause incorrect parsing



* spotlessApply



---------


(cherry picked from commit 7f8fbe9)

Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
Throw OpensearchSecurityException incase of datasource authorization …
bce7758 
…error (opensearch-project#2626) (opensearch-project#2634)

(cherry picked from commit 5464bfc)

Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @opensearch-ci-bot
Increment version to 2.14.0-SNAPSHOT (opensearch-project#2585)
294566f 
Signed-off-by: opensearch-ci-bot <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
@opensearch-trigger-bot @github-actions
Use EMR serverless bundled iceberg JAR. (opensearch-project#2632) (op…
7921635 
…ensearch-project#2635)

Instead of downloading the JAR from Maven, the JAR in the EMR serverless
root file system can be used.


(cherry picked from commit e578a57)

Signed-off-by: Adi Suresh <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions
release notes for 2.14.0.0 (opensearch-project#2647) (opensearch-proj…
da7b01e 
…ect#2648)

(cherry picked from commit b454a2c)

Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot @github-actions @vmmusings
Add option to use LakeFormation in S3Glue data source. (opensearch-pr…
22db591 
…oject#2624) (opensearch-project#2639)

* Add option to use LakeFormation in S3Glue data source.



* Update s3glue_connector.rst

corrected formatting issue.



---------




(cherry picked from commit ea08c8f)

Signed-off-by: Adi Suresh <[email protected]>
Signed-off-by: Vamsi Manohar <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Vamsi Manohar <[email protected]>
@opensearch-trigger-bot @github-actions
Update maintainers list (opensearch-project#2663) (opensearch-project…
5413096 
…#2664)

(cherry picked from commit cfd222c)

Signed-off-by: Sean Kao <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@ykmr1224 @github-actions
[Backport 2.x] Delete Spark datasource (opensearch-project#2638) (ope…
8a9d38c 
…nsearch-project#2667)

* Delete Spark datasource (opensearch-project#2638)

* Delete Spark datasource

Signed-off-by: Tomoyuki Morita <[email protected]>

* Fix build error

Signed-off-by: Tomoyuki Morita <[email protected]>

* Delete spark_connector.rst

Signed-off-by: Tomoyuki Morita <[email protected]>

* Add missing test

Signed-off-by: Tomoyuki Morita <[email protected]>

---------

Signed-off-by: Tomoyuki Morita <[email protected]>
(cherry picked from commit de7b367)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Fix test failure in DefaultSparkSqlFunctionResponseHandleTest

Signed-off-by: Tomoyuki Morita <[email protected]>

* Reformat

Signed-off-by: Tomoyuki Morita <[email protected]>

---------

Signed-off-by: Tomoyuki Morita <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@ykmr1224 ykmr1224 closed this May 13, 2024
@ykmr1224 ykmr1224 deleted the backport-2636 branch May 18, 2024 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ps48 ps48 Awaiting requested review from ps48 ps48 will be requested when the pull request is marked ready for review ps48 is a code owner

@kavithacm kavithacm Awaiting requested review from kavithacm kavithacm will be requested when the pull request is marked ready for review kavithacm is a code owner

@derek-ho derek-ho Awaiting requested review from derek-ho derek-ho will be requested when the pull request is marked ready for review derek-ho is a code owner

@joshuali925 joshuali925 Awaiting requested review from joshuali925 joshuali925 will be requested when the pull request is marked ready for review joshuali925 is a code owner

@dai-chen dai-chen Awaiting requested review from dai-chen dai-chen will be requested when the pull request is marked ready for review dai-chen is a code owner

@YANG-DB YANG-DB Awaiting requested review from YANG-DB YANG-DB will be requested when the pull request is marked ready for review YANG-DB is a code owner

@mengweieric mengweieric Awaiting requested review from mengweieric mengweieric will be requested when the pull request is marked ready for review mengweieric is a code owner

@Swiddis Swiddis Awaiting requested review from Swiddis Swiddis will be requested when the pull request is marked ready for review Swiddis is a code owner

@penghuo penghuo Awaiting requested review from penghuo penghuo will be requested when the pull request is marked ready for review penghuo is a code owner

@seankao-az seankao-az Awaiting requested review from seankao-az seankao-az will be requested when the pull request is marked ready for review seankao-az is a code owner

@MaxKsyunz MaxKsyunz Awaiting requested review from MaxKsyunz MaxKsyunz will be requested when the pull request is marked ready for review MaxKsyunz is a code owner

@Yury-Fridlyand Yury-Fridlyand Awaiting requested review from Yury-Fridlyand Yury-Fridlyand will be requested when the pull request is marked ready for review Yury-Fridlyand is a code owner

@anirudha anirudha Awaiting requested review from anirudha anirudha will be requested when the pull request is marked ready for review anirudha is a code owner

@forestmvey forestmvey Awaiting requested review from forestmvey forestmvey will be requested when the pull request is marked ready for review forestmvey is a code owner

@acarbonetto acarbonetto Awaiting requested review from acarbonetto acarbonetto will be requested when the pull request is marked ready for review acarbonetto is a code owner

@GumpacG GumpacG Awaiting requested review from GumpacG GumpacG will be requested when the pull request is marked ready for review GumpacG is a code owner

@LantaoJin LantaoJin Awaiting requested review from LantaoJin LantaoJin will be requested when the pull request is marked ready for review LantaoJin is a code owner

@noCharger noCharger Awaiting requested review from noCharger noCharger will be requested when the pull request is marked ready for review noCharger is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.