Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport multiple PRs to main from 2.x #1663

Merged
merged 24 commits into from
Nov 20, 2023

Conversation

rbhavna
Copy link
Collaborator

@rbhavna rbhavna commented Nov 20, 2023

Description

List of PR's backported from 2.x to main:

#1362
#1447
#1443
#1452
#1448
#1463
#1350
#1465
#1484
#1472
#1479
#1473
#1482
#1485
#1487
#1499
#1516
#1514
#1593
#1461
#1595

Issues Resolved

Multiple commits from PR #1652 were squashed into one commit accidentally. All of them have been reverted and this PR merges all of them as individual commits to main

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

ylwu-amzn and others added 24 commits November 16, 2023 21:07
opensearch-project#1362)

* fix parameter name in preprocess function; fix remote model function name

Signed-off-by: Yaliang Wu <[email protected]>

* fix failed unit test

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
* fixing metrics correlation algorithm

Signed-off-by: Dhrubo Saha <[email protected]>
…pensearch-project#1463)

* if model version fails to register, update model group accordingly

Signed-off-by: Bhavana Ramaram <[email protected]>
* Update Model API POC

Signed-off-by: Sicheng Song <[email protected]>

* Using GetRequest to get model

Signed-off-by: Sicheng Song <[email protected]>

* Finalize model update API

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix compileTest

Signed-off-by: Sicheng Song <[email protected]>

* Add Unit Test Cases for Update Model API

Signed-off-by: Sicheng Song <[email protected]>

* Tune back test coverage thereshold

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit tests on Update model API

Signed-off-by: Sicheng Song <[email protected]>

* Add unit test for TransportUpdateModelAction class

Signed-off-by: Sicheng Song <[email protected]>

* Fix a test error

Signed-off-by: Sicheng Song <[email protected]>

* Change exception thrown to failure response

Signed-off-by: Sicheng Song <[email protected]>

* Move the function judgement to the outter block

Signed-off-by: Sicheng Song <[email protected]>

* Check if model is undeployed before update model

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit test for update model API

Signed-off-by: Sicheng Song <[email protected]>

* Fix unit test due to blocking java 11 CI workflow

Signed-off-by: Sicheng Song <[email protected]>

* Enabling auto bumping model version during registering to a new model group and address reviewers' other concern

Signed-off-by: Sicheng Song <[email protected]>

* Autobump new model groups' latest version when register to a new model

Signed-off-by: Sicheng Song <[email protected]>

* Change the REST API method from POST to PUT

Signed-off-by: Sicheng Song <[email protected]>

* Change the update REST API endpoint

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>
…#1465)

* Add a setting to control the update connector API

Signed-off-by: Sicheng Song <[email protected]>

* Enabling the update connnector setting in unit test

Signed-off-by: Sicheng Song <[email protected]>

* Enabling the update connnector setting in corresponding unit test

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>
* fix update connector API

Signed-off-by: Yaliang Wu <[email protected]>
…search-project#1472) (opensearch-project#1508)

* Performance enhacement for predict action by caching model info

Signed-off-by: zane-neo <[email protected]>

* Add context.restore() to avoid missing info

Signed-off-by: zane-neo <[email protected]>

---------

Signed-off-by: zane-neo <[email protected]>
(cherry picked from commit a985f6e)

Co-authored-by: zane-neo <[email protected]>
…ect#1510)

* fix failed ut from PR 1472

Signed-off-by: Yaliang Wu <[email protected]>

* exclude class for low coverage

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
(cherry picked from commit da5d829)

Co-authored-by: Yaliang Wu <[email protected]>
… status code; fix p… (opensearch-project#1477) (opensearch-project#1509)

* throw exception if remote model doesn't return 2xx status code; fix p… (opensearch-project#1473)

* throw exception if remote model doesn't return 2xx status code; fix predict runner

Signed-off-by: Yaliang Wu <[email protected]>

* fix kmeans model deploy bug

Signed-off-by: Yaliang Wu <[email protected]>

* support multiple docs for remote embedding model

Signed-off-by: Yaliang Wu <[email protected]>

* fix ut

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>

* fix wrong class

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
(cherry picked from commit 201c8a8)

Co-authored-by: Yaliang Wu <[email protected]>
…roject#1482) (opensearch-project#1511)

* fix no worker node exception for remote embedding model

Signed-off-by: Yaliang Wu <[email protected]>

* only add model info to cache if model cache exist

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
(cherry picked from commit 6f83b9f)

Co-authored-by: Yaliang Wu <[email protected]>
…dex not created (opensearch-project#1485) (opensearch-project#1486) (opensearch-project#1512)

* fix for delete model group API throwing incorrect error when model index not created

Signed-off-by: Bhavana Ramaram <[email protected]>
(cherry picked from commit 60ef0fd)

Co-authored-by: Bhavana Ramaram <[email protected]>
(cherry picked from commit 5544681)

Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
…) (opensearch-project#1513)

Signed-off-by: Yaliang Wu <[email protected]>
(cherry picked from commit cea1cd6)

Co-authored-by: Yaliang Wu <[email protected]>
Signed-off-by: Bhavana Ramaram <[email protected]>
…h-project#1595)

* Fix model/connector update API to address appsec concern

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile and build failure

Signed-off-by: Sicheng Song <[email protected]>

* Improve unit test coverage

Signed-off-by: Sicheng Song <[email protected]>

* Fix spotless

Signed-off-by: Sicheng Song <[email protected]>

* Merge update connector feature flag to remote inference feature flag

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix exception status

Signed-off-by: Sicheng Song <[email protected]>

* Keep fixing exception status

Signed-off-by: Sicheng Song <[email protected]>

* Spotless fix

Signed-off-by: Sicheng Song <[email protected]>

* Add UT on parsing exception

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>
@rbhavna rbhavna had a problem deploying to ml-commons-cicd-env November 20, 2023 20:55 — with GitHub Actions Failure
@opensearch-trigger-bot opensearch-trigger-bot bot added the documentation Improvements or additions to documentation label Nov 20, 2023
@rbhavna rbhavna had a problem deploying to ml-commons-cicd-env November 20, 2023 20:56 — with GitHub Actions Failure
@ylwu-amzn ylwu-amzn merged commit 9f438d3 into opensearch-project:main Nov 20, 2023
12 of 22 checks passed
@dhrubo-os
Copy link
Collaborator

I understand this is easier to get approval for multiple PRs altogether. But I feel like this way it's not quite easy to verify the files and perfom the actual review. It seems bit risky IMO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
Status: Planned work items
Development

Successfully merging this pull request may close these issues.

5 participants