Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.11] Rebased - return parsing exception 400 for parsing errors #1603

Merged
merged 1 commit into from
Nov 6, 2023

Conversation

Zhangxunmt
Copy link
Collaborator

Description

Rebased the PR #1593 and backport into 2.11.

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Xun Zhang <[email protected]>

add more ut in restupdateconnector

Signed-off-by: Xun Zhang <[email protected]>
Copy link

codecov bot commented Nov 6, 2023

Codecov Report

Merging #1603 (325e585) into 2.11 (bf51943) will decrease coverage by 0.04%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               2.11    #1603      +/-   ##
============================================
- Coverage     79.97%   79.94%   -0.04%     
+ Complexity     2461     2459       -2     
============================================
  Files           197      197              
  Lines          9829     9831       +2     
  Branches        992      992              
============================================
- Hits           7861     7859       -2     
- Misses         1500     1503       +3     
- Partials        468      469       +1     
Flag Coverage Δ
ml-commons 79.94% <100.00%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...pensearch/ml/rest/RestMLUpdateConnectorAction.java 90.47% <100.00%> (+1.00%) ⬆️

... and 1 file with indirect coverage changes

@Zhangxunmt Zhangxunmt temporarily deployed to ml-commons-cicd-env November 6, 2023 19:20 — with GitHub Actions Inactive
@Zhangxunmt Zhangxunmt temporarily deployed to ml-commons-cicd-env November 6, 2023 19:20 — with GitHub Actions Inactive
@Zhangxunmt Zhangxunmt temporarily deployed to ml-commons-cicd-env November 6, 2023 19:20 — with GitHub Actions Inactive
@dhrubo-os dhrubo-os merged commit 6ef198f into opensearch-project:2.11 Nov 6, 2023
8 of 9 checks passed
b4sjoo pushed a commit to b4sjoo/ml-commons that referenced this pull request Nov 13, 2023
)

add more ut in restupdateconnector

Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>
ylwu-amzn pushed a commit that referenced this pull request Nov 13, 2023
* Update Model API (#1350)

* Update Model API POC

Signed-off-by: Sicheng Song <[email protected]>

* Using GetRequest to get model

Signed-off-by: Sicheng Song <[email protected]>

* Finalize model update API

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix compileTest

Signed-off-by: Sicheng Song <[email protected]>

* Add Unit Test Cases for Update Model API

Signed-off-by: Sicheng Song <[email protected]>

* Tune back test coverage thereshold

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit tests on Update model API

Signed-off-by: Sicheng Song <[email protected]>

* Add unit test for TransportUpdateModelAction class

Signed-off-by: Sicheng Song <[email protected]>

* Fix a test error

Signed-off-by: Sicheng Song <[email protected]>

* Change exception thrown to failure response

Signed-off-by: Sicheng Song <[email protected]>

* Move the function judgement to the outter block

Signed-off-by: Sicheng Song <[email protected]>

* Check if model is undeployed before update model

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit test for update model API

Signed-off-by: Sicheng Song <[email protected]>

* Fix unit test due to blocking java 11 CI workflow

Signed-off-by: Sicheng Song <[email protected]>

* Enabling auto bumping model version during registering to a new model group and address reviewers' other concern

Signed-off-by: Sicheng Song <[email protected]>

* Autobump new model groups' latest version when register to a new model

Signed-off-by: Sicheng Song <[email protected]>

* Change the REST API method from POST to PUT

Signed-off-by: Sicheng Song <[email protected]>

* Change the update REST API endpoint

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>

* Fix java compile when merging

Signed-off-by: Sicheng Song <[email protected]>

* Fix model/connector update API to address security concern (#1595)

* Fix model/connector update API to address appsec concern

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile and build failure

Signed-off-by: Sicheng Song <[email protected]>

* Improve unit test coverage

Signed-off-by: Sicheng Song <[email protected]>

* Fix spotless

Signed-off-by: Sicheng Song <[email protected]>

* Merge update connector feature flag to remote inference feature flag

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix exception status

Signed-off-by: Sicheng Song <[email protected]>

* Keep fixing exception status

Signed-off-by: Sicheng Song <[email protected]>

* Spotless fix

Signed-off-by: Sicheng Song <[email protected]>

* Add UT on parsing exception

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>

* return parsing exception 400 for parsing errors (#1603)

add more ut in restupdateconnector

Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>

* throttling and quota feature on single node cluster

Signed-off-by: Sicheng Song <[email protected]>

* Fix java compile when merging

Signed-off-by: Sicheng Song <[email protected]>

* Enabling in-place update on multi-node

Signed-off-by: Sicheng Song <[email protected]>

* Fix confidential rotation in update internal connector

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Co-authored-by: Xun Zhang <[email protected]>
b4sjoo added a commit to b4sjoo/ml-commons that referenced this pull request Nov 17, 2023
…roject#1624)

* Update Model API (opensearch-project#1350)

* Update Model API POC

Signed-off-by: Sicheng Song <[email protected]>

* Using GetRequest to get model

Signed-off-by: Sicheng Song <[email protected]>

* Finalize model update API

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix compileTest

Signed-off-by: Sicheng Song <[email protected]>

* Add Unit Test Cases for Update Model API

Signed-off-by: Sicheng Song <[email protected]>

* Tune back test coverage thereshold

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit tests on Update model API

Signed-off-by: Sicheng Song <[email protected]>

* Add unit test for TransportUpdateModelAction class

Signed-off-by: Sicheng Song <[email protected]>

* Fix a test error

Signed-off-by: Sicheng Song <[email protected]>

* Change exception thrown to failure response

Signed-off-by: Sicheng Song <[email protected]>

* Move the function judgement to the outter block

Signed-off-by: Sicheng Song <[email protected]>

* Check if model is undeployed before update model

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit test for update model API

Signed-off-by: Sicheng Song <[email protected]>

* Fix unit test due to blocking java 11 CI workflow

Signed-off-by: Sicheng Song <[email protected]>

* Enabling auto bumping model version during registering to a new model group and address reviewers' other concern

Signed-off-by: Sicheng Song <[email protected]>

* Autobump new model groups' latest version when register to a new model

Signed-off-by: Sicheng Song <[email protected]>

* Change the REST API method from POST to PUT

Signed-off-by: Sicheng Song <[email protected]>

* Change the update REST API endpoint

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>

* Fix java compile when merging

Signed-off-by: Sicheng Song <[email protected]>

* Fix model/connector update API to address security concern (opensearch-project#1595)

* Fix model/connector update API to address appsec concern

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile and build failure

Signed-off-by: Sicheng Song <[email protected]>

* Improve unit test coverage

Signed-off-by: Sicheng Song <[email protected]>

* Fix spotless

Signed-off-by: Sicheng Song <[email protected]>

* Merge update connector feature flag to remote inference feature flag

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix exception status

Signed-off-by: Sicheng Song <[email protected]>

* Keep fixing exception status

Signed-off-by: Sicheng Song <[email protected]>

* Spotless fix

Signed-off-by: Sicheng Song <[email protected]>

* Add UT on parsing exception

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>

* return parsing exception 400 for parsing errors (opensearch-project#1603)

add more ut in restupdateconnector

Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>

* throttling and quota feature on single node cluster

Signed-off-by: Sicheng Song <[email protected]>

* Fix java compile when merging

Signed-off-by: Sicheng Song <[email protected]>

* Enabling in-place update on multi-node

Signed-off-by: Sicheng Song <[email protected]>

* Fix confidential rotation in update internal connector

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Co-authored-by: Xun Zhang <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>
b4sjoo added a commit that referenced this pull request Nov 17, 2023
* Merge update model API and model level throttling/quota (#1624)

* Update Model API (#1350)

* Update Model API POC

Signed-off-by: Sicheng Song <[email protected]>

* Using GetRequest to get model

Signed-off-by: Sicheng Song <[email protected]>

* Finalize model update API

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix compileTest

Signed-off-by: Sicheng Song <[email protected]>

* Add Unit Test Cases for Update Model API

Signed-off-by: Sicheng Song <[email protected]>

* Tune back test coverage thereshold

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit tests on Update model API

Signed-off-by: Sicheng Song <[email protected]>

* Add unit test for TransportUpdateModelAction class

Signed-off-by: Sicheng Song <[email protected]>

* Fix a test error

Signed-off-by: Sicheng Song <[email protected]>

* Change exception thrown to failure response

Signed-off-by: Sicheng Song <[email protected]>

* Move the function judgement to the outter block

Signed-off-by: Sicheng Song <[email protected]>

* Check if model is undeployed before update model

Signed-off-by: Sicheng Song <[email protected]>

* Add more unit test for update model API

Signed-off-by: Sicheng Song <[email protected]>

* Fix unit test due to blocking java 11 CI workflow

Signed-off-by: Sicheng Song <[email protected]>

* Enabling auto bumping model version during registering to a new model group and address reviewers' other concern

Signed-off-by: Sicheng Song <[email protected]>

* Autobump new model groups' latest version when register to a new model

Signed-off-by: Sicheng Song <[email protected]>

* Change the REST API method from POST to PUT

Signed-off-by: Sicheng Song <[email protected]>

* Change the update REST API endpoint

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>

* Fix java compile when merging

Signed-off-by: Sicheng Song <[email protected]>

* Fix model/connector update API to address security concern (#1595)

* Fix model/connector update API to address appsec concern

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile and build failure

Signed-off-by: Sicheng Song <[email protected]>

* Improve unit test coverage

Signed-off-by: Sicheng Song <[email protected]>

* Fix spotless

Signed-off-by: Sicheng Song <[email protected]>

* Merge update connector feature flag to remote inference feature flag

Signed-off-by: Sicheng Song <[email protected]>

* Fix compile

Signed-off-by: Sicheng Song <[email protected]>

* Fix exception status

Signed-off-by: Sicheng Song <[email protected]>

* Keep fixing exception status

Signed-off-by: Sicheng Song <[email protected]>

* Spotless fix

Signed-off-by: Sicheng Song <[email protected]>

* Add UT on parsing exception

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>

* return parsing exception 400 for parsing errors (#1603)

add more ut in restupdateconnector

Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>

* throttling and quota feature on single node cluster

Signed-off-by: Sicheng Song <[email protected]>

* Fix java compile when merging

Signed-off-by: Sicheng Song <[email protected]>

* Enabling in-place update on multi-node

Signed-off-by: Sicheng Song <[email protected]>

* Fix confidential rotation in update internal connector

Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Co-authored-by: Xun Zhang <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>

* merge conflict

Signed-off-by: Sicheng Song <[email protected]>

* Change rate limiter token capacity setting (#1635)

Signed-off-by: Sicheng Song <[email protected]>

* support step size for embedding model which outputs less embeddings (#1586)

* support step size for embedding model which outputs less embeddings

Signed-off-by: Yaliang Wu <[email protected]>

* tune parameter name

Signed-off-by: Yaliang Wu <[email protected]>

* fine tune processed doc to always respect step size

Signed-off-by: Yaliang Wu <[email protected]>

---------

Signed-off-by: Yaliang Wu <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>

* validate step size (#1587)

Signed-off-by: Yaliang Wu <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>

* backport to 2.11 (#1639)

Signed-off-by: xinyual <[email protected]>
Signed-off-by: Sicheng Song <[email protected]>

---------

Signed-off-by: Sicheng Song <[email protected]>
Signed-off-by: Xun Zhang <[email protected]>
Signed-off-by: Yaliang Wu <[email protected]>
Signed-off-by: xinyual <[email protected]>
Co-authored-by: Xun Zhang <[email protected]>
Co-authored-by: Yaliang Wu <[email protected]>
Co-authored-by: xinyual <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants