extend describeAPI to support customized metadata #1421

lxning · 2022-02-08T19:26:16Z

Description

Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #(issue)
#1406

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Feature/Issue validation/testing

Please describe the tests [UT/IT] that you ran to verify your changes and relevent result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

Test A
Test B
UT/IT execution results
Positive and negative tests are added in ModelServerTest.java. All tests pass frontend build.
Logs
Test on noop-customized.mar

curl "http://localhost:8081/models/noop-customized/1.0?customized=true"
[
{
"modelName": "noop-customized",
"modelVersion": "1.0",
"modelUrl": "noop-customized.mar",
"runtime": "python",
"minWorkers": 12,
"maxWorkers": 12,
"batchSize": 1,
"maxBatchDelay": 100,
"loadedAtStartup": true,
"workers": [
{
"id": "9000",
"startTime": "2022-02-08T17:12:54.764Z",
"status": "READY",
"memoryUsage": 123383808,
"pid": 25916,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9001",
"startTime": "2022-02-08T17:12:54.765Z",
"status": "READY",
"memoryUsage": 122703872,
"pid": 25915,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9002",
"startTime": "2022-02-08T17:12:54.765Z",
"status": "READY",
"memoryUsage": 122970112,
"pid": 25923,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9003",
"startTime": "2022-02-08T17:12:54.766Z",
"status": "READY",
"memoryUsage": 122798080,
"pid": 25920,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9004",
"startTime": "2022-02-08T17:12:54.766Z",
"status": "READY",
"memoryUsage": 122564608,
"pid": 25917,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9005",
"startTime": "2022-02-08T17:12:54.766Z",
"status": "READY",
"memoryUsage": 122920960,
"pid": 25921,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9006",
"startTime": "2022-02-08T17:12:54.766Z",
"status": "READY",
"memoryUsage": 123609088,
"pid": 25924,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9007",
"startTime": "2022-02-08T17:12:54.767Z",
"status": "READY",
"memoryUsage": 123834368,
"pid": 25919,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9008",
"startTime": "2022-02-08T17:12:54.767Z",
"status": "READY",
"memoryUsage": 123125760,
"pid": 25918,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9009",
"startTime": "2022-02-08T17:12:54.767Z",
"status": "READY",
"memoryUsage": 122621952,
"pid": 25922,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9010",
"startTime": "2022-02-08T17:12:54.767Z",
"status": "READY",
"memoryUsage": 123154432,
"pid": 25926,
"gpu": false,
"gpuUsage": "N/A"
},
{
"id": "9011",
"startTime": "2022-02-08T17:12:54.768Z",
"status": "READY",
"memoryUsage": 124149760,
"pid": 25925,
"gpu": false,
"gpuUsage": "N/A"
}
],
"customizedMetadata": "{\n "data1": "1",\n "data2": "2"\n}"
}
]

Checklist:

Have you added tests that prove your fix is effective or that this feature works?
New and existing unit tests pass locally with these changes?
Has code been commented, particularly in hard-to-understand areas?
Have you made corresponding changes to the documentation?

sagemaker-neo-ci-bot · 2022-02-08T19:49:40Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: 9b91d20
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-08T20:06:43Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: 9b91d20
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-08T20:12:52Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: 9b91d20
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

HamidShojanazeri

Thanks @lxning I could successfully try it, by adding the desribe_handle() to the HF_bert_handler(). output looks like this.

The steps to test would be

adding the describe_handler to the your handler.
setting customized =True when querying describe API.

curl  http://localhost:8081/models/my_tc/?customized=true

docs/management_api.md

sagemaker-neo-ci-bot · 2022-02-11T02:19:12Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: fd5d992
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-11T02:19:56Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: 3cbd322
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-11T02:23:22Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: 3cbd322
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-11T02:33:00Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: fd5d992
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-11T02:34:21Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: 3cbd322
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-11T02:39:23Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: fd5d992
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-11T17:44:26Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: 3e00fe5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-11T17:58:44Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: 3e00fe5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-11T18:05:10Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: 3e00fe5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

HamidShojanazeri · 2022-02-11T18:46:13Z

docs/management_api.md

+        return output_describe
+```
+
+* Implement function _is_describe if handler is inherited from BaseHandler.


@lxning , I think it should be if handler is NOT inherited from BaseHandler

We should make sure we have documented that, it is recommended for custom handlers to extend the BaseHandler

docs/management_api.md

sagemaker-neo-ci-bot · 2022-02-23T04:18:27Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: c3107ee
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T04:33:11Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: c3107ee
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T04:35:49Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: c3107ee
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T05:05:56Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: b55cb30
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T05:17:53Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: b55cb30
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T05:23:54Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: b55cb30
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T18:30:38Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: 8552284
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T18:44:17Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: 8552284
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T18:53:55Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: 8552284
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T19:04:10Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: 5e8e610
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T19:18:06Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: 5e8e610
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T19:24:10Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: 5e8e610
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T22:19:12Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: 543b881
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T22:33:46Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: 543b881
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-23T22:40:07Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: 543b881
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-24T18:13:45Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: a61a3fb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-24T18:25:52Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: a61a3fb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-24T18:32:53Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: a61a3fb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-24T19:04:42Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: a61a3fb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-02-24T19:31:08Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: a61a3fb
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

lxning added 6 commits February 2, 2022 20:12

add describe command in frontend

ef2a354

update describeAPI handler

4c6d8e0

update grpc proto

bc08feb

Merge branch 'master' into issue_1406

b4794f5

unit test

3ac4296

update doc

9b91d20

lxning self-assigned this Feb 8, 2022

lxning added the enhancement New feature or request label Feb 8, 2022

lxning added this to the v0.5.3 milestone Feb 8, 2022

lxning requested review from HamidShojanazeri and maaquib February 8, 2022 19:27

HamidShojanazeri reviewed Feb 9, 2022

View reviewed changes

docs/management_api.md Show resolved Hide resolved

HamidShojanazeri reviewed Feb 9, 2022

View reviewed changes

docs/management_api.md Show resolved Hide resolved

lxning and others added 2 commits February 10, 2022 17:52

update doc

fd5d992

Merge branch 'master' into issue_1406

3cbd322

Trigger build

3e00fe5

HamidShojanazeri reviewed Feb 11, 2022

View reviewed changes

Merge branch 'master' into issue_1406

c3107ee

fix lint issue

b55cb30

Merge branch 'master' into issue_1406

8552284

Merge branch 'master' into issue_1406

5e8e610

Merge branch 'master' into issue_1406

543b881

Merge branch 'master' into issue_1406

a61a3fb

maaquib approved these changes Feb 24, 2022

View reviewed changes

lxning merged commit 162af4f into master Feb 24, 2022

cceyda mentioned this pull request Jul 14, 2022

update torchserve-dashboard to match v0.6 cceyda/torchserve-dashboard#17

Open

cceyda mentioned this pull request Aug 1, 2022

better caching, init option, v0.6 update cceyda/torchserve-dashboard#19

Merged

extend describeAPI to support customized metadata #1421

extend describeAPI to support customized metadata #1421

Conversation

lxning commented Feb 8, 2022 • edited Loading

Description

Type of change

Feature/Issue validation/testing

Checklist:

sagemaker-neo-ci-bot commented Feb 8, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 8, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 8, 2022

AWS CodeBuild CI Report

HamidShojanazeri left a comment • edited Loading

Choose a reason for hiding this comment

sagemaker-neo-ci-bot commented Feb 11, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 11, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 11, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 11, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 11, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 11, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 11, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 11, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 11, 2022

AWS CodeBuild CI Report

HamidShojanazeri Feb 11, 2022

Choose a reason for hiding this comment

HamidShojanazeri Feb 11, 2022

Choose a reason for hiding this comment

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 23, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 24, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 24, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 24, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 24, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Feb 24, 2022

AWS CodeBuild CI Report

lxning commented Feb 8, 2022 •

edited

Loading

HamidShojanazeri left a comment •

edited

Loading