Skip to content

Commit

Permalink
docs(triton): update triton adapter docs to make it run
Browse files Browse the repository at this point in the history
  • Loading branch information
lizzzcai committed Oct 19, 2021
1 parent 1be4a07 commit 38c3976
Showing 1 changed file with 14 additions and 7 deletions.
21 changes: 14 additions & 7 deletions model-mesh-triton-adapter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,33 @@ This is an adapter which implements the internal model-mesh model management API

## How to

1. Pull Triton Serving Docker Image
1. Clone the repository

$ git clone https://github.com/kserve/modelmesh-runtime-adapter.git
$ cd modelmesh-runtime-adapter/model-mesh-triton-adapter

2. Pull Triton Serving Docker Image

$ docker pull nvcr.io/nvidia/tritonserver:20.09-py3

2. Run Triton Serving Container with model data mounted
3. Run Triton Serving Container with model data mounted

By default, Triton Serving Docker expose Port `8000` for HTTP and Port `8001` for gRPC.

Using following command to forward container's `8000` to your workstation's `8000` and container's `8001` to your workstation's `8001`.

$ docker run -p 8000:8000 -p 8001:8001 -v /Users/tnarayan/AI/model-mesh-triton-adapter/examples/models/:/models nvcr.io/nvidia/tritonserver:20.09-py3 tritonserver --model-store=/models --model-control-mode=explicit --strict-model-config=false --strict-readiness=false
$ docker run -p 8000:8000 -p 8001:8001 -v $(pwd)/examples/models:/models nvcr.io/nvidia/tritonserver:20.09-py3 tritonserver --model-store=/models --model-control-mode=explicit --strict-model-config=false --strict-readiness=false

3. Setup your Golang, gRPC and Protobuff Development Environment locally
4. Setup your Golang, gRPC and Protobuff Development Environment locally

Follow this [gRPC Go Quick Start Guide](https://grpc.io/docs/quickstart/go/)

4. Run Triton adapter with:
5. Run Triton adapter with:

$ export ROOT_MODEL_DIR=$(pwd)/examples/models
$ export CONTAINER_MEM_REQ_BYTES=268435456 # 256MB
$ go run main.go

5. Test adapter with this client from another terminal:
6. Test adapter with this client from another terminal:

$ go run triton/client/client.go
$ go run triton/adapter_client/adapter_client.go

0 comments on commit 38c3976

Please sign in to comment.