Skip to content

Latest commit

 

History

History
14 lines (11 loc) · 1.14 KB

triton_server.md

File metadata and controls

14 lines (11 loc) · 1.14 KB

Prepare Triton Server For Native Inferencing

As mentioned in the README, the DeepStream applications should work as Triton client with Triton Server running natively for cAPIs. So the Triton Inference Server libraries should be installed in the machine. An easier way is to run the DeepStream application in the DeepStream Triton container.

Running DeepStream Triton container, takes the DeepStream 7.1 container as the example:

    docker run --gpus all -it  --ipc=host --rm --privileged -v /tmp/.X11-unix:/tmp/.X11-unix  -v $(pwd):/samples   -e DISPLAY=$DISPLAY -w /samples nvcr.io/nvidia/deepstream:7.1-gc-triton-devel

Inside the container, prepare model engines for Triton server, the tao-converter links inside the prepare_triton_models.sh scripts can be changed to proper versions according to the actual TensorRT version:

    ./build_triton_engine.sh

Then the DeepStream sample application can be build and run inside this container.