How to use a fixed TensorRT version in triton server?

[niuzheng168]

We converted our ONNX model to TensorRT and host it in triton server with TensorRT backend.
Our system scan Vulnerable Image everyday so we have to keep triton server docker image up to date. However, the server update usually contains TRT runtime update so we may hit error like
Serialization assertion stdVersionRead == serializationVersion failed.Version tag does not match. Note: Current Version: 232, Serialized Engine Version: 237

Then we have to reconvert our ONNX model. And a full test aginst the model.

Is there a way to build a fixed version in Triton Server?

[krishung5]

Hi @niuzheng168, if you'd like to use some specific TRT version, you can try to use LD_LIBRARY_PATH to link with the TRT library with the version you want. For example, after downloading the TRT tar file, do something like

tar -xf TensorRT-8.5.2.2.Linux.x86_64-gnu.cuda-11.8.cudnn8.6.tar.gz
cd TensorRT-8.5.2.2/lib
export LD_LIBRARY_PATH=$PWD:$LD_LIBRARY_PATH
// run Triton again, should be loading the 8.5.2 libs now