What's the difference compared to NVIDIA-AI-IOT / torch2trt? #883
-
|
It seems both torch-tensorrt and torch2trt (https://github.com/NVIDIA-AI-IOT/torch2trt) are from Nvidia. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Torch-TensorRT is designed to be a robust path from PyTorch and TorchScript to TensorRT supporting C++ (via LibTorch) and Python (via PyTorch). Under the hood, Torch-TensorRT compiles stand alone torchscript code (no python dependency) to TensorRT and wraps it in a module, where as torch2trt monkey-patches PyTorch python functions to emit TensorRT layers when they are run, using that to construct the engine which is returned as a module. In terms of advantages Torch-TensorRT looks to be easy to use, support advanced TensorRT features like quantization as well as give you many different deployment options (C++ w/o Python for example). It also has the ability to automatically segment your model in to supported and unsupported components so you can still get some performance gains even if TensorRT does not support all the operators you are using. |
Beta Was this translation helpful? Give feedback.
Torch-TensorRT is designed to be a robust path from PyTorch and TorchScript to TensorRT supporting C++ (via LibTorch) and Python (via PyTorch).
Under the hood, Torch-TensorRT compiles stand alone torchscript code (no python dependency) to TensorRT and wraps it in a module, where as torch2trt monkey-patches PyTorch python functions to emit TensorRT layers when they are run, using that to construct the engine which is returned as a module.
In terms of advantages Torch-TensorRT looks to be easy to use, support advanced TensorRT features like quantization as well as give you many different deployment options (C++ w/o Python for example). It also has the ability to automatically segment your m…