🐛 [Bug] Windows native library support (works!)

[markivfb]

Bug Description

Hi, based on some past posts like this and this and this, I was able to get a bazel build working for windows for the c++ library, with both model compilation and inference, by building torch_tensorrt.dll, which I believe includes both the runtime and the compilation plugins.

I am wondering what it would take for official windows support. I've attached a full patch, I think the declspec may not be needed.

Once this builds, all one has to do to use torch_tensorrt in their project is, link to torch_tensorrt.dll.if.lib and ensure torch_tensorrt.dll is loaded up with LoadLibrary as mentioned in other issue reports.

    HMODULE hLib = LoadLibrary(TEXT("torch_tensorrt"));
    if (hLib == NULL) {
        std::cerr << "Library torch_tensorrt.dll not found" << std::endl;
        exit(1);
    }

More info about exact versions of libraries used:
cuda_11.8.0_522.06_windows.exe
TensorRT-8.6.1.6.Windows10.x86_64.cuda-11.8
libtorch-win-shared-with-deps-2.0.1+cu118
cudnn-windows-x86_64-8.8.0.121_cuda11-archive

Build command:
bazel-6.3.2-windows-x86_64.exe build //:libtorchtrt --compilation_mode opt

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• Torch-TensorRT Version (e.g. 1.0.0): 8.6.1.6
• PyTorch Version (e.g. 1.0): 2.0.1
• CPU Architecture:
• OS (e.g., Linux): Windows
• How you installed PyTorch (conda, pip, libtorch, source): libtorch, source
• Build command you used (if compiling from source): bazel
• Are you using local sources or building from archives: latest github?
• Python version:
• CUDA version: 11.8
• GPU models and configuration: 3080Ti
• Any other relevant information:

Additional context

0001-WINDOWS-SUPPORT.patch

[jensdraht1999]

@markivfb Can you use this with Pytorch? Or just native C/C++?

If yes, can you tell me, what you did with the dlls and where did you put them?

[Justin62628]

This script works on my cuda 11.7 too with torch 2.0.1. torch_tensorrt.dll is successfully built. Thanks for your marvelous work.

[jensdraht1999]

@Justin62628 I have build torch_tensorrt.dll successfully too, but something more complicated. However: What do I actually do with these dll files for Pytorch?

[Justin62628]

@Justin62628 I have build torch_tensorrt.dll successfully too, but something more complicated. However: What do I actually do with these dll files for Pytorch?

so far I can only work with these dll files with libtorch.
Normally we should compile wheels of torch_tensorrt for python first, which means you are to exec python setup.py install from this repo to get the _C.pyd extension for torch_tensorrt(the setup script will automatically get all those done for you, which installs torch_tensorrt python api to your site-pakcages), then you are able to import torch_tensorrt in your python project,
I have managed to compile torch_tensorrt _C.pyd with a modified compilation script bazel build //:libtorchtrt --compilation_mode opt --features=windows_export_all_symbols, but there seems to be some linking and symbols exporting mistakes, where I encounters external dlls not found error when trying to import torch_tensorrt into python project like yours

[jensdraht1999]

@Justin62628 Could you share your wheel file for Python, so I could install this? You can upload as a image file here and then I can rename the extension back to .py and install it.

[phineas-pta]

for me i also have to edit file .bazelrc: change -std=c++17 to /std:c++17

[jensdraht1999]

for me i also have to edit file .bazelrc: change -std=c++17 to /std:c++17

Did you manage to somehow work it with pytorch with python?

[phineas-pta]

@jensdraht1999 wheel built successfully but unusable (dll linking error like #856)