Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨[Feature] Add precompiled Windows binaries for Libtorch #1008

Closed
jonahclarsen opened this issue Apr 27, 2022 · 3 comments
Closed

✨[Feature] Add precompiled Windows binaries for Libtorch #1008

jonahclarsen opened this issue Apr 27, 2022 · 3 comments
Assignees
Labels
channel: windows bugs, questions, & RFEs around Windows feature request New feature or request

Comments

@jonahclarsen
Copy link

Is your feature request related to a problem? Please describe.
I am new to machine learning, my current project involves inference with a large CNN on Windows. As is, it's too large to fit in VRAM, so I need to quantize it - hence why I need Torch-TensorRT.

However, while I am trying my best to follow #856 and #960, their instructions are not easy to understand since I have no experience with Docker or bazel.

Describe the solution you'd like
I would like Windows binaries (.lib and .dll's) to be officially released.

Describe alternatives you've considered
I briefly considered compiling the torchscript modules in docker first, but then I learned I can't run them without the libtorchtrt_runtime.dll, and I also need it to run on many different hardware setups, and as I understand it, this means I need Torch-TensorRT to compile the AOT module every time. So it seems the only solution is having a fully compiled Torch-TensorRT for Windows.

I am also considering using an older version of TRTorch where Windows binaries were easier to compile, at least to see that it does actually do what I expect and therefore it's worth it to continue - but I expect the older versions don't work with CUDA 11, which I want to deploy this project with, and are also probably less performant and more buggy, so that is not an ideal solution.

Additional context
I am happy to do as much legwork as I can to help get this finished, both for me and for others who could benefit from precompiled binaries in the future. However, as I have such little experience in this area, I expect other members of the dev team could probably get it done in much less time than I could.

@jonahclarsen jonahclarsen added the feature request New feature or request label Apr 27, 2022
@narendasan
Copy link
Collaborator

Yes we have had a lot of requests for this. Look at #1007 to track progress for this thread going forward.

@ncomly-nvidia ncomly-nvidia added the channel: windows bugs, questions, & RFEs around Windows label May 3, 2022
@github-actions
Copy link

github-actions bot commented Aug 2, 2022

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

@ncomly-nvidia
Copy link
Contributor

Please take a look at #1058. Should support your needs!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
channel: windows bugs, questions, & RFEs around Windows feature request New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants