Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TensorRT対応 #26

Open
sanamiy opened this issue Sep 12, 2024 · 3 comments
Open

TensorRT対応 #26

sanamiy opened this issue Sep 12, 2024 · 3 comments
Labels
enhancement New feature or request 優先度　低

Comments

@sanamiy
Copy link
Contributor

sanamiy commented Sep 12, 2024 •
edited
Loading

NVIDIA GPUでの推論をより早く軽くしたい

TensorRT 10.4 supports operators in the inclusive range of opset 9 to opset 20
https://github.com/onnx/onnx-tensorrt/blob/10.4-GA/docs/operators.md
@tuna2134
Copy link
Owner

tuna2134 commented Sep 12, 2024 •
edited
Loading

Googlefanさんが昨日話していたのですが、対応が難しいのとCUDAがあるので十分早いのではないかっていう話があります。ただ挑戦する価値はあると思います。

@tuna2134 tuna2134 added enhancement New feature or request 優先度　低 labels Sep 12, 2024
@Googlefan256
Copy link
Collaborator

参考程度に4060tiでは1秒で25回短いテキストを推論できます
ただもっと早くするロマンはありますよね...!

@Googlefan256
Copy link
Collaborator

bertのほうはtrt対応できました。
本体はほかの人に託します...

@Googlefan256 Googlefan256 mentioned this issue Sep 12, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request 優先度　低
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sanamiy @tuna2134 @Googlefan256