Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

register_jit_hooks: Remove confusing error message #3150

Merged
merged 1 commit into from
Sep 9, 2024

Conversation

HolyWu
Copy link
Contributor

@HolyWu HolyWu commented Sep 8, 2024

Description

import torch
import torch_tensorrt

model = torch.nn.ReLU().eval().cuda()

inputs = [torch.randn(1, 3, 224, 224, device="cuda")]

trt_model = torch_tensorrt.compile(
    model,
    ir="dynamo",
    inputs=inputs,
    enabled_precisions={torch.float},
    min_block_size=1,
    make_refitable=False,
    cache_built_engines=False,
    reuse_cached_engines=False,
)

torch_tensorrt.save(trt_model, "trt.ts", output_format="torchscript", inputs=inputs)
loaded_model = torch_tensorrt.load("trt.ts")
holywu@HOLYWU:~$ python3 test.py
Unable to import quantization op. Please install modelopt library (https://github.com/NVIDIA/TensorRT-Model-Optimizer?tab=readme-ov-file#installation) to add support for compiling quantized models
WARNING:torch_tensorrt.dynamo.utils:Could not detect the device on which the model exists. Assuming the model is on CPU
WARNING: [Torch-TensorRT] - Using default stream in enqueueV3() may lead to performance issues due to additional calls to cudaStreamSynchronize() by TensorRT to ensure correct synchronization. Please use non-default stream instead.
ERROR: [Torch-TensorRT] - linux_x86_64
holywu@HOLYWU:~$

@github-actions github-actions bot added component: core Issues re: The core compiler component: runtime labels Sep 8, 2024
@narendasan narendasan merged commit 1d0916f into pytorch:main Sep 9, 2024
3 checks passed
@HolyWu HolyWu deleted the remove_log_error branch September 10, 2024 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants