[Bug] For mmdet3d in8 model convert problem

[QLI037]

Checklist

• I have searched related issues but cannot get the expected help.
• 2. I have read the FAQ documentation but cannot get the expected help.
• 3. The bug has not been fixed in the latest version.

Describe the bug

I'm using the “./configs/mmdet3d/voxel-detection/voxel-detection_tensorrt_dynamic-kitti-32x4.py” config file and using int8 mode to try to convert my model configuration file and model weights file to a tensorRT engine file, but I'm having some issues with the conversion process, and the error is basically the same as issue #1048 The error is basically the same as issue #1048, I would like to ask if I need to fix the configuration file as well!
The error message is as follows：
root@b5914d4fa2e0:/mmdetection3d# export MODEL_CONFIG=/root/mmdetection3d/work_dirs/my_pointpillars_kitti/my_pointpillars_kitti.py
root@b5914d4fa2e0:/mmdetection3d# export MODEL_PATH=/root/mmdetection3d/work_dirs/my_pointpillars_kitti/epoch_4.pth
root@b5914d4fa2e0:/mmdetection3d# export TEST_DATA=/root/mmdetection3d/demo/data/kitti/000008.bin
root@b5914d4fa2e0:/mmdetection3d# python3 /root/mmdeploy/tools/deploy.py /root/mmdeploy/configs/mmdet3d/voxel-detection/voxel-detection_tensorrt_dynamic-kitti-32x4.py $MODEL_CONFIG $MODEL_PATH $TEST_DATA --work-dir pointpillars --device cuda
12/08 10:27:26 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
/usr/local/lib/python3.8/dist-packages/mmdet3d/evaluation/functional/kitti_utils/eval.py:10: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
def get_thresholds(scores: np.ndarray, num_gt, num_sample_pts=41):
/usr/local/lib/python3.8/dist-packages/pandas/core/computation/expressions.py:20: UserWarning: Pandas requires version '2.7.3' or newer of 'numexpr' (version '2.7.1' currently installed).
from pandas.core.computation.check import NUMEXPR_INSTALLED
12/08 10:27:29 - mmengine - WARNING - Failed to search registry with scope "mmdet3d" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet3d" is a correct scope, or whether the registry is initialized.
12/08 10:27:29 - mmengine - WARNING - Failed to search registry with scope "mmdet3d" in the "mmdet3d_tasks" registry tree. As a workaround, the current "mmdet3d_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet3d" is a correct scope, or whether the registry is initialized.
/usr/local/lib/python3.8/dist-packages/mmdet3d/models/dense_heads/anchor3d_head.py:94: UserWarning: dir_offset and dir_limit_offset will be depressed and be incorporated into box coder in the future
warnings.warn(
Loads checkpoint by local backend from path: /root/mmdetection3d/work_dirs/my_pointpillars_kitti/epoch_4.pth
12/08 10:27:31 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
12/08 10:27:31 - mmengine - INFO - Export PyTorch model to ONNX: pointpillars/end2end.onnx.
12/08 10:27:32 - mmengine - WARNING - Can not find torch.nn.functional.scaled_dot_product_attention, function rewrite will not be applied
/usr/local/lib/python3.8/dist-packages/mmdeploy/codebase/mmdet3d/models/pillar_encode.py:42: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
f_center = features[..., :3] - (coors[..., 1:] * torch.tensor(
/usr/local/lib/python3.8/dist-packages/mmdeploy/codebase/mmdet3d/models/pillar_encode.py:43: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
[self.vz, self.vy, self.vx]).to(device) + torch.tensor([
/usr/local/lib/python3.8/dist-packages/mmdet3d/models/detectors/voxelnet.py:42: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
batch_size = voxel_dict['coors'][-1, 0].item() + 1
/usr/local/lib/python3.8/dist-packages/mmcv/cnn/bricks/wrappers.py:87: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 4)):
12/08 10:27:33 - mmengine - INFO - Execute onnx optimize passes.
12/08 10:27:33 - mmengine - WARNING - Can not optimize model, please build torchscipt extension.
More details: https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/experimental/onnx_optimizer.md
12/08 10:27:33 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
12/08 10:27:35 - mmengine - INFO - Start pipeline mmdeploy.apis.calibration.create_calib_input_data in subprocess
/usr/local/lib/python3.8/dist-packages/mmdet3d/evaluation/functional/kitti_utils/eval.py:10: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
def get_thresholds(scores: np.ndarray, num_gt, num_sample_pts=41):
/usr/local/lib/python3.8/dist-packages/pandas/core/computation/expressions.py:20: UserWarning: Pandas requires version '2.7.3' or newer of 'numexpr' (version '2.7.1' currently installed).
from pandas.core.computation.check import NUMEXPR_INSTALLED
12/08 10:27:37 - mmengine - WARNING - Failed to search registry with scope "mmdet3d" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet3d" is a correct scope, or whether the registry is initialized.
12/08 10:27:37 - mmengine - WARNING - Failed to search registry with scope "mmdet3d" in the "mmdet3d_tasks" registry tree. As a workaround, the current "mmdet3d_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet3d" is a correct scope, or whether the registry is initialized.
/usr/local/lib/python3.8/dist-packages/mmdet3d/models/dense_heads/anchor3d_head.py:94: UserWarning: dir_offset and dir_limit_offset will be depressed and be incorporated into box coder in the future
warnings.warn(
Loads checkpoint by local backend from path: /root/mmdetection3d/work_dirs/my_pointpillars_kitti/epoch_4.pth
12/08 10:27:40 - mmengine - INFO - ------------------------------
12/08 10:27:40 - mmengine - INFO - The length of test dataset: 3769
12/08 10:27:40 - mmengine - INFO - The number of instances per category in the dataset:
+------------+--------+
| category | number |
+------------+--------+
| Pedestrian | 2280 |
| Cyclist | 893 |
| Car | 14385 |
+------------+--------+
0%| | 0/3769 [00:01<?, ?it/s]
Process Process-3:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 107, in call
ret = func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/calibration.py", line 73, in create_calib_input_data
create_calib_input_data_impl(
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 356, in wrap
return self.call_function(func_name, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
return self.call_function_local(func_name, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
return pipe_caller(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 107, in call
ret = func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/utils/calibration.py", line 67, in create_calib_input_data
input_ndarray = input_tensor.detach().cpu().numpy()
AttributeError: 'dict' object has no attribute 'detach'
12/08 10:27:42 - mmengine - ERROR - /usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 - mmdeploy.apis.calibration.create_calib_input_data with Call id: 1 failed. exit.

Reproduction

root@b5914d4fa2e0:/mmdetection3d# export MODEL_CONFIG=/root/mmdetection3d/work_dirs/my_pointpillars_kitti/my_pointpillars_kitti.py
root@b5914d4fa2e0:/mmdetection3d# export MODEL_PATH=/root/mmdetection3d/work_dirs/my_pointpillars_kitti/epoch_4.pth
root@b5914d4fa2e0:/mmdetection3d# export TEST_DATA=/root/mmdetection3d/demo/data/kitti/000008.bin
root@b5914d4fa2e0:/mmdetection3d# python3 /root/mmdeploy/tools/deploy.py /root/mmdeploy/configs/mmdet3d/voxel-detection/voxel-detection_tensorrt_dynamic-kitti-32x4.py $MODEL_CONFIG $MODEL_PATH $TEST_DATA --work-dir pointpillars --device cuda
12/08 10:27:26 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
/usr/local/lib/python3.8/dist-packages/mmdet3d/evaluation/functional/kitti_utils/eval.py:10: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
def get_thresholds(scores: np.ndarray, num_gt, num_sample_pts=41):
/usr/local/lib/python3.8/dist-packages/pandas/core/computation/expressions.py:20: UserWarning: Pandas requires version '2.7.3' or newer of 'numexpr' (version '2.7.1' currently installed).
from pandas.core.computation.check import NUMEXPR_INSTALLED
12/08 10:27:29 - mmengine - WARNING - Failed to search registry with scope "mmdet3d" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet3d" is a correct scope, or whether the registry is initialized.
12/08 10:27:29 - mmengine - WARNING - Failed to search registry with scope "mmdet3d" in the "mmdet3d_tasks" registry tree. As a workaround, the current "mmdet3d_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet3d" is a correct scope, or whether the registry is initialized.
/usr/local/lib/python3.8/dist-packages/mmdet3d/models/dense_heads/anchor3d_head.py:94: UserWarning: dir_offset and dir_limit_offset will be depressed and be incorporated into box coder in the future
warnings.warn(
Loads checkpoint by local backend from path: /root/mmdetection3d/work_dirs/my_pointpillars_kitti/epoch_4.pth
12/08 10:27:31 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
12/08 10:27:31 - mmengine - INFO - Export PyTorch model to ONNX: pointpillars/end2end.onnx.
12/08 10:27:32 - mmengine - WARNING - Can not find torch.nn.functional.scaled_dot_product_attention, function rewrite will not be applied
/usr/local/lib/python3.8/dist-packages/mmdeploy/codebase/mmdet3d/models/pillar_encode.py:42: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
f_center = features[..., :3] - (coors[..., 1:] * torch.tensor(
/usr/local/lib/python3.8/dist-packages/mmdeploy/codebase/mmdet3d/models/pillar_encode.py:43: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
[self.vz, self.vy, self.vx]).to(device) + torch.tensor([
/usr/local/lib/python3.8/dist-packages/mmdet3d/models/detectors/voxelnet.py:42: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
batch_size = voxel_dict['coors'][-1, 0].item() + 1
/usr/local/lib/python3.8/dist-packages/mmcv/cnn/bricks/wrappers.py:87: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if x.numel() == 0 and obsolete_torch_version(TORCH_VERSION, (1, 4)):
12/08 10:27:33 - mmengine - INFO - Execute onnx optimize passes.
12/08 10:27:33 - mmengine - WARNING - Can not optimize model, please build torchscipt extension.
More details: https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/experimental/onnx_optimizer.md
12/08 10:27:33 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
12/08 10:27:35 - mmengine - INFO - Start pipeline mmdeploy.apis.calibration.create_calib_input_data in subprocess
/usr/local/lib/python3.8/dist-packages/mmdet3d/evaluation/functional/kitti_utils/eval.py:10: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
def get_thresholds(scores: np.ndarray, num_gt, num_sample_pts=41):
/usr/local/lib/python3.8/dist-packages/pandas/core/computation/expressions.py:20: UserWarning: Pandas requires version '2.7.3' or newer of 'numexpr' (version '2.7.1' currently installed).
from pandas.core.computation.check import NUMEXPR_INSTALLED
12/08 10:27:37 - mmengine - WARNING - Failed to search registry with scope "mmdet3d" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet3d" is a correct scope, or whether the registry is initialized.
12/08 10:27:37 - mmengine - WARNING - Failed to search registry with scope "mmdet3d" in the "mmdet3d_tasks" registry tree. As a workaround, the current "mmdet3d_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet3d" is a correct scope, or whether the registry is initialized.
/usr/local/lib/python3.8/dist-packages/mmdet3d/models/dense_heads/anchor3d_head.py:94: UserWarning: dir_offset and dir_limit_offset will be depressed and be incorporated into box coder in the future
warnings.warn(
Loads checkpoint by local backend from path: /root/mmdetection3d/work_dirs/my_pointpillars_kitti/epoch_4.pth
12/08 10:27:40 - mmengine - INFO - ------------------------------
12/08 10:27:40 - mmengine - INFO - The length of test dataset: 3769
12/08 10:27:40 - mmengine - INFO - The number of instances per category in the dataset:
+------------+--------+
| category | number |
+------------+--------+
| Pedestrian | 2280 |
| Cyclist | 893 |
| Car | 14385 |
+------------+--------+
0%| | 0/3769 [00:01<?, ?it/s]
Process Process-3:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 107, in call
ret = func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/calibration.py", line 73, in create_calib_input_data
create_calib_input_data_impl(
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 356, in wrap
return self.call_function(func_name, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
return self.call_function_local(func_name, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
return pipe_caller(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py", line 107, in call
ret = func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/mmdeploy/apis/utils/calibration.py", line 67, in create_calib_input_data
input_ndarray = input_tensor.detach().cpu().numpy()
AttributeError: 'dict' object has no attribute 'detach'
12/08 10:27:42 - mmengine - ERROR - /usr/local/lib/python3.8/dist-packages/mmdeploy/apis/core/pipeline_manager.py - pop_mp_output - 80 - mmdeploy.apis.calibration.create_calib_input_data with Call id: 1 failed. exit.

Environment

TensorRT:8.6.1
CUDA:11.7
pytorch:1.13.0
Nvidia-driver：551.86

Error traceback

No response

[QLI037]

@grimoire Hello, I see that you have solved a similar problem before, could you please provide me with some ideas on how to solve my problem