Does mmdeploy supports onnx model quantization(onnx model with fp16 mode)??

[sanjaypavo]

Hi. I need to deploy my model(any object detection model) in onnx format in fp16 mode.Is it possible in mmdeploy?

Thanks in advance..

[tpoisonooo]

In theory, fp16 mode is not a kind of quantization, it is just convert fp32 value with bf16 format.

onnx format with fp16 precision not tested now.

mmdeploy using ppq to quantize ncnn int8. please check

• https://github.com/open-mmlab/mmdeploy/blob/master/docs/zh_cn/03-benchmark/quantization.md to get some test result.
• https://github.com/open-mmlab/mmdeploy/blob/master/docs/zh_cn/02-how-to-run/quantize_model.md to quantize an ncnn int8 model

[tpoisonooo]

Looks like I have to write an English version of the quantization doc +_+

[tpoisonooo]

#842

[JIAOJINYU]

@tpoisonooo
I apologize for the sudden bother you.
I would like to ask you 2 questions.

1. Does mmdeploy only support fp16 level quantization for onnxruntime at this moment?
2. I currently would like to quantize rtmpose to int8. I try to quantize rtmpose to int8 using onnxruntime's static quantization. but the accuracy of the quantized model has been zero. Can I modify the pose-detection_onnxruntime-fp16_static.py to pose-detection_onnxruntime-int8_static.py by myself to realize the int8 quantization using mmdeploy?