-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Tutorial - QNN] Prequantized MXNet model compilation. #5362
Conversation
@jwfromm @siju-samuel You might also be interested |
# print(mod) | ||
|
||
# Compile Relay module. Set the target platform. Replace the target with the your target type. | ||
target = 'llvm -mcpu=cascadelake' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this doesn't work on CI
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few nits.
############################################################################### | ||
# Helper functions | ||
# ---------------- | ||
def download_calib_dataset(dataset_url, calib_dataset): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you use from tvm.contrib.download import download_testdata' as in other tutorials to download the calib record, currently its downloading to
./data`
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need more images to perform the calibration. I moved the dataset to /tmp
folder for now.
In this series of tutorials, we demonstrate how to load and run models quantized by PyTorch (Part | ||
1), MXNet (Part 2), and TFLite (Part 3). Once loaded, we can run compiled, quantized models on any | ||
hardware TVM supports. | ||
|
||
This is part 1 of the tutorial, where we will focus on PyTorch-prequantized models. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the 3 tutorials are in different files, suggest we can remove the references to MxNet and TFLite here. May be the below line is enough.
Here, we demonstrate how to load and run models quantized by PyTorch.
Once loaded, we can run compiled, quantized models on any hardware TVM supports.
def get_mxnet_fp32_model(): | ||
""" Read the MXNet symbol. """ | ||
model_name = 'resnet50_v1' | ||
dir_path = os.path.dirname(os.path.realpath(__file__)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dir_path not used, can be removed.
|
||
import mxnet as mx | ||
from gluoncv.model_zoo import get_model | ||
from mxnet.contrib.quantization import * |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if possible, remove wildcard imports
# affected. Output of the following code is as follows | ||
# | ||
# TVM Top-5 labels: [236 211 178 165 168] | ||
# MXNet Top-5 labels: [236 211 178 165 168] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be removed, while document is rendered, it will print.
@tqchen This tutorial requires Currently, the CI failure is
There is no workaround here, I am using MXNet-MKL quantizer to quantize the model. If we want to have this tutorial anyways, I can wrap the tutorial into a function and comment its invocation. And when in future, we have the package, I can remove the comment. |
Can we skip calibration (uncomment to encourage testing locally) and still get meaningful output? I think it is unlikely we can update our CI for the sake of this tutorial. |
We can just use the floating pt model for the reference pt |
@masahi @tqchen The MXNet quantized models has operators that can only work with MKLDNN. Example is as follows. Note the
Even if I quantize the model outside of this tutorial. I would still need |
OK, let us wait for the mxnet-mkl then, currently blocked by #5396 hopefully we can land this week |
@anijain2305 @tqchen What is the status on this? I see there have been a recent CI change. Are we ready to have |
see #5458 |
@masahi Earlier the plan was to have a single tutorial. However, I get seg faults if I import torch and mxnet in the same file. So, I am breaking it down into 3 parts - PyTorch, MXNet and TFLite. I added for Part 2 and massaged the intro to part 1.