-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance regression of quantization on CUDA after [Relay][AutoTVM] Relay op strategy (#4644) #4972
Comments
cc @icemelon9 seems to be using some default schedule? accuracy problem looks weird though. |
After the op strategy the schedule configs on top hub are no longer compatible. Since only LLVM configs have been updated, other targets like CUDA will use default schedule configs and result in performance regression. |
The full precision resnet18v1 model, however, runs well on CUDA after that commit. |
I guess the performance regression could be due to https://github.com/apache/incubator-tvm/blob/master/python/tvm/relay/op/strategy/cuda.py#L88. Previously TVM might use I don't have any idea what causes the accuracy problem. Could you share more about what happens in the |
Thank you for your reply.
I didn't change any code in tvm. |
After auto-tuning on 1070 Max-Q, the speed is much more faster:
The log file follows.
However, the accuracy is still close to zero. Like this:
|
Close for now as the perf regression has beenr esolved. Please open new threads on https://discuss.tvm.ai/ to discuss the accuracy issue :) |
My environment:
Here is my code, which uses resnet18v1 onnx model.
Output when TVM is at ([Fix] Fix get_valid_count flaky test for cuda (#4901)):
Output when TVM is at ([Relay][AutoTVM] Relay op strategy (#4644)):
Besides, the accuracy after the commit is close to zero on ILSVRC2012_img_val dataset.
The text was updated successfully, but these errors were encountered: