Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TOPI] Fix CUDA Library Tuning #6132

Merged
merged 3 commits into from
Jul 25, 2020
Merged

Conversation

comaniac
Copy link
Contributor

Although we make conv2d_cudnn.cuda and dense_cublas.cuda as AutoTVM tasks so that they can be "tuned" and compared with other implementations, there have some issues prevent us from actually "tuning" them.

  • conv2d_cudnn.cuda: I constantly got the following errors in cudnnFindConvolutionForwardAlgorithm (on T4 GPU). Note that this function is called when extracting tasks without issues. I guess it might be the issue of CUDA context and threading.

    cuDNN: Check failed: e == CUDNN_STATUS_SUCCESS (2 vs. 0) : CUDNN_STATUS_ALLOC_FAILED
    

    The solution in this PR is to create a knob so that it becomes a template with 8 candidates. In this case, cudnnFindConvolutionForwardAlgorithm will not be called during tuning and everything works well. We still set the knob value to -1 in the fallback config to achieve the same behavoir as now.

  • dense_cblas.cuda: The error comes from the callback function that tries to display the FLOPS. The reason is that task.flops is FloatImm instead of float, so float(flops) will throw type error. This PR lets add_flop function support FloatImm and IntImm types.

cc @icemelon9 @merrymercy

@comaniac comaniac force-pushed the fix_cuda_lib_tuning branch from 88e5f0a to 735a1de Compare July 25, 2020 00:15
@tqchen tqchen merged commit 00578b7 into apache:master Jul 25, 2020
@comaniac comaniac deleted the fix_cuda_lib_tuning branch July 27, 2020 18:15
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Sep 2, 2020
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Sep 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants