rcnn example throws CUDNN_STATUS_BAD_PARAM when running under cudnn 6.0

[ghost]

After I update the mxnet version from 1.1.0 to 1.2.0 and build the repository with CUDA 8.0.61 and cudnn 6.0, the rcnn training throws the following error when evaluating the rpn accuracy.

check failed: e == cuDNN: CUDNN_STATUS_SUCCESS(3 vs. 0) cuDNN: CUDNN_STATUS_BAD_PARAM

The error occurred when executing the following code in example/rcnn/rcnn/core/metric.py:
pred_label = mx.ndarray.argmax_channel(pred).asnumpy().astype('int32')
Any ideas to address this without disabling cudnn or rolling to a former version?

[kalyc]

Thanks for submitting this issue @xioryu You could post this on discuss.mxnet.io for further details on usage of mxnet.

[kalyc]

@nswamy could you add label "Question", "CUDA" to this?

[vrakesh]

@nswamy requesting a label for "Question", or CUDA to this issue

[ijkguo]

It was probably related to old SoftmaxActivation layer. Now changed to mx.sym.softmax in #11373.

[thomelane]

@xioryu are you able to provide some sample code that reproduces this issue? many thanks!

It's not possible to diagnose from just knowing the error occurred on line pred_label = mx.ndarray.argmax_channel(pred).asnumpy().astype('int32'). I'd expect any fatal error in the network to appear when this line is run, just because .asnumpy() blocks and waits for all the async operations to complete (i.e. waits for the network computation to complete).

[ghost]

@thomelane The new simplified repo is OK.

[ijkguo]

The cause of this issue is operator SoftmaxActivation, used in the old complex rcnn example. Two fixes were made and either fixed this issue:

• fix a bug in cudnn softmax activation. #10918 fixed SoftmaxActivation cudnn bug.
• update rcnn example #11373 removed any usage of SoftmaxActivation.

[thomelane]

@xioryu @ijkguo great, and thanks for confirming!

@sandeep-krishnamurthy good to close this ticket now, cheers.