Grouped Convolution on VPU

[dhpalle]

Hello All,
I am trying to understand the logic behind the assertion at line 404

in the file
https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/src/vpu/graph_transformer/src/stages/convolution.cpp

I copied the assertion below for ease of reference:

    VPU_THROW_UNLESS(output_channels / groups == biases->desc().dim(Dim::C),
                     "number of biases must equal to number of output channels per group, but: "
                     "channels per group=%d, biases=%d",
                     output_channels / groups, biases->desc().dim(Dim::C));

Assertion expects that the number of number of biases to be equal to number of output channels per group.

If the network has layer like the one described below:

from torch import nn
m = nn.Conv3d(in_channels=3,out_channels=3,kernel_size=[3,3,3],stride=[1,2,2],padding=[1,1,1,1,1,1],groups=3)
### We will see that m.weight.shape = torch.Size([3,1,3,3,3])
### m.bias.shape = torch.Size([3])
### m.out_channels = 3
### m.groups = 3

So out_channels/groups /= len(bias)

I converted a model with such layer from onnx to openvino format. Model runs on CPU but fails to run on VPU due to the assertion above.

How can we fix this ?

[jgespino]

Hi @dhpalle

Thanks for reaching out, according to the documentation Convolution-Grouped 3D layer is only supported on CPU and GPU plugins.

@ilya-lavrenov Could you confirm?

Regards,
Jesus

[ilya-lavrenov]

@Maxim-Doronin to comment about MYRIAD layers support.

[dhpalle]

@jgespino
I read the documentation and suspected the issue could be due to lack of support for grouped conv3d. If that is the case, the error message is misleading then. In fact, model fails to execute even with HETERO plugin. If the layer is not supported, then i was expecting a fall back to CPU in case of HETERO. Also, i checked another model with conv3d with group=1 and that works. Is that because at group=1 layer is like regular conv3d. But then, according to documentation that is not supported on VPU.

Executing inference test with  HETERO:MYRIAD,CPU
  0%|                                                                                                                               | 0/53 [00:06<?, ?it/s]
Traceback (most recent call last):
  File "run.py", line 191, in <module>
    main()
  File "run.py", line 188, in main
    run_vino(args)
  File "run.py", line 151, in run_vino
    summ = runBM_session_openvino(ie,net,args.num_iter,bs,provider)
  File "run.py", line 119, in runBM_session_openvino
    exec_net = ie.load_network(network=net, device_name=provider)
  File "ie_api.pyx", line 306, in openvino.inference_engine.ie_api.IECore.load_network
  File "ie_api.pyx", line 315, in openvino.inference_engine.ie_api.IECore.load_network
RuntimeError: Failed to compile layer "Conv_76": number of biases must equal to number of output channels per group, but: channels per group=1, biases=3

[Maxim-Doronin]

@elatkin, could you please comment on it as Conv3D stage author?

[dhpalle]

@elatkin I am not sure how to HETERO plugin works. According to the documentation here (https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_HETERO.html), HETERO plugin should take care of fallback to CPU for case of provider="HETERO:MYRIAD,CPU". If the detection of the no support for groups>1 occurs inside convolution.cpp, will the fallback still occur ?

[ghost]

@dhpalle
Sorry for long delay before responding!

Here is my vision of the situation:

• Grouped 3D convolution is not supported by the Myriad X plugin, unfortunately
• The error message about that is misleading. This is the bug: I am working on fixing the error message, see the pull-request ConvND: fix errmsg if groups > 1 #4199
• We are currently investigating the possibility for supporting grouped 3D convolution on Myriad X processor

Concerning the hetero plugin, I am not sure about performance on ARM processor. I think, by CPU we primarily implied x86, see:

• Supported Devices - OpenVINO™ Toolkit

Anyway, please let me find another person who could explain the hetero plugin. I must confess that I do not know much about the hetero

[alvoron]

@dhpalle
Recently ARM CPU plugin was released as contrib module. It allows to perform deep neural networks inference on ARM CPUs, using OpenVINO API.
The plugin is built as a wrapper over ARM Compute library. Unfortunately, ARM Compute library doesn’t support Conv3D, only 2D is supported.

[avitial]

Ref. 48529

[ilya-lavrenov]

MYRIAD plugin is removed from master branch.