onnx ver.1.4 is not supported

[ZyuZyuZyu]

I am using the Pytorch ver 1.0.0 which is exporting onnx model in ver 1.4.0, directly lead to the upsample operation transfering failed.
the representation of Upsample-op in the older version is now supported looks like
{
'Upsample',
inputs=['X'],
outputs=['Y'],
mode='nearest',
width_scale=2,
height_scale=2,
}
but in the new verison of ONNX, Upsample-op looks like
{
'Upsample',
inputs=['X', 'scales'],
outputs=['Y'],
mode='nearest',
}

Is there a way to modify the model_optimizer source code to make it work tranfering the new ops in ONNX and if yes how can I do it efficiently?

Any advice is welcome !

[shubha-ramani]

Zyu Zyu what is the exact error which Model Optimizer is giving you for the new version ?

[ZyuZyuZyu]

@shubha-ramani Thanks for your reply. The error message is 'One/both of widths_scale = {} and height_scale = {} is not defined for Upsampe node {}.' which located on model_optimizer/extensions/front/onnx/upsample_ext.py line 70.

[shubha-ramani]

Zyu it seems you are right, given the current code located at : https://github.com/opencv/dldt/blob/2018/model-optimizer/extensions/front/onnx/upsample_ext.py
Which does not match the latest PyTorch documentation:
https://pytorch.org/docs/stable/nn.html

I will file a bug. Thank you for using OpenVino.

PS. If you want to hack upsample_ext.py as a temporary workaround and later submit a pull-request yourself feel free.

Shubha

[ZyuZyuZyu]

@shubha-ramani Thank you for comfirming my work. I'm new to the sourse code here now. I will submit a pull-request as long as I finished this work.

[wuhy08]

I wonder if there will be timeline to fix this issue. In ONNX, Upsample-9 op supports dynamic scales, which makes "scales" an input instead of an attribute. I didn't find a way to extract the value of input and convert to attribute in model-optimizer. Is there any hint on that?

[Aeroxander]

@wuhy08 Wondering the same, cause my model has the "Expand" layer what isn't supported < Opset v9

[wuhy08]

Hi @Aeroxander
I don't have Expand Op in my model so I cannot comment on that. But based on ONNX manual, Expand is just broadcasting. I wonder if you can change that Op to Reshape in the model.

[Aeroxander]

@wuhy08 I tried it with this issue but I get Blob "37" not found in const blobs during the loading of the ONNX model after the Expand layer is initialized when I try to map the Expand layer to type "Scale".

I'm just not that familiar with adding layers to OpenCV yet and how OpenCV works with broadcasting.
So tried to emulate how Numpy broadcast in this way:

layerParams.blobs.push_back(Mat::ones(blob.cols, blob.rows, blob.type()));
layerParams.type = "Scale";

But then I get the blob not found error.

When I try to map it to "Reshape" instead of "Scale" I get:
Can't create layer 37 of type Expand in function 'cv::dnn::LayerData...

EDIT: Fixed it with adding constBlobs.insert instead of .push_back, but the same issue arises with the "MatMul" layer now. I'm very confused I keep getting these Blobs not found errors

[lazarevevgeny]

@ZyuZyuZyu is this issue still actual?

[AnastasiaKazantaeva]

It seems that the issues is not actual anymore as no response. Closing it. Feel free to reopen it or create a new one.