Not able to reshape input in replace.json

[longngng]

Issue Type

Others

onnx2tf version number

1.0.27

Download URL for ONNX

https://drive.google.com/file/d/1Wkb-xi8NBICJttIyctUL3m7ETGHv2BZ8/view

Parameter Replacement JSON

{
    "format_version": 1,
    "operations": [
        {
            "op_name": "Flatten_10",
            "param_target": "inputs",
            "param_name": "input.16",
            "pre_process_transpose_perm": [3,1,2,0]
        },
        {
            "op_name": "Flatten_10",
            "param_target": "outputs",
            "param_name": "onnx::Gemm_31",
            "post_process_transpose_perm": [1, 0]
        }
    ]
}

Description

1. School research project on TinyML.
2. I have simplified the model with onnxsim, then I ran this command onnx2tf -i baseline_simplified.onnx -b 1

The onnx model has a Flatten layer. Input shape is ['batch_size', 32, 1, 3], output shape is ['batch_size', 96]. The OP get converted to tf.reshape, the input shape is (1, 1, 3, 32), output shape is (3, 32).

3. I attempted to solve it with the Parameter Replacement JSON to tranpose both the input and output. The transpose for the output works, but the transpose for the input gives me this error.
   

4. I need the tflite model so that I can quantise it and deploy using TFLiteMicro.

5. I have tried to use openvino library to convert onnx to openvino, then openvino to tf. I can convert but the output of tf model is very different from the onnx output.

[PINTO0309]

1. there was a bug in the parameter transposition logic that I have fixed. Thanks for pointing this out. Fixes: fca5c2a
2. Parameter Replacement JSON can be written as follows to correctly modify Flatten behavior. Please try it.

{
  "format_version": 1,
  "operations": [
    {
      "op_name": "Flatten_10",
      "param_target": "inputs",
      "param_name": "input.16",
      "pre_process_transpose_perm": [0,3,1,2]
    },
    {
      "op_name": "Flatten_10",
      "param_target": "attributes",
      "param_name": "axis",
      "values": 1
    }
  ]
}

3. Release: https://github.com/PINTO0309/onnx2tf/releases/tag/1.0.28

You seem to be aware of the trick to change the batch size to a fixed size so please ignore the wording here. I only mention it as a reminder as reference information to other engineers.
#############
The optimization will be suboptimal with variable batch sizes. We recommend changing the batch size to a fixed size whenever possible. If the batch size is variable (batch_size), the shape estimation process before and after Flatten (Reshape) becomes redundant. When using TFLiteMicro, I assume that you are aiming to optimize performance to the limit, so it would be better to avoid redundant models such as the one shown below to further improve performance.
#############

I have tried to use openvino library to convert onnx to openvino, then openvino to tf. I can convert but the output of tf model is very different from the onnx output.

It is understandable that the output results of the model will be corrupted if the dimensions are not reverted to the same dimensional order as the ONNX dimensions before flattening the dimensions. This is also true for openvino2tensorflow. It must be understood that the order of the elements of the array after flattening is different from the expected order.

a[1,32,1,3].flatten() to b[1,96] and a[1,1,3,32].flatten() to b[1,96] are not equivalent.

This is a problem already described in the "Key concept" of the README: the tool's automatic conversion from NCHW to NHWC does not accurately estimate the shape of the pattern.

• Before ONNX
  

• After TFLite
  

• Variable Batch Size
  model: model_float32.tflite.zip
  

• Fixed Batch Size
  model: model_float32.tflite.zip
  

[longngng]

The conversion works now. Thank you very much for the fix and detailed explanation!

[PINTO0309]

Partial fix, Ref: 8954da0
Release: https://github.com/PINTO0309/onnx2tf/releases/tag/1.1.6

• Added Flatten transposition process, transposition behavior similar to Reshape.
• Just before the Flatten process, a forced inversion to NCHW format is applied.
• Improved replace.json to be unnecessary only in some situations.