Softmax Detect conversion errors in axis and identify the axis with the smallest possible error and replace it.

[PINTO0309]

1. Content and background

• Softmax
  • Detect conversion errors in axis and identify the axis with the smallest possible error and replace it.
  • Automatic selection of the axis with the smallest error by measuring the error of all axes in the process of model transformation.
  • Because the minimum inference is performed multiple times, the conversion speed of the model including Softmax is slightly slower, instead of no longer incurring an accuracy error.
  • Almost eliminates conversion errors in Softmax operations in Transformer models that contain a large number of Softmax, such as MobileFormer.
  • This implementation can be applied to various OPs that handle axis. For example, ReduceXX, ArgMax.
  • The sample JSON file was modified to reflect this improvement. https://github.com/PINTO0309/onnx2tf/blob/main/json_samples/replace_MobileFormer-e9.json

2. Summary of corrections

3. Before/After (If there is an operating log that can be used as a reference)

4. Issue number (only if there is a related issue)

Implementation of strict mode #145
[MobileFormer] Converted model outputs values mismatch with original ones. #105
[MobileFormer]Dimensions must be equal [Add Layer] #103