[Bug]: OpenVINO CPU gave inconsistent results with TensorFlow Keras for the GlobalAveragePooling2D

[jikechao]

OpenVINO Version

2023.1.0-12185-9e6b00e51cd-releases/2023/1

Operating System

Ubuntu 18.04 (LTS)

Device used for inference

CPU

Framework

Keras (TensorFlow 2)

Model used

self defined model shown in the following script

Issue description

OpenVINO and Keras gave different inference results about the model containing GlobalAveragePooling2D operator.

Specifically, only OpenVINO using the CPU for inference gave different results with OpenVINO(GPU) and PyTorch(CPU/GPU).

Step-by-step reproduction

import tensorflow as tf
from tensorflow import keras as keras
from tensorflow.keras import layers, models
import numpy as np

import openvino as ov

layer = keras.layers.GlobalAveragePooling2D()
input_shape = [1, 11, 1, 7]
input_data = np.random.randint(2, size=input_shape)
x = layers.Input(shape=input_shape[1:], dtype="int8")
y = layer(x)
model = models.Model(x, y)
# model.summary()
res_keras = model(input_data)

tf2_model_path = f"_temp_model"
tf.saved_model.save(model, tf2_model_path)
ov_model = ov.convert_model(tf2_model_path,  input=input_shape)

ir_path = f"_temp_OVIR.xml"
ov.save_model(ov_model, ir_path, compress_to_fp16=False)
core = ov.Core()
model = core.read_model(ir_path)
compiled_model = core.compile_model(model=model, device_name="CPU")  # GPU: run well

output_key = compiled_model.output(0)

res_dlc = compiled_model(input_data)[output_key]
print(res_keras)
print(res_dlc)
np.testing.assert_allclose(res_keras, res_dlc, atol=1e-3, rtol=1e-3)

Relevant log output

AssertionError: 
Not equal to tolerance rtol=0.001, atol=0.001

Mismatched elements: 5 / 7 (71.4%)
Max absolute difference: 1
Max relative difference: 1.
 x: array([[0, 0, 0, 0, 0, 0, 0]], dtype=int8)
 y: array([[0, 1, 0, 1, 1, 1, 1]], dtype=int8)

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[jikechao]

BTW, GlobalAveragePooling1D, GlobalAveragePooling2D, and GlobalAveragePooling3D have the same issue.

[rkazants]

PR: #21439

[rkazants]

@jikechao, there is one more bug in CPU plugin. Once my PR is merged, I will move it to CPU plugin team. There is also problem with ReduceMean for int8 type implementation there.

Best regards,
Roman

[jikechao]

@rkazants, I understood! Thank you for your explanation.

[rkazants]

Hi @mg-intel,

I fixed the first part of the issue when TF FE was unable to handle int8 type. #21439 was merged.

Now it comes to the second issue on the CPU side. ReduceMean for input int8 type output incorrect results different from TF, example:

import numpy as np
import openvino.runtime.opset9 as ov
from openvino.runtime import Model, Core

param = ov.parameter([1,2,3,1], name="data", dtype=np.int8)
axes = ov.constant(np.array([1, 2], dtype=np.int32))
reduce_mean = ov.reduce_mean(param, axes, False)
model = Model([reduce_mean.output(0)], [param], "model")
core = Core()
compiled_model = core.compile_model(model, "CPU")
data = np.array([[[[1], [1], [1]], [[1], [1], [0]]]], dtype=np.int8)
output = compiled_model.infer_new_request({0: data})
print("ov output = ", output)

# TF part
output = tf.raw_ops.Mean(input=data,axis=[1,2])
print("tf output = ", output)

OV reports 1 and TF reports 0 in the result. Now I re-assign this to CPU to align it with TF.

Best regards,
Roman

[xuchen-intel]

Thanks @jikechao @rkazants for the detailed information!
PR: #25217

[jikechao]

@xuchen-intel Thanks for your efforts!