[Good First Issue]: Align behavior of ONNX Frontend operator AveragePool-7, 10, 11, 19 with original framework

[gkrivor]

Context

Neural networks are graphs consisting of nodes called operators. Each operator corresponds to a mathematical function, usually described in framework's documentation or an AI standard, such as ONNX.
OpenVINO ONNX Frontend is a component responsible for working with ONNX graphs and requires implementation of different ONNX operators in order to use ONNX models.
This task requires alignment between OpenVINO ONNX Frontend and original framework implementations of AveragePool for next list of opsets: opset 7, opset 10, opset 11, opset 19
Necessary help will be provided by ONNX Fronted team.

What needs to be done?

Operator details can be found in ONNX Operators
More details can be found in ONNX Changelog: opset 7, opset 10, opset 11, opset 19

1. Operator already has a common implementation in OpenVINO. First of all, you need to review a documentation and prepare a table with differences between versions. It could be, for instance, a missing property, extended/reduced coverage of existing property, etc...
2. Copy existing implementation here to make it aligned with original framework (extend or reduce coverage of a common implementation). Copy of modified implementation should be in a defined opset, or in opset 1 in case it implements oldest implementation. Example of multi-opset operation.
3. Register the function in ops_bridge.cpp while keeping alphabetical order
4. Create test model(s) in ONNX models directory. OpenVINO test infrastructure then converts prototxt files to ONNX models - you will use those models later in tests
5. Add tests covering all use cases here
6. Check Python xfailed tests to find a test marked as a xfailed for added functionality. If any exist - remove corresponding lines and try to verify by using cmdline "python -m pytest -k name_of_test".
   More details in adding operators to ONNX Frontend guide

Example Pull Requests

No response

Resources

• Contribution guide - start here!
• Intel DevHub Discord channel - engage in discussions, ask questions and talk to OpenVINO developers
• What is OpenVINO?
• User documentation
• Guides on ONNX Frontend architecture and ONNX Frontend tests are also available
• You can also see a blog post on contributing to OpenVINO

Contact points

@gkrivor

Ticket

No response

[Sukhvansh2004]

Hello I woulld like to work on this issue.

[Sukhvansh2004]

Thank you for giving me this issue to work on.

[Sukhvansh2004]

I have created a table regarding the differences and changes in different versions of average pooling, however I am not getting on how to put these changes into the repository as mentioned in the issue. It would be great if I could get some help regarding this.

[gkrivor]

Hi, could you put your table here?

[Sukhvansh2004]

Sure, here is the table of differences I infered and created from the documentation link provided in the issue.

I am also attatching my fork so you can see the changes which I have made till now
https://github.com/Sukhvansh2004/openvino.git

[Sukhvansh2004]

@gkrivor sir, I had forgotten to put a small change in op-7 to the initial version of avg pool. Could you furthur assist me regarding the onnx frontend on how and where all to change the code. I have read the frontend docs but still facing some difficulties and issues, would be great if you could assist.

[Sukhvansh2004]

@p-wysocki @gkrivor Sir, I have quite some changes in the repo to align op-7, 10, 11 and 19, however to make any changes furthur I would need to run test cases for which I would be required to build the repo, but for some reason my CMake command is giving errors while building the repo could you pls help me with this.

I am getting the following error.

cmake -G "Visual Studio 16 2019" "C:\Users\Sukhvansh Jain\Desktop\Open source\OpenVINO\openvino"
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22631.
-- OpenVINO version is 2024.0.0 (Build 13907)
-- OpenVINO Runtime enabled features:

-- CI_BUILD_NUMBER: 2024.0.0-13907-c689a4096d4
-- CPACK_GENERATOR = TGZ
-- ENABLE_LTO = OFF
-- OS_FOLDER = OFF
-- USE_BUILD_TYPE_SUBFOLDER = OFF
-- CMAKE_COMPILE_WARNING_AS_ERROR = OFF
-- ENABLE_QSPECTRE = OFF
-- ENABLE_INTEGRITYCHECK = OFF
-- ENABLE_SANITIZER = OFF
-- ENABLE_UB_SANITIZER = OFF
-- ENABLE_THREAD_SANITIZER = OFF
-- ENABLE_COVERAGE = OFF
-- ENABLE_SSE42 = ON
-- ENABLE_AVX2 = ON
-- ENABLE_AVX512F = ON
-- BUILD_SHARED_LIBS = ON
-- ENABLE_LIBRARY_VERSIONING = OFF
-- ENABLE_FASTER_BUILD = OFF
-- ENABLE_CPPLINT = OFF
-- ENABLE_CPPLINT_REPORT = OFF
-- ENABLE_CLANG_FORMAT = OFF
-- ENABLE_NCC_STYLE = OFF
-- ENABLE_UNSAFE_LOCATIONS = OFF
-- ENABLE_FUZZING = OFF
-- ENABLE_PROXY = ON
-- ENABLE_INTEL_CPU = ON
-- ENABLE_ARM_COMPUTE_CMAKE = OFF
-- ENABLE_TESTS = OFF
-- ENABLE_INTEL_GPU = ON
-- ENABLE_ONEDNN_FOR_GPU = ON
-- ENABLE_DEBUG_CAPS = OFF
-- ENABLE_GPU_DEBUG_CAPS = OFF
-- ENABLE_CPU_DEBUG_CAPS = OFF
-- ENABLE_SNIPPETS_DEBUG_CAPS = OFF
-- ENABLE_PROFILING_ITT = OFF
-- ENABLE_PROFILING_FILTER = ALL
-- ENABLE_PROFILING_FIRST_INFERENCE = ON
-- SELECTIVE_BUILD = OFF
-- ENABLE_DOCS = OFF
-- ENABLE_PKGCONFIG_GEN = OFF
-- THREADING = TBB
-- ENABLE_TBBBIND_2_5 = ON
-- ENABLE_TBB_RELEASE_ONLY = OFF
-- ENABLE_MULTI = ON
-- ENABLE_AUTO = ON
-- ENABLE_AUTO_BATCH = ON
-- ENABLE_HETERO = ON
-- ENABLE_TEMPLATE = ON
-- ENABLE_PLUGINS_XML = OFF
-- ENABLE_FUNCTIONAL_TESTS = OFF
-- ENABLE_SAMPLES = ON
-- ENABLE_OV_ONNX_FRONTEND = ON
-- ENABLE_OV_PADDLE_FRONTEND = ON
-- ENABLE_OV_IR_FRONTEND = ON
-- ENABLE_OV_PYTORCH_FRONTEND = ON
-- ENABLE_OV_IR_FRONTEND = ON
-- ENABLE_OV_TF_FRONTEND = ON
-- ENABLE_OV_TF_LITE_FRONTEND = ON
-- ENABLE_SNAPPY_COMPRESSION = ON
-- ENABLE_STRICT_DEPENDENCIES = OFF
-- ENABLE_SYSTEM_TBB = OFF
-- ENABLE_SYSTEM_PUGIXML = OFF
-- ENABLE_SYSTEM_FLATBUFFERS = ON
-- ENABLE_SYSTEM_OPENCL = OFF
-- ENABLE_SYSTEM_PROTOBUF = OFF
-- ENABLE_SYSTEM_SNAPPY = OFF
-- ENABLE_PYTHON_PACKAGING = OFF
-- ENABLE_OPENVINO_DEBUG = OFF

-- CMAKE_VERSION ......................... 3.28.1
-- OpenVINO_SOURCE_DIR ................... C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino
-- OpenVINO_BINARY_DIR ................... C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino/build
-- CMAKE_GENERATOR ....................... Visual Studio 16 2019
-- CPACK_GENERATOR ....................... TGZ
-- CMAKE_C_COMPILER_ID ................... MSVC
-- CMAKE_CXX_COMPILER_ID ................. MSVC
-- CMAKE_CXX_STANDARD .................... 14
-- CMAKE_CONFIGURATION_TYPES ............. Debug Release MinSizeRel RelWithDebInfo
CMake Error at thirdparty/ocl/CMakeLists.txt:41 (add_subdirectory):
The source directory

C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino/thirdparty/ocl/icd_loader

does not contain a CMakeLists.txt file.

CMake Error at thirdparty/ocl/CMakeLists.txt:44 (target_compile_definitions):
Cannot specify compile definitions for target "OpenCL" which is not built
by this project.

CMake Error at thirdparty/ocl/CMakeLists.txt:46 (target_include_directories):
Cannot specify include directories for target "OpenCL" which is not built
by this project.

CMake Error at thirdparty/dependencies.cmake:98 (get_target_property):
get_target_property() called with non-existent target "OpenCL::OpenCL".
Call Stack (most recent call first):
CMakeLists.txt:136 (include)

-- Protocol Buffers Configuring...

-- 3.20.3.0
-- Configuration script parsing status [
-- Description : Protocol Buffers
-- Version : 3.20.3.0 (3.20.3)
-- Contact : [email protected]
-- ]
-- Protocol Buffers Configuring done
-- Proceeding with version: 23.3.3.0
-- CMAKE_CXX_FLAGS: /DWIN32 /D_WINDOWS /GR /EHsc /D_CRT_SECURE_NO_WARNINGS /D_SCL_SECURE_NO_WARNINGS /EHsc /Gy /W3 /bigobj /MP /wd4251 /wd4275
CMake Warning (dev) at thirdparty/onnx/onnx/CMakeLists.txt:112 (find_package):
Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
are removed. Run "cmake --help-policy CMP0148" for policy details. Use
the cmake_policy command to set the policy and suppress this warning.

This warning is for project developers. Use -Wno-dev to suppress it.

Generated: C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino/build/thirdparty/onnx/onnx/onnx/onnx_openvino_onnx-ml.proto
Generated: C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino/build/thirdparty/onnx/onnx/onnx/onnx-operators_openvino_onnx-ml.proto
Generated: C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino/build/thirdparty/onnx/onnx/onnx/onnx-data_openvino_onnx.proto

-- ******** Summary ********
-- CMake version : 3.28.1
-- CMake command : C:/Program Files/CMake/bin/cmake.exe
-- System : Windows
-- C++ compiler : C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe
-- C++ compiler version : 19.29.30153.0
-- CXX flags : /DWIN32 /D_WINDOWS /GR /EHsc /D_CRT_SECURE_NO_WARNINGS /D_SCL_SECURE_NO_WARNINGS /EHsc /Gy /W3 /bigobj /MP /wd4251 /wd4275 /wd4244 /EHsc /wd26812
-- Build type : Release
-- Compile definitions : OV_BUILD_POSTFIX="$<$CONFIG:Debug:d>$<$CONFIG:Release:>";IE_BUILD_POSTFIX="$<$CONFIG:Debug:d>$<$CONFIG:Release:>";OV_NATIVE_PARENT_PROJECT_ROOT_DIR="openvino";__STDC_FORMAT_MACROS
-- CMAKE_PREFIX_PATH :
-- CMAKE_INSTALL_PREFIX : C:/Program Files (x86)/OpenVINO
-- CMAKE_MODULE_PATH :

-- ONNX version : 1.15.0
-- ONNX NAMESPACE : openvino_onnx
-- ONNX_USE_LITE_PROTO : ON
-- USE_PROTOBUF_SHARED_LIBS : OFF
-- Protobuf_USE_STATIC_LIBS : ON
-- ONNX_DISABLE_EXCEPTIONS : OFF
-- ONNX_DISABLE_STATIC_REGISTRATION : OFF
-- ONNX_WERROR : OFF
-- ONNX_BUILD_TESTS : OFF
-- ONNX_BUILD_BENCHMARKS : OFF
-- ONNX_BUILD_SHARED_LIBS :
-- BUILD_SHARED_LIBS : OFF

-- Protobuf compiler :
-- Protobuf includes :
-- Protobuf libraries :
-- BUILD_ONNX_PYTHON : OFF
-- Using the multi-header code from C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino/thirdparty/json/nlohmann_json/include/
-- TBB (2021.2.2) is found at C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino/temp/tbb/lib/cmake/TBB
-- Static tbbbind_2_5 package usage is disabled, since oneTBB (ver. 2021.2.2) provides dynamic TBBBind 2.5+
CMake Error at src/plugins/intel_cpu/thirdparty/CMakeLists.txt:139 (add_subdirectory):
The source directory

C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino/src/plugins/intel_cpu/thirdparty/mlas

does not contain a CMakeLists.txt file.

CMake Error at src/plugins/intel_cpu/thirdparty/CMakeLists.txt:118 (add_subdirectory):
The source directory

C:/Users/Sukhvansh Jain/Desktop/Open source/OpenVINO/openvino/src/plugins/intel_cpu/thirdparty/onednn

does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
src/plugins/intel_cpu/thirdparty/CMakeLists.txt:143 (ov_add_onednn)

-- Python3 executable: C:/Users/Sukhvansh Jain/AppData/Local/Programs/Python/Python310/python.exe
-- Python3 version: 3.10.9
-- Python3 interpreter ID: Python
-- Python3 SOABI: cp310-win_amd64
-- Python3 include dirs: C:/Users/Sukhvansh Jain/AppData/Local/Programs/Python/Python310/include
-- Python3 libraries: C:/Users/Sukhvansh Jain/AppData/Local/Programs/Python/Python310/libs/python310.lib
CMake Warning at samples/cpp/common/format_reader/CMakeLists.txt:22 (message):
OpenCV ver. 3.0+ is not found, format_reader will be built without OpenCV
support

CMake Warning at samples/cpp/benchmark_app/CMakeLists.txt:104 (message):
OpenCL CPP header is not found, benchmark_app will be built without OpenCL
support and you will not be able to use the '-use_device_mem' option.
Please, install '<apt | yum> install opencl-headers' to enable the option

CMake Warning at samples/cpp/benchmark_app/CMakeLists.txt:112 (message):
OpenCV ver. 3.0+ is not found, benchmark_app will be built without OpenCV
support. Set OpenCV_DIR

CMake Warning at samples/c/common/opencv_c_wrapper/CMakeLists.txt:20 (message):
OpenCV ver. 3.0+ is not found, opencv_c_wrapper is built without OPENCV
support

-- Register template to be built in build-modules/template
-- Register template_extension to be built in build-modules/template_extension
CMake Warning at tools/mo/CMakeLists.txt:5 (message):
Please enable OpenVINO Runtime Python API (set -DENABLE_PYTHON=ON) targets
to enable Model Optimizer target

CMake Warning at cmake/developer_package/shellcheck/shellcheck.cmake:22 (message):
shellcheck tool is not found
Call Stack (most recent call first):
scripts/CMakeLists.txt:18 (ov_shellcheck_process)

-- Configuring incomplete, errors occurred!

[gkrivor]

Hi, sorry for a delay, I have a time to check github tuesday-wednesday :(

Looks like a great solution, but you touched a core part of OpenVINO, I'm not sure it is necessary.

I think you've overcomplicated the solution. Let me check. I thought about something like this, it should be enough:
https://github.com/openvinotoolkit/openvino/blob/master/src/frontends/onnx/frontend/src/op/softmax.cpp

Currently, biggest problem which requires are alignment/refactoring is a "some common implementation" of a translation function. What does it mean? It means it looks like as an any of documented opset (for example, 7,10,11,19), but doesn't implement exact opset functionality (allows to use an undocumented input types, use wrong inputs, etc).

That's why we want to align behavior with the original documentation. Because otherwise, when you pass a model with a 19 opset it fails due to a common implementation uses wrong input (input instead of an attribute, as example), but you will get any possible error and will spend a lot of time to understand what goes wrong. I'm sure previous approach was very good for a fast implementation. But now it cause a misunderstanding while debugging and enabling new opsets.

[mitruska]

Hello, I've left review comments within the mentioned PR #22075, but let me put the summary here as well.

Most of the currently failing ONNX AvgPool compliance tests are related to the "dilations" attribute, and in fact to cover it, new version of the ov AvgPool is needed (as it was done for ov MaxPool op).
But to enable new ov Op not only new class of AvgPool is needed, but also the plugin implementation.

@gkrivor We should cover such request internally or by a separate, precisely described task for contributors.

[awayzjj]

.take

[github-actions]

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

[p-wysocki]

Hello @awayzjj, are you still working on this? Do you need any help?

[awayzjj]

Hello @awayzjj, are you still working on this? Do you need any help?

Yes, sorry for the late, I will create a PR this weekend.

[awayzjj]

@gkrivor @mitruska Hi I aligned the behavior of AveragePool-7, 10, and 11, and created a PR.

The AveragePool-19 added the dilations attribute as referenced in the ONNX documentation, but the average_pool operator in OpenVINO does not support it. I want to reference how the max-pool operator handles the dilations attribute:

Here is how the max-pool operator handles the dilations attribute in its evaluate method:

openvino/src/core/src/op/max_pool.cpp

Lines 419 to 429 in 999e20a

	bool MaxPool::evaluate(TensorVector& outputs, const TensorVector& inputs) const {
	OV_OP_SCOPE(v14_MaxPool_evaluate);
	const auto input_shapes = std::vector<PartialShape>{inputs[0].get_shape()};
	auto pads_begin = m_pads_begin;
	auto pads_end = m_pads_end;
	const auto output_shape = shape_infer(this, input_shapes, pads_begin, pads_end).front();
	outputs[0].set_shape(output_shape.get_shape());
	return ov::op::maxpool::evaluate_util(this, outputs, inputs, get_dilations(), get_axis());
	}

Its evaluate method ultimately calls reference::max_pool:

openvino/src/core/src/op/max_pool.cpp

Lines 189 to 196 in 999e20a

	reference::max_pool(in.data<const T>(),
	out.data<T>(),
	in_shape,
	out_shape,
	kernel,
	strides,
	pads_begin,
	pads_end);

However, I found that the AvgPool operator does not override the evaluate method:

openvino/src/core/include/openvino/op/avg_pool.hpp

Lines 51 to 91 in 999e20a

	namespace v14 {
	/// \brief Batched average pooling operation.
	/// \ingroup ov_ops_cpp_api
	class OPENVINO_API AvgPool : public util::AvgPoolBase {
	public:
	OPENVINO_OP("AvgPool", "opset14", util::AvgPoolBase);
	/// \brief Constructs a batched average pooling operation.
	AvgPool() = default;
	/// \brief Constructs a batched average pooling operation.
	///
	/// \param arg The output producing the input data batch tensor.<br>
	/// `[d1, dn]`
	/// \param strides The strides.<br> `[n]`
	/// \param pads_begin The beginning of padding shape.<br> `[n]`
	/// \param pads_end The end of padding shape.<br> `[n]`
	/// \param kernel The kernel shape.<br> `[n]`
	/// \param exclude_pad If false then averages include padding elements, each
	/// treated as the number zero. If true, padding
	/// elements
	/// are entirely ignored when computing averages.
	/// \param rounding_type Whether to use ceiling or floor rounding type while
	/// computing output shape.
	/// \param auto_pad Padding type to use for additional padded dimensions
	AvgPool(const Output<Node>& arg,
	const Strides& strides,
	const Shape& pads_begin,
	const Shape& pads_end,
	const Shape& kernel,
	bool exclude_pad,
	RoundingType rounding_type = RoundingType::FLOOR,
	const PadType& auto_pad = PadType::EXPLICIT);
	void validate_and_infer_types() override;
	std::shared_ptr<Node> clone_with_new_inputs(const OutputVector& new_args) const override;
	};
	} // namespace v14
	} // namespace op
	} // namespace ov

Could you provide some guidance on where and how I should add the dilations attribute to the op?

[mlukasze]

@p-wysocki fyi