Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CPU]: Added ROIAlignRotated basic impl. #23844

Merged
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
4a5775b
[Op]: Added definition of ROIAlignedRotated.
pkowalc1 Mar 14, 2024
e0c13d1
[Op]:[ROIAlignRotated]: Fixed tests code style.
pkowalc1 Mar 14, 2024
e713a7b
[Op]:[ROIAlignRotated]: Fixerd codestyle.
pkowalc1 Mar 14, 2024
fb113e6
[Op][ROIAlignRotated]: Removed form opset14_tbl, to disable failing c…
pkowalc1 Mar 14, 2024
b09db55
Merge branch 'master' into roi_align_rotated_operator_def
pkowalc1 Mar 15, 2024
f5e31b9
Merge branch 'master' into roi_align_rotated_operator_def
pkowalc1 Mar 15, 2024
77a4223
[Op]:[ROIAlign]: Refactoring, introduced ROIAlignBase.
pkowalc1 Mar 15, 2024
5943fa8
Update src/core/shape_inference/include/roi_align_shape_utils.hpp
pkowalc1 Mar 15, 2024
eb7720d
Update src/core/shape_inference/include/roi_align_shape_inference.hpp
pkowalc1 Mar 15, 2024
ce5dcc6
Update src/core/tests/type_prop/roi_align.cpp
pkowalc1 Mar 15, 2024
014f931
Update src/core/src/op/roi_align_rotated.cpp
pkowalc1 Mar 15, 2024
d3a3c57
[Op][ROIAlignRotated]: Review comments.
pkowalc1 Mar 15, 2024
7b56861
[Op]:[ROIAlignRotated]: code sytle fixes.
pkowalc1 Mar 15, 2024
0756ecf
[Op]:{ROIAlign]: Fixed code style.
pkowalc1 Mar 15, 2024
9008667
[Op]:[ROIAlignRotated]: Fixed code style.
pkowalc1 Mar 15, 2024
ac066f7
[Op]:[ROIAlign]: Fixed code style.
pkowalc1 Mar 15, 2024
e0e78a3
[Op]:[ROIAlign]: Fixed type bug.
pkowalc1 Mar 15, 2024
190cc6b
[Op]:[ROIAlignBase]: Fixed naming convention.
pkowalc1 Mar 15, 2024
af1b95e
[Op][ROIAlignRotated]: Temporary removed new op from tests, since now…
pkowalc1 Mar 15, 2024
ee705be
Merge branch 'master' into roi_align_rotated_operator_def
pkowalc1 Mar 15, 2024
38206ee
[Op]:[ROIAlign]: Review fixes.
pkowalc1 Mar 15, 2024
5e369cc
Merge branch 'roi_align_rotated_operator_def' of https://github.com/p…
pkowalc1 Mar 15, 2024
074a209
[Op][ROIAlign]: Fixed code style.
pkowalc1 Mar 18, 2024
995c542
[Op]:[ROIAlign]: Fixed crash in template plugin tests
pkowalc1 Mar 18, 2024
2b4aec4
Merge branch 'master' into roi_align_rotated_operator_def
pkowalc1 Mar 18, 2024
c2a1228
Merge branch 'master' into roi_align_rotated_operator_def
pkowalc1 Mar 19, 2024
4b831c5
Merge branch 'master' into roi_align_rotated_operator_def
pkowalc1 Mar 19, 2024
4ecbfa7
[Op][ROIAlignRotated]: Changed size of second dim of roi input due to…
pkowalc1 Mar 20, 2024
242d91f
Merge branch 'roi_align_rotated_operator_def' of https://github.com/p…
pkowalc1 Mar 20, 2024
fbe71f8
Merge branch 'master' into roi_align_rotated_operator_def
pkowalc1 Mar 20, 2024
dc2600a
Merge branch 'master' into roi_align_rotated_operator_def
pkowalc1 Mar 20, 2024
456b71b
[Op][ROIAlignRotated]: Adding Implementation. WIP.
pkowalc1 Mar 21, 2024
bd0bb00
[Op][ROIAlignRotated]: WIP, fixed a bug and added more test cases
pkowalc1 Mar 22, 2024
1483b8d
[Op][ROIAlignRotated]: Added more tests and refactoring
pkowalc1 Mar 25, 2024
75186a3
[Op][ROIAlignRotated]: Tests refactoring, changed test cases names.
pkowalc1 Mar 25, 2024
8a19209
[Op][ROIAlign]: Refactoring of reference impl.
pkowalc1 Mar 25, 2024
ec02c6c
[Op][ROIAlign]: Fixed code style.
pkowalc1 Mar 25, 2024
7e78b9c
[frontend]: Added mmdepoy.ROIAlignRotated with test.
pkowalc1 Mar 27, 2024
3f067ae
[ROIAlignRotated]: Added ROIAlignRotated simple impl.
pkowalc1 Apr 3, 2024
6f30a18
Merge branch 'master' into roi_align_rotated_intel_cpu_plugin_impl
pkowalc1 Apr 29, 2024
3f44778
[onnx_test]: Added missing space.
pkowalc1 Apr 30, 2024
7c07606
Merge branch 'master' into roi_align_rotated_intel_cpu_plugin_impl
pkowalc1 May 8, 2024
047237b
[frontend][roi_align_rotated]: Fixed code afer bad reslution of merge…
pkowalc1 May 8, 2024
7d88f29
[frontend]: Fixing code after bad resolution of merge conflict.
pkowalc1 May 8, 2024
5e8aa9a
[cpu_plugin][cmake]: Added comment.
pkowalc1 May 9, 2024
7fa0b88
[test]: Fixed opset tets.
pkowalc1 May 9, 2024
cfd0104
[conformance_test]: Added generate function for ROIAlignRotated.
pkowalc1 May 9, 2024
0f1d0bc
Merge branch 'master' into roi_align_rotated_intel_cpu_plugin_impl
pkowalc1 May 9, 2024
848af5a
[cpu_plugin][ROIAlignRotated]: Fxies due to review.
pkowalc1 May 9, 2024
947862c
Merge branch 'master' into roi_align_rotated_intel_cpu_plugin_impl
pkowalc1 May 10, 2024
a7de6e2
[cpu_plugin]: Added support for ROIAlignRotated by adding evaluate me…
pkowalc1 May 13, 2024
44d34fa
Merge branch 'roi_align_rotated_intel_cpu_plugin_impl' of https://git…
pkowalc1 May 13, 2024
fe8159b
[cpu_plugin]: CMakelists.txt: Removed not needed new line.
pkowalc1 May 13, 2024
aca6338
Merge branch 'master' into roi_align_rotated_intel_cpu_plugin_impl
pkowalc1 May 14, 2024
42a534b
Review fix: src/core/src/op/roi_align_rotated.cpp
pkowalc1 May 14, 2024
f6dd302
Review fix: Update src/core/src/op/roi_align_rotated.cpp
pkowalc1 May 14, 2024
5624e0a
Merge branch 'master' into roi_align_rotated_intel_cpu_plugin_impl
pkowalc1 May 14, 2024
328dabf
[opset]: Fixed error after merge conflict resolution.
pkowalc1 May 14, 2024
6ad51f3
Merge branch 'roi_align_rotated_intel_cpu_plugin_impl' of https://git…
pkowalc1 May 14, 2024
33960ed
Revert "[cpu_plugin]: Added support for ROIAlignRotated by adding eva…
pkowalc1 May 14, 2024
a9a6316
Merge branch 'master' into roi_align_rotated_intel_cpu_plugin_impl
pkowalc1 May 14, 2024
c68e148
Merge branch 'master' into roi_align_rotated_intel_cpu_plugin_impl
pkowalc1 May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/core/include/openvino/opsets/opset14_tbl.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -221,3 +221,4 @@ _OPENVINO_OP_REG(FakeConvert, ov::op::v13)
// New operations added in opset14
_OPENVINO_OP_REG(ConvertPromoteTypes, ov::op::v14)
_OPENVINO_OP_REG(Inverse, ov::op::v14)
_OPENVINO_OP_REG(ROIAlignRotated, ov::op::v14)
5 changes: 3 additions & 2 deletions src/frontends/onnx/tests/onnx_import.in.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -6617,8 +6617,9 @@ OPENVINO_TEST(${BACKEND_NAME}, onnx_model_mish_activation) {
}

OPENVINO_TEST(${BACKEND_NAME}, onnx_model_mmdeploy_roi_align_rotated) {
if (std::string("${BACKEND_NAME}") != std::string("INTERPRETER")) {
return; //< Skip this test for non-INTERPRETER backends, since op is not yet implemented on those backends.
if (!(std::string("${BACKEND_NAME}") == std::string("INTERPRETER") ||
std::string("${BACKEND_NAME}") == std::string("IE_CPU"))) {
return; //< Skip this test for not-yet-implemented on those backends.
}

auto model = convert_model("mmdeploy_roi_align_rotated.onnx");
Expand Down
6 changes: 6 additions & 0 deletions src/plugins/intel_cpu/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,11 @@ if (ENABLE_SNIPPETS_LIBXSMM_TPP)
target_include_directories(${TARGET_NAME} SYSTEM PRIVATE $<TARGET_PROPERTY:xsmm,INCLUDE_DIRECTORIES>)
endif ()
target_include_directories(${TARGET_NAME} SYSTEM PRIVATE $<TARGET_PROPERTY:dnnl,INCLUDE_DIRECTORIES>)

# Temporal solution to use template reference implementations in cases where optimizied implementation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a ticket number

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# is not (yet) needed.
target_include_directories(${TARGET_NAME} PRIVATE $<TARGET_PROPERTY:openvino::reference,INTERFACE_INCLUDE_DIRECTORIES>)

# Cross compiled function
# TODO: The same for proposal, proposalONNX, topk
cross_compiled_file(${TARGET_NAME}
Expand Down Expand Up @@ -249,6 +254,7 @@ if(BUILD_SHARED_LIBS)
$<TARGET_PROPERTY:openvino::itt,INTERFACE_INCLUDE_DIRECTORIES>
$<TARGET_PROPERTY:openvino::shape_inference,INTERFACE_INCLUDE_DIRECTORIES>
$<TARGET_PROPERTY:openvino::snippets,INTERFACE_INCLUDE_DIRECTORIES>
$<TARGET_PROPERTY:openvino::reference,INTERFACE_INCLUDE_DIRECTORIES>
PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}/src
$<TARGET_PROPERTY:openvino::conditional_compilation,INTERFACE_INCLUDE_DIRECTORIES>)
Expand Down
2 changes: 2 additions & 0 deletions src/plugins/intel_cpu/src/cpu_types.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ static const TypeToNameMap& get_type_to_name_tbl() {
{"Slice", Type::StridedSlice},
{"Tile", Type::Tile},
{"ROIAlign", Type::ROIAlign},
{"ROIAlignRotated", Type::ROIAlignRotated},
{"ROIPooling", Type::ROIPooling},
{"PSROIPooling", Type::PSROIPooling},
{"DeformablePSROIPooling", Type::PSROIPooling},
Expand Down Expand Up @@ -280,6 +281,7 @@ std::string NameFromType(const Type type) {
CASE(NonZero);
CASE(Tile);
CASE(ROIAlign);
CASE(ROIAlignRotated);
CASE(ROIPooling);
CASE(PSROIPooling);
CASE(DepthToSpace);
Expand Down
1 change: 1 addition & 0 deletions src/plugins/intel_cpu/src/cpu_types.h
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ enum class Type {
NonZero,
Tile,
ROIAlign,
ROIAlignRotated,
ROIPooling,
PSROIPooling,
BatchToSpace,
Expand Down
110 changes: 110 additions & 0 deletions src/plugins/intel_cpu/src/nodes/roi_align_rotated.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
// Copyright (C) 2018-2024 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//

#include "roi_align_rotated.h"

#include <openvino/opsets/opset14.hpp>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include just required operation headers.


#include "common/cpu_convert.h"
#include "openvino/reference/roi_align.hpp"

namespace ov {
namespace intel_cpu {
namespace node {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add fn isSupportedOperation(), like in other CPU nodes

ROIAlignRotated::ROIAlignRotated(const std::shared_ptr<ov::Node>& op, const GraphContext::CPtr context)
: Node(op, context, NgraphShapeInferFactory(op, EMPTY_PORT_MASK)) {
std::string errorMessage;
praasz marked this conversation as resolved.
Show resolved Hide resolved
const auto roiAlign = ov::as_type_ptr<const ov::opset14::ROIAlignRotated>(op);
pooledH = roiAlign->get_pooled_h();
pooledW = roiAlign->get_pooled_w();
spatialScale = roiAlign->get_spatial_scale();
samplingRatio = roiAlign->get_sampling_ratio();
clockwiseMode = roiAlign->get_clockwise_mode();
}

void ROIAlignRotated::getSupportedDescriptors() {
// Validation is already done in the ov::opset14::ROIAlignRotated.
}

void ROIAlignRotated::initSupportedPrimitiveDescriptors() {
if (!supportedPrimitiveDescriptors.empty())
return;

ov::element::Type inputPrec0 = getOriginalInputPrecisionAtPort(0);
ov::element::Type outputPrec = getOriginalOutputPrecisionAtPort(0);

addSupportedPrimDesc(
{{LayoutType::ncsp, inputPrec0}, {LayoutType::ncsp, ov::element::f32}, {LayoutType::ncsp, ov::element::i32}},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per specification, 1st and 2nd inputs have the same precisions. Why the second one is always f32?
Also need to check this value. Plugin supports only f32, f16 and bf16 floating point types.

{{LayoutType::ncsp, outputPrec}},
impl_desc_type::ref);
}

bool ROIAlignRotated::created() const {
return getType() == Type::ROIAlignRotated;
}

bool ROIAlignRotated::needPrepareParams() const {
return false;
}

void ROIAlignRotated::executeDynamicImpl(dnnl::stream strm) {
execute(strm);
}

template <ov::element::Type_t OV_TYPE>
void ROIAlignRotated::executeImpl() {
using T = typename ov::element_type_traits<OV_TYPE>::value_type;

const size_t batch_indices_size = getSrcMemoryAtPort(2)->getShape().getElementsCount();

std::vector<int64_t> batch_indices_vec_scaled_up(batch_indices_size);
cpu_convert(getSrcMemoryAtPort(2)->getData(),
batch_indices_vec_scaled_up.data(),
getSrcMemoryAtPort(2)->getPrecision(),
ov::element::i64,
batch_indices_size);

ov::reference::roi_align<T, ov::reference::roi_policy::ROIAlignRotatedOpDefPolicy>(
getSrcDataAtPortAs<const T>(0),
getSrcDataAtPortAs<const T>(1),
batch_indices_vec_scaled_up.data(),
getDstDataAtPortAs<T>(0),
getSrcMemoryAtPort(0)->getShape().toPartialShape().to_shape(), //< Conversion from intel_cpu::Shape to
praasz marked this conversation as resolved.
Show resolved Hide resolved
// ov::Shape....
getSrcMemoryAtPort(1)->getShape().toPartialShape().to_shape(),
getSrcMemoryAtPort(2)->getShape().toPartialShape().to_shape(),
getDstMemoryAtPort(0)->getShape().toPartialShape().to_shape(),
pooledH,
pooledW,
samplingRatio,
spatialScale,
ov::op::v3::ROIAlign::PoolingMode::AVG,
ov::op::v9::ROIAlign::AlignedMode::ASYMMETRIC,
clockwiseMode);
}

void ROIAlignRotated::execute(dnnl::stream) {
const ov::element::Type type = getOriginalInputPrecisionAtPort(0);
executeImpl<ov::element::f32>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this fn is called twice?


#define CASE(OV_TYPE) \
case ov::element::OV_TYPE: \
executeImpl<ov::element::OV_TYPE>(); \
break;

switch (type) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use OV_SWITCH instead

CASE(bf16);
CASE(f16);
CASE(f32);
CASE(f64);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not supported by plugin

default:
OPENVINO_THROW("[ROIAlignRotated]: Unhandled data type ", type, " in execute()");
}
#undef CASE
}

} // namespace node
} // namespace intel_cpu
} // namespace ov
37 changes: 37 additions & 0 deletions src/plugins/intel_cpu/src/nodes/roi_align_rotated.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// Copyright (C) 2018-2024 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//

#pragma once

#include "node.h"

namespace ov {
namespace intel_cpu {
namespace node {

class ROIAlignRotated : public Node {
public:
ROIAlignRotated(const std::shared_ptr<ov::Node>& op, const GraphContext::CPtr context);

void getSupportedDescriptors() override;
void initSupportedPrimitiveDescriptors() override;
void execute(dnnl::stream strm) override;
bool created() const override;
bool needPrepareParams() const override;
void executeDynamicImpl(dnnl::stream strm) override;

private:
template <ov::element::Type_t OV_TYPE>
void executeImpl();

int pooledH;
int pooledW;
int samplingRatio;
float spatialScale;
bool clockwiseMode;
};

} // namespace node
} // namespace intel_cpu
} // namespace ov
2 changes: 2 additions & 0 deletions src/plugins/intel_cpu/src/nodes_factory.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@
#include "nodes/reverse_sequence.h"
#include "nodes/rnn.h"
#include "nodes/roi_align.h"
#include "nodes/roi_align_rotated.h"
#include "nodes/roi_pooling.h"
#include "nodes/roll.h"
#include "nodes/rope.h"
Expand Down Expand Up @@ -193,6 +194,7 @@ Node::NodesFactory::NodesFactory() : Factory("NodesFactory") {
INTEL_CPU_NODE(NonMaxSuppression, Type::NonMaxSuppression);
INTEL_CPU_NODE(ROIPooling, Type::ROIPooling);
INTEL_CPU_NODE(ROIAlign, Type::ROIAlign);
INTEL_CPU_NODE(ROIAlignRotated, Type::ROIAlignRotated);
INTEL_CPU_NODE(TopK, Type::TopK);
INTEL_CPU_NODE(Proposal, Type::Proposal);
INTEL_CPU_NODE(RegionYolo, Type::RegionYolo);
Expand Down
Loading