[IE TESTS] [CPU] extended cpu specific tests to support int8 precision #6546

antonvor · 2021-07-06T20:56:48Z

This PR contains int8 tests for nodes:

Convolution
GroupConvolution
ConvolutionBackpropData
GroupConvolutionBackpropData

TODO

fill Convolution and GroupConvolution tests
fill ConvolutionBackpropData and GroupConvolutionBackpropData tests
split tests into files depending on the precision (fp32, bf16, int8)
add dw conv fusing tests for [CPU] fixed conv + dw conv fusing #6333
add tests for issue 63021
add tests for issue 62549

antonvor · 2021-07-21T07:08:37Z

@dmitry-gorokhov could you please review?

1. extended cpu specific tests to support int8 presision 2. added int8 deconvolution fusing tests

antonvor · 2021-08-24T12:43:36Z

@maxnick could you review please?

maxnick · 2021-08-25T12:53:24Z

inference-engine/tests/functional/plugin/cpu/test_utils/cpu_test_utils.hpp

+
+    ngraph::element::Type outElemType = ngraph::element::f32;
+    // only for int8 testing
+    int quantizeInHigh = 1;


The name is not very descriptive. Can we change the name so that the meaning become more obvious?

maxnick · 2021-08-25T13:07:07Z

inference-engine/tests/functional/plugin/cpu/test_utils/cpu_test_utils.cpp

+   if (outElemType == ngraph::element::u8) {
+       newLastNode = ngraph::builder::makeFakeQuantize(newLastNode, ngPrc, 256, {1, 1, 1, 1}, {0}, {static_cast<float>(quantizeInHigh)}, {0}, {255});
+   } else if (outElemType == ngraph::element::i8) {
+       newLastNode = ngraph::builder::makeFakeQuantize(newLastNode, ngPrc, 255, {1, 1, 1, 1}, {0}, {static_cast<float>(quantizeInHigh)}, {-127}, {127});
+   }
+


It looks like CPUTestsBase::modifyGraph is better place for this code. Do we have to place the FQ node after the simple operations chain that we add in fusing tests?

maxnick · 2021-08-25T13:24:50Z

inference-engine/tests/functional/plugin/cpu/single_layer_tests/convolution.cpp

+        if (outPrc == Precision::U8 || outPrc == Precision::I8) {
+            threshold = 1.001f;
+            outElemType = FuncTestUtils::PrecisionUtils::convertIE2nGraphPrc(outPrc);
+            quantizeInHigh = calculateQuantizeInHigh(kernel, inputShape[1]);
+        }


looks like a code block that could be extracted to the base class method since it is repeated in every test SetUp() function that tests a quantized layer. Since we do not have a time to properly refactor CPU single layer tests, we can solve it using some functional stuff. For example we can define a functor pointer in the CPUTestBase that is called in this extracted code, but is initialized in implementations. If it is not initialized - the method throws. So that we define at least some int8 related stages: changing threshold, outElemType, quantizeInHigh calculation in one common place. Unfortunately this method itself should be called from the SetUp function but it insolate some amount of common code.

maxnick · 2021-08-25T13:30:48Z

inference-engine/tests/functional/plugin/cpu/single_layer_tests/convolution.cpp

+        if (inPrc == Precision::U8 || inPrc == Precision::I8) {
+            additionalPasses.push_back(std::make_shared<pass::ConvertPrecision<element::i8, element::f32>>());
+            additionalPasses.push_back(std::make_shared<pass::ConvertPrecision<element::u8, element::f32>>());
+        }


This is also a good candidate for method extraction. If we had a time for refactoring we would inherit CPUTestsBase from LayerTestCommon then using algorithm pattern we could define some mandatory setUp stages and extract such repeated code blocks to the base class methods.

maxnick · 2021-08-25T13:34:00Z

inference-engine/tests/functional/plugin/cpu/single_layer_tests/convolution_backprop_data.cpp

        InferenceEngine::SizeVector kernel, stride, dilation;
        std::vector<ptrdiff_t> padBegin, padEnd, outPadding;
        size_t convOutChannels;
        std::tie(kernel, stride, padBegin, padEnd, dilation, convOutChannels, padType, outPadding) = convParams;
-        auto ngPrc = FuncTestUtils::PrecisionUtils::convertIE2nGraphPrc(netPrecision);
+        auto inElementType = FuncTestUtils::PrecisionUtils::convertIE2nGraphPrc(inPrc);
+        auto outElementType = FuncTestUtils::PrecisionUtils::convertIE2nGraphPrc(outPrc);


It seems that this variable is not used.

maxnick · 2021-08-25T14:01:17Z

inference-engine/tests/functional/plugin/cpu/single_layer_tests/group_convolution.cpp

+                                         ::testing::Values(Precision::FP32, Precision::U8, Precision::I8),
+                                         ::testing::Values(InferenceEngine::Layout::ANY),
+                                         ::testing::Values(InferenceEngine::Layout::ANY),
+                                         ::testing::Values(std::vector<size_t >({2, 64, 7, 7})),


Here we testing only channels shape evenly divided by the SIMD vector size, that could hide some problems related to tails processing. Suggest adding more shape variations where input and output channels number is a fraction of blocks number. Please check other new tests instances including other layers.

maxnick · 2021-08-25T14:25:43Z

inference-engine/tests/functional/plugin/cpu/single_layer_tests/convolution.cpp

+                        ::testing::Values(Precision::FP32, Precision::U8, Precision::I8),
+                        ::testing::Values(Layout::ANY),
+                        ::testing::Values(Layout::ANY),
+                        ::testing::Values(std::vector<size_t >({ 2, 12, 7, 7, 7 })),


12 channels does not form a full vector on avx512 VNNI, m.b. it makes sense to have a shape that contains at least one full SIMD vector and some fractional part to cover more possible code paths? Here in in the other new test instances.

maxnick · 2021-08-25T14:30:41Z

...ce-engine/tests/functional/plugin/cpu/single_layer_tests/group_convolution_backprop_data.cpp

        InferenceEngine::SizeVector kernel, stride, dilation;
        std::vector<ptrdiff_t> padBegin, padEnd, outputPadding;
        size_t convOutChannels, numGroups;
        std::tie(kernel, stride, padBegin, padEnd, dilation, convOutChannels, numGroups, padType, outputPadding) = groupConvParams;
+        auto inElementType = FuncTestUtils::PrecisionUtils::convertIE2nGraphPrc(inPrc);
+        auto outElementType = FuncTestUtils::PrecisionUtils::convertIE2nGraphPrc(outPrc);


It seems that the variable is not used.

ilya-lavrenov · 2022-10-26T22:20:40Z

@antonvor @dmitry-gorokhov is it still valid?

akladiev · 2023-05-11T16:21:27Z

This PR will be closed in 2 weeks in case of no activity.

akladiev · 2023-05-23T14:42:17Z

This PR was closed because it has been stalled for 2 week with no activity.

antonvor self-assigned this Jul 6, 2021

antonvor requested review from a team July 6, 2021 20:56

antonvor mentioned this pull request Jul 6, 2021

WIP: [IE TESTS] [CPU] extended cpu specific tests to support int8 precision #5348

Closed

4 tasks

openvino-pushbot added category: IE Tests OpenVINO Test: plugins and common category: Core OpenVINO Core (aka ngraph) labels Jul 6, 2021

antonvor force-pushed the feature/cpu_tests_improvements branch 3 times, most recently from 2c95845 to a3cffa1 Compare July 19, 2021 12:00

antonvor requested review from a team July 20, 2021 13:54

antonvor force-pushed the feature/cpu_tests_improvements branch 2 times, most recently from d85f86f to 22fa239 Compare July 21, 2021 07:07

antonvor changed the title ~~WIP: [IE TESTS] [CPU] extended cpu specific tests to support int8 precision~~ [IE TESTS] [CPU] extended cpu specific tests to support int8 precision Jul 21, 2021

antonvor assigned dmitry-gorokhov Jul 21, 2021

antonvor force-pushed the feature/cpu_tests_improvements branch 2 times, most recently from aafeae7 to 1cedb93 Compare July 22, 2021 13:33

antonvor mentioned this pull request Jul 27, 2021

MVN test extensions #6291

Closed

antonvor force-pushed the feature/cpu_tests_improvements branch 4 times, most recently from 64d613f to ae2d28e Compare August 3, 2021 08:08

antonvor force-pushed the feature/cpu_tests_improvements branch from ae2d28e to 549258b Compare August 4, 2021 03:48

[CPU] [TESTS] cpu tests improvements

8741933

1. extended cpu specific tests to support int8 presision 2. added int8 deconvolution fusing tests

antonvor force-pushed the feature/cpu_tests_improvements branch from 549258b to 8741933 Compare August 24, 2021 12:21

some fixes

65d5691

antonvor assigned maxnick and unassigned dmitry-gorokhov Aug 24, 2021

antonvor requested a review from maxnick August 24, 2021 12:43

maxnick reviewed Aug 25, 2021

View reviewed changes

antonvor mentioned this pull request Oct 19, 2021

[CPU] [TESTS] cpu tests improvements #8082

Closed

7 tasks

akladiev added the Stale label May 11, 2023

akladiev closed this May 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[IE TESTS] [CPU] extended cpu specific tests to support int8 precision #6546

[IE TESTS] [CPU] extended cpu specific tests to support int8 precision #6546

antonvor commented Jul 6, 2021 •

edited

Loading

antonvor commented Jul 21, 2021

antonvor commented Aug 24, 2021

maxnick Aug 25, 2021

maxnick Aug 25, 2021

maxnick Aug 25, 2021

maxnick Aug 25, 2021

maxnick Aug 25, 2021

maxnick Aug 25, 2021

maxnick Aug 25, 2021

maxnick Aug 25, 2021

ilya-lavrenov commented Oct 26, 2022

akladiev commented May 11, 2023

akladiev commented May 23, 2023

[IE TESTS] [CPU] extended cpu specific tests to support int8 precision #6546

[IE TESTS] [CPU] extended cpu specific tests to support int8 precision #6546

Conversation

antonvor commented Jul 6, 2021 • edited Loading

antonvor commented Jul 21, 2021

antonvor commented Aug 24, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ilya-lavrenov commented Oct 26, 2022

akladiev commented May 11, 2023

akladiev commented May 23, 2023

antonvor commented Jul 6, 2021 •

edited

Loading