Strided slice mo (#12)

* fix ss * successfully converted * successfully run moved infer and normalizer unit-tests * successfully rewritten StridedSlice infer unittests * int64 array * Successfully converter crash-when-loading, xj_feauture and toy nets (cherry-picked maxpoolV4 and tf_broadcast_ext) * successfully moved PermuteAttrs to general mechanism * successfully converted xj_feauture and crash when loading with the new rewritten SS infer * fixed get_shape_from_slice and moved to common utils * fixed extending masks and some other * some refactoring * fixed extending masks in extractor, fixed licence year and some other code clearing * corrected a couple of unittests * fox permute for 5 rank slice and 4 rank inputs/ * WIP * Added comments * fixed StridedSlice in ProposalMutation.py * rechecked shape_infer unittests added some new cases * added shape_infer unit-tests after StridedSliceNormalizer pass and Permute unit-tests * corrected unittests * Applied review comments * general permutations for inputs implemented, corrected ellipsis unrolling when shrink_axis is at the beginning, some other corrections * removed code duplication in infer and normalizer, moved 'slices' attr normalizing to StridedSliceNormalizer.py * removed some code duplication and other minor improvements * Added tests * minor corrections * [GNA] Support 1d tensors (openvinotoolkit#4270) * [OpenVINO Scripts] Updated with RHEL8 (openvinotoolkit#4296) * Updated install_NEO_OCL_driver.sh & install_openvino_dependencies.sh with rhel8 case * install_NEO_OCL_driver: Fixed conditional * script: Updated with actual revisions * [CPU] MKLDNN NCHW pooling primitive performance fix. (openvinotoolkit#4235) * [IE][VPU]: Check memory capacity after finding the corresponding device (openvinotoolkit#4314) This issue relates to multi-device mode. While we are trying to allocate a graph on one of the devices we should check memory capacity only for the corresponding device, not for the last opened as far devices may have different memory capacity or the latest opened device may have still not sent its attributes (including memory capacity) * Azure CI: Enable IB initiators as helpers * Result rename operation (openvinotoolkit#4242) * Added result rename operation * Optimize imports * Added ResultRename to package_BOM * ResultRename moved to the end of back phase, code refactoring * Revert incorrect changes * Optimize imports * Added comments and optimized imports. * DPC++ link error workaround. (openvinotoolkit#4192) * DPC++ link error workaround. OpenVINO C++ program failed to link when DPC++ compiler is used. 'make_shared_blob' causes 'unresolved external symbol' error on linking. Commented out some __clang__ specific directives to workaround the issue in "ie_blob.h". * DPC++ compatibility issue fix #2 1. Removed type-by-type template class definition for __clang__. 2. Modified TBlob() destructor. The 'unresolved reference' error occur again if I left 'virtual ~TBlob();' only. It seems it needs to be 'virtual ~TBlob() {};'. * DPC++ compatibility fix #3 - Add DPC++ conditional code Uses '__SYCL_COMPILER_VERSION' predefined macro to check if the compiler is a DPC++ or not. Added conditional directive to switch code based of the detected compiler. NOTE: User program must include <CL/sycl.hpp>, or the '__SYCL_COMPILER_VERSION' macro won't be defined and this fix won't take effect. * DPC++ compatibility issue fix #4 Changed from #ifdef to #if + logical formulas. * DPC++ compatibility issue fix #5 Added compiler check logic in src/ie_rtti.cpp * DPC++ Compatibility issue #6 - ie_parameter.cpp Added compiler check macro for DPC++ to ie_parameter.cpp as well. Co-authored-by: Yasunori Shimura <[email protected]> * Azure CI: Disable IB stop * Remove generic ie op (openvinotoolkit#4213) * Removed legacy IE shape infer * Removed GenericIE operation * Removed legacy shape infer tests * Removed legacy test with legacy IE reshape * Fixed compilation issues related to removal of GenericIE * Fixed one more compilation issue with clDNN * Fixed test for reading experimental ops * Updated tests and make IR Reader to load old experimenal and extension ops as opset6 * Change opset of some ops only if they are currently experimental/extension to avoid situation like opset1::Proposal -> opset6::Proposal * Removed more legacy code * Returned back code removed by mistake * Fixed issues related to incorrect merge with master * Merge fixes * Fixed unit tests which starts to fail because now loading the model with unknown operation is failed earlier * Removed incorrectly added code Co-authored-by: Evgeny Lazarev <[email protected]> * [IE CLDNN] Added CTCGreedyDecoderSeqLen operation (openvinotoolkit#4119) * [CPU] Refactors jitters for nGraph interop (openvinotoolkit#4255) * [IE CLDNN] Fixed CTCGreedyDecoderSeqLenLayerTest gpu instances (openvinotoolkit#4326) * Fix comparison of constant with short float NAN values (openvinotoolkit#4299) * fix comparison of constant with short float NAN values * adjust precision, remove elvises * more templates * add ir serialization test with float16 const * remove unused prototxt * [ONNX] Remove linking libonnx from unit-test (openvinotoolkit#4298) * Remove linking libonnx from unit-test * Consider all flavors of protobuf libraries * Avoid passing on NOTFOUND properties * Set system deps for mac * Revert include dirs set up * wider range of unittests added (froze the number) * ONNX RNN/GRU enable dynamic input shape (openvinotoolkit#4241) * Add support for custom ONNX operator PriorBoxClustered (openvinotoolkit#4202) * review comments applied * Force test use bash for setupvars.sh (openvinotoolkit#4321) setupvars.sh is incomatible with sh shell. * enabled skipped unit-test * comment corrections Co-authored-by: Pavel Esir <[email protected]> Co-authored-by: Elizaveta Lobanova <[email protected]> Co-authored-by: Artyom Anokhov <[email protected]> Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Maksim Doronin <[email protected]> Co-authored-by: Alexander Zhogov <[email protected]> Co-authored-by: Anastasia Popova <[email protected]> Co-authored-by: Yasunori Shimura <[email protected]> Co-authored-by: Yasunori Shimura <[email protected]> Co-authored-by: Evgeny Lazarev <[email protected]> Co-authored-by: Evgeny Lazarev <[email protected]> Co-authored-by: Roman Lyamin <[email protected]> Co-authored-by: Marina Kolpakova <[email protected]> Co-authored-by: Vladimir Paramuzov <[email protected]> Co-authored-by: Bartosz Lesniewski <[email protected]> Co-authored-by: Tomasz Jankowski <[email protected]> Co-authored-by: Katarzyna Mitrus <[email protected]> Co-authored-by: Bartosz Sledz <[email protected]> Co-authored-by: Andrey Somsikov <[email protected]>
a-sidorova · Feb 15, 2021 · dbcf809 · dbcf809
1 parent 4e515e1
commit dbcf809
Show file tree

Hide file tree

Showing 125 changed files with 6,607 additions and 2,284 deletions.
diff --git a/.ci/azure/windows.yml b/.ci/azure/windows.yml
@@ -98,10 +98,6 @@ jobs:
     workingDirectory: $(BUILD_DIR)
     displayName: 'Build Win'
 
-  - script: echo Stop IncrediBuild_Agent && net stop IncrediBuild_Agent
-    displayName: Stop IncrediBuild
-    continueOnError: true
-
   - script: dir $(REPO_DIR)\bin\ /s
     displayName: 'List files'
 
@@ -204,4 +200,9 @@ jobs:
       #testRunTitle: 'Pre/Post-Commit' # Optional
       buildPlatform: 'x64' # Optional
       buildConfiguration: 'Windows' # Optional
-      #publishRunAttachments: true # Optional
+      #publishRunAttachments: true # Optional
+
+  - script: echo Stop IncrediBuild_Agent && net stop IncrediBuild_Agent
+    displayName: Stop IncrediBuild
+    continueOnError: true
+    enabled: false
diff --git a/docs/IE_PLUGIN_DG/Doxyfile b/docs/IE_PLUGIN_DG/Doxyfile
@@ -844,8 +844,7 @@ EXCLUDE_SYMLINKS       = NO
 # Note that the wildcards are matched against the file with absolute path, so to
 # exclude all test directories for example use the pattern */test/*
 
-EXCLUDE_PATTERNS       = generic_ie.hpp \
-                         function_name.hpp \
+EXCLUDE_PATTERNS       = function_name.hpp \
                          macro_overload.hpp
 
 # The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names

diff --git a/docs/doxygen/ie_plugin_api.config b/docs/doxygen/ie_plugin_api.config
@@ -38,8 +38,6 @@ FILE_PATTERNS          = *.c \
                          *.hpp \
                          *.md
 
-EXCLUDE_PATTERNS       = generic_ie.hpp
-
 EXCLUDE_SYMBOLS        = InferenceEngine::details
 
 TAGFILES               = "@DOCS_BUILD_DIR@/ie_api.tag=.." \

diff --git a/inference-engine/include/ie_blob.h b/inference-engine/include/ie_blob.h
@@ -577,13 +577,14 @@ class TBlob : public MemoryBlob {
     /**
      *@brief Virtual destructor.
      */
-#ifdef __clang__
+
+#if defined(__clang__) && !defined(__SYCL_COMPILER_VERSION)
     virtual ~TBlob();
 #else
     virtual ~TBlob() {
         free();
     }
-#endif  // __clang__
+#endif  // __clang__ && !__SYCL_COMPILER_VERSION
 
     /**
      * @brief Gets the size of the given type.
@@ -806,7 +807,7 @@ class TBlob : public MemoryBlob {
     }
 };
 
-#ifdef __clang__
+#if defined(__clang__) && !defined(__SYCL_COMPILER_VERSION)
 extern template class INFERENCE_ENGINE_API_CLASS(InferenceEngine::TBlob<float>);
 extern template class INFERENCE_ENGINE_API_CLASS(InferenceEngine::TBlob<double>);
 extern template class INFERENCE_ENGINE_API_CLASS(InferenceEngine::TBlob<int8_t>);
@@ -819,7 +820,7 @@ extern template class INFERENCE_ENGINE_API_CLASS(InferenceEngine::TBlob<long>);
 extern template class INFERENCE_ENGINE_API_CLASS(InferenceEngine::TBlob<long long>);
 extern template class INFERENCE_ENGINE_API_CLASS(InferenceEngine::TBlob<unsigned long>);
 extern template class INFERENCE_ENGINE_API_CLASS(InferenceEngine::TBlob<unsigned long long>);
-#endif  // __clang__
+#endif  // __clang__ && !__SYCL_COMPILER_VERSION
 
 /**
  * @brief Creates a blob with the given tensor descriptor.

diff --git a/inference-engine/include/ie_parameter.hpp b/inference-engine/include/ie_parameter.hpp
@@ -265,11 +265,11 @@ class INFERENCE_ENGINE_API_CLASS(Parameter) {
     struct HasOperatorEqual : CheckOperatorEqual<T, EqualTo>::type {};
 
     struct Any {
-#ifdef __clang__
+#if defined(__clang__) && !defined(__SYCL_COMPILER_VERSION)
         virtual ~Any();
 #else
         virtual ~Any() = default;
-#endif
+#endif  // __clang__ && !__SYCL_COMPILER_VERSION
         virtual bool is(const std::type_info&) const = 0;
         virtual Any* copy() const = 0;
         virtual bool operator==(const Any& rhs) const = 0;
@@ -326,7 +326,7 @@ class INFERENCE_ENGINE_API_CLASS(Parameter) {
     Any* ptr = nullptr;
 };
 
-#ifdef __clang__
+#if defined(__clang__) && !defined(__SYCL_COMPILER_VERSION)
 extern template struct INFERENCE_ENGINE_API_CLASS(InferenceEngine::Parameter::RealData<InferenceEngine::Blob::Ptr>);
 extern template struct INFERENCE_ENGINE_API_CLASS(InferenceEngine::Parameter::RealData<int>);
 extern template struct INFERENCE_ENGINE_API_CLASS(InferenceEngine::Parameter::RealData<bool>);
@@ -341,6 +341,6 @@ extern template struct INFERENCE_ENGINE_API_CLASS(
     InferenceEngine::Parameter::RealData<std::tuple<unsigned int, unsigned int>>);
 extern template struct INFERENCE_ENGINE_API_CLASS(
     InferenceEngine::Parameter::RealData<std::tuple<unsigned int, unsigned int, unsigned int>>);
-#endif  // __clang__
+#endif  // __clang__ && !__SYCL_COMPILER_VERSION
 
 }  // namespace InferenceEngine
diff --git a/inference-engine/src/cldnn_engine/cldnn_engine.cpp b/inference-engine/src/cldnn_engine/cldnn_engine.cpp
@@ -21,7 +21,6 @@
 #include <ngraph/opsets/opset5.hpp>
 #include <ngraph/pass/manager.hpp>
 #include <ngraph/pass/constant_folding.hpp>
-#include <generic_ie.hpp>
 #include <ie_ngraph_utils.hpp>
 
 #include <transformations/opset_conversions/convert_opset3_to_opset2.hpp>
@@ -137,8 +136,6 @@ InferenceEngine::CNNNetwork clDNNEngine::CloneAndTransformNetwork(const Inferenc
     if (clonedNetwork.getFunction()) {
         OV_ITT_SCOPED_TASK(itt::domains::CLDNNPlugin, "clDNNEngine::TransformNetwork");
         auto nGraphFunc = clonedNetwork.getFunction();
-        // Disable shape inference (WA for generic operations)
-        ngraph::op::GenericIE::DisableReshape noReshape(nGraphFunc);
 
         bool enableInt8;
         {

diff --git a/inference-engine/src/cldnn_engine/cldnn_graph.cpp b/inference-engine/src/cldnn_engine/cldnn_graph.cpp
@@ -25,8 +25,8 @@
 #include <sys/stat.h>
 #include <exec_graph_info.hpp>
 #include <ie_ngraph_utils.hpp>
-#include "generic_ie.hpp"
 #include <ngraph/variant.hpp>
+#include <ngraph/ngraph.hpp>
 #include "cldnn_itt.h"
 
 using namespace InferenceEngine;
@@ -464,7 +464,6 @@ InferenceEngine::CNNNetwork CLDNNGraph::GetExecGraphInfoByPrimitivesInfo(std::ve
         create_ngraph_node(pi);
     }
 
-    ngraph::op::GenericIE::DisableReshape reshape(nodes);
     auto function = std::make_shared<ngraph::Function>(results, params, "runtime_gpu_graph");
     InferenceEngine::CNNNetwork net(function);
     return net;

diff --git a/inference-engine/src/cldnn_engine/cldnn_primitives_list.hpp b/inference-engine/src/cldnn_engine/cldnn_primitives_list.hpp
@@ -201,5 +201,8 @@ REGISTER_FACTORY(v5, Round);
 // REGISTER_FACTORY(v5, Loop);
 // REGISTER_FACTORY(v5, RNNSequence);
 
+// ------------------------------ Supported v6 ops ------------------------------ //
+REGISTER_FACTORY(v6, CTCGreedyDecoderSeqLen);
+
 // --------------------------- Supported internal ops --------------------------- //
 REGISTER_FACTORY(internal, NonMaxSuppressionIEInternal);
diff --git a/inference-engine/src/cldnn_engine/ops/ctc_greedy_decoder.cpp b/inference-engine/src/cldnn_engine/ops/ctc_greedy_decoder.cpp
@@ -1,32 +1,122 @@
-// Copyright (C) 2020 Intel Corporation
+// Copyright (C) 2020-2021 Intel Corporation
 // SPDX-License-Identifier: Apache-2.0
 //
 
 #include "cldnn_program.h"
 #include "cldnn_common_utils.h"
 
 #include "ngraph/op/ctc_greedy_decoder.hpp"
+#include "ngraph/op/ctc_greedy_decoder_seq_len.hpp"
 
 #include "api/ctc_greedy_decoder.hpp"
+#include "api/reorder.hpp"
+#include "api/mutable_data.hpp"
+
+#include "transformations/utils/utils.hpp"
 
 namespace CLDNNPlugin {
 
-void CreateCTCGreedyDecoderOp(Program& p, const std::shared_ptr<ngraph::op::v0::CTCGreedyDecoder>& op) {
-    p.ValidateInputs(op, {2});
+void CreateCommonCTCGreedyDecoderOp(Program& p, const std::shared_ptr<ngraph::Node>& op, bool ctc_merge_repeated) {
+    p.ValidateInputs(op, {2, 3});
     auto inputPrimitives = p.GetInputPrimitiveIDs(op);
-    std::string layerName = layer_type_name_ID(op);
 
-    auto primitive = cldnn::ctc_greedy_decoder(layerName,
-                                               inputPrimitives[0],
-                                               inputPrimitives[1],
-                                               op->get_ctc_merge_repeated(),
-                                               DataTypeFromPrecision(op->get_output_element_type(0)),
-                                               CldnnTensorFromIEDims(op->get_output_shape(0)));
+    std::vector<cldnn::primitive_id> reorderedInputs;
+    reorderedInputs.resize(inputPrimitives.size());
+
+    for (size_t portIndex = 0; portIndex < inputPrimitives.size(); portIndex++) {
+        auto inputDataType = DataTypeFromPrecision(op->get_input_element_type(portIndex));
+        if (inputDataType == cldnn::data_types::i64) {
+            // clDNN primitive supports only i32 data type for 'sequence_length' and 'blank_index' inputs
+            // so we need additional reorder if it's provided as i64
+            auto reorderPrimName = inputPrimitives[portIndex] + "_" + op->get_friendly_name() + Program::m_preProcessTag;
+            auto targetFormat = DefaultFormatForDims(op->get_input_shape(portIndex).size());
+            auto preprocessPrim = cldnn::reorder(reorderPrimName,
+                                                 inputPrimitives[portIndex],
+                                                 targetFormat,
+                                                 cldnn::data_types::i32);
+            p.AddPrimitive(preprocessPrim);
+            p.AddInnerPrimitiveToProfiler(reorderPrimName, layer_type_name_ID(op), op);
+            reorderedInputs[portIndex] = (reorderPrimName);
+        } else {
+            reorderedInputs[portIndex] = inputPrimitives[portIndex];
+        }
+    }
+
+    uint32_t blank_index = op->get_input_shape(0).back() - 1;
+    if (reorderedInputs.size() == 3) {
+        auto blank_index_node = std::dynamic_pointer_cast<ngraph::op::v0::Constant>(op->get_input_node_shared_ptr(2));
+        if (!blank_index_node) {
+            THROW_IE_EXCEPTION << "Unsupported blank_index node type in " << op->get_friendly_name() << " (" << op->get_type_name() << ")";
+        }
+        float val;
+        if (ngraph::shape_size(blank_index_node->get_output_shape(0)) != 1 || !ngraph::op::util::get_single_value(blank_index_node, val)) {
+            THROW_IE_EXCEPTION << "Unsupported parameter size in " << op->get_friendly_name() << " (" << op->get_type_name() << ")";
+        }
+        blank_index = static_cast<uint32_t>(val);
+        reorderedInputs.pop_back();
+    }
+
+    std::size_t num_output = op->get_output_size();
+
+    std::vector<cldnn::memory> shared_memory;
+    if (num_output == 2) {
+        auto mutable_precision = op->get_output_element_type(1);
+         if (mutable_precision == ngraph::element::i64) {
+            mutable_precision = ngraph::element::i32;
+        }
+
+        cldnn::layout mutableLayout = cldnn::layout(
+            DataTypeFromPrecision(mutable_precision),
+            DefaultFormatForDims(op->get_output_shape(1).size()),
+            CldnnTensorFromIEDims(op->get_output_shape(1)));
+
+        shared_memory.emplace_back(cldnn::memory::allocate(p.GetEngine(), mutableLayout));
+
+        cldnn::primitive_id ctc_gd_mutable_id_w = layer_type_name_ID(op) + "_md_write";
+        auto ctc_gd_mutable_prim = cldnn::mutable_data(ctc_gd_mutable_id_w, shared_memory[0]);
+        p.primitivesToIRLayersMap[ctc_gd_mutable_id_w] = { op->get_friendly_name() };
+        p.primitiveIDs[ctc_gd_mutable_id_w] = ctc_gd_mutable_id_w;
+        p.AddPrimitive(ctc_gd_mutable_prim);
+        reorderedInputs.push_back(ctc_gd_mutable_id_w);
+    }
+
+    auto CTCGreedyDecoderLayerName = num_output == 2 ? layer_type_name_ID(op) + ".0" : layer_type_name_ID(op);
+    auto primitive = cldnn::ctc_greedy_decoder(
+                CTCGreedyDecoderLayerName,
+                reorderedInputs,
+                blank_index,
+                ctc_merge_repeated,
+                CldnnTensorFromIEDims(op->get_output_shape(0)));
+
+    // clDNN primitive supports only i32 as output data type
+    primitive.output_data_type = DataTypeFromPrecision(ngraph::element::i32);
+
+    if (num_output == 2) {
+        primitive.second_output = reorderedInputs.back();
+    }
 
     p.AddPrimitive(primitive);
-    p.AddPrimitiveToProfiler(op);
+
+    if (num_output == 2) {
+        cldnn::primitive_id ctc_gd_mutable_id_r = layer_type_name_ID(op) + ".1";
+        auto ctc_gd_mutable_prim_r = cldnn::mutable_data(ctc_gd_mutable_id_r, { CTCGreedyDecoderLayerName }, shared_memory[0]);
+        p.primitivesToIRLayersMap[ctc_gd_mutable_id_r] = { op->get_friendly_name() };
+        p.primitiveIDs[ctc_gd_mutable_id_r] = ctc_gd_mutable_id_r;
+        p.AddPrimitive(ctc_gd_mutable_prim_r);
+    }
+
+    p.AddPrimitiveToProfiler(CTCGreedyDecoderLayerName, op);
+}
+
+void CreateCTCGreedyDecoderOp(Program& p, const std::shared_ptr<ngraph::op::v0::CTCGreedyDecoder>& op) {
+    CreateCommonCTCGreedyDecoderOp(p, op, op->get_ctc_merge_repeated());
+}
+
+void CreateCTCGreedyDecoderSeqLenOp(Program& p, const std::shared_ptr<ngraph::op::v6::CTCGreedyDecoderSeqLen>& op) {
+    CreateCommonCTCGreedyDecoderOp(p, op, op->get_merge_repeated());
 }
 
 REGISTER_FACTORY_IMPL(v0, CTCGreedyDecoder);
+REGISTER_FACTORY_IMPL(v6, CTCGreedyDecoderSeqLen);
 
 }  // namespace CLDNNPlugin
diff --git a/inference-engine/src/gna_plugin/gna_graph_compiler.cpp b/inference-engine/src/gna_plugin/gna_graph_compiler.cpp
@@ -1283,8 +1283,9 @@ void GNAGraphCompiler::AffinePrimitive(InferenceEngine::CNNLayerPtr layer, bool
 
     auto input_data = HasTo2DReshapeData(layer) ? Get2DReshapedData(inputs, 8) : inputs;
     auto in_dims = input_data->getDims();
-    uint32_t num_rows_in = InferenceEngine::details::product(in_dims) / in_dims.front();
-    uint32_t num_columns_in = in_dims.front();
+    auto batch_size = (in_dims.size() == 1) ? 1 : in_dims.front();
+    uint32_t num_rows_in = InferenceEngine::details::product(in_dims) / batch_size;
+    uint32_t num_columns_in = batch_size;
     uint32_t num_rows_out = isDiag ? num_rows_in : FROM_IR_DIM(outputs, 1);
     uint32_t num_padding = ALIGN(num_rows_in, 8) - num_rows_in;
     uint32_t num_padding_out = isDiag ? num_padding : 0;

diff --git a/inference-engine/src/gna_plugin/gna_groups.hpp b/inference-engine/src/gna_plugin/gna_groups.hpp
@@ -31,7 +31,8 @@ inline InferenceEngine::DataPtr Get2DReshapedData(InferenceEngine::DataPtr input
         }
     }
 
-    InferenceEngine::SizeVector newDims(dims.size(), 1);
+    size_t newDimsSize = (dims.size() > 1) ? dims.size() : 2;
+    InferenceEngine::SizeVector newDims(newDimsSize, 1);
     newDims[0] = numColumnsIn;
     newDims[1] = numRowsIn;
     return std::make_shared<InferenceEngine::Data>(input->getName(),
-Original file line number
+Diff line change
@@ Expand Up / @@ -38,8 +38,6 @@ FILE_PATTERNS = *.c \ @@
                              *.hpp \
                              *.md
-    EXCLUDE_PATTERNS       = generic_ie.hpp
     EXCLUDE_SYMBOLS        = InferenceEngine::details
     TAGFILES               = "@DOCS_BUILD_DIR@/ie_api.tag=.." \
@@ Expand Down @@