+
+ **Option 2: Manually create the Guest VM**
+ 1. Create an empty virtual disk image:
+ ```sh
+ sudo qemu-img create -f qcow2 /ovsa_ovsa_runtime_vm_disk.qcow2 20G
+ ```
+ 2. Install Ubuntu 18.04 on the Guest VM. Name the Guest VM `ovsa_runtime`:
+ ```sh
+ sudo qemu-system-x86_64 -m 8192 -enable-kvm \
+ -cpu host \
+ -drive if=virtio,file=/ovsa_ovsa_runtime_vm_disk.qcow2,cache=none \
+ -cdrom /ubuntu-18.04.5-live-server-amd64.iso \
+ -device e1000,netdev=hostnet1,mac=52:54:00:d1:66:5f \
+ -netdev tap,id=hostnet1,script=/virbr0-qemu-ifup, downscript=/virbr0-qemu-ifdown \
+ -vnc :2
+ ```
+ 3. Connect a VNC client with `:2`.
+ 4. Follow the prompts on the screen to finish installing the Guest VM. Name the Guest VM `ovsa_runtime`.
+ 5. Shut down the Guest VM.
+ 6. Restart the Guest VM:
+ ```sh
+ sudo qemu-system-x86_64 -m 8192 -enable-kvm \
+ -cpu host \
+ -drive if=virtio,file=/ovsa_ovsa_runtime_vm_disk.qcow2,cache=none \
+ -device e1000,netdev=hostnet1,mac=52:54:00:d1:66:5f \
+ -netdev tap,id=hostnet1,script=/virbr0-qemu-ifup, downscript=/virbr0-qemu-ifdown \
+ -vnc :2
+ ```
+ 7. Choose ONE of these options to install additional required software:
+
+ **Option 1: Use a script to install additional software**
+ 1. Copy the script `install_guest_deps.sh` from the `Scripts/reference` directory of the OVSA repository to the Guest VM
+ 2. Run the script.
+ 3. Shut down the Guest VM.
+
+ **Option 2: Manually install additional software**
+ 1. Install the software tool [`tpm2-tss`](https://github.com/tpm2-software/tpm2-tss/releases/download/2.4.4/tpm2-tss-2.4.4.tar.gz)
+ Installation information is at https://github.com/tpm2-software/tpm2-tss/blob/master/INSTALL.md
+ 2. Install the software tool [`tpm2-abmrd`](https://github.com/tpm2-software/tpm2-abrmd/releases/download/2.3.3/tpm2-abrmd-2.3.3.tar.gz)
+ Installation information is at https://github.com/tpm2-software/tpm2-abrmd/blob/master/INSTALL.md
+ 3. Install the [`tpm2-tools`](https://github.com/tpm2-software/tpm2-tools/releases/download/4.3.0/tpm2-tools-4.3.0.tar.gz)
+ Installation information is at https://github.com/tpm2-software/tpm2-tools/blob/master/INSTALL.md
+ 4. Install the [Docker packages](https://docs.docker.com/engine/install/ubuntu/)
+ 5. Shut down the Guest VM.
+
+2. Create a directory to support the virtual TPM device. Only `root` should have read/write permission to this directory:
+ ```sh
+ sudo mkdir /var/OVSA/vtpm/vtpm_runtime
+ ```
+ **NOTE**: For steps 3 and 4, you can copy and edit the script named `start_ovsa_runtime_vm.sh` in the scripts directory in the OpenVINO™ Security Add-on repository instead of manually running the commands. Edit the script to point to the correct directory locations and increment `vnc` for each Guest VM. This means that if you are creating a third Guest VM on the same Host Machine, change `-vnc :2` to `-vnc :3`
+3. Start the vTPM:
+ ```sh
+ swtpm socket --tpmstate dir=/var/OVSA/vtpm/vtpm_runtime \
+ --tpm2 \
+ --ctrl type=unixio,path=/var/OVSA/vtpm/vtpm_runtime/swtpm-sock \
+ --log level=20
+ ```
+4. Start the Guest VM in a new terminal. To do so, either copy and edit the script named `start_ovsa_runtime_vm.sh` in the scripts directory in the OpenVINO™ Security Add-on repository or manually run the command:
+ ```sh
+ sudo qemu-system-x86_64 \
+ -cpu host \
+ -enable-kvm \
+ -m 8192 \
+ -smp 8,sockets=1,cores=8,threads=1 \
+ -device e1000,netdev=hostnet2,mac=52:54:00:d1:67:6f \
+ -netdev tap,id=hostnet2,script=/br0-qemu-ifup,downscript=/br0-qemu-ifdown \
+ -device e1000,netdev=hostnet3,mac=52:54:00:d1:67:5f \
+ -netdev tap,id=hostnet3,script=/virbr0-qemu-ifup,downscript=/virbr0-qemu-ifdown \
+ -drive if=virtio,file=/ovsa_runtime_vm_disk.qcow2,cache=none \
+ -chardev socket,id=chrtpm,path=/var/OVSA/vtpm/vtpm_runtime/swtpm-sock \
+ -tpmdev emulator,id=tpm0,chardev=chrtpm \
+ -device tpm-tis,tpmdev=tpm0 \
+ -vnc :2
+ ```
+ Use the QEMU runtime options in the command to change the memory amount or CPU assigned to this Guest VM.
+5. Use a VNC client to log on to the Guest VM at `:` where `` corresponds to the vnc number in the `start_ovsa_isv_vm.sh` or in step 8.
+
+## How to Build and Install the OpenVINO™ Security Add-on Software
+
+Follow the below steps to build and Install OpenVINO™ Security Add-on on host and different VMs.
+
+### Step 1: Build the OpenVINO™ Model Server image
+Building OpenVINO™ Security Add-on depends on OpenVINO™ Model Server docker containers. Download and build OpenVINO™ Model Server first on the host.
+
+1. Download the [OpenVINO™ Model Server software](https://github.com/openvinotoolkit/model_server)
+2. Build the [OpenVINO™ Model Server Docker images](https://github.com/openvinotoolkit/model_server/blob/main/docs/docker_container.md)
+ ```sh
+ git clone https://github.com/openvinotoolkit/model_server.git
+ cd model_server
+ make docker_build
+ ```
+### Step 2: Build the software required for all roles
+
+This step is for the combined role of Model Developer and Independent Software Vendor, and the User
+
+1. Download the [OpenVINO™ Security Add-on](https://github.com/openvinotoolkit/security_addon)
+
+2. Go to the top-level OpenVINO™ Security Add-on source directory.
+ ```sh
+ cd security_addon
+ ```
+3. Build the OpenVINO™ Security Add-on:
+ ```sh
+ make clean all
+ sudo make package
+ ```
+ The following packages are created under the `release_files` directory:
+ - `ovsa-kvm-host.tar.gz`: Host Machine file
+ - `ovsa-developer.tar.gz`: For the Model Developer and the Independent Software Developer
+ - `ovsa-model-hosting.tar.gz`: For the User
+
+### Step 3: Install the host software
+This step is for the combined role of Model Developer and Independent Software Vendor, and the User.
+
+1. Go to the `release_files` directory:
+ ```sh
+ cd release_files
+ ```
+2. Set up the path:
+ ```sh
+ export OVSA_RELEASE_PATH=$PWD
+ ```
+3. Install the OpenVINO™ Security Add-on Software on the Host Machine:
+ ```sh
+ cd $OVSA_RELEASE_PATH
+ tar xvfz ovsa-kvm-host.tar.gz
+ cd ovsa-kvm-host
+ ./install.sh
+ ```
+
+If you are using more than one Host Machine repeat Step 3 on each.
+
+### Step 4: Set up packages on the Guest VM
+This step is for the combined role of Model Developer and Independent Software Vendor. References to the Guest VM are to `ovsa_isv_dev`.
+
+1. Log on to the Guest VM.
+2. Create the OpenVINO™ Security Add-on directory in the home directory
+ ```sh
+ mkdir OVSA
+ ```
+3. Go to the Host Machine, outside of the Guest VM.
+4. Copy `ovsa-developer.tar.gz` from `release_files` to the Guest VM:
+ ```sh
+ cd $OVSA_RELEASE_PATH
+ scp ovsa-developer.tar.gz username@://OVSA
+ ```
+5. Go to the Guest VM.
+6. Install the software to the Guest VM:
+ ```sh
+ cd OVSA
+ tar xvfz ovsa-developer.tar.gz
+ cd ovsa-developer
+ sudo -s
+ ./install.sh
+ ```
+7. Create a directory named `artefacts`. This directory will hold artefacts required to create licenses:
+ ```sh
+ cd //OVSA
+ mkdir artefacts
+ cd artefacts
+ ```
+8. Start the license server on a separate terminal.
+ ```sh
+ sudo -s
+ source /opt/ovsa/scripts/setupvars.sh
+ cd /opt/ovsa/bin
+ ./license_server
+ ```
+
+### Step 5: Install the OpenVINO™ Security Add-on Model Hosting Component
+
+This step is for the User. References to the Guest VM are to `ovsa_runtime`.
+
+The Model Hosting components install the OpenVINO™ Security Add-on Runtime Docker container based on OpenVINO™ Model Server NGINX Docker to host a access controlled model.
+
+1. Log on to the Guest VM as ``.
+2. Create the OpenVINO™ Security Add-on directory in the home directory
+ ```sh
+ mkdir OVSA
+ ```
+3. While on the Host Machine copy the ovsa-model-hosting.tar.gz from release_files to the Guest VM:
+ ```sh
+ cd $OVSA_RELEASE_PATH
+ scp ovsa-model-hosting.tar.gz username@://OVSA
+ ```
+4. Install the software to the Guest VM:
+ ```sh
+ cd OVSA
+ tar xvfz ovsa-model-hosting.tar.gz
+ cd ovsa-model-hosting
+ sudo -s
+ ./install.sh
+ ```
+5. Create a directory named `artefacts`:
+ ```sh
+ cd //OVSA
+ mkdir artefacts
+ cd artefacts
+ ```
+
+## How to Use the OpenVINO™ Security Add-on
+
+This section requires interactions between the Model Developer/Independent Software vendor and the User. All roles must complete all applicable set up steps and installation steps before beginning this section.
+
+This document uses the [face-detection-retail-0004](@ref omz_models_intel_face_detection_retail_0004_description_face_detection_retail_0004) model as an example.
+
+The following figure describes the interactions between the Model Developer, Independent Software Vendor, and User.
+
+**Remember**: The Model Developer/Independent Software Vendor and User roles are related to virtual machine use and one person might fill the tasks required by multiple roles. In this document the tasks of Model Developer and Independent Software Vendor are combined and use the Guest VM named `ovsa_isv`. It is possible to have all roles set up on the same Host Machine.
+
+
+
+### Model Developer Instructions
+
+The Model Developer creates model, defines access control and creates the user license. References to the Guest VM are to `ovsa_isv_dev`. After the model is created, access control enabled, and the license is ready, the Model Developer provides the license details to the Independent Software Vendor before sharing to the Model User.
+
+#### Step 1: Create a key store and add a certificate to it
+
+1. Set up a path to the artefacts directory:
+ ```sh
+ sudo -s
+ cd //OVSA/artefacts
+ export OVSA_RUNTIME_ARTEFACTS=$PWD
+ source /opt/ovsa/scripts/setupvars.sh
+ ```
+2. Create files to request a certificate:
+ This example uses a self-signed certificate for demonstration purposes. In a production environment, use CSR files to request for a CA-signed certificate.
+ ```sh
+ cd $OVSA_DEV_ARTEFACTS
+ /opt/ovsa/bin/ovsatool keygen -storekey -t ECDSA -n Intel -k isv_keystore -r isv_keystore.csr -e "/C=IN/CN=localhost"
+ ```
+ Two files are created:
+ - `isv_keystore.csr`- A Certificate Signing Request (CSR)
+ - `isv_keystore.csr.crt` - A self-signed certificate
+
+ In a production environment, send `isv_keystore.csr` to a CA to request a CA-signed certificate.
+
+3. Add the certificate to the key store
+ ```sh
+ /opt/ovsa/bin/ovsatool keygen -storecert -c isv_keystore.csr.crt -k isv_keystore
+ ```
+
+#### Step 2: Create the model
+
+This example uses `curl` to download the `face-detection-retail-004` model from the OpenVINO Model Zoo. If you are behind a firewall, check and set your proxy settings.
+
+1. Log on to the Guest VM.
+
+2. Download a model from the Model Zoo:
+ ```sh
+ cd $OVSA_DEV_ARTEFACTS
+ curl --create-dirs https://download.01.org/opencv/2021/openvinotoolkit/2021.1/open_model_zoo/models_bin/1/face-detection-retail-0004/FP32/face-detection-retail-0004.xml https:// download.01.org/opencv/2021/openvinotoolkit/2021.1/open_model_zoo/models_bin/1/face-detection-retail-0004/FP32/face-detection-retail-0004.bin -o model/face-detection-retail-0004.xml -o model/face-detection-retail-0004.bin
+ ```
+ The model is downloaded to the `OVSA_DEV_ARTEFACTS/model` directory.
+
+#### Step 3: Define access control for the model and create a master license for it
+
+1. Go to the `artefacts` directory:
+ ```sh
+ cd $OVSA_DEV_ARTEFACTS
+ ```
+2. Run the `uuidgen` command:
+ ```sh
+ uuidgen
+ ```
+3. Define and enable the model access control and master license:
+ ```sh
+ /opt/ovsa/bin/ovsatool protect -i model/face-detection-retail-0004.xml model/face-detection-retail-0004.bin -n "face detection" -d "face detection retail" -v 0004 -p face_detection_model.dat -m face_detection_model.masterlic -k isv_keystore -g
+ ```
+The Intermediate Representation files for the `face-detection-retail-0004` model are encrypted as `face_detection_model.dat` and a master license is generated as `face_detection_model.masterlic`.
+
+#### Step 4: Create a Runtime Reference TCB
+
+Use the runtime reference TCB to create a customer license for the access controlled model and the specific runtime.
+
+Generate the reference TCB for the runtime
+```sh
+cd $OVSA_DEV_ARTEFACTS
+source /opt/ovsa/scripts/setupvars.sh
+ /opt/ovsa/bin/ovsaruntime gen-tcb-signature -n "Face Detect @ Runtime VM" -v "1.0" -f face_detect_runtime_vm.tcb -k isv_keystore
+```
+
+#### Step 5: Publish the access controlled Model and Runtime Reference TCB
+The access controlled model is ready to be shared with the User and the reference TCB is ready to perform license checks.
+
+#### Step 6: Receive a User Request
+1. Obtain artefacts from the User who needs access to a access controlled model:
+ * Customer certificate from the customer's key store.
+ * Other information that apply to your licensing practices, such as the length of time the user needs access to the model
+
+2. Create a customer license configuration
+ ```sh
+ cd $OVSA_DEV_ARTEFACTS
+ /opt/ovsa/bin/ovsatool licgen -t TimeLimit -l30 -n "Time Limit License Config" -v 1.0 -u ":" -k isv_keystore -o 30daylicense.config
+ ```
+3. Create the customer license
+ ```sh
+ cd $OVSA_DEV_ARTEFACTS
+ /opt/ovsa/bin/ovsatool sale -m face_detection_model.masterlic -k isv_keystore -l 30daylicense.config -t face_detect_runtime_vm.tcb -p custkeystore.csr.crt -c face_detection_model.lic
+ ```
+
+4. Update the license server database with the license.
+ ```sh
+ cd /opt/ovsa/DB
+ python3 ovsa_store_customer_lic_cert_db.py ovsa.db $OVSA_DEV_ARTEFACTS/face_detection_model.lic $OVSA_DEV_ARTEFACTS/custkeystore.csr.crt
+ ```
+
+5. Provide these files to the User:
+ * `face_detection_model.dat`
+ * `face_detection_model.lic`
+
+### User Instructions
+References to the Guest VM are to `ovsa_rumtime`.
+
+#### Step 1: Add a CA-Signed Certificate to a Key Store
+
+1. Set up a path to the artefacts directory:
+ ```sh
+ sudo -s
+ cd //OVSA/artefacts
+ export OVSA_RUNTIME_ARTEFACTS=$PWD
+ source /opt/ovsa/scripts/setupvars.sh
+ ```
+2. Generate a Customer key store file:
+ ```sh
+ cd $OVSA_RUNTIME_ARTEFACTS
+ /opt/ovsa/bin/ovsatool keygen -storekey -t ECDSA -n Intel -k custkeystore -r custkeystore.csr -e "/C=IN/CN=localhost"
+ ```
+ Two files are created:
+ * `custkeystore.csr` - A Certificate Signing Request (CSR)
+ * `custkeystore.csr.crt` - A self-signed certificate
+
+3. Send `custkeystore.csr` to the CA to request a CA-signed certificate.
+
+4. Add the certificate to the key store:
+ ```sh
+ /opt/ovsa/bin/ovsatool keygen -storecert -c custkeystore.csr.crt -k custkeystore
+ ```
+
+#### Step 2: Request an access controlled Model from the Model Developer
+This example uses scp to share data between the ovsa_runtime and ovsa_dev Guest VMs on the same Host Machine.
+
+1. Communicate your need for a model to the Model Developer. The Developer will ask you to provide the certificate from your key store and other information. This example uses the length of time the model needs to be available.
+2. Generate an artefact file to provide to the Developer:
+ ```sh
+ cd $OVSA_RUNTIME_ARTEFACTS
+ scp custkeystore.csr.crt username@://OVSA/artefacts
+ ```
+#### Step 3: Receive and load the access controlled model into the OpenVINO™ Model Server
+1. Receive the model as files named
+ * `face_detection_model.dat`
+ * `face_detection_model.lic`
+2. Prepare the environment:
+ ```sh
+ cd $OVSA_RUNTIME_ARTEFACTS/..
+ cp /opt/ovsa/example_runtime ovms -r
+ cd ovms
+ mkdir -vp model/fd/1
+ ```
+ The `$OVSA_RUNTIME_ARTEFACTS/../ovms` directory contains scripts and a sample configuration JSON file to start the model server.
+3. Copy the artefacts from the Model Developer:
+ ```sh
+ cd $OVSA_RUNTIME_ARTEFACTS/../ovms
+ cp $OVSA_RUNTIME_ARTEFACTS/face_detection_model.dat model/fd/1/.
+ cp $OVSA_RUNTIME_ARTEFACTS/face_detection_model.lic model/fd/1/.
+ cp $OVSA_RUNTIME_ARTEFACTS/custkeystore model/fd/1/.
+ ```
+4. Rename and edit `sample.json` to include the names of the access controlled model artefacts you received from the Model Developer. The file looks like this:
+ ```sh
+ {
+ "custom_loader_config_list":[
+ {
+ "config":{
+ "loader_name":"ovsa",
+ "library_path": "/ovsa-runtime/lib/libovsaruntime.so"
+ }
+ }
+ ],
+ "model_config_list":[
+ {
+ "config":{
+ "name":"protected-model",
+ "base_path":"/sampleloader/model/fd",
+ "custom_loader_options": {"loader_name": "ovsa", "keystore": "custkeystore", "protected_file": "face_detection_model"}
+ }
+ }
+ ]
+ }
+ ```
+#### Step 4: Start the NGINX Model Server
+The NGINX Model Server publishes the access controlled model.
+ ```sh
+ ./start_secure_ovsa_model_server.sh
+ ```
+For information about the NGINX interface, see https://github.com/openvinotoolkit/model_server/blob/main/extras/nginx-mtls-auth/README.md
+
+#### Step 5: Prepare to run Inference
+
+1. Log on to the Guest VM from another terminal.
+
+2. Install the Python dependencies for your set up. For example:
+ ```sh
+ sudo apt install pip3
+ pip3 install cmake
+ pip3 install scikit-build
+ pip3 install opencv-python
+ pip3 install futures==3.1.1
+ pip3 install tensorflow-serving-api==1.14.0
+ ```
+3. Copy the `face_detection.py` from the example_client in `/opt/ovsa/example_client`
+ ```sh
+ cd /home/intel/OVSA/ovms
+ cp /opt/ovsa/example_client/* .
+ ```
+4. Copy the sample images for inferencing. An image directory is created that includes a sample image for inferencing.
+ ```sh
+ curl --create-dirs https://raw.githubusercontent.com/openvinotoolkit/model_server/master/example_client/images/people/people1.jpeg -o images/people1.jpeg
+ ```
+#### Step 6: Run Inference
+
+Run the `face_detection.py` script:
+```sh
+python3 face_detection.py --grpc_port 3335 --batch_size 1 --width 300 --height 300 --input_images_dir images --output_dir results --tls --server_cert server.pem --client_cert client.pem --client_key client.key --model_name protected-model
+```
+
+## Summary
+You have completed these tasks:
+- Set up one or more computers (Host Machines) with one KVM per machine and one or more virtual machines (Guest VMs) on the Host Machines
+- Installed the OpenVINO™ Security Add-on
+- Used the OpenVINO™ Model Server to work with OpenVINO™ Security Add-on
+- As a Model Developer or Independent Software Vendor, you access controlled a model and prepared a license for it.
+- As a Model Developer or Independent Software Vendor, you prepared and ran a License Server and used the License Server to verify a User had a valid license to use a access controlled model.
+- As a User, you provided information to a Model Developer or Independent Software Vendor to get a access controlled model and the license for the model.
+- As a User, you set up and launched a Host Server on which you can run licensed and access controlled models.
+- As a User, you loaded a access controlled model, validated the license for the model, and used the model to run inference.
+
+## References
+Use these links for more information:
+- [OpenVINO™ toolkit](https://software.intel.com/en-us/openvino-toolkit)
+- [OpenVINO Model Server Quick Start Guide](https://github.com/openvinotoolkit/model_server/blob/main/docs/ovms_quickstart.md)
+- [Model repository](https://github.com/openvinotoolkit/model_server/blob/main/docs/models_repository.md)
diff --git a/docs/template_plugin/src/template_async_infer_request.cpp b/docs/template_plugin/src/template_async_infer_request.cpp
index 3facaf7327d5f6..41c1f62724f6b5 100644
--- a/docs/template_plugin/src/template_async_infer_request.cpp
+++ b/docs/template_plugin/src/template_async_infer_request.cpp
@@ -25,16 +25,19 @@ TemplateAsyncInferRequest::TemplateAsyncInferRequest(
if (remoteDevice) {
_pipeline = {
{cpuTaskExecutor, [this] {
- IE_PROFILING_AUTO_SCOPE(PreprocessingAndStartPipeline)
+ OV_ITT_SCOPED_TASK(itt::domains::TemplatePlugin,
+ "TemplateAsyncInferRequest::PreprocessingAndStartPipeline");
_inferRequest->inferPreprocess();
_inferRequest->startPipeline();
}},
{_waitExecutor, [this] {
- IE_PROFILING_AUTO_SCOPE(WaitPipeline)
+ OV_ITT_SCOPED_TASK(itt::domains::TemplatePlugin,
+ "TemplateAsyncInferRequest::WaitPipeline");
_inferRequest->waitPipeline();
}},
{cpuTaskExecutor, [this] {
- IE_PROFILING_AUTO_SCOPE(Postprocessing)
+ OV_ITT_SCOPED_TASK(itt::domains::TemplatePlugin,
+ "TemplateAsyncInferRequest::Postprocessing");
_inferRequest->inferPostprocess();
}}
};
diff --git a/docs/template_plugin/src/template_executable_network.cpp b/docs/template_plugin/src/template_executable_network.cpp
index 7eff8da41484be..0a2193342d8af5 100644
--- a/docs/template_plugin/src/template_executable_network.cpp
+++ b/docs/template_plugin/src/template_executable_network.cpp
@@ -10,6 +10,7 @@
#include "template/template_config.hpp"
#include "template_plugin.hpp"
#include "template_executable_network.hpp"
+#include "template_itt.hpp"
using namespace TemplatePlugin;
@@ -61,7 +62,7 @@ TemplatePlugin::ExecutableNetwork::ExecutableNetwork(std::istream & model,
model.read(dataBlob->buffer(), dataSize);
}
- // TODO: implement Import / Export of configuration options
+ // TODO: implement Import / Export of configuration options and merge with `cfg`
// TODO: implement Import / Export of network precisions, layouts, preprocessing info
auto cnnnetwork = _plugin->GetCore()->ReadNetwork(xmlString, std::move(dataBlob));
@@ -142,7 +143,7 @@ InferenceEngine::IInferRequest::Ptr TemplatePlugin::ExecutableNetwork::CreateInf
auto internalRequest = CreateInferRequestImpl(_networkInputs, _networkOutputs);
auto asyncThreadSafeImpl = std::make_shared(std::static_pointer_cast(internalRequest),
_taskExecutor, _plugin->_waitExecutor, _callbackExecutor);
- asyncRequest.reset(new InferenceEngine::InferRequestBase(asyncThreadSafeImpl),
+ asyncRequest.reset(new InferenceEngine::InferRequestBase(asyncThreadSafeImpl),
[](InferenceEngine::IInferRequest *p) { p->Release(); });
asyncThreadSafeImpl->SetPointerToPublicInterface(asyncRequest);
return asyncRequest;
@@ -188,6 +189,8 @@ InferenceEngine::Parameter TemplatePlugin::ExecutableNetwork::GetMetric(const st
// ! [executable_network:export_impl]
void TemplatePlugin::ExecutableNetwork::ExportImpl(std::ostream& modelStream) {
+ OV_ITT_SCOPED_TASK(itt::domains::TemplatePlugin, "ExecutableNetwork::ExportImpl");
+
// Note: custom ngraph extensions are not supported
std::map custom_opsets;
std::stringstream xmlFile, binFile;
diff --git a/docs/template_plugin/src/template_infer_request.cpp b/docs/template_plugin/src/template_infer_request.cpp
index 30c9a4c0f9fa95..61f4cb15120b7b 100644
--- a/docs/template_plugin/src/template_infer_request.cpp
+++ b/docs/template_plugin/src/template_infer_request.cpp
@@ -244,7 +244,8 @@ void TemplateInferRequest::inferPostprocess() {
// ! [infer_request:infer_postprocess]
// ! [infer_request:get_performance_counts]
-void TemplateInferRequest::GetPerformanceCounts(std::map &perfMap) const {
+std::map TemplateInferRequest::GetPerformanceCounts() const {
+ std::map perfMap;
InferenceEngineProfileInfo info;
info.execution_index = 0;
info.status = InferenceEngineProfileInfo::EXECUTED;
@@ -259,5 +260,6 @@ void TemplateInferRequest::GetPerformanceCounts(std::map& perfMap) const override;
+ std::map GetPerformanceCounts() const override;
InferenceEngine::StatusCode Cancel() override;
diff --git a/docs/template_plugin/src/template_plugin.cpp b/docs/template_plugin/src/template_plugin.cpp
index 6b9610f722f00e..ff339499645cb0 100644
--- a/docs/template_plugin/src/template_plugin.cpp
+++ b/docs/template_plugin/src/template_plugin.cpp
@@ -16,6 +16,7 @@
#include
#include "template/template_config.hpp"
+#include "template_itt.hpp"
#include "template_plugin.hpp"
#include "template_executable_network.hpp"
#include "template_infer_request.hpp"
@@ -74,6 +75,8 @@ std::shared_ptr TransformNetwork(const std::shared_ptr& config) {
- // TODO: Import network from stream is not mandatory functionality;
- // Can just throw an exception and remove the code below
- Configuration exportedCfg;
-
- // some code below which reads exportedCfg from `model` stream
- // ..
+ OV_ITT_SCOPED_TASK(itt::domains::TemplatePlugin, "Plugin::ImportNetworkImpl");
- auto cfg = Configuration(config, exportedCfg);
- auto exec_network_impl = std::make_shared(model, cfg, std::static_pointer_cast(shared_from_this()));
+ Configuration cfg(config);
+ auto exec_network_impl = std::make_shared(model, cfg,
+ std::static_pointer_cast(shared_from_this()));
return make_executable_network(exec_network_impl);
}
@@ -129,6 +128,8 @@ InferenceEngine::ExecutableNetwork Plugin::ImportNetworkImpl(std::istream& model
// ! [plugin:query_network]
InferenceEngine::QueryNetworkResult Plugin::QueryNetwork(const InferenceEngine::CNNNetwork &network, const ConfigMap& config) const {
+ OV_ITT_SCOPED_TASK(itt::domains::TemplatePlugin, "Plugin::QueryNetwork");
+
InferenceEngine::QueryNetworkResult res;
Configuration cfg{config, _cfg, false};
diff --git a/inference-engine/cmake/dependencies.cmake b/inference-engine/cmake/dependencies.cmake
index 81e01c6de25bf2..ae69fcc016a54b 100644
--- a/inference-engine/cmake/dependencies.cmake
+++ b/inference-engine/cmake/dependencies.cmake
@@ -136,7 +136,6 @@ endif ()
## TBB package
if (THREADING STREQUAL "TBB" OR THREADING STREQUAL "TBB_AUTO")
-
reset_deps_cache(TBBROOT TBB_DIR)
if (WIN32 AND X86_64)
@@ -235,22 +234,23 @@ if (ENABLE_OPENCV)
elseif (ARM)
set(OPENCV_SUFFIX "debian9arm")
set(OPENCV_HASH "0e787d6738092993bc92bb55975f52caabae45dc73473b5196d15e65e87d6b9d")
- elseif (LINUX_OS_NAME STREQUAL "CentOS 7" OR CMAKE_CXX_COMPILER_VERSION VERSION_LESS "4.9")
+ elseif ((LINUX_OS_NAME STREQUAL "CentOS 7" OR
+ CMAKE_CXX_COMPILER_VERSION VERSION_LESS "4.9") AND X86_64)
set(OPENCV_SUFFIX "centos7")
set(OPENCV_HASH "9b813af064d463b31fa1603b11b6559532a031d59bb0782d234380955fd397e0")
- elseif (LINUX_OS_NAME MATCHES "CentOS 8")
+ elseif (LINUX_OS_NAME MATCHES "CentOS 8" AND X86_64)
set(OPENCV_SUFFIX "centos8")
set(OPENCV_HASH "8ec3e3552500dee334162386b98cc54a5608de1f1a18f283523fc0cc13ee2f83")
- elseif (LINUX_OS_NAME STREQUAL "Ubuntu 16.04")
+ elseif (LINUX_OS_NAME STREQUAL "Ubuntu 16.04" AND X86_64)
set(OPENCV_SUFFIX "ubuntu16")
set(OPENCV_HASH "cd46831b4d8d1c0891d8d22ff5b2670d0a465a8a8285243059659a50ceeae2c3")
- elseif (LINUX_OS_NAME STREQUAL "Ubuntu 18.04")
+ elseif (LINUX_OS_NAME STREQUAL "Ubuntu 18.04" AND X86_64)
set(OPENCV_SUFFIX "ubuntu18")
set(OPENCV_HASH "8ec3e3552500dee334162386b98cc54a5608de1f1a18f283523fc0cc13ee2f83")
- elseif (LINUX_OS_NAME STREQUAL "Ubuntu 20.04")
+ elseif (LINUX_OS_NAME STREQUAL "Ubuntu 20.04" AND X86_64)
set(OPENCV_SUFFIX "ubuntu20")
set(OPENCV_HASH "2b7808d002864acdc5fc0b19cd30dadc31a37cc267931cad605f23f2383bfc21")
- else()
+ elseif(NOT DEFINED OpenCV_DIR AND NOT DEFINED ENV{OpenCV_DIR})
message(FATAL_ERROR "OpenCV is not available on current platform (${LINUX_OS_NAME})")
endif()
RESOLVE_DEPENDENCY(OPENCV
diff --git a/inference-engine/cmake/features.cmake b/inference-engine/cmake/features.cmake
index 8c83eaf24536e7..56aeba29020835 100644
--- a/inference-engine/cmake/features.cmake
+++ b/inference-engine/cmake/features.cmake
@@ -2,9 +2,9 @@
# SPDX-License-Identifier: Apache-2.0
#
-#these options are aimed to optimize build time on development system
+# these options are aimed to optimize build time on development system
-ie_dependent_option (ENABLE_GNA "GNA support for inference engine" ON "NOT APPLE;NOT ANDROID;X86 OR X86_64" OFF)
+ie_dependent_option (ENABLE_GNA "GNA support for inference engine" ON "NOT APPLE;NOT ANDROID;X86_64" OFF)
ie_dependent_option (ENABLE_CLDNN_TESTS "Enable clDNN unit tests" OFF "ENABLE_CLDNN" OFF)
@@ -26,7 +26,7 @@ if (ENABLE_MKL_DNN)
endif()
# "MKL-DNN library based on OMP or TBB or Sequential implementation: TBB|OMP|SEQ"
-if(ARM OR (MSVC AND (ARM OR AARCH64)) )
+if(X86 OR ARM OR (MSVC AND (ARM OR AARCH64)) )
set(THREADING_DEFAULT "SEQ")
else()
set(THREADING_DEFAULT "TBB")
diff --git a/inference-engine/cmake/vpu_dependencies.cmake b/inference-engine/cmake/vpu_dependencies.cmake
index 3c5fd59cb9589f..3ffce54ee1ffd8 100644
--- a/inference-engine/cmake/vpu_dependencies.cmake
+++ b/inference-engine/cmake/vpu_dependencies.cmake
@@ -1,4 +1,4 @@
-# Copyright (C) 2018-2020 Intel Corporation
+# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -6,14 +6,14 @@ include_guard(GLOBAL)
set(VPU_SUPPORTED_FIRMWARES usb-ma2x8x pcie-ma2x8x)
set(VPU_SUPPORTED_FIRMWARES_HASH
- "39a35758b76463f633f377616057c7d2a24562c7c1cfc36744f28949619e57c9"
- "798df21b5b3a8c4a6faab61f9220b2b216ba6c4a5acf75aaa17a8520bc639bfe")
+ "7892e82f8ba90b487c4b115bfc266265d8ceb6f3cfc3e7e203ec6150d041fa2c"
+ "bec36fa7a8b64cd50df8b7782c594df32267c5081d7aa2e77a701dcfa18b3ec6")
#
# Default packages
#
-set(FIRMWARE_PACKAGE_VERSION 1579)
+set(FIRMWARE_PACKAGE_VERSION 1599)
set(VPU_CLC_MA2X8X_VERSION "movi-cltools-20.09.2")
#
diff --git a/inference-engine/ie_bridges/c/samples/hello_classification/README.md b/inference-engine/ie_bridges/c/samples/hello_classification/README.md
index 845a19e1bf52dc..6bf0ddf0b6369b 100644
--- a/inference-engine/ie_bridges/c/samples/hello_classification/README.md
+++ b/inference-engine/ie_bridges/c/samples/hello_classification/README.md
@@ -14,7 +14,7 @@ To properly demonstrate this API, it is required to run several networks in pipe
## Running
-To run the sample, you can use public or pre-trained models. To download the pre-trained models, use the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/ie_bridges/c/samples/hello_nv12_input_classification/README.md b/inference-engine/ie_bridges/c/samples/hello_nv12_input_classification/README.md
index eeadef10cdbf88..a9e1e20056b049 100644
--- a/inference-engine/ie_bridges/c/samples/hello_nv12_input_classification/README.md
+++ b/inference-engine/ie_bridges/c/samples/hello_nv12_input_classification/README.md
@@ -34,9 +34,7 @@ ffmpeg -i cat.jpg -pix_fmt nv12 cat.yuv
## Running
-To run the sample, you can use public or pre-trained models. To download pre-trained models, use
-the OpenVINO™ [Model Downloader](@ref omz_tools_downloader_README)
-or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the
> Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
diff --git a/inference-engine/ie_bridges/c/samples/object_detection_sample_ssd/README.md b/inference-engine/ie_bridges/c/samples/object_detection_sample_ssd/README.md
index 2e70a23f0576a6..55916a129f9473 100644
--- a/inference-engine/ie_bridges/c/samples/object_detection_sample_ssd/README.md
+++ b/inference-engine/ie_bridges/c/samples/object_detection_sample_ssd/README.md
@@ -39,7 +39,7 @@ Options:
Running the application with the empty list of options yields the usage message given above and an error message.
-To run the sample, you can use public or pre-trained models. To download the pre-trained models, use the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/ie_bridges/c/tests/ie_c_api_test.cpp b/inference-engine/ie_bridges/c/tests/ie_c_api_test.cpp
index 561ede3b3c76fc..b8c1f61adfcc9d 100644
--- a/inference-engine/ie_bridges/c/tests/ie_c_api_test.cpp
+++ b/inference-engine/ie_bridges/c/tests/ie_c_api_test.cpp
@@ -235,6 +235,7 @@ TEST(ie_core_get_metric, getMetric) {
const char *device_name = "CPU";
const char *metric_name = "SUPPORTED_CONFIG_KEYS";
ie_param_t param;
+ param.params = nullptr;
IE_EXPECT_OK(ie_core_get_metric(core, device_name, metric_name, ¶m));
ie_param_free(¶m);
@@ -249,6 +250,7 @@ TEST(ie_core_get_config, getConfig) {
const char *device_name = "CPU";
const char *config_name = "CPU_THREADS_NUM";
ie_param_t param;
+ param.params = nullptr;
IE_EXPECT_OK(ie_core_get_config(core, device_name, config_name, ¶m));
EXPECT_STREQ(param.params, "0");
@@ -847,6 +849,7 @@ TEST(ie_exec_network_get_config, getConfig) {
EXPECT_NE(nullptr, exe_network);
ie_param_t param;
+ param.params = nullptr;
IE_EXPECT_OK(ie_exec_network_get_config(exe_network, "CPU_THREADS_NUM", ¶m));
ie_param_free(¶m);
@@ -901,6 +904,7 @@ TEST(ie_exec_network_get_metric, getMetric) {
EXPECT_NE(nullptr, exe_network);
ie_param_t param;
+ param.params = nullptr;
IE_EXPECT_OK(ie_exec_network_get_metric(exe_network, "SUPPORTED_CONFIG_KEYS", ¶m));
ie_param_free(¶m);
@@ -1735,11 +1739,16 @@ TEST(ie_blob_make_memory_nv12, inferRequestWithNV12Blob) {
ie_blob_t *output_blob = nullptr;
IE_EXPECT_OK(ie_infer_request_get_blob(infer_request, "fc_out", &output_blob));
+ EXPECT_NE(nullptr, output_blob);
ie_blob_buffer_t buffer;
+ buffer.buffer = nullptr;
IE_EXPECT_OK(ie_blob_get_buffer(output_blob, &buffer));
- float *output_data = (float *)(buffer.buffer);
- EXPECT_NEAR(output_data[1], 0.f, 1.e-5);
+ EXPECT_NE(buffer.buffer, nullptr);
+ if (buffer.buffer) {
+ float *output_data = (float *)(buffer.buffer);
+ EXPECT_NEAR(output_data[1], 0.f, 1.e-5);
+ }
ie_blob_free(&output_blob);
ie_blob_free(&blob_nv12);
diff --git a/inference-engine/ie_bridges/python/sample/classification_sample_async/README.md b/inference-engine/ie_bridges/python/sample/classification_sample_async/README.md
index d7a20f5037333d..80dc537b9a5702 100644
--- a/inference-engine/ie_bridges/python/sample/classification_sample_async/README.md
+++ b/inference-engine/ie_bridges/python/sample/classification_sample_async/README.md
@@ -59,7 +59,7 @@ Options:
Running the application with the empty list of options yields the usage message given above and an error message.
-To run the sample, you can use AlexNet and GoogLeNet or other image classification models. You can download the pre-trained models with the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or from [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use AlexNet and GoogLeNet or other image classification models. You can download [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/ie_bridges/python/sample/hello_classification/README.md b/inference-engine/ie_bridges/python/sample/hello_classification/README.md
index 488278c87d2ff4..34858bb437a1bb 100644
--- a/inference-engine/ie_bridges/python/sample/hello_classification/README.md
+++ b/inference-engine/ie_bridges/python/sample/hello_classification/README.md
@@ -46,7 +46,7 @@ Options:
Running the application with the empty list of options yields the usage message given above.
-To run the sample, you can use AlexNet and GoogLeNet or other image classification models. You can download the pre-trained models with the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or from [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use AlexNet and GoogLeNet or other image classification models. You can download [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/ie_bridges/python/sample/ngraph_function_creation_sample/README.md b/inference-engine/ie_bridges/python/sample/ngraph_function_creation_sample/README.md
index 75b05f78c5f5df..bdba6c38ab46e3 100644
--- a/inference-engine/ie_bridges/python/sample/ngraph_function_creation_sample/README.md
+++ b/inference-engine/ie_bridges/python/sample/ngraph_function_creation_sample/README.md
@@ -13,7 +13,7 @@ When the inference is done, the application outputs inference results to the sta
> **NOTE**: This sample supports models with FP32 weights only.
-The `lenet.bin` weights file was generated by the [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
+The `lenet.bin` weights file was generated by the [Model Optimizer](../../../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
tool from the public LeNet model with the `--input_shape [64,1,28,28]` parameter specified.
The original model is available in the [Caffe* repository](https://github.com/BVLC/caffe/tree/master/examples/mnist) on GitHub\*.
@@ -69,4 +69,4 @@ By default, the application outputs top-1 inference result for each inference re
## See Also
-* [Using Inference Engine Samples](../../../docs/IE_DG/Samples_Overview.md)
+* [Using Inference Engine Samples](../../../../../docs/IE_DG/Samples_Overview.md)
diff --git a/inference-engine/ie_bridges/python/sample/object_detection_sample_ssd/README.md b/inference-engine/ie_bridges/python/sample/object_detection_sample_ssd/README.md
index 26b8394cdb7260..90bc09ff2e75bf 100644
--- a/inference-engine/ie_bridges/python/sample/object_detection_sample_ssd/README.md
+++ b/inference-engine/ie_bridges/python/sample/object_detection_sample_ssd/README.md
@@ -55,7 +55,7 @@ Options:
Running the application with the empty list of options yields the usage message given above and an error message.
-To run the sample, you can use RMNet_SSD or other object-detection models. You can download the pre-trained models with the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or from [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use RMNet_SSD or other object-detection models. You can download [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/ie_bridges/python/sample/object_detection_sample_ssd/object_detection_sample_ssd.py b/inference-engine/ie_bridges/python/sample/object_detection_sample_ssd/object_detection_sample_ssd.py
index 0ff4523c931b0d..5bbd3a4609268b 100644
--- a/inference-engine/ie_bridges/python/sample/object_detection_sample_ssd/object_detection_sample_ssd.py
+++ b/inference-engine/ie_bridges/python/sample/object_detection_sample_ssd/object_detection_sample_ssd.py
@@ -73,13 +73,7 @@ def main():
# -----------------------------------------------------------------------------------------------------
# --------------------------- 3. Read and preprocess input --------------------------------------------
-
- print("inputs number: " + str(len(net.input_info.keys())))
- assert len(net.input_info.keys()) == 1, 'Sample supports networks with one input'
-
for input_key in net.input_info:
- print("input shape: " + str(net.input_info[input_key].input_data.shape))
- print("input key: " + input_key)
if len(net.input_info[input_key].input_data.layout) == 4:
n, c, h, w = net.input_info[input_key].input_data.shape
@@ -96,7 +90,6 @@ def main():
image = cv2.resize(image, (w, h))
image = image.transpose((2, 0, 1)) # Change data layout from HWC to CHW
images[i] = image
-
# -----------------------------------------------------------------------------------------------------
# --------------------------- 4. Configure input & output ---------------------------------------------
@@ -122,23 +115,30 @@ def main():
data[input_name] = images
if input_info_name != "":
- infos = np.ndarray(shape=(n, c), dtype=float)
+ detection_size = net.input_info[input_info_name].input_data.shape[1]
+ infos = np.ndarray(shape=(n, detection_size), dtype=float)
for i in range(n):
infos[i, 0] = h
infos[i, 1] = w
- infos[i, 2] = 1.0
+ for j in range(2, detection_size):
+ infos[i, j] = 1.0
data[input_info_name] = infos
# --------------------------- Prepare output blobs ----------------------------------------------------
log.info('Preparing output blobs')
+ output_name, output_info = "", None
func = ng.function_from_cnn(net)
- ops = func.get_ordered_ops()
- output_name, output_info = "", net.outputs[next(iter(net.outputs.keys()))]
- output_ops = {op.friendly_name : op for op in ops \
- if op.friendly_name in net.outputs and op.get_type_name() == "DetectionOutput"}
- if len(output_ops) != 0:
- output_name, output_info = output_ops.popitem()
+ if func:
+ ops = func.get_ordered_ops()
+ for op in ops:
+ if op.friendly_name in net.outputs and op.get_type_name() == "DetectionOutput":
+ output_name = op.friendly_name
+ output_info = net.outputs[output_name]
+ break
+ else:
+ output_name = list(net.outputs.keys())[0]
+ output_info = net.outputs[output_name]
if output_name == "":
log.error("Can't find a DetectionOutput layer in the topology")
@@ -189,12 +189,12 @@ def main():
else:
print()
+ tmp_image = cv2.imread(args.input)
for imid in classes:
- tmp_image = cv2.imread(args.input)
for box in boxes[imid]:
cv2.rectangle(tmp_image, (box[0], box[1]), (box[2], box[3]), (232, 35, 244), 2)
- cv2.imwrite("out.bmp", tmp_image)
- log.info("Image out.bmp created!")
+ cv2.imwrite("out.bmp", tmp_image)
+ log.info("Image out.bmp created!")
# -----------------------------------------------------------------------------------------------------
log.info("Execution successful\n")
diff --git a/inference-engine/ie_bridges/python/src/openvino/inference_engine/constants.pyx b/inference-engine/ie_bridges/python/src/openvino/inference_engine/constants.pyx
index 188d38940bd422..ce5ca4d6dede81 100644
--- a/inference-engine/ie_bridges/python/src/openvino/inference_engine/constants.pyx
+++ b/inference-engine/ie_bridges/python/src/openvino/inference_engine/constants.pyx
@@ -18,7 +18,7 @@ from .cimport ie_api_impl_defs as C
import numpy as np
from enum import Enum
-supported_precisions = ["FP32", "FP64", "FP16", "I64", "U64", "I32", "U32", "I16", "I8", "U16", "U8"]
+supported_precisions = ["FP32", "FP64", "FP16", "I64", "U64", "I32", "U32", "I16", "I8", "U16", "U8", "BOOL"]
known_plugins = ['CPU', 'GPU', 'FPGA', 'MYRIAD', 'HETERO', 'HDDL', 'MULTI']
@@ -34,7 +34,8 @@ format_map = {
'U16' : np.uint16,
'I8' : np.int8,
'U8' : np.uint8,
- 'I64' : np.int64
+ 'I64' : np.int64,
+ 'BOOL' : np.uint8
}
layout_str_to_enum = {'ANY': C.Layout.ANY,
diff --git a/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api.pyx b/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api.pyx
index e92dcaaa58e8ba..c92040d40cf76a 100644
--- a/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api.pyx
+++ b/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api.pyx
@@ -173,7 +173,8 @@ cdef class Blob:
fp64_array_memview = self._array_data
self._ptr = C.make_shared_blob[double](c_tensor_desc, &fp64_array_memview[0], fp64_array_memview.shape[0])
elif precision == "FP16":
- raise RuntimeError("Currently, it's impossible to set_blob with FP16 precision")
+ I16_array_memview = self._array_data.view(dtype=np.int16)
+ self._ptr = C.make_shared_blob[int16_t](c_tensor_desc, &I16_array_memview[0], I16_array_memview.shape[0])
elif precision == "I16":
I16_array_memview = self._array_data
self._ptr = C.make_shared_blob[int16_t](c_tensor_desc, &I16_array_memview[0], I16_array_memview.shape[0])
@@ -1222,7 +1223,10 @@ cdef class InferRequest:
def _fill_inputs(self, inputs):
for k, v in inputs.items():
assert k in self._inputs_list, f"No input with name {k} found in network"
- self.input_blobs[k].buffer[:] = v
+ if self.input_blobs[k].tensor_desc.precision == "FP16":
+ self.input_blobs[k].buffer[:] = v.view(dtype=np.int16)
+ else:
+ self.input_blobs[k].buffer[:] = v
## This class contains the information about the network model read from IR and allows you to manipulate with
@@ -1439,6 +1443,14 @@ cdef class IENetwork:
def _get_function_capsule(self):
return self.impl.getFunction()
+ def get_ov_name_for_tensor(self, orig_name: str):
+ name = bytes(orig_name, 'utf-8')
+ return self.impl.getOVNameForTensor(name).decode('utf-8')
+
+ def get_ov_name_for_operation(self, orig_name: str):
+ name = bytes(orig_name, 'utf-8')
+ return self.impl.getOVNameForOperation(name).decode('utf-8')
+
cdef class BlobBuffer:
"""Copy-less accessor for Inference Engine Blob"""
diff --git a/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.cpp b/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.cpp
index 226cc73bc2ee42..7a2bd205a0837d 100644
--- a/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.cpp
+++ b/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.cpp
@@ -260,6 +260,14 @@ const std::map InferenceEnginePython::IE
return outputs;
}
+std::string InferenceEnginePython::IENetwork::getOVNameForTensor(const std::string& orig_name) {
+ return actual->getOVNameForTensor(orig_name);
+}
+
+std::string InferenceEnginePython::IENetwork::getOVNameForOperation(const std::string& orig_name) {
+ return actual->getOVNameForOperation(orig_name);
+}
+
void
InferenceEnginePython::IENetwork::addOutput(const std::string &out_layer, size_t port_id) {
actual->addOutput(out_layer, port_id);
diff --git a/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.hpp b/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.hpp
index 5534d1ddb53215..eff8c8cec3f504 100644
--- a/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.hpp
+++ b/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.hpp
@@ -71,6 +71,9 @@ struct IENetwork {
IENetwork() = default;
void convertToOldRepresentation();
+
+ std::string getOVNameForTensor(const std::string& orig_name);
+ std::string getOVNameForOperation(const std::string& orig_name);
};
diff --git a/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl_defs.pxd b/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl_defs.pxd
index d11d8b526a8743..91b3e9af849e90 100644
--- a/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl_defs.pxd
+++ b/inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl_defs.pxd
@@ -175,6 +175,8 @@ cdef extern from "ie_api_impl.hpp" namespace "InferenceEnginePython":
void load_from_buffer(const char*xml, size_t xml_size, uint8_t*bin, size_t bin_size) except +
object getFunction() except +
void convertToOldRepresentation() except +
+ string getOVNameForTensor(const string &) except +
+ string getOVNameForOperation(const string &) except +
cdef cppclass InferRequestWrap:
double exec_time;
diff --git a/inference-engine/ie_bridges/python/tests/test_Blob.py b/inference-engine/ie_bridges/python/tests/test_Blob.py
index 7220f87cbd8adf..2353e60c61c293 100644
--- a/inference-engine/ie_bridges/python/tests/test_Blob.py
+++ b/inference-engine/ie_bridges/python/tests/test_Blob.py
@@ -36,87 +36,48 @@ def test_get_buffer():
blob = Blob(tensor_desc, array)
assert np.array_equal(blob.buffer, array)
-
-def test_write_to_buffer_fp32():
- tensor_desc = TensorDesc("FP32", [1, 3, 127, 127], "NCHW")
- array = np.zeros(shape=(1, 3, 127, 127), dtype=np.float32)
+def write_to_buffer(precision, numpy_precision):
+ tensor_desc = TensorDesc(precision, [1, 3, 127, 127], "NCHW")
+ array = np.zeros(shape=(1, 3, 127, 127), dtype=numpy_precision)
blob = Blob(tensor_desc, array)
- ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=np.float32)
+ ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=numpy_precision)
blob.buffer[:] = ones_arr
assert np.array_equal(blob.buffer, ones_arr)
+def test_write_to_buffer_fp32():
+ write_to_buffer("FP32", np.float32)
+
def test_write_to_buffer_fp64():
- tensor_desc = TensorDesc("FP64", [1, 3, 127, 127], "NCHW")
- array = np.zeros(shape=(1, 3, 127, 127), dtype=np.float64)
- blob = Blob(tensor_desc, array)
- ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=np.float64)
- blob.buffer[:] = ones_arr
- assert np.array_equal(blob.buffer, ones_arr)
+ write_to_buffer("FP64", np.float64)
-@pytest.mark.skip(reason="Need to figure out how to implement right conversion")
def test_write_to_buffer_fp16():
- tensor_desc = TensorDesc("FP16", [1, 3, 127, 127], "NCHW")
- array = np.zeros(shape=(1, 3, 127, 127), dtype=np.float16)
- blob = Blob(tensor_desc, array)
- ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=np.float16)
- blob.buffer[:] = ones_arr
- assert np.array_equal(blob.buffer, ones_arr)
+ write_to_buffer("FP16", np.float16)
def test_write_to_buffer_int8():
- tensor_desc = TensorDesc("I8", [1, 3, 127, 127], "NCHW")
- array = np.zeros(shape=(1, 3, 127, 127), dtype=np.int8)
- blob = Blob(tensor_desc, array)
- ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=np.int8)
- blob.buffer[:] = ones_arr
- assert np.array_equal(blob.buffer, ones_arr)
+ write_to_buffer("I8", np.int8)
def test_write_to_buffer_uint8():
- tensor_desc = TensorDesc("U8", [1, 3, 127, 127], "NCHW")
- array = np.zeros(shape=(1, 3, 127, 127), dtype=np.uint8)
- blob = Blob(tensor_desc, array)
- ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=np.uint8)
- blob.buffer[:] = ones_arr
- assert np.array_equal(blob.buffer, ones_arr)
+ write_to_buffer("U8", np.uint8)
def test_write_to_buffer_int32():
- tensor_desc = TensorDesc("I32", [1, 3, 127, 127], "NCHW")
- array = np.zeros(shape=(1, 3, 127, 127), dtype=np.int32)
- blob = Blob(tensor_desc, array)
- ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=np.int32)
- blob.buffer[:] = ones_arr
- assert np.array_equal(blob.buffer, ones_arr)
+ write_to_buffer("I32", np.int32)
def test_write_to_buffer_int16():
- tensor_desc = TensorDesc("I16", [1, 3, 127, 127], "NCHW")
- array = np.zeros(shape=(1, 3, 127, 127), dtype=np.int16)
- blob = Blob(tensor_desc, array)
- ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=np.int16)
- blob.buffer[:] = ones_arr
- assert np.array_equal(blob.buffer, ones_arr)
+ write_to_buffer("I16", np.int16)
def test_write_to_buffer_uint16():
- tensor_desc = TensorDesc("U16", [1, 3, 127, 127], "NCHW")
- array = np.zeros(shape=(1, 3, 127, 127), dtype=np.uint16)
- blob = Blob(tensor_desc, array)
- ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=np.uint16)
- blob.buffer[:] = ones_arr
- assert np.array_equal(blob.buffer, ones_arr)
+ write_to_buffer("U16", np.uint16)
def test_write_to_buffer_int64():
- tensor_desc = TensorDesc("I64", [1, 3, 127, 127], "NCHW")
- array = np.zeros(shape=(1, 3, 127, 127), dtype=np.int64)
- blob = Blob(tensor_desc, array)
- ones_arr = np.ones(shape=(1, 3, 127, 127), dtype=np.int64)
- blob.buffer[:] = ones_arr
- assert np.array_equal(blob.buffer, ones_arr)
+ write_to_buffer("I64", np.int64)
def test_write_numpy_scalar_int64():
diff --git a/inference-engine/ie_bridges/python/tests/test_IENetwork.py b/inference-engine/ie_bridges/python/tests/test_IENetwork.py
index e3c52497814e1f..a1192fe64e9ccf 100644
--- a/inference-engine/ie_bridges/python/tests/test_IENetwork.py
+++ b/inference-engine/ie_bridges/python/tests/test_IENetwork.py
@@ -247,3 +247,61 @@ def test_multi_out_data():
assert net.outputs["28/Reshape"].name == "28/Reshape" and net.outputs["28/Reshape"].shape == [1, 5184]
assert net.outputs["fc_out"].name == "fc_out" and net.outputs["fc_out"].shape == [1, 10]
pass
+
+def test_tensor_names():
+ model = """
+
+
+
+
+
+
+
+
+
+ 1
+ 3
+ 22
+ 22
+
+
+
+
+
+
+
+ 1
+ 3
+ 22
+ 22
+
+
+
+
+
+
+
+
+
+ """
+ ie = IECore()
+ weights = b''
+ net = ie.read_network(model=model.encode('utf-8'), weights=weights, init_from_buffer=True)
+ assert net.get_ov_name_for_tensor("relu_t") == "activation"
+ assert net.get_ov_name_for_tensor("identity_t") == "activation"
+ assert net.get_ov_name_for_tensor("input") == "in1"
+ assert net.get_ov_name_for_operation("output") == "activation"
diff --git a/inference-engine/include/cpp/ie_cnn_network.h b/inference-engine/include/cpp/ie_cnn_network.h
index 9544646a6d089d..8fc28ec41351d0 100644
--- a/inference-engine/include/cpp/ie_cnn_network.h
+++ b/inference-engine/include/cpp/ie_cnn_network.h
@@ -1,4 +1,4 @@
-// Copyright (C) 2018-2020 Intel Corporation
+// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@@ -189,6 +189,32 @@ class INFERENCE_ENGINE_API_CLASS(CNNNetwork) {
*/
void serialize(const std::string& xmlPath, const std::string& binPath = {}) const;
+ /**
+ * @brief Method maps framework tensor name to OpenVINO name
+ *
+ * @param orig_name Framework tensor name
+ *
+ * @return OpenVINO name
+ */
+ std::string getOVNameForTensor(const std::string& orig_name) const {
+ std::string ov_name;
+ CALL_STATUS_FNC(getOVNameForTensor, ov_name, orig_name);
+ return ov_name;
+ }
+
+ /**
+ * @brief Method maps framework operator name to OpenVINO name
+ *
+ * @param orig_name Framework operation name
+ *
+ * @return OpenVINO name
+ */
+ std::string getOVNameForOperation(const std::string& orig_name) const {
+ std::string ov_name;
+ CALL_STATUS_FNC(getOVNameForOperation, ov_name, orig_name);
+ return ov_name;
+ }
+
protected:
IE_SUPPRESS_DEPRECATED_START
/**
diff --git a/inference-engine/include/cpp/ie_infer_request.hpp b/inference-engine/include/cpp/ie_infer_request.hpp
index bb635b05c45c63..ea13c956948b30 100644
--- a/inference-engine/include/cpp/ie_infer_request.hpp
+++ b/inference-engine/include/cpp/ie_infer_request.hpp
@@ -241,7 +241,7 @@ class InferRequest {
auto res = actual->Wait(millis_timeout, &resp);
if (res != OK && res != RESULT_NOT_READY &&
res != INFER_NOT_STARTED && res != INFER_CANCELLED) {
- InferenceEngine::details::extract_exception(res, resp.msg);
+ THROW_IE_EXCEPTION << InferenceEngine::details::as_status << res << resp.msg;
}
return res;
}
diff --git a/inference-engine/include/gpu/gpu_params.hpp b/inference-engine/include/gpu/gpu_params.hpp
index 43d93bf8d69780..308f0af6f78963 100644
--- a/inference-engine/include/gpu/gpu_params.hpp
+++ b/inference-engine/include/gpu/gpu_params.hpp
@@ -101,7 +101,7 @@ DECLARE_GPU_PARAM_KEY(MEM_HANDLE, gpu_handle_param);
* @brief This key identifies video decoder surface handle
* in a shared memory blob parameter map
*/
-#ifdef WIN32
+#ifdef _WIN32
DECLARE_GPU_PARAM_KEY(DEV_OBJECT_HANDLE, gpu_handle_param);
#else
DECLARE_GPU_PARAM_KEY(DEV_OBJECT_HANDLE, uint32_t);
diff --git a/inference-engine/include/ie_common.h b/inference-engine/include/ie_common.h
index e9d228d6653c06..cb04cf049a2f0b 100644
--- a/inference-engine/include/ie_common.h
+++ b/inference-engine/include/ie_common.h
@@ -83,6 +83,7 @@ enum Layout : uint8_t {
// Single image layouts
CHW = 128, //!< A single image layout (e.g. for mean image)
+ HWC = 129, //!< A single image layout (e.g. for mean image)
// 2D
HW = 192, //!< HW 2D layout
@@ -113,6 +114,7 @@ inline std::ostream& operator<<(std::ostream& out, const Layout& p) {
PRINT_LAYOUT(OIHW);
PRINT_LAYOUT(C);
PRINT_LAYOUT(CHW);
+ PRINT_LAYOUT(HWC);
PRINT_LAYOUT(HW);
PRINT_LAYOUT(NC);
PRINT_LAYOUT(CN);
diff --git a/inference-engine/include/ie_compound_blob.h b/inference-engine/include/ie_compound_blob.h
index 526402b9dfd85e..b52347a7b6175e 100644
--- a/inference-engine/include/ie_compound_blob.h
+++ b/inference-engine/include/ie_compound_blob.h
@@ -289,8 +289,8 @@ class INFERENCE_ENGINE_API_CLASS(BatchedBlob) : public CompoundBlob {
* @brief Constructs a batched blob from a vector of blobs
* @details All passed blobs should meet following requirements:
* - all blobs have equal tensor descriptors,
- * - blobs layouts should be one of: NCHW, NHWC, NCDHW, NDHWC, NC, CN, C, CHW
- * - batch dimensions should be equal to 1 or not defined (C, CHW).
+ * - blobs layouts should be one of: NCHW, NHWC, NCDHW, NDHWC, NC, CN, C, CHW, HWC
+ * - batch dimensions should be equal to 1 or not defined (C, CHW, HWC).
* Resulting blob's tensor descriptor is constructed using tensor descriptors
* of passed blobs by setting batch dimension to blobs.size()
*
@@ -302,8 +302,8 @@ class INFERENCE_ENGINE_API_CLASS(BatchedBlob) : public CompoundBlob {
* @brief Constructs a batched blob from a vector of blobs
* @details All passed blobs should meet following requirements:
* - all blobs have equal tensor descriptors,
- * - blobs layouts should be one of: NCHW, NHWC, NCDHW, NDHWC, NC, CN, C, CHW
- * - batch dimensions should be equal to 1 or not defined (C, CHW).
+ * - blobs layouts should be one of: NCHW, NHWC, NCDHW, NDHWC, NC, CN, C, CHW, HWC
+ * - batch dimensions should be equal to 1 or not defined (C, CHW, HWC).
* Resulting blob's tensor descriptor is constructed using tensor descriptors
* of passed blobs by setting batch dimension to blobs.size()
*
diff --git a/inference-engine/include/ie_icnn_network.hpp b/inference-engine/include/ie_icnn_network.hpp
index 946e3044a30daf..2c6b5bea3ff2f0 100644
--- a/inference-engine/include/ie_icnn_network.hpp
+++ b/inference-engine/include/ie_icnn_network.hpp
@@ -1,4 +1,4 @@
-// Copyright (C) 2018-2020 Intel Corporation
+// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@@ -69,9 +69,11 @@ class INFERENCE_ENGINE_ICNNNETWORK_CLASS(ICNNNetwork) : public details::IRelease
*
* For single and multiple outputs networks.
*
- * This method need to be called to find output names for using them later
+ * This method need to be called to find out OpenVINO output names for using them later
* when calling InferenceEngine::InferRequest::GetBlob or InferenceEngine::InferRequest::SetBlob
*
+ * If you want to use framework names, you can use InferenceEngine::ICNNNetwork::getOVNameForTensor or
+ * InferenceEngine::ICNNNetwork::getOVNameForOperation methods to map framework names to OpenVINO names
*
* @param out Reference to the OutputsDataMap object
*/
@@ -82,9 +84,12 @@ class INFERENCE_ENGINE_ICNNNETWORK_CLASS(ICNNNetwork) : public details::IRelease
* object.
*
* For single and multiple inputs networks.
- * This method need to be called to find out input names for using them later
+ * This method need to be called to find out OpenVINO input names for using them later
* when calling InferenceEngine::InferRequest::SetBlob
*
+ * If you want to use framework names, you can use InferenceEngine::ICNNNetwork::getOVNameForTensor or
+ * InferenceEngine::ICNNNetwork::getOVNameForOperation methods to map framework names to OpenVINO names
+ *
* @param inputs Reference to InputsDataMap object.
*/
virtual void getInputsInfo(InputsDataMap& inputs) const noexcept = 0;
@@ -179,6 +184,38 @@ class INFERENCE_ENGINE_ICNNNETWORK_CLASS(ICNNNetwork) : public details::IRelease
virtual StatusCode serialize(const std::string& xmlPath, const std::string& binPath, ResponseDesc* resp) const
noexcept = 0;
+ /**
+ * @brief Methods maps framework tensor name to OpenVINO name
+ *
+ * @param ov_name OpenVINO name
+ * @param orig_name Framework tensor name
+ * @param resp Pointer to the response message that holds a description of an error if any occurred
+ *
+ * @return Status code of the operation
+ */
+ virtual StatusCode getOVNameForTensor(std::string& ov_name, const std::string& orig_name, ResponseDesc* resp) const noexcept {
+ (void) ov_name;
+ (void) orig_name;
+ (void) resp;
+ return NOT_IMPLEMENTED;
+ }
+
+ /**
+ * @brief Methods maps framework operation name to OpenVINO name
+ *
+ * @param ov_name OpenVINO name
+ * @param orig_name Framework operation name
+ * @param resp Pointer to the response message that holds a description of an error if any occurred
+ *
+ * @return Status code of the operation
+ */
+ virtual StatusCode getOVNameForOperation(std::string& ov_name, const std::string& orig_name, ResponseDesc* resp) const noexcept {
+ (void) ov_name;
+ (void) orig_name;
+ (void) resp;
+ return NOT_IMPLEMENTED;
+ }
+
/**
* @brief A virtual destructor.
*/
diff --git a/inference-engine/include/ie_input_info.hpp b/inference-engine/include/ie_input_info.hpp
index 5d6b8f8680383b..fe2d92b4e3ee13 100644
--- a/inference-engine/include/ie_input_info.hpp
+++ b/inference-engine/include/ie_input_info.hpp
@@ -70,6 +70,7 @@ class InputInfo {
* NC - for 2-dimensional,
* CHW - for 3-dimensional,
* NCHW - for 4-dimensional
+ * NCDHW - for 5-dimensional
* The default input layout might be changed preferred one using setLayout() function.
* @return The precision used for input blob creation
*/
diff --git a/inference-engine/samples/benchmark_app/README.md b/inference-engine/samples/benchmark_app/README.md
index 41f48e4735d886..3bba703c68bb71 100644
--- a/inference-engine/samples/benchmark_app/README.md
+++ b/inference-engine/samples/benchmark_app/README.md
@@ -4,6 +4,12 @@ This topic demonstrates how to use the Benchmark C++ Tool to estimate deep learn
> **NOTE:** This topic describes usage of C++ implementation of the Benchmark Tool. For the Python* implementation, refer to [Benchmark Python* Tool](../../tools/benchmark_tool/README.md).
+> **TIP**: You also can work with the Benchmark Tool inside the OpenVINO™ [Deep Learning Workbench](@ref workbench_docs_Workbench_DG_Introduction) (DL Workbench).
+> [DL Workbench](@ref workbench_docs_Workbench_DG_Introduction) is a platform built upon OpenVINO™ and provides a web-based graphical environment that enables you to optimize, fine-tune, analyze, visualize, and compare
+> performance of deep learning models on various Intel® architecture
+> configurations. In the DL Workbench, you can use most of OpenVINO™ toolkit components.
+>
+> Proceed to an [easy installation from Docker](@ref workbench_docs_Workbench_DG_Install_from_Docker_Hub) to get started.
## How It Works
@@ -43,6 +49,7 @@ The application also saves executable graph information serialized to an XML fil
## Run the Tool
+
Note that the benchmark_app usually produces optimal performance for any device out of the box.
**So in most cases you don't need to play the app options explicitly and the plain device name is enough**, for example, for CPU:
@@ -115,7 +122,7 @@ If a model has only image input(s), please provide a folder with images or a pat
If a model has some specific input(s) (not images), please prepare a binary file(s) that is filled with data of appropriate precision and provide a path to them as input.
If a model has mixed input types, input folder should contain all required files. Image inputs are filled with image files one by one. Binary inputs are filled with binary inputs one by one.
-To run the tool, you can use public or Intel's pre-trained models. To download the models, use the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the tool, you can use [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the tool with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/samples/classification_sample_async/README.md b/inference-engine/samples/classification_sample_async/README.md
index 5d9abb063350f8..32df39493566ed 100644
--- a/inference-engine/samples/classification_sample_async/README.md
+++ b/inference-engine/samples/classification_sample_async/README.md
@@ -49,7 +49,7 @@ Options:
Running the application with the empty list of options yields the usage message given above and an error message.
-To run the sample, use AlexNet and GoogLeNet or other public or pre-trained image classification models. To download the pre-trained models, use the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, use AlexNet and GoogLeNet or other public or pre-trained image classification models. You can download [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/samples/hello_classification/README.md b/inference-engine/samples/hello_classification/README.md
index f390cf9a874b8a..1244e68343a770 100644
--- a/inference-engine/samples/hello_classification/README.md
+++ b/inference-engine/samples/hello_classification/README.md
@@ -19,7 +19,7 @@ Refer to [Integrate the Inference Engine New Request API with Your Application](
## Running
-To run the sample, you can use public or pre-trained models. To download the pre-trained models, use the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/samples/hello_nv12_input_classification/README.md b/inference-engine/samples/hello_nv12_input_classification/README.md
index b80a5a86a59121..5d781bc66923c7 100644
--- a/inference-engine/samples/hello_nv12_input_classification/README.md
+++ b/inference-engine/samples/hello_nv12_input_classification/README.md
@@ -35,9 +35,7 @@ ffmpeg -i cat.jpg -pix_fmt nv12 cat.yuv
## Running
-To run the sample, you can use public or pre-trained models. To download pre-trained models, use
-the OpenVINO™ [Model Downloader](@ref omz_tools_downloader_README)
-or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the
> Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
diff --git a/inference-engine/samples/hello_reshape_ssd/README.md b/inference-engine/samples/hello_reshape_ssd/README.md
index ae14ddcc5a92c2..4392a3eafcf369 100644
--- a/inference-engine/samples/hello_reshape_ssd/README.md
+++ b/inference-engine/samples/hello_reshape_ssd/README.md
@@ -7,7 +7,7 @@ networks like SSD-VGG. The sample shows how to use [Shape Inference feature](../
## Running
-To run the sample, you can use public or pre-trained models. To download the pre-trained models, use the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/samples/object_detection_sample_ssd/README.md b/inference-engine/samples/object_detection_sample_ssd/README.md
index b0a4f4e84652f9..46849d90bfecc2 100644
--- a/inference-engine/samples/object_detection_sample_ssd/README.md
+++ b/inference-engine/samples/object_detection_sample_ssd/README.md
@@ -36,7 +36,7 @@ Options:
Running the application with the empty list of options yields the usage message given above and an error message.
-To run the sample, you can use public or pre-trained models. To download the pre-trained models, use the OpenVINO [Model Downloader](@ref omz_tools_downloader_README) or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
+To run the sample, you can use [public](@ref omz_models_public_index) or [Intel's](@ref omz_models_intel_index) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader_README).
> **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
diff --git a/inference-engine/samples/speech_sample/main.cpp b/inference-engine/samples/speech_sample/main.cpp
index b2932ece759010..dced4a7f9abbaa 100644
--- a/inference-engine/samples/speech_sample/main.cpp
+++ b/inference-engine/samples/speech_sample/main.cpp
@@ -269,7 +269,7 @@ float StdDevRelError(score_error_t error) {
}
#if !defined(__arm__) && !defined(_M_ARM) && !defined(__aarch64__) && !defined(_M_ARM64)
-#if defined(_WIN32) || defined(WIN32)
+#ifdef _WIN32
#include
#include
#else
@@ -281,7 +281,7 @@ float StdDevRelError(score_error_t error) {
inline void native_cpuid(unsigned int *eax, unsigned int *ebx,
unsigned int *ecx, unsigned int *edx) {
size_t level = *eax;
-#if defined(_WIN32) || defined(WIN32)
+#ifdef _WIN32
int regs[4] = {static_cast(*eax), static_cast(*ebx), static_cast(*ecx), static_cast(*edx)};
__cpuid(regs, level);
*eax = static_cast(regs[0]);
diff --git a/inference-engine/src/cldnn_engine/cldnn_custom_layer.cpp b/inference-engine/src/cldnn_engine/cldnn_custom_layer.cpp
index 09032abd9148e8..523187dc64b4ce 100644
--- a/inference-engine/src/cldnn_engine/cldnn_custom_layer.cpp
+++ b/inference-engine/src/cldnn_engine/cldnn_custom_layer.cpp
@@ -53,7 +53,7 @@ void CLDNNCustomLayer::ProcessKernelNode(const pugi::xml_node & node) {
CheckAndReturnError(m_kernelEntry.length() == 0, "No Kernel entry in layer: " << GetStrAttr(node.parent(), "name"));
// Handle Source nodes
- for (auto sourceNode = node.child("Source"); !sourceNode.empty(); sourceNode = sourceNode.next_sibling("Source")) {
+ FOREACH_CHILD(sourceNode, node, "Source") {
// open file
std::string filename = m_configDir + "/" + GetStrAttr(sourceNode, "filename", "");
std::ifstream inputFile(filename);
@@ -74,7 +74,7 @@ void CLDNNCustomLayer::ProcessKernelNode(const pugi::xml_node & node) {
}
// Handle Define nodes
- for (auto defineNode = node.child("Define"); !defineNode.empty(); defineNode = defineNode.next_sibling("Define")) {
+ FOREACH_CHILD(defineNode, node, "Define") {
KernelDefine kd;
kd.name = GetStrAttr(defineNode, "name", "");
CheckAndReturnError((kd.name.length() == 0), "Missing name for define node");
@@ -91,7 +91,7 @@ void CLDNNCustomLayer::ProcessKernelNode(const pugi::xml_node & node) {
void CLDNNCustomLayer::ProcessBuffersNode(const pugi::xml_node & node) {
CheckNodeTypeAndReturnError(node, "Buffers");
- for (auto tensorNode = node.child("Tensor"); !tensorNode.empty(); tensorNode = tensorNode.next_sibling("Tensor")) {
+ FOREACH_CHILD(tensorNode, node, "Tensor") {
KerenlParam kp;
kp.format = FormatFromString(GetStrAttr(tensorNode, "format", "BFYX"));
CheckAndReturnError(kp.format == cldnn::format::format_num, "Tensor node has an invalid format: " << GetStrAttr(tensorNode, "format"));
@@ -109,7 +109,7 @@ void CLDNNCustomLayer::ProcessBuffersNode(const pugi::xml_node & node) {
}
m_kernelParams.push_back(kp);
}
- for (auto dataNode = node.child("Data"); !dataNode.empty(); dataNode = dataNode.next_sibling("Data")) {
+ FOREACH_CHILD(dataNode, node, "Data") {
KerenlParam kp;
kp.type = ParamType::Data;
kp.paramIndex = GetIntAttr(dataNode, "arg-index", -1);
diff --git a/inference-engine/src/cldnn_engine/cldnn_engine.cpp b/inference-engine/src/cldnn_engine/cldnn_engine.cpp
index c39cdc20433f45..f82e866630ddb8 100644
--- a/inference-engine/src/cldnn_engine/cldnn_engine.cpp
+++ b/inference-engine/src/cldnn_engine/cldnn_engine.cpp
@@ -1,4 +1,4 @@
-// Copyright (C) 2018-2020 Intel Corporation
+// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@@ -30,6 +30,7 @@
#include
#include
+#include "transformations/common_optimizations/convert_quantize_dequantize.hpp"
#include
#include
#include
@@ -47,6 +48,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -60,9 +62,11 @@
#include
#include
+#include
#include
#include
#include
+#include
#include "cldnn_engine.h"
#include "cldnn_executable_network.h"
@@ -135,8 +139,15 @@ InferenceEngine::CNNNetwork clDNNEngine::CloneAndTransformNetwork(const Inferenc
// Disable shape inference (WA for generic operations)
ngraph::op::GenericIE::DisableReshape noReshape(nGraphFunc);
+ bool enableInt8;
{
ngraph::pass::Manager manager;
+ enableInt8 = config.enableInt8 && ngraph::pass::low_precision::LowPrecisionTransformer::isFunctionQuantized(nGraphFunc);
+ if (enableInt8) {
+ manager.register_pass(
+ std::vector{ ngraph::element::i8, ngraph::element::u8 });
+ }
+
manager.register_pass();
manager.register_pass();
manager.register_pass();
@@ -271,10 +282,19 @@ InferenceEngine::CNNNetwork clDNNEngine::CloneAndTransformNetwork(const Inferenc
pass_config->enable();
+ if (enableInt8) {
+ pass_config->set_callback([](const_node_ptr &node) -> bool {
+ return ngraph::pass::low_precision::NetworkHelper::areQuantizeAndDequantizeSupportedForMultiply(node);
+ });
+
+ pass_config->set_callback([](const_node_ptr &node) -> bool {
+ return ngraph::pass::low_precision::NetworkHelper::areQuantizeAndDequantizeSupportedForSubtract(node);
+ });
+ }
+
manager.run_passes(nGraphFunc);
}
- bool enableInt8 = config.enableInt8 && ngraph::pass::low_precision::LowPrecisionTransformer::isFunctionQuantized(nGraphFunc);
if (enableInt8) {
OV_ITT_SCOPED_TASK(itt::domains::CLDNNPlugin, "clDNNEngine::TransformNetwork::LPT");
using namespace ngraph::pass::low_precision;
@@ -445,11 +465,11 @@ RemoteContext::Ptr clDNNEngine::CreateContext(const ParamMap& params) {
auto context = std::make_shared(shared_from_this(), params, _impl->m_config);
return std::dynamic_pointer_cast(context);
} else if (GPU_PARAM_VALUE(VA_SHARED) == contextTypeStr) {
- #ifdef WIN32
+#ifdef _WIN32
auto context = std::make_shared(shared_from_this(), params, _impl->m_config);
- #else
+#else
auto context = std::make_shared(shared_from_this(), params, _impl->m_config);
- #endif
+#endif
return std::dynamic_pointer_cast(context);
} else {
THROW_IE_EXCEPTION << "Invalid remote context type" << contextTypeStr;
diff --git a/inference-engine/src/cldnn_engine/cldnn_graph.cpp b/inference-engine/src/cldnn_engine/cldnn_graph.cpp
index bff05e90077ff3..340853cf44e5d0 100644
--- a/inference-engine/src/cldnn_engine/cldnn_graph.cpp
+++ b/inference-engine/src/cldnn_engine/cldnn_graph.cpp
@@ -591,8 +591,9 @@ void CLDNNGraph::UpdateImplementationsMap() {
}
}
-void CLDNNGraph::GetPerformanceCounts(std::map &result) const {
+std::map CLDNNGraph::GetPerformanceCounts() const {
OV_ITT_SCOPED_TASK(itt::domains::CLDNNPlugin, "CLDNNGraph::GetPerformanceCounts");
+ std::map result;
bool combinePrimByIRLayers = false;
unsigned i = 0;
auto allIds = GetNetwork()->get_all_primitive_org_ids();
@@ -738,6 +739,7 @@ void CLDNNGraph::GetPerformanceCounts(std::map CLDNNGraph::GetNetwork(size_t idx) const {
diff --git a/inference-engine/src/cldnn_engine/cldnn_graph.h b/inference-engine/src/cldnn_engine/cldnn_graph.h
index 40dc4bab68b971..45a4510843defc 100644
--- a/inference-engine/src/cldnn_engine/cldnn_graph.h
+++ b/inference-engine/src/cldnn_engine/cldnn_graph.h
@@ -38,7 +38,7 @@ class CLDNNGraph {
bool IsLoaded() const;
- void GetPerformanceCounts(std::map& perfMap) const;
+ std::map GetPerformanceCounts() const;
void UpdatePerfStatistics();
const Config& getConfig() const { return m_config; }
diff --git a/inference-engine/src/cldnn_engine/cldnn_infer_request.cpp b/inference-engine/src/cldnn_engine/cldnn_infer_request.cpp
index 0ce5dfbcb47b3f..2e7b1b00d9a1dd 100644
--- a/inference-engine/src/cldnn_engine/cldnn_infer_request.cpp
+++ b/inference-engine/src/cldnn_engine/cldnn_infer_request.cpp
@@ -421,8 +421,9 @@ void CLDNNInferRequest::checkBlobs() {
}
}
-void CLDNNInferRequest::GetBlob(const char *name, Blob::Ptr &data) {
+Blob::Ptr CLDNNInferRequest::GetBlob(const std::string& name) {
OV_ITT_SCOPED_TASK(itt::domains::CLDNNPlugin, "CLDNNInferRequest::GetBlob");
+ Blob::Ptr data;
InputInfo::Ptr foundInput;
DataPtr foundOutput;
bool is_input = findInputAndOutputBlobByName(name, foundInput, foundOutput);
@@ -440,13 +441,14 @@ void CLDNNInferRequest::GetBlob(const char *name, Blob::Ptr &data) {
data = _outputs[name];
checkOutputBlob(data, name, foundOutput);
}
+ return data;
}
-void CLDNNInferRequest::SetBlob(const char *name, const Blob::Ptr &data) {
+void CLDNNInferRequest::SetBlob(const std::string& name, const Blob::Ptr &data) {
OV_ITT_SCOPED_TASK(itt::domains::CLDNNPlugin, "CLDNNInferRequest::SetBlob");
// perform all common checks first
- if (name == nullptr) {
+ if (name.empty()) {
THROW_IE_EXCEPTION << NOT_FOUND_str + "Failed to set blob with empty name";
}
if (!data)
@@ -884,13 +886,12 @@ void CLDNNInferRequest::InferImpl() {
}
}
-void CLDNNInferRequest::GetPerformanceCounts(
- std::map &perfMap) const {
+std::map CLDNNInferRequest::GetPerformanceCounts() const {
OV_ITT_SCOPED_TASK(itt::domains::CLDNNPlugin, "CLDNNInferRequest::GetPerformanceCounts");
if (!m_useProfiling) {
THROW_IE_EXCEPTION << "Performance counters were not enabled";
} else {
- m_graph->GetPerformanceCounts(perfMap);
+ return m_graph->GetPerformanceCounts();
}
}
diff --git a/inference-engine/src/cldnn_engine/cldnn_infer_request.h b/inference-engine/src/cldnn_engine/cldnn_infer_request.h
index 739be8594dd48f..a557c9b94af303 100644
--- a/inference-engine/src/cldnn_engine/cldnn_infer_request.h
+++ b/inference-engine/src/cldnn_engine/cldnn_infer_request.h
@@ -29,7 +29,7 @@ class CLDNNInferRequest : public InferenceEngine::InferRequestInternal {
void checkBlobs() override;
void InferImpl() override;
- void GetPerformanceCounts(std::map &perfMap) const override;
+ std::map GetPerformanceCounts() const override;
CLDNNInferRequest(InferenceEngine::InputsDataMap networkInputs, InferenceEngine::OutputsDataMap networkOutputs,
const std::shared_ptr& execNetwork);
@@ -38,8 +38,8 @@ class CLDNNInferRequest : public InferenceEngine::InferRequestInternal {
virtual ~CLDNNInferRequest() = default;
- void GetBlob(const char *name, InferenceEngine::Blob::Ptr &data) override;
- void SetBlob(const char *name, const InferenceEngine::Blob::Ptr &data) override;
+ InferenceEngine::Blob::Ptr GetBlob(const std::string& name) override;
+ void SetBlob(const std::string& name, const InferenceEngine::Blob::Ptr &data) override;
void SetBatch(int batch = -1) override;
void SetGraph(std::shared_ptr graph);
diff --git a/inference-engine/src/cldnn_engine/cldnn_primitives_list.hpp b/inference-engine/src/cldnn_engine/cldnn_primitives_list.hpp
index 8ce7c0ebe86534..c3fb7cad324eb0 100644
--- a/inference-engine/src/cldnn_engine/cldnn_primitives_list.hpp
+++ b/inference-engine/src/cldnn_engine/cldnn_primitives_list.hpp
@@ -1,4 +1,4 @@
-// Copyright (C) 2020 Intel Corporation
+// Copyright (C) 2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@@ -156,18 +156,17 @@ REGISTER_FACTORY(v3, EmbeddingBagOffsetsSum);
REGISTER_FACTORY(v3, EmbeddingBagPackedSum);
REGISTER_FACTORY(v3, EmbeddingSegmentsSum);
REGISTER_FACTORY(v3, ExtractImagePatches);
+REGISTER_FACTORY(v3, ScatterUpdate);
+REGISTER_FACTORY(v3, ScatterElementsUpdate);
// REGISTER_FACTORY(v3, NonMaxSuppression); Supported via v3 -> v5 internal conversion
// ----------------------------- Unsupported v3 ops ----------------------------- //
-// REGISTER_FACTORY(v3, ScatterUpdate); // There is the scatter_update primitive, but seems like it produces wrong results
// REGISTER_FACTORY(v3, Assign);
// REGISTER_FACTORY(v3, Bucketize);
// REGISTER_FACTORY(v3, GRUCell);
// REGISTER_FACTORY(v3, NonZero);
// REGISTER_FACTORY(v3, ROIAlign);
// REGISTER_FACTORY(v3, ReadValue);
-// REGISTER_FACTORY(v3, ScatterElementsUpdate);
-// REGISTER_FACTORY(v3, ScatterUpdate);
// REGISTER_FACTORY(v3, ScatterNDUpdate);
// REGISTER_FACTORY(v3, ShapeOf);
// REGISTER_FACTORY(v3, TopK);
diff --git a/inference-engine/src/cldnn_engine/cldnn_remote_context.cpp b/inference-engine/src/cldnn_engine/cldnn_remote_context.cpp
index 66d0949fb81940..6cc88ee77df85f 100644
--- a/inference-engine/src/cldnn_engine/cldnn_remote_context.cpp
+++ b/inference-engine/src/cldnn_engine/cldnn_remote_context.cpp
@@ -36,7 +36,7 @@ ParamMap CLDNNRemoteBlobImpl::getParams() const {
{ GPU_PARAM_KEY(OCL_CONTEXT), params.context },
{ GPU_PARAM_KEY(MEM_HANDLE), params.mem }
};
-#ifdef WIN32
+#ifdef _WIN32
case BT_DX_BUF_SHARED:
return{
{ GPU_PARAM_KEY(SHARED_MEM_TYPE), GPU_PARAM_VALUE(DX_BUFFER) },
@@ -94,7 +94,7 @@ void CLDNNRemoteBlobImpl::allocate_if_needed() {
case BlobType::BT_BUF_SHARED:
m_memObject = std::unique_ptr(new cldnn::memory(cldnn::memory::share_buffer(*eng, m_layout, m_mem)));
break;
-#ifdef WIN32
+#ifdef _WIN32
case BlobType::BT_SURF_SHARED:
m_memObject = std::unique_ptr(new cldnn::memory(cldnn::memory::share_surface(*eng, m_layout, m_mem, m_plane)));
break;
@@ -130,7 +130,7 @@ void CLDNNRemoteBlobImpl::allocate() noexcept {
case BlobType::BT_BUF_SHARED:
m_memObject = std::unique_ptr(new cldnn::memory(cldnn::memory::share_buffer(*eng, m_layout, m_mem)));
break;
-#ifdef WIN32
+#ifdef _WIN32
case BlobType::BT_SURF_SHARED:
m_memObject = std::unique_ptr(new cldnn::memory(cldnn::memory::share_surface(*eng, m_layout, m_mem, m_plane)));
break;
diff --git a/inference-engine/src/cldnn_engine/cldnn_remote_context.h b/inference-engine/src/cldnn_engine/cldnn_remote_context.h
index 0b52527a244829..a539e2c6b50a84 100644
--- a/inference-engine/src/cldnn_engine/cldnn_remote_context.h
+++ b/inference-engine/src/cldnn_engine/cldnn_remote_context.h
@@ -19,10 +19,10 @@
# define NOMINMAX
#endif
-#ifdef WIN32
-#include
+#ifdef _WIN32
+# include
#else
-#include
+# include
#endif
namespace CLDNNPlugin {
@@ -122,7 +122,7 @@ class typedCLDNNRemoteBlob : public TpublicAPI {
using CLDNNRemoteCLbuffer = typedCLDNNRemoteBlob;
using CLDNNRemoteCLImage2D = typedCLDNNRemoteBlob;
-#ifdef WIN32
+#ifdef _WIN32
using CLDNNRemoteD3DBuffer = typedCLDNNRemoteBlob;
using CLDNNRemoteD3DSurface = typedCLDNNRemoteBlob;
#else
@@ -130,7 +130,7 @@ using CLDNNRemoteVASurface = typedCLDNNRemoteBlobas();
if (ptr) return ptr->getImpl();
@@ -257,7 +257,7 @@ class typedCLDNNExecutionContext : public TpublicContextAPI,
}
};
-#ifdef WIN32
+#ifdef _WIN32
using surf_key = _Key;
#else
using surf_key = _Key;
@@ -270,7 +270,7 @@ class typedCLDNNExecutionContext : public TpublicContextAPI,
using InferenceEngine::gpu::details::param_map_obj_getter;
InferenceEngine::RemoteBlob::Ptr ret = nullptr;
uint32_t plane = param_map_obj_getter::_ObjFromParamSimple(params, GPU_PARAM_KEY(VA_PLANE));
-#ifdef WIN32
+#ifdef _WIN32
cldnn::shared_handle mem = param_map_obj_getter::_ObjFromParamSimple(params, GPU_PARAM_KEY(DEV_OBJECT_HANDLE));
surf_key skey(mem, plane);
#else
@@ -291,7 +291,7 @@ class typedCLDNNExecutionContext : public TpublicContextAPI,
auto smart_this =
std::dynamic_pointer_cast
(std::enable_shared_from_this>::shared_from_this());
-#ifdef WIN32
+#ifdef _WIN32
ret = std::make_shared(smart_this,
tensorDesc, layout, mem, 0, plane,
CLDNNRemoteBlobImpl::BlobType::BT_SURF_SHARED);
@@ -335,7 +335,7 @@ class typedCLDNNExecutionContext : public TpublicContextAPI,
layout.format = ImageFormatFromLayout(tensorDesc.getLayout());
ret = std::make_shared(smart_this, tensorDesc, layout, mem, 0, 0, blob_type);
break;
-#ifdef WIN32
+#ifdef _WIN32
case CLDNNRemoteBlobImpl::BlobType::BT_DX_BUF_SHARED:
ret = std::make_shared(smart_this, tensorDesc, layout, mem, 0, 0, blob_type);
break;
@@ -402,7 +402,7 @@ class typedCLDNNExecutionContext : public TpublicContextAPI,
} else if (GPU_PARAM_VALUE(OCL_IMAGE2D) == memTypeStr) {
blob_type = CLDNNRemoteBlobImpl::BlobType::BT_IMG_SHARED;
mem = param_map_obj_getter::_ObjFromParamSimple(params, GPU_PARAM_KEY(MEM_HANDLE));
-#ifdef WIN32
+#ifdef _WIN32
} else if (GPU_PARAM_VALUE(DX_BUFFER) == memTypeStr) {
blob_type = CLDNNRemoteBlobImpl::BlobType::BT_DX_BUF_SHARED;
mem = param_map_obj_getter::_ObjFromParamSimple(params, GPU_PARAM_KEY(DEV_OBJECT_HANDLE));
@@ -426,14 +426,14 @@ class typedCLDNNExecutionContext : public TpublicContextAPI,
};
using CLDNNRemoteCLContext = typedCLDNNExecutionContext;
-#ifdef WIN32
+#ifdef _WIN32
using CLDNNRemoteD3DContext = typedCLDNNExecutionContext;
#else
using CLDNNRemoteVAContext = typedCLDNNExecutionContext;
#endif
inline CLDNNExecutionContextImpl* getContextImpl(InferenceEngine::gpu::ClContext::Ptr ctxPtr) {
-#ifdef WIN32
+#ifdef _WIN32
{
auto ptr = ctxPtr->as();
if (ptr) return ptr->getImpl();
diff --git a/inference-engine/src/cldnn_engine/ops/result.cpp b/inference-engine/src/cldnn_engine/ops/result.cpp
index 56ad5e9f5c017a..536caf22eb7555 100644
--- a/inference-engine/src/cldnn_engine/ops/result.cpp
+++ b/inference-engine/src/cldnn_engine/ops/result.cpp
@@ -18,7 +18,9 @@ void CreateResultOp(Program& p, const std::shared_ptr& o
p.ValidateInputs(op, {1});
auto prev = op->get_input_node_shared_ptr(0);
+ NGRAPH_SUPPRESS_DEPRECATED_START
auto inputID = op->get_input_source_output(0).get_tensor().get_name();
+ NGRAPH_SUPPRESS_DEPRECATED_END
if (inputID.empty()) {
inputID = prev->get_friendly_name();
if (prev->get_output_size() > 1) {
diff --git a/inference-engine/src/cldnn_engine/ops/scatter_elements_update.cpp b/inference-engine/src/cldnn_engine/ops/scatter_elements_update.cpp
new file mode 100644
index 00000000000000..5c9002c64aaa44
--- /dev/null
+++ b/inference-engine/src/cldnn_engine/ops/scatter_elements_update.cpp
@@ -0,0 +1,68 @@
+// Copyright (C) 2021 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#include "cldnn_program.h"
+#include "cldnn_common_utils.h"
+
+#include "ngraph/op/scatter_elements_update.hpp"
+#include "ngraph/op/constant.hpp"
+
+#include "api/scatter_elements_update.hpp"
+
+namespace CLDNNPlugin {
+
+static inline cldnn::scatter_elements_update::scatter_elements_update_axis GetScatterElementsUpdateAxis(int axis, unsigned rank) {
+ if (axis < 0)
+ axis += rank;
+ if (axis < 0 || axis >= rank)
+ THROW_IE_EXCEPTION << "ScatterElementsUpdate axis is not correspond to number of dimensions";
+
+ // Difference in dimension ordering between IE and clDNN,
+ // reverse spatial dimensions after batch and feature.
+ unsigned cldnn_axis = axis;
+ if (axis >= 2) {
+ auto spatial_axis = axis - 2;
+ // Default and minimum number of dimensions is 4
+ auto spatial_size = std::max(rank, 4u) - 2;
+ cldnn_axis = spatial_size - spatial_axis - 1 + 2;
+ }
+
+ switch (cldnn_axis) {
+ case 0: return cldnn::scatter_elements_update::scatter_elements_update_axis::along_b;
+ case 1: return cldnn::scatter_elements_update::scatter_elements_update_axis::along_f;
+ case 2: return cldnn::scatter_elements_update::scatter_elements_update_axis::along_x;
+ case 3: return cldnn::scatter_elements_update::scatter_elements_update_axis::along_y;
+ case 4: return cldnn::scatter_elements_update::scatter_elements_update_axis::along_z;
+ case 5: return cldnn::scatter_elements_update::scatter_elements_update_axis::along_w;
+ default: THROW_IE_EXCEPTION << "Unsupported ScatterElementsUpdate axis: " << axis;
+ }
+
+ return cldnn::scatter_elements_update::scatter_elements_update_axis::along_f; // shouldn't get here
+}
+
+void CreateScatterElementsUpdateOp(Program& p, const std::shared_ptr& op) {
+ p.ValidateInputs(op, {4});
+ auto inputPrimitives = p.GetInputPrimitiveIDs(op);
+ std::string layerName = layer_type_name_ID(op);
+
+ size_t rank = op->get_input_shape(0).size();
+ auto axes_constant = std::dynamic_pointer_cast(op->get_input_node_shared_ptr(3));
+ if (!axes_constant) {
+ THROW_IE_EXCEPTION << "Unsupported parameter nodes type in " << op->get_friendly_name() << " (" << op->get_type_name() << ")";
+ }
+ int32_t axis = axes_constant->cast_vector()[0];
+
+ auto primitive = cldnn::scatter_elements_update(layerName,
+ inputPrimitives[0],
+ inputPrimitives[1],
+ inputPrimitives[2],
+ GetScatterElementsUpdateAxis(axis, rank));
+
+ p.AddPrimitive(primitive);
+ p.AddPrimitiveToProfiler(op);
+}
+
+REGISTER_FACTORY_IMPL(v3, ScatterElementsUpdate);
+
+} // namespace CLDNNPlugin
diff --git a/inference-engine/src/cldnn_engine/ops/split.cpp b/inference-engine/src/cldnn_engine/ops/split.cpp
index 65cbf59873b831..3639a3c583a2e5 100644
--- a/inference-engine/src/cldnn_engine/ops/split.cpp
+++ b/inference-engine/src/cldnn_engine/ops/split.cpp
@@ -24,6 +24,7 @@ void CreateCommonSplitOp(Program& p, const std::shared_ptr& op) {
for (size_t i = 0; i < op->get_output_size(); i++) {
std::string outLayerName = layerName + (is_single_out_split ? "" : "." + std::to_string(i));
const auto outLayerDims = op->get_output_shape(i);
+ NGRAPH_SUPPRESS_DEPRECATED_START
if (outLayerDims.size() != startOffset.size()) {
THROW_IE_EXCEPTION << "Invalid dimesions in split layer: " << op->get_friendly_name()
<< " output: " << op->get_output_tensor_name(i);
@@ -34,6 +35,7 @@ void CreateCommonSplitOp(Program& p, const std::shared_ptr& op) {
<< " output: " << op->get_output_tensor_name(i);
}
}
+ NGRAPH_SUPPRESS_DEPRECATED_END
auto outTensor = CldnnTensorFromIEDims(outLayerDims, 1);
auto offsetTensor = CldnnTensorFromIEDims(startOffset, 0);
diff --git a/inference-engine/src/gna_plugin/gna_infer_request.hpp b/inference-engine/src/gna_plugin/gna_infer_request.hpp
index 52b9c0fdd24e49..9b04e759d813d4 100644
--- a/inference-engine/src/gna_plugin/gna_infer_request.hpp
+++ b/inference-engine/src/gna_plugin/gna_infer_request.hpp
@@ -67,9 +67,8 @@ class GNAInferRequest : public InferenceEngine::AsyncInferRequestInternal {
* Note: not all plugins may provide meaningful data
* @param perfMap - a map of layer names to profiling information for that layer.
*/
- void GetPerformanceCounts(std::map &perfMap) const override {
- plg->GetPerformanceCounts(perfMap);
+ std::map GetPerformanceCounts() const override {
+ return plg->GetPerformanceCounts();
}
/**
diff --git a/inference-engine/src/gna_plugin/gna_plugin.cpp b/inference-engine/src/gna_plugin/gna_plugin.cpp
index e500722d5e6669..c8e3c225cfb8dd 100644
--- a/inference-engine/src/gna_plugin/gna_plugin.cpp
+++ b/inference-engine/src/gna_plugin/gna_plugin.cpp
@@ -1451,9 +1451,13 @@ void GNAPlugin::Export(const std::string &fileName) {
serial.Export(gnamem->getBasePtr(), gnamem->getTotalBytes(), outStream);
}
-void GNAPlugin::GetPerformanceCounts(std::map &perfMap) {
+std::map GNAPlugin::GetPerformanceCounts() {
if (gnaFlags->performance_counting) {
+ std::map perfMap;
gnadevice->getGnaPerfCounters(perfMap);
+ return perfMap;
+ } else {
+ return {};
}
}
diff --git a/inference-engine/src/gna_plugin/gna_plugin.hpp b/inference-engine/src/gna_plugin/gna_plugin.hpp
index 6c5acd33eaef49..6441e8e5f5e762 100644
--- a/inference-engine/src/gna_plugin/gna_plugin.hpp
+++ b/inference-engine/src/gna_plugin/gna_plugin.hpp
@@ -102,7 +102,7 @@ class GNAPlugin : public InferenceEngine::IInferencePlugin {
void LoadNetwork(InferenceEngine::CNNNetwork &network);
bool Infer(const InferenceEngine::BlobMap &input, InferenceEngine::BlobMap &result);
- void GetPerformanceCounts(std::map &perfMap);
+ std::map GetPerformanceCounts();
void AddExtension(InferenceEngine::IExtensionPtr extension) override;
void SetConfig(const std::map &config) override;
diff --git a/inference-engine/src/hetero_plugin/hetero_executable_network.cpp b/inference-engine/src/hetero_plugin/hetero_executable_network.cpp
index 7c5555e4cb78db..6241118511a7ce 100644
--- a/inference-engine/src/hetero_plugin/hetero_executable_network.cpp
+++ b/inference-engine/src/hetero_plugin/hetero_executable_network.cpp
@@ -383,6 +383,7 @@ HeteroExecutableNetwork::HeteroExecutableNetwork(const InferenceEngine::CNNNetwo
if (itClonedInput != clonedInputs.end() && nullptr != itClonedInput->second) {
itClonedInput->second->getPreProcess() = externalInput.second->getPreProcess();
itClonedInput->second->setPrecision(externalInput.second->getPrecision());
+ itClonedInput->second->setLayout(externalInput.second->getLayout());
}
}
isInputSubnetwork[id] = std::any_of(std::begin(subgraph._parameters),
@@ -440,28 +441,24 @@ HeteroExecutableNetwork::HeteroExecutableNetwork(std::istream&
std::unordered_set networkInputs;
pugi::xml_node inputsNode = heteroNode.child("inputs");
- for (auto inputNode = inputsNode.child("input"); !inputNode.empty();
- inputNode = inputNode.next_sibling("input")) {
+ FOREACH_CHILD(inputNode, inputsNode, "input") {
networkInputs.insert(GetStrAttr(inputNode, "name"));
}
std::unordered_set networkOutputs;
pugi::xml_node outputsNode = heteroNode.child("outputs");
- for (auto outputNode = outputsNode.child("output"); !outputNode.empty();
- outputNode = outputNode.next_sibling("output")) {
+ FOREACH_CHILD(outputNode, outputsNode, "output") {
networkOutputs.insert(GetStrAttr(outputNode, "name"));
}
Engine::Configs importedConfigs;
auto configsNode = heteroNode.child("configs");
- for (auto configNode = configsNode.child("config"); !configNode.empty();
- configNode = configNode.next_sibling("config")) {
+ FOREACH_CHILD(configNode, configsNode, "config") {
importedConfigs.emplace(GetStrAttr(configNode, "key"), GetStrAttr(configNode, "value"));
}
auto blobNamesNode = heteroNode.child("blob_names_map");
- for (auto blobNameNode = blobNamesNode.child("blob_name_map"); !blobNameNode.empty();
- blobNameNode = blobNameNode.next_sibling("blob_name_map")) {
+ FOREACH_CHILD(blobNameNode, blobNamesNode, "blob_name_map") {
_blobNameMap.emplace(GetStrAttr(blobNameNode, "key"), GetStrAttr(blobNameNode, "value"));
}
@@ -471,8 +468,7 @@ HeteroExecutableNetwork::HeteroExecutableNetwork(std::istream&
std::vector descs;
pugi::xml_node subnetworksNode = heteroNode.child("subnetworks");
- for (auto subnetworkNode = subnetworksNode.child("subnetwork"); !subnetworkNode.empty();
- subnetworkNode = subnetworkNode.next_sibling("subnetwork")) {
+ FOREACH_CHILD(subnetworkNode, subnetworksNode, "subnetwork") {
auto deviceName = GetStrAttr(subnetworkNode, "device");
auto metaDevices = _heteroPlugin->GetDevicePlugins(deviceName, importedConfigs);
@@ -507,16 +503,14 @@ HeteroExecutableNetwork::HeteroExecutableNetwork(std::istream&
cnnnetwork = _heteroPlugin->GetCore()->ReadNetwork(xmlString, std::move(dataBlob));
auto inputs = cnnnetwork.getInputsInfo();
auto inputsNode = subnetworkNode.child("inputs");
- for (auto inputNode = inputsNode.child("input"); !inputNode.empty();
- inputNode = inputNode.next_sibling("input")) {
+ FOREACH_CHILD(inputNode, inputsNode, "input") {
auto inputName = GetStrAttr(inputNode, "name");
inputs[inputName]->setPrecision(Precision::FromStr(GetStrAttr(inputNode, "precision")));
}
auto outputs = cnnnetwork.getOutputsInfo();
auto outputsNode = subnetworkNode.child("outputs");
- for (auto outputNode = outputsNode.child("output"); !outputNode.empty();
- outputNode = outputNode.next_sibling("output")) {
+ FOREACH_CHILD(outputNode, outputsNode, "output") {
auto outputName = GetStrAttr(outputNode, "name");
outputs[outputName]->setPrecision(Precision::FromStr(GetStrAttr(outputNode, "precision")));
}
diff --git a/inference-engine/src/hetero_plugin/hetero_infer_request.cpp b/inference-engine/src/hetero_plugin/hetero_infer_request.cpp
index 94be11eb594ece..9d831f7703082f 100644
--- a/inference-engine/src/hetero_plugin/hetero_infer_request.cpp
+++ b/inference-engine/src/hetero_plugin/hetero_infer_request.cpp
@@ -64,7 +64,7 @@ HeteroInferRequest::HeteroInferRequest(InferenceEngine::InputsDataMap networkInp
}
}
-void HeteroInferRequest::SetBlob(const char* name, const InferenceEngine::Blob::Ptr& data) {
+void HeteroInferRequest::SetBlob(const std::string& name, const InferenceEngine::Blob::Ptr& data) {
InferenceEngine::InferRequestInternal::SetBlob(name, data);
assert(!_inferRequests.empty());
for (auto &&desc : _inferRequests) {
@@ -95,14 +95,15 @@ void HeteroInferRequest::InferImpl() {
}
}
-void HeteroInferRequest::GetPerformanceCounts(std::map &perfMap) const {
- perfMap.clear();
+std::map HeteroInferRequest::GetPerformanceCounts() const {
+ std::map perfMap;
for (size_t i = 0; i < _inferRequests.size(); i++) {
auto perfMapRequest = _inferRequests[i]._request->GetPerformanceCounts();
for (auto &&r : perfMapRequest) {
perfMap[std::string("subgraph") + std::to_string(i) + ": " + r.first] = r.second;
}
}
+ return perfMap;
}
void HeteroInferRequest::updateInOutIfNeeded() {
diff --git a/inference-engine/src/hetero_plugin/hetero_infer_request.hpp b/inference-engine/src/hetero_plugin/hetero_infer_request.hpp
index 069e36339151d2..aee93ced131e21 100644
--- a/inference-engine/src/hetero_plugin/hetero_infer_request.hpp
+++ b/inference-engine/src/hetero_plugin/hetero_infer_request.hpp
@@ -40,9 +40,9 @@ class HeteroInferRequest : public InferenceEngine::InferRequestInternal {
void InferImpl() override;
- void SetBlob(const char* name, const InferenceEngine::Blob::Ptr& data) override;
+ void SetBlob(const std::string& name, const InferenceEngine::Blob::Ptr& data) override;
- void GetPerformanceCounts(std::map &perfMap) const override;
+ std::map GetPerformanceCounts() const override;
void updateInOutIfNeeded();
diff --git a/inference-engine/src/inference_engine/cnn_network_ngraph_impl.cpp b/inference-engine/src/inference_engine/cnn_network_ngraph_impl.cpp
index 2355a5674ab20c..02f8d2fc3fd8e8 100644
--- a/inference-engine/src/inference_engine/cnn_network_ngraph_impl.cpp
+++ b/inference-engine/src/inference_engine/cnn_network_ngraph_impl.cpp
@@ -1,4 +1,4 @@
-// Copyright (C) 2018-2020 Intel Corporation
+// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@@ -65,7 +65,7 @@ void CNNNetworkNGraphImpl::createDataForResult(const ::ngraph::Output<::ngraph::
case 2:
return l == Layout::CN || l == Layout::HW || l == Layout::NC;
case 3:
- return l == Layout::CHW;
+ return l == Layout::CHW || l == Layout::HWC;
case 4:
return l == Layout::NCHW || l == Layout::NHWC;
case 5:
@@ -122,6 +122,12 @@ CNNNetworkNGraphImpl::CNNNetworkNGraphImpl(
std::string outName = layer->get_friendly_name();
IE_ASSERT(layer->get_output_size() == 1); // Parameter as only singly output port
+ // map original names to OpenVINO name
+ _opNames[outName] = outName;
+ for (const auto& name : layer->get_output_tensor(0).get_names()) {
+ _tensorNames[name] = outName;
+ }
+
DataPtr& ptr = _data[outName];
IE_ASSERT(ptr); // Data must be allocated after the reshape method
@@ -139,7 +145,10 @@ CNNNetworkNGraphImpl::CNNNetworkNGraphImpl(
}
CNNNetworkNGraphImpl::CNNNetworkNGraphImpl(const CNNNetwork& network) {
- if (network.getFunction() == nullptr) {
+ IE_SUPPRESS_DEPRECATED_START
+ const ICNNNetwork& iNetwork = network;
+ const auto net = dynamic_cast(&iNetwork);
+ if (network.getFunction() == nullptr || !net) {
THROW_IE_EXCEPTION << "Cannot create CNNNetwork with nGraph from legacy network format!";
}
@@ -147,6 +156,9 @@ CNNNetworkNGraphImpl::CNNNetworkNGraphImpl(const CNNNetwork& network) {
InputsDataMap inputs = network.getInputsInfo();
OutputsDataMap outputs = network.getOutputsInfo();
+ _opNames = net->_opNames;
+ _tensorNames = net->_tensorNames;
+
for (const auto& outputInfo : outputs) {
const auto& name = outputInfo.second->getName();
DataPtr output = std::make_shared(name, outputInfo.second->getTensorDesc());
@@ -164,6 +176,7 @@ CNNNetworkNGraphImpl::CNNNetworkNGraphImpl(const CNNNetwork& network) {
info->setLayout(inputInfo.second->getLayout());
_inputData[name] = info;
}
+ IE_SUPPRESS_DEPRECATED_END
}
void CNNNetworkNGraphImpl::setInputInfo(InputInfo::Ptr data) {
@@ -204,19 +217,22 @@ StatusCode CNNNetworkNGraphImpl::addOutput(const std::string& layerName, size_t
try {
for (const auto & layer : _ngraph_function->get_ops()) {
- if (layer->get_friendly_name() == layerName) {
+ // Result can have the same name as previous operation
+ if (layer->get_friendly_name() == layerName && !std::dynamic_pointer_cast(layer)) {
+ std::string outputName = layerName;
+ if (layer->outputs().size() != 1) {
+ outputName += "." + std::to_string(outputIndex);
+ }
+
// Check that we don't have a result for the output port
for (const auto& port : layer->output(outputIndex).get_target_inputs()) {
if (dynamic_cast(port.get_node()))
return OK;
}
auto result = make_shared<::ngraph::op::Result>(layer->output(outputIndex));
+ result->set_friendly_name(outputName);
_ngraph_function->add_results({result});
- std::string outputName = layerName;
- if (layer->outputs().size() != 1) {
- outputName += "." + std::to_string(outputIndex);
- }
if (_outputData.count(outputName) == 0) {
reshape();
}
@@ -237,6 +253,17 @@ void CNNNetworkNGraphImpl::addOutput(const ::ngraph::Output<::ngraph::Node> & ou
createDataForResult(output, dataName, data);
_data[dataName] = data;
_outputData[dataName] = data;
+
+ // Save original framework names
+ for (const auto& name : output.get_tensor().get_names()) {
+ _tensorNames[name] = dataName;
+ }
+ for (const auto consumerInput : output.get_target_inputs()) {
+ const auto &consumerLayer = consumerInput.get_node()->shared_from_this();
+ if (std::dynamic_pointer_cast(consumerLayer)) {
+ _opNames[consumerLayer->get_friendly_name()] = dataName;
+ }
+ }
}
size_t CNNNetworkNGraphImpl::getBatchSize() const noexcept {
@@ -306,6 +333,7 @@ CNNNetworkNGraphImpl::reshape(const std::map>&
auto params = _ngraph_function->get_parameters();
+ bool parameter_replaced = false;
for (size_t i = 0; i < params.size(); i++) {
const auto& param = params[i];
if (inputShapes.find(param->get_friendly_name()) == inputShapes.end())
@@ -314,23 +342,35 @@ CNNNetworkNGraphImpl::reshape(const std::map>&
auto newParam = std::make_shared<::ngraph::op::Parameter>(param->get_element_type(), shape);
newParam->set_friendly_name(param->get_friendly_name());
_ngraph_function->replace_parameter(i, newParam);
+ parameter_replaced = true;
}
- _ngraph_function->validate_nodes_and_infer_types();
+ if (parameter_replaced)
+ _ngraph_function->validate_nodes_and_infer_types();
+
+ const auto& results = _ngraph_function->get_results();
+ bool outputs_are_static = all_of(
+ begin(results), end(results),
+ [](const std::shared_ptr& n){ return n->get_output_partial_shape(0).is_static(); });
{
- auto specialized_ngraph_function = cloneFunction(false);
- {
- OV_ITT_SCOPED_TASK(itt::domains::IE, "CNNNetworkNGraphImpl::ConvertToLegacy");
- ::ngraph::pass::Manager manager;
- // resolves dynamism by replacing dynamic operation with static version
- manager.register_pass<::ngraph::pass::ConvertNMS5ToLegacyMatcher>(false);
- manager.register_pass<::ngraph::pass::ConstantFolding>();
- // OneHotToLegacy changes output precision
- manager.register_pass<::ngraph::pass::ConvertOneHotToOneHotIEMatcher>()->detect_output_type(
- specialized_ngraph_function);
- manager.run_passes(specialized_ngraph_function);
+ shared_ptr specialized_ngraph_function = nullptr;
+ if (outputs_are_static) {
+ specialized_ngraph_function = _ngraph_function;
+ } else {
+ specialized_ngraph_function = cloneFunction(false);
+ {
+ OV_ITT_SCOPED_TASK(itt::domains::IE, "CNNNetworkNGraphImpl::ConvertToLegacy");
+ ::ngraph::pass::Manager manager;
+ // resolves dynamism by replacing dynamic operation with static version
+ manager.register_pass<::ngraph::pass::ConvertNMS5ToLegacyMatcher>(false);
+ manager.register_pass<::ngraph::pass::ConstantFolding>();
+ // OneHotToLegacy changes output precision
+ manager.register_pass<::ngraph::pass::ConvertOneHotToOneHotIEMatcher>()->detect_output_type(
+ specialized_ngraph_function);
+ manager.run_passes(specialized_ngraph_function);
+ }
+ specialized_ngraph_function->validate_nodes_and_infer_types();
}
- specialized_ngraph_function->validate_nodes_and_infer_types();
#if 0
for (const auto &op : specialized_ngraph_function->get_ordered_ops()) {
@@ -391,7 +431,7 @@ StatusCode CNNNetworkNGraphImpl::serialize(const std::string& xmlPath,
ResponseDesc* resp) const noexcept {
try {
std::map custom_opsets;
- for (auto extension : _ie_extensions) {
+ for (const auto& extension : _ie_extensions) {
auto opset = extension->getOpSets();
custom_opsets.insert(begin(opset), end(opset));
}
@@ -410,6 +450,20 @@ StatusCode CNNNetworkNGraphImpl::serialize(const std::string& xmlPath,
return OK;
}
+StatusCode CNNNetworkNGraphImpl::getOVNameForTensor(std::string& ov_name, const std::string& orig_name, ResponseDesc* resp) const noexcept {
+ if (_tensorNames.find(orig_name) == _tensorNames.end())
+ return DescriptionBuffer(NOT_FOUND, resp) << "Framework tensor with name \"" << orig_name << "\" was not mapped to OpenVINO data!";
+ ov_name = _tensorNames.at(orig_name);
+ return OK;
+}
+
+StatusCode CNNNetworkNGraphImpl::getOVNameForOperation(std::string& ov_name, const std::string& orig_name, ResponseDesc* resp) const noexcept {
+ if (_opNames.find(orig_name) == _opNames.end())
+ return DescriptionBuffer(NOT_FOUND, resp) << "Framework operation with name \"" << orig_name << "\" was not mapped to OpenVINO data!";
+ ov_name = _opNames.at(orig_name);
+ return OK;
+}
+
StatusCode CNNNetworkNGraphImpl::setBatchSize(size_t size, ResponseDesc* responseDesc) noexcept {
try {
if (getBatchSize() == size) return OK;
diff --git a/inference-engine/src/inference_engine/cnn_network_ngraph_impl.hpp b/inference-engine/src/inference_engine/cnn_network_ngraph_impl.hpp
index 7d3070afaec472..7778ec8ae82424 100644
--- a/inference-engine/src/inference_engine/cnn_network_ngraph_impl.hpp
+++ b/inference-engine/src/inference_engine/cnn_network_ngraph_impl.hpp
@@ -1,4 +1,4 @@
-// Copyright (C) 2018-2020 Intel Corporation
+// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@@ -11,6 +11,7 @@
#include
#include
+#include
#include