Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add openvino interpreter samples #159

Merged
merged 7 commits into from
Mar 17, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions datumaro/plugins/openvino/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# OpenVINO™ Inference Interpreter
Interpreter samples to parse OpenVINO™ inference outputs.

## Models supported from interpreter samples
There are 5 SSD and 1 image classification interpreter samples.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
There are 5 SSD and 1 image classification interpreter samples.
There are detection and image classification examples.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


- 5 SSD interpreter samples support the following models.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- 5 SSD interpreter samples support the following models.
- Detection (SSD-based)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

- Intel Pre-trained Models > Object Detection
- [face-detection-0200](https://docs.openvinotoolkit.org/latest/omz_models_intel_face_detection_0200_description_face_detection_0200.html)
- [face-detection-0202](https://docs.openvinotoolkit.org/latest/omz_models_intel_face_detection_0202_description_face_detection_0202.html)
- [face-detection-0204](https://docs.openvinotoolkit.org/latest/omz_models_intel_face_detection_0204_description_face_detection_0204.html)
- [person-detection-0200](https://docs.openvinotoolkit.org/latest/omz_models_intel_person_detection_0200_description_person_detection_0200.html)
- [person-detection-0201](https://docs.openvinotoolkit.org/latest/omz_models_intel_person_detection_0201_description_person_detection_0201.html)
- [person-detection-0202](https://docs.openvinotoolkit.org/latest/omz_models_intel_person_detection_0202_description_person_detection_0202.html)
- [person-vehicle-bike-detection-2000](https://docs.openvinotoolkit.org/latest/omz_models_intel_person_vehicle_bike_detection_2000_description_person_vehicle_bike_detection_2000.html)
- [person-vehicle-bike-detection-2001](https://docs.openvinotoolkit.org/latest/omz_models_intel_person_vehicle_bike_detection_2001_description_person_vehicle_bike_detection_2001.html)
- [person-vehicle-bike-detection-2002](https://docs.openvinotoolkit.org/latest/omz_models_intel_person_vehicle_bike_detection_2002_description_person_vehicle_bike_detection_2002.html)
- [vehicle-detection-0200](https://docs.openvinotoolkit.org/latest/omz_models_intel_vehicle_detection_0200_description_vehicle_detection_0200.html)
- [vehicle-detection-0201](https://docs.openvinotoolkit.org/latest/omz_models_intel_vehicle_detection_0201_description_vehicle_detection_0201.html)
- [vehicle-detection-0202](https://docs.openvinotoolkit.org/latest/omz_models_intel_vehicle_detection_0202_description_vehicle_detection_0202.html)

- Public Pre-Trained Models(OMZ) > Object Detection
- [ssd_mobilenet_v1_coco](https://docs.openvinotoolkit.org/latest/omz_models_public_ssd_mobilenet_v1_coco_ssd_mobilenet_v1_coco.html)
- [ssd_mobilenet_v2_coco](https://docs.openvinotoolkit.org/latest/omz_models_public_ssd_mobilenet_v2_coco_ssd_mobilenet_v2_coco.html)

- Image classification interpreter sample supports the following model.
- Public Pre-Trained Models(OMZ) > Classification
- [mobilenet-v2-pytorch](https://docs.openvinotoolkit.org/latest/omz_models_public_mobilenet_v2_pytorch_mobilenet_v2_pytorch.html)

You can find more OpenVINO™ Trained Models [here](https://docs.openvinotoolkit.org/latest/omz_models_intel_index.html)
To run the inference with OpenVINO™, the model format should be Intermediate Representation(IR).
For the Caffe/TensorFlow/MXNet/Kaldi/ONNX models, please see the [Model Conversion Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model.html)

You need to implement your own interpreter samples to support the other OpenVINO™ Trained Models.

## Model download
- Prerequisites
- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)
Copy link
Contributor

@zhiltsov-max zhiltsov-max Mar 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing )

Suggested change
- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)
- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)
Copy link
Contributor

@zhiltsov-max zhiltsov-max Mar 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing )

Suggested change
- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)
- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


```bash
# cd <openvino_dir>/deployment_tools/open_model_zoo/tools/downloader
# ./downloader.py --name <model_name>
#
# Examples
cd /opt/intel/openvino_2021/deployment_tools/open_model_zoo/tools/downloader
./downloader.py --name face-detection-0200
```

## Model inference
- Prerequisites:
- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
Copy link
Contributor

@zhiltsov-max zhiltsov-max Mar 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing ) (+3 next)

Suggested change
- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html)
- OpenVINO™ (To install OpenVINO™, please see the [OpenVINO™ Installation Instruction](https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

- Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md)
- Datumaro (To install Datumaro, please see the [User Manual](docs/user_manual.md))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html)
- OpenVINO™ models (To download OpenVINO™ models, please see the [Model Downloader Instruction] (https://docs.openvinotoolkit.org/latest/omz_tools_downloader_README.html))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit)
- PASCAL VOC 2012 dataset (To download VOC 2012 dataset, please go [VOC2012 download](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


- To run the inference with OpenVINO™ models and the interpreter samples, please follow the instructions below.

```bash
# <openvino_dir>/bin/setupvars.sh
# datum create -o <proj_dir>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why these commands have different indents?

# datum model add -l <launcher> -p <proj_dir> --copy -- -d <path_to_xml> -w <path_to_bin> -i <path_to_interpreter_script>
# datum add path -p <proj_dir> -f <format> <path_to_dataset>
# datum model run -p <proj_dir> -m model-0
#
# Examples
# Detection> ssd_mobilenet_v2_coco
/opt/intel/openvino_2021/bin/setupvars.sh
Copy link
Contributor

@zhiltsov-max zhiltsov-max Mar 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

. or source in the beginning is required

Suggested change
/opt/intel/openvino_2021/bin/setupvars.sh
. /opt/intel/openvino/bin/setupvars.sh

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we install openvino 2021.r1, the default path is /opt/intel/openvino_2021, not /opt/intel/openvino. The default path like openvino_2021 had been using since openvino 2021.r1. Do you still want me to change it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to openvino because we don't know user which Openvino version will install. openvino is more general. So I changed it.

cd /home/cvalgo/workspace/datumaro/datumaro/plugins/openvino/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please clean /home/cvalgo/workspace. Make the examples as simple and ready to use, as possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
cd /home/cvalgo/workspace/datumaro/datumaro/plugins/openvino/
cd datumaro/plugins/openvino/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

datum create -o proj_ssd_mobilenet_v2_coco_detection
datum model add -l openvino -p proj_ssd_mobilenet_v2_coco_detection --copy -- --output-layers=do_ExpandDims_conf/sigmoid -d model/ssd_mobilenet_v2_coco.xml -w model/ssd_mobilenet_v2_coco.bin -i samples/ssd_mobilenet_coco_detection_interp.py
datum add path -p proj_ssd_mobilenet_v2_coco_detection -f voc /home/cvalgo/workspace/datumaro/VOCdevkit
datum model run -p proj_ssd_mobilenet_v2_coco_detection -m model-0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
datum create -o proj_ssd_mobilenet_v2_coco_detection
datum model add -l openvino -p proj_ssd_mobilenet_v2_coco_detection --copy -- --output-layers=do_ExpandDims_conf/sigmoid -d model/ssd_mobilenet_v2_coco.xml -w model/ssd_mobilenet_v2_coco.bin -i samples/ssd_mobilenet_coco_detection_interp.py
datum add path -p proj_ssd_mobilenet_v2_coco_detection -f voc /home/cvalgo/workspace/datumaro/VOCdevkit
datum model run -p proj_ssd_mobilenet_v2_coco_detection -m model-0
datum create -o proj_ssd_mobilenet_v2_coco_detection
datum model add -l openvino -p proj_ssd_mobilenet_v2_coco_detection --copy -- \
--output-layers=do_ExpandDims_conf/sigmoid \
-d model/ssd_mobilenet_v2_coco.xml \
-w model/ssd_mobilenet_v2_coco.bin \
-i samples/ssd_mobilenet_coco_detection_interp.py
datum add path -p proj_ssd_mobilenet_v2_coco_detection -f voc VOCdevkit/
datum model run -p proj_ssd_mobilenet_v2_coco_detection -m model-0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


# Classification> mobilenet-v2-pytorch
/opt/intel/openvino_2021/bin/setupvars.sh
cd /home/cvalgo/workspace/datumaro/datumaro/plugins/openvino/
datum create -o proj_mobilenet_v2_classification
datum model add -l openvino -p proj_mobilenet_v2_classification --copy -- -d model/mobilenet-v2-pytorch.xml -w model/mobilenet-v2-pytorch.bin -i samples/mobilenet_v2_pytorch_interp.py
datum add path -p proj_mobilenet_v2_classification -f voc /home/cvalgo/workspace/datumaro/VOCdevkit
datum model run -p proj_mobilenet_v2_classification -m model-0
```
Empty file.
91 changes: 91 additions & 0 deletions datumaro/plugins/openvino/samples/coco.class
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
person
bicycle
car
motorcycle
airplane
bus
train
truck
boat
trafficlight
firehydrant
streetsign
stopsign
parkingmeter
bench
bird
cat
dog
horse
sheep
cow
elephant
bear
zebra
giraffe
hat
backpack
umbrella
shoe
eyeglasses
handbag
tie
suitcase
frisbee
skis
snowboard
sportsball
kite
baseballbat
baseballglove
skateboard
surfboard
tennisracket
bottle
plate
wineglass
cup
fork
knife
spoon
bowl
banana
apple
sandwich
orange
broccoli
carrot
hotdog
pizza
donut
cake
chair
couch
pottedplant
bed
mirror
diningtable
window
desk
toilet
door
tv
laptop
mouse
remote
keyboard
cellphone
microwave
oven
toaster
sink
refrigerator
blender
book
clock
vase
scissors
teddybear
hairdrier
toothbrush
hairbrush
Loading