OpenVino integration

.gitignore

@@ -6,6 +6,7 @@
 /.env
 /keys
 /logs
+/components/openvino/*.tgz
 
 # Ignore temporary files
 docker-compose.override.yml

CHANGELOG.md

@@ -12,6 +12,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Ability to draw/change polyshapes (except for points) by slip method. Just press ENTER and moving a cursor.
 - Ability to switch lock/hide properties via label UI element (in right menu) for all objects with same label.
 - Shortcuts for outside/keyframe properties
+- OpenVINO for accelerated model inference
+- Tensorflow annotation now works without CUDA. It can use CPU only. OpenVINO and CUDA are supported optionally.
 
 ### Changed
 - Polyshape editing method has been improved. You can redraw part of shape instead of points cloning.

Dockerfile

@@ -13,8 +13,6 @@ ENV LANG='C.UTF-8'  \
     LC_ALL='C.UTF-8'
 
 ARG USER
-ARG TF_ANNOTATION
-ENV TF_ANNOTATION=${TF_ANNOTATION}
 ARG DJANGO_CONFIGURATION
 ENV DJANGO_CONFIGURATION=${DJANGO_CONFIGURATION}
 
@@ -50,13 +48,28 @@ ENV HOME /home/${USER}
 WORKDIR ${HOME}
 RUN adduser --shell /bin/bash --disabled-password --gecos "" ${USER}
 
-# Install tf annotation if need
-COPY cvat/apps/tf_annotation/docker_setup_tf_annotation.sh /tmp/tf_annotation/
-COPY cvat/apps/tf_annotation/requirements.txt /tmp/tf_annotation/
-ENV TF_ANNOTATION_MODEL_PATH=${HOME}/rcnn/frozen_inference_graph.pb
+COPY components /tmp/components
+
+# OpenVINO toolkit support
+ARG OPENVINO_TOOLKIT
+ENV OPENVINO_TOOLKIT=${OPENVINO_TOOLKIT}
+RUN if [ "$OPENVINO_TOOLKIT" = "yes" ]; then \
+        /tmp/components/openvino/install.sh; \
+    fi
+
+# CUDA support
+ARG CUDA_SUPPORT
+ENV CUDA_SUPPORT=${CUDA_SUPPORT}
+RUN if [ "$CUDA_SUPPORT" = "yes" ]; then \
+        /tmp/components/cuda/install.sh; \
+    fi
 
+# Tensorflow annotation support
+ARG TF_ANNOTATION
+ENV TF_ANNOTATION=${TF_ANNOTATION}
+ENV TF_ANNOTATION_MODEL_PATH=${HOME}/rcnn/inference_graph
 RUN if [ "$TF_ANNOTATION" = "yes" ]; then \
-        /tmp/tf_annotation/docker_setup_tf_annotation.sh; \
+        bash -i /tmp/components/tf_annotation/install.sh; \
     fi
 
 ARG WITH_TESTS

README.md

@@ -31,26 +31,6 @@ The instructions below should work for `Ubuntu 16.04`. It will probably work on
 
 Please read official manual [here](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/).
 
-### Install the latest driver for your graphics card
-
-The step is necessary only to run tf_annotation app. If you don't have a Nvidia GPU you can skip the step.
-
-```bash
-sudo add-apt-repository ppa:graphics-drivers/ppa
-sudo apt-get update
-sudo apt-cache search nvidia-*   # find latest nvidia driver
-sudo apt-get install nvidia-*    # install the nvidia driver
-sudo apt-get install mesa-common-dev
-sudo apt-get install freeglut3-dev
-sudo apt-get install nvidia-modprobe
-```
-
-Reboot your PC and verify installation by `nvidia-smi` command.
-
-### Install [Nvidia-Docker](https://github.com/NVIDIA/nvidia-docker)
-
-The step is necessary only to run tf_annotation app. If you don't have a Nvidia GPU you can skip the step. See detailed installation instructions on repository page.
-
 ### Install docker-compose (1.19.0 or newer)
 
 ```bash
@@ -61,25 +41,26 @@ sudo pip install docker-compose
 
 To build all necessary docker images run `docker-compose build` command. By default, in production mode the tool uses PostgreSQL as database, Redis for caching.
 
-### Run containers without tf_annotation app
+### Run docker containers
 
-To start all containers run `docker-compose up -d` command. Go to [localhost:8080](http://localhost:8080/). You should see a login page.
+To start default container run `docker-compose up -d` command. Go to [localhost:8080](http://localhost:8080/). You should see a login page.
 
-### Run containers with tf_annotation app
+### You can include any additional components. Just add corresponding docker-compose file to build or run command:
 
-If you would like to enable tf_annotation app first of all be sure that nvidia-driver, nvidia-docker and docker-compose>=1.19.0 are installed properly (see instructions above) and `docker info | grep 'Runtimes'` output contains `nvidia`.
-
-Run following command:
 ```bash
-docker-compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d --build
+# Build image with CUDA and OpenVINO support
+docker-compose -f docker-compose.yml -f docker-compose.cuda.yml -f docker-compose.openvino.yml build
+
+# Run containers with CUDA and OpenVINO support
+docker-compose -f docker-compose.yml -f docker-compose.cuda.yml -f docker-compose.openvino.yml up -d
 ```
 
 ### Create superuser account
 
 You can [register a user](http://localhost:8080/auth/register) but by default it will not have rights even to view list of tasks. Thus you should create a superuser. The superuser can use admin panel to assign correct groups to the user. Please use the command below:
 
 ```bash
-docker exec -it cvat sh -c '/usr/bin/python3 ~/manage.py createsuperuser'
+docker exec -it cvat bash -ic '/usr/bin/python3 ~/manage.py createsuperuser'
 ```
 
 Type your login/password for the superuser [on the login page](http://localhost:8080/auth/login) and press **Login** button. Now you should be able to create a new annotation task. Please read documentation for more details.

components/README.md

@@ -0,0 +1,6 @@
+### There are some additional components for CVAT
+
+* [NVIDIA CUDA](cuda/README.md)
+* [OpenVINO](openvino/README.md)
+* [Tensorflow Object Detector](tf_annotation/README.md)
+* [Analytics](analytics/README.md)

analytics/README.md → components/analytics/README.md

@@ -1,4 +1,18 @@
-# Analytics for Computer Vision Annotation Tool (CVAT)
+## Analytics for Computer Vision Annotation Tool (CVAT)
+
+It is possible to proxy annotation logs from client to ELK. To do that run the following command below:
+
+### Build docker image
+```bash
+# From project root directory
+docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml build
+```
+
+### Run docker container
+```bash
+# From project root directory
+docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml up -d
+```
 
 At the moment it is not possible to save advanced settings. Below values should be specified manually.
 

analytics/docker-compose.yml → components/analytics/docker-compose.analytics.yml

@@ -8,7 +8,7 @@ services:
         aliases:
           - elasticsearch
     build:
-      context: ./analytics/elasticsearch
+      context: ./components/analytics/elasticsearch
       args:
         ELK_VERSION: 6.4.0
     volumes:
@@ -23,7 +23,7 @@ services:
         aliases:
           - kibana
     build:
-      context: ./analytics/kibana
+      context: ./components/analytics/kibana
       args:
         ELK_VERSION: 6.4.0
     depends_on: ['cvat_elasticsearch']
@@ -32,7 +32,7 @@ services:
   cvat_kibana_setup:
     container_name: cvat_kibana_setup
     image: cvat
-    volumes: ['./analytics/kibana:/home/django/kibana:ro']
+    volumes: ['./components/analytics/kibana:/home/django/kibana:ro']
     depends_on: ['cvat']
     working_dir: '/home/django'
     entrypoint: ['bash', 'wait-for-it.sh', 'elasticsearch:9200', '-t', '0', '--',
@@ -49,7 +49,7 @@ services:
         aliases:
           - logstash
     build:
-      context: ./analytics/logstash
+      context: ./components/analytics/logstash
       args:
         ELK_VERSION: 6.4.0
         http_proxy:  ${http_proxy}

components/cuda/README.md

@@ -0,0 +1,41 @@
+## [NVIDIA CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit)
+
+### Requirements
+
+* NVIDIA GPU with a compute capability [3.0 - 7.2]
+* Latest GPU driver
+
+### Installation
+
+#### Install the latest driver for your graphics card
+
+```bash
+sudo add-apt-repository ppa:graphics-drivers/ppa
+sudo apt-get update
+sudo apt-cache search nvidia-*   # find latest nvidia driver
+sudo apt-get install nvidia-*    # install the nvidia driver
+sudo apt-get install mesa-common-dev
+sudo apt-get install freeglut3-dev
+sudo apt-get install nvidia-modprobe
+```
+
+#### Reboot your PC and verify installation by `nvidia-smi` command.
+
+#### Install [Nvidia-Docker](https://github.com/NVIDIA/nvidia-docker)
+
+Please be sure that installation was successful.
+```bash
+docker info | grep 'Runtimes'   # output should contains 'nvidia'
+```
+
+### Build docker image
+```bash
+# From project root directory
+docker-compose -f docker-compose.yml -f components/cuda/docker-compose.cuda.yml build
+```
+
+### Run docker container
+```bash
+# From project root directory
+docker-compose -f docker-compose.yml -f components/cuda/docker-compose.cuda.yml up -d
+```

docker-compose.nvidia.yml → components/cuda/docker-compose.cuda.yml

@@ -10,7 +10,7 @@ services:
     build:
       context: .
       args:
-        TF_ANNOTATION: "yes"
+        CUDA_SUPPORT: "yes"
     runtime: "nvidia"
     environment:
       NVIDIA_VISIBLE_DEVICES: all

cvat/apps/tf_annotation/docker_setup_tf_annotation.sh → components/cuda/install.sh

@@ -18,8 +18,8 @@ CUDA_VERSION=9.0.176
 NCCL_VERSION=2.1.15
 CUDNN_VERSION=7.0.5.15
 CUDA_PKG_VERSION="9-0=${CUDA_VERSION}-1"
-echo "export PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}" >> ${HOME}/.bashrc
-echo "export LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:${LD_LIBRARY_PATH}" >> ${HOME}/.bashrc
+echo 'export PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}' >> ${HOME}/.bashrc
+echo 'export LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:${LD_LIBRARY_PATH}' >> ${HOME}/.bashrc
 
 apt-get update && apt-get install -y --no-install-recommends --allow-unauthenticated \
     libprotobuf-dev \
@@ -32,11 +32,3 @@ apt-get update && apt-get install -y --no-install-recommends --allow-unauthentic
     ln -s cuda-9.0 /usr/local/cuda && \
 rm -rf /var/lib/apt/lists/* \
     /etc/apt/sources.list.d/nvidia-ml.list /etc/apt/sources.list.d/cuda.list
-
-pip3 install --no-cache-dir -r "$(cd `dirname $0` && pwd)/requirements.txt"
-
-cd ${HOME}
-wget -O model.tar.gz http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_coco_11_06_2017.tar.gz
-tar -xzf model.tar.gz
-rm model.tar.gz
-mv faster_rcnn_inception_resnet_v2_atrous_coco_11_06_2017 rcnn

components/openvino/README.md

@@ -0,0 +1,23 @@
+## [Intel OpenVINO toolkit](https://software.intel.com/en-us/openvino-toolkit)
+
+### Requirements
+
+* Intel Core with 6th generation and higher or Intel Xeon CPUs.
+
+### Preparation
+
+* Download latest OpenVINO toolkit for Ubuntu 16.04 platform. It should be .tgz archive. Minimum required version is 2018.3.*
+* Put downloaded file into components/openvino
+* Accept EULA in the eula.cfg file
+
+### Build docker image
+```bash
+# From project root directory
+docker-compose -f docker-compose.yml -f components/openvino/docker-compose.openvino.yml build
+```
+
+### Run docker container
+```bash
+# From project root directory
+docker-compose -f docker-compose.yml -f components/openvino/docker-compose.openvino.yml up -d
+```

components/openvino/docker-compose.openvino.yml

@@ -0,0 +1,13 @@
+#
+# Copyright (C) 2018 Intel Corporation
+#
+# SPDX-License-Identifier: MIT
+#
+version: "2.3"
+
+services:
+  cvat:
+    build:
+      context: .
+      args:
+        OPENVINO_TOOLKIT: "yes"

components/openvino/eula.cfg

@@ -0,0 +1,3 @@
+# Accept actual EULA from openvino installation archive. Valid values are: {accept, decline}
+ACCEPT_EULA=accept
+

components/openvino/install.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+#
+# Copyright (C) 2018 Intel Corporation
+#
+# SPDX-License-Identifier: MIT
+#
+set -e
+
+if [[ `lscpu | grep -o "GenuineIntel"` != "GenuineIntel" ]]; then
+    echo "OpenVINO supports only Intel CPUs"
+    exit 1
+fi
+
+if [[ `lscpu | grep -o "sse4" | head -1` != "sse4" ]] && [[ `lscpu | grep -o "avx2" | head -1` != "avx2" ]]; then
+    echo "You Intel CPU should support sse4 or avx2 instruction if you want use OpenVINO"
+    exit 1
+fi
+
+apt update && apt install -y libpng12-dev libcairo2-dev \
+    libpango1.0-dev libglib2.0-dev libgtk2.0-dev \
+    libgstreamer0.10-dev libswscale-dev \
+    libavcodec-dev libavformat-dev cmake libusb-1.0-0-dev cpio
+
+# OpenCV which included into OpenVino toolkit was compiled with other version ffmpeg
+# Need to install these packages for it works
+apt install -y libavcodec-ffmpeg56 libavformat-ffmpeg56 libswscale-ffmpeg3
+
+cd /tmp/components/openvino
+tar -xzf `ls | grep "openvino_toolkit"`
+cd `ls -d */ | grep "openvino_toolkit"`
+
+cat ../eula.cfg >> silent.cfg
+./install.sh -s silent.cfg
+
+cd /tmp/components && rm openvino -r
+
+echo "source /opt/intel/computer_vision_sdk/bin/setupvars.sh" >> ${HOME}/.bashrc
+echo -e '\nexport IE_PLUGINS_PATH=${IE_PLUGINS_PATH}' >> /opt/intel/computer_vision_sdk/bin/setupvars.sh

components/tf_annotation/README.md

@@ -0,0 +1,41 @@
+## [Tensorflow Object Detector](https://github.com/tensorflow/models/tree/master/research/object_detection)
+
+### What is it?
+* This application allows you automatically to annotate many various objects on images.
+* It uses [Faster RCNN Inception Resnet v2 Atrous Coco Model](http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_resnet_v2_atrous_coco_2018_01_28.tar.gz) from [tensorflow detection model zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md)
+* It can work on CPU (with Tensorflow or OpenVINO) or GPU (with Tensorflow GPU).
+* It supports next classes (just specify them in "labels" row):
+```
+'surfboard', 'car', 'skateboard', 'boat', 'clock',
+'cat', 'cow', 'knife', 'apple', 'cup', 'tv',
+'baseball_bat', 'book', 'suitcase', 'tennis_racket',
+'stop_sign', 'couch', 'cell_phone', 'keyboard',
+'cake', 'tie', 'frisbee', 'truck', 'fire_hydrant',
+'snowboard', 'bed', 'vase', 'teddy_bear',
+'toaster', 'wine_glass', 'traffic_light',
+'broccoli', 'backpack', 'carrot', 'potted_plant',
+'donut', 'umbrella', 'parking_meter', 'bottle',
+'sandwich', 'motorcycle', 'bear', 'banana',
+'person', 'scissors', 'elephant', 'dining_table',
+'toothbrush', 'toilet', 'skis', 'bowl', 'sheep',
+'refrigerator', 'oven', 'microwave', 'train',
+'orange', 'mouse', 'laptop', 'bench', 'bicycle',
+'fork', 'kite', 'zebra', 'baseball_glove', 'bus',
+'spoon', 'horse', 'handbag', 'pizza', 'sports_ball',
+'airplane', 'hair_drier', 'hot_dog', 'remote',
+'sink', 'dog', 'bird', 'giraffe', 'chair'.
+```
+* Component adds "Run TF Annotation" button into dashboard.
+
+
+### Build docker image
+```bash
+# From project root directory
+docker-compose -f docker-compose.yml -f components/tf_annotation/docker-compose.tf_annotation.yml build
+```
+
+### Run docker container
+```bash
+# From project root directory
+docker-compose -f docker-compose.yml -f components/tf_annotation/docker-compose.tf_annotation.yml up -d
+```

components/tf_annotation/docker-compose.tf_annotation.yml

@@ -0,0 +1,13 @@
+#
+# Copyright (C) 2018 Intel Corporation
+#
+# SPDX-License-Identifier: MIT
+#
+version: "2.3"
+
+services:
+  cvat:
+    build:
+      context: .
+      args:
+        TF_ANNOTATION: "yes"