Skip to content
This repository has been archived by the owner on Nov 14, 2023. It is now read-only.

Commit

Permalink
YoloV7 serverless detector feature for auto annotation (cvat-ai#5552)
Browse files Browse the repository at this point in the history 
### Motivation and context

Integration of YOLOv7 as a serverless nuclio function that can be used
for auto-labeling. YoloV7 is the SOTA at the time of this PR therefore
it would make sense to support it in CVAT. The integration is quite
simple into CVAT as docker based on Ultralytics YoloV5 with coco
pretrained model (https://github.com/WongKinYiu/yolov7) and a docker
image (https://hub.docker.com/r/ultralytics/yolov5).

related issue: cvat-ai#5548 

### How has this been tested?

Automatic annotation was run using YOLOv7 on a custom dataset.
The serverless function was deployed using

```
nuctl deploy --project-name cvat \
  --path serverless/onnx/WongKinYiu/yolov7/nuclio \
  --volume `pwd`/serverless/common:/opt/nuclio/common \
  --platform local
```

Then using the 'Automatic annotation' action the function was tested and
the auto-generated labels were controlled to check that no coordinates
misfit is happening.

### Use custom model:
1. Export your model with NMS for image resolution of 640x640
(preferable).
2. Copy your custom model yolov7-custom.onnx to /serverless/common
3. Modify function.yaml file according to your labels.
4. Modify model_handler.py as follow:
```
 self.model_path = "yolov7-custom.onnx"
```

Co-authored-by: Nikita Manovich <[email protected]>
Co-authored-by: yasakova-anastasia <[email protected]>
  • Loading branch information
@hardikdava @nmanovic
@yasakova-anastasia @mikhail-treskin
3 people authored and mikhail-treskin committed Jul 1, 2023
1 parent 5942ec9 commit c1f9ff2
Show file tree
Hide file tree
Showing 6 changed files with 421 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
(<https://github.com/opencv/cvat/pull/5523>)
- \[SDK\] A PyTorch adapter setting to disable cache updates
(<https://github.com/opencv/cvat/pull/5549>)
- YOLO v7 serverless feature added using ONNX backend (<https://github.com/opencv/cvat/pull/5552>)

### Changed
- The Docker Compose files now use the Compose Specification version
Expand Down
1 change: 1 addition & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,7 @@ up to 10x. Here is a list of the algorithms we support, and the platforms they c
| [Faster RCNN](/serverless/openvino/omz/public/faster_rcnn_inception_v2_coco/nuclio) | detector | OpenVINO | ✔️ | |
| [Mask RCNN](/serverless/openvino/omz/public/mask_rcnn_inception_resnet_v2_atrous_coco/nuclio) | detector | OpenVINO | ✔️ | |
| [YOLO v3](/serverless/openvino/omz/public/yolo-v3-tf/nuclio) | detector | OpenVINO | ✔️ | |
| [YOLO v7](/serverless/onnx/WongKinYiu/yolov7/nuclio) | detector | ONNX | ✔️ | ✔️ |
| [Object reidentification](/serverless/openvino/omz/intel/person-reidentification-retail-300/nuclio) | reid | OpenVINO | ✔️ | |
| [Semantic segmentation for ADAS](/serverless/openvino/omz/intel/semantic-segmentation-adas-0001/nuclio) | detector | OpenVINO | ✔️ | |
| [Text detection v4](/serverless/openvino/omz/intel/text-detection-0004/nuclio) | detector | OpenVINO | ✔️ | |
Expand Down
135 changes: 135 additions & 0 deletions serverless/onnx/WongKinYiu/yolov7/nuclio/function-gpu.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
metadata:
name: onnx-yolov7
namespace: cvat
annotations:
name: YOLO v7
type: detector
framework: onnx
spec: |
[
{ "id": 0, "name": "person" },
{ "id": 1, "name": "bicycle" },
{ "id": 2, "name": "car" },
{ "id": 3, "name": "motorbike" },
{ "id": 4, "name": "aeroplane" },
{ "id": 5, "name": "bus" },
{ "id": 6, "name": "train" },
{ "id": 7, "name": "truck" },
{ "id": 8, "name": "boat" },
{ "id": 9, "name": "traffic light" },
{ "id": 10, "name": "fire hydrant" },
{ "id": 11, "name": "stop sign" },
{ "id": 12, "name": "parking meter" },
{ "id": 13, "name": "bench" },
{ "id": 14, "name": "bird" },
{ "id": 15, "name": "cat" },
{ "id": 16, "name": "dog" },
{ "id": 17, "name": "horse" },
{ "id": 18, "name": "sheep" },
{ "id": 19, "name": "cow" },
{ "id": 20, "name": "elephant" },
{ "id": 21, "name": "bear" },
{ "id": 22, "name": "zebra" },
{ "id": 23, "name": "giraffe" },
{ "id": 24, "name": "backpack" },
{ "id": 25, "name": "umbrella" },
{ "id": 26, "name": "handbag" },
{ "id": 27, "name": "tie" },
{ "id": 28, "name": "suitcase" },
{ "id": 29, "name": "frisbee" },
{ "id": 30, "name": "skis" },
{ "id": 31, "name": "snowboard" },
{ "id": 32, "name": "sports ball" },
{ "id": 33, "name": "kite" },
{ "id": 34, "name": "baseball bat" },
{ "id": 35, "name": "baseball glove" },
{ "id": 36, "name": "skateboard" },
{ "id": 37, "name": "surfboard" },
{ "id": 38, "name": "tennis racket" },
{ "id": 39, "name": "bottle" },
{ "id": 40, "name": "wine glass" },
{ "id": 41, "name": "cup" },
{ "id": 42, "name": "fork" },
{ "id": 43, "name": "knife" },
{ "id": 44, "name": "spoon" },
{ "id": 45, "name": "bowl" },
{ "id": 46, "name": "banana" },
{ "id": 47, "name": "apple" },
{ "id": 48, "name": "sandwich" },
{ "id": 49, "name": "orange" },
{ "id": 50, "name": "broccoli" },
{ "id": 51, "name": "carrot" },
{ "id": 52, "name": "hot dog" },
{ "id": 53, "name": "pizza" },
{ "id": 54, "name": "donut" },
{ "id": 55, "name": "cake" },
{ "id": 56, "name": "chair" },
{ "id": 57, "name": "sofa" },
{ "id": 58, "name": "pottedplant" },
{ "id": 59, "name": "bed" },
{ "id": 60, "name": "diningtable" },
{ "id": 61, "name": "toilet" },
{ "id": 62, "name": "tvmonitor" },
{ "id": 63, "name": "laptop" },
{ "id": 64, "name": "mouse" },
{ "id": 65, "name": "remote" },
{ "id": 66, "name": "keyboard" },
{ "id": 67, "name": "cell phone" },
{ "id": 68, "name": "microwave" },
{ "id": 69, "name": "oven" },
{ "id": 70, "name": "toaster" },
{ "id": 71, "name": "sink" },
{ "id": 72, "name": "refrigerator" },
{ "id": 73, "name": "book" },
{ "id": 74, "name": "clock" },
{ "id": 75, "name": "vase" },
{ "id": 76, "name": "scissors" },
{ "id": 77, "name": "teddy bear" },
{ "id": 78, "name": "hair drier" },
{ "id": 79, "name": "toothbrush" }
]
spec:
description: YOLO v7 via onnx-runtime
runtime: 'python:3.8'
handler: main:handler
eventTimeout: 30s
build:
image: cvat/onnx-yolov7
baseImage: ultralytics/yolov5:latest

directives:
preCopy:
- kind: USER
value: root
- kind: RUN
value: apt update && apt install --no-install-recommends -y libglib2.0-0
- kind: WORKDIR
value: /opt/nuclio
- kind: RUN
value: pip install onnxruntime
- kind: WORKDIR
value: /opt/nuclio
- kind: RUN
value: wget https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7-nms-640.onnx
- kind: RUN
value: ln -s /usr/bin/python3 /usr/bin/python

triggers:
myHttpTrigger:
maxWorkers: 1
kind: 'http'
workerAvailabilityTimeoutMilliseconds: 10000
attributes:
maxRequestBodySize: 33554432 # 32MB

resources:
limits:
nvidia.com/gpu: 1

platform:
attributes:
restartPolicy:
name: always
maximumRetryCount: 3
mountMode: volume
128 changes: 128 additions & 0 deletions serverless/onnx/WongKinYiu/yolov7/nuclio/function.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
metadata:
name: onnx-yolov7
namespace: cvat
annotations:
name: YOLO v7
type: detector
framework: onnx
spec: |
[
{ "id": 0, "name": "person" },
{ "id": 1, "name": "bicycle" },
{ "id": 2, "name": "car" },
{ "id": 3, "name": "motorbike" },
{ "id": 4, "name": "aeroplane" },
{ "id": 5, "name": "bus" },
{ "id": 6, "name": "train" },
{ "id": 7, "name": "truck" },
{ "id": 8, "name": "boat" },
{ "id": 9, "name": "traffic light" },
{ "id": 10, "name": "fire hydrant" },
{ "id": 11, "name": "stop sign" },
{ "id": 12, "name": "parking meter" },
{ "id": 13, "name": "bench" },
{ "id": 14, "name": "bird" },
{ "id": 15, "name": "cat" },
{ "id": 16, "name": "dog" },
{ "id": 17, "name": "horse" },
{ "id": 18, "name": "sheep" },
{ "id": 19, "name": "cow" },
{ "id": 20, "name": "elephant" },
{ "id": 21, "name": "bear" },
{ "id": 22, "name": "zebra" },
{ "id": 23, "name": "giraffe" },
{ "id": 24, "name": "backpack" },
{ "id": 25, "name": "umbrella" },
{ "id": 26, "name": "handbag" },
{ "id": 27, "name": "tie" },
{ "id": 28, "name": "suitcase" },
{ "id": 29, "name": "frisbee" },
{ "id": 30, "name": "skis" },
{ "id": 31, "name": "snowboard" },
{ "id": 32, "name": "sports ball" },
{ "id": 33, "name": "kite" },
{ "id": 34, "name": "baseball bat" },
{ "id": 35, "name": "baseball glove" },
{ "id": 36, "name": "skateboard" },
{ "id": 37, "name": "surfboard" },
{ "id": 38, "name": "tennis racket" },
{ "id": 39, "name": "bottle" },
{ "id": 40, "name": "wine glass" },
{ "id": 41, "name": "cup" },
{ "id": 42, "name": "fork" },
{ "id": 43, "name": "knife" },
{ "id": 44, "name": "spoon" },
{ "id": 45, "name": "bowl" },
{ "id": 46, "name": "banana" },
{ "id": 47, "name": "apple" },
{ "id": 48, "name": "sandwich" },
{ "id": 49, "name": "orange" },
{ "id": 50, "name": "broccoli" },
{ "id": 51, "name": "carrot" },
{ "id": 52, "name": "hot dog" },
{ "id": 53, "name": "pizza" },
{ "id": 54, "name": "donut" },
{ "id": 55, "name": "cake" },
{ "id": 56, "name": "chair" },
{ "id": 57, "name": "sofa" },
{ "id": 58, "name": "pottedplant" },
{ "id": 59, "name": "bed" },
{ "id": 60, "name": "diningtable" },
{ "id": 61, "name": "toilet" },
{ "id": 62, "name": "tvmonitor" },
{ "id": 63, "name": "laptop" },
{ "id": 64, "name": "mouse" },
{ "id": 65, "name": "remote" },
{ "id": 66, "name": "keyboard" },
{ "id": 67, "name": "cell phone" },
{ "id": 68, "name": "microwave" },
{ "id": 69, "name": "oven" },
{ "id": 70, "name": "toaster" },
{ "id": 71, "name": "sink" },
{ "id": 72, "name": "refrigerator" },
{ "id": 73, "name": "book" },
{ "id": 74, "name": "clock" },
{ "id": 75, "name": "vase" },
{ "id": 76, "name": "scissors" },
{ "id": 77, "name": "teddy bear" },
{ "id": 78, "name": "hair drier" },
{ "id": 79, "name": "toothbrush" }
]
spec:
description: YOLO v7 via onnx
runtime: 'python:3.8'
handler: main:handler
eventTimeout: 30s
build:
image: cvat/onnx-yolov7
baseImage: ultralytics/yolov5:latest-cpu

directives:
preCopy:
- kind: USER
value: root
- kind: RUN
value: apt update && apt install --no-install-recommends -y libglib2.0-0 && apt install wget
- kind: RUN
value: pip install onnxruntime
- kind: WORKDIR
value: /opt/nuclio
- kind: RUN
value: wget https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7-nms-640.onnx
- kind: RUN
value: ln -s /usr/bin/python3 /usr/bin/python

triggers:
myHttpTrigger:
maxWorkers: 2
kind: 'http'
workerAvailabilityTimeoutMilliseconds: 10000
attributes:
maxRequestBodySize: 33554432 # 32MB

platform:
attributes:
restartPolicy:
name: always
maximumRetryCount: 3
mountMode: volume
37 changes: 37 additions & 0 deletions serverless/onnx/WongKinYiu/yolov7/nuclio/main.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import base64
import io
import json

import yaml
from model_handler import ModelHandler
from PIL import Image


def init_context(context):
context.logger.info("Init context... 0%")

# Read labels
with open("/opt/nuclio/function.yaml", 'rb') as function_file:
functionconfig = yaml.safe_load(function_file)

labels_spec = functionconfig['metadata']['annotations']['spec']
labels = {item['id']: item['name'] for item in json.loads(labels_spec)}

# Read the DL model
model = ModelHandler(labels)
context.user_data.model = model

context.logger.info("Init context...100%")


def handler(context, event):
context.logger.info("Run YoloV7 ONNX model")
data = event.body
buf = io.BytesIO(base64.b64decode(data["image"]))
threshold = float(data.get("threshold", 0.5))
image = Image.open(buf)

results = context.user_data.model.infer(image, threshold)

return context.Response(body=json.dumps(results), headers={},
content_type='application/json', status_code=200)
Loading

0 comments on commit c1f9ff2

Please sign in to comment.