Skip to content

Commit

Permalink
Add user documentation file and integration tests for YOLO format (cv…
Browse files Browse the repository at this point in the history 
…at-ai#246)

* add user documentation file for yolo

* add integraion tests

* update user manual

* update changelog
  • Loading branch information
Kirill Sizov authored May 20, 2021
1 parent 8c7bbc5 commit 6782295
Show file tree
Hide file tree
Showing 5 changed files with 369 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Documentation file and integration tests for Pascal VOC format (<https://github.com/openvinotoolkit/datumaro/pull/228>)
- Support for MNIST and MNIST in CSV dataset formats (<https://github.com/openvinotoolkit/datumaro/pull/234>)
- Documentation file for COCO format (<https://github.com/openvinotoolkit/datumaro/pull/241>)
- Documentation file and integration tests for YOLO format (<https://github.com/openvinotoolkit/datumaro/pull/246>)

### Changed
- LabelMe format saves dataset items with their relative paths by subsets without changing names (<https://github.com/openvinotoolkit/datumaro/pull/200>)
Expand Down
210 changes: 210 additions & 0 deletions docs/formats/yolo_user_manual.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
# YOLO user manual

## Contents
- [Format specification](#format-specification)
- [Load YOLO dataset](#load-yolo-dataset)
- [Export to other formats](#export-to-other-formats)
- [Export to YOLO format](#export-to-yolo-format)
- [Particular use cases](#particular-use-cases)

## Format specification

- The YOLO dataset format is for training and validating object detection models.
Specification for this format available
[here](https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects).
And also you can find some official examples on working with YOLO dataset
[here](https://pjreddie.com/darknet/yolo/);

- The YOLO dataset format support the following types of annotations:
- `Bounding boxes`

- YOLO format doesn't support attributes for annotations;

- The format only supports subsets named `train` or `valid`.

## Load YOLO dataset

Few ways to create Datumaro project and add YOLO dataset to it:

```bash
datum import -o project -f yolo -i <path/to/yolo/dataset>

# another way to do the same:
datum create -o project
datum add path -p project -f yolo -i <path/to/yolo/dataset>

# and you can add another one yolo dataset:
datum add path -p project -f yolo -i <path/to/other/yolo/dataset>
```

YOLO dataset directory should have the following structure:

<!--lint disable fenced-code-flag-->
```
└─ yolo_dataset/
│
├── obj.names # file with list of classes
├── obj.data # file with dataset information
├── train.txt # list of image paths in train subset
├── valid.txt # list of image paths in valid subset
│
├── obj_train_data/ # directory with annotations and images for train subset
│ ├── image1.txt # list of labeled bounding boxes for image1
│ ├── image1.jpg
│ ├── image2.txt
│ ├── image2.jpg
│ ├── ...
│
├── obj_valid_data/ # directory with annotations and images for valid subset
│ ├── image101.txt
│ ├── image101.jpg
│ ├── image102.txt
│ ├── image102.jpg
│ ├── ...
```
> YOLO dataset cannot contain a subset with a name other than `train` or `valid`.
If imported dataset contains such subsets, they will be ignored.
If you are exporting a project into yolo format,
all subsets different from `train` and `valid` will be skipped.
If there is no subset separation in a project, the data
will be saved in `train` subset.

- `obj.data` should have the following content, it is not necessary to have both
subsets, but necessary to have one of them:
```
classes = 5 # optional
names = <path/to/obj.names>
train = <path/to/train.txt>
valid = <path/to/valid.txt>
backup = backup/ # optional
```
- `obj.names` contain list of classes.
The line number for the class is the same as its index:
```
label1 # label1 has index 0
label2 # label2 has index 1
label3 # label2 has index 2
...
```
- Files `train.txt` and `valid.txt` should have the following structure:
```
<path/to/image1.jpg>
<path/to/image2.jpg>
...
```
- Files in directories `obj_train_data/` and `obj_valid_data/`
should contain information about labeled bounding boxes
for images:
```
# image1.txt:
# <label_index> <x> <y> <width> <height>
0 0.250000 0.400000 0.300000 0.400000
3 0.600000 0.400000 0.400000 0.266667
```
Here `x`, `y`, `width`, and `height` are relative to the image's width and height.

## Export to other formats

Datumaro can convert YOLO dataset into any other format
[Datumaro supports](../docs/user_manual.md#supported-formats).
For successful conversion the output format should support
object detection task (e.g. Pascal VOC, COCO, TF Detection API etc.)

Examples:
```bash
datum import -o project -f yolo -i <path/to/yolo/dataset>
datum export -p project -f voc -o <path/to/output/voc/dataset>
```

```bash
datum convert -if yolo -i <path/to/yolo/dataset> \
-f coco_instances -o <path/to/output/coco/dataset>
```

## Export to YOLO format

Datumaro can convert an existing dataset to YOLO format,
if the dataset supports object detection task.

Example:

```
datum import -p project -f coco_instances -i <path/to/coco/dataset>
datum export -p project -f yolo -o <path/to/output/yolo/dataset> -- --save-images
```

Extra options for export to YOLO format:

- `--save-images` allow to export dataset with saving images
(default: `False`);
- `--image-ext <IMAGE_EXT>` allow to specify image extension
for exporting dataset (default: use original or `.jpg`, if none).

## Particular use cases

### How to prepare PASCAL VOC dataset for exporting to YOLO format dataset?

```bash
datum import -o project -f voc -i ./VOC2012
datum filter -p project -e '/item[subset="train" or subset="val"]' -o trainval_voc
datum transform -p trainval_voc -o trainvalid_voc \
-t map_subsets -- -s train:train -s val:valid
datum export -p trainvalid_voc -f yolo -o ./yolo_dataset -- --save-images
```

### How to remove some class from YOLO dataset?
Delete all items, which contain `cat` objects and remove
`cat` from list of classes:
```bash
datum import -o project -f yolo -i ./yolo_dataset
datum filter -p project -o filtered -m i+a -e '/item/annotation[label!="cat"]'
datum transform -p filtered -o without_cat -t remap_labels -- -l cat:
datum export -p without_cat -f yolo -o ./yolo_without_cats
```

### How to create custom dataset in YOLO format?
```python
import numpy as np
from datumaro.components.dataset import Dataset
from datumaro.components.extractor import Bbox, DatasetItem

dataset = Dataset.from_iterable([
DatasetItem(id='image_001', subset='train',
image=np.ones((20, 20, 3)),
annotations=[
Bbox(3.0, 1.0, 8.0, 5.0, label=1),
Bbox(1.0, 1.0, 10.0, 1.0, label=2)
]
),
DatasetItem(id='image_002', subset='train',
image=np.ones((15, 10, 3)),
annotations=[
Bbox(4.0, 4.0, 4.0, 4.0, label=3)
]
)
], categories=['house', 'bridge', 'crosswalk', 'traffic_light'])

dataset.export('../yolo_dataset', format='yolo', save_images=True)
```

### How to get information about objects on each images?

If you only want information about label names for each
images, then you can get it from code:
```python
from datumaro.components.dataset import Dataset
from datumaro.components.extractor import AnnotationType

dataset = Dataset.import_from('./yolo_dataset', format='yolo')
cats = dataset.categories()[AnnotationType.label]

for item in dataset:
for ann in item.annotations:
print(item.id, cats[ann.label].name)
```

And If you want complete information about each items you can run:
```bash
datum import -o project -f yolo -i ./yolo_dataset
datum filter -p project --dry-run -e '/item'
```
1 change: 1 addition & 0 deletions docs/user_manual.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ List of supported formats:
- YOLO (`bboxes`)
- [Format specification](https://github.com/AlexeyAB/darknet#how-to-train-pascal-voc-data)
- [Dataset example](../tests/assets/yolo_dataset)
- [Format documentation](./formats/yolo_user_manual.md)
- TF Detection API (`bboxes`, `masks`)
- Format specifications: [bboxes](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/using_your_own_dataset.md), [masks](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/instance_segmentation.md)
- [Dataset example](../tests/assets/tf_detection_api_dataset)
Expand Down
Binary file added tests/assets/voc_dataset/voc_dataset1/JPEGImages/2007_000001.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
157 changes: 157 additions & 0 deletions tests/cli/test_yolo_format.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
import numpy as np
import os.path as osp

from unittest import TestCase

from datumaro.cli.__main__ import main
from datumaro.components.dataset import Dataset
from datumaro.components.extractor import (DatasetItem,
AnnotationType, Bbox)
from datumaro.util.test_utils import TestDir, compare_datasets
import datumaro.plugins.voc_format.format as VOC

def run(test, *args, expected_code=0):
test.assertEqual(expected_code, main(args), str(args))

class YoloIntegrationScenarios(TestCase):
def test_can_save_and_load_yolo_dataset(self):
target_dataset = Dataset.from_iterable([
DatasetItem(id='1', subset='train',
image=np.ones((10, 15, 3)),
annotations=[
Bbox(3.0, 3.0, 2.0, 3.0, label=4),
Bbox(0.0, 2.0, 4.0, 2.0, label=2)
]
)
], categories=['label_' + str(i) for i in range(10)])

with TestDir() as test_dir:
yolo_dir = osp.join(__file__[:__file__.rfind(osp.join('tests', ''))],
'tests', 'assets', 'yolo_dataset')

run(self, 'import', '-o', test_dir, '-f', 'yolo', '-i', yolo_dir)

export_dir = osp.join(test_dir, 'export_dir')
run(self, 'export', '-p', test_dir, '-o', export_dir,
'-f', 'yolo', '--', '--save-images')

parsed_dataset = Dataset.import_from(export_dir, format='yolo')
compare_datasets(self, target_dataset, parsed_dataset)

def test_can_export_mot_as_yolo(self):
target_dataset = Dataset.from_iterable([
DatasetItem(id='1', subset='train',
annotations=[
Bbox(0.0, 4.0, 4.0, 8.0, label=2)
]
)
], categories=['label_' + str(i) for i in range(10)])

with TestDir() as test_dir:
mot_dir = osp.join(__file__[:__file__.rfind(osp.join('tests', ''))],
'tests', 'assets', 'mot_dataset')

run(self, 'create', '-o', test_dir)
run(self, 'add', 'path', '-p', test_dir, '-f', 'mot_seq', mot_dir)

yolo_dir = osp.join(test_dir, 'yolo_dir')
run(self, 'export', '-p', test_dir, '-o', yolo_dir,
'-f', 'yolo', '--', '--save-images')

parsed_dataset = Dataset.import_from(yolo_dir, format='yolo')
compare_datasets(self, target_dataset, parsed_dataset)

def test_can_convert_voc_to_yolo(self):
target_dataset = Dataset.from_iterable([
DatasetItem(id='2007_000001', subset='train',
annotations=[
Bbox(8.0, 2.5, 4.0, 1.0, label=15),
Bbox(2.0, 1.0, 4.0, 1.0, label=8),
Bbox(11.0, 3.0, 4.0, 1.0, label=22)
]
)
], categories=[label.name for label in
VOC.make_voc_categories()[AnnotationType.label]])

with TestDir() as test_dir:
voc_dir = osp.join(__file__[:__file__.rfind(osp.join('tests', ''))],
'tests', 'assets', 'voc_dataset', 'voc_dataset1')
yolo_dir = osp.join(test_dir, 'yolo_dir')

run(self, 'convert', '-if', 'voc', '-i', voc_dir,
'-f', 'yolo', '-o', yolo_dir, '--', '--save-images')

parsed_dataset = Dataset.import_from(yolo_dir, format='yolo')
compare_datasets(self, target_dataset, parsed_dataset)

def test_can_ignore_non_supported_subsets(self):
source_dataset = Dataset.from_iterable([
DatasetItem(id='img1', subset='test',
image=np.ones((10, 20, 3)),
annotations=[
Bbox(1.0, 2.0, 1.0, 1.0, label=0)
]
),
DatasetItem(id='img2', subset='train',
image=np.ones((10, 5, 3)),
annotations=[
Bbox(3.0, 1.0, 2.0, 1.0, label=1)
]
)
], categories=[str(i) for i in range(4)])

target_dataset = Dataset.from_iterable([
DatasetItem(id='img2', subset='train',
image=np.ones((10, 5, 3)),
annotations=[
Bbox(3.0, 1.0, 2.0, 1.0, label=1)
]
)
], categories=[str(i) for i in range(4)])

with TestDir() as test_dir:
dataset_dir = osp.join(test_dir, 'dataset_dir')
source_dataset.save(dataset_dir, save_images=True)

run(self, 'create', '-o', test_dir)
run(self, 'add', 'path', '-p', test_dir, '-f', 'datumaro', dataset_dir)

yolo_dir = osp.join(test_dir, 'yolo_dir')
run(self, 'export', '-p', test_dir, '-o', yolo_dir,
'-f', 'yolo', '--', '--save-images')

parsed_dataset = Dataset.import_from(yolo_dir, format='yolo')
compare_datasets(self, target_dataset, parsed_dataset)

def test_can_delete_labels_from_yolo_dataset(self):
target_dataset = Dataset.from_iterable([
DatasetItem(id='1', subset='train',
image=np.ones((10, 15, 3)),
annotations=[
Bbox(0.0, 2.0, 4.0, 2.0, label=0)
]
)
], categories=['label_2'])

with TestDir() as test_dir:
yolo_dir = osp.join(__file__[:__file__.rfind(osp.join('tests', ''))],
'tests', 'assets', 'yolo_dataset')

run(self, 'create', '-o', test_dir)
run(self, 'add', 'path', '-p', test_dir, '-f', 'yolo', yolo_dir)

filtered_path = osp.join(test_dir, 'filtered')
run(self, 'filter', '-p', test_dir, '-o', filtered_path,
'-m', 'i+a', '-e', "/item/annotation[label='label_2']")

result_path = osp.join(test_dir, 'result')
run(self, 'transform', '-p', filtered_path, '-o', result_path,
'-t', 'remap_labels', '--', '-l', 'label_2:label_2',
'--default', 'delete')

export_dir = osp.join(test_dir, 'export')
run(self, 'export', '-p', result_path, '-o', export_dir,
'-f', 'yolo', '--', '--save-image')

parsed_dataset = Dataset.import_from(export_dir, format='yolo')
compare_datasets(self, target_dataset, parsed_dataset)

0 comments on commit 6782295

Please sign in to comment.