make formats docs folder, create COCO format documentation

CHANGELOG.md

@@ -14,6 +14,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Support COCO panoptic and stuff format (<https://github.com/openvinotoolkit/datumaro/pull/210>)
 - Documentation file and integration tests for Pascal VOC format (<https://github.com/openvinotoolkit/datumaro/pull/228>)
 - Support for MNIST and MNIST in CSV dataset formats (<https://github.com/openvinotoolkit/datumaro/pull/234>)
+- Documentation file for COCO format (<https://github.com/openvinotoolkit/datumaro/pull/241>)
 
 ### Changed
 - LabelMe format saves dataset items with their relative paths by subsets without changing names (<https://github.com/openvinotoolkit/datumaro/pull/200>)

docs/formats/coco_user_manual.md

@@ -0,0 +1,177 @@
+# COCO user manual
+
+## Contents
+
+- [Format specification](#format-specification)
+- [Load COCO dataset](#load-COCO-dataset)
+- [Export to other formats](#export-to-other-formats)
+- [Export to COCO](#export-to-COCO)
+- [Particular use cases](#particular-use-cases)
+
+## Format specification
+
+COCO format specification available [here](https://cocodataset.org/#format-data).
+
+COCO dataset format supports `captions`, `image_info`, `instances`, `panoptic`,
+`person_keypoints`, `stuff` annotation tasks
+and, as Datumaro extension, `label` (like `instances` with only `category_id`)
+
+##  Load COCO dataset
+
+The COCO dataset is available for free download:
+
+Images:
+- [train images](http://images.cocodataset.org/zips/train2017.zip)
+- [val images](http://images.cocodataset.org/zips/val2017.zip)
+- [test images](http://images.cocodataset.org/zips/test2017.zip)
+- [unlabeled images](http://images.cocodataset.org/zips/unlabeled2017.zip)
+
+Annotations:
+- [captions](http://images.cocodataset.org/annotations/annotations_trainval2017.zip)
+- [image_info](http://images.cocodataset.org/annotations/image_info_test2017.zip)
+- [instances](http://images.cocodataset.org/annotations/annotations_trainval2017.zip)
+- [panoptic](http://images.cocodataset.org/annotations/panoptic_annotations_trainval2017.zip)
+- [person_keypoints](http://images.cocodataset.org/annotations/annotations_trainval2017.zip)
+- [stuff](http://images.cocodataset.org/annotations/stuff_annotations_trainval2017.zip)
+
+There are two ways to create Datumaro project and add COCO dataset to it:
+
+``` bash
+datum import --format coco --input-path <path/to/dataset>
+# or
+datum create
+datum add path -f coco <path/to/dataset>
+```
+
+It is possible to specify project name and project directory run
+`datum create --help` for more information.
+
+COCO dataset directory should have the following structure:
+
+<!--lint disable fenced-code-flag-->
+```
+└─ Dataset/
+    ├── images/
+    │   ├── train<year>
+    │   │   ├── <image_name1.ext>
+    │   │   ├── <image_name2.ext>
+    │   │   └── ...
+    │   ├── val<year>
+    │   │   ├── <image_name1.ext>
+    │   │   ├── <image_name2.ext>
+    │   │   └── ...
+    ├── annotations/
+    │   └── <tasks>_train<year>.json
+    │   └── <tasks>_test<year>.json
+```
+
+For `panoptic` COCO dataset directory should have the following structure:
+
+<!--lint disable fenced-code-flag-->
+```
+└─ Dataset/
+    ├── images/
+    │   ├── train<year>
+    │   │   ├── <image_name1.ext>
+    │   │   ├── <image_name2.ext>
+    │   │   └── ...
+    │   ├── val<year>
+    │   │   ├── <image_name1.ext>
+    │   │   ├── <image_name2.ext>
+    │   │   └── ...
+    ├── annotations/
+    │   ├── panoptic_train<year>
+    │   │   ├── <image_name1.ext>
+    │   │   ├── <image_name2.ext>
+    │   │   └── ...
+    │   ├── panoptic_train<year>.json
+    │   ├── panoptic_val<year>
+    │   │   ├── <image_name1.ext>
+    │   │   ├── <image_name2.ext>
+    │   │   └── ...
+    │   └── panoptic_val<year>.json
+```
+
+You can import dataset for specific tasks
+of COCO dataset instead of the whole dataset,
+for example:
+
+``` bash
+datum import --format coco_stuff --input-path <path/to/stuff.json>
+```
+
+Datumaro supports the following COCO tasks:
+- [Image Captioning](https://cocodataset.org/#captions-2015) (`coco_caption`)
+- [Object Detection](https://cocodataset.org/#detection-2020) (`coco_instances`, `coco_label`)
+- [Panoptic Segmentation](https://cocodataset.org/#panoptic-2020) (`coco_panoptic`)
+- [Keypoint Detection](https://cocodataset.org/#keypoints-2020) (`coco_person_keypoints`)
+- [Stuff Segmentation](https://cocodataset.org/#stuff-2019) (`coco_stuff`)
+
+To make sure that the selected dataset has been added to the project, you can run
+`datum info`, which will display the project and dataset information.
+
+##  Export to other formats
+
+Datumaro can convert COCO dataset into any other format [Datumaro supports](../user_manual.md#supported-formats).
+To get the expected result, the dataset needs to be converted to formats
+that support the specified task (e.g. for panoptic segmentation - VOC, CamVID)
+There are few ways to convert COCO dataset to other dataset format:
+
+``` bash
+datum project import -f coco -i <path/to/coco>
+datum export -f voc -o <path/to/output/dir>
+# or
+datum convert -if coco -i <path/to/coco> -f voc -o <path/to/output/dir>
+```
+
+Some formats provide extra options for conversion.
+These options are passed after double dash (`--`) in the command line.
+To get information about them, run
+
+`datum export -f <FORMAT> -- -h`
+
+##  Export to COCO
+
+There are few ways to convert dataset to COCO format:
+
+``` bash
+# export dataset into COCO format from existing project
+datum export -p <path/to/project> -f coco -o <path/to/export/dir> \
+    -- --save-images
+# converting to COCO format from other format
+datum convert -if voc -i <path/to/voc/dataset> \
+    -f coco -o <path/to/export/dir> -- --save-images
+```
+
+Extra options for export to COCO format:
+- `--save-images` allow to export dataset with saving images
+(by default `False`);
+- `--image-ext IMAGE_EXT` allow to specify image extension
+for exporting dataset (by default `.png`);
+- `--segmentation-mode MODE` allow to specify save mode for instance segmentation:
+    - 'guess': guess the mode for each instance (using 'is_crowd' attribute as hint)
+    - 'polygons': save polygons( merge and convert masks, prefer polygons)
+    - 'mask': save masks (merge and convert polygons, prefer masks)
+(by default `guess`);
+- `--crop-covered` allow torop covered segments so that background objects
+segmentation was more accurate (by default `False`);
+- `--allow-attributes ALLOW_ATTRIBUTES` allow export of attributes
+(by default `True`);
+- `--reindex REINDEX` allow to assign new indices to images and annotations,
+useful to avoid merge conflicts (by default `False`);
+- `--merge-images` allow to save all images into a single directory
+(by default `False`);
+- `--tasks TASKS` allow to specify tasks for export dataset,
+by default Datumaro uses all tasks. Example:
+
+```bash
+datum import -o project -f coco -i <dataset>
+datum export -p project -f coco -- --tasks instances,stuff
+```
+
+## Particular use cases
+
+Datumaro supports filtering, transformation, merging etc. for all formats
+and for the COCO format in particular. Follow
+[user manual](../user_manual.md)
+to get more information about these operations.

docs/mnist_user_manual.md → docs/formats/mnist_user_manual.md

@@ -99,7 +99,7 @@ MNIST format only supports single channel 28 x 28 images.
 
 ##  Export to other formats
 
-Datumaro can convert MNIST dataset into any other format [Datumaro supports](../docs/user_manual.md#supported-formats).
+Datumaro can convert MNIST dataset into any other format [Datumaro supports](../user_manual.md#supported-formats).
 To get the expected result, the dataset needs to be converted to formats
 that support the classification task (e.g. CIFAR-10/100, ImageNet, PascalVOC, etc.)
 There are few ways to convert MNIST dataset to other dataset format:
@@ -138,7 +138,7 @@ These commands also work for MNIST in CSV if you use `mnist_csv` instead of `mni
 ##  Particular use cases
 
 Datumaro supports filtering, transformation, merging etc. for all formats
-and for the MNIST format in particular. Follow [user manual](../docs/user_manual.md)
+and for the MNIST format in particular. Follow [user manual](../user_manual.md)
 to get more information about these operations.
 
 There are few examples of using Datumaro operations to solve

docs/pascal_voc_user_manual.md → docs/formats/pascal_voc_user_manual.md

@@ -136,7 +136,7 @@ To make sure that the selected dataset has been added to the project, you can ru
 ## Export to other formats
 
 Datumaro can convert Pascal VOC dataset into any other format
-[Datumaro supports](../docs/user_manual.md#supported-formats).
+[Datumaro supports](../user_manual.md#supported-formats).
 
 Such conversion will only be successful if the output
 format can represent the type of dataset you want to convert,
@@ -211,7 +211,7 @@ datum export -f voc_segmentation -- --label-map voc
 
 Datumaro supports filtering, transformation, merging etc. for all formats
 and for the Pascal VOC format in particular. Follow
-[user manual](../docs/user_manual.md)
+[user manual](../user_manual.md)
 to get more information about these operations.
 
 There are few examples of using Datumaro operations to solve

docs/user_manual.md

@@ -89,10 +89,11 @@ List of supported formats:
   - [Format specification](http://cocodataset.org/#format-data)
   - [Dataset example](../tests/assets/coco_dataset)
   - `labels` are our extension - like `instances` with only `category_id`
+  - [Format documentation](./formats/coco_user_manual.md)
 - PASCAL VOC (`classification`, `detection`, `segmentation` (class, instances), `action_classification`, `person_layout`)
   - [Format specification](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/htmldoc/index.html)
   - [Dataset example](../tests/assets/voc_dataset)
-  - [Format documentation](./pascal_voc_user_manual.md)
+  - [Format documentation](./formats/pascal_voc_user_manual.md)
 - YOLO (`bboxes`)
   - [Format specification](https://github.com/AlexeyAB/darknet#how-to-train-pascal-voc-data)
   - [Dataset example](../tests/assets/yolo_dataset)
@@ -121,11 +122,11 @@ List of supported formats:
 - MNIST (`classification`)
   - [Format specification](http://yann.lecun.com/exdb/mnist/)
   - [Dataset example](../tests/assets/mnist_dataset)
-  - [Format documentation](./mnist_user_manual.md)
+  - [Format documentation](./formats/mnist_user_manual.md)
 - MNIST in CSV (`classification`)
   - [Format specification](https://pjreddie.com/projects/mnist-in-csv/)
   - [Dataset example](../tests/assets/mnist_csv_dataset)
-  - [Format documentation](./mnist_user_manual.md)
+  - [Format documentation](./formats/mnist_user_manual.md)
 - CamVid (`segmentation`)
   - [Format specification](http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/)
   - [Dataset example](../tests/assets/camvid_dataset)