Relate points with boxes for pose estimation

[sramirez]

Hello,

Is there any way to associate boxes with keypoints following the COCO keypoint detection format?
For instance, I would like to associate person with the face point so that after exporting to COCO format the point and the box are associated to the same instance ID. However, neither they are associated nor the point even appear in the JSON file.

I've been reading in the issues track about how CVAT faces pose detection problem, but not sure if funcionality is now being developed or not. Seems Datamuro deals witht the export problem, but no solution for relating detection (boxes) with keypoints.

[zhiltsov-max]

@sramirez, hi. While not provided in CVAT UI (i.e. CVAT does not support COCO keypoints officially), it can be done with Datumaro. It exports annotations of the points type as COCO keypoints, ids are assigned automatically. If there are grouped rectangles and points, then rectangles are used as bboxes for points, otherwise bboxes are calculated from points.
Be aware that COCO categories will only contain label names and no point names (keypoints and skeleton fields). Points should be created in the same order to make sense in COCO format.

# export the task as Datumaro dataset
# unzip, go to export dir
# read README.md for setup instructions
datum project export -f coco (or coco_person_keypoints to be specific) [-- --save-images (if you like)]

If you have any ideas on improvement - you are welcome.

[sramirez]

From that, I thus understand it's not possible to create a bbox and associate it to a subsequent set of keypoints, right? This is a requirement for my pose estimation task, bc I need to distinguish for each body point if it belongs to one box or another (alike people).

[zhiltsov-max]

Join points and boxes into a group.

[sramirez]

Still not valid bc some points could be skipped, thus breaking the correct point order.

[nmanovic]

@sramirez ,

CVAT has group feature. You can assign the same group to different objects. For example, in CVAT XML format you will have the same group_id attribute for grouped objects. Will it solve the problem? If it doesn't please provide more details and context (what you need to get).

Documentation: https://github.com/opencv/cvat/blob/develop/cvat/apps/documentation/user_guide.md#shape-grouping

[bsekachev]

I will close the issue.
There is a similar issue about supporting better annotation scenario for this case #494