[Feature] Support evaluating CocoMetric without ann_file.

[liqikai9]

Motivation

Support evaluating coco metric without ann_file thus other non-COCO format datasets can take CocoMetric as well.

Modification

1. Add the 'CLASSES' in the dataset metainfo to store the information about category.
2. Add 'raw_ann_info' key in the definition of BaseCocoStyleDataset
3. Add 'raw_ann_info' key in mmpose/datasets/transforms/formatting.py
4. Modify CocoMetric, add gt_to_coco_json function. Also fix the inheritance dependency of PoseTrack18Metric
5. Add unittests.

BC-breaking (Optional)

Use cases (Optional)

In order to use this function, we need to change some places:

1. Add 'raw_ann_info' to the data_info in the dataset to store the raw annotation information:

data_info = {
    'img_id': ann['image_id'],
    'img_path': img_path,
    'bbox': bbox,
    'bbox_score': np.ones(1, dtype=np.float32),
    'num_keypoints': num_keypoints,
    'keypoints': keypoints,
    'keypoints_visible': keypoints_visible,
    'iscrowd': ann.get('iscrowd', 0),
    'segmentation': ann.get('segmentation', None),
    'id': ann['id'],
    # store the raw annotation of the instance
    # it is useful for evaluation without providing ann_file
    'raw_ann_info': copy.deepcopy(ann),
}

2. Add 'raw_ann_info' to meta_keys so that the annotation can be readed in the CocoMetric

test_pipeline = [
    dict(type='LoadImageFromFile'),
    ...,
    dict(type='PackPoseInputs', meta_keys=('id',
            'img_id', 'img_path', 'ori_shape', 'img_shape', 'input_size',
            'flip', 'flip_direction', 'flip_indices', 'raw_ann_info'))
]

3. Remove the 'ann_file' in CocoMetric:

val_evaluator = dict(
    type='CocoMetric',
    # ann_file=data_root + 'annotations/person_keypoints_val2017.json',
)

Verification

I have tested on these following datasets and the results are exactly the same:

Comment some codes of mmpose/datasets/datasets/base/base_coco_style_dataset.py to omit the filtering:

1. CocoDataset: can only evaluate gt_bbox, because can not assign gt keypoints and bboxes information to each detection bbox. Besides, the filtering condition should be deleted, so that each gt instance will be inferenced. Under these conditions, the results with and without ann_file are exactly the same (with 11004 samples in total, which is equal to the number of samples before filtering).

Test command:

python tools/test.py configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth

With ann_file:

Without ann_file:

2. AnimalPose dataset: it only evaluates on gt_bbox so it can fit this function naturally. The results with and without ann_file are exactly the same (with 1117 samples in total).

Test command:

python tools/test.py configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth

With ann_file:

Without ann_file:

3. CrowdPose dataset: Fix some evaluation metric items, modified the _do_python_keypoint_eval function in CocoMetric.

Can only evaluate gt_bbox, because can not assign gt keypoints and bboxes information to each detection bbox. Besides, the filtering condition should be deleted, so that each gt instance will be inferenced. Under these conditions, the results with and without ann_file are exactly the same (with 35283 samples in total, which is equal to the number of samples before filtering).

Test command:

python tools/test.py configs/body_2d_keypoint/topdown_heatmap/crowdpose/td-hm_hrnet-w32_8xb64-210e_crowdpose-256x192.py https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_crowdpose_256x192-960be101_20201227.pth

With ann_file:

Without ann_file:

Checklist

Before PR:

• I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
• Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
• Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
• New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
• The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

• CLA has been signed and all committers have signed the CLA in this PR.