Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] enable CocoMetric to use bbox score in pred_instances #2145

Merged
merged 3 commits into from
Apr 4, 2023
+29 −18
Merged

[Fix] enable CocoMetric to use bbox score in pred_instances #2145

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
acf63f2
fix CocoMetric and update yolo-pose results
Ben-Louis Apr 4, 2023
295308a
refine code of visualizer
Ben-Louis Apr 4, 2023
c09fc5f
inferencer support filtering predicted instance via bbox_thr
Ben-Louis Apr 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions demo/docs/2d_human_pose_demo.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py \
https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth \
https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192-81c58e40_20220909.pth \
--input tests/data/posetrack18/videos/000001_mpiinew_test/000001_mpiinew_test.mp4 \
--output-root=vis_results/demo --show --draw-heatmap
```
Expand All @@ -116,7 +116,7 @@ Example:
```shell
python demo/bottomup_demo.py \
configs/body_2d_keypoint/dekr/coco/dekr_hrnet-w32_8xb10-140e_coco-512x512.py \
https://download.openmmlab.com/mmpose/bottom_up/dekr/hrnet_w32_coco_512x512-2a3056de_20220928.pth \
https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/dekr/coco/dekr_hrnet-w32_8xb10-140e_coco-512x512_ac7c17bf-20221228.pth \
--input tests/data/coco/000000197388.jpg --output-root=vis_results \
--show --save-predictions
```
Expand Down
2 changes: 2 additions & 0 deletions mmpose/apis/inferencers/mmpose_inferencer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,8 @@ def __call__(
inputs = self.preprocess(
inputs, batch_size=batch_size, **preprocess_kwargs)

forward_kwargs['bbox_thr'] = preprocess_kwargs.get('bbox_thr', -1)

preds = []
if 'pose2d' not in self.mode or not hasattr(self.pose2d_inferencer,
'detector'):
Expand Down
8 changes: 7 additions & 1 deletion mmpose/apis/inferencers/pose2d_inferencer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,10 +184,15 @@ def preprocess_single(self,

return data_infos

def forward(self, inputs: Union[dict, tuple]):
def forward(self, inputs: Union[dict, tuple], bbox_thr=-1):
data_samples = super().forward(inputs)
if self.cfg.data_mode == 'topdown':
data_samples = [merge_data_samples(data_samples)]
if bbox_thr > 0:
for ds in data_samples:
if 'bbox_scores' in ds.pred_instances:
ds.pred_instances = ds.pred_instances[
ds.pred_instances.bbox_scores > bbox_thr]
return data_samples

def __call__(
Expand Down Expand Up @@ -241,6 +246,7 @@ def __call__(
else:
inputs = self._inputs_to_list(inputs)

forward_kwargs['bbox_thr'] = preprocess_kwargs.get('bbox_thr', -1)
inputs = self.preprocess(
inputs, batch_size=batch_size, **preprocess_kwargs)

Expand Down
12 changes: 9 additions & 3 deletions mmpose/evaluation/metrics/coco_metric.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,13 +178,19 @@ def process(self, data_batch: Sequence[dict],
pred['keypoint_scores'] = keypoint_scores
pred['category_id'] = data_sample.get('category_id', 1)

if ('bbox_scores' not in data_sample['gt_instances']
or len(data_sample['gt_instances']['bbox_scores']) !=
len(keypoints)):
if 'bbox_scores' in data_sample['pred_instances']:
# some one-stage models will predict bboxes and scores
# together with keypoints
bbox_scores = data_sample['pred_instances']['bbox_scores']
elif ('bbox_scores' not in data_sample['gt_instances']
or len(data_sample['gt_instances']['bbox_scores']) !=
len(keypoints)):
# bottom-up models might output different number of
# instances from annotation
bbox_scores = np.ones(len(keypoints))
else:
# top-down models use detected bboxes, the scores of which
# are contained in the gt_instances
bbox_scores = data_sample['gt_instances']['bbox_scores']
pred['bbox_scores'] = bbox_scores

Expand Down
9 changes: 3 additions & 6 deletions mmpose/visualization/local_visualizer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,12 +264,9 @@ def _draw_instances_kpts(self,
# compute neck joint
neck = np.mean(keypoints_info[:, [5, 6]], axis=1)
# neck score when visualizing pred
neck[:, 2] = 1 - np.logical_or(
keypoints_info[:, 5, 2] < kpt_thr,
keypoints_info[:, 6, 2] < kpt_thr).astype(int)
neck[:, 3] = 1 - np.logical_or(
keypoints_info[:, 5, 3] < kpt_thr,
keypoints_info[:, 6, 3] < kpt_thr).astype(int)
neck[:, 2:4] = np.logical_and(
keypoints_info[:, 5, 2:4] < kpt_thr,
keypoints_info[:, 6, 2:4] < kpt_thr).astype(int)
new_keypoints_info = np.insert(
keypoints_info, 17, neck, axis=1)

Expand Down
8 changes: 4 additions & 4 deletions projects/yolox-pose/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,10 +91,10 @@ Results on COCO val2017

| Model | Input Size | AP | AP<sup>50</sup> | AP<sup>75</sup> | AR | AR<sup>50</sup> | Download |
| :-------------------------------------------------------------: | :--------: | :---: | :-------------: | :-------------: | :---: | :-------------: | :----------------------------------------------------------------------: |
| [YOLOX-tiny-Pose](./configs/yolox-pose_tiny_4xb64-300e_coco.py) | 640 | 0.477 | 0.756 | 0.506 | 0.547 | 0.802 | [model](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_tiny_4xb64-300e_coco-c47dd83b_20230321.pth) \| [log](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_tiny_4xb64-300e_coco_20230321.json) |
| [YOLOX-s-Pose](./configs/yolox-pose_s_8xb32-300e_coco.py) | 640 | 0.595 | 0.836 | 0.653 | 0.658 | 0.878 | [model](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_s_8xb32-300e_coco-9f5e3924_20230321.pth) \| [log](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_s_8xb32-300e_coco_20230321.json) |
| [YOLOX-m-Pose](./configs/yolox-pose_m_4xb64-300e_coco.py) | 640 | 0.659 | 0.870 | 0.729 | 0.713 | 0.903 | [model](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_m_4xb64-300e_coco-cbd11d30_20230321.pth) \| [log](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_m_4xb64-300e_coco_20230321.json) |
| [YOLOX-l-Pose](./configs/yolox-pose_l_4xb64-300e_coco.py) | 640 | 0.679 | 0.882 | 0.749 | 0.733 | 0.911 | [model](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_l_4xb64-300e_coco-122e4cf8_20230321.pth) \| [log](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_l_4xb64-300e_coco_20230321.json) |
| [YOLOX-tiny-Pose](./configs/yolox-pose_tiny_4xb64-300e_coco.py) | 640 | 0.518 | 0.799 | 0.545 | 0.566 | 0.841 | [model](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_tiny_4xb64-300e_coco-c47dd83b_20230321.pth) \| [log](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_tiny_4xb64-300e_coco_20230321.json) |
| [YOLOX-s-Pose](./configs/yolox-pose_s_8xb32-300e_coco.py) | 640 | 0.632 | 0.875 | 0.692 | 0.676 | 0.907 | [model](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_s_8xb32-300e_coco-9f5e3924_20230321.pth) \| [log](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_s_8xb32-300e_coco_20230321.json) |
| [YOLOX-m-Pose](./configs/yolox-pose_m_4xb64-300e_coco.py) | 640 | 0.685 | 0.897 | 0.753 | 0.727 | 0.925 | [model](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_m_4xb64-300e_coco-cbd11d30_20230321.pth) \| [log](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_m_4xb64-300e_coco_20230321.json) |
| [YOLOX-l-Pose](./configs/yolox-pose_l_4xb64-300e_coco.py) | 640 | 0.706 | 0.907 | 0.775 | 0.747 | 0.934 | [model](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_l_4xb64-300e_coco-122e4cf8_20230321.pth) \| [log](https://download.openmmlab.com/mmpose/v1/projects/yolox-pose/yolox-pose_l_4xb64-300e_coco_20230321.json) |

We have only trained models with an input size of 640, as we couldn't replicate the performance enhancement mentioned in the paper when increasing the input size from 640 to 960. We warmly welcome any contributions if you can successfully reproduce the results from the paper!

Expand Down
4 changes: 2 additions & 2 deletions projects/yolox-pose/configs/yolox-pose_s_8xb32-300e_coco.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@
test_cfg=dict(
yolox_style=True,
multi_label=False,
score_thr=0.2,
score_thr=0.001,
max_per_img=300,
nms=dict(type='nms', iou_threshold=0.65)))

Expand Down Expand Up @@ -189,7 +189,7 @@
val_evaluator = dict(
type='mmpose.CocoMetric',
ann_file=data_root + 'annotations/person_keypoints_val2017.json',
)
score_mode='bbox')
test_evaluator = val_evaluator

default_hooks = dict(checkpoint=dict(save_best='coco/AP', rule='greater'))
Expand Down