Skip to content

Commit

Permalink
[MMSIG-77] Translate demo/docs (open-mmlab#2228)
Browse files Browse the repository at this point in the history
  • Loading branch information
xin-li-67 authored and Tau-J committed Apr 25, 2023
1 parent 3d4ba0b commit bbe2ece
Show file tree
Hide file tree
Showing 25 changed files with 760 additions and 45 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ https://user-images.githubusercontent.com/15977946/124654387-0fd3c500-ded1-11eb-
- **Support diverse tasks**

We support a wide spectrum of mainstream pose analysis tasks in current research community, including 2d multi-person human pose estimation, 2d hand pose estimation, 2d face landmark detection, 133 keypoint whole-body human pose estimation, 3d human mesh recovery, fashion landmark detection and animal pose estimation.
See [Demo](demo/docs/) for more information.
See [Demo](demo/docs/en) for more information.

- **Higher efficiency and higher accuracy**

Expand Down
2 changes: 1 addition & 1 deletion README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ https://user-images.githubusercontent.com/15977946/124654387-0fd3c500-ded1-11eb-
- **支持多种人体姿态分析相关任务**

MMPose 支持当前学界广泛关注的主流姿态分析任务:主要包括 2D多人姿态估计、2D手部姿态估计、2D人脸关键点检测、133关键点的全身人体姿态估计、3D人体形状恢复、服饰关键点检测、动物关键点检测等。
具体请参考 [功能演示](demo/docs/)
具体请参考 [功能演示](demo/docs/zh_cn/)

- **更高的精度和更快的速度**

Expand Down
2 changes: 1 addition & 1 deletion configs/animal_2d_keypoint/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Please follow [DATA Preparation](/docs/en/dataset_zoo/2d_animal_keypoint.md) to

## Demo

Please follow [DEMO](/demo/docs/2d_animal_demo.md) to generate fancy demos.
Please follow [DEMO](/demo/docs/en/2d_animal_demo.md) to generate fancy demos.

<img src="https://user-images.githubusercontent.com/26127467/187655602-907db86e-710b-447a-8ec9-5b623d43d160.gif" height="500px" alt><br>

Expand Down
2 changes: 1 addition & 1 deletion configs/body_2d_keypoint/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Please follow [DATA Preparation](/docs/en/dataset_zoo/2d_body_keypoint.md) to pr

## Demo

Please follow [Demo](/demo/docs/2d_human_pose_demo.md#2d-human-pose-demo) to run demos.
Please follow [Demo](/demo/docs/en/2d_human_pose_demo.md#2d-human-pose-demo) to run demos.

<div align=center>
<img src="https://user-images.githubusercontent.com/87690686/187824368-1f1631c3-52bf-4b45-bf9a-a70cd6551e1a.jpg" height="500px" alt><br>
Expand Down
2 changes: 1 addition & 1 deletion configs/body_3d_keypoint/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ Please follow [DATA Preparation](/docs/en/dataset_zoo/3d_body_keypoint.md) to pr

## Demo

Please follow [Demo](/demo/docs/3d_human_pose_demo.md) to run demos.
Please follow [Demo](/demo/docs/en/3d_human_pose_demo.md) to run demos.

<img src="https://user-images.githubusercontent.com/15977946/118820606-02df2000-b8e9-11eb-9984-b9228101e780.gif" width="600px" alt><br>
2 changes: 1 addition & 1 deletion configs/face_2d_keypoint/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,6 @@ Please follow [DATA Preparation](/docs/en/dataset_zoo/2d_face_keypoint.md) to pr

## Demo

Please follow [Demo](/demo/docs/2d_face_demo.md) to run demos.
Please follow [Demo](/demo/docs/en/2d_face_demo.md) to run demos.

<img src="https://user-images.githubusercontent.com/11788150/109144943-ccd44900-779c-11eb-9e9d-8682e7629654.gif" width="600px" alt><br>
2 changes: 1 addition & 1 deletion configs/hand_2d_keypoint/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Please follow [DATA Preparation](/docs/en/dataset_zoo/2d_hand_keypoint.md) to pr

## Demo

Please follow [Demo](/demo/docs/2d_hand_demo.md) to run demos.
Please follow [Demo](/demo/docs/en/2d_hand_demo.md) to run demos.

<img src="https://user-images.githubusercontent.com/11788150/109098558-8c54db00-775c-11eb-8966-85df96b23dc5.gif" width="600px" alt><br>

Expand Down
2 changes: 1 addition & 1 deletion configs/hand_gesture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ Please follow [DATA Preparation](/docs/en/dataset_zoo/2d_hand_gesture.md) to pre

## Demo

Please follow [Demo](/demo/docs/gesture_recognition_demo.md) to run the demo.
Please follow [Demo](/demo/docs/en/gesture_recognition_demo.md) to run the demo.

<img src="https://user-images.githubusercontent.com/15977946/172213082-afb9d71a-f2df-4509-932c-e47dc61ec7d7.gif" width="600px" alt>
2 changes: 1 addition & 1 deletion configs/wholebody_2d_keypoint/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@ Please follow [DATA Preparation](/docs/en/dataset_zoo/2d_wholebody_keypoint.md)

## Demo

Please follow [Demo](/demo/docs/2d_wholebody_pose_demo.md) to run demos.
Please follow [Demo](/demo/docs/en/2d_wholebody_pose_demo.md) to run demos.

<img src="https://user-images.githubusercontent.com/9464825/95552839-00a61080-0a40-11eb-818c-b8dad7307217.gif" width="600px" alt><br>
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ The augement `--det-cat-id=15` selected detected bounding boxes with label 'cat'
**COCO-animals**
In COCO dataset, there are 80 object categories, including 10 common `animal` categories (14: 'bird', 15: 'cat', 16: 'dog', 17: 'horse', 18: 'sheep', 19: 'cow', 20: 'elephant', 21: 'bear', 22: 'zebra', 23: 'giraffe').

For other animals, we have also provided some pre-trained animal detection models (1-class models). Supported models can be found in [detection model zoo](/demo/docs/mmdet_modelzoo.md).
For other animals, we have also provided some pre-trained animal detection models (1-class models). Supported models can be found in [detection model zoo](/demo/docs/en/mmdet_modelzoo.md).

To save visualized results on disk:

Expand Down
6 changes: 3 additions & 3 deletions demo/docs/2d_face_demo.md → demo/docs/en/2d_face_demo.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
## 2D Face Keypoint Demo

We provide a demo script to test a single image or video with hand detectors and top-down pose estimators. Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection) with version >= 3.0.
We provide a demo script to test a single image or video with face detectors and top-down pose estimators. Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection) with version >= 3.0.

**Face Box Model Preparation:** The pre-trained face box estimation model can be found in [mmdet model zoo](/demo/docs/mmdet_modelzoo.md).
**Face Bounding Box Model Preparation:** The pre-trained face box estimation model can be found in [mmdet model zoo](/demo/docs/en/mmdet_modelzoo.md#face-bounding-box-detection-models).

### 2D Face Image Demo

Expand Down Expand Up @@ -98,4 +98,4 @@ In addition, the Inferencer supports saving predicted poses. For more informatio

### Speed Up Inference

For 2D face keypoint estimation models, try to edit the config file. For example, set `model.test_cfg.flip_test=False` in [aflw_hrnetv2](../../configs/face_2d_keypoint/topdown_heatmap/aflw/td-hm_hrnetv2-w18_8xb64-60e_aflw-256x256.py#90).
For 2D face keypoint estimation models, try to edit the config file. For example, set `model.test_cfg.flip_test=False` in line 90 of [aflw_hrnetv2](../../../configs/face_2d_keypoint/topdown_heatmap/aflw/td-hm_hrnetv2-w18_8xb64-60e_aflw-256x256.py).
3 changes: 1 addition & 2 deletions demo/docs/2d_hand_demo.md → demo/docs/en/2d_hand_demo.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

We provide a demo script to test a single image or video with hand detectors and top-down pose estimators. Assume that you have already installed [mmdet](https://github.com/open-mmlab/mmdetection) with version >= 3.0.

**Hand Box Model Preparation:** The pre-trained hand box estimation model can be found in [mmdet model zoo](/demo/docs/mmdet_modelzoo.md).
**Hand Box Model Preparation:** The pre-trained hand box estimation model can be found in [mmdet model zoo](/demo/docs/en/mmdet_modelzoo.md#hand-bounding-box-detection-models).

### 2D Hand Image Demo

Expand All @@ -14,7 +14,6 @@ python demo/topdown_demo_with_mmdet.py \
[--show] [--device ${GPU_ID or CPU}] [--save-predictions] \
[--draw-heatmap ${DRAW_HEATMAP}] [--radius ${KPT_RADIUS}] \
[--kpt-thr ${KPT_SCORE_THR}] [--bbox-thr ${BBOX_SCORE_THR}]

```

The pre-trained hand pose estimation model can be downloaded from [model zoo](https://mmpose.readthedocs.io/en/latest/model_zoo/hand_2d_keypoint.html).
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ MMDetection provides 80-class COCO-pretrained models, which already includes the

### Hand Bounding Box Detection Models

For hand bounding box detection, we simply train our hand box models on onehand10k dataset using MMDetection.
For hand bounding box detection, we simply train our hand box models on OneHand10K dataset using MMDetection.

#### Hand detection results on OneHand10K test set

Expand All @@ -19,7 +19,7 @@ For hand bounding box detection, we simply train our hand box models on onehand1

For face bounding box detection, we train a YOLOX detector on COCO-face data using MMDetection.

#### Hand detection results on OneHand10K test set
#### Face detection results on COCO-face test set

| Arch | Box AP | ckpt |
| :-------------------------------------------------------------- | :----: | :----------------------------------------------------------------------------------------------------: |
Expand All @@ -29,8 +29,6 @@ For face bounding box detection, we train a YOLOX detector on COCO-face data usi

#### COCO animals

<!-- TODO: update link to the doc of MMDetection 3.0 -->

In COCO dataset, there are 80 object categories, including 10 common `animal` categories (14: 'bird', 15: 'cat', 16: 'dog', 17: 'horse', 18: 'sheep', 19: 'cow', 20: 'elephant', 21: 'bear', 22: 'zebra', 23: 'giraffe')
For animals in the categories, please download from [MMDetection Model Zoo](https://mmdetection.readthedocs.io/en/3.x/model_zoo.html).

Expand Down
File renamed without changes.
124 changes: 124 additions & 0 deletions demo/docs/zh_cn/2d_animal_demo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
## 2D Animal Pose Demo

本系列文档我们会来介绍如何使用提供了的脚本进行完成基本的推理 demo ,本节先介绍如何对 top-down 结构和动物的 2D 姿态进行单张图片和视频推理,请确保你已经安装了 3.0 以上版本的 [MMDetection](https://github.com/open-mmlab/mmdetection)

### 2D 动物图片姿态识别推理

```shell
python demo/topdown_demo_with_mmdet.py \
${MMDET_CONFIG_FILE} ${MMDET_CHECKPOINT_FILE} \
${MMPOSE_CONFIG_FILE} ${MMPOSE_CHECKPOINT_FILE} \
--input ${INPUT_PATH} --det-cat-id ${DET_CAT_ID} \
[--show] [--output-root ${OUTPUT_DIR}] [--save-predictions] \
[--draw-heatmap ${DRAW_HEATMAP}] [--radius ${KPT_RADIUS}] \
[--kpt-thr ${KPT_SCORE_THR}] [--bbox-thr ${BBOX_SCORE_THR}] \
[--device ${GPU_ID or CPU}]
```

用户可以在 [model zoo](https://mmpose.readthedocs.io/zh_CN/dev-1.x/model_zoo/animal_2d_keypoint.html) 获取预训练好的关键点识别模型。

这里我们用 [animalpose model](https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth) 来进行演示:

```shell
python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
--input tests/data/animalpose/ca110.jpeg \
--show --draw-heatmap --det-cat-id=15
```

可视化结果如下:

<img src="https://user-images.githubusercontent.com/26127467/187644168-5915551a-0876-4b85-9454-7f92c84ba6fb.jpeg" height="500px" alt><br>

如果使用了 heatmap-based 模型同时设置了 `--draw-heatmap` ,预测的热图也会跟随关键点一同可视化出来。

`--det-cat-id=15` 参数用来指定模型只检测 `cat` 类型,这是基于 COCO 数据集的数据。

**COCO 数据集动物信息**

COCO 数据集共包含 80 个类别,其中有 10 种常见动物,类别如下:

(14: 'bird', 15: 'cat', 16: 'dog', 17: 'horse', 18: 'sheep', 19: 'cow', 20: 'elephant', 21: 'bear', 22: 'zebra', 23: 'giraffe')

对于其他类型的动物,我们也提供了一些训练好的动物检测模型,用户可以前往 [detection model zoo](/demo/docs/zh_cn/mmdet_modelzoo.md) 下载。

如果想本地保存可视化结果可使用如下命令:

```shell
python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
--input tests/data/animalpose/ca110.jpeg \
--output-root vis_results --draw-heatmap --det-cat-id=15
```

如果想本地保存预测结果,需要使用 `--save-predictions`

```shell
python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
--input tests/data/animalpose/ca110.jpeg \
--output-root vis_results --save-predictions --draw-heatmap --det-cat-id=15
```

仅使用 CPU:

```shell
python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
--input tests/data/animalpose/ca110.jpeg \
--show --draw-heatmap --det-cat-id=15 --device cpu
```

### 2D 动物视频姿态识别推理

视频和图片使用了同样的接口,区别在于视频推理时 `${INPUT_PATH}` 既可以是本地视频文件的路径也可以是视频文件的 **URL** 地址。

例如:

```shell
python demo/topdown_demo_with_mmdet.py \
demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
https://download.openmmlab.com/mmpose/animal/hrnet/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
--input demo/resources/<demo_dog.mp4> \
--output-root vis_results --draw-heatmap --det-cat-id=16
```

<img src="https://user-images.githubusercontent.com/26127467/187655602-907db86e-710b-447a-8ec9-5b623d43d160.gif" height="500px" alt><br>

这段视频可以在 [Google Drive](https://drive.google.com/file/d/18d8K3wuUpKiDFHvOx0mh1TEwYwpOc5UO/view?usp=sharing) 下载。

### 使用 Inferencer 进行 2D 动物姿态识别推理

Inferencer 提供一个更便捷的推理接口,使得用户可以绕过模型的配置文件和 checkpoint 路径直接使用 model aliases ,支持包括图片路径、视频路径、图片文件夹路径和 webcams 在内的多种输入方式,例如可以这样使用:

```shell
python demo/inferencer_demo.py tests/data/ap10k \
--pose2d animal --vis-out-dir vis_results/ap10k
```

该命令会对输入的 `tests/data/ap10k` 下所有的图片进行推理并且把可视化结果都存入 `vis_results/ap10k` 文件夹下。

<img src="https://user-images.githubusercontent.com/26127467/229789306-83ea56fa-12f2-4e27-9031-329d335ec26d.jpg" alt="Image 1" height="200"/> <img src="https://user-images.githubusercontent.com/26127467/229789324-7fef5688-422d-4663-a57c-d1e1d511e83c.jpg" alt="Image 2" height="200"/>

Inferencer 同样支持保存预测结果,更多的信息可以参考 [Inferencer 文档](https://mmpose.readthedocs.io/en/dev-1.x/user_guides/inference.html#inferencer-a-unified-inference-interface)

### 加速推理

用户可以通过修改配置文件来加速,更多具体例子可以参考:

1. 设置 `model.test_cfg.flip_test=False`,如 [animalpose_hrnet-w32](../../configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py#85) 所示。
2. 使用更快的 bounding box 检测器,可参考 [MMDetection](https://mmdetection.readthedocs.io/zh_CN/3.x/model_zoo.html)
Loading

0 comments on commit bbe2ece

Please sign in to comment.