Merge 93119ae into dab2fdf

open-mmlab · May 15, 2023 · 7491e60 · 7491e60
2 parents dab2fdf + 93119ae
commit 7491e60
Show file tree

Hide file tree

Showing 48 changed files with 6,231 additions and 137 deletions.
diff --git a/configs/body_2d_keypoint/rtmpose/body8/rtmpose-l_8xb256-210e_body8-256x192.py b/configs/body_2d_keypoint/rtmpose/body8/rtmpose-l_8xb256-210e_body8-256x192.py
diff --git a/configs/body_2d_keypoint/rtmpose/body8/rtmpose-l_8xb256-210e_body8-384x288.py b/configs/body_2d_keypoint/rtmpose/body8/rtmpose-l_8xb256-210e_body8-384x288.py
diff --git a/configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-210e_body8-256x192.py b/configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-210e_body8-256x192.py
diff --git a/configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-210e_body8-384x288.py b/configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-210e_body8-384x288.py
diff --git a/configs/body_2d_keypoint/rtmpose/body8/rtmpose-s_8xb256-210e_body8-256x192.py b/configs/body_2d_keypoint/rtmpose/body8/rtmpose-s_8xb256-210e_body8-256x192.py
diff --git a/configs/body_2d_keypoint/rtmpose/body8/rtmpose-t_8xb256-210e_body8-256x192.py b/configs/body_2d_keypoint/rtmpose/body8/rtmpose-t_8xb256-210e_body8-256x192.py
diff --git a/configs/body_2d_keypoint/rtmpose/body8/rtmpose_body8.md b/configs/body_2d_keypoint/rtmpose/body8/rtmpose_body8.md
@@ -0,0 +1,76 @@
+<!-- [ALGORITHM] -->
+
+<details>
+<summary align="right"><a href="https://link.springer.com/chapter/10.1007/978-3-030-58580-8_27">RTMPose (arXiv'2023)</a></summary>
+
+```bibtex
+@misc{https://doi.org/10.48550/arxiv.2303.07399,
+  doi = {10.48550/ARXIV.2303.07399},
+  url = {https://arxiv.org/abs/2303.07399},
+  author = {Jiang, Tao and Lu, Peng and Zhang, Li and Ma, Ningsheng and Han, Rui and Lyu, Chengqi and Li, Yining and Chen, Kai},
+  keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
+  title = {RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose},
+  publisher = {arXiv},
+  year = {2023},
+  copyright = {Creative Commons Attribution 4.0 International}
+}
+
+```
+
+</details>
+
+<!-- [BACKBONE] -->
+
+<details>
+<summary align="right"><a href="https://arxiv.org/abs/2212.07784">RTMDet (arXiv'2022)</a></summary>
+
+```bibtex
+@misc{lyu2022rtmdet,
+      title={RTMDet: An Empirical Study of Designing Real-Time Object Detectors},
+      author={Chengqi Lyu and Wenwei Zhang and Haian Huang and Yue Zhou and Yudong Wang and Yanyi Liu and Shilong Zhang and Kai Chen},
+      year={2022},
+      eprint={2212.07784},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```
+
+</details>
+
+<!-- [DATASET] -->
+
+<details>
+<summary align="right"><a href="https://link.springer.com/chapter/10.1007/978-3-319-10602-1_48">COCO (ECCV'2014)</a></summary>
+
+```bibtex
+@inproceedings{lin2014microsoft,
+  title={Microsoft coco: Common objects in context},
+  author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C Lawrence},
+  booktitle={European conference on computer vision},
+  pages={740--755},
+  year={2014},
+  organization={Springer}
+}
+```
+
+</details>
+
+- Results on COCO val2017 with detector having human AP of 56.4 on COCO val2017 dataset.
+- `*` denotes model trained on 7 public datasets:
+  - [AI Challenger](https://mmpose.readthedocs.io/en/latest/dataset_zoo/2d_body_keypoint.html#aic)
+  - [MS COCO](https://mmpose.readthedocs.io/en/latest/dataset_zoo/2d_body_keypoint.html#coco)
+  - [CrowdPose](https://mmpose.readthedocs.io/en/latest/dataset_zoo/2d_body_keypoint.html#crowdpose)
+  - [MPII](https://mmpose.readthedocs.io/en/latest/dataset_zoo/2d_body_keypoint.html#mpii)
+  - [sub-JHMDB](https://mmpose.readthedocs.io/en/latest/dataset_zoo/2d_body_keypoint.html#sub-jhmdb-dataset)
+  - [Halpe](https://mmpose.readthedocs.io/en/latest/dataset_zoo/2d_wholebody_keypoint.html#halpe)
+  - [PoseTrack18](https://mmpose.readthedocs.io/en/latest/dataset_zoo/2d_body_keypoint.html#posetrack18)
+- `Body8` denotes the addition of the [OCHuman](https://mmpose.readthedocs.io/en/latest/dataset_zoo/2d_body_keypoint.html#ochuman) dataset, in addition to the 7 datasets mentioned above, for evaluation.
+
+|                     Config                     | Input Size | AP<sup><br>(COCO) | [email protected]<sup><br>(Body8) | AUC<sup><br>(Body8) | EPE<sup><br>(Body8) | Params(M) | FLOPS(G) |                     Download                      |
+| :--------------------------------------------: | :--------: | :---------------: | :---------------------: | :-----------------: | :-----------------: | :-------: | :------: | :-----------------------------------------------: |
+| [RTMPose-t\*](/configs/body_2d_keypoint/rtmpose/body8/rtmpose-t_8xb256-210e_body8-256x192.py) |  256x192   |       65.9        |          91.44          |        63.18        |        19.45        |   3.34    |   0.36   | [Model](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-t_simcc-body7_pt-body7_420e-256x192-026a1439_20230504.pth) |
+| [RTMPose-s\*](/configs/body_2d_keypoint/rtmpose/body8/rtmpose-s_8xb256-210e_body8-256x192.py) |  256x192   |       69.7        |          92.45          |        65.15        |        17.85        |   5.47    |   0.68   | [Model](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-s_simcc-body7_pt-body7_420e-256x192-acd4a1ef_20230504.pth) |
+| [RTMPose-m\*](/configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-210e_body8-256x192.py) |  256x192   |       74.9        |          94.25          |        68.59        |        15.12        |   13.59   |   1.93   | [Model](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth) |
+| [RTMPose-l\*](/configs/body_2d_keypoint/rtmpose/body8/rtmpose-l_8xb256-210e_body8-256x192.py) |  256x192   |       76.7        |          95.08          |        70.14        |        13.79        |   27.66   |   4.16   | [Model](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-body7_pt-body7_420e-256x192-4dba18fc_20230504.pth) |
+| [RTMPose-m\*](/configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-210e_body8-384x288.py) |  384x288   |       76.6        |          94.64          |        70.38        |        13.98        |   13.72   |   4.33   | [Model](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-384x288-65e718c4_20230504.pth) |
+| [RTMPose-l\*](/configs/body_2d_keypoint/rtmpose/body8/rtmpose-l_8xb256-210e_body8-384x288.py) |  384x288   |       78.3        |          95.36          |        71.58        |        13.08        |   27.79   |   9.35   | [Model](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-body7_pt-body7_420e-384x288-3f5a1437_20230504.pth) |
diff --git a/configs/body_2d_keypoint/rtmpose/body8/rtmpose_body8.yml b/configs/body_2d_keypoint/rtmpose/body8/rtmpose_body8.yml
@@ -0,0 +1,93 @@
+Collections:
+- Name: RTMPose
+  Paper:
+    Title: "RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose"
+    URL: https://arxiv.org/abs/2303.07399
+  README: https://github.com/open-mmlab/mmpose/blob/main/projects/rtmpose/README.md
+Models:
+- Config: configs/body_2d_keypoint/rtmpose/body8/rtmpose-t_8xb256-210e_body8-256x192.py
+  In Collection: RTMPose
+  Metadata:
+    Architecture: &id001
+    - RTMPose
+    Training Data: &id002
+    - AI Challenger
+    - COCO
+    - CrowdPose
+    - MPII
+    - sub-JHMDB
+    - Halpe
+    - PoseTrack18
+  Name: rtmpose-t_8xb256-210e_body8-256x192
+  Results:
+  - Dataset: Body8
+    Metrics:
+      AP: 0.659
+      [email protected]: 0.914
+    Task: Body 2D Keypoint
+  Weights: https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-t_simcc-body7_pt-body7_420e-256x192-026a1439_20230504.pth
+- Config: configs/body_2d_keypoint/rtmpose/body8/rtmpose-s_8xb256-210e_body8-256x192.py
+  In Collection: RTMPose
+  Metadata:
+    Architecture: *id001
+    Training Data: *id002
+  Name: rtmpose-s_8xb256-210e_body8-256x192
+  Results:
+  - Dataset: Body8
+    Metrics:
+      AP: 0.697
+      [email protected]: 0.925
+    Task: Body 2D Keypoint
+  Weights: https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-s_simcc-body7_pt-body7_420e-256x192-acd4a1ef_20230504.pth
+- Config: configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-210e_body8-256x192.py
+  In Collection: RTMPose
+  Metadata:
+    Architecture: *id001
+    Training Data: *id002
+  Name: rtmpose-m_8xb256-210e_body8-256x192
+  Results:
+  - Dataset: Body8
+    Metrics:
+      AP: 0.749
+      [email protected]: 0.943
+    Task: Body 2D Keypoint
+  Weights: https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.pth
+- Config: configs/body_2d_keypoint/rtmpose/body8/rtmpose-l_8xb256-210e_body8-256x192.py
+  In Collection: RTMPose
+  Metadata:
+    Architecture: *id001
+    Training Data: *id002
+  Name: rtmpose-l_8xb256-210e_body8-256x192
+  Results:
+  - Dataset: Body8
+    Metrics:
+      AP: 0.767
+      [email protected]: 0.951
+    Task: Body 2D Keypoint
+  Weights: https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-body7_pt-body7_420e-256x192-4dba18fc_20230504.pth
+- Config: configs/body_2d_keypoint/rtmpose/body8/rtmpose-m_8xb256-210e_body8-384x288.py
+  In Collection: RTMPose
+  Metadata:
+    Architecture: *id001
+    Training Data: *id002
+  Name: rtmpose-m_8xb256-210e_body8-384x288
+  Results:
+  - Dataset: Body8
+    Metrics:
+      AP: 0.766
+      [email protected]: 0.946
+    Task: Body 2D Keypoint
+  Weights: https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-m_simcc-body7_pt-body7_420e-384x288-65e718c4_20230504.pth
+- Config: configs/body_2d_keypoint/rtmpose/body8/rtmpose-l_8xb256-210e_body8-384x288.py
+  In Collection: RTMPose
+  Metadata:
+    Architecture: *id001
+    Training Data: *id002
+  Name: rtmpose-l_8xb256-210e_body8-384x288
+  Results:
+  - Dataset: Body8
+    Metrics:
+      AP: 0.783
+      [email protected]: 0.964
+    Task: Body 2D Keypoint
+  Weights: https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-l_simcc-body7_pt-body7_420e-384x288-3f5a1437_20230504.pth
diff --git a/configs/face_2d_keypoint/rtmpose/README.md b/configs/face_2d_keypoint/rtmpose/README.md
@@ -29,4 +29,4 @@ Results on LaPa dataset
 
 |   Model   | Input Size | NME  |           Details and Download            |
 | :-------: | :--------: | :--: | :---------------------------------------: |
-| RTMPose-m |  256x256   | 1.29 | [rtmpose_lapa.md](./wflw/rtmpose_lapa.md) |
+| RTMPose-m |  256x256   | 1.29 | [rtmpose_lapa.md](./lapa/rtmpose_lapa.md) |
diff --git a/configs/face_2d_keypoint/rtmpose/lapa/rtmpose-m_8xb64-120e_lapa-256x256.py b/configs/face_2d_keypoint/rtmpose/lapa/rtmpose-m_8xb64-120e_lapa-256x256.py
@@ -187,7 +187,7 @@
         data_root=data_root,
         data_mode=data_mode,
         ann_file='annotations/lapa_train.json',
-        data_prefix=dict(img='train/images/'),
+        data_prefix=dict(img=''),
         pipeline=train_pipeline,
     ))
 val_dataloader = dict(
@@ -201,7 +201,7 @@
         data_root=data_root,
         data_mode=data_mode,
         ann_file='annotations/lapa_val.json',
-        data_prefix=dict(img='val/images/'),
+        data_prefix=dict(img=''),
         test_mode=True,
         pipeline=val_pipeline,
     ))
@@ -216,7 +216,7 @@
         data_root=data_root,
         data_mode=data_mode,
         ann_file='annotations/lapa_test.json',
-        data_prefix=dict(img='test/images/'),
+        data_prefix=dict(img=''),
         test_mode=True,
         pipeline=val_pipeline,
     ))

diff --git a/configs/face_2d_keypoint/rtmpose/lapa/rtmpose_lapa.md b/configs/face_2d_keypoint/rtmpose/lapa/rtmpose_lapa.md
@@ -33,7 +33,7 @@
 
 </details>
 
-Results on COCO-WholeBody-Face val set
+Results on LaPa val set
 
 | Arch                                                           | Input Size | NME  |                              ckpt                              |                              log                               |
 | :------------------------------------------------------------- | :--------: | :--: | :------------------------------------------------------------: | :------------------------------------------------------------: |

diff --git a/configs/face_2d_keypoint/topdown_regression/README.md b/configs/face_2d_keypoint/topdown_regression/README.md
@@ -15,4 +15,5 @@ Result on WFLW test set
 | Model                                                           | Input Size | NME  |                              ckpt                              |                              log                              |
 | :-------------------------------------------------------------- | :--------: | :--: | :------------------------------------------------------------: | :-----------------------------------------------------------: |
 | [ResNet-50](/configs/face_2d_keypoint/topdown_regression/wflw/td-reg_res50_8xb64-210e_wflw-256x256.py) |  256x256   | 4.88 | [ckpt](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256-92d0ba7f_20210303.pth) | [log](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_20210303.log.json) |
-| [ResNet-50+SoftWingLoss](/configs/face_2d_keypoint/topdown_regression/wflw/td-reg_res50_softwingloss_8xb64-210e_wflw-256x256.py) |  256x256   | 4.67 | [ckpt](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_softwingloss-4d34f22a_20211212.pth) | [log](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_softwingloss_20211212.log.json) |
+| [ResNet-50+WingLoss](/configs/face_2d_keypoint/topdown_regression/wflw/td-reg_res50_wingloss_8xb64-210e_wflw-256x256.py) |  256x256   | 4.67 | [ckpt](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_wingloss-f82a5e53_20210303.pth) | [log](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_wingloss_20210303.log.json) |
+| [ResNet-50+SoftWingLoss](/configs/face_2d_keypoint/topdown_regression/wflw/td-reg_res50_softwingloss_8xb64-210e_wflw-256x256.py) |  256x256   | 4.44 | [ckpt](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_softwingloss-4d34f22a_20211212.pth) | [log](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_softwingloss_20211212.log.json) |
diff --git a/configs/face_2d_keypoint/topdown_regression/wflw/resnet_wingloss_wflw.md b/configs/face_2d_keypoint/topdown_regression/wflw/resnet_wingloss_wflw.md
@@ -0,0 +1,76 @@
+<!-- [ALGORITHM] -->
+
+<details>
+<summary align="right"><a href="http://openaccess.thecvf.com/content_cvpr_2014/html/Toshev_DeepPose_Human_Pose_2014_CVPR_paper.html">DeepPose (CVPR'2014)</a></summary>
+
+```bibtex
+@inproceedings{toshev2014deeppose,
+  title={Deeppose: Human pose estimation via deep neural networks},
+  author={Toshev, Alexander and Szegedy, Christian},
+  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+  pages={1653--1660},
+  year={2014}
+}
+```
+
+</details>
+
+<!-- [BACKBONE] -->
+
+<details>
+<summary align="right"><a href="http://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html">ResNet (CVPR'2016)</a></summary>
+
+```bibtex
+@inproceedings{he2016deep,
+  title={Deep residual learning for image recognition},
+  author={He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian},
+  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+  pages={770--778},
+  year={2016}
+}
+```
+
+</details>
+
+<!-- [ALGORITHM] -->
+
+<details>
+<summary align="right"><a href="http://openaccess.thecvf.com/content_cvpr_2018/html/Feng_Wing_Loss_for_CVPR_2018_paper.html">Wingloss (CVPR'2018)</a></summary>
+
+```bibtex
+@inproceedings{feng2018wing,
+  title={Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks},
+  author={Feng, Zhen-Hua and Kittler, Josef and Awais, Muhammad and Huber, Patrik and Wu, Xiao-Jun},
+  booktitle={Computer Vision and Pattern Recognition (CVPR), 2018 IEEE Conference on},
+  year={2018},
+  pages ={2235-2245},
+  organization={IEEE}
+}
+```
+
+</details>
+
+<!-- [DATASET] -->
+
+<details>
+<summary align="right"><a href="http://openaccess.thecvf.com/content_cvpr_2018/html/Wu_Look_at_Boundary_CVPR_2018_paper.html">WFLW (CVPR'2018)</a></summary>
+
+```bibtex
+@inproceedings{wu2018look,
+  title={Look at boundary: A boundary-aware face alignment algorithm},
+  author={Wu, Wayne and Qian, Chen and Yang, Shuo and Wang, Quan and Cai, Yici and Zhou, Qiang},
+  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
+  pages={2129--2138},
+  year={2018}
+}
+```
+
+</details>
+
+Results on WFLW dataset
+
+The model is trained on WFLW train set.
+
+| Model                                                           | Input Size | NME  |                              ckpt                              |                              log                              |
+| :-------------------------------------------------------------- | :--------: | :--: | :------------------------------------------------------------: | :-----------------------------------------------------------: |
+| [ResNet-50+WingLoss](/configs/face_2d_keypoint/topdown_regression/wflw/td-reg_res50_wingloss_8xb64-210e_wflw-256x256.py) |  256x256   | 4.67 | [ckpt](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_wingloss-f82a5e53_20210303.pth) | [log](https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_wingloss_20210303.log.json) |
diff --git a/configs/face_2d_keypoint/topdown_regression/wflw/resnet_wingloss_wflw.yml b/configs/face_2d_keypoint/topdown_regression/wflw/resnet_wingloss_wflw.yml
@@ -0,0 +1,16 @@
+Models:
+- Config: configs/face_2d_keypoint/topdown_regression/wflw/td-reg_res50_wingloss_8xb64-210e_wflw-256x256.py
+  In Collection: ResNet
+  Metadata:
+    Architecture:
+    - DeepPose
+    - ResNet
+    - WingLoss
+    Training Data: WFLW
+  Name: td-reg_res50_wingloss_8xb64-210e_wflw-256x256
+  Results:
+  - Dataset: WFLW
+    Metrics:
+      NME: 4.67
+    Task: Face 2D Keypoint
+  Weights: https://download.openmmlab.com/mmpose/face/deeppose/deeppose_res50_wflw_256x256_wingloss-f82a5e53_20210303.pth