Roadmap of MMPose

[hellock]

We keep this issue open to collect feature requests from users and hear your voice. Our monthly release plan is also available here.

You can either:

1. Suggest a new feature by leaving a comment.
2. Vote for a feature request with 👍 or be against with 👎. (Remember that developers are busy and cannot respond to all feature requests, so vote for your most favorable one!)
3. Tell us that you would like to help implement one of the features in the list or review the PRs. (This is the greatest things to hear about!)

[jin-s13]

TODO List (continuously updated... [last edit: 2023.1.14]) :
Here is a collection of feature requests.
Items that have already been implemented in MMPose will be removed from the list.

1. More popular backbones

• ConvNeXt

2. Add more popular datasets:

• LSP
• MPII test set
• MPII multi-person dataset for bottom-up methods
• FashionAI
• DeepFashion2
• 3dpw
• MuCo-3DHP

3. More 2d human pose estimation method.

• CPN
• OpenPose [Feature] Paf detector #651
• PifPaf
• ZoomNet
• OPEC-Net
• CenterNet
• MoveNet
• MIPNet
• ViTPose [Feature] Support ViTPose #1876

4. More 2d face alignment algorithms.

5. More 3d human pose algorithms.

• DecoMR
• PoseFormer [Feature] Add PoseFormer backbone #1215
• C2F [Feature] C2F dataset part #868 [Feature] C2F train part #975
• SmoothNet training codes and models trained on 3DPW and AIST.
• PoseAug

6. Support 2d video pose estimation and tracking

• Penn Action
• LSTM Pose Machine
• DetTrack
• KeyTrack
• DCPose Are you planing to add DCPose with Darkpose? #759
• Skip-Convolultion Skip-Convolution Processing #836

7. Support Vehicle pose estimation

• CarFusion dataset
• Occlusion-net
• Apollocar3D

8. Add 3D Pose Consistency Benchmark Documentation: Add 3D Pose Consistency Benchmark #828

9. Mano based hand keypoints detection

10. Depth-based 3d hand pose estimation

• A2J

11. Multi-view 3d pose estimation

• Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images
• 3DMPPE PoseNet Add 3DMPPE POSENET #964
• Learnable Triangulation of Human Pose Multi-view 3d pose estimation #864
• Epipolar Transformers Multi-view 3d pose estimation #864
• Cross view fusion for 3d human pose estimation Multi-view 3d pose estimation #864

12. Support memonger

13. Support Pytorch AMP training Regarding multi-gpu training #339

14. Hyperparameter tuner Optuna

15. Support Unity plugin

16. print loss during evaluation. No log INFO show on terminal when training? #333

17. Quantization Aware Training Quantization Aware Training #359

18. Easier Usage (API)

19. Export to Torchscript How to export the model to torchscript？ #576

[innerlee]

Speed up inference #40

[Witek-]

Add Yolov4 and OpenPose

[san-guy]

Bottom up for MPII dataset?

[HuangJunJie2017]

support memonger : https://github.com/Lyken17/pytorch-memonger

[francescotaioli]

Background : 3d pose estimation (with video generation) with a high number of people (es: official video, minute 00:19 sec, but with a lot of people

Result video: the original video is put on the top-left, with the subsuquent 3d pose of the people on the right. If there are a lot of people, the final video has strange resolution (i.e 6000x400) because every people detected is on put on the same row.

What could be improve: split the people 3d pose visualization into multiple row

[liwenssss]

update Interhand2.6M dataset which contains MANO hand mesh parameters.......

[walsvid]

It would be nice to have Depth-Based 3D Hand Pose Estimation methods like A2J.

[vra]

It would be great to have SmoothNet trained on 3DPW and AIST++ :)

[qinb]

It would be nice to add SmoothNet training code about pose estimation, hoping it could easily retrain on my own dataset.

[levan92]

Will be really helpful to implement MIPNet into mmpose:

• It is particularly useful to tackle data where there are crowded/highly occluded humans. Was previously the SOTA on OCHuman before ViTPose came along. Within the realms of convnets, it should still be the SOTA, and it seems like the idea is general enough to be applied to different types of backbones.

[levan92]

Also similar to #1389 request, will be nice to integrate ViTPose into mmpose. ViTPose is already implemented in mmpose, so I expect integration to be much easier 😄

[liuzhuang13]

It would be nice to support ConvNeXt backbones. It is a very simple model that is purely convolutional. They can serve as a drop-in replacement for ResNet or Swin Transformer architectures. ImageNet-22k pretrained ConvNeXt variants are considered state-of-the-art in this regime.

Official code: https://github.com/facebookresearch/ConvNeXt
ConvNeXt was also implemented in the mmsegmentation and mmdetection libraries.

Thanks!

[darcula1993]

It would be nice to have Poseaug augmentation pipeline for 3d pose estimation.
official code: [(https://github.com/jfzhang95/PoseAug)]

Thanks!

[AshvantSelvam]

Would like to have Tracing support for mmPose models.
I have been able to successfully use 'torch2torchscript' under mmdeploy.apis to trace mmSegmentation Models. However, using the same on mmPose (triedconfigs under dekr and associative_embedding) with the following mmdeploy config: '\mmdeploy\configs\mmpose\pose-detection_torchscript.py' would throw the following error:

File "mmdetection\mmpose\mmpose\datasets\pipelines\shared_transform.py", line 176, in call
meta[key_tgt] = results[key_src]

KeyError: 'flip_index'

Would really appreciate having this feature! Thanks

[qwert31639]

Can SCRFD be added to mmpose?
https://github.com/deepinsight/insightface/tree/master/detection/scrfd

[jin-s13]

Can SCRFD be added to mmpose? https://github.com/deepinsight/insightface/tree/master/detection/scrfd

It seems that SCRFD is for face detection. MMPose will focus on pose estimation/keypoint detection. Maybe it is more appropriate to support it in mmdet.

[BobDLA]

Are you still woking on the openpose which has list on the ROADMAP for years?

Thanks