[Feature] Liu yi/surpport edpose #2688

LiuYi-Up · 2023-09-13T05:31:26Z

Motivation

Support inference pipline for ED-Pose

Modification

新文件： configs/body_2d_keypoint/edpose/coco/edpose_coco.md
新文件： configs/body_2d_keypoint/edpose/coco/edpose_coco.yml
新文件： configs/body_2d_keypoint/edpose/coco/edpose_res50_coco.py
新文件： docs/src/papers/algorithms/edpose.md
修改： mmpose/codecs/init.py
新文件： mmpose/codecs/edpose_label.py
修改： mmpose/datasets/transforms/init.py
修改： mmpose/datasets/transforms/bottomup_transforms.py
修改： mmpose/models/data_preprocessors/init.py
修改： mmpose/models/data_preprocessors/data_preprocessor.py
修改： mmpose/models/heads/init.py
新文件： mmpose/models/heads/transformer_heads/init.py
新文件： mmpose/models/heads/transformer_heads/base_transformer_head.py
新文件： mmpose/models/heads/transformer_heads/edpose_head.py
新文件： mmpose/models/heads/transformer_heads/transformers/init.py
新文件： mmpose/models/heads/transformer_heads/transformers/deformable_detr_layers.py
新文件： mmpose/models/heads/transformer_heads/transformers/detr_layers.py
新文件： mmpose/models/heads/transformer_heads/transformers/utils.py
修改： mmpose/models/necks/init.py
新文件： mmpose/models/necks/chanel_mapper_neck.py
修改： tests/test_datasets/test_transforms/test_bottomup_transforms.py

BC-breaking (Optional)

Use cases (Optional)

Checklist

Before PR:

I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

CLA has been signed and all committers have signed the CLA in this PR.

CLAassistant · 2023-09-13T05:31:35Z

All committers have signed the CLA.

mmpose/codecs/edpose_label.py

codecov · 2023-09-14T06:54:47Z

Codecov Report

Patch coverage is 30.33% of modified lines.

❗ Current head 82fa9f4 differs from pull request most recent head b19fffa. Consider uploading reports for the commit b19fffa to get more accurate results

Files Changed	Coverage
mmpose/datasets/transforms/__init__.py	`ø`
...ose/models/data_preprocessors/data_preprocessor.py	`ø`
...pose/models/heads/transformer_heads/edpose_head.py	`7.70%`
...dels/heads/transformer_heads/transformers/utils.py	`23.07%`
...eads/transformer_heads/transformers/detr_layers.py	`25.53%`
...ormer_heads/transformers/deformable_detr_layers.py	`27.27%`
mmpose/models/utils/ops.py	`27.58%`
...s/heads/transformer_heads/base_transformer_head.py	`47.82%`
mmpose/codecs/edpose_label.py	`86.36%`
mmpose/datasets/transforms/bottomup_transforms.py	`90.25%`
... and 9 more

📢 Thoughts on this report? Let us know!.

mmpose/version.py

Tau-J · 2023-09-15T05:51:31Z

mmpose/models/data_preprocessors/data_preprocessor.py

+                    elif self.normalize_bakend == 'pillow':
+                        _batch_input = tvF.normalize(
+                            _batch_input, mean=self.mean, std=self.std)


normalize_backend seems to be confusing, since the normalization is actually conducted by torchvision.
If we want to emphasize the img is loaded by pillow, maybe imdecode_backend is more explicit, just like mmcv does https://github.com/open-mmlab/mmcv/blob/b361a81ad4c5d31611dad7e8d5fbcb7d7df22f34/mmcv/transforms/loading.py#L33

Tau-J · 2023-09-15T05:57:53Z

mmpose/models/heads/transformer_heads/edpose_head.py

+        return [[itm_out.transpose(0, 1) for itm_out in intermediate],
+                [
+                    itm_refpoint.transpose(0, 1)
+                    for itm_refpoint in intermediate_reference_points
+                ]]


Please refine this code

Tau-J · 2023-09-15T06:03:24Z

mmpose/models/heads/transformer_heads/edpose_head.py

+        out = {
+            'pred_logits': outputs_class[-1],
+            'pred_boxes': outputs_coord_list[-1],
+            'pred_keypoints': outputs_keypoints_list[-1]
+        }
+
+        return out
+


It seems not to be necessary to pack outputs into dict

mmpose/models/heads/transformer_heads/base_transformer_head.py

Co-authored-by: Tau <[email protected]>

Tau-J

LGTM

* update * [Fix] Fix HRFormer log link * [Feature] Add Application 'Just dance' (#2528) * [Docs] Add advanced tutorial of implement new model. (#2539) * [Doc] Update img (#2541) * [Feature] Support MotionBERT (#2482) * [Fix] Fix demo scripts (#2542) * [Fix] Fix Pose3dInferencer keypoint shape bug (#2543) * [Enhance] Add notifications when saving visualization results (#2545) * [Fix] MotionBERT training and flip-test (#2548) * [Docs] Enhance docs (#2555) * [Docs] Fix links in doc (#2557) * [Docs] add details (#2558) * [Refactor] 3d human pose demo (#2554) * [Docs] Update MotionBERT docs (#2559) * [Refactor] Update the arguments of 3d inferencer to align with the demo script (#2561) * [Enhance] Combined dataset supports custom sampling ratio (#2562) * [Docs] Add MultiSourceSampler docs (#2563) * [Doc] Refine docs (#2564) * [Feature][MMSIG] Add UniFormer Pose Estimation to Projects folder (#2501) * [Fix] Check the compatibility of inferencer's input/output (#2567) * [Fix]Fix 3d visualization (#2565) * [Feature] Add bear example in just dance (#2568) * [Doc] Add example and openxlab link for just dance (#2571) * [Fix] Configs' paths of VideoPose3d (#2572) * [Docs] update docs (#2573) * [Fix] Fix new config bug in train.py (#2575) * [Fix] Configs' of MotionBERT (#2574) * [Enhance] Normalization option in 3d human pose demo and inferencer (#2576) * [Fix] Fix the incorrect labels for training vis_head with combined datasets (#2550) * [Enhance] Enhance 3dpose demo and docs (#2578) * [Docs] Enhance Codecs documents (#2580) * [Feature] Add DWPose distilled WholeBody RTMPose models (#2581) * [Docs] Add deployment docs (#2582) * [Fix] Refine 3dpose (#2583) * [Fix] Fix config typo in rtmpose-x (#2585) * [Fix] Fix 3d inferencer (#2593) * [Feature] Add a simple visualize api (#2596) * [Feature][MMSIG] Support badcase analyze in test (#2584) * [Fix] fix bug in flip_bbox with xyxy format (#2598) * [Feature] Support ubody dataset (2d keypoints) (#2588) * [Fix] Fix visualization bug in 3d pose (#2594) * [Fix] Remove use-multi-frames option (#2601) * [Enhance] Update demos (#2602) * [Enhance] wholebody support openpose style visualization (#2609) * [Docs] Documentation regarding 3d pose (#2599) * [CodeCamp2023-533] Migration Deepfashion topdown heatmap algorithms to 1.x (#2597) * [Fix] fix badcase hook (#2616) * [Fix] Update dataset mim downloading source to OpenXLab (#2614) * [Docs] Update docs structure (#2617) * [Docs] Refine Docs (#2619) * [Fix] Fix numpy error (#2626) * [Docs] Update error info and docs (#2624) * [Fix] Fix inferencer argument name (#2627) * [Fix] fix links for coco+aic hrnet (#2630) * [Fix] fix a bug when visualize keypoint indices (#2631) * [Docs] Update rtmpose docs (#2642) * [Docs] update README.md (#2647) * [Docs] Add onnx of RTMPose models (#2656) * [Docs] Fix mmengine link (#2655) * [Docs] Update QR code (#2653) * [Feature] Add DWPose (#2643) * [Refactor] Reorganize distillers (#2658) * [CodeCamp2023-259]Document Writing: Advanced Tutorial - Custom Data Augmentation (#2605) * [Docs] Fix installation docs(#2668) * [Fix] Fix expired links in README (#2673) * [Feature] Support multi-dataset evaluation (#2674) * [Refactor] Specify labels to pack in codecs (#2659) * [Refactor] update mapping tables (#2676) * [Fix] fix link (#2677) * [Enhance] Enable CocoMetric to get ann_file from MessageHub (#2678) * [Fix] fix vitpose pretrained ckpts (#2687) * [Refactor] Refactor YOLOX-Pose into mmpose core package (#2620) * [Fix] Fix typo in COCOMetric(#2691) * [Fix] Fix bug raised by changing bbox_center to input_center (#2693) * [Feature] Surpport EDPose for inference(#2688) * [Refactor] Internet for 3d hand pose estimation (#2632) * [Fix] Change test batch_size of edpose to 1 (#2701) * [Docs] Add OpenXLab Badge (#2698) * [Doc] fix inferencer doc (#2702) * [Docs] Refine dataset config tutorial (#2707) * [Fix] modify yoloxpose test settings (#2706) * [Fix] add compatibility for argument `return_datasample` (#2708) * [Feature] Support ubody3d dataset (#2699) * [Fix] Fix 3d inferencer (#2709) * [Fix] Move ubody3d dataset to wholebody3d (#2712) * [Refactor] Refactor config and dataset file structures (#2711) * [Fix] give more clues when loading img failed (#2714) * [Feature] Add demo script for 3d hand pose (#2710) * [Fix] Fix Internet demo (#2717) * [codecamp: mmpose-315] 300W-LP data set support (#2716) * [Fix] Fix the typo in YOLOX-Pose (#2719) * [Feature] Add detectors trained on humanart (#2724) * [Feature] Add RTMPose-Wholebody (#2721) * [Doc] Fix github action badge in README (#2727) * [Fix] Fix bug of dwpose (#2728) * [Feature] Support hand3d inferencer (#2729) * [Fix] Fix new config of RTMW (#2731) * [Fix] Align visualization color of 3d demo (#2734) * [Fix] Refine h36m data loading and add head_size to PackPoseInputs (#2735) * [Refactor] Align test accuracy for AE (#2737) * [Refactor] Separate evaluation mappings from KeypointConverter (#2738) * [Fix] MotionbertLabel codec (#2739) * [Fix] Fix mask shape (#2740) * [Feature] Add training datasets of RTMW (#2743) * [Doc] update RTMPose README (#2744) * [Fix] skip warnings in demo (#2746) * Bump 1.2 (#2748) * add comments in dekr configs (#2751) --------- Co-authored-by: Peng Lu <[email protected]> Co-authored-by: Yifan Lareina WU <[email protected]> Co-authored-by: Xin Li <[email protected]> Co-authored-by: Indigo6 <[email protected]> Co-authored-by: 谢昕辰 <[email protected]> Co-authored-by: tpoisonooo <[email protected]> Co-authored-by: zhengjie.xu <[email protected]> Co-authored-by: Mesopotamia <[email protected]> Co-authored-by: chaodyna <[email protected]> Co-authored-by: lwttttt <[email protected]> Co-authored-by: Kanji Yomoda <[email protected]> Co-authored-by: LiuYi-Up <[email protected]> Co-authored-by: ZhaoQiiii <[email protected]> Co-authored-by: Yang-ChangHui <[email protected]> Co-authored-by: Xuan Ju <[email protected]>

mm-assistant bot assigned Ben-Louis Sep 13, 2023

Ben-Louis changed the base branch from main to dev-1.x September 13, 2023 05:36

Ben-Louis changed the title ~~Liu yi/surpport edpose~~ [Feature] Liu yi/surpport edpose Sep 13, 2023

Ben-Louis reviewed Sep 13, 2023

View reviewed changes

mmpose/codecs/edpose_label.py Outdated Show resolved Hide resolved

Ben-Louis reviewed Sep 13, 2023

View reviewed changes

mmpose/codecs/edpose_label.py Outdated Show resolved Hide resolved

Ben-Louis force-pushed the LiuYi/surpport_edpose branch from 537f084 to e4ec320 Compare September 14, 2023 06:32

Tau-J and others added 9 commits September 15, 2023 11:03

Bump version to v1.0.0 (open-mmlab#2169) (open-mmlab#2170)

eb17d3b

support edpose

f7106c8

rebase & fix conflict

dd9d295

simplify codec

e9edf28

move some modules to utils

43309fb

refine model code

ffbb246

merge BatchShapeDataPreprocessor into PoseDataPreprocessor

f2fd3ea

fix ut & simplify decode

4a71560

add license notification

88a9445

Ben-Louis force-pushed the LiuYi/surpport_edpose branch from 9ecf22f to 88a9445 Compare September 15, 2023 03:37

Ben-Louis added 4 commits September 15, 2023 11:38

refine code

26d8d78

refine doc

6a10bba

add ut for edpose_label

aa55d56

update config name && model yaml

bed4321

Tau-J reviewed Sep 15, 2023

View reviewed changes

mmpose/version.py Outdated Show resolved Hide resolved

Tau-J reviewed Sep 15, 2023

View reviewed changes

fix ut error raised by matplotlib 3.8.0

4e3dc00

Tau-J reviewed Sep 15, 2023

View reviewed changes

Ben-Louis added 2 commits September 15, 2023 14:34

simplify datapreprocessor

ba556e5

fix comments

084db23

Tau-J reviewed Sep 15, 2023

View reviewed changes

mmpose/models/heads/transformer_heads/base_transformer_head.py Outdated Show resolved Hide resolved

Ben-Louis and others added 2 commits September 15, 2023 15:07

Update mmpose/models/heads/transformer_heads/base_transformer_head.py

e781fcf

Co-authored-by: Tau <[email protected]>

switch to new config

b19fffa

Tau-J approved these changes Sep 18, 2023

View reviewed changes

Tau-J merged commit 8c34ddc into open-mmlab:dev-1.x Sep 18, 2023
8 of 9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Liu yi/surpport edpose #2688

[Feature] Liu yi/surpport edpose #2688

LiuYi-Up commented Sep 13, 2023

CLAassistant commented Sep 13, 2023 •

edited

Loading

codecov bot commented Sep 14, 2023 •

edited

Loading

Tau-J Sep 15, 2023

Tau-J Sep 15, 2023

Tau-J Sep 15, 2023

Tau-J left a comment

[Feature] Liu yi/surpport edpose #2688

[Feature] Liu yi/surpport edpose #2688

Conversation

LiuYi-Up commented Sep 13, 2023

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

CLAassistant commented Sep 13, 2023 • edited Loading

codecov bot commented Sep 14, 2023 • edited Loading

Codecov Report

Tau-J Sep 15, 2023

Choose a reason for hiding this comment

Tau-J Sep 15, 2023

Choose a reason for hiding this comment

Tau-J Sep 15, 2023

Choose a reason for hiding this comment

Tau-J left a comment

Choose a reason for hiding this comment

CLAassistant commented Sep 13, 2023 •

edited

Loading

codecov bot commented Sep 14, 2023 •

edited

Loading