[Refactor] Internet for 3d hand pose estimation #2632

LareinaM · 2023-08-18T06:44:12Z

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

Before PR:

I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

CLA has been signed and all committers have signed the CLA in this PR.

codecov · 2023-08-18T07:01:02Z

Codecov Report

Patch coverage is 66.86% of modified lines.

❗ Current head 2058c8b differs from pull request most recent head 8e00d4a. Consider uploading reports for the commit 8e00d4a to get more accurate results

Files Changed	Coverage
mmpose/datasets/transforms/formatting.py	`ø`
mmpose/models/heads/__init__.py	`ø`
mmpose/models/heads/heatmap_heads/heatmap_head.py	`ø`
mmpose/models/heads/regression_heads/rle_head.py	`ø`
mmpose/models/heads/heatmap_heads/internet_head.py	`23.07%`
mmpose/datasets/transforms/hand_transforms.py	`42.10%`
.../datasets/datasets/base/base_coco_style_dataset.py	`50.00%`
mmpose/codecs/utils/post_processing.py	`64.70%`
...ose/datasets/datasets/hand/interhand_3d_dataset.py	`76.78%`
mmpose/codecs/hand_3d_heatmap.py	`95.45%`
... and 10 more

📢 Thoughts on this report? Let us know!.

mmpose/evaluation/metrics/hand_metric.py

Tau-J · 2023-09-15T05:34:32Z

mmpose/evaluation/metrics/hand_metric.py

+                keypoints_cam[..., :21, :] -= keypoints_cam[..., 20, :]
+                keypoints_cam[..., 21:, :] -= keypoints_cam[..., 41, :]
+
+                pred_coords_cam[..., :21, 2] += abs_depth[0]
+                pred_coords_cam[..., 21:, 2] += abs_depth[1]
+                pred_coords_cam = pixel_to_camera(pred_coords_cam, focal[0],
+                                                  focal[1], principal_pt[0],
+                                                  principal_pt[1])
+
+                pred_coords_cam[..., :21, :] -= pred_coords_cam[..., 20, :]
+                pred_coords_cam[..., 21:, :] -= pred_coords_cam[..., 41, :]


Please avoid assuming the number of keypoints is 21 in metric level. If this metric is specifically designed for InterHand, please use a particular name

renamed metric to InterHandMetric

…o interhand26m

Tau-J

LGTM

* update * [Fix] Fix HRFormer log link * [Feature] Add Application 'Just dance' (#2528) * [Docs] Add advanced tutorial of implement new model. (#2539) * [Doc] Update img (#2541) * [Feature] Support MotionBERT (#2482) * [Fix] Fix demo scripts (#2542) * [Fix] Fix Pose3dInferencer keypoint shape bug (#2543) * [Enhance] Add notifications when saving visualization results (#2545) * [Fix] MotionBERT training and flip-test (#2548) * [Docs] Enhance docs (#2555) * [Docs] Fix links in doc (#2557) * [Docs] add details (#2558) * [Refactor] 3d human pose demo (#2554) * [Docs] Update MotionBERT docs (#2559) * [Refactor] Update the arguments of 3d inferencer to align with the demo script (#2561) * [Enhance] Combined dataset supports custom sampling ratio (#2562) * [Docs] Add MultiSourceSampler docs (#2563) * [Doc] Refine docs (#2564) * [Feature][MMSIG] Add UniFormer Pose Estimation to Projects folder (#2501) * [Fix] Check the compatibility of inferencer's input/output (#2567) * [Fix]Fix 3d visualization (#2565) * [Feature] Add bear example in just dance (#2568) * [Doc] Add example and openxlab link for just dance (#2571) * [Fix] Configs' paths of VideoPose3d (#2572) * [Docs] update docs (#2573) * [Fix] Fix new config bug in train.py (#2575) * [Fix] Configs' of MotionBERT (#2574) * [Enhance] Normalization option in 3d human pose demo and inferencer (#2576) * [Fix] Fix the incorrect labels for training vis_head with combined datasets (#2550) * [Enhance] Enhance 3dpose demo and docs (#2578) * [Docs] Enhance Codecs documents (#2580) * [Feature] Add DWPose distilled WholeBody RTMPose models (#2581) * [Docs] Add deployment docs (#2582) * [Fix] Refine 3dpose (#2583) * [Fix] Fix config typo in rtmpose-x (#2585) * [Fix] Fix 3d inferencer (#2593) * [Feature] Add a simple visualize api (#2596) * [Feature][MMSIG] Support badcase analyze in test (#2584) * [Fix] fix bug in flip_bbox with xyxy format (#2598) * [Feature] Support ubody dataset (2d keypoints) (#2588) * [Fix] Fix visualization bug in 3d pose (#2594) * [Fix] Remove use-multi-frames option (#2601) * [Enhance] Update demos (#2602) * [Enhance] wholebody support openpose style visualization (#2609) * [Docs] Documentation regarding 3d pose (#2599) * [CodeCamp2023-533] Migration Deepfashion topdown heatmap algorithms to 1.x (#2597) * [Fix] fix badcase hook (#2616) * [Fix] Update dataset mim downloading source to OpenXLab (#2614) * [Docs] Update docs structure (#2617) * [Docs] Refine Docs (#2619) * [Fix] Fix numpy error (#2626) * [Docs] Update error info and docs (#2624) * [Fix] Fix inferencer argument name (#2627) * [Fix] fix links for coco+aic hrnet (#2630) * [Fix] fix a bug when visualize keypoint indices (#2631) * [Docs] Update rtmpose docs (#2642) * [Docs] update README.md (#2647) * [Docs] Add onnx of RTMPose models (#2656) * [Docs] Fix mmengine link (#2655) * [Docs] Update QR code (#2653) * [Feature] Add DWPose (#2643) * [Refactor] Reorganize distillers (#2658) * [CodeCamp2023-259]Document Writing: Advanced Tutorial - Custom Data Augmentation (#2605) * [Docs] Fix installation docs(#2668) * [Fix] Fix expired links in README (#2673) * [Feature] Support multi-dataset evaluation (#2674) * [Refactor] Specify labels to pack in codecs (#2659) * [Refactor] update mapping tables (#2676) * [Fix] fix link (#2677) * [Enhance] Enable CocoMetric to get ann_file from MessageHub (#2678) * [Fix] fix vitpose pretrained ckpts (#2687) * [Refactor] Refactor YOLOX-Pose into mmpose core package (#2620) * [Fix] Fix typo in COCOMetric(#2691) * [Fix] Fix bug raised by changing bbox_center to input_center (#2693) * [Feature] Surpport EDPose for inference(#2688) * [Refactor] Internet for 3d hand pose estimation (#2632) * [Fix] Change test batch_size of edpose to 1 (#2701) * [Docs] Add OpenXLab Badge (#2698) * [Doc] fix inferencer doc (#2702) * [Docs] Refine dataset config tutorial (#2707) * [Fix] modify yoloxpose test settings (#2706) * [Fix] add compatibility for argument `return_datasample` (#2708) * [Feature] Support ubody3d dataset (#2699) * [Fix] Fix 3d inferencer (#2709) * [Fix] Move ubody3d dataset to wholebody3d (#2712) * [Refactor] Refactor config and dataset file structures (#2711) * [Fix] give more clues when loading img failed (#2714) * [Feature] Add demo script for 3d hand pose (#2710) * [Fix] Fix Internet demo (#2717) * [codecamp: mmpose-315] 300W-LP data set support (#2716) * [Fix] Fix the typo in YOLOX-Pose (#2719) * [Feature] Add detectors trained on humanart (#2724) * [Feature] Add RTMPose-Wholebody (#2721) * [Doc] Fix github action badge in README (#2727) * [Fix] Fix bug of dwpose (#2728) * [Feature] Support hand3d inferencer (#2729) * [Fix] Fix new config of RTMW (#2731) * [Fix] Align visualization color of 3d demo (#2734) * [Fix] Refine h36m data loading and add head_size to PackPoseInputs (#2735) * [Refactor] Align test accuracy for AE (#2737) * [Refactor] Separate evaluation mappings from KeypointConverter (#2738) * [Fix] MotionbertLabel codec (#2739) * [Fix] Fix mask shape (#2740) * [Feature] Add training datasets of RTMW (#2743) * [Doc] update RTMPose README (#2744) * [Fix] skip warnings in demo (#2746) * Bump 1.2 (#2748) * add comments in dekr configs (#2751) --------- Co-authored-by: Peng Lu <[email protected]> Co-authored-by: Yifan Lareina WU <[email protected]> Co-authored-by: Xin Li <[email protected]> Co-authored-by: Indigo6 <[email protected]> Co-authored-by: 谢昕辰 <[email protected]> Co-authored-by: tpoisonooo <[email protected]> Co-authored-by: zhengjie.xu <[email protected]> Co-authored-by: Mesopotamia <[email protected]> Co-authored-by: chaodyna <[email protected]> Co-authored-by: lwttttt <[email protected]> Co-authored-by: Kanji Yomoda <[email protected]> Co-authored-by: LiuYi-Up <[email protected]> Co-authored-by: ZhaoQiiii <[email protected]> Co-authored-by: Yang-ChangHui <[email protected]> Co-authored-by: Xuan Ju <[email protected]>

LareinaM added 7 commits August 4, 2023 17:40

first commit

3bddad0

edit args

04bd7c6

align shapes and rename

efea8bb

remove checks

0a88d2f

transform and codec

8c18138

update codec and heads

78a0dd3

config, metrics, dataset, codec...

269b6b0

mm-assistant bot assigned Ben-Louis Aug 18, 2023

LareinaM added 3 commits August 21, 2023 12:01

config

7610165

dataset problems

8f4b299

add yml file

2e5b059

LareinaM marked this pull request as ready for review August 23, 2023 03:34

LareinaM added 8 commits August 28, 2023 16:30

fix encoder and loss

0612252

loss and model

b21cf73

fix test

48c31dc

remove debug lines

8c59298

migrate codec

63f0c8a

Merge branch 'dev-1.x' into interhand26m

2f6b9ac

fix lint

f99d8e5

fix lint

b1d51e3

Tau-J reviewed Sep 15, 2023

View reviewed changes

mmpose/evaluation/metrics/hand_metric.py Outdated Show resolved Hide resolved

Tau-J reviewed Sep 15, 2023

View reviewed changes

LareinaM added 7 commits September 15, 2023 20:31

Merge branch 'dev-1.x' into interhand26m

8fcaca5

rename

f073604

Merge branch 'interhand26m' of https://github.com/LareinaM/mmpose int…

9c2efc8

…o interhand26m

fix formatting

f61640b

update results

decbd7d

Merge branch 'dev-1.x' into interhand26m

6b3335d

Update __init__.py

8e00d4a

Tau-J approved these changes Sep 19, 2023

View reviewed changes

Tau-J merged commit bf8b363 into open-mmlab:dev-1.x Sep 19, 2023
8 of 9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Refactor] Internet for 3d hand pose estimation #2632

[Refactor] Internet for 3d hand pose estimation #2632

LareinaM commented Aug 18, 2023

codecov bot commented Aug 18, 2023 •

edited

Loading

Tau-J Sep 15, 2023 •

edited

Loading

LareinaM Sep 15, 2023

Tau-J left a comment

[Refactor] Internet for 3d hand pose estimation #2632

[Refactor] Internet for 3d hand pose estimation #2632

Conversation

LareinaM commented Aug 18, 2023

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

codecov bot commented Aug 18, 2023 • edited Loading

Codecov Report

Tau-J Sep 15, 2023 • edited Loading

Choose a reason for hiding this comment

LareinaM Sep 15, 2023

Choose a reason for hiding this comment

Tau-J left a comment

Choose a reason for hiding this comment

codecov bot commented Aug 18, 2023 •

edited

Loading

Tau-J Sep 15, 2023 •

edited

Loading