Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Refactor dataset evaluation interface #1209

Merged
merged 1 commit into from
Feb 28, 2022

Conversation

ly015
Copy link
Member

@ly015 ly015 commented Feb 25, 2022

Motivation

Refactor the argument list of Dataset.evaluate() to align with OpenMMLab standard.

Modification

  • Refactor Dataset.evaluate() argument list: 1) rename outputs as results; 2) change res_folder to an optional argument, and automatically create temp folder if res_folder is None.
# Old
def evaluate(self, outputs, res_folder, **kwargs):
      ...

# New
@deprecated_api_warning(name_dict=dict(outputs='results'))
def evaluate(self, results, res_folder=None, **kwargs):
   ...
  • Modify mmpose/core/evaluation/eval_hooks.py
  • Update unit tests of datasets.

BC-breaking (Optional)

Use cases (Optional)

Checklist

Before PR:

  • I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
  • Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
  • Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
  • New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
  • The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

  • CLA has been signed and all committers have signed the CLA in this PR.

@codecov
Copy link

codecov bot commented Feb 25, 2022

Codecov Report

Merging #1209 (c28a397) into dev-0.24 (d5fd7c6) will increase coverage by 0.20%.
The diff coverage is 92.29%.

Impacted file tree graph

@@             Coverage Diff              @@
##           dev-0.24    #1209      +/-   ##
============================================
+ Coverage     82.46%   82.67%   +0.20%     
============================================
  Files           203      204       +1     
  Lines         16069    16361     +292     
  Branches       2862     2942      +80     
============================================
+ Hits          13252    13527     +275     
+ Misses         2095     2090       -5     
- Partials        722      744      +22     
Flag Coverage Δ
unittests 82.61% <92.29%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmpose/core/evaluation/eval_hooks.py 78.12% <ø> (+27.21%) ⬆️
...sets/datasets/top_down/topdown_mpii_trb_dataset.py 44.36% <29.41%> (-1.68%) ⬇️
...datasets/datasets/top_down/topdown_mpii_dataset.py 40.16% <50.00%> (+0.49%) ⬆️
...ets/body3d/body3d_mview_direct_panoptic_dataset.py 81.36% <73.68%> (-1.06%) ⬇️
...e/datasets/datasets/animal/animal_ap10k_dataset.py 85.38% <77.77%> (-1.74%) ⬇️
...asets/datasets/face/face_coco_wholebody_dataset.py 88.88% <77.77%> (-4.87%) ⬇️
mmpose/datasets/datasets/face/face_cofw_dataset.py 82.89% <77.77%> (-3.87%) ⬇️
...asets/datasets/hand/hand_coco_wholebody_dataset.py 88.57% <77.77%> (-4.98%) ⬇️
...s/datasets/top_down/topdown_posetrack18_dataset.py 91.54% <81.25%> (-1.84%) ⬇️
...sets/top_down/topdown_posetrack18_video_dataset.py 86.02% <86.36%> (-0.75%) ⬇️
... and 37 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d5fd7c6...c28a397. Read the comment docs.

@ly015 ly015 requested a review from jin-s13 February 25, 2022 10:14
@ly015 ly015 merged commit fd1e4fb into open-mmlab:dev-0.24 Feb 28, 2022
ly015 added a commit that referenced this pull request Mar 7, 2022
shuheilocale pushed a commit to shuheilocale/mmpose that referenced this pull request May 6, 2023
ajgrafton pushed a commit to ajgrafton/mmpose that referenced this pull request Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants