Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] Refine algorithm readme with model performance table #1627

Merged
merged 13 commits into from
Oct 9, 2022

Conversation

ly015
Copy link
Member

@ly015 ly015 commented Sep 2, 2022

Motivation

Update algorithm readme with model performance table and page links for algorithms for better user experience.

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

Before PR:

  • I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
  • Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
  • Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
  • New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
  • The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

  • CLA has been signed and all committers have signed the CLA in this PR.

@ly015 ly015 force-pushed the refine-model-readme branch from 1a9f764 to bd33144 Compare September 2, 2022 09:19
@Tau-J
Copy link
Collaborator

Tau-J commented Sep 2, 2022

Do you plan to use this page as an alternative of current algorithm docs? I suggest adding it into the docs instead of placing under the config folder. Besides, maybe an accuracy-sorted version is also necessary.

@ly015
Copy link
Member Author

ly015 commented Sep 2, 2022

Do you plan to use this page as an alternative of current algorithm docs? I suggest adding it into the docs instead of placing under the config folder. Besides, maybe an accuracy-sorted version is also necessary.

This page is expected to serve as an index for users navigating the configs folder on GitHub. We can modify the doc compiling script to automatically involves these pages in the model zoo document page, or even generate a figure with the table content. That can be a future plan.

As for the sorting, one concern is that the performance comparison may not be completely fair (network scale, input size, ...) so the sorted results could be misleading in some cases. Maybe sort them by the publishing year?

@ly015 ly015 force-pushed the refine-model-readme branch from ec5c826 to a99c255 Compare September 2, 2022 09:54
@Tau-J
Copy link
Collaborator

Tau-J commented Sep 2, 2022

Do you plan to use this page as an alternative of current algorithm docs? I suggest adding it into the docs instead of placing under the config folder. Besides, maybe an accuracy-sorted version is also necessary.

This page is expected to serve as an index for users navigating the configs folder on GitHub. We can modify the doc compiling script to automatically involves these pages in the model zoo document page, or even generate a figure with the table content. That can be a future plan.

As for the sorting, one concern is that the performance comparison may not be completely fair (network scale, input size, ...) so the sorted results could be misleading in some cases. Maybe sort them by the publishing year?

I understand your concern. But popular repos like timm and paperwithcodes, whose list is also not completely fair.
image

image

So this kind of ranking is just for convenience, and you know, users are always wondering "which one is the best model in your repo"

@ly015 ly015 force-pushed the refine-model-readme branch from fe4915c to 2df9dab Compare October 8, 2022 11:29
@ly015 ly015 changed the base branch from 1.x to dev-1.x October 8, 2022 11:30
@ly015 ly015 force-pushed the refine-model-readme branch from f771955 to 95b7ecb Compare October 9, 2022 04:04
@ly015 ly015 force-pushed the refine-model-readme branch from 95b7ecb to 43a98fb Compare October 9, 2022 09:46
@codecov
Copy link

codecov bot commented Oct 9, 2022

Codecov Report

Base: 77.73% // Head: 77.77% // Increases project coverage by +0.04% 🎉

Coverage data is based on head (43a98fb) compared to base (4693395).
Patch has no changes to coverable lines.

Additional details and impacted files
@@             Coverage Diff             @@
##           dev-1.x    #1627      +/-   ##
===========================================
+ Coverage    77.73%   77.77%   +0.04%     
===========================================
  Files          204      204              
  Lines        11716    11716              
  Branches      1956     1956              
===========================================
+ Hits          9107     9112       +5     
+ Misses        2213     2206       -7     
- Partials       396      398       +2     
Flag Coverage Δ
unittests 77.77% <ø> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmpose/datasets/transforms/common_transforms.py 83.47% <0.00%> (+1.42%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@ly015 ly015 force-pushed the refine-model-readme branch from bfb77ce to fc0de64 Compare October 9, 2022 10:18
@ly015 ly015 merged commit 8895a4c into open-mmlab:dev-1.x Oct 9, 2022
ly015 added a commit that referenced this pull request Oct 14, 2022
ly015 added a commit that referenced this pull request Oct 14, 2022
shuheilocale pushed a commit to shuheilocale/mmpose that referenced this pull request May 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants