Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add two traditional filters to smooth poses #1127

Merged
merged 13 commits into from
Mar 11, 2022
Merged

[Feature] Add two traditional filters to smooth poses #1127

merged 13 commits into from
Mar 11, 2022

Conversation

ailingzengzzz
Copy link
Collaborator

@ailingzengzzz ailingzengzzz commented Jan 4, 2022

Motivation

Smooth poses.

Modification

  1. Add Smoother module for pose smoothing, which applies a temporal filter on pose sequence.
  2. Support 3 filters: Savitzky-Golay 2d filter, Gaussian 1d filter, and One-Euro filter (derived from the existing implementation in MMpose)
  3. Update inference APIs, demos, and Webcam APIs.
  4. Add unit tests of smoother and filters
  5. Fix the bug reported in Mistake in one_euro_filter.py #1176

Use Cases

Init smoother

from mmpose.core import Smoother

smoother = Smoother(filter_cfg='configs/_base_/filters/one_euro.py', keypoint_dim=2)

Online smoothing (frame-by-frame)

for frame in video:
    # pose estimation
    results = inference_top_down_pose_model(model, frame, ...)
    # pose tracking
    results, next_id = get_track_id(results, last_results, next_id, ...)
    # smoothing
    results_smoothed = smoother.smooth(results)

Offline smoothing

results_sequence = []

for frame in video:
    # pose estimation
    results = inference_top_down_pose_model(model, frame, ...)
    # pose tracking
    results, next_id = get_track_id(results, last_results, next_id, ...)

    results_sequence.append(results)

results_sequence_smoothed = smoother.smooth(results_sequence)

BC-Breaking

The arguments use_one_euro and fps in function get_track_id() will be deprecated in the future. Deprecation warnings have been added in this PR.

Checklist

Before PR:

  • I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
  • Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
  • Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
  • New functionalities are covered by complete unit tests. If not, please add more unit tests to ensure correctness.
  • The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

  • CLA has been signed and all committers have signed the CLA in this PR.

@CLAassistant
Copy link

CLAassistant commented Jan 4, 2022

CLA assistant check
All committers have signed the CLA.

@jin-s13 jin-s13 requested a review from ly015 January 4, 2022 12:52
.gitignore Outdated Show resolved Hide resolved
@jin-s13
Copy link
Collaborator

jin-s13 commented Jan 4, 2022

@codecov
Copy link

codecov bot commented Jan 4, 2022

Codecov Report

Merging #1127 (b77a93c) into dev-0.25 (c09a693) will increase coverage by 0.03%.
The diff coverage is 96.41%.

❗ Current head b77a93c differs from pull request most recent head 605bbae. Consider uploading reports for the commit 605bbae to get more accurate results

Impacted file tree graph

@@             Coverage Diff              @@
##           dev-0.25    #1127      +/-   ##
============================================
+ Coverage     83.44%   83.47%   +0.03%     
============================================
  Files           205      212       +7     
  Lines         16625    16821     +196     
  Branches       2976     3010      +34     
============================================
+ Hits          13872    14042     +170     
- Misses         2001     2012      +11     
- Partials        752      767      +15     
Flag Coverage Δ
unittests 83.46% <96.41%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmpose/apis/inference_tracking.py 59.05% <50.00%> (-0.30%) ⬇️
mmpose/core/post_processing/smoother.py 94.59% <94.59%> (ø)
...ost_processing/temporal_filters/one_euro_filter.py 98.07% <98.07%> (ø)
mmpose/core/post_processing/__init__.py 100.00% <100.00%> (ø)
mmpose/core/post_processing/one_euro_filter.py 100.00% <100.00%> (ø)
.../core/post_processing/temporal_filters/__init__.py 100.00% <100.00%> (ø)
...e/core/post_processing/temporal_filters/builder.py 100.00% <100.00%> (ø)
...se/core/post_processing/temporal_filters/filter.py 100.00% <100.00%> (ø)
...ost_processing/temporal_filters/gaussian_filter.py 100.00% <100.00%> (ø)
...rocessing/temporal_filters/savizky_golay_filter.py 100.00% <100.00%> (ø)
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c09a693...605bbae. Read the comment docs.

@ly015 ly015 changed the base branch from master to dev-0.23 January 5, 2022 06:24
@ly015 ly015 changed the title add two traditional filters to smooth poses [Feature] Add two traditional filters to smooth poses Jan 5, 2022
@ly015 ly015 changed the base branch from dev-0.23 to dev-0.25 March 8, 2022 03:05
@ly015 ly015 changed the base branch from dev-0.25 to dev-0.23 March 8, 2022 03:05
@ly015 ly015 changed the base branch from dev-0.23 to dev-0.25 March 8, 2022 03:05
@ly015 ly015 marked this pull request as draft March 10, 2022 14:34
@ly015 ly015 marked this pull request as ready for review March 10, 2022 20:16
@ly015 ly015 requested a review from jin-s13 March 10, 2022 20:16


class OneEuro:

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need any description here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is part of the newly reimplemented One-Euro filter. The one that will be deprecated is in mmpose/core/post_processing/one_euro_filter.py.

@ly015 ly015 merged commit b26b5bf into dev-0.25 Mar 11, 2022
liqikai9 pushed a commit to liqikai9/mmpose that referenced this pull request Mar 28, 2022
* add savgol&gaus1d filters

* Update __init__.py

* Update test_temporal_filter.py

* Update and rename gaus1d_filter.py to gauss1d_filter.py

* Update __init__.py

* pass pre-commit hooks

* Add smoother

* add Smoother
* refactor filters with base filter
* add unittest for Smoother

* add temporal filter unittests

* remove misadded file

* fix bugs in smoother and filters
update get_track_id and demos

* fix unittest

* fix smoother bug with empty input

* fix smoothing in 3d video demo

Co-authored-by: ly015 <[email protected]>
ly015 added a commit that referenced this pull request Apr 2, 2022
* add savgol&gaus1d filters

* Update __init__.py

* Update test_temporal_filter.py

* Update and rename gaus1d_filter.py to gauss1d_filter.py

* Update __init__.py

* pass pre-commit hooks

* Add smoother

* add Smoother
* refactor filters with base filter
* add unittest for Smoother

* add temporal filter unittests

* remove misadded file

* fix bugs in smoother and filters
update get_track_id and demos

* fix unittest

* fix smoother bug with empty input

* fix smoothing in 3d video demo

Co-authored-by: ly015 <[email protected]>
@OpenMMLab-Assistant-004
Copy link

Hi @ailingzengzzz !First of all, we want to express our gratitude for your significant PR in the mmpose project. Your contribution is highly appreciated, and we are grateful for your efforts in helping improve this open-source project during your personal time. We believe that many developers will benefit from your PR.

We would also like to invite you to join our Special Interest Group (SIG) private channel on Discord, where you can share your experiences, ideas, and build connections with like-minded peers. To join the SIG channel, simply message moderator— OpenMMLab on Discord or briefly share your open-source contributions in the #introductions channel and we will assist you. Look forward to seeing you there! Join us :https://discord.gg/raweFPmdzG

If you have WeChat,welcome to join our community on WeChat. You can add our assistant :openmmlabwx. Please add "mmsig + Github ID" as a remark when adding friends:)
Thank you again for your contribution❤

shuheilocale pushed a commit to shuheilocale/mmpose that referenced this pull request May 6, 2023
* add savgol&gaus1d filters

* Update __init__.py

* Update test_temporal_filter.py

* Update and rename gaus1d_filter.py to gauss1d_filter.py

* Update __init__.py

* pass pre-commit hooks

* Add smoother

* add Smoother
* refactor filters with base filter
* add unittest for Smoother

* add temporal filter unittests

* remove misadded file

* fix bugs in smoother and filters
update get_track_id and demos

* fix unittest

* fix smoother bug with empty input

* fix smoothing in 3d video demo

Co-authored-by: ly015 <[email protected]>
HAOCHENYE pushed a commit to HAOCHENYE/mmpose that referenced this pull request Jun 27, 2023
ajgrafton pushed a commit to ajgrafton/mmpose that referenced this pull request Mar 6, 2024
* add savgol&gaus1d filters

* Update __init__.py

* Update test_temporal_filter.py

* Update and rename gaus1d_filter.py to gauss1d_filter.py

* Update __init__.py

* pass pre-commit hooks

* Add smoother

* add Smoother
* refactor filters with base filter
* add unittest for Smoother

* add temporal filter unittests

* remove misadded file

* fix bugs in smoother and filters
update get_track_id and demos

* fix unittest

* fix smoother bug with empty input

* fix smoothing in 3d video demo

Co-authored-by: ly015 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants