Build a markerless motion capture system to extract kinematics from bicycling videos

[moorepants]

Description

We have at least two sets of data that could be useful for building a markerless motion capture algorithm for bicycling.

This data from bicycling on a treadmill using a traditional motion capture system:

https://figshare.com/account/projects/1860/articles/1082512

We also have the matching videos for this.

We also have videos and IMU type data for this: https://figshare.com/account/home#/projects/22067

There is video from a rear frame fixed camera and IMU type data from this paper too: http://moorepants.github.io/dissertation/delftbicycle.html

There may be more.

The idea would to use modern machine learning techniques to build an open source tool that can extract bicycle kinematics from videos.

Prior Art

This software is getting recent press: http://www.mousemotorlab.org/deeplabcut but was based on https://github.com/eldar/pose-tensorflow, the later which is specifically for humans.

Extracting gait data from cerebal palsy gait: http://gaitlab.stanford.edu/

Heike mentioned this paper about elderly falls as a helpful reference: https://pubmed.ncbi.nlm.nih.gov/23083889/

Openpose is used to extract kinematics of people.

Frame by frame analysis of horse racing falls: https://doi.org/10.1007/s12283-020-00323-0 (manual way of doing something similar to what we could do here).

2021-02: University of Gent says it is going to extract data from old vidoes https://www.vrt.be/vrtnws/nl/2021/02/03/ugent-onderzoek-wielrennen-valpartijen/, this guys is quoted in the article: https://ai.ugent.be/people/StevenVerstockt.en.html

https://twitter.com/JonMatthis/status/1351531974364688385, https://github.com/jonmatthis/freemocap

https://github.com/geaxgx/depthai_blazepose

https://anipose.readthedocs.io/en/latest/

Proposed Methods

Required Resources

[moorepants]

I think that there are a substantial number of videos on youtube of bicycle crashes. If we got this working, we could have a really nice dataset of the kinematics of falls.

[moorepants]

One note: getting 3D kinematics from 2D videos may still be difficult.