3D Whole-body Pose Estimation Using Graph High-Resolution Network for Humanoid Robot Teleoperation

This repository contains the pytorch implementation of the approach described in the paper:

Mingyu Zhang, Qing Gao , Yuanchuan Lai, Ye Zhang, Tao Chang, Yulan Guo 3D Whole-body Pose Estimation Using Graph High-Resolution Network for Humanoid Robot Teleoperation

Introduction & Dataset

We train and evaluate models for 3D whole-body pose estimation on the H3WB.

Train_data Download Link.

Test_data Download Link.

Please put the train_data and test_data in the data directory

Whole-body 2D joints of the human pose are exploited as inputs. 2D poses are scaled according to the image resolution and normalized to [-1, 1]; 3D poses are aligned with respect to the root joint and the unit of 3D pose data changed from millimeter to meter.

Quickstart

This repository is build upon Python v3.8 and Pytorch v1.13.1 on Ubuntu 22.04 LTS. All experiments are conducted on a single NVIDIA RTX 3090 GPU. See requirements.txt for other dependencies. Then you can install dependencies with the following commands.

git clone https://github.com/Z-mingyu/GraphHRNet.git
cd GraphHRNet
pip install -r requirements.txt

Evaluation

python HRNet_GCN_WB.py --model {1/2/3/4/5} --gcn {gcn_name}  --evaluate checkpoint/{weight_name}.pth.tar -cfg checkpoint/w32_adam_lr1e-3.yaml

Training

# Decoupled Vanilla GCN
python HRNet_GCN_WB.py --gcn dc_vanilla

# Decoupled Pre-Aggresive GCN (What method used in paper)
python HRNet_GCN_WB.py --gcn dc_preagg

# Semantic GCN
python HRNet_GCN_WB.py --gcn semantic

# Decoupled Post-Aggresive GCN
python HRNet_GCN_WB.py --gcn dc_postagg

# Convolution-style GCN
python HRNet_GCN_WB.py --gcn convst

# No-sharing GCN
python HRNet_GCN_WB.py --gcn nosharing

# Modulated GCN
python HRNet_GCN_WB.py --gcn modulated

Pre-trained weight

The best checkpoint has been uploaded with hyper-parameters and configuration file

Download Link.

Acknowledgement

This code is extended from the following repositories.

• VideoPose3D
• Semantic GCN
• Modulated-GCN
• GraphSH
• H3WB

Thank you to authors for releasing their code. Please also consider citing their works.