DGTR: Dual-Branch Graph Transformer Network for 3D Human Mesh Reconstruction from Video (IROS 2024)

Introduction

This repository is the official Pytorch implementation of "Dual-Branch Graph Transformer Network for 3D Human Mesh Reconstruction from Video"

Video

IROS24_1464_VI_i.mp4

Running DGTR

Installation

conda create -n DGTR python=3.7 -y
pip install torch==1.4.0 torchvision==0.5.0
pip install -r requirements.txt

Data preparation

1. Download base_data and SMPL pkl (male&female and neutral), and then put them into ${ROOT}/data/base_data/. Rename SMPL pkl as SMPL_{GENDER}.pkl format. For example, mv basicModel_neutral_lbs_10_207_0_v1.0.0.pkl SMPL_NEUTRAL.pkl.

2. Download data provided by TCMR (except InstaVariety dataset). Pre-processed InstaVariety is uploaded by VIBE authors here. Put them into ${ROOT}/data/preprocessed_data/

3. Download models for testing. Put them into ${ROOT}/data/pretrained_models/

4. Download images (e.g., 3DPW) for rendering. Put them into ${ROOT}/data/3dpw/

The data directory structure should follow the below hierarchy.

${ROOT}  
|-- data  
  |-- base_data  
    |-- J_regressor_extra.npy  
    |-- ...
  |-- preprocessed_data
    |-- 3dpw_train_db.pt
    |-- ...
  |-- pretrained_models
    |-- table1_3dpw_weights.pth.tar
    |-- ...
  |-- 3dpw
    |-- imageFiles
      |-- courtyard_arguing_00
      |-- ...