FreiHAND | HO-3Dv2 |
---|---|
-
Objective: Generate realistic 3D hand meshes with accurate textures from a single image.
-
Supervision Levels: Utilize self-supervision, weak supervision, and full supervision.
-
Contributions of High-Fidelity Textures: Enhance hand pose and shape estimation with learned high-fidelity textures.
-
Benchmark Performance: Experimental evaluations on public benchmarks (FreiHAND and HO-3D). Outperform state-of-the-art methods in texture quality, while maintaining accurate pose and shape estimation.
This code is developed under Python 3.9, Pytorch 1.13, and cuda 11.7.
- (Optional) You may need to wake up your conda:
conda update -n base -c default conda
conda config --append channels conda-forge
conda update --all
- Create the environment and install the requirements:
conda env remove -n hifihr
conda create -n hifihr python=3.9
conda activate hifihr
conda install pytorch=1.13.0 torchvision pytorch-cuda=11.7 -c pytorch -c nvidia
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
# conda install pytorch3d -c pytorch3d
pip install "git+https://github.com/facebookresearch/pytorch3d.git"
conda install tqdm tensorboard transforms3d scikit-image timm trimesh rtree opencv matplotlib rich lpips
pip install chumpy
For 3D hand reconstruction task on the FreiHAND dataset:
- Download the FreiHAND dataset from the website.
For HO3D dataset:
- Download the HO-3Dv2 dataset from the website.
Pre-trained models can be downloaded from the Google Drive link.
- Evaluation:
python train_hrnet.py --config_json config/FreiHAND/evaluation.json
- Training:
python train_hrnet.py --config_json config/FreiHAND/full_rhd_freihand.json
Note: remember to check and inplace the dirs and files in the *.json
files.
- Evaluation:
python3 train_hrnet.py --config_json config/HO3D/evaluation.json
- Training: Please refer to FreiHAND training scripts.
We would like to thank to the great project in S2HAND.
If you find this code useful for your research, please consider citing:
@inproceedings{zhu2023hifihr,
title={HiFiHR: Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texture},
author={Zhu, Jiayin and Zhao, Zhuoran and Yang, Linlin and Yao, Angela},
booktitle={German Conference on Pattern Recognition},
year={2023},
organization={Springer}
}