Skip to content

Latest commit

 

History

History
107 lines (76 loc) · 3.98 KB

README.md

File metadata and controls

107 lines (76 loc) · 3.98 KB
Raw

GP-NeRF: Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering (ECCV2022)

Paper

ENV

  • System: Ubuntu 16.04
  • Nvidia: TITAN Xp
  • Torch version: 1.4.0+cu100, torchvision: 0.4.0

Install Env

We provide two ways to install the environment:

  1. Conda-based:
conda create -n gpnerf python=3.7
conda activate gpnerf

# make sure that the pytorch cuda is consistent with the system cuda
# e.g., if your system cuda is 10.0, install torch 1.4 built from cuda 10.0
pip install torch==1.4.0+cu100 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirements.txt

# Install spconv
git clone https://github.com/traveller59/spconv --recursive
cd spconv
git checkout abf0acf30f5526ea93e687e3f424f62d9cd8313a
export CUDA_HOME="/usr/local/cuda-10.0"
python setup.py bdist_wheel
cd dist
pip install spconv-1.2.1-cp36-cp36m-linux_x86_64.whl
  1. Docker-based:
docker pull caiyj/neuralbody:imi-server
# if there's any problem with spconv, try the following commands
cd ~/spconv
python setup.py bdist_wheel
cd dist
pip uninstall spconv-1.2.1-cp36-cp36m-linux_x86_64.whl
pip install spconv-1.2.1-cp36-cp36m-linux_x86_64.whl
pip install yacs

Set up datasets

  1. Download the ZJU-Mocap dataset here.
  2. Create a soft link: 
    ROOT=/path/to/neuralbody
cd $ROOT/data
ln -s /path/to/zju_mocap zju_mocap

Train

Train on ZJU-Mocap dataset, eval on ZJU-Mocap dataset:

CUDA_VISIBLE_DEVICES=0 python tools/train.py -cfg configs/trainzju_valzju.yaml

Train on THuman dataset, eval on ZJU-Mocap dataset:

CUDA_VISIBLE_DEVICES=0 python3 tools/train.py --cfg configs/trainthu_valzju.yaml

Evaluation and Visualization

Train on ZJU-Mocap dataset, eval on ZJU-Mocap dataset:

CUDA_VISIBLE_DEVICES=0 python3 tools/inference.py --cfg configs/trainzju_valzju.yaml render.resume_path checkpoints/cam3zju_zjuval_70.pth test.test_seq 'demo_zjutrain_zjuval' dataset.test.seq_list "'CoreView_387', 'CoreView_393', 'CoreView_392'," \ test.is_vis True dataset.test.sampler 'FrameSampler' dataset.test.shuffle False render.file 'demo_render'

Train on THuman dataset, eval on ZJU-Mocap dataset:

CUDA_VISIBLE_DEVICES=0 python3 tools/inference.py --cfg configs/trainthu_valzju.yaml render.resume_path checkpoints/cam3thu_zjuval_dim16_100.pth test.test_seq 'demo_thutrain_zjuval' dataset.test.seq_list "'CoreView_387', 'CoreView_393', 'CoreView_392'," \ test.is_vis True dataset.test.sampler 'FrameSampler' dataset.test.shuffle False render.file 'demo_render' head.sigma.code_dim 16

Visualization setting: test.is_vis = True, visualization dir path = [result_dir]/[test.test_seq].

We have provided two pretrained weights files and the corresponding training logs and visualization results here for reference.

Notice

  • If loading the provided checkpoint, please make sure the version of PyTorch and spconv are consistent with the provided code.
  • You can change the dataset root by modifying dataset.train.data_root and dataset.test.data_root in the config file.
  • Please set dataset.test.sampler to '' if you want to test every frame of the test set.
  • To ensure the testing performance, the environment especially PyTorch version and spconv version should be consistent between the training and testing.

Acknowledgement

BibTex

Please cite this paper if you find the code/model helpful in your research:

@inproceedings{chen2022gpnerf,
	title={Geometry-guided progressive NeRF for generalizable and efficient neural human rendering},
	author={Chen, Mingfei and Zhang, Jianfeng and Xu, Xiangyu and Liu, Lijuan and Cai, Yujun and Feng, Jiashi and Yan, Shuicheng},
	booktitle={ECCV},
	year={2022}
}