F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting

Yuxin Wang¹, Qianyi Wu², Dan Xu^1✉️
1Hong Kong University of Science and Technology, ²Monash University

Demo

Novel_VIew_Synthesis.mp4

Updates

• 2025/01/12: We released this repo with the pre-trained model and inference code.

Installation

git clone https://github.com/W-Ted/F3D-Gaus.git

cd F3D-Gaus
conda create -n f3d_gaus python=3.10.14 -y
conda activate f3d_gaus
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 # pytorch=2.0.1=py3.10_cuda11.7_cudnn8.5.0_0
pip install -r requirements.txt

# GOF
cd src/gaussian-splatting
pip install submodules/diff-gof-rasterization
pip install submodules/simple-knn/

# tetra-nerf for triangulation (mesh extraction)
cd submodules/tetra-triangulation
conda install cmake -y
conda install conda-forge::gmp
conda install conda-forge::cgal
cmake .
# you can specify your own cuda path
# export CPATH=/usr/local/cuda-11.3/targets/x86_64-linux/include:$CPATH
make 
pip install -e .

Pre-trained model

We provide the pre-trained model here(~720MB). You could download it to the ''pretrained_models'' directory.

cd pretrained_models
pip install gdown && gdown 'https://drive.google.com/uc?id=1Uar3kyI5Oi5f3cZytUl5YKBkcg4HNALz'
cd ..

Inference

We provide two scripts for inference of F3D-Gaus: one for novel view synthesis and the other for subsequent mesh extraction.

# single-image novel view synthesis
bash scripts/test_nvs.sh 

# single-image mesh extraction
bash scripts/test_mesh.sh

Acknowledgements

This project is built upon G3DR and Splatter-Image. The 3DGS representation is borrowed from GOF. Kudos to these researchers.

Citation

@article{wang2025f3dgaus,
    title={F3D-Gaus: Feed-forward 3D-aware Generation on ImageNet with Cycle-Consistent Gaussian Splatting},
    author={Wang, Yuxin and Wu, Qianyi and Xu, Dan},
    journal={arXiv preprint arXiv:2501.06714},
    year={2025}
}