🎢 RS-NeRF: Neural Radiance Fields from Rolling Shutter Images (ECCV 2024)

Muyao Niu   Tong Chen   Yifan Zhan   Zhuoxiao Li   Xiang Ji   Yinqiang Zheng^*  

The University of Tokyo   ^* Corresponding Author  

In European Conference on Computer Vision (ECCV) 2024

---

Stay tuned. Feel free to contact me for bugs or missing files.

Setup Procedures

Python Environment

conda create -n rsnerf python==3.10
conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch
pip install -r requirements.txt

Dataset

We contribute synthetic and real datasets for evaluating RS-related novel-view synthesis techniques that follows the forward-facing manner.

Download the synthetic and real dataset from this link and unzip them to the current directory.

Pretrained RAFT model for multi-sampling

Download the pretrained RAFT model (raft-things.pth) from this link and unzip it to ./raft_models.

Training

Synthetic dataset

python train.py \
--config configs/wine.txt

Real dataset

python train_real.py \
--config configs/real_toy.txt

Acknowledgments

We appreciate for nerf-pytorch and BAD-NeRF, upon which we build our code implementation. We would also appreciate the code release of USB-NeRF, rspy, JAMNet, CVR, and DeepUnroll for comparison and evaluation.