This repository contains the official implementation for VideoINR introduced in the following paper:
VideoINR: Learning Video Implicit Neural Representation
for Continuous Space-Time Super-Resolution
Zeyuan Chen, Yinbo Chen, Jingwen Liu, Xingqian Xu, Vidit Goel, Zhangyang Wang, Humphrey Shi, Xiaolong Wang
CVPR 2022
You can find more visual results and a brief introduction to VideoINR at our project page.
If you find our work useful in your research, please cite:
@inproceedings{chen2022vinr,
author = {Chen, Zeyuan and Chen, Yinbo and Liu, Jingwen and Xu, Xingqian and Goel, Vidit and Wang, Zhangyang and Shi, Humphrey and Wang, Xiaolong},
title = {VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution},
journal = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2022},
}
The code is tested in:
- Python 3.6
- Pytorch 1.6.0
- Cuda 10.1
- Deformable Convolution v2. Following Zooming Slowmo, we adopt CharlesShang's implementation in the submodule.
If you are using Anaconda, the following command can be used to build the environment:
conda create -n videoinr
conda activate videoinr
conda install pytorch=1.6 -c pytorch
cd models/modules/DCNv2/
python setup.py install
-
Download the pre-trained model from google drive.
-
Convert your video of interest to a sequence of images. This process can be completed by many apps, e.g. ffmpeg and AdobePR.
The folder that contains this image sequence should have a structure as follows:
data_path
├── img_1.jpg
├── img_2.jpg
├── ...
├── img_n.jpg
- Using VideoINR for performing space-time super-resolution. You can adjust up-sampling scales by setting different
space_scale
andtime_scale
.
python demo.py --space_scale 4 --time_scale 8 --data_path [YOUR_DATA_PATH]
- The output would be three folders including low-resolution images, bicubic-upsampling images, and the results of VideoINR.