Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BasicVSR config #254

Merged
merged 9 commits into from
Apr 14, 2021
Merged

Add BasicVSR config #254

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
b4372d1
Add configuration files
ckkelvinchan Apr 13, 2021
feb16a4
Modify README
ckkelvinchan Apr 13, 2021
c4fe7ad
update README
ckkelvinchan Apr 13, 2021
bd0c83e
update config
ckkelvinchan Apr 13, 2021
fbc9ba1
update config
ckkelvinchan Apr 13, 2021
ab6ec90
update config
ckkelvinchan Apr 13, 2021
ee04a0d
update config
ckkelvinchan Apr 13, 2021
05d6f08
fix bug
ckkelvinchan Apr 14, 2021
c77867d
update config and readme
ckkelvinchan Apr 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions configs/restorers/basicvsr/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond

## Introduction

```
@InProceedings{chan2021basicvsr,
author = {Chan, Kelvin CK and Wang, Xintao and Yu, Ke and Dong, Chao and Loy, Chen Change},
title = {BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond},
booktitle = {Proceedings of the IEEE conference on computer vision and pattern recognition},
year = {2021}
}
```

## Results and Models

Evaluated on RGB channels for REDS4 and Y channel for others. The metrics are `PSNR`/`SSIM`.

The pretrained weights of SPyNet can be found [here](https://download.openmmlab.com/mmediting/restorers/basicvsr/spynet_20210409-322d39be.pth) .

| Method | REDS4 (BIx4) | Vimeo-90K-T (BIx4) | Vid4 (BIx4) | Download |
|--------------------------------------------------------------------------------------------------------------------------------|:--------------:|:------------------:|:--------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| [basicvsr_reds4](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/basicvsr/basicvsr_reds.py) | 31.4170/0.8909 | - | - | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_reds4_20120409-b4b03f4d.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_reds4_20210409_092646.log.json) |
| [basicvsr_vimeo90k_bi](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/basicvsr/basicvsr_vimeo90k_bi.py) | - | 37.2299/0.9447 | 27.2296/0.8227 | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bi_20210409-ef89bf61.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bi_20210409_132702.log.json) |

<br />

| Method | UDM10 (BDx4) | Vimeo-90K-T (BDx4) | Vid4 (BDx4) | Download |
|--------------------------------------------------------------------------------------------------------------------------------|:--------------:|:------------------:|:--------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| [basicvsr_vimeo90k_bd](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/basicvsr/basicvsr_vimeo90k_bd.py) | 39.8802/0.9683 | 37.5730/0.9495 | 27.9278/0.8537 | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bd_20210409-b5a982fc.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bd_20210409_132740.log.json) |
139 changes: 139 additions & 0 deletions configs/restorers/basicvsr/basicvsr_reds4.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
exp_name = 'basicvsr_reds4'

# model settings
model = dict(
type='BasicVSR',
generator=dict(
type='BasicVSRNet',
mid_channels=64,
num_blocks=30,
spynet_pretrained='pretrained_models/spynet.pth'),
pixel_loss=dict(type='CharbonnierLoss', loss_weight=1.0, reduction='mean'))
# model training and testing settings
train_cfg = dict(fix_iter=5000)
test_cfg = dict(metrics=['PSNR'], crop_border=0)

# dataset settings
train_dataset_type = 'SRREDSMultipleGTDataset'
val_dataset_type = 'SRREDSMultipleGTDataset'

train_pipeline = [
dict(type='GenerateFrameIndicesForRecurrent', interval_list=[1]),
dict(type='TemporalReverse', keys='lq_path', reverse_ratio=0),
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='lq',
channel_order='rgb'),
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='gt',
channel_order='rgb'),
dict(type='RescaleToZeroOne', keys=['lq', 'gt']),
dict(type='PairedRandomCrop', gt_patch_size=256),
dict(
type='Flip', keys=['lq', 'gt'], flip_ratio=0.5,
direction='horizontal'),
dict(type='Flip', keys=['lq', 'gt'], flip_ratio=0.5, direction='vertical'),
dict(type='RandomTransposeHW', keys=['lq', 'gt'], transpose_ratio=0.5),
dict(type='FramesToTensor', keys=['lq', 'gt']),
dict(type='Collect', keys=['lq', 'gt'], meta_keys=['lq_path', 'gt_path'])
]

test_pipeline = [
dict(type='GenerateFrameIndicesForRecurrent', interval_list=[1]),
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='lq',
channel_order='rgb'),
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='gt',
channel_order='rgb'),
dict(type='RescaleToZeroOne', keys=['lq', 'gt']),
dict(type='FramesToTensor', keys=['lq', 'gt']),
dict(
type='Collect',
keys=['lq', 'gt'],
meta_keys=['lq_path', 'gt_path', 'key'])
]

data = dict(
workers_per_gpu=6,
train_dataloader=dict(samples_per_gpu=4, drop_last=True), # 2 gpus
val_dataloader=dict(samples_per_gpu=1),
test_dataloader=dict(samples_per_gpu=1, workers_per_gpu=1),

# train
train=dict(
type='RepeatDataset',
times=1000,
dataset=dict(
type=train_dataset_type,
lq_folder='data/REDS/train_sharp_bicubic/X4',
gt_folder='data/REDS/train_sharp',
num_input_frames=15,
pipeline=train_pipeline,
scale=4,
val_partition='REDS4',
test_mode=False)),
# val
val=dict(
type=val_dataset_type,
lq_folder='data/REDS/train_sharp_bicubic/X4',
gt_folder='data/REDS/train_sharp',
num_input_frames=100,
pipeline=test_pipeline,
scale=4,
val_partition='REDS4',
test_mode=True),
# test
test=dict(
type=val_dataset_type,
lq_folder='data/REDS/train_sharp_bicubic/X4',
gt_folder='data/REDS/train_sharp',
num_input_frames=100,
pipeline=test_pipeline,
scale=4,
val_partition='REDS4',
test_mode=True),
)

# optimizer
optimizers = dict(
generator=dict(
type='Adam',
lr=2e-4,
betas=(0.9, 0.99),
paramwise_cfg=dict(custom_keys={'spynet': dict(lr_mult=0.125)})))

# learning policy
total_iters = 300000
lr_config = dict(
policy='CosineRestart',
by_epoch=False,
periods=[300000],
restart_weights=[1],
min_lr=1e-7)

checkpoint_config = dict(interval=5000, save_optimizer=True, by_epoch=False)
# remove gpu_collect=True in non distributed training
evaluation = dict(interval=5000, save_image=False, gpu_collect=True)
log_config = dict(
interval=100,
hooks=[
dict(type='TextLoggerHook', by_epoch=False),
dict(type='TensorboardLoggerHook'),
])
visual_config = None

# runtime settings
dist_params = dict(backend='nccl')
log_level = 'INFO'
work_dir = f'./work_dirs/{exp_name}'
load_from = None
resume_from = None
workflow = [('train', 1)]
156 changes: 156 additions & 0 deletions configs/restorers/basicvsr/basicvsr_vimeo90k_bd.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
exp_name = 'basicvsr_vimeo90k_bd'

# model settings
model = dict(
type='BasicVSR',
generator=dict(
type='BasicVSRNet',
mid_channels=64,
num_blocks=30,
spynet_pretrained='pretrained_models/spynet.pth'),
pixel_loss=dict(type='CharbonnierLoss', loss_weight=1.0, reduction='mean'))
# model training and testing settings
train_cfg = dict(fix_iter=5000)
test_cfg = dict(metrics=['PSNR'], crop_border=0, convert_to='y')

# dataset settings
train_dataset_type = 'SRVimeo90KMultipleGTDataset'
val_dataset_type = 'SRTestMultipleGTDataset'
test_dataset_type = 'SRVimeo90KDataset'

train_pipeline = [
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='lq',
channel_order='rgb'),
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='gt',
channel_order='rgb'),
dict(type='RescaleToZeroOne', keys=['lq', 'gt']),
dict(type='PairedRandomCrop', gt_patch_size=256),
dict(
type='Flip', keys=['lq', 'gt'], flip_ratio=0.5,
direction='horizontal'),
dict(type='Flip', keys=['lq', 'gt'], flip_ratio=0.5, direction='vertical'),
dict(type='RandomTransposeHW', keys=['lq', 'gt'], transpose_ratio=0.5),
dict(type='MirrorExtend', keys=['lq', 'gt']),
innerlee marked this conversation as resolved.
Show resolved Hide resolved
dict(type='FramesToTensor', keys=['lq', 'gt']),
dict(type='Collect', keys=['lq', 'gt'], meta_keys=['lq_path', 'gt_path'])
]

val_pipeline = [
dict(type='GenerateFrameIndicesForRecurrent', interval_list=[1]),
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='lq',
channel_order='rgb'),
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='gt',
channel_order='rgb'),
dict(type='RescaleToZeroOne', keys=['lq', 'gt']),
dict(type='FramesToTensor', keys=['lq', 'gt']),
dict(
type='Collect',
keys=['lq', 'gt'],
meta_keys=['lq_path', 'gt_path', 'key'])
]

test_pipeline = [
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='lq',
channel_order='rgb'),
dict(
type='LoadImageFromFileList',
io_backend='disk',
key='gt',
channel_order='rgb'),
dict(type='RescaleToZeroOne', keys=['lq', 'gt']),
dict(type='MirrorExtend', keys=['lq']),
dict(type='FramesToTensor', keys=['lq', 'gt']),
dict(
type='Collect',
keys=['lq', 'gt'],
meta_keys=['lq_path', 'gt_path', 'key'])
]

data = dict(
workers_per_gpu=6,
train_dataloader=dict(samples_per_gpu=4, drop_last=True), # 2 gpus
val_dataloader=dict(samples_per_gpu=1),
test_dataloader=dict(samples_per_gpu=1, workers_per_gpu=1),

# train
train=dict(
type='RepeatDataset',
times=1000,
dataset=dict(
type=train_dataset_type,
lq_folder='data/vimeo90k/BDx4',
gt_folder='data/vimeo90k/GT',
ann_file='data/vimeo90k/meta_info_Vimeo90K_train_GT.txt',
pipeline=train_pipeline,
scale=4,
test_mode=False)),
# val
val=dict(
type=val_dataset_type,
lq_folder='data/Vid4/BDx4',
gt_folder='data/Vid4/GT',
pipeline=val_pipeline,
scale=4,
test_mode=True),
# test
test=dict(
type=test_dataset_type,
lq_folder='data/vimeo90k/BDx4',
gt_folder='data/vimeo90k/GT',
ann_file='data/vimeo90k/meta_info_Vimeo90K_test_GT.txt',
pipeline=test_pipeline,
scale=4,
num_input_frames=7,
test_mode=True),
)

# optimizer
optimizers = dict(
generator=dict(
type='Adam',
lr=2e-4,
betas=(0.9, 0.99),
paramwise_cfg=dict(custom_keys={'spynet': dict(lr_mult=0.125)})))

# learning policy
total_iters = 300000
lr_config = dict(
policy='CosineRestart',
by_epoch=False,
periods=[300000],
restart_weights=[1],
min_lr=1e-7)

checkpoint_config = dict(interval=5, save_optimizer=True, by_epoch=False)
# remove gpu_collect=True in non distributed training
evaluation = dict(interval=5000, save_image=False, gpu_collect=True)
log_config = dict(
interval=100,
hooks=[
dict(type='TextLoggerHook', by_epoch=False),
dict(type='TensorboardLoggerHook'),
])
visual_config = None

# runtime settings
dist_params = dict(backend='nccl')
log_level = 'INFO'
work_dir = f'./work_dirs/{exp_name}'
load_from = None
resume_from = None
workflow = [('train', 1)]
Loading