Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] fix model index #1559

Merged
merged 3 commits into from
Dec 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion .dev_scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
- [8. Deterministic training](#8-deterministic-training)
- [9. Automatically check links](#9-automatically-check-links)
- [10. Calculate flops](#10-calculate-flops)
- [11. Update model idnex](#11-update-model-index)

## 1. Check UT

Expand Down Expand Up @@ -237,10 +238,18 @@ You can specify the `--target` by a file or a directory.

**Notes:** DO NOT use it in CI, because requiring too many http requirements by CI will cause 503 and CI will propabaly fail.

10. Calculate flops
## 10. Calculate flops

To summarize the flops of different models, you can run the following commands:

```bash
python .dev_scripts/benchmark_valid_flop.py --flops --flops-str
```

## 11. Update model index

To update model-index according to `README.md`, please run the following commands,

```bash
python .dev_scripts/update_model_index.py
```
10 changes: 7 additions & 3 deletions .dev_scripts/update_model_index.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ def dump_yaml_and_check_difference(obj, file):

if osp.isfile(file):
file_exists = True
print(f' exist {file}')
# print(f' exist {file}')
with open(file, 'r', encoding='utf-8') as f:
str_orig = f.read()
else:
Expand Down Expand Up @@ -181,13 +181,15 @@ def parse_md(md_file):
f'line {i+1} in {md_file}')
i += 1
continue

if 'Method' in cols:
config_idx = cols.index('Method')
elif 'Config' in cols:
config_idx = cols.index('Config')
else:
print(cols)
raise ValueError('Cannot find config Table.')

checkpoint_idx = cols.index('Download')
try:
flops_idx = cols.index('FLOPs')
Expand Down Expand Up @@ -318,7 +320,7 @@ def parse_md(md_file):
i += 1

if len(models) == 0:
warnings.warn('no model is found in this md file')
warnings.warn(f'no model is found in {md_file}')

result = {'Collections': [collection], 'Models': models}
yml_file = md_file.replace('README.md', 'metafile.yml')
Expand Down Expand Up @@ -366,9 +368,11 @@ def update_model_index():
sys.exit(0)

file_modified = False
# pbar = tqdm.tqdm(range(len(file_list)), initial=0, dynamic_ncols=True)
for fn in file_list:
print(f'process {fn}')
file_modified |= parse_md(fn)
# pbar.update(1)
# pbar.set_description(f'processing {fn}')

file_modified |= update_model_index()

Expand Down
29 changes: 6 additions & 23 deletions configs/basicvsr_pp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,29 +39,12 @@ The pretrained weights of SPyNet can be found [here](https://download.openmmlab.

Note that the following models are finetuned from smaller models. The training schemes of these models will be released when MMEditing reaches 5k stars. We provide the pre-trained models here.

**NTIRE 2021 Video Super-Resolution**

[basicvsr-pp_c128n25_600k_ntire-vsr](./basicvsr-pp_c128n25_600k_ntire-vsr.py)

[model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_vsr_20210311-1ff35292.pth)

**NTIRE 2021 Quality Enhancement of Compressed Video - Track 1**

[basicvsr-pp_c128n25_600k_ntire-decompress-track1](./basicvsr-pp_c128n25_600k_ntire-decompress-track1.py)

[model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track1_20210223-7b2eba02.pth)

**NTIRE 2021 Quality Enhancement of Compressed Video - Track 2**

[basicvsr-pp_c128n25_600k_ntire-decompress-track2](./basicvsr-pp_c128n25_600k_ntire-decompress-track2.py)

[model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track2_20210314-eeae05e6.pth)

**NTIRE 2021 Quality Enhancement of Compressed Video - Track 3**

[basicvsr-pp_c128n25_600k_ntire-decompress-track3](./basicvsr-pp_c128n25_600k_ntire-decompress-track3.py)

[model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track3_20210304-6daf4a40.pth)
| Method | Download | Track |
| :-------------------------------------------------------------------: | :---------------------------------------------------------------------: | :----------------------------------------------------------: |
| [basicvsr-pp_c128n25_600k_ntire-vsr](./basicvsr-pp_c128n25_600k_ntire-vsr.py) | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_vsr_20210311-1ff35292.pth) | NTIRE 2021 Video Super-Resolution |
| [basicvsr-pp_c128n25_600k_ntire-decompress-track1](./basicvsr-pp_c128n25_600k_ntire-decompress-track1.py) | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track1_20210223-7b2eba02.pth) | NTIRE 2021 Quality Enhancement of Compressed Video - Track 1 |
| [basicvsr-pp_c128n25_600k_ntire-decompress-track2](./basicvsr-pp_c128n25_600k_ntire-decompress-track2.py) | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track2_20210314-eeae05e6.pth) | NTIRE 2021 Quality Enhancement of Compressed Video - Track 2 |
| [basicvsr-pp_c128n25_600k_ntire-decompress-track3](./basicvsr-pp_c128n25_600k_ntire-decompress-track3.py) | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track3_20210304-6daf4a40.pth) | NTIRE 2021 Quality Enhancement of Compressed Video - Track 3 |

</details>

Expand Down
13 changes: 6 additions & 7 deletions configs/basicvsr_pp/README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,12 @@ SPyNet 的 预训练权重在[这里](https://download.openmmlab.com/mmediting/r

请注意,以下模型是从较小的模型中微调而来的。 这些模型的训练方案将在 MMEditing 达到 5k star 时发布。 我们在这里提供预训练的模型。

[NTIRE 2021 Video Super-Resolution](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_vsr_20210311-1ff35292.pth)

[NTIRE 2021 Quality Enhancement of Compressed Video - Track 1](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track1_20210223-7b2eba02.pth)

[NTIRE 2021 Quality Enhancement of Compressed Video - Track 2](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track2_20210314-eeae05e6.pth)

[NTIRE 2021 Quality Enhancement of Compressed Video - Track 3](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track3_20210304-6daf4a40.pth)
| 算法 | 模型 | 赛道 |
| ---------------------------------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------ |
| [basicvsr-pp_c128n25_600k_ntire-vsr](./basicvsr-pp_c128n25_600k_ntire-vsr.py) | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_vsr_20210311-1ff35292.pth) | NTIRE 2021 Video Super-Resolution |
| [basicvsr-pp_c128n25_600k_ntire-decompress-track1](./basicvsr-pp_c128n25_600k_ntire-decompress-track1.py) | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track1_20210223-7b2eba02.pth) | NTIRE 2021 Quality Enhancement of Compressed Video - Track 1 |
| [basicvsr-pp_c128n25_600k_ntire-decompress-track2](./basicvsr-pp_c128n25_600k_ntire-decompress-track2.py) | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track2_20210314-eeae05e6.pth) | NTIRE 2021 Quality Enhancement of Compressed Video - Track 2 |
| [basicvsr-pp_c128n25_600k_ntire-decompress-track3](./basicvsr-pp_c128n25_600k_ntire-decompress-track3.py) | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track3_20210304-6daf4a40.pth) | NTIRE 2021 Quality Enhancement of Compressed Video - Track 3 |

</details>
```
Expand Down
40 changes: 40 additions & 0 deletions configs/basicvsr_pp/metafile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -109,3 +109,43 @@ Models:
Vimeo-90K-T (BIx4) SSIM (Y): 0.9351
Task: Video Super-Resolution
Weights: https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c64n7_8x1_300k_vimeo90k_bd_20210305-ab315ab1.pth
- Config: configs/basicvsr_pp/basicvsr-pp_c128n25_600k_ntire-vsr.py
In Collection: BasicVSR++
Metadata:
Training Data: Others
Name: basicvsr-pp_c128n25_600k_ntire-vsr
Results:
- Dataset: Others
Metrics: {}
Task: Video Super-Resolution
Weights: https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_vsr_20210311-1ff35292.pth
- Config: configs/basicvsr_pp/basicvsr-pp_c128n25_600k_ntire-decompress-track1.py
In Collection: BasicVSR++
Metadata:
Training Data: Others
Name: basicvsr-pp_c128n25_600k_ntire-decompress-track1
Results:
- Dataset: Others
Metrics: {}
Task: Video Super-Resolution
Weights: https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track1_20210223-7b2eba02.pth
- Config: configs/basicvsr_pp/basicvsr-pp_c128n25_600k_ntire-decompress-track2.py
In Collection: BasicVSR++
Metadata:
Training Data: Others
Name: basicvsr-pp_c128n25_600k_ntire-decompress-track2
Results:
- Dataset: Others
Metrics: {}
Task: Video Super-Resolution
Weights: https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track2_20210314-eeae05e6.pth
- Config: configs/basicvsr_pp/basicvsr-pp_c128n25_600k_ntire-decompress-track3.py
In Collection: BasicVSR++
Metadata:
Training Data: Others
Name: basicvsr-pp_c128n25_600k_ntire-decompress-track3
Results:
- Dataset: Others
Metrics: {}
Task: Video Super-Resolution
Weights: https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c128n25_ntire_decompress_track3_20210304-6daf4a40.pth

This file was deleted.

4 changes: 4 additions & 0 deletions configs/cyclegan/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,17 @@ https://download.openmmlab.com/mmediting/cyclegan/refactor/cyclegan_in_1x1_80k_f

`FID` comparison with official:

<!-- SKIP THIS TABLE -->

| Dataset | facades | facades-id0 | summer2winter | summer2winter-id0 | winter2summer | winter2summer-id0 | horse2zebra | horse2zebra-id0 | zebra2horse | zebra2horse-id0 | average |
| :------: | :---------: | :---------: | :-----------: | :---------------: | :-----------: | :---------------: | :---------: | :-------------: | :---------: | :-------------: | :--------: |
| official | **123.626** | **119.726** | **77.342** | **76.773** | **72.631** | 74.239 | **62.111** | 77.202 | **138.646** | **137.050** | **95.935** |
| ours | 124.8033 | 125.1694 | 83.7177 | 83.1418 | 72.8025 | **73.5001** | 64.5225 | **74.7770** | 141.1571 | **134.3728** | 97.79 |

`IS` comparison with evaluation:

<!-- SKIP THIS TABLE -->

| Dataset | facades | facades-id0 | summer2winter | summer2winter-id0 | winter2summer | winter2summer-id0 | horse2zebra | horse2zebra-id0 | zebra2horse | zebra2horse-id0 | average |
| :------: | :-------: | :---------: | :-----------: | :---------------: | :-----------: | :---------------: | :---------: | :-------------: | :---------: | :-------------: | :-------: |
| official | 1.638 | 1.697 | 2.762 | **2.750** | **3.293** | **3.110** | 1.375 | **1.584** | **3.186** | 3.047 | 2.444 |
Expand Down
15 changes: 8 additions & 7 deletions configs/dim/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,14 @@ Image matting is a fundamental computer vision problem and has many applications

## Results and models

| Method | SAD | MSE | GRAD | CONN | GPU Info | Download |
| :-------------------------------------------------------: | :------: | :-------: | :------: | :------: | :------: | :----------------------------------------------------------------------------------------------------: |
| stage1 (paper) | 54.6 | 0.017 | 36.7 | 55.3 | - | - |
| stage3 (paper) | **50.4** | **0.014** | 31.0 | 50.8 | - | - |
| [stage1 (our)](./dim_stage1-v16_1xb1-1000k_comp1k.py) | 53.8 | 0.017 | 32.7 | 54.5 | 1 | [model](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage1_v16_1x1_1000k_comp1k_SAD-53.8_20200605_140257-979a420f.pth) \| [log](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage1_v16_1x1_1000k_comp1k_20200605_140257.log.json) |
| [stage2 (our)](./dim_stage2-v16-pln_1xb1-1000k_comp1k.py) | 52.3 | 0.016 | 29.4 | 52.4 | 1 | [model](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage2_v16_pln_1x1_1000k_comp1k_SAD-52.3_20200607_171909-d83c4775.pth) \| [log](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage2_v16_pln_1x1_1000k_comp1k_20200607_171909.log.json) |
| [stage3 (our)](./dim_stage3-v16-pln_1xb1-1000k_comp1k.py) | 50.6 | 0.015 | **29.0** | **50.7** | 1 | [model](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage3_v16_pln_1x1_1000k_comp1k_SAD-50.6_20200609_111851-647f24b6.pth) \| [log](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage3_v16_pln_1x1_1000k_comp1k_20200609_111851.log.json) |
| Method | SAD | MSE | GRAD | CONN | GPU Info | Download |
| :-------------------------------------------------------------------------: | :------: | :-------: | :------: | :------: | :------: | :----------------------------------------------------------------------------------: |
| stage1 (paper) | 54.6 | 0.017 | 36.7 | 55.3 | - | - |
| stage3 (paper) | **50.4** | **0.014** | 31.0 | 50.8 | - | - |
| [stage1 (our)](./dim_stage1-v16_1xb1-1000k_comp1k.py) | 53.8 | 0.017 | 32.7 | 54.5 | 1 | [model](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage1_v16_1x1_1000k_comp1k_SAD-53.8_20200605_140257-979a420f.pth) \| [log](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage1_v16_1x1_1000k_comp1k_20200605_140257.log.json) |
| [stage2 (our)](./dim_stage2-v16-pln_1xb1-1000k_comp1k.py) | 52.3 | 0.016 | 29.4 | 52.4 | 1 | [model](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage2_v16_pln_1x1_1000k_comp1k_SAD-52.3_20200607_171909-d83c4775.pth) \| [log](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage2_v16_pln_1x1_1000k_comp1k_20200607_171909.log.json) |
| [stage3 (our)](./dim_stage3-v16-pln_1xb1-1000k_comp1k.py) | 50.6 | 0.015 | **29.0** | **50.7** | 1 | [model](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage3_v16_pln_1x1_1000k_comp1k_SAD-50.6_20200609_111851-647f24b6.pth) \| [log](https://download.openmmlab.com/mmediting/mattors/dim/dim_stage3_v16_pln_1x1_1000k_comp1k_20200609_111851.log.json) |
| [stage1 (online merge)](./dim_stage1-v16_1xb1-1000k_comp1k_online-merge.py) | - | - | - | - | - | - |

**NOTE**

Expand Down
Loading