[Feature] Support ConvNeXt #1216

MengzhangLI · 2022-01-18T09:33:19Z

Motivation

Support ConvNext.
Paper: https://arxiv.org/abs/2201.03545
Github: https://github.com/facebookresearch/ConvNeXt

Modification

Need some refactor. WIP.

Refactor backbone code.
Refactor mmcv_custome code and align inference metric.
Add unit test
Align training metric.

MengzhangLI · 2022-01-19T10:23:50Z

In our refactor code, (NCHW -> permute to NHWC -> PyTorch LN -> linear layers -> layer scale -> permute back to NCHW) takes over (NCHW -> custom LN -> 1x1 convs -> layer scale), the former style may be faster.

Reference: facebookresearch/ConvNeXt#18.

xvjiarui

Please refer to ResNet for some details

mmseg/models/backbones/convnext.py

codecov · 2022-01-21T07:46:30Z

Codecov Report

Merging #1216 (b77b971) into master (2d66179) will increase coverage by 0.04%.
The diff coverage is 94.93%.

@@            Coverage Diff             @@
##           master    #1216      +/-   ##
==========================================
+ Coverage   90.27%   90.32%   +0.04%     
==========================================
  Files         131      132       +1     
  Lines        7621     7699      +78     
  Branches     1267     1290      +23     
==========================================
+ Hits         6880     6954      +74     
  Misses        531      531              
- Partials      210      214       +4

Flag	Coverage Δ
unittests	`90.32% <94.93%> (+0.04%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...eg/core/utils/layer_decay_optimizer_constructor.py	`94.80% <94.80%> (ø)`
mmseg/core/utils/__init__.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 369a2ee...b77b971. Read the comment docs.

mzr1996 · 2022-01-27T07:14:40Z

Please look at open-mmlab/mmpretrain#670

…into ConvNext

mmseg/core/utils/layer_decay_optimizer_constructor.py

tests/test_core/test_learning_rate_decay_optimizer_constructor.py

mmseg/core/utils/layer_decay_optimizer_constructor.py

tests/test_core/test_learning_rate_decay_optimizer_constructor.py

mmseg/core/utils/layer_decay_optimizer_constructor.py

tests/test_core/test_learning_rate_decay_optimizer_constructor.py

MeowZheng

might add metafile and readme

tests/test_core/test_learning_rate_decay_optimizer_constructor.py

README_zh-CN.md

README.md

mmseg/core/utils/layer_decay_optimizer_constructor.py

configs/_base_/models/upernet_convnext.py

configs/convnext/README.md

configs/convnext/upernet_convnext_tiny_fp16_512x512_160k_ade20k.py

configs/convnext/upernet_convnext_small_fp16_512x512_160k_ade20k.py

configs/convnext/upernet_convnext_xlarge_fp16_640x640_160k_ade20k.py

mmseg/core/utils/layer_decay_optimizer_constructor.py

tests/test_core/test_learning_rate_decay_optimizer_constructor.py

…_layer()

…into ConvNext

mmseg/core/utils/layer_decay_optimizer_constructor.py

* upload original backbone and configs * ConvNext Refactor * ConvNext Refactor * convnext customization refactor with mmseg style * convnext customization refactor with mmseg style * add ade20k_640x640.py * upload files for training * delete dist_optimizer_hook and remove layer_decay_optimizer_constructor * check max(out_indices) < num_stages * add unittest * fix lint error * use MMClassification backbone * fix bugs in base_1k * add mmcls in requirements/mminstall.txt * add mmcls in requirements/mminstall.txt * fix drop_path_rate and layer_scale_init_value * use logger.info instead of print * add mmcls in runtime.txt * fix f string && delete * add doctring in LearningRateDecayOptimizerConstructor and fix mmcls version in requirements * fix typo in LearningRateDecayOptimizerConstructor * use ConvNext models in unit test for LearningRateDecayOptimizerConstructor * add unit test * fix typo * fix typo * add layer_wise and fix redundant backbone.downsample_norm in it * fix unit test * give a ground truth lr_scale and weight_decay * upload models and readme * delete 'backbone.stem_norm' and 'backbone.downsample_norm' in get_num_layer() * fix unit test and use mmcls url * update md2yml.py and metafile * fix typo

* make sure fp16 runs well * add fp16 test for superes * Update src/diffusers/models/unet_2d.py Co-authored-by: Pedro Cuenca <[email protected]> * gen on cuda * always run fast inferecne test on cpu * run on cpu Co-authored-by: Pedro Cuenca <[email protected]>

upload original backbone and configs

b66ab67

MengzhangLI added the WIP Work in process label Jan 18, 2022

MengzhangLI self-assigned this Jan 18, 2022

MengzhangLI added 2 commits January 19, 2022 17:44

ConvNext Refactor

abceb28

ConvNext Refactor

6c48587

MengzhangLI requested review from xvjiarui and MeowZheng January 19, 2022 10:00

xvjiarui reviewed Jan 19, 2022

View reviewed changes

MengzhangLI added 4 commits January 20, 2022 16:01

convnext customization refactor with mmseg style

7260a4f

convnext customization refactor with mmseg style

46a529a

add ade20k_640x640.py

fd73adc

upload files for training

8cb0748

MengzhangLI added 4 commits January 22, 2022 22:54

delete dist_optimizer_hook and remove layer_decay_optimizer_constructor

e0892c6

check max(out_indices) < num_stages

9289262

add unittest

2d998a3

fix lint error

adcdd2c

shinya7y mentioned this pull request Jan 22, 2022

[Feature] Support TIMMBackbone open-mmlab/mmdetection#7020

Merged

MengzhangLI added 5 commits February 10, 2022 20:25

use MMClassification backbone

01c0d93

fix bugs in base_1k

6a5592a

Merge branch 'master' of https://github.com/open-mmlab/mmsegmentation …

bc78936

…into ConvNext

add mmcls in requirements/mminstall.txt

54837c2

add mmcls in requirements/mminstall.txt

575312b

Ezra-Yu mentioned this pull request Feb 15, 2022

[Feature] Support ConvNeXt open-mmlab/mmpretrain#670

Merged

6 tasks

Merge branch 'master' of https://github.com/open-mmlab/mmsegmentation …

f9d4068

…into ConvNext

MeowZheng reviewed Feb 24, 2022

View reviewed changes

mmseg/core/utils/layer_decay_optimizer_constructor.py Outdated Show resolved Hide resolved

MengzhangLI added 2 commits February 24, 2022 16:21

fix drop_path_rate and layer_scale_init_value

bb4f35a

use logger.info instead of print

a7efb3b

MengzhangLI added 2 commits March 2, 2022 19:46

add unit test

e8db7fe

fix typo

b424568

MeowZheng reviewed Mar 2, 2022

View reviewed changes

tests/test_core/test_learning_rate_decay_optimizer_constructor.py Outdated Show resolved Hide resolved

mmseg/core/utils/layer_decay_optimizer_constructor.py Outdated Show resolved Hide resolved

MengzhangLI added 2 commits March 2, 2022 22:06

fix typo

d2a3ea4

add layer_wise and fix redundant backbone.downsample_norm in it

e671ee3

MengzhangLI commented Mar 2, 2022

View reviewed changes

mmseg/core/utils/layer_decay_optimizer_constructor.py Outdated Show resolved Hide resolved

MeowZheng reviewed Mar 3, 2022

View reviewed changes

fix unit test

755dc7e

MeowZheng reviewed Mar 3, 2022

View reviewed changes

tests/test_core/test_learning_rate_decay_optimizer_constructor.py Outdated Show resolved Hide resolved

MengzhangLI added 2 commits March 3, 2022 21:29

give a ground truth lr_scale and weight_decay

09be63b

upload models and readme

3c8436a

MengzhangLI commented Mar 3, 2022

View reviewed changes

README_zh-CN.md Outdated Show resolved Hide resolved

MengzhangLI commented Mar 3, 2022

View reviewed changes

README.md Outdated Show resolved Hide resolved

MengzhangLI commented Mar 3, 2022

View reviewed changes

mmseg/core/utils/layer_decay_optimizer_constructor.py Outdated Show resolved Hide resolved

MeowZheng reviewed Mar 4, 2022

View reviewed changes

MengzhangLI added 4 commits March 4, 2022 12:45

delete 'backbone.stem_norm' and 'backbone.downsample_norm' in get_num…

614c352

…_layer()

fix unit test and use mmcls url

c39ff5f

Merge branch 'master' of https://github.com/open-mmlab/mmsegmentation …

dd6e669

…into ConvNext

update md2yml.py and metafile

2845ba7

MeowZheng reviewed Mar 4, 2022

View reviewed changes

mmseg/core/utils/layer_decay_optimizer_constructor.py Outdated Show resolved Hide resolved

mmseg/core/utils/layer_decay_optimizer_constructor.py Outdated Show resolved Hide resolved

fix typo

b77b971

MeowZheng approved these changes Mar 4, 2022

View reviewed changes

MeowZheng merged commit 7ddd2fe into open-mmlab:master Mar 4, 2022

MengzhangLI removed the WIP Work in process label Mar 4, 2022

MengzhangLI changed the title ~~[Feature] Support ConvNext~~ [Feature] Support ConvNeXt Mar 4, 2022

MengzhangLI mentioned this pull request Mar 6, 2022

Invitation of making PR for OpenMMLab / MMSegmentation. sail-sg/poolformer#30

Closed

MengzhangLI deleted the ConvNext branch March 17, 2022 09:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Support ConvNeXt #1216

[Feature] Support ConvNeXt #1216

MengzhangLI commented Jan 18, 2022 •

edited by RockeyCoss

Loading

MengzhangLI commented Jan 19, 2022

xvjiarui left a comment

codecov bot commented Jan 21, 2022 •

edited

Loading

mzr1996 commented Jan 27, 2022

MeowZheng left a comment

[Feature] Support ConvNeXt #1216

[Feature] Support ConvNeXt #1216

Conversation

MengzhangLI commented Jan 18, 2022 • edited by RockeyCoss Loading

Motivation

Modification

MengzhangLI commented Jan 19, 2022

xvjiarui left a comment

Choose a reason for hiding this comment

codecov bot commented Jan 21, 2022 • edited Loading

Codecov Report

mzr1996 commented Jan 27, 2022

MeowZheng left a comment

Choose a reason for hiding this comment

MengzhangLI commented Jan 18, 2022 •

edited by RockeyCoss

Loading

codecov bot commented Jan 21, 2022 •

edited

Loading