MobileNetV2 #9614

dwSun · 2018-01-29T14:56:59Z

Description

MobileNetV2 model from the
"Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation" <https://arxiv.org/abs/1801.04381>_ paper.

Checklist

Essentials

Passed code style checking (pylint)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Code is well-documented:
To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Comments

Not sure this is the correct implement, but this model did works.

szha

Thanks for the contribution! Would you put the mobilenet v2 class in the gluon model zoo? python/mxnet/gluon/model_zoo/vision/mobilenet.py

dwSun · 2018-01-30T15:59:59Z

I have put the MobileNetV2 class in the gluon vision model zoo.

I don't have pretrained models, so this should be modified:

We provide pre-trained models for all the listed models.
These models can constructed by passing pretrained=True:

BTW, pylint is so strict that I have to disable some rules.

marcoabreu · 2018-01-30T17:05:47Z

example/image-classification/symbols/mobilenetv2.py

+# under the License.
+
+# -*- coding:utf-8 -*-
+# pylint: disable= arguments-differ,line-too-long,invalid-name


Please provide reason for disabling pylint.

szha

Please delete duplicate code in example, and follow exactly the convention in existing interfaces here:
https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/model_zoo/vision/mobilenet.py#L78-L160

Don't worry about the doc and pretrained model as long as it's not added to the api doc yet.

szha · 2018-01-30T19:49:38Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

+        return x
+
+
+def get_mobilenetv2(w, pretrained=False, ctx=cpu(),


add a version argument to get_mobilenet.

szha · 2018-01-30T19:50:00Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

+    return net
+
+
+def mobilenetv2_1_0(**kwargs):


provide the same variants as the v1 mobilenet

szha · 2018-01-30T19:51:21Z

example/image-classification/symbols/mobilenetv2.py

+# under the License.
+
+# -*- coding:utf-8 -*-
+# pylint: disable= arguments-differ,line-too-long,invalid-name


Don't use file-level lint disable. Please use inline lint disables to identify the lines that violate the lint rules and we can try to decide whether to fix or disable.

szha · 2018-01-30T19:52:04Z

example/image-classification/symbols/mobilenetv2.py

+# -*- coding:utf-8 -*-
+# pylint: disable= arguments-differ,line-too-long,invalid-name
+'''
+MobileNetV2, implemented in Gluon.


delete duplicate and load from gluon model zoo instead.

dwSun · 2018-01-31T09:14:50Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

 from ....context import cpu
 from ...block import HybridBlock
-from ... import nn
+


iosrt made this change.

dwSun · 2018-01-31T09:16:08Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py


 # Helpers
+# pylint: disable= too-many-arguments


This is mobilenet v1 code, add this to pass the style check. Code not changed.

please remove and use our pylintrc as standard.

dwSun · 2018-01-31T09:17:50Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

                strides = [1, 2] * 3 + [1] * 5 + [2, 1]
+                # pylint: disable= invalid-name


This is mobilenet v1 code, add this to pass the style check.
autopep8 made those changes.

please use make pylint as the standard for checking code style. we have a 100 char-per-line limit.

dwSun · 2018-01-31T09:26:12Z

Requested changes have been made. I am trying to make my code more clear by merging commit logs.
Seems the situation not going as I wish.

szha · 2018-01-31T16:49:40Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

+
+                c_bgn = int(160 * multiplier)
+                c_end = int(320 * multiplier)
+                self.features.add(BottleNeck(c_in=c_bgn, c_out=c_end, t=6, s=1))


the construction looks like it could be refactored. can you use similar implementation from mobilenet v1?

szha · 2018-01-31T16:50:05Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

    Parameters
    ----------
+    ver : int, default 1


version. same below

dwSun · 2018-02-01T09:42:14Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

+                strides = [1, 2] * 2 + [1] * 2 + [2] + [1] * 3 + [1] * 3 + [2] + [1] * 3
+
+                for c_bgn, c_end, t, s in zip(c_bgns, c_ends, ts, strides):
+                    self.features.add(BottleNeck(c_in=c_bgn, c_out=c_end, t=t, s=s))


The code here looks ugly and very hard to follow the data flow.
Is this kind of code really necessary?

20 lines of self.features.add(BottleNeck... is unnecessarily verbose. I personally prefer the more compact code.

Could you use the same variable naming as in V1?

I prefer compact code too. But shouldn't we keep our code simply and clear.
Some people use the word 'scary' to describe code inside caffe and torch. Maybe it will be helpful to keep the code simple and stupid.
This kind of code take twice the time to write and more time to understand. As a programmer, I am a bit lazy. So, personally, I prefer previous version.

szha · 2018-02-01T18:04:38Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

+                self.features.add(nn.GlobalAvgPool2D())
+
+            self.output = nn.Conv2D(classes, 1, use_bias=False, prefix='pred_')
+            self.flatten = nn.Flatten(prefix='flat_')


don't introduce new field. flatten should be included in output.

szha · 2018-02-01T18:08:00Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

+    t : int
+        Layer expansion ratio.
+    s : int
+        strides


Use the argument naming from https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/gluon/model_zoo/vision/resnet.py#L95-L102

s -> stride

marcoabreu

While I don't feel comfortable having unchecked code, this is the way we apparently go for examples. Approved from my side.

szha · 2018-02-09T04:37:15Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

+                self.features.add(nn.GlobalAvgPool2D())
+
+            self.output = nn.HybridSequential(prefix='output_')
+            self.output.add(


with self.output.name_scope():
and manual prefix can be skipped.

A name_scope is indeed necessary.

zhreshold · 2018-02-14T00:02:59Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

+                channels_group = [int(x * multiplier) for x in [16] + [24] * 2 + [32] * 3
+                                  + [64] * 4 + [96] * 3 + [160] * 3 + [320]]
+                ts = [1] + [6] * 16
+                strides = [1, 2] * 2 + [1] * 2 + [2] + [1] * 3 + [1] * 3 + [2] + [1] * 3


I think there's a mistake in paper:

Therefore the strides here should be:

strides = [1, 2] * 2 + [1] * 3 + [2] + [1] * 2 + [1] * 3 + [2] + [1] * 3

More inputs and eyes are welcome if you find this not correct @dwSun @szha

BTW, I am comparing http://ethereon.github.io/netscope/#/gist/d01b5b8783b4582a42fe07bd46243986
with
plot.gv.pdf

The strides are correct in paper. We are applying stride 2 to last block of previous group of blocks.

BTW, we are using output node as top-most node in model graph. While the output node should be the last few lines in our code. So the graph is looking like upside down. Should it be more convenient if we revert it?

@dwSun from 28^2x32 to 28^2x64, it should be stride 1. I am observing 14^2x64 feature map, which corresponds to stride 2.
The input sizes and strides in this table is conflicting, but I am not sure which one is correct unless we compute the Mult-add and compare with the claims in paper.

My mistake. Those 2 lines indeed can't make sense. Will correct it in next commit.

dwSun · 2018-02-15T02:53:34Z

for reference, this is the struct I am using:

input shape     stride  output shape
# conv2d
224x224x3       2       112x112x32

# bottleneck
112x112x32      1       112x112x16

# bottleneck
112x112x16      2       56x56x24
56x56x24        1       56x56x24

# bottleneck
56x56x24        2       28x28x32
28x28x32        1       28x28x32
28x28x32        1       28x28x32

# bottleneck
28x28x32        1       28x28x64
28x28x64        1       28x28x64
28x28x64        1       28x28x64
28x28x64        1       28x28x64

# bottleneck
28x28x64        2       14x14x96
14x14x96        1       14x14x96
14x14x96        1       14x14x96

# bottleneck
14x14x96        2       7x7x160
7x7x160         1       7x7x160
7x7x160         1       7x7x160

# bottleneck
7x7x160         1       7x7x320

# conv2d
7x7x320         1       7x7x1280

# avgpool
7x7x1280        _       1x1x1280

# the output
1x1x1280        _       1x1xk

szha · 2018-02-15T03:52:18Z

LGTM. Would you mind adding the variants to tests/python/unittest/test_gluon_model_zoo.py? Click the unchecked checkbox at the top of this PR once you're done.

szha · 2018-02-15T03:55:02Z

@zhreshold please approve once you confirm that it's implemented correctly.

dwSun · 2018-02-15T09:24:47Z

unittest finished.

zhreshold · 2018-02-15T18:03:33Z

python/mxnet/gluon/model_zoo/vision/mobilenet.py

+            self.features = nn.HybridSequential(prefix='features_')
+            with self.features.name_scope():
+                _add_conv(self.features, int(32 * multiplier), kernel=3,
+                          stride=2, pad=1, active=False)


no relu for the first conv?

The MobileNetV2 paper not mentioned relu after first conv2d.
Also the channels of first conv2d is 32, while the channels of first bottleneck is 16, this doesn't sense.
I have tested 3 different model architectures:

with relu in first conv2d

without relu in first conv2d

without relu and use 16 instead of 32 as output channels in first conv2d

The result is a bit confusing, since performance among those architectures looks similar.
Here is the result:

The dataset contains 100 classes, 19W pics, all gray-scale image. This dataset maybe too small. But I don't have enough resource to test on coco or imagenet.

dwSun · 2018-02-18T00:48:40Z

Add relu in first conv2d as requested.

fix pylint style check error. add docstring, disable some check. delete duplicate code in example.

Jing-Luo · 2018-03-20T03:29:54Z

Any plan to release the pretrained model? I try to train the model by my own and it looks not so well...

For more details, please look at

https://discuss.gluon.ai/t/topic/5295

szha · 2018-03-20T03:36:38Z

@Jing-Luo It's currently WIP.

* MobileNetV2 * add mobilenetv2 to gluon vision model zoo. * code reformat use autopep8 and isort. fix pylint style check error. add docstring, disable some check. delete duplicate code in example. * change ver to version. * use exist helper. * model code refactor. * fix line too long. * remove invalid name option. * merge output operations. * change variables name * fix line too long. * s -> stride * add output name_scope * remove relu in first conv2d. * change block name from BottleNeck to LinearBottleneck. * resolve conflict * fix parameter name. * correct strides * add mobilenetv2 to unittest. * use autopep8 to reformat code. * add mobilenetv2 symbols to gluon.vision * add relu in 1st conv2d. * code refactor by using helpers. * split mobilenet v1 and v2 apis.

dwSun requested a review from szha as a code owner January 29, 2018 14:56

szha reviewed Jan 29, 2018

View reviewed changes

szha self-assigned this Jan 29, 2018

marcoabreu suggested changes Jan 30, 2018

View reviewed changes

szha suggested changes Jan 30, 2018

View reviewed changes

dwSun commented Jan 31, 2018

View reviewed changes

szha reviewed Jan 31, 2018

View reviewed changes

python/mxnet/gluon/model_zoo/vision/mobilenet.py Outdated

Parameters

----------

ver : int, default 1

Copy link

Member

szha Jan 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

version. same below

dwSun commented Feb 1, 2018

View reviewed changes

szha reviewed Feb 1, 2018

View reviewed changes

marcoabreu approved these changes Feb 1, 2018

View reviewed changes

szha reviewed Feb 9, 2018

View reviewed changes

zhreshold suggested changes Feb 14, 2018

View reviewed changes

zhreshold reviewed Feb 15, 2018

View reviewed changes

szha approved these changes Feb 15, 2018

View reviewed changes

zhreshold approved these changes Feb 18, 2018

View reviewed changes

dwSun added 4 commits February 21, 2018 20:58

MobileNetV2

32d677a

add mobilenetv2 to gluon vision model zoo.

a9ff4a4

code reformat use autopep8 and isort.

d91f562

fix pylint style check error. add docstring, disable some check. delete duplicate code in example.

resolve conflict

66b43d9

dwSun added 20 commits February 21, 2018 20:58

change ver to version.

5e42af3

use exist helper.

e52e201

model code refactor.

9206f5b

fix line too long.

776ff38

remove invalid name option.

506582f

merge output operations.

31016c8

change variables name

098950c

fix line too long.

877d499

s -> stride

3754c38

add output name_scope

7e72f92

remove relu in first conv2d.

98ca217

change block name from BottleNeck to LinearBottleneck.

988d7d4

fix parameter name.

5cceadf

correct strides

4798599

add mobilenetv2 to unittest.

6012ada

use autopep8 to reformat code.

ea49267

add mobilenetv2 symbols to gluon.vision

58e7026

add relu in 1st conv2d.

9220a34

code refactor by using helpers.

6d47271

split mobilenet v1 and v2 apis.

c071ed8

szha merged commit 5ca6efe into apache:master Feb 21, 2018

		strides = [1, 2] * 3 + [1] * 5 + [2, 1]
		# pylint: disable= invalid-name

MobileNetV2 #9614

MobileNetV2 #9614

Conversation

dwSun commented Jan 29, 2018 • edited Loading

Description

Checklist

Essentials

Comments

szha left a comment

Choose a reason for hiding this comment

dwSun commented Jan 30, 2018

Choose a reason for hiding this comment

szha left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dwSun commented Jan 31, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

szha Feb 8, 2018 • edited Loading

Choose a reason for hiding this comment

marcoabreu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dwSun Feb 14, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dwSun commented Feb 15, 2018

szha commented Feb 15, 2018 • edited Loading

szha commented Feb 15, 2018

dwSun commented Feb 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dwSun commented Feb 18, 2018

Jing-Luo commented Mar 20, 2018

szha commented Mar 20, 2018

dwSun commented Jan 29, 2018 •

edited

Loading

szha Feb 8, 2018 •

edited

Loading

dwSun Feb 14, 2018 •

edited

Loading

szha commented Feb 15, 2018 •

edited

Loading