Mobile Deeplab-V3+ model for Segmentation

This project is used for deploying people segmentation model to mobile device and learning. The people segmentation android project is here. The model is another Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (Deeplab-V3+) implementation base on MobilenetV2 / MobilenetV3 on TensorFlow.

This project uses tf.estimator API to do training, and many code come from TensorFlow model,

Dataset

This project supports 2 dataset.

1. PASCAL VOC2012
   • Base dataset used for test.
   • You could download the augmented segmentation dataset from DrSleep.
2. Supervisely People dataset.
   • People Segmentation dataset.
   • More data could refer to the blog
   • Download from Google Drive, baidu pan, code: 8pdv

Environment

• TensorFlow v1.12.0
  1. No module named keras_applications: solution
  2. AttributeError: 'Estimator' object has no attribute '_distribution': solution.

Usage

0. Prepare pretrained MobilenetV2 model(optional)

   bash prepare.sh

1. Train
   • train on pascal voc 2012 dataset

   bash train_pascal_voc2012.sh
   # Tensorboard
   tensorboard --logdir=datasets/pascal_voc2012/exp/deeplab-v3-plus/train

   • train on people segmentation dataset

   mv people_segmentation.tar.gz datasets
   bash train_people_seg.sh
   # Tensorboard
   tensorboard --logdir=datasets/people_segmentation/exp/deeplab-v3-plus/train

2. Export and Freeze model
   • pascal voc 2012 dataset

   bash export_pascal_voc2012.sh
   # Output
   # 1. TensorFlow Serving: datasets/pascal_voc2012/exp/deeplab-v3-plus/export/
   # 2. Frozen model: datasets/pascal_voc2012/exp/deeplab-v3-plus/export/frozen_model.pb

   • people segmentation dataset

   bash export_people_seg.sh
   # Output
   # 1. TensorFlow Serving: datasets/people_segmentation/exp/deeplab-v3-plus/export/
   # 2. Frozen model: datasets/people_segmentation/exp/deeplab-v3-plus/export/frozen_model.pb

Explanation

You could modify the parameters and add new backbones network easily.

1. Use MobilenetV3 backbone: Change parameter from --backbone="MobielentV2" to --backbone="MobilenetV3"
2. quant_friendly: Remove BN layer after Depthwise Convolution, replace relu6 with relu, method from A Quantization-Friendly Separable Convolution for MobileNets.

Result

1. Pascal VOC2012

Backbone	Pretrained backbone	Augmented	Input Size	ASPP	Decoder	mIOU(%)
MobilenetV2	YES	YES	513x513	NO	NO	70.51

• Environment: base_lr=0.007, training_steps=30000, learning_policy=poly, num_gpu=1.

2. People Segmentation Dataset

Backbone	Input Size	Augmented	ASPP	Decoder	Quant-Friendly	Steps	mIOU(%)	Pretrained Model
MobilenetV2	256x256	YES	NO	YES	YES	150000	81.68	model
MobilenetV2	256x256	YES	YES	YES	YES	150000	83.33	model
MobilenetV3	256x256	YES	No	YES	YES	150000	81.86	model
MobilenetV2	513x513	YES	No	YES	YES	150000	89.55	model

• Environment: base_lr=0.05, training_steps=150000, learning_policy=poly, num_gpu=1.
• Augmented: use augmented dataset.

TODO

• Add LR-ASPP module in MobilenetV3.

References

1. https://github.com/tensorflow/models/tree/master/research/deeplab
2. https://github.com/rishizek/tensorflow-deeplab-v3-plus
3. https://github.com/kuan-wang/pytorch-mobilenet-v3/blob/master/mobilenetv3.py
4. https://github.com/DrSleep/tensorflow-deeplab-resnet