Skip to content

Commit

Permalink
Update supported mmseg models (#181)
Browse files Browse the repository at this point in the history
* fix ocrnet cascade decoder

* update mmseg support models

* update mmseg configs

* support emanet and icnet

* set max K of TopK for tensorrt

* update supported models for mmseg in docs

* add test for emamodule

* add configs and update docs

* Update docs

* update benchmark
  • Loading branch information
RunningLeon authored and lvhan028 committed Apr 1, 2022
1 parent db79695 commit 5adbfc5
Show file tree
Hide file tree
Showing 20 changed files with 700 additions and 8 deletions.
3 changes: 0 additions & 3 deletions configs/mmseg/segmentation_ncnn_static.py

This file was deleted.

3 changes: 3 additions & 0 deletions configs/mmseg/segmentation_onnxruntime_static-1024x2048.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
_base_ = ['./segmentation_static.py', '../_base_/backends/onnxruntime.py']

onnx_config = dict(input_shape=[2048, 1024])
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
_base_ = ['./segmentation_static.py', '../_base_/backends/onnxruntime.py']

onnx_config = dict(input_shape=None)
onnx_config = dict(input_shape=[512, 512])
4 changes: 4 additions & 0 deletions configs/mmseg/segmentation_openvino_static-512x512.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
_base_ = ['./segmentation_static.py', '../_base_/backends/openvino.py']

backend_config = dict(
model_inputs=[dict(opt_shapes=dict(input=[1, 3, 512, 512]))])
13 changes: 13 additions & 0 deletions configs/mmseg/segmentation_tensorrt-fp16_static-1024x1024.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
_base_ = ['./segmentation_static.py', '../_base_/backends/tensorrt-fp16.py']

onnx_config = dict(input_shape=[1024, 1024])
backend_config = dict(
common_config=dict(max_workspace_size=1 << 30),
model_inputs=[
dict(
input_shapes=dict(
input=dict(
min_shape=[1, 3, 1024, 1024],
opt_shape=[1, 3, 1024, 1024],
max_shape=[1, 3, 1024, 1024])))
])
13 changes: 13 additions & 0 deletions configs/mmseg/segmentation_tensorrt-fp16_static-512x512.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
_base_ = ['./segmentation_static.py', '../_base_/backends/tensorrt-fp16.py']

onnx_config = dict(input_shape=[512, 512])
backend_config = dict(
common_config=dict(max_workspace_size=1 << 30),
model_inputs=[
dict(
input_shapes=dict(
input=dict(
min_shape=[1, 3, 512, 512],
opt_shape=[1, 3, 512, 512],
max_shape=[1, 3, 512, 512])))
])
13 changes: 13 additions & 0 deletions configs/mmseg/segmentation_tensorrt-int8_static-1024x1024.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
_base_ = ['./segmentation_static.py', '../_base_/backends/tensorrt-int8.py']

onnx_config = dict(input_shape=[1024, 1024])
backend_config = dict(
common_config=dict(max_workspace_size=1 << 30),
model_inputs=[
dict(
input_shapes=dict(
input=dict(
min_shape=[1, 3, 1024, 1024],
opt_shape=[1, 3, 1024, 1024],
max_shape=[1, 3, 1024, 1024])))
])
13 changes: 13 additions & 0 deletions configs/mmseg/segmentation_tensorrt-int8_static-512x512.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
_base_ = ['./segmentation_static.py', '../_base_/backends/tensorrt-int8.py']

onnx_config = dict(input_shape=[512, 512])
backend_config = dict(
common_config=dict(max_workspace_size=1 << 30),
model_inputs=[
dict(
input_shapes=dict(
input=dict(
min_shape=[1, 3, 512, 512],
opt_shape=[1, 3, 512, 512],
max_shape=[1, 3, 512, 512])))
])
13 changes: 13 additions & 0 deletions configs/mmseg/segmentation_tensorrt_static-1024x1024.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
_base_ = ['./segmentation_static.py', '../_base_/backends/tensorrt.py']

onnx_config = dict(input_shape=[1024, 1024])
backend_config = dict(
common_config=dict(max_workspace_size=1 << 30),
model_inputs=[
dict(
input_shapes=dict(
input=dict(
min_shape=[1, 3, 1024, 1024],
opt_shape=[1, 3, 1024, 1024],
max_shape=[1, 3, 1024, 1024])))
])
13 changes: 13 additions & 0 deletions configs/mmseg/segmentation_tensorrt_static-512x512.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
_base_ = ['./segmentation_static.py', '../_base_/backends/tensorrt.py']

onnx_config = dict(input_shape=[512, 512])
backend_config = dict(
common_config=dict(max_workspace_size=1 << 30),
model_inputs=[
dict(
input_shapes=dict(
input=dict(
min_shape=[1, 3, 512, 512],
opt_shape=[1, 3, 512, 512],
max_shape=[1, 3, 512, 512])))
])
234 changes: 234 additions & 0 deletions docs/en/benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -1480,6 +1480,240 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](tut
<td align="center">-</td>
<td>$MMSEG_DIR/configs/unet/fcn_unet_s5-d16_4x4_512x1024_160k_cityscapes.py</td>
</tr>
<tr>
<td align="center">ANN</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.40</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">77.32</td>
<td align="center">77.32</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/ann/ann_r50-d8_512x1024_40k_cityscapes.py</td>
</tr>
<tr>
<td align="center">APCNet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.40</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">77.32</td>
<td align="center">77.32</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/apcnet/apcnet_r50-d8_512x1024_40k_cityscapes.py</td>
</tr>
<tr>
<td align="center">BiSeNetV1</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">74.44</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">74.44</td>
<td align="center">74.43</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/bisenetv1/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes.py</td>
</tr>
<tr>
<td align="center">BiSeNetV2</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">73.21</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">73.21</td>
<td align="center">73.21</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/bisenetv2/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes.py</td>
</tr>
<tr>
<td align="center">CGNet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">68.25</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">68.27</td>
<td align="center">68.27</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/cgnet/cgnet_512x1024_60k_cityscapes.py</td>
</tr>
<tr>
<td align="center">EMANet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.59</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">77.59</td>
<td align="center">77.6</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/emanet/emanet_r50-d8_512x1024_80k_cityscapes.py</td>
</tr>
<tr>
<td align="center">EncNet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">75.67</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">75.66</td>
<td align="center">75.66</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/encnet/encnet_r50-d8_512x1024_40k_cityscapes.py</td>
</tr>
<tr>
<td align="center">ERFNet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">71.08</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">71.08</td>
<td align="center">71.07</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/erfnet/erfnet_fcn_4x4_512x1024_160k_cityscapes.py</td>
</tr>
<tr>
<td align="center">FastFCN</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">79.12</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">79.12</td>
<td align="center">79.12</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py</td>
</tr>
<tr>
<td align="center">GCNet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.69</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">77.69</td>
<td align="center">77.69</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/gcnet/gcnet_r50-d8_512x1024_40k_cityscapes.py</td>
</tr>
<tr>
<td align="center">ICNet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">76.29</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">76.36</td>
<td align="center">76.36</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/icnet/icnet_r18-d8_832x832_80k_cityscapes.py</td>
</tr>
<tr>
<td align="center">ISANet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">78.49</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">78.49</td>
<td align="center">78.49</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/isanet/isanet_r50-d8_512x1024_40k_cityscapes.py</td>
</tr>
<tr>
<td align="center">OCRNet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">74.30</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">73.66</td>
<td align="center">73.67</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/ocrnet/ocrnet_hr18s_512x1024_40k_cityscapes.py</td>
</tr>
<tr>
<td align="center">PointRend</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">76.47</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">76.41</td>
<td align="center">76.42</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/point_rend/pointrend_r50_512x1024_80k_cityscapes.py</td>
</tr>
<tr>
<td align="center">Semantic FPN</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">74.52</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">74.52</td>
<td align="center">74.52</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/sem_fpn/fpn_r50_512x1024_80k_cityscapes.py</td>
</tr>
<tr>
<td align="center">STDC</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">75.10</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">75.10</td>
<td align="center">75.10</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/stdc/stdc1_in1k-pre_512x1024_80k_cityscapes.py</td>
</tr>
<tr>
<td align="center">STDC</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.17</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">77.17</td>
<td align="center">77.17</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/stdc/stdc2_in1k-pre_512x1024_80k_cityscapes.py</td>
</tr>
<tr>
<td align="center">UPerNet</td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.10</td>
<td align="center">-</td>
<td align="center">-</td>
<td align="center">77.19</td>
<td align="center">77.18</td>
<td align="center">-</td>
<td align="center">-</td>
<td>$MMSEG_DIR/configs/upernet/upernet_r50_512x1024_40k_cityscapes.py</td>
</tr>
</tbody>
</table>
</div>
Expand Down
27 changes: 27 additions & 0 deletions docs/en/codebases/mmseg.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,33 @@ Please refer to [get_started.md](https://github.com/open-mmlab/mmsegmentation/bl
| DeepLabV3+ | Y | Y | Y | Y | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/deeplabv3plus) |
| Fast-SCNN[*](#static_shape) | Y | Y | N | Y | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fastscnn) |
| UNet[*](#static_shape) | Y | Y | Y | Y | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/unet) |
| ANN[*](#static_shape) | Y | Y | N | N | N | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/ann) |
| APCNet | Y | Y | Y | N | N | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/apcnet) |
| BiSeNetV1 | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/bisenetv1) |
| BiSeNetV2 | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/bisenetv2) |
| CGNet | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/cgnet) |
| DMNet | Y | N | N | N | N | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/dmnet) |
| DNLNet | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/dnlnet) |
| EMANet | Y | Y | N | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/emanet) |
| EncNet | Y | Y | N | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/encnet) |
| ERFNet | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/erfnet) |
| FastFCN | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fastfcn) |
| GCNet | Y | Y | N | N | N | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/gcnet) |
| ICNet[*](#static_shape) | Y | Y | N | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/icnet) |
| ISANet | Y | Y | N | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/isanet) |
| NonLocal Net | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/nonlocal_net) |
| OCRNet | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/ocrnet) |
| PointRend | Y | Y | N | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/point_rend) |
| Semantic FPN | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/sem_fpn) |
| STDC | Y | Y | Y | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/stdc) |
| UPerNet[*](#static_shape) | Y | Y | N | N | N | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/upernet) |
| DANet | Y | Y | N | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/danet) |
| Segmenter[*](#static_shape) | Y | Y | N | N | N | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/segmenter) |
| SegFormer[*](#static_shape) | Y | Y | N | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/segformer) |
| SETR | Y | N | N | N | Y | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/setr) |
| CCNet | N | N | N | N | N | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/ccnet) |
| PSANet | N | N | N | N | N | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/psanet) |
| DPT | N | N | N | N | N | [config](https://github.com/open-mmlab/mmsegmentation/tree/master/configs/dpt) |

### Reminder

Expand Down
Loading

0 comments on commit 5adbfc5

Please sign in to comment.