ISDA-Pytorch

The Implicit Semantic Data Augmentation (ISDA) algorithm implemented in Pytorch.

• (NeurIPS 2019) Implicit Semantic Data Augmentation for Deep Networks

• (T-PAMI) Regularizing Deep Networks with Semantic Data Augmentation

Update on 2021/04/23: Release Code for Visualizing Deep Features on ImageNet!

Update on 2021/01/17: Journal Version of ISDA is Accepted by T-PAMI!

Update on 2020/04/25: Release Pre-trained Models on ImageNet.

Update on 2020/04/24: Release Code for Image Classification on ImageNet and Semantic Segmentation on Cityscapes.

Introduction

We propose a novel implicit semantic data augmentation (ISDA) approach to complement traditional augmentation techniques like flipping, translation or rotation. ISDA consistently improves the generalization performance of popular deep networks on supervised & semi-supervised image classification, semantic segmentation, object detection and instance segmentation.

Citation

If you find this work valuable or use our code in your own research, please consider citing us with the following bibtex:

@inproceedings{NIPS2019_9426,
        title = {Implicit Semantic Data Augmentation for Deep Networks},
       author = {Wang, Yulin and Pan, Xuran and Song, Shiji and Zhang, Hong and Huang, Gao and Wu, Cheng},
    booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
        pages = {12635--12644},
         year = {2019},
}

@article{wang2021regularizing,
        title = {Regularizing deep networks with semantic data augmentation},
       author = {Wang, Yulin and Huang, Gao and Song, Shiji and Pan, Xuran and Xia, Yitong and Wu, Cheng},
      journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
         year = {2021},
          doi = {10.1109/TPAMI.2021.3052951}
}

Get Started

Please go to the folder Image classification on CIFAR, Image classification on ImageNet, Semantic segmentation on Cityscapes and Visualizing deep features for specific docs.

Pre-trained Models on ImageNet

• Measured by Top-1 error.

Model	Params	Baseline	ISDA	Model
ResNet-50	25.6M	23.0	21.9	Tsinghua Cloud / Google Drive
ResNet-101	44.6M	21.7	20.8	Tsinghua Cloud / Google Drive
ResNet-152	60.3M	21.3	20.3	Tsinghua Cloud / Google Drive
DenseNet-BC-121	8.0M	23.7	23.2	Tsinghua Cloud / Google Drive
DenseNet-BC-265	33.3M	21.9	21.2	Tsinghua Cloud / Google Drive
ResNeXt50, 32x4d	25.0M	22.5	21.3	Tsinghua Cloud / Google Drive
ResNeXt101, 32x8d	88.8M	21.1	20.1	Tsinghua Cloud / Google Drive

Visualization of Augmented Samples

• ImageNet

Results

• Supervised image classification on ImageNet

• Complementing traditional data augmentation techniques

• Semi-supervised image classification on CIFAR & SVHN

• Semantic segmentation on Cityscapes

• Object detection on MS COCO

• Instance segmentation on MS COCO

Acknowledgment

Our code for semantic segmentation is mainly based on pytorch-segmentation-toolbox.

To Do

Update code for semi-supervised learning.