Attention-based Deep Multiple Instance Learning could be applied in a wide range of medical imaging applications. Supported by the project "Deep Learning for Survival Prediction"@UTA-SMILE, I wrote the Keras version of ICML 2018 paper "Attention-based Deep Multiple Instance Learning" (https://arxiv.org/pdf/1802.04712.pdf) in this repo to share the solution for Keras users.
The official Pytorch implementation can be found here. I built it with Keras using Tensorflow backend. I wrote attention layers described in the paper and did experiments in colon images with 10-fold cross validation. I got the very close average accuracy described in the paper and visualization results can be seen as below. Parts of codes are from https://github.com/yanyongluan/MINNs.
When train the model, we only use the image-level label (0 or 1 to see if it is a cancer image). The attention layer can provide an interpretation of the decision by presenting only a small subset of positive patches.
- Colon cancer dataset [Data]
- Processed patches [Google Drive]
I put my processed data here and you can also set up according to the paper. If you have any problem, please feel free to contact me.
Several applications can be found recently. I will summarize them in the following and the first one is our recent work.
- Jiawen Yao, Xinliang Zhu, et al. "Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks", Medical Image Analysis, 101789, 2020.[PDF] [Code]
Other important work used multiple-instance learning in medical imaging include (list will be updated frequently)
| Year | Author list | Title | Conference/Journal |
|---|---|---|---|
| 2021 | Ming Y. Lu, Tiffany Y. Chen, Drew F. K. Williamson, Melissa Zhao, Maha Shady, Jana Lipkova & Faisal Mahmood | AI-based pathology predicts origins for cancers of unknown primary. [Pytorch] | Nature, arxiv |
| 2021 | Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Richard J. Chen, Matteo Barbieri & Faisal Mahmood | Data-efficient and weakly supervised computational pathology on whole-slide images. [Pytorch] | Nature Biomedical Engineering, arxiv |
| 2021 | Jianan Chen, Helen M. C. Cheung, Laurent Milot and Anne L. Martel | AMINN: Autoencoder-based Multiple Instance Neural Network Improves Outcome Prediction of Multifocal Liver Metastases. [Keras] | MICCAI 2021 arxiv |
| 2020 | Ole-Johan Skrede et al. | Deep learning for prediction of colorectal cancer outcome: a discovery and validation study | Lancet |
| 2019 | Shujun Wang, Yaxi Zhu, et al. | RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification [Keras] | Medical Image Analysis arxiv |
If you have any questions about this code, I am happy to answer your issues or emails (to [email protected]).
I plan to review recent work using Deep MIL techniques in medical imaging and Your suggestions are very welcome !
The work conducted by Jiawen Yao was funded by Grants from the UTA-SMILE Lab.