MouseGAN++

Unsupervised Disentanglement and Contrastive Representation for Multiple MRI Modalities Synthesis and Structural Segmentation of Mouse Brain

---

Paper | Usage demo | Replicate demo | MRI data release | Pretrained weight | Documentation | Contents | See also: BEN

See also: | Github project link

"BEN: a generalizable Brain Extraction Net for multimodal MRI data from rodents, nonhuman primates, and humans." bioRxiv (2022).

---

Usage Demo

Name	Modality	Colab link
MouseGAN++	T1w, T2w, T2*w, QSM, Mag
MouseGAN++	T1w, T2w

Replicate Demo and More Results

Name	Description	Details
MRM NeAt segmentation	Segmentation results & pretrained weights.	link
Ablation study loss curve	Loss curve for ablation study from Tensorboard.	link
Pretrained weight	Pretrained weight for MouseGAN++.	link
More examples	More results of MouseGAN++ and SOTA methods, including failure cases.	link
Rater study	Qualitatively results of synthesized images by medical experts.	link

---

Quick Start Contents

Visit our documentation for installation, tutorials and more.

• Installation
• Quick Start / Tutorial
  • Run translation alone
  • Run segmentation alone
  • Choice of pretext task
  • Plan list
• Resources
  • Dataset release
  • Pretained weight

Installation

Requirements:

• torch == 1.3
• numpy == 1.19
• SimpleITK == 2.0
• opencv-python == 4.2

Install dependencies:

git clone https://github.com/yu02019/MouseGAN-pp.git
cd MouseGAN-pp
conda env create -f environment.yml

Run translation

For multi-modality dataset:

python translation.py --dataroot DATAROOT --name NAME --num_domains NUM_DOMAINS --out_dir OUT_DIR --resume MODEL_DIR --num NUM_PER_IMG

Run segmentation

For example, run on MRM NeAt dataset:

Note: The previous version of the code required modifying parameters directly within the .py file and processed .npy format files. We are currently updating the code to accept command line arguments and to handle nii/nii.gz format files. This update is still in progress. Please stay tuned for updates or modify the source code directly.

In the meantime, you can refer to our upcoming project for improved functionalities (Github repo: Todo)

# This functionality is currently under development.
# Please stay tuned for updates or modify the source code directly.
python segmentation.py -i input_folder -o output_folder -w model_weight 

Choice of pretext task

In our paper, we used modality translation as our pretext task, as we wanted to impute missing modality. However, if readers are faced with multi-center single modality data (e.g, T2w MR images from 11.7T and 9.4T scanners), our pretext task could change to center-style translation easily.

Plan list

• Update interfaces (Before October 12th)
• Update Documentation (Before October 18th)
• Update Colab demo (Before October 13th)
• Update Tutorials (Before October 16th) (see details in demo)
• Rewrite dataloader
• Consolidate BEN and MouesGAN++ as one fully end-to-end pipeline for the mouse brain.

Resources

Dataset release / Pretrained weight / Contributing to MouseGAN++

We will release our multi-modality MR mouse dataset images for more extensive communities for both neuroscience and deep learning.

The details can be found in this folder.

---

Citation

If you find our work / datasets useful for your research, please consider citing:

@ARTICLE{9966838,
  author={Yu, Ziqi and Han, Xiaoyang and Zhang, Shengjie and Feng, Jianfeng and Peng, Tingying and Zhang, Xiao-Yong},
  journal={IEEE Transactions on Medical Imaging}, 
  title={MouseGAN++: Unsupervised Disentanglement and Contrastive Representation for Multiple MRI Modalities Synthesis and Structural Segmentation of Mouse Brain}, 
  year={2022},
  volume={},
  number={},
  pages={1-1},
  doi={10.1109/TMI.2022.3225528}}

Disclaimer: This toolkit is only for research purpose. If used on an additional dataset, the model might need to be fine-tuned before running (refer to Limitation).