Skip to content

Repo for MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images

License

Notifications You must be signed in to change notification settings

batmanlab/MedSyn

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

75 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MedSyn

Official PyTorch implementation for paper MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images, accepted by IEEE Transactions on Medical Imaging.

This code is made by Yanwu Xu and Li Sun.

Table of Contents

  1. Environment Setup
  2. Pretrained Checkpoint
  3. Pre-processing Data
  4. Training
  5. Inference
  6. Additional Scripts
  7. Generated Samples
  8. Citation
  9. License and Copyright
  10. Contact

Environment Setup

Before running and doing inference based on our code, we highly recommend preparing at least two GPUs with 48G GPU memory each.

conda create -n medsyn python==3.9

In addition to this, you need to also install several packages by:

pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121
pip install monai==0.8.0
pip install accelerate
pip install einops
pip install einops_exts

Pretrained Checkpoint

Refer to the src folder

Our checkpoint for pre-trained language model is available here. Our checkpoint for model pre-trained on UPMC dataset is available here (Application required).

Pre-processing Data

Refer to the preprocess folder

Training

Refer to the src folder

This is a one-key running bash, which will run both low-res and high-res. But the training can be done independently

sh run_train.sh

Inference

Refer to the src folder

sh run_inference.sh

Additional Scripts

We give the inference for our text conditional generation in "prompt.ipynb" and the conditional generation with segmentation in "seg_conditional.ipynb"

Generated Samples

Low-Res High-Res

Comparisons

Generation Conditioned on Reports

Generation Conditioned on Segmentation Mask

Citation

@ARTICLE{medsyn2024,
  author={Xu, Yanwu and Sun, Li and Peng, Wei and Jia, Shuyue and Morrison, Katelyn and Perer, Adam and Zandifar, Afrooz and Visweswaran, Shyam and Eslami, Motahhare and Batmanghelich, Kayhan},
  journal={IEEE Transactions on Medical Imaging}, 
  title={MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images}, 
  year={2024},
  doi={10.1109/TMI.2024.3415032}}

License and Copyright

CC-BY-NC

Contact

Yanwu Xu [[email protected]], Li Sun [[email protected]], Kayhan Batmanghelich [[email protected]]

About

Repo for MedSyn: Text-guided Anatomy-aware Synthesis of High-Fidelity 3D CT Images

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published