MedDr: Diagnosis-Guided Bootstrapping for Large-Scale Medical Vision-Language Learning

A generalist foundation model for healthcare capable of handling diverse medical data modalities.

[Project Page] [🤗Model]

MedDr: Diagnosis-Guided Bootstrapping for Large-Scale Medical Vision-Language Learning [Paper]
Sunan He*, Yuxiang Nie*, Zhixuan Chen, Zhiyuan Cai, Hongmei Wang, Shu Yang, Hao Chen** (*Equal Contribution, **Corresponding author)

Release

• [04/23] 🔥 We released MedDr: Diagnosis-Guided Bootstrapping for Large-Scale Medical Vision-Language Learning. We developed MedDr, a generalist foundation model for healthcare capable of handling diverse medical data modalities, including radiology, pathology, dermatology, retinography, and endoscopy. Check out the paper.

Schedule

• Release the demo code.
• Release the evaluation code.
• Release the training code.
• Release the data generation code.

Environment

We build our model based on InternVL. Please refer to the INSTALLATION.md to prepare the environment.

Demo

Download the checkpoint and change the model_path in demo.py.

python3 demo.py

Acknowledgement

• InternVL: Thanks for their efforts in the open-source community. InternVL is a highly valuable work that contributes significantly to the VLM domain.