Skip to content

Latest commit

 

History

History
69 lines (46 loc) · 2.51 KB

README.md

File metadata and controls

69 lines (46 loc) · 2.51 KB

CVPR2023 Highlight | MAESTER: Masked Autoencoder Guided Segmentation at Pixel Resolution for Accurate, Self-Supervised Subcellular Structure Recognition

Check out the Paper! and our Youtube Talk!

💥 Introduction

We introduce MAESTER (Masked AutoEncoder guided SegmenTation at pixEl Resolution), a self-supervised method for accurate, subcellular structure segmentation at pixel resolution. MAESTER treats volume electron microscopy(vEM) image segmentation as a representation learning and clustering problem. Specifically, MAESTER learns semantically meaningful token representations of multi-pixel image patches while simultaneously maintaining a sufficiently large field of view for contextual learning. We also develop a cover-and-stride inference strategy to achieve pixel-level subcellular strueture segmentation.

⚙️ Installation

  • Clone the repository:
git clone https://github.com/bowang-lab/MAESTER
  • Set up the environment:
poetry install
poetry shell
pip install torch==2.0.1 torchvision==0.15.2
  • Download the trained model for demo

    • Google drive: https://drive.google.com/drive/folders/143W_VSl5ONE3NGbnI0i19S8lBRml7lRz?usp=sharing
    • Put it under ./MAESTER/model_weights/
  • Dataset:

    wget https://cloud.mpi-cbg.de/index.php/s/UJopHTRuh6f4wR8/download
    
    • and unzip the dataset, put it under ./MAESTER/data/

🎉 Example

  • Check out our detailed demo:
    • Inference with MAESTER ./examples/inference_demo.ipynb.

📝 To-do

  • Add inference demo
  • Add scalable inference example
  • Add DDP training example

📄 Citation

@InProceedings{Xie_2023_CVPR,
    author    = {Xie, Ronald and Pang, Kuan and Bader, Gary D. and Wang, Bo},
    title     = {MAESTER: Masked Autoencoder Guided Segmentation at Pixel Resolution for Accurate, Self-Supervised Subcellular Structure Recognition},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2023},
    pages     = {3292-3301}
}

Acknowledgement

  • This repository is built upon MAE.