Annotating a dataset for YOLO using SAM

[nihegde18]

Hello ,
I am a researcher attempting to train a yolo-11 model to separate blood cells in images (object detection and classification). I have an 18,000 image dataset that wouldn't make sense to annotate manually - so i am attempting to use SAM-2 to generate masks and then classify the masks as RBC or WBC subtype based on color thresholding (since WBC's are purple) which would then be used to generate bounding boxes for the annotations. I was looking for more ideas to auto annotate a dataset like this - since i often run into issues with overlapping cells , granularities etc. any help would be appreciated - here's a sample image of my dataset.

[UltralyticsAssistant]

👋 Hello @nihegde18, thank you for sharing your research and interest in Ultralytics 🚀! It sounds like a fascinating project, and leveraging tools like SAM-2 for auto-annotation is a great approach to expedite dataset preparation for YOLO models.

We recommend visiting the Docs to explore training workflows, datasets, and annotation guidelines as you progress. Specifically, the Tips for Best Training Results guide may be particularly useful in achieving accurate results for complex datasets like yours.

If this is a ❓ question about dataset preparation or YOLO training, providing more details such as your current annotation pipeline, challenges you're facing with overlapping objects, and specific issues with granularities will help us assist you better. If possible, sharing additional dataset samples (e.g., via links or GitHub uploads) would also be valuable.

If you suspect issues with any aspect of ultralytics or YOLO, please ensure you're running the latest version of the ultralytics package. You can upgrade it using the following command:

pip install -U ultralytics

Additionally, here are a few pre-verified 😍 environments where you can train and test YOLO models seamlessly:

• Notebooks with free GPU:
• Docker Images: See the Docker Quickstart Guide

For ideas or inspiration from the community, you can join discussions on Discourse or check out threads on our Subreddit. If you'd like to chat in real time, consider joining us on Discord 🎧.

This is an automated response to guide you, and an Ultralytics engineer will be here to assist you further soon. Feel free to provide additional information or ask more questions in the meantime. Good luck with your dataset annotation journey! 🚀🔬

[glenn-jocher]

@nihegde18 thank you for sharing your use case. To auto-annotate your dataset, you can use the auto_annotate function from the Ultralytics framework, which integrates YOLO detection and SAM segmentation models. This may help address challenges like overlapping cells. For your scenario, applying SAM-2 with fine-tuned parameters and leveraging post-processing techniques like color segmentation or IoU filtering could improve annotation quality. For more details, refer to the auto_annotate documentation.