Semantic segmentation-aireal

[itaim18]

I have frames and images of drones.
I want to train a model for semantic segmentation of it including: roads, roundabouts, crosswalks, junctions etc.
What tools and methods should I use first this task.
Would hate to go over all frames one by one etc.
Thinking of SAM 2 as a helper but don’t see how exactly…
Any ideas?

[UltralyticsAssistant]

👋 Hello @itaim18, thank you for reaching out and sharing your use case for semantic segmentation with aerial drone images 🚀! This sounds like an exciting project, and we’re here to help guide you in the right direction.

We recommend first checking out the Ultralytics Docs for resources on the tasks you wish to accomplish. Semantic segmentation tasks may also benefit from reviewing the model training tips and popular segmentation techniques for optimal results.

If this is a ❓ general question about workflows or methods, could you please provide more details about your dataset or tools you've considered (like SAM)? This will help provide additional clarity for recommendations. If you're considering using SAM, you might want to look into its applications for data labeling and model initialization, as these can significantly reduce your manual efforts in preparing segmentation datasets.

If this is a 🐛 technical issue or bug report related to the tools or methods, we kindly ask you to provide a minimum reproducible example, along with error logs or screenshots, to help us identify and debug the issue.

---

Key Resources to Explore 🔍

• Python Usage Guide – Learn about integrating Python for task automation.
• CLI Usage Guide – Get started with command-line interface workflows.
• Custom Training Guide – Improve your model training with segmentation-specific tips.

Explore the Community 🎧

Join and engage with the vibrant Ultralytics community for support or tips:

• Discord for real-time discussions.
• Discourse for in-depth conversations.
• Subreddit for knowledge sharing and project inspiration.

---

Upgrade Ultralytics 🔄

Make sure you’re running the latest version of the ultralytics package for new features and bug fixes. Upgrade it in your environment with:

pip install -U ultralytics

Verify that you are using a Python>=3.8 environment with PyTorch>=1.8.

---

Verified Environments 🖥️

YOLO is tested and verified to work seamlessly in the following environments:

• Notebooks with a GPU:
• Cloud Platforms: GCP and AWS.
• Docker Quickstart Guide

---

CI Status ✅

If this badge is green, all Ultralytics CI tests are passing, ensuring the reliability of YOLO in supported environments.

---

🤖 This is an automated response, and an Ultralytics engineer will assist you further soon! 🎉

[itaim18]

This is a general question. Unfortunately, I don't believe the visdrone dataset would help much in achieving training my model.

[glenn-jocher]

@itaim18 thank you for your feedback. If the VisDrone dataset isn't suitable, I recommend exploring datasets like DeepGlobe or OpenStreetMap for aerial imagery segmentation tasks, or you can leverage SAM2 for efficient annotation of your custom dataset. Let us know if you need further assistance with this!

[itaim18]

Hey, @glenn-jocher , Let's say I was able to segment abt 150 images into roboflow, how should i train my model from here?
would you recommend one of their notebooks to do it? was looking at this: https://github.com/roboflow/notebooks/blob/main/notebooks/fine-tune-sam-2.1.ipynb but not sure if this is the right approach...

[glenn-jocher]

@itaim18 you can train your model using Roboflow's tools or export your dataset in YOLO format and train it directly with the Ultralytics framework. Both approaches are effective, but if you're focused on YOLO models, training within the Ultralytics ecosystem ensures alignment with the model's architecture and optimization.

[Y-T-G]

@itaim18 Ultralytics doesn't support semantic segmentation. It trains an instance segmentation model. And you need lots of data. That's how deep learning works.

[glenn-jocher]

@itaim18 to train a semantic segmentation model for aerial imagery, you can leverage Ultralytics tools for auto-annotation. Use the auto_annotate function in our framework to efficiently label your dataset by combining a YOLO detection model (e.g., YOLO11) with a segmentation model like SAM or SAM 2. This will save you from manually labeling each frame. Refer to the SAM 2 documentation for integration and usage details.

[itaim18]

Hey :)
I actually figured some stuff in this area, probably gonna leverage the project’s abilities in the few days and look more into it
Thank you very much!!

[glenn-jocher]

@itaim18 you're welcome! If you have any further questions while exploring the project, feel free to reach out. Best of luck with your work!