Ultralytics
tasks/segment/ #8734
giscus[bot]
bot
Replies: 85 comments 210 replies
-
|
I have a question about segmentation dataset and really wish you can answer, my dataset contains lots of information including bbox and segmentations, however it has wrong category_id and all categories have been marked as the same category, does this has bad influence on finetuing? If so, how can I fix that? |
Beta Was this translation helpful? Give feedback.
-
|
@jrryzh hey there! 👋 Absolutely, having incorrect To fix this, you'll need to correct the For instance, if you're working with a JSON format similar to COCO, you could use Python to update the category IDs: import json
# Load your dataset
with open('your_dataset.json') as f:
data = json.load(f)
# Assuming all categories are incorrectly marked as category_id 1 and you want to update them to correct IDs
correct_mapping = {1: 'new_correct_id'} # Update this with your correct category IDs
# Update category IDs
for ann in data['annotations']:
if ann['category_id'] in correct_mapping:
ann['category_id'] = correct_mapping[ann['category_id']]
# Save the corrected dataset
with open('your_corrected_dataset.json', 'w') as f:
json.dump(data, f)Remember to replace If you're working with segmentation, our Instance Segmentation guide might be helpful. It covers how to use our segmentation models, like Hope this helps! Let us know if you have any more questions. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
How can I run my code on GPU ? From the processor the recognition is freezing |
Beta Was this translation helpful? Give feedback.
-
|
@Boaruzhanchik hey there! 😊 To run your code on a GPU, you typically just need to ensure your model is set to use the GPU. With Ultralytics YOLO, your model should automatically utilize a CUDA-compatible GPU if available. Here's a quick check to make sure: from ultralytics import YOLO
# Make sure 'device' is set to 'cuda' if you want to use GPU (default if CUDA is available)
model = YOLO('yolov8n.pt', device='cuda')This small addition directs the model to use the GPU. Make sure you have a CUDA-compatible Nvidia GPU and have installed the necessary CUDA and CuDNN libraries. If recognition is freezing, it's usually due to the heavy computation not being efficiently processed by the CPU. Running it on the GPU should significantly speed things up! 🚀 If CUDA is correctly set up and you're still encountering issues, make sure to check your GPU drivers and CUDA installation. Keep me posted on how it goes or if you run into any further questions! |
Beta Was this translation helpful? Give feedback.
-
|
Hi, i'm install pythorch and have problem when program detect object Traceback (most recent call last): CPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\cpu\nms_kernel.cpp:112 [kernel] |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for the inconvenience, everything was solved, thank you ! |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 Glad to hear everything got sorted out. If you ever run into another hiccup or have any questions, please don't hesitate to reach out. Wishing you continued success with your projects! If instance segmentation or anything else piques your interest, our Ultralytics Docs are there to help guide you along. Cheers! 🎉 |
Beta Was this translation helpful? Give feedback.
-
|
I am building a streamlit app on YOLOv8 models, |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 For displaying the result or the annotated image/video in your Streamlit app using YOLOv8 models, you can follow this approach: After running the prediction with a YOLOv8 model, you get a from ultralytics import YOLO
import streamlit as st
from PIL import Image
import numpy as np
# Load a YOLO model
model = YOLO('yolov8n.pt') # Or your custom model
# Run prediction
results = model('path/to/image.jpg') # Replace with your source
# Convert results image to PIL Image for Streamlit
annotated_img = Image.fromarray(results[0].plot()[..., ::-1]) # Convert BGR to RGB
# Display the image in Streamlit
st.image(annotated_img, caption='Annotated Image')Replace Hope this helps you integrate YOLOv8 with your Streamlit app smoothly! 🚀 Let me know if you have any more questions. |
Beta Was this translation helpful? Give feedback.
-
|
has the ability to use image masks instead of contour points as labels ever been implemented ? |
Beta Was this translation helpful? Give feedback.
-
|
Hello! 👋 Yes, Ultralytics YOLO models have the ability to perform instance segmentation, which can utilize image masks to segment individual objects in an image. This is particularly useful for datasets where contour points might not adequately represent the object due to internal gaps or complex shapes. For instance segmentation, you can use models with the Here's a quick example on how to predict with a pretrained segmentation model: from ultralytics import YOLO
# Load a segmentation model
model = YOLO('yolov8n-seg.pt')
# Predict with the model
results = model('https://ultralytics.com/images/bus.jpg')This will give you both the bounding boxes and segmentation masks for detected objects, handling cases where simple contours are not sufficient. 🎭 Let us know if you need further assistance or examples! |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! I am using the pre-trained segmentation model by Yolov8. I have an image that contains several objects, and I only want to receive the mask of any person present in the frame, while ignoring all the other masks. My requirements:
|
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 For your requirement to only get the person mask from an image using the YOLOv8 pre-trained segmentation model, and ignoring other detections, you can do something like this: from ultralytics import YOLO
# Load the YOLOv8 segmentation model
model = YOLO('yolov8n-seg.pt')
# Perform prediction
results = model('path/to/your/image.jpg')
# Filter for 'person' class masks only (class_id for 'person' is usually 0)
person_masks = results.masks[results.boxes.cls == 0]
# Loop through each person mask to process/visualize as needed
for mask in person_masks:
# Here, 'mask' represents individual person masks
# You can proceed to draw contours or other operations as required
# Save or visualize person_masks as needed, maintaining original image dimensionsThis snippet is a starting point. The By utilizing this approach, you'll be able to:
Hope this helps! Let me know if you need further assistance. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
results.masks gives an attribute error. It was fixed with results[0].masks. I have the tensor containing the mask data of the person through person_masks.data. Just help me save the masked image now, that only contains mask of the detected person. (There is always going to be one person in my usecase) |
Beta Was this translation helpful? Give feedback.
-
|
Hey! 👋 Great to hear you've successfully isolated the person mask using YOLOv8 segmentation! To save the mask image with the same spatial dimensions as the original, here is a quick way: import cv2
import torch
# Assuming `person_masks.data` is your mask tensor
# Convert mask to a binary image (0 or 255), change the shape to match the original image height and width
mask_image = person_masks.data.squeeze().cpu().numpy() * 255 # Adjust the multiplication factor as necessary
mask_image = mask_image.astype(np.uint8)
# Save the mask image
cv2.imwrite('path/to/save/person_mask.jpg', mask_image)This code snippet takes your Let me know if this helps or if you have any further questions! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
I have a question, how do you visualize the segmentation mask with the detected label? it seems it's not discussed in the documentation, it just shows here how to predict using it but not visualizing it with an image. I even checked the list of results but it seems its just returning segmentation masks, not the classes that it predicts, with that said my question is can you visualize the segmentation mask with a label only because of what i see in the tutorial it has a bounding box too which i think that's how you put the label there because i think the segmentation masks are not directly related in the class ID, my goal is to detect object with segmentation and put label to it without bounding box. looking forward to your answer thank you |
Beta Was this translation helpful? Give feedback.
-
|
Hello there! 👋 To visualize the segmentation mask with its corresponding label (without bounding boxes), you can use the from ultralytics import YOLO
import matplotlib.pyplot as plt
# Load a pretrained segmentation model
model = YOLO('yolov8n-seg.pt')
# Predict on an image and return a Results object
results = model('https://ultralytics.com/images/bus.jpg')
# Visualize segmentation with labels (without bounding boxes)
fig = plt.figure(figsize=(10, 10))
plt.imshow(results.render()[0]) # `render()` method overlays masks & class labels onto the image
plt.axis('off')
plt.show()This generates an output where each segmentation mask is color-coded, and the class labels are overlaid directly on the masks. The Unfortunately, our default method integrates both segmentation masks and bounding boxes for comprehensive detection visualization. If you specifically want only the masks with labels and without the bounding boxes in your plots, the code tweak above should help. Hope this answers your question! 😊 If you have any further queries or need more insights, feel free to reach out. |
Beta Was this translation helpful? Give feedback.
-
|
I have a question. Can anyone suggest me how can I save the predicted bounding box and segmented mask in the format of : |
Beta Was this translation helpful? Give feedback.
-
|
@utokyo-sm96 hey there! 😊 To save the predicted bounding box and segmentation mask in your specified format, you can iterate over the detected objects and their corresponding masks to build the desired string for each detection. Below is a Python code snippet using the Ultralytics YOLO model to demonstrate how you could achieve this: from ultralytics import YOLO
# Load your YOLOv8 segment model
model = YOLO('yolov8n-seg.pt')
# Run a prediction
results = model('path/to/your/image.jpg')
# Iterate through results to format and save your detections and masks
for i, det in enumerate(results[0].boxes.xywh):
class_id = int(det[4])
bbox_center_x, bbox_center_y, bbox_width, bbox_height = det[:4]
segment = results[0].masks.xyn[i] # Get mask points for this detection
# Build the desired format string
output_str = f"{class_id} {bbox_center_x} {bbox_center_y} {bbox_width} {bbox_height}"
for x, y in zip(segment[0::2], segment[1::2]): # Iterate over points in mask
output_str += f" {x} {y}"
print(output_str) # Or write to a fileThis code will print out the details for each detection in your specified format. Note that Hope this helps! Let me know if you need further assistance. |
Beta Was this translation helpful? Give feedback.
-
|
When I try to predict some images using the best weights of this trained model (!python3 segment/predict.py --save-crop --save-txt) I expect to get predicted segment coordinates in the saved txt files as: |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 It looks like you're encountering an issue where you're expecting segmentation polygon coordinates in the For including segmentation mask coordinates in your output, you'd need to programmatically access the segmentation masks from the model's output and convert them into your desired format. Here's a quick example on how you might approach it: from ultralytics import YOLO
import numpy as np
model = YOLO('yolov8n-seg.pt') # Load your segment model
results = model('path/to/image.jpg') # Perform inference
for i, (box, mask) in enumerate(zip(results[0].boxes, results[0].masks)):
class_id = box[4].int()
bbox = box[:4] # bbox coordinates (center_x, center_y, width, height)
polygon = mask.to_polygons()[0] # Assuming one polygon per mask for simplification
# Convert polygon to your desired format here
# Example: polygon might need flattening and conversion to (x, y) pairs
flat_polygon = np.array(polygon).flatten()
output_line = f"{class_id} {bbox[0]} {bbox[1]} {bbox[2]} {bbox[3]} " + ' '.join(map(str, flat_polygon))
# Now write output_line to your .txt fileThis is just a starter; you might need to adjust the mask processing part depending on your exact needs. Remember, the actual implementation details may vary as this example assumes direct access to methods for converting masks to polygons which might not exist as is and you may need to implement or adapt them based on the format of the masks. Don't hesitate to reach out again if you need more help! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
` ` if len(det): ` Masks Shape: torch.Size([3, 640, 640]) can you help me out where am i wrong. |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 It looks like your script is running into an issue where it's not properly identifying the segmentation masks, as indicated by "Number of non-zero values in class mask: 0" and "Found 0 polygons." This typically happens if the predicted masks are not properly generated or if there's an issue in the conversion process from the mask tensor to the segmentation polygons. A couple of things you might want to check:
If the masks are indeed being generated but not captured correctly, you might need to adjust the process of converting masks to polygons. Remember, masks are typically provided as binary arrays (or tensors), where you need to find contours and then approximate those contours to polygons. Here's a simplified approach to debugging and trying to fix your issue:
Also, for your specific case of wanting to write the segmentation polygons alongside bounding box details, ensure your Lastly, remember that each segmentation mask corresponds to one detected object, so ensure your processing loop correctly matches each mask with its corresponding detection. If after checking these aspects you still face issues, consider sharing a minimal but complete snippet of how you're processing the image through the model and attempting to save the results. It might offer more clues on where things might be going astray. Hope this helps you to debug the situation! Keep experimenting, and don't hesitate to reach out for more guidance. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
I have an important question, i want going to train yolov8 model for instance segmentation but i need to save all the segmented mask on my disk after training the model. how i can get that |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! Saving segmented masks to disk with Ultralytics YOLO is doable, and arranging them by class labels adds a nice level of organization for further training or analysis. 🧐 Here's a quick guide to help you out:
from ultralytics import YOLO
# Load your trained segmentation model
model = YOLO('path/to/yolov8n-seg.pt')
# Predict on an image, replace with your image path or URL
results = model('path/to/image.jpg')
# Save results, this includes saving masks
results.save()
Here's a very basic starter: import os
import numpy as np
# Assuming 'results' from the prediction step
for det in results.xyxy[0]: # Iterate through detections
class_id = int(det[-2]) # Class ID for detected object
class_name = results.names[class_id] # Class name
mask = results.masks[class_id].numpy() # Convert mask to numpy array
# Create directory for the class if it doesn't exist
os.makedirs(class_name, exist_ok=True)
# Filename for the mask
mask_path = os.path.join(class_name, f"{det[5]}.png") # det[5] is the unique ID, you might adjust this
# Save mask, you may need to adjust the saving depending on mask format
np.save(mask_path, mask)Note: You'll need to adjust the looping and saving part based on your actual needs and mask data structure. Remember, getting into the specifics might require tweaking according to your dataset, and the exact implementation might vary. If you're dealing with a custom dataset or more complex requirements, you might need to dive deeper into handling the output data. Happy coding! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Query: |
Beta Was this translation helpful? Give feedback.
-
|
@utokyo-sm96 hello! For saving instance segmentation predictions, including masks, Here's a quick example on how you might go about this for saving segmentation masks as images, but you can adapt the logic to save in your preferred format: from ultralytics import YOLO
import numpy as np
from PIL import Image
# Load a model
model = YOLO('yolov8n-seg.pt') # Assuming you're using a segmentation model
# Predict with the model
results = model('path/to/image.jpg')
# Save masks
for i, mask in enumerate(results[0].masks):
img = Image.fromarray((mask.numpy() * 255).astype(np.uint8))
img.save(f'mask_{i}.png')In this example, each object's mask is saved as a separate Let us know if you need further assistance! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
hi i am trying to get each mask in a separate photo and i used this code why an i getting this error |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! It looks like you're encountering an error because you're trying to directly operate on the You're on the right track with wanting to convert each mask to an image format! However, you need to access the numpy representation of each mask by iterating over the masks within the from ultralytics import YOLO
from PIL import Image
# Assuming you're using a YOLOv8 segmentation model
model = YOLO('yolov8n-seg.pt')
results = model('path/to/your/image.jpg')
# Iterate over each detected object's mask
for i, mask in enumerate(results[0].masks.numpy()):
# Convert mask to an image
mask_img = Image.fromarray((mask * 255).astype(np.uint8))
# Save each mask as an image file
mask_img.save(f'mask_{i}.png')In this adjusted approach, Hope this helps! Let me know if you have further questions. 😄 |
Beta Was this translation helpful? Give feedback.
-
|
Hello, |
Beta Was this translation helpful? Give feedback.
-
|
Hello 😊! To get the mIoU (mean Intersection over Union) for each epoch end during training/validation with our segmentation models (e.g., from ultralytics import YOLO
# Load your segmentation model
model = YOLO('yolov8n-seg.pt') # using an official segmentation model for example
# Perform validation (after training or directly if the model is pre-trained)
metrics = model.val() # Assumption: model and dataset are correctly configured
# Access mIoU (mean IoU for segmentation)
miou = metrics.seg.map # This is your mIoU value
print(f'mIoU: {miou}')This will give you the mIoU for segmentation masks at the end of each validation phase, which is typically run after each training epoch. Hope this helps! If you have further questions or need more assistance, feel free to ask. 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for sharing, but I don't know how to add MIOU evaluation metrics in v8, if you have time, can you give a detailed guide? Thank you very much! |
Beta Was this translation helpful? Give feedback.
-
|
@handsome2456 hello! 😊 To add mIoU (mean Intersection over Union) evaluation metrics in YOLOv8, you can access these metrics directly after each validation phase. Here's a detailed guide to help you get started: Step-by-Step Guide
Example CodeHere is a complete example that includes training and validation with mIoU metrics: from ultralytics import YOLO
# Load a model
model = YOLO("yolov8n-seg.pt")
# Train the model and validate at the end of each epoch
results = model.train(data="coco8-seg.yaml", epochs=100, imgsz=640)
# Validate the model
metrics = model.val()
miou = metrics.seg.map
print(f'mIoU: {miou}')For more detailed information on instance segmentation and other tasks, you can visit the Ultralytics documentation on segmentation. Feel free to reach out if you have any more questions or need further assistance. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! I'd like to share with you a project I've recently worked on. Together with a colleague, we've created a repository that serves as a tool with a SAHI-like inference but specifically tailored for instance segmentation tasks. Our repository allows for segmenting small objects in images by combining mask predictions from various overlapping patches. We support both YOLOv8-seg and FastSAM. Additionally, we have a variant for object detection tasks, and the key distinction from SAHI is the support for all the current models from the Ultralytics team: YOLOv9, YOLOv8, RTDTR, and others. I'm a huge fan of Ultralytics, so I'd be thrilled to assist you if you're interested in our project. I'm confident that for many people, the task of finding a large number of segments would be beneficial, especially when using standard Ultralytics models. Here's the link to the project: YOLO-Patch-Based-Inference. Honestly, I wasn't sure where to share information about the project, so I decided to start with the YOLO-seg support section, as it seemed like the most obvious choice. I would greatly appreciate your feedback on the potential usefulness of our project for you. Thank you! Example: |
Beta Was this translation helpful? Give feedback.
-
|
Hello there! Wow, your project sounds absolutely fascinating! 🚀 The idea of tailoring SAHI-like inference for instance segmentation, and supporting an impressive range of Ultralytics models, including YOLOv8-seg and FastSAM, is a commendable effort. It's particularly exciting that you're extending support to the latest models like YOLOv9, RTDTR, and so forth. Your repository is a brilliant testament to the passion and expertise in the community. I'm thrilled to see your innovative use of our models, and I'm confident that your project could significantly benefit those needing refined segmentation, especially for small objects. Your approach to combining mask predictions from overlapping patches is a smart solution to a common challenge in instance segmentation. I've taken a look at your YOLO-Patch-Based-Inference repo, and it’s clearly well-thought-out. The instance segmentation community will surely find tremendous value in the tools you've developed. Sharing it in the YOLO-seg support section was a great choice! If there's anything more we can do to support your project or if you have further questions or insights, feel free to reach out. Feedback like yours is invaluable—it helps us continue to innovate and support projects that enhance the capabilities of Ultralytics models. Once again, thank you for your enthusiasm for Ultralytics and for sharing your project with us. Keep up the fantastic work! 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for your feedback. |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Thank you so much for reaching out and offering to collaborate on documenting your project for Ultralytics documentation. Your initiative aligns perfectly with our goal of enriching the community with valuable resources and tools. 😊 Given the potential impact and the innovative approach of your SAHI-like tool for instance segmentation, featuring support for Ultralytics models including YOLOv8-seg, FastSAM, and more, it sounds like something that would greatly interest and benefit our users. Documenting your solution in a manner similar to our SAHI guide could indeed foster wider adoption and assist others in their instance segmentation endeavors. Let's connect to discuss how we can work together on creating this documentation. Please feel free to reach out to us via our repository's "Discussions" section or directly via email with a brief outline or proposal for the documentation content. This will help us kickstart the process and determine how best to feature your work within our documentation ecosystem. Again, thank you for your enthusiasm and looking forward to collaborating with you! 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much. I wrote a proposal in the "Discussions" section - https://github.com/orgs/ultralytics/discussions/9381 |
Beta Was this translation helpful? Give feedback.
-
|
@Koldim2001 that's fantastic! 😊 Your proposal in the "Discussions" section here is a great step forward, and I appreciate your willingness to contribute and help enhance the Ultralytics documentation. Your project's unique approach to instance segmentation, especially for small objects and full support for our model range, sounds like a valuable resource for our users. Collaborating on similar documentation to what we've done with SAHI would not only benefit our community but also raise awareness of your innovative solution. Let's definitely explore this further. Please expect a follow-up in the "Discussions" section as we discuss next steps and how we can integrate your project into our ecosystem effectively. Thank you for reaching out and looking forward to working together! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
YOLO V8 has pre-trained models that are performed on the COCO dataset. However, I am working on a project for image segmentation where the classes are not available in the COCO dataset. In this case, do I need to train a model from scratch? About the dataset: Looking for an answer. Thankyou! |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 It's great that you've already annotated your custom dataset in YOLOv8 format, especially for such an impactful project involving property damage assessment from satellite and aerial imagery. Given your custom classes are outside the COCO dataset, you're on the right track thinking about training a model specifically for your use case. The good news is, you don't need to start completely from scratch! You can leverage one of our pre-trained segmentation models like Here's a quick example of how you might kick off your training process: from ultralytics import YOLO
# Initialize your model for training
model = YOLO('yolov8n-seg.pt') # Load a pre-trained segmentation model
# Fine-tune the model on your custom dataset
results = model.train(data='your_custom_dataset.yaml', epochs=100, imgsz=640)Ensure your dataset YAML file is correctly set up with paths to your training and validation data, classes, etc. This method will help your model learn the specific classes and damage levels you're interested in! Best of luck with your project, and don't hesitate to reach out if you have more questions. Happy detecting! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Thanks a lot again. |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 Since your custom dataset annotations are ready and in the proper YOLOv8 format, you're all set to start fine-tuning one of our pre-trained segmentation models, such as from ultralytics import YOLO
# Load the pre-trained model
model = YOLO('yolov8n-seg.pt')
# Fine-tune the model on your custom dataset
results = model.train(data='your_custom_dataset.yaml', epochs=100, imgsz=640)Make sure your |
Beta Was this translation helpful? Give feedback.
-
|
Is Panoptic Segmentation supported by Yolov8 or in any version of Yolo? |
Beta Was this translation helpful? Give feedback.
-
|
Hello! At the moment, YOLOv8 specifically supports Instance Segmentation with models like For panoptic segmentation, which combines both semantic segmentation (segmenting all pixels in an image) and instance segmentation (distinguishing between individual object instances), we're always exploring new features and capabilities to incorporate into our models. Your interest is definitely noted! 🚀 If you're specifically looking to work with panoptic segmentation, I'd encourage staying tuned to our updates. Also, feel free to contribute or suggest improvements; we welcome collaboration from the community! 🤝 Here's a quick example of how you can work with our segmentation models: from ultralytics import YOLO
# Load a pretrained segment model
model = YOLO('yolov8n-seg.pt')
# Segment objects in an image
results = model('https://ultralytics.com/images/bus.jpg')
# Visualize results
results.show()Let us know if there's anything else we can help you with! |
Beta Was this translation helpful? Give feedback.
-
|
I have trained custom yolov8 model for Instance segmentation on video. Now I have to make video inference for the same please guide me. And I have to deploy it in mobile application which was developed in flutter give me guidance on these for Yolov8 .tflite format model for instance segmentation. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @mayurkatre18, Thank you for reaching out! It's fantastic to hear that you've successfully trained a custom YOLOv8 model for instance segmentation on video. Let's guide you through the process of making video inferences and deploying your model in a Flutter mobile application. Video InferenceTo perform video inference with your trained YOLOv8 model, you can use the following Python code: from ultralytics import YOLO
# Load your trained model
model = YOLO('path/to/your/custom-model.pt')
# Predict on a video
results = model('path/to/your/video/file.mp4')
# Iterate over results if needed
for result in results:
# Process your results here
result.show() # Display the frame with predictionsThis code will load your custom model and run predictions on each frame of the video. If you need to process real-time video streams, you can read frames from the stream and pass them to the Exporting to TensorFlow Lite (.tflite)To deploy your model in a Flutter mobile application, you need to export it to TensorFlow Lite format. Here’s how you can do it: from ultralytics import YOLO
# Load your trained model
model = YOLO('path/to/your/custom-model.pt')
# Export the model to TensorFlow Lite format
model.export(format='tflite')This command will create a Integrating with FlutterOnce you have the
dependencies:
tflite: ^1.0.3
import 'package:flutter/material.dart';
import 'package:tflite/tflite.dart';
void main() => runApp(MyApp());
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
home: Scaffold(
appBar: AppBar(
title: Text('YOLOv8 Instance Segmentation'),
),
body: Center(
child: Text('Running YOLOv8 Model'),
),
),
);
}
}
class MyHomePage extends StatefulWidget {
@override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
@override
void initState() {
super.initState();
loadModel();
}
Future<void> loadModel() async {
String res = await Tflite.loadModel(
model: "assets/yolov8n-seg.tflite",
labels: "assets/labels.txt",
);
print(res);
}
Future<void> runInference() async {
var recognitions = await Tflite.detectObjectOnImage(
path: "path/to/your/image.jpg",
model: "YOLO",
threshold: 0.5,
);
print(recognitions);
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text('YOLOv8 Instance Segmentation'),
),
body: Center(
child: Text('Running YOLOv8 Model'),
),
);
}
}This example demonstrates how to load the TensorFlow Lite model and run inference on an image. You can extend this to process video frames as needed. For more detailed instructions on exporting models and integrating them into mobile applications, please refer to the Ultralytics documentation. If you encounter any issues or need further assistance, feel free to ask. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hi, |
Beta Was this translation helpful? Give feedback.
-
|
Hi @mayurkatre18, Thank you for reaching out! 😊 To perform video inference using a Step 1: Install TensorFlow LiteFirst, ensure you have TensorFlow Lite installed: pip install tflite-runtimeStep 2: Load and Run Inference on Video FramesHere's a Python script to load your import cv2
import numpy as np
import tflite_runtime.interpreter as tflite
# Load the TFLite model and allocate tensors
interpreter = tflite.Interpreter(model_path="path/to/your/model.tflite")
interpreter.allocate_tensors()
# Get input and output tensors
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Open the video file
cap = cv2.VideoCapture("path/to/your/video.mp4")
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
# Preprocess the frame
input_data = cv2.resize(frame, (input_details[0]['shape'][2], input_details[0]['shape'][1]))
input_data = np.expand_dims(input_data, axis=0)
input_data = input_data.astype(np.float32) / 255.0
# Set the tensor to point to the input data to be inferred
interpreter.set_tensor(input_details[0]['index'], input_data)
# Run the inference
interpreter.invoke()
# Get the results
output_data = interpreter.get_tensor(output_details[0]['index'])
# Process the results (e.g., draw bounding boxes and masks on the frame)
# This part will depend on your specific model's output format
# Display the frame with predictions
cv2.imshow('Frame', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()Step 3: Deploying in FlutterTo deploy your
For more detailed instructions, you can refer to the TensorFlow Lite Flutter Plugin documentation. If you encounter any issues or need further assistance, feel free to ask. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
I have trained YOLOv8 segmentation model in .tflite format. Now I have to integrate it into mobile application with flutter langauge. For this I have to make all configurations in laptop only to make integration of video and providing result and after this we have to make an API to have integration of this video processing in mobile app. Please provide the neccesary steps. |
Beta Was this translation helpful? Give feedback.
-
|
To integrate your YOLOv8 |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I have a question. I have trained a custom model based on a pre trained one to segment the shirts from cyclists. The problem however is that it recognizes 2 slightly different shirt on 1 image. My idea was to only use the one with the highest confidence score, but this object appears to be none (results.probs or results.keypoint are both none objects). Is it possible to get the confidence score or do I have to apply a different methode? |
Beta Was this translation helpful? Give feedback.
-
|
@pietervhij hey there! 👋 With only 100 training images, there's indeed a risk of overfitting when training over a large number of epochs like 100, especially for complex tasks like instance segmentation. Here are a couple of tips to mitigate that:
Training on a smaller Keep innovating! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the response, i have a little followup question. You say Ultralytics supports augmentations directly in the training pipeline. What do you mean by this? Is there a parameter that could implement this? Or is it you should adjust your dataset beforehand? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 🤗 Great follow-up question. Ultralytics YOLO integrates data augmentations directly into the training pipeline, meaning you don't have to modify your dataset beforehand. These augmentations are applied on-the-fly during training to enhance the diversity of your dataset and improve model generalization. When you train your model using the No need to set a specific parameter for basic augmentations; they're handled for you by default! However, you can customize augmentations to some extent using the training configuration file ( If you've missed it, don't worry. It's a fairly common question, and the details are indeed more implicit within the documentation. Always feel free to ask for clarifications! 😄 Happy training! |
Beta Was this translation helpful? Give feedback.
-
|
So if I understand correctly, when i train my model with following code, the ultralytics YOLO is already implementing augmentations? from ultralytics import YOLO model = YOLO("yolov8n-seg.pt") # load a pretrained model (recommended for training) freeze_layers = 20 results = model.train(data="shirt.yaml", epochs=50) # train the model The yaml file looks as following: train: "train/images" roboflow: |
Beta Was this translation helpful? Give feedback.
-
|
Yes, you've got it right! 😊 When you train your model with the Ultralytics YOLO framework, data augmentations are indeed automatically applied during training. This helps enhance the diversity of your dataset, which is especially valuable for small datasets or to improve model generalization. Here's a quick look at your training setup: from ultralytics import YOLO
model = YOLO("yolov8n-seg.pt") # Pretrained model loaded
# ... your freezing logic ...
results = model.train(data="shirt.yaml", epochs=50) # Augmentations are applied here!In this setup, augmentations such as random scaling, cropping, and flipping are employed. This is seamlessly handled by the Ultralytics framework, so there's no need for manual augmentation of your dataset before training. Your training configuration through Keep up the great work on your project! If you have more questions or need further assistance, feel free to ask. |
Beta Was this translation helpful? Give feedback.
-
|
hi, i have done a segmentation model and it works pretty well. How can i extract segmentation information? just like object detection it is x1,x2,y1,y2. So how can i do that on segmentation? i labelled using polygon then export as coco before converting to yolo format |
Beta Was this translation helpful? Give feedback.
-
|
@TanWeeKang to extract segmentation information, you can use the |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Is there a paper for yolov11? Thanks in advance! |
Beta Was this translation helpful? Give feedback.
-
|
@to505to505 hello! Currently, there isn't a published paper for YOLOv11. For the latest updates and detailed information, please refer to the Ultralytics documentation. |
Beta Was this translation helpful? Give feedback.
-
|
Hello! I want to ask you about how to make the segmentation label when some situation. i have problem to choose how to set the label.. |
Beta Was this translation helpful? Give feedback.
-
|
@byunbriana for intersection situations in segmentation, consider using separate masks for each object, ensuring each mask accurately outlines the object's shape, even if they overlap. This approach maintains clarity in object boundaries. |
Beta Was this translation helpful? Give feedback.
-
|
Hello I want to give some advice how to draw the segmentation mask when two products in intersection situation. |
Beta Was this translation helpful? Give feedback.
-
|
@byunbriana for handling intersection situations in segmentation, consider refining your model's training data to include more examples of overlapping objects, or adjust the model's hyperparameters to improve mask accuracy. You might also explore post-processing techniques to better separate intersecting objects. |
Beta Was this translation helpful? Give feedback.
-
|
Is the detection box identified by YOLOv8 fitted to the surface of the object, or is it just a random plane？ |
Beta Was this translation helpful? Give feedback.
-
|
@wcy7788 the detection box in YOLOv8 is not a random plane; it is fitted to the object's surface based on the model's predictions to best encompass the detected object. For precise shapes, consider using instance segmentation models, which provide detailed masks. |
Beta Was this translation helpful? Give feedback.
-
|
After the segmentation recognition function of YOLOv8 is simplified into a point, is this point also fitted to the surface of the object? Recently, monocular analysis of depth detection is being done. |
Beta Was this translation helpful? Give feedback.
-
|
The point in YOLOv8's segmentation is not fitted to the object's surface; it represents a simplified representation of the object's location. For detailed surface fitting, consider using depth estimation techniques alongside segmentation. |
Beta Was this translation helpful? Give feedback.
-
|
Hello,I have a question of load checkpoint,I've already download 11-seg.pt. However,when I try to tain with my datasets,it's always try to download yolo11n.pt, |
Beta Was this translation helpful? Give feedback.
-
|
@GenJin it seems like the model path might be incorrect in your training command. Ensure you're specifying the correct model file, such as |
Beta Was this translation helpful? Give feedback.
-
|
thank you,I found thar if I rename 'yolo11n-seg.pt ' to 'yolo11n.pt',It won't download again.It’s weird but effective. |
Beta Was this translation helpful? Give feedback.
-
|
It's great that you found a workaround by renaming the file. This behavior might be due to the model path configuration in your script. Ensure that the correct model file name is specified to avoid unnecessary downloads. If you have further questions, feel free to ask! |
Beta Was this translation helpful? Give feedback.
-
|
Hello, I'm trying to do the same with ultralytics with this example: https://github.com/opencv/opencv_zoo/tree/main/models/image_segmentation_efficientsam in which one data point on the image is used to get the segmented object. |
Beta Was this translation helpful? Give feedback.
-
|
@quanshuli you can achieve similar instance segmentation with Ultralytics by using YOLO11-seg models, which provide masks for each object. While the approach differs from OpenCV's point-based method, YOLO11-seg models are pretrained on COCO and can be fine-tuned for your specific needs. For more details, check the Ultralytics documentation. |
Beta Was this translation helpful? Give feedback.
-
|
It is. See the documentation section titled “ SAM prediction example” https://docs.ultralytics.com/models/sam/#how-to-use-sam-versatility-and-power-in-image-segmentation |
Beta Was this translation helpful? Give feedback.
-
|
@KDeser for detailed guidance on using SAM for segmentation, please refer to the "SAM prediction example" section in the documentation you mentioned. If you have further questions, feel free to ask here. |
Beta Was this translation helpful? Give feedback.
-
|
yolov8l-seg.pt Is this still available for download |
Beta Was this translation helpful? Give feedback.
-
|
@wcy7788 yes, the |
Beta Was this translation helpful? Give feedback.
-
|
I encountered a problem while working on the YOLOv8-seg segmentation task, and I would like to understand the possible reasons and solutions. When I used the following code for inference: results = model(r'/root/2_4.jpg', save=True, save_crop=False, imgsz=640, classes=[1], max_det=1, show_labels=False, show_conf=False, show_boxes=False)and tried to save the binary mask results from the how can i resolve this problem. |
Beta Was this translation helpful? Give feedback.
-
|
@nihil127 the offset issue you're experiencing with the binary mask is likely due to the padding added during mask processing. To resolve this, you can adjust the dimensions of the mask by cropping the padding area before saving the mask. The |
Beta Was this translation helpful? Give feedback.
-
|
Not sure how to fix this but try removing the
imgsz=640
Parameter from your code
…On Mon, 18 Nov, 2024, 7:06 pm Glenn Jocher, ***@***.***> wrote:
@nihil127 <https://github.com/nihil127> the offset issue you're
experiencing with the binary mask is likely due to the padding added during
mask processing. To resolve this, you can adjust the dimensions of the mask
by cropping the padding area before saving the mask. The utils/plotting.py
script may need modification to ensure the mask aligns correctly with the
image size. Alternatively, you could extract and adjust the mask dimensions
manually within your inference loop. For more detailed guidance, reviewing
the source code for adjustments or consulting the Ultralytics community for
script customization tips might be beneficial.
—
Reply to this email directly, view it on GitHub
<#8734 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ATLJGEBKXAC5FB3TBKL6THL2BHUNTAVCNFSM6AAAAABEKFCKICVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMRZGIYTIMA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
I will have a try thx |
Beta Was this translation helpful? Give feedback.
-
|
@nihil127 thank you for your interest. If you encounter any issues or need further assistance, feel free to reach out here or consult the Ultralytics documentation for more detailed guidance. Your success with YOLO is important to us! |
Beta Was this translation helpful? Give feedback.
-
|
well thanks for your patience! |
Beta Was this translation helpful? Give feedback.
-
|
You're welcome! If you have any more questions or need further assistance with YOLO models, feel free to ask. The Ultralytics community and documentation are great resources for support as well. |
Beta Was this translation helpful? Give feedback.
-
|
I have a question about segmentation competition https://www.kaggle.com/competitions/ultrasound-segmentation-of-thyroid-nodules/overview ，which dataset contains images and masks .First, I deal with masks to txt annotations. |
Beta Was this translation helpful? Give feedback.
-
|
@changhuazuo it seems you're encountering issues with prediction quality after training your YOLO11 segmentation model. To address the null mask values, consider refining your dataset and reviewing label quality. Ensure that the data preparation and augmentation steps are optimal, and experiment with different hyperparameters or training strategies. For evaluating performance using metrics like Surface Dice, you may need to implement custom evaluation scripts tailored to your dataset's requirements. If you need further guidance, the YOLO segmentation documentation might offer additional insights. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks ,The dataset label is mask ,every image has a mask .The label quality I think is good . when model.train(data='data_seg.yaml', epochs=1, imgsz=640) epoch is 1,some image has two masks,but every image has a RLE encode ,how to understand it？def rle_encode(img): |
Beta Was this translation helpful? Give feedback.
-
|
Thanks ，I deal with it,but Surface Dice number is very small |
Beta Was this translation helpful? Give feedback.
-
|
@changhuazuo if you're experiencing a low Surface Dice score, consider revisiting your dataset and annotations to ensure they're correctly formatted and high-quality. Additionally, experimenting with different model architectures or training parameters might help improve the segmentation results. For evaluating metrics like Surface Dice, you might need to implement custom evaluation scripts tailored to your dataset's needs. |
Beta Was this translation helpful? Give feedback.
-
|
I have used the code in (How do I train a YOLO11 segmentation model on a custom dataset?) using google colab. How to proceed further to segment custom objects which i had trained earlier? |
Beta Was this translation helpful? Give feedback.
-
|
@AbhishekMitra-AIT to proceed with segmenting custom objects after training on your dataset, use the trained model for predictions. In Python, load your custom model (e.g., from ultralytics import YOLO
model = YOLO("path/to/best.pt") # Load your trained model
results = model("path/to/image.jpg") # Predict on an image
results.show() # Display segmentation resultsYou can also use the CLI: |
Beta Was this translation helpful? Give feedback.
-
|
I used yolo11n-seg.pt model .Image[360 width,640 Height]. The mask values from the model has values below:| |
Beta Was this translation helpful? Give feedback.
-
|
Input image size is (360, 640, 3), passed to the model without preprocessing. |
Beta Was this translation helpful? Give feedback.
-
|
Model Output for reference |
Beta Was this translation helpful? Give feedback.
-
|
@pakshi10 the height of 384 in the model output is due to internal resizing during inference, as YOLO models typically resize input images to maintain a fixed aspect ratio for processing (e.g., 640x640 or similar). Regarding the mask's maximum y-value being 360 instead of 359, this might reflect 1-based indexing for the mask output. Ensure consistent indexing throughout your workflow to avoid confusion. Let us know if you need further assistance! |
Beta Was this translation helpful? Give feedback.
-
|
FYI . I didn't do any indexing . This is from the output from the yolo mask . I think this is either model or computation issue . The mask.xy comes from the model . Look into the issue why the index is 360 and not 359 from yolo mask . |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for pointing this out. The model uses height 384 due to internal resizing during inference to align with its architecture, even if the input height is 360. As for the maximum |
Beta Was this translation helpful? Give feedback.
-
|
I also have a question
1. I have exported the default yolov8n.pt model in onnx format and noticed
that the model size doubled from 6 MB to 12 MB, why is that ?
2. For some models, while exporting, I can enable both half=True and
int8=True. Why is that? There should only be one of the possible size half
enables FP16 and int8 enable 8 bit integer value?
3. Does enabling int8 or half also quantizes the model ? I want to learn
more about quantization
…On Wed, 18 Dec, 2024, 10:46 pm Akshay Kumar, ***@***.***> wrote:
Model Output for reference
0: 384x640 1 0, 1 3, 6.5ms
Speed: 1.1ms preprocess, 6.5ms inference, 1.7ms postprocess per image at
shape (1, 3, 384, 640)
—
Reply to this email directly, view it on GitHub
<#8734 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ATLJGEERU4OZHGJAP5A65BD2GGUVXAVCNFSM6AAAAABEKFCKICVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRQHAZDSNI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
@RishiKumarSoni 1. The increase in size when exporting to ONNX is due to differences in model serialization formats; ONNX often includes additional metadata and structure for compatibility. |
Beta Was this translation helpful? Give feedback.
and others
-
tasks/segment/
Learn how to use instance segmentation models with Ultralytics YOLO. Instructions on training, validation, image prediction, and model export.
https://docs.ultralytics.com/tasks/segment/
Beta Was this translation helpful? Give feedback.
All reactions