Some questions about yolov11 training custom data

[jaffe-fly]

i am use yolov11 train my custon datasets, and now have two question
1,when infer often have a object with two class of boxes and high conf, eg object A has a class A with conf 0.85 box and has a class B with conf 0.77 box. why？ if this is due to train datasets, if there is a open source tool can find these errors?
2, if a object rotate 90 °， yolo cant detect it, even though I have set degree when training. i cant find any guide for set custom augment.How should I perform effective data augmentation in this case?

[UltralyticsAssistant]

👋 Hello @jaffe-fly, thank you for your interest in Ultralytics 🚀! We appreciate you taking the time to share your questions.

If this is a 🐛 Bug Report, please ensure you provide a minimum reproducible example (MRE) to help us better understand and debug the issue.

For the questions you raised:

1. Regarding the overlapping boxes with high confidence for different classes, it could indeed be related to your training dataset or annotations. Please review your dataset for potential issues such as mislabels or overlapping annotations that may cause this behavior. If possible, share a detailed description of how your dataset is structured and any images/tests showcasing the issue.

2. For detecting rotated objects, ensure that your augmentation implementation matches the intended result. If you’ve set specific augmentation parameters like degree, it’s important to validate their effect during training via visualization of augmented samples. You can elaborate on how you are applying your custom augmentations to help us provide more tailored feedback.

Please also verify you are using the latest ultralytics package in a clean Python environment to ensure no outdated dependencies are impacting your results. Run the following command for an update:

pip install -U ultralytics

If you’re seeking guidance on custom training, data augmentation, or other advanced configurations, sharing additional details such as dataset specifics, training logs, or examples of your inference outputs can go a long way in assisting us to help you effectively.

This is an automated response, but rest assured, an Ultralytics engineer will review your question in greater detail soon! 😊

[glenn-jocher]

@jaffe-fly 1. The issue of an object being detected with two classes and high confidence could arise from ambiguous or overlapping labels in your training dataset. Regularly reviewing and ensuring high-quality annotations can help. Tools like Label Studio or other annotation platforms may assist in identifying labeling issues.

2. For detecting rotated objects, ensure your dataset includes diverse orientations of objects and verify that augmentations are applied correctly. Albumentations is a powerful library for augmentations, including rotations. You can integrate it for better augmentation during training. Learn more here: Albumentations Guide.

[jaffe-fly]

so should i use Albumentations to augment datasets offline, then start traing? can use it to custom augment on the fly?

[glenn-jocher]

@jaffe-fly albumentations can be used both offline to augment datasets and on the fly during training with YOLO11. For on-the-fly augmentation, ensure Albumentations is installed, and it will integrate seamlessly during the training process. Details here: Albumentations Integration.

[jaffe-fly]

this doc no code example for how to use Albumentations to custom augment, i am change

ultralytics/ultralytics/data/augment.py

Line 1850 in 0504d3a

A.ImageCompression(quality_lower=75, p=0.0),

here and add A.RandomRotate90(p=0.5). Is there a better way than this?

[glenn-jocher]

Thank you for your question! While modifying augment.py directly works, a more maintainable approach is to integrate custom augmentations using Albumentations externally. You can pass your transformations through the Albumentations class during training by importing and customizing it. For reference, see the Albumentations guide. This avoids altering the core code and simplifies updates.

[Y-T-G]

1. You can use agnostic_nms=True during inference to remove duplicate objects that are of different classes.
2. What did you set it to?

[jaffe-fly]

Is the generalization ability of yolov11 better than yolov8? Under the same dataset training and the same parameters, and then tested in a new scene, yolo11 can detect part of object， but yolov8 cannot，Even if you set conf= 0.3，yolov8 still cant detect anything

[jaffe-fly]

What is the smallest pixel size that yolov8 and yolov11 can detect?

[glenn-jocher]

The smallest detectable object size for YOLOv8 and YOLO11 depends on the input image resolution and the model's stride, as smaller objects may be missed if they fall below the receptive field. For optimal performance, ensure the image resolution is high enough to represent small objects clearly.

[jaffe-fly]

original image is 2560*1440，training imgsz is 1024, yolov11l-obb model,, how to get smallest detectable object size ,

[glenn-jocher]

The smallest detectable object size depends on the model's stride and the image resolution during training. For YOLO11, the stride for yolov11l-obb is typically 32 pixels. With a 2560x1440 image resized to 1024x576 during training, the smallest detectable object would be approximately (1024 / 32) = 32 pixels in the resized image, equivalent to (32 * original_image_width / imgsz) in the original resolution.