Segmentation polygons2masks_overlap() in np.int32

[glenn-jocher]

May resolve #9461

WARNING: Masks should be uint8 for fastest speed, change needs profiling results to determine impact.

@AyushExel @Laughing-q

Signed-off-by: Glenn Jocher [email protected]

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Improved mask generation for overlapping segments in image segmentation tasks.

📊 Key Changes

• Altered the data type of the generated masks in polygons2masks_overlap function based on the number of segments.
• np.uint8 is used when the number of segments is 255 or fewer.
• np.int32 is utilized for scenarios with more than 255 segments.

🎯 Purpose & Impact

• 🎨 Enhances segmentation mask creation to support a larger number of overlapping regions.
• 🛠️ Solves the issue of mask overflow when dealing with a high count of segments, which typically results in incorrect mask values.
• 💡 This change will directly aid in the accurate training and inference of models on complex images with numerous overlapping objects, improving the robustness and reliability of image segmentation applications.

[Laughing-q]

@glenn-jocher that's strange, I can't reproduce the issue either. Don't know why uint8 is converted to uint16. This fix might be a temporary way.

[glenn-jocher]

@Laughing-q yes it is strange. The CI passes both ways, but ideally we want the masks at 8 bits, or 16 bits vs 32 bits.

[glenn-jocher]

@Laughing-q what do you think is the best solution here? Should we replace np.uint8 with np.uint16?

[Laughing-q]

@glenn-jocher The issue is that torch can't convert numpy with uint16 type to tensor. So the uint16 can not be used. Actually what bothers me the most is why uint16 appears. If you want to fix this quickly, maybe we just change it to int32. Or wait for a while to see if the guys who's opened the issue can provide us more info.

[Laughing-q]

@glenn-jocher I've reproduced the issue, this is because the number of objects exceeds 255 which is beyond the range of uint8. Then automatically this line will convert uint8 to uint16:

yolov5/utils/segment/dataloaders.py

Line 327 in 63368e7

mask = ms[i] * (i + 1)

As torch do not support uint16, then I think the np.int32 is the best solution.

EDIT: Or we can do this update if you want to keep uint8 as a part of it. I think this might be better as mostly the number of objects per image from users wouldn't exceed 255.

    masks = np.zeros((img_size[0] // downsample_ratio, img_size[1] // downsample_ratio), 
            dtype=np.int32 if len(segments) > 255 else np.uint8)

[glenn-jocher]

@Laughing-q ah ok! Yes I like the second solution, I'll update the PR for that and merge.