Please share Instance Segmentation custom training steps

[prateekgml]

Hi @WongKinYiu ,

Please document and share the instance segmentation training steps for a custom dataset.

[nick-konovalchuk]

Yeah, we're really grateful for adding an option of instance segmentation in the first place. Please, consider adding segmentation training script so we can use yolov7-mask at a full extent

[WongKinYiu]

For object detection, two type of label formats are supported.

1. bbox: class center_x center_y width height
2. polygon: class x1 y1 x2 y2 x3 y3 ... xn yn
   (where n could be different value of different objects)

For instance segmentation, only polygon label format is supported.

1. polygon: class x1 y1 x2 y2 x3 y3 ... xn yn
   (where n could be different value of different objects, you could take a look coco example)

After finish prepare your dataset, you have to do what you also need do when training custom object dataset. For example, create data yaml and model yaml of your custom dataset.

1. change path and names according to coco.yaml
2. change nc according to yolov7-seg.yaml

Finally, you could use the training script to train your instance segmentation model. (please remember to change data yaml and model yaml for your custom dataset)

[WeiChihChern]

@WongKinYiu

Another issue that I am facing with my custom dataset for instance segmentation is from loss calculation:

seg/utils/loss.py, at line 198:
anchors, shape = self.anchors[i], p[i].shape

Traceback (most recent call last):
File "train.py", line 630, in
main(opt)
File "train.py", line 526, in main
train(opt.hyp, opt, device, callbacks)
File "train.py", line 308, in train
loss, loss_items = compute_loss(pred, targets.to(device)) # loss scaled by batch_size
File "/hub/yolov7/seg/utils/loss.py", line 125, in call
tcls, tbox, indices, anchors = self.build_targets(p, targets) # targets
File "/hub/yolov7/seg/utils/loss.py", line 198, in build_targets
anchors, shape = self.anchors[i], p[i].shape
AttributeError: 'list' object has no attribute 'shape'

I assume it has nothing to do with my annotation format as it's using the prediction feature maps for detection. In my case, p is a list of list of tensors:
print(p[0][0].size()) --> torch.Size([16, 3, 120, 120, 43])
print(p[0][1].size()) --> torch.Size([16, 3, 60, 60, 43])
print(p[0][2].size()) --> torch.Size([16, 3, 30, 30, 43])

And just to double check, I have .txt annotation files placed in the same folder as the images, and each .txt contains polygon annotation only as you mentioned above.

Do you have any insight regarding this? Thank you.

Update: Resolved. Was using the wrong train.py.

[nick-konovalchuk]

@WeiChihChern
Did you use a training script from u7 branch or mask branch? I've managed to make the training script from u7 work

[nick-konovalchuk]

Btw, @WongKinYiu, could you tell us what's the difference between mask and seg versions of yolov7?

[WeiChihChern]

@bottledmind I used the wrong training code apparently, should be using the one in ./seg/segments/train.py rather than the one in ./seg/train.py. It's training now. Thanks.

[BandyKenny]

@WeiChihChern 我发现我也是遇到这个问题，居然是没有找到正确的train.py😂

[prateekgml]

@WongKinYiu thanks! I am able to do the instance training on my dataset-
https://dsbyprateekg.blogspot.com/2022/09/how-to-train-custom-dataset-with-yolov7.html

[achbogga]

AttributeError: Can't get attribute 'Merge' on <module 'models.common' from '/home/aboggaram/projects/yolov7-mask/seg/models/common.py'>

Getting this error while trying the above instructions

[ahasan-haque]

@WongKinYiu I am facing problems annotating objects formed by two polygons (see the figure attached). In coco style format, I can use segmentation as a list of list. How should I do it here? all x, y of two polygons at the same line? if I make different lines for them; I guess they will be recognized as two distinct instance of same class right?

[aurelm95]

@WongKinYiu I am facing problems annotating objects formed by two polygons (see the figure attached). In coco style format, I can use segmentation as a list of list. How should I do it here? all x, y of two polygons at the same line? if I make different lines for them; I guess they will be recognized as two distinct instance of same class right?

I have the same question. Did you solve it? @ahasan-haque

[ichsan2895]

@WongKinYiu thanks! I am able to do the instance training on my dataset- https://dsbyprateekg.blogspot.com/2022/09/how-to-train-custom-dataset-with-yolov7.html

Hello, could you share the script for creating YOLO custom dataset for Instance segmentation?

For instance segmentation, only polygon label format is supported.

1. polygon: class x1 y1 x2 y2 x3 y3 ... xn yn
   (where n could be different value of different objects, you could take a look coco example)

After finish prepare your dataset, you have to do what you also need do when training custom object dataset. For example, create data yaml and model yaml of your custom dataset.

1. change path and names according to coco.yaml
2. change nc according to yolov7-seg.yaml

Finally, you could use the training script to train your instance segmentation model. (please remember to change data yaml and model yaml for your custom dataset)

I dont find any idea for converting COCO format to the suggested format. I also use labelme for labelling but not found any YOLO format.

[jasonchanhku]

@WongKinYiu thanks for the great work, I have a question related to this, which is, does it work for polyline? i.e the segmentation labels are polylines instead of polygons

[utokyo-sm96]

@WongKinYiu , can you share the idea of model ensembling with 3 pre trained weights in YOLOv7 instance segmentation

[polarisV0V]

I have a small question. Since the dataset format for segmentation task is : class x1 y1 x2 y2 ...... What if I want to segment a circle object with a hole inside? This kind of dataset format will include the inside part which I don't want!

[aurelm95]

@polarisV0V I have the same problem. I thinks there is no quick solution for this :(