Freezing layers YOLOv8 for fine-tuning

[Rayen023]

Hello, I'm currently attempting to freeze the backbone of YOLOv8 for fine-tuning purposes. As a baseline, I tried training the model with all layers frozen. Surprisingly, it produced favorable results. Initially, I believed this to be an issue with PyTorch's implementation of requires_grad = False. However, after perusing the documentation at the following link:

Transfer Learning with Frozen Layers

It appears this is the expected behavior.
Consequently, I'm trying to understand how the weights and biases can still update even when requires_grad = False has been applied to all layers.

Here's my code:

model = YOLO("yolov8l.yaml")  # Build a new model from scratch
for k, v in model.model.named_parameters(): 
    v.requires_grad = False  # Freeze all layers 

# Use the model
model.train(data=data_path, epochs=50, batch=32, imgsz=544, device=0)

I also tried to set the momentum and weight_decay to zero after freezing the layers, as one user suggested that the model might still be learning because these parameters might still provide access to gradients. Any insights would be appreciated.

[atharvavaidya14]

Hello,
Did you find any solution to the problem? I am also working with fine-tuning yolov8 with freezing the backbone. But if it is not getting frozen in the way they have mentioned in the documentation, then that's a big problem.

[glenn-jocher]

@atharvavaidya14 hello! When you're fine-tuning YOLOv8 and you want to freeze certain layers, setting requires_grad = False should indeed prevent those layers from updating during training. If you're observing changes in performance, it could be due to other parts of the model or training process that are still active.

It's important to ensure that all layers you intend to freeze are correctly set. Additionally, if you're using pre-trained weights, make sure to load them before freezing the layers. After freezing, no updates should occur to the weights and biases of those layers during backpropagation.

If you're still seeing unexpected behavior, it might be worth double-checking your training loop and optimizer setup to ensure that the frozen state of the parameters is respected throughout the training process.

Remember that fine-tuning typically involves freezing the early layers (like the backbone) and only training the later layers. If all layers are frozen, the model should not learn or change its predictions, as no learning is taking place.

If you continue to face issues, please provide more details about your training setup and any observations you've made, and we can look into it further. The YOLOv8 documentation on our modes and tasks might also offer additional insights into the training process. 📘🔍

[ksooklall]

From the documentation you can supply freeze as a parameter to train.
freeze, None, (int or list, optional) freeze first n layers, or freeze list of layer indices during training

[atharvavaidya14]

@ksooklall unfortunately, it does not work with yolov8-pose models. or maybe V8 in general. I haven't tried for other models
It gives this error-
SyntaxError: 'freeze' is not a valid YOLO argument

with this code:

model = YOLO("yolov8x-pose.pt")
model.train(
        data=f'data/data.yaml',
        rect=True, 
        epochs=no_epochs, 
        patience=patience,
        imgsz=640, 
        batch=16, 
        close_mosaic=1, 
        save=True, 
        project=dir, 
        name=proj_name,
        warmup_epochs=0,
        freeze=2,
    )

[jamesquags]

how many layers is "the backbone"?

[zohaibjan]

Can you freeze the backbone if you are using a yolov9.pt existing pretrained weights ?. When I tried the code but with the pt file it gave me this error: "AttributeError: 'DetectionModel' object has no attribute 'backbone'"

[pderrenger]

@zohaibjan hello!

For freezing the backbone in YOLOv9 models, it's important to ensure that the model architecture explicitly defines a 'backbone' attribute that can be accessed as you've tried. It seems like the YOLOv9 model structure might not have a directly accessible 'backbone' attribute like some other models.

A good approach would be to review the specific architecture of the YOLOv9 model you are using to identify how the layers are structured and named. You can then freeze the appropriate layers by their names or indices directly.

If you need further assistance on how to navigate the model's architecture or how to freeze specific layers, feel free to ask!

Happy coding! 🚀

[Sakethnoothi775]

@zohaibjan hello!

For freezing the backbone in YOLOv9 models, it's important to ensure that the model architecture explicitly defines a 'backbone' attribute that can be accessed as you've tried. It seems like the YOLOv9 model structure might not have a directly accessible 'backbone' attribute like some other models.

A good approach would be to review the specific architecture of the YOLOv9 model you are using to identify how the layers are structured and named. You can then freeze the appropriate layers by their names or indices directly.

If you need further assistance on how to navigate the model's architecture or how to freeze specific layers, feel free to ask!

Happy coding! 🚀

Hello, i am trying to train a yolov9s for 2500 phone and 3000 book images i.e phone,book classes, can you please suggest the exact number for parameter 'freeze' in training

[rishith2]

Did anyone figure it out, how to do transfer learning. Do we have to freeze the backbone(first 53 layers) and give dataset with base images and new class images? Or just pass the dataset(base+new) without freezing.
I tried this:
model = YOLO("yolov8l.yaml")
for k, v in model.model.named_parameters():
v.requires_grad = False # Freeze all layers
model.train(data=data_path, epochs=50, batch=32, imgsz=544, device=0)

with dataset that only has new class images and freezing all layers, it gives predictions for just new classes but not the base coco classes. Logically if youre freeezing all the layers, it shouldnt learn anything. Do we have to give the coco classes images as well? Does transfer learning happen without freezing, as i am giving the base classes images ?

[glenn-jocher]

For effective transfer learning with YOLOv8, you don't need to freeze the backbone. Simply include both the base and new class images in your dataset and train the model. Freezing layers is optional and typically used to retain specific learned features. Ensure your dataset and configuration file correctly reflect all classes. If issues persist, please verify with the latest package version.

[Y-T-G]

It's still getting updated because the BatchNorm layers are in train mode.

I freeze the layers completely here by putting BatchNorm layers to eval mode during training.

[WindFreeFoliage]

So cool, Thanks

[glenn-jocher]

@WindFreeFoliage thank you for your kind words! The credit goes to the YOLO community and the Ultralytics team for their efforts. Let us know if you have any questions!