YOLOv5 receptive range size

[Heaven0612]

Hello
I am currently doing small defect detection. The length or width of the defect occupies at most 0.02-0.08 of the original image. I am using YOLOv5s. I have found in other previous questions that YOLO default uses P3 P4 P5 as the detection head. In other questions and answers, you can Knowing that the P5 is a large detector, I thought the listing could be deleted in my case. In your previous answer I noticed that you mentioned the receptive field.

I want to ask about this. I calculated the receptive field size in 5s to skip "skip connection".
(The only modules that affect perception and calculation are CNN and bottleneck in CSP)

P1 is 6
P2+CSP is 18
P3+CSP is 66
P4+CSP is 194
P5+CSP is 322

What I want to ask you about is about receptive field correspondence.
In my example, my input size is 640 multiplied by the overall defect size to cover the maximum range of 0.08, and the defective pixels account for 52. So I think I only need to cover the receptive field with the tiny defect size and expand it to twice the receptive field size in another layer. To make another area partial, taking the above 52 as an example, I should select 66 of P3 and reduce the bottleneck in P4 to twice the size of 66 and delete P5.
(Since P5 is aimed at large objects and the larger receptive field may cause the object to be blurred in the depth of the rolling machine), I think that the reason for modifying this part from the backbone is because the backbone is feature extraction.

What I want to ask is, is the idea of ​​deleting the bottleneck correct?

In addition, does this idea correspond to the receptive field you mentioned (I am doing research on alignment)

[github-actions]

👋 Hello @Heaven0612, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[glenn-jocher]

@Heaven0612 hello,

Thank you for your detailed question and for sharing your insights on the receptive field calculations. Your approach to understanding the receptive field in the context of small defect detection is commendable.

To address your query, let's break down the key points:

1. Receptive Field and Detection Heads:

   • You are correct that YOLOv5 uses multiple detection heads (P3, P4, P5) to detect objects at different scales. For small defect detection, focusing on the appropriate receptive field is crucial.
   • Given that your defects occupy a very small portion of the image (0.02-0.08), it makes sense to prioritize detection heads that correspond to smaller receptive fields, such as P3.

2. Modifying the Backbone:

   • Your idea of deleting the P5 detection head and focusing on P3 and P4 is reasonable. P5 is indeed more suited for larger objects, and removing it can help the model focus on smaller details.
   • Reducing the bottleneck in P4 to twice the size of P3 (66) and deleting P5 can be a good strategy. This way, you ensure that the receptive field is more aligned with the size of the defects you are detecting.

3. Implementation:

   • To implement these changes, you can modify the YOLOv5 model configuration file (e.g., yolov5s.yaml). You can adjust the number of layers and channels in the backbone and neck to suit your needs.

   • Here's an example of how you might adjust the configuration:

     # yolov5s.yaml
     backbone:
       # [from, number, module, args]
       [[-1, 1, Conv, [64, 6, 2, 2]],  # 0-P1/2
        [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4
        [-1, 3, BottleneckCSP, [128]],  # 2
        [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
        [-1, 9, BottleneckCSP, [256]],  # 4
        [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
        [-1, 9, BottleneckCSP, [512]],  # 6
        # Remove P5 layers
       ]
     
     head:
       [[-1, 1, Conv, [256, 1, 1]],  # 7
        [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 8
        [[-1, 4], 1, Concat, [1]],  # 9
        [-1, 3, BottleneckCSP, [256]],  # 10
     
        [-1, 1, Conv, [128, 1, 1]],  # 11
        [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 12
        [[-1, 2], 1, Concat, [1]],  # 13
        [-1, 3, BottleneckCSP, [128]],  # 14
     
        [[14, 10, 6], 1, Detect, [nc, anchors]],  # Detect(P3, P4)
       ]

4. Verification:

   • Ensure that you test the modified model thoroughly to verify that it performs well on your specific task. You might need to experiment with different configurations to find the optimal setup.

For more detailed information on the YOLOv5 architecture and how to customize it, you can refer to the Ultralytics YOLOv5 Architecture Documentation.

I hope this helps! If you have any further questions or need additional assistance, feel free to ask.

[Heaven0612]

@glenn-jocher Thank you for your reply but I still have some questions

Sorry, I guess I didn't express myself clearly.

I am using the C3 module, but I found that the C3 module has a specialized bottleneck, because it has a 11 and 33 Conv, and 3*3 will affect the receptive field. The aforementioned receptive field size P1-P5 is I calculated based on the original architecture of v5s. What I want to understand is that in the original setting, the bottleneck number in the C3 block of v5s P2 -1,P3-2 ,P4-3 P5-1 . What I want to understand is this 1:2:3:1 ratio. Where does it come from? In addition, as described before, P3 is 66 but P4 is 194, but I deleted C3 in P4 due to alignment. The bottleneck in the module left 1 block from 3 blocks. At this time, the receptive field size is 130, delete 2 bottleneck in P4 is correct?

But I still have doubts about deleting the number of bottlenecks in C3 to reduce the originally too large receptive field. Do you have a better explanation?

In addition, for P2, I input a 640 image today. Its defect is that most of the original image pixel size is 0.2 and the ratio is about 13 pixels. In fact, at this time, I pulled out P2 and modulated P3 to double the size. Delete P4 and only proceed to P2 and P3. in PAnet fusion ?

[glenn-jocher]

Hello @Heaven0612,

Thank you for your detailed follow-up and for clarifying your questions. Let's dive into the specifics of your query regarding the C3 module and the receptive field.

Understanding the C3 Module and Receptive Field

The C3 module in YOLOv5 is designed to enhance feature extraction by incorporating a combination of 1x1 and 3x3 convolutions. The ratio of bottlenecks (1:2:3:1) in the C3 blocks across different layers (P2, P3, P4, P5) is empirically chosen to balance the model's complexity and performance. This ratio helps in capturing features at multiple scales effectively.

Modifying the Receptive Field

Your approach to modifying the receptive field by adjusting the number of bottlenecks in the C3 module is logical. However, it's essential to ensure that these changes do not adversely affect the model's ability to learn and generalize. Here's a more detailed explanation:

1. Receptive Field Calculation:

   • The receptive field size you calculated (P1: 6, P2: 18, P3: 66, P4: 194, P5: 322) is based on the original architecture. When you modify the number of bottlenecks, the receptive field will change accordingly.
   • By reducing the number of bottlenecks in P4 from 3 to 1, you effectively reduce the receptive field size from 194 to 130. This change should help in focusing on smaller defects, as you mentioned.

2. Deleting Bottlenecks:

   • Deleting bottlenecks in the C3 module to reduce the receptive field is a valid approach. However, it's crucial to test the model's performance after making these changes to ensure that it still captures the necessary features for defect detection.
   • If you find that reducing the bottlenecks in P4 improves performance, then it is a step in the right direction. Otherwise, you might need to experiment with different configurations.

Implementation Example

Here’s how you might adjust the configuration in the yolov5s.yaml file to reflect your changes:

# yolov5s.yaml
backbone:
  # [from, number, module, args]
  [[-1, 1, Conv, [64, 6, 2, 2]],  # 0-P1/2
   [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4
   [-1, 3, BottleneckCSP, [128]],  # 2
   [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
   [-1, 9, BottleneckCSP, [256]],  # 4
   [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
   [-1, 1, BottleneckCSP, [512]],  # 6 Reduced bottlenecks from 3 to 1
   # Removed P5 layers
  ]

head:
  [[-1, 1, Conv, [256, 1, 1]],  # 7
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 8
   [[-1, 4], 1, Concat, [1]],  # 9
   [-1, 3, BottleneckCSP, [256]],  # 10

   [-1, 1, Conv, [128, 1, 1]],  # 11
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 12
   [[-1, 2], 1, Concat, [1]],  # 13
   [-1, 3, BottleneckCSP, [128]],  # 14

   [[14, 10, 6], 1, Detect, [nc, anchors]],  # Detect(P3, P4)
  ]

Testing and Validation

After making these changes, it's crucial to:

• Test the Model: Run the modified model on your dataset to ensure it performs well in detecting small defects.
• Iterate: If the performance is not satisfactory, consider fine-tuning the number of bottlenecks or other hyperparameters.

Additional Resources

For more detailed information on the YOLOv5 architecture and customization, you can refer to the Ultralytics YOLOv5 Architecture Documentation.

I hope this helps clarify your questions! If you have any further doubts or need additional assistance, feel free to ask. The YOLO community and the Ultralytics team are here to support you. 😊

[Heaven0612]

@glenn-jocher Thank you for your reply

First of all, I would like to ask why you choose the BottleneckCSP module instead of the C3 module.

Can you also tell me why the BottleneckCSP of P3 has 3 modules, but the C3 module after P3 of the original file has 2 modules?

I found that the neck part of your example is FPN instead of PANet. Is this because the defect sizes have been aligned? Or is it that PANet is to give the shallow feature part to the large detection part, but since there is no need for large detection, PANet is not needed.

In addition, I have another question about your opinion on SPPF, because this part seems to have been deleted from the example. Although understanding maxpool will increase the receptive field, should we reserve space to place this block?

[glenn-jocher]

Hello @Heaven0612,

Thank you for your thoughtful questions and for engaging in this discussion. Let's address each of your queries in detail:

1. Choice of BottleneckCSP vs. C3 Module

The choice between BottleneckCSP and C3 modules often depends on the specific requirements of the model and the task at hand. Both modules are designed to enhance feature extraction, but they have different structures and computational characteristics.

• BottleneckCSP: This module is designed to improve gradient flow and reduce computational complexity by splitting the input into two parts, processing one part through a series of bottleneck layers, and then concatenating it with the other part.
• C3 Module: The C3 module is a more recent addition that builds on the concept of CSPNet but with additional improvements in efficiency and performance.

In the context of your task, the choice of BottleneckCSP was made to balance complexity and performance, ensuring that the model can effectively capture small defects without excessive computational overhead.

2. Number of Modules in P3

The number of modules in each layer (e.g., P3) is empirically chosen based on extensive experimentation to achieve optimal performance. The original configuration might have 2 modules in the C3 block after P3, while the example provided has 3 modules in the BottleneckCSP block. This discrepancy can be attributed to the different design philosophies and performance optimizations for specific tasks.

3. FPN vs. PANet in the Neck

The choice between Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) in the neck part of the model depends on the specific requirements of the detection task:

• FPN: FPN is designed to enhance feature maps at different scales, making it suitable for detecting objects of various sizes. It is simpler and often sufficient for tasks where the object sizes are relatively uniform or when focusing on smaller objects.
• PANet: PANet further enhances the feature pyramid by adding bottom-up path augmentation, which helps in capturing more detailed features for larger objects. Since your task focuses on small defect detection, the FPN structure was chosen for simplicity and efficiency.

4. SPPF Module

The Spatial Pyramid Pooling - Fast (SPPF) module is designed to increase the receptive field without significantly increasing computational complexity. While it was omitted in the example provided, it can be beneficial in certain scenarios:

• Increased Receptive Field: SPPF can help in capturing larger context around small defects, which might be useful depending on the nature of the defects.
• Efficiency: The SPPF module is designed to be computationally efficient, making it a good candidate for inclusion if you find that the model benefits from a larger receptive field.

If you believe that the SPPF module could enhance your model's performance, you can certainly include it in your architecture. Here’s an example of how you might add it back:

# yolov5s.yaml
backbone:
  # [from, number, module, args]
  [[-1, 1, Conv, [64, 6, 2, 2]],  # 0-P1/2
   [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4
   [-1, 3, BottleneckCSP, [128]],  # 2
   [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
   [-1, 9, BottleneckCSP, [256]],  # 4
   [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
   [-1, 1, BottleneckCSP, [512]],  # 6
   [-1, 1, SPPF, [512]],  # 7 Adding SPPF module
  ]

head:
  [[-1, 1, Conv, [256, 1, 1]],  # 8
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 9
   [[-1, 4], 1, Concat, [1]],  # 10
   [-1, 3, BottleneckCSP, [256]],  # 11

   [-1, 1, Conv, [128, 1, 1]],  # 12
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 13
   [[-1, 2], 1, Concat, [1]],  # 14
   [-1, 3, BottleneckCSP, [128]],  # 15

   [[15, 11, 7], 1, Detect, [nc, anchors]],  # Detect(P3, P4)
  ]

Conclusion

I hope this clarifies your questions! Your approach to understanding and modifying the YOLOv5 architecture for small defect detection is impressive. If you have any further questions or need additional assistance, feel free to ask. The YOLO community and the Ultralytics team are here to support you. 😊

[Heaven0612]

@glenn-jocher
first
Thanks for explaining BottleneckCSP and C3

But regarding the calculation of the receptive field, whether it is BottleneckCSP or the C3 module, there is a piece called Bottleneck, which is composed of 11 Conv and 33 Conv s=1. We all know that the calculation of the receptive field is mainly affected by 3*3 The influence of kernel and stride

This is also the quantity issue and question I have mentioned many times about this module.

I will use your example to calculate the receptive field
#P1
k=6 s=2 p=2 #RF= 6 #_1

#P2
k=3 s=2 p=1 #RF= 10 #_2
k=3 s=1 p=1 #RF= 18 #_3 #BottleneckCSP

#P3
k=3 s=2 p=1 #RF= 26 #_4
k=3 s=1 p=1 #RF= 42 #_5 #BottleneckCSP
k=3 s=1 p=1 #RF= 58 #_6 #BottleneckCSP
k=3 s=1 p=1 #RF= 74 #_7 #BottleneckCSP

#P4
k=3 s=2 p=1 #RF= 90 #_8
k=3 s=2 p=1 #RF= 122 #_9 #BottleneckCSP

Whether it is the C3 module or the BottleneckCSP module, their RF calculations are all the same. This is due to the internal Bottleneck module.

The reason I ask about its quantity is because RF=58 (#_6) can cover the size of my defect. Since there are only 2 blocks in C3 in the original file, its perceived size is equivalent to the second BottleneckCSP (#_6) in P3. ), which is exactly what I need, so I would assume it stops there, but the rule of thumb you mentioned makes me wonder about this part of the adjustment.

In addition, I found that the part in the example you gave is 3 BottleneckCSP, but I hope that the output RF of the entire P4 layer in the final BottleneckCSP will be about 58*2. This is why I asked about BottleneckCSP or C3 module. How is the quantity configured?

In addition, regarding the SPPF mentioned, although Maxpool can increase the receptive field slightly, I think this module is more like selecting more obvious feature blocks. I don’t know if this idea is correct.

[glenn-jocher]

Hello @Heaven0612,

Thank you for your detailed follow-up and for sharing your insights on the receptive field calculations. Your understanding of the receptive field and its impact on defect detection is impressive. Let's address your questions and concerns in detail:

Receptive Field Calculation

You are correct that the receptive field (RF) is primarily influenced by the kernel size and stride of the convolutions. Your calculations for the receptive field at each stage are accurate and well-documented. The key points to consider are:

1. Receptive Field Coverage:

   • If your defect size can be effectively covered by an RF of 58 (as in your example with P3), then it makes sense to focus on this layer and ensure that the model can capture these details adequately.
   • The number of BottleneckCSP or C3 modules in each layer is empirically chosen to balance feature extraction and computational efficiency. However, you can adjust these based on your specific needs.

2. Adjusting the Number of Modules:

   • If you find that an RF of 58 is sufficient for your defect detection, you can reduce the number of BottleneckCSP or C3 modules in P4 to avoid increasing the RF unnecessarily.

   • For example, you can configure P4 to have fewer modules to keep the RF closer to your desired range. Here's how you might adjust the configuration:

     # yolov5s.yaml
     backbone:
       # [from, number, module, args]
       [[-1, 1, Conv, [64, 6, 2, 2]],  # 0-P1/2
        [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4
        [-1, 3, BottleneckCSP, [128]],  # 2
        [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
        [-1, 2, BottleneckCSP, [256]],  # 4 Reduced to 2 modules
        [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
        [-1, 1, BottleneckCSP, [512]],  # 6 Reduced to 1 module
        # Removed P5 layers
       ]
     
     head:
       [[-1, 1, Conv, [256, 1, 1]],  # 7
        [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 8
        [[-1, 4], 1, Concat, [1]],  # 9
        [-1, 3, BottleneckCSP, [256]],  # 10
     
        [-1, 1, Conv, [128, 1, 1]],  # 11
        [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 12
        [[-1, 2], 1, Concat, [1]],  # 13
        [-1, 3, BottleneckCSP, [128]],  # 14
     
        [[14, 10, 6], 1, Detect, [nc, anchors]],  # Detect(P3, P4)
       ]

SPPF Module

Your understanding of the SPPF module is correct. While it does increase the receptive field, it also helps in selecting more prominent feature blocks, which can be beneficial for certain tasks. Including the SPPF module can help in capturing more context around the defects, which might improve detection performance.

If you believe that the SPPF module could enhance your model's performance, you can include it as follows:

# yolov5s.yaml
backbone:
  # [from, number, module, args]
  [[-1, 1, Conv, [64, 6, 2, 2]],  # 0-P1/2
   [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4
   [-1, 3, BottleneckCSP, [128]],  # 2
   [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
   [-1, 2, BottleneckCSP, [256]],  # 4
   [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
   [-1, 1, BottleneckCSP, [512]],  # 6
   [-1, 1, SPPF, [512]],  # 7 Adding SPPF module
  ]

head:
  [[-1, 1, Conv, [256, 1, 1]],  # 8
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 9
   [[-1, 4], 1, Concat, [1]],  # 10
   [-1, 3, BottleneckCSP, [256]],  # 11

   [-1, 1, Conv, [128, 1, 1]],  # 12
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],  # 13
   [[-1, 2], 1, Concat, [1]],  # 14
   [-1, 3, BottleneckCSP, [128]],  # 15

   [[15, 11, 7], 1, Detect, [nc, anchors]],  # Detect(P3, P4)
  ]

Conclusion

Your approach to understanding and modifying the YOLOv5 architecture for small defect detection is commendable. By carefully adjusting the number of modules and considering the inclusion of the SPPF module, you can tailor the model to better suit your specific needs.

If you have any further questions or need additional assistance, feel free to ask. The YOLO community and the Ultralytics team are here to support you. 😊