About image resize problem

[brian208579]

Hi @jkjung-avt
I read the issue about image resize : AlexeyAB/darknet#232
I realize it will affect performance ( accuracy )
So I check the _preprocess_yolov3 function in yolov3.py

def _preprocess_yolov3(img, shape):
    """Preprocess an image before TRT YOLOv3 inferencing."""
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    img = cv2.resize(img, shape)
    img = img.transpose((2, 0, 1)).astype(np.float32)
    img = img / 255.0
    return np.ascontiguousarray(img)

In my perception, cv2.resize is without keeping aspect ratio and same as AlexeyAB version
But I'm not sure ...
Can you give me some suggestion and confirm with me ?
Appreciat ~

[jkjung-avt]

Indeed, cv2.resize() does not keep aspect ratio.

Normally, object detection models are trained with random aspect-ratioed crops of the original images. That is, we expect the trained object detection model capable of handling aspect-ratio distortions up to certain range.

For example, check out source code of tensorflow object detection API: https://github.com/tensorflow/models/blob/master/research/object_detection/protos/preprocessor.proto#L153

By default, the tensorflow ssd or faster_rcnn models are all trained by input image crops with random aspect ratios in the range of 0.75~1.33. That's why, at inference time, we usually don't preserve aspect ratio and just resize the input images directly.

I haven't studied whether the YOLOv3 model is trained with varying aspect ratios, though. If you do find presevring aspect ratio is helpful (producing better mAP), feel free to modify the preprocessing and postprocessing code to handle that.

[brian208579]

@jkjung-avt Thanks your suggestion

In pjreddie's darknet
Resized image with keeping aspect ratio
In AlexeyAB's darknet
Resized image without keeping aspect ratio

For me, AlexeyAB's darknet has better performance ( mAP )
So I should not need to modify the code 👍

But I have another question
I want to create a yolov3 with only two scale yolo layers ( remove last yolo layer )
Do I need to modify the yolov3_to_onnx.py or onnx_to_tensorrt.py ?
Could you provide me some information ( I am a beginner of tensorRT ) ?

Appreciat !!!

[jkjung-avt]

For a modified YOLOv3 network with only 2 yolo layers, I think you'd need to:

• Remove the output tensor (for the yolo layer which you have removed) in yolov3_to_onnx.py: https://github.com/jkjung-avt/tensorrt_demos/blob/master/yolov3_onnx/yolov3_to_onnx.py#L795
• No need to modify onnx_to_tensorrt.py.
• Modify output shapes, yolo_masks and yolo_anchors in yolov3.py:
  • https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/yolov3.py#L410
  • https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/yolov3.py#L428
  • https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/yolov3.py#L430

[brian208579]

@jkjung-avt
ok , I will try it , thank you a lot !!!