Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add model ScaledYOLOv4 Support #34

Merged
merged 49 commits into from
Jul 25, 2022
Merged

Conversation

ziqi-jin
Copy link
Contributor

  • add model ScaledYOLOv4 Support
  • fixed the boundary problem in ScaledYOLOv4, YOLOv7 and YOLOR
  • optimized the normalization function in ScaledYOLOv4, YOLOv7 and YOLOR

ziqi-jin and others added 16 commits July 19, 2022 11:57
* Fix compile problem in different python version (#26)

* fix some usage problem in linux

* Fix compile problem

Co-authored-by: root <[email protected]>

* Add PaddleDetetion/PPYOLOE model support (#22)

* add ppdet/ppyoloe

* Add demo code and documents

* add convert processor to vision (#27)

* update .gitignore

* Added checking for cmake include dir

* fixed missing trt_backend option bug when init from trt

* remove un-need data layout and add pre-check for dtype

* changed RGB2BRG to BGR2RGB in ppcls model

* add model_zoo yolov6 c++/python demo

* fixed CMakeLists.txt typos

* update yolov6 cpp/README.md

* add yolox c++/pybind and model_zoo demo

* move some helpers to private

* fixed CMakeLists.txt typos

* add normalize with alpha and beta

* add version notes for yolov5/yolov6/yolox

* add copyright to yolov5.cc

* revert normalize

* fixed some bugs in yolox

* fixed examples/CMakeLists.txt to avoid conflicts

* add convert processor to vision

* format examples/CMakeLists summary

* Fix bug while the inference result is empty with YOLOv5 (#29)

* Add multi-label function for yolov5

* Update README.md

Update doc

* Update fastdeploy_runtime.cc

fix variable option.trt_max_shape wrong name

* Update runtime_option.md

Update resnet model dynamic shape setting name from images to x

* Fix bug when inference result boxes are empty

* Delete detection.py

Co-authored-by: Jason <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: DefTruth <[email protected]>
Co-authored-by: huangjianhui <[email protected]>
* Fix compile problem in different python version (#26)

* fix some usage problem in linux

* Fix compile problem

Co-authored-by: root <[email protected]>

* Add PaddleDetetion/PPYOLOE model support (#22)

* add ppdet/ppyoloe

* Add demo code and documents

* add convert processor to vision (#27)

* update .gitignore

* Added checking for cmake include dir

* fixed missing trt_backend option bug when init from trt

* remove un-need data layout and add pre-check for dtype

* changed RGB2BRG to BGR2RGB in ppcls model

* add model_zoo yolov6 c++/python demo

* fixed CMakeLists.txt typos

* update yolov6 cpp/README.md

* add yolox c++/pybind and model_zoo demo

* move some helpers to private

* fixed CMakeLists.txt typos

* add normalize with alpha and beta

* add version notes for yolov5/yolov6/yolox

* add copyright to yolov5.cc

* revert normalize

* fixed some bugs in yolox

* fixed examples/CMakeLists.txt to avoid conflicts

* add convert processor to vision

* format examples/CMakeLists summary

* Fix bug while the inference result is empty with YOLOv5 (#29)

* Add multi-label function for yolov5

* Update README.md

Update doc

* Update fastdeploy_runtime.cc

fix variable option.trt_max_shape wrong name

* Update runtime_option.md

Update resnet model dynamic shape setting name from images to x

* Fix bug when inference result boxes are empty

* Delete detection.py

Co-authored-by: Jason <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: DefTruth <[email protected]>
Co-authored-by: huangjianhui <[email protected]>
* Develop (#11) (#12)

* Fix compile problem in different python version (#26)

* fix some usage problem in linux

* Fix compile problem

Co-authored-by: root <[email protected]>

* Add PaddleDetetion/PPYOLOE model support (#22)

* add ppdet/ppyoloe

* Add demo code and documents

* add convert processor to vision (#27)

* update .gitignore

* Added checking for cmake include dir

* fixed missing trt_backend option bug when init from trt

* remove un-need data layout and add pre-check for dtype

* changed RGB2BRG to BGR2RGB in ppcls model

* add model_zoo yolov6 c++/python demo

* fixed CMakeLists.txt typos

* update yolov6 cpp/README.md

* add yolox c++/pybind and model_zoo demo

* move some helpers to private

* fixed CMakeLists.txt typos

* add normalize with alpha and beta

* add version notes for yolov5/yolov6/yolox

* add copyright to yolov5.cc

* revert normalize

* fixed some bugs in yolox

* fixed examples/CMakeLists.txt to avoid conflicts

* add convert processor to vision

* format examples/CMakeLists summary

* Fix bug while the inference result is empty with YOLOv5 (#29)

* Add multi-label function for yolov5

* Update README.md

Update doc

* Update fastdeploy_runtime.cc

fix variable option.trt_max_shape wrong name

* Update runtime_option.md

Update resnet model dynamic shape setting name from images to x

* Fix bug when inference result boxes are empty

* Delete detection.py

Co-authored-by: Jason <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: DefTruth <[email protected]>
Co-authored-by: huangjianhui <[email protected]>

Co-authored-by: Jason <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: DefTruth <[email protected]>
Co-authored-by: huangjianhui <[email protected]>

* Develop (#13)

* Fix compile problem in different python version (#26)

* fix some usage problem in linux

* Fix compile problem

Co-authored-by: root <[email protected]>

* Add PaddleDetetion/PPYOLOE model support (#22)

* add ppdet/ppyoloe

* Add demo code and documents

* add convert processor to vision (#27)

* update .gitignore

* Added checking for cmake include dir

* fixed missing trt_backend option bug when init from trt

* remove un-need data layout and add pre-check for dtype

* changed RGB2BRG to BGR2RGB in ppcls model

* add model_zoo yolov6 c++/python demo

* fixed CMakeLists.txt typos

* update yolov6 cpp/README.md

* add yolox c++/pybind and model_zoo demo

* move some helpers to private

* fixed CMakeLists.txt typos

* add normalize with alpha and beta

* add version notes for yolov5/yolov6/yolox

* add copyright to yolov5.cc

* revert normalize

* fixed some bugs in yolox

* fixed examples/CMakeLists.txt to avoid conflicts

* add convert processor to vision

* format examples/CMakeLists summary

* Fix bug while the inference result is empty with YOLOv5 (#29)

* Add multi-label function for yolov5

* Update README.md

Update doc

* Update fastdeploy_runtime.cc

fix variable option.trt_max_shape wrong name

* Update runtime_option.md

Update resnet model dynamic shape setting name from images to x

* Fix bug when inference result boxes are empty

* Delete detection.py

Co-authored-by: Jason <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: DefTruth <[email protected]>
Co-authored-by: huangjianhui <[email protected]>

* documents

* documents

* documents

* documents

* documents

* documents

* documents

* documents

* documents

* documents

* documents

* documents

* Develop (#14)

* Fix compile problem in different python version (#26)

* fix some usage problem in linux

* Fix compile problem

Co-authored-by: root <[email protected]>

* Add PaddleDetetion/PPYOLOE model support (#22)

* add ppdet/ppyoloe

* Add demo code and documents

* add convert processor to vision (#27)

* update .gitignore

* Added checking for cmake include dir

* fixed missing trt_backend option bug when init from trt

* remove un-need data layout and add pre-check for dtype

* changed RGB2BRG to BGR2RGB in ppcls model

* add model_zoo yolov6 c++/python demo

* fixed CMakeLists.txt typos

* update yolov6 cpp/README.md

* add yolox c++/pybind and model_zoo demo

* move some helpers to private

* fixed CMakeLists.txt typos

* add normalize with alpha and beta

* add version notes for yolov5/yolov6/yolox

* add copyright to yolov5.cc

* revert normalize

* fixed some bugs in yolox

* fixed examples/CMakeLists.txt to avoid conflicts

* add convert processor to vision

* format examples/CMakeLists summary

* Fix bug while the inference result is empty with YOLOv5 (#29)

* Add multi-label function for yolov5

* Update README.md

Update doc

* Update fastdeploy_runtime.cc

fix variable option.trt_max_shape wrong name

* Update runtime_option.md

Update resnet model dynamic shape setting name from images to x

* Fix bug when inference result boxes are empty

* Delete detection.py

Co-authored-by: root <[email protected]>
Co-authored-by: DefTruth <[email protected]>
Co-authored-by: huangjianhui <[email protected]>

Co-authored-by: Jason <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: DefTruth <[email protected]>
Co-authored-by: huangjianhui <[email protected]>
Co-authored-by: Jason <[email protected]>
Copy link
Collaborator

@DefTruth DefTruth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fastdeploy/vision/wongkinyiu/scaledyolov4.cc 中 LetterBox 处理

Resize::Run(mat, resize_w, resize_h);

在resize前加一个判断,只在宽高不相同时进行resize

if ((mat->Height() != resize_h) || (mat->Width() != resize_w)) {
  Resize::Run(mat, resize_w, resize_h);
}

其他几个模型yolor、yolov7对应的地方也修改一下。因为在预处理中已经先做了一次resize,这次resize其实已经resize到目标维度size了,这里的mat的维度应该是和目标size一致的,所以LetterBox里面的这个resize可以做一次维度判断,避免多做一次冗余的resize

Copy link
Collaborator

@DefTruth DefTruth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fastdeploy/vision/wongkinyiu/scaledyolov4.cc 中 Preprocess 处理

double ratio = (size[0] * 1.0) / std::max(static_cast<float>(mat->Height()),
                                          static_cast<float>(mat->Width()));

这里只用了size[0],也就是只支持方形输入,不支持矩形输入,但是我们无法判断用户是不是只用方形输入,比如[640,1280]、[320,640]等这种推理的输入size也是很常见的。所以这里最好修改成既支持方形也支持矩形。比如:

float ratio = std::min(size[1] * 1.0f / static_cast<float>(mat->Height()), 
                       size[0] * 1.0f / static_cast<float>(mat->Width()));

其他几个模型yolor、yolov7对应的地方也修改一下。

Copy link
Collaborator

@DefTruth DefTruth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

在 fastdeploy/vision/wongkinyiu/scaledyolov4.cc 中 Postprocess 的 pad处理逻辑有遗漏:

float scale = std::min(out_h / ipt_h, out_w / ipt_w);
for (size_t i = 0; i < result->boxes.size(); ++i) {
    float pad_h = (out_h - ipt_h * scale) / 2;
    float pad_w = (out_w - ipt_w * scale) / 2;
    // ...
}

这里的pad计算逻辑没有完全和LetterBox中的逻辑对上,因为LetterBox中有2中pad逻辑,一种是is_mini_pad=false(默认的,也就是_auto=false),另一种是is_mini_pad=true(也就是_auto=true),这两种模式下pad的计算方式是不一样的,所以在decode的时候也要做不一样的处理,不能只做is_mini_pad=false的处理。可参考以下逻辑(可以把pad计算提到循环外面):

float scale = std::min(out_h / ipt_h, out_w / ipt_w);
float pad_h = (out_h - ipt_h * scale) / 2.0f;
float pad_w = (out_w - ipt_w * scale) / 2.0f;
if (is_mini_pad) {
  // 和 LetterBox中_auto=true的处理逻辑对应 
  pad_h = static_cast<float>(static_cast<int>(pad_h) % stride);
  pad_w = static_cast<float>(static_cast<int>(pad_w) % stride);
}
for (size_t i = 0; i < result->boxes.size(); ++i) {
  // ... decode 逻辑
}

yolor、yolov7中的相关处理也可以修改一下。

Copy link
Collaborator

@DefTruth DefTruth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

在 model_zoo/vision/scaledyolov4/api.md 中 函数签名中的参数没有对齐

Predict函数

ScaledYOLOv4::Predict(cv::Mat* im, DetectionResult* result,
                float conf_threshold = 0.25,
                float nms_iou_threshold = 0.5)

建议对齐一下

@ziqi-jin

This comment was marked as duplicate.

@ziqi-jin ziqi-jin closed this Jul 24, 2022
@ziqi-jin ziqi-jin reopened this Jul 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants