-
Notifications
You must be signed in to change notification settings - Fork 465
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add model ScaledYOLOv4 Support #34
Conversation
ziqi-jin
commented
Jul 22, 2022
- add model ScaledYOLOv4 Support
- fixed the boundary problem in ScaledYOLOv4, YOLOv7 and YOLOR
- optimized the normalization function in ScaledYOLOv4, YOLOv7 and YOLOR
* Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <[email protected]> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]>
* Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <[email protected]> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]>
* Develop (#11) (#12) * Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <[email protected]> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> * Develop (#13) * Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <[email protected]> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * Develop (#14) * Fix compile problem in different python version (#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <[email protected]> * Add PaddleDetetion/PPYOLOE model support (#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> Co-authored-by: Jason <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fastdeploy/vision/wongkinyiu/scaledyolov4.cc 中 LetterBox 处理
Resize::Run(mat, resize_w, resize_h);
在resize前加一个判断,只在宽高不相同时进行resize
if ((mat->Height() != resize_h) || (mat->Width() != resize_w)) {
Resize::Run(mat, resize_w, resize_h);
}
其他几个模型yolor、yolov7对应的地方也修改一下。因为在预处理中已经先做了一次resize,这次resize其实已经resize到目标维度size了,这里的mat的维度应该是和目标size一致的,所以LetterBox里面的这个resize可以做一次维度判断,避免多做一次冗余的resize
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fastdeploy/vision/wongkinyiu/scaledyolov4.cc 中 Preprocess 处理
double ratio = (size[0] * 1.0) / std::max(static_cast<float>(mat->Height()),
static_cast<float>(mat->Width()));
这里只用了size[0],也就是只支持方形输入,不支持矩形输入,但是我们无法判断用户是不是只用方形输入,比如[640,1280]、[320,640]等这种推理的输入size也是很常见的。所以这里最好修改成既支持方形也支持矩形。比如:
float ratio = std::min(size[1] * 1.0f / static_cast<float>(mat->Height()),
size[0] * 1.0f / static_cast<float>(mat->Width()));
其他几个模型yolor、yolov7对应的地方也修改一下。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
在 fastdeploy/vision/wongkinyiu/scaledyolov4.cc 中 Postprocess 的 pad处理逻辑有遗漏:
float scale = std::min(out_h / ipt_h, out_w / ipt_w);
for (size_t i = 0; i < result->boxes.size(); ++i) {
float pad_h = (out_h - ipt_h * scale) / 2;
float pad_w = (out_w - ipt_w * scale) / 2;
// ...
}
这里的pad计算逻辑没有完全和LetterBox中的逻辑对上,因为LetterBox中有2中pad逻辑,一种是is_mini_pad=false(默认的,也就是_auto=false),另一种是is_mini_pad=true(也就是_auto=true),这两种模式下pad的计算方式是不一样的,所以在decode的时候也要做不一样的处理,不能只做is_mini_pad=false的处理。可参考以下逻辑(可以把pad计算提到循环外面):
float scale = std::min(out_h / ipt_h, out_w / ipt_w);
float pad_h = (out_h - ipt_h * scale) / 2.0f;
float pad_w = (out_w - ipt_w * scale) / 2.0f;
if (is_mini_pad) {
// 和 LetterBox中_auto=true的处理逻辑对应
pad_h = static_cast<float>(static_cast<int>(pad_h) % stride);
pad_w = static_cast<float>(static_cast<int>(pad_w) % stride);
}
for (size_t i = 0; i < result->boxes.size(); ++i) {
// ... decode 逻辑
}
yolor、yolov7中的相关处理也可以修改一下。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
在 model_zoo/vision/scaledyolov4/api.md 中 函数签名中的参数没有对齐
Predict函数
ScaledYOLOv4::Predict(cv::Mat* im, DetectionResult* result, float conf_threshold = 0.25, float nms_iou_threshold = 0.5)
建议对齐一下