Text Detection: add ppocr-v2 detect -WIP #66

the-star-sea · 2022-06-29T03:06:14Z

add demo
add example in readme

zihaomu

Thanks for your contribution!
Newly added models do not need the old copyright.

models/text_detection_ppdetect/demo.py

models/text_detection_ppdetect/ppdetect.py

zihaomu · 2022-07-20T05:09:16Z

Hi @the-star-sea. Since in OpenCV the DB has been supported by High-Level API, can you provide more speed and accuracy test data for DB and proposed ppocr?

the-star-sea · 2022-07-20T07:53:03Z

Hi @the-star-sea. Since in OpenCV the DB has been supported by High-Level API, can you provide more speed and accuracy test data for DB and proposed ppocr?

ok.I will do it

the-star-sea · 2022-07-24T03:47:47Z

zihaomu · 2022-07-25T03:22:26Z

@the-star-sea the ppocr's speed is super fast.
two questions:

Are the pre-process and the post-process of ppocr's the same as the DB net?
Is there accuracy test result?

the-star-sea · 2022-07-25T04:41:46Z

@the-star-sea the ppocr's speed is super fast. two questions:

Are the pre-process and the post-process of ppocr's the same as the DB net?

Is there accuracy test result?

1.almost the same
2.

zihaomu · 2022-07-25T05:01:48Z

Thank you @the-star-sea! The accuracy result is good. Which validation data are used? How many data it has?

the-star-sea · 2022-07-25T05:32:41Z

icdar2015.500 imgs.

zihaomu · 2022-07-25T05:46:20Z

Thanks for the clarification. The DB model has hidded the post-process in the high level api, some thing like the following:

model = cv.dnn_TextDetectionModel_DB(
            cv.dnn.readNet(self._modelPath)
        )

# time start
model.detect(image)
# time stop

In order to get a fair speed comparison result, how did you test the speed of ppocr?

the-star-sea · 2022-07-25T07:48:12Z

I write the config file in benchmark and add function support in ppdetect.It just tests the speed of infering onnx model.
Besides.the onnx filesize of ppdetect is 2284 while DB is 47628.

zihaomu · 2022-07-25T08:06:04Z

47628

That's a great answer. Thanks!
Can we re-use the High-Level API of DB for PPocr with no change or little change?

model = cv.dnn_TextDetectionModel_DB(
            cv.dnn.readNet(modelPath_ppocr)
        )
model.detect(image)

the-star-sea · 2022-07-26T08:04:08Z

I think there need some changes because ppocr doesnot need polygon threhold.I will do it.

zihaomu · 2022-08-30T02:15:09Z

@the-star-sea Please keep one in the mobile and normal models, as we discussed before, to avoid confusing users.

the-star-sea · 2022-08-31T06:45:24Z

@the-star-sea Please keep one in the mobile and normal models, as we discussed before, to avoid confusing users.

ok.I will do it

zihaomu

LGTM!

zihaomu · 2022-09-05T05:40:21Z

models/text_detection_ppdetect/README.md

+
+Real-time Scene Text Detection with Differentiable Binarization
+
+This model is ported from [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR).


How about adding the original model link here?

fengyuentau

I suggest we remove DB from opencv_zoo after this pull request is merged, make a new high level api specifically for this model in opencv and update the wrapper class.

models/__init__.py

fengyuentau · 2022-09-05T05:57:02Z

models/__init__.py

@@ -37,3 +37,4 @@ def register(self, item):
 MODELS.register(MobileNetV2)
 MODELS.register(MPPalmDet)
 MODELS.register(LPD_YuNet)
+MODELS.register(PPDetect)


EOL before EOF

zihaomu · 2022-09-05T07:23:11Z

I suggest we remove DB from opencv_zoo after this pull request is merged.

Hi @fengyuentau, how about leaving it to the next PR to remove DB model, because we have a lot of speed tests based on the DB model. In addition, the high-level API based ppocr model has not been completed and is expected to be completed by the end of September.

fengyuentau · 2022-09-06T03:31:04Z

Sure, thats exactly what I meant. Just want to emphasize this plan.

fengyuentau · 2022-09-08T01:57:33Z

@zihaomu Please update benchmark results on this model.

zihaomu · 2022-09-08T02:02:26Z

@zihaomu Please update benchmark results on this model.

Hi, @fengyuentau the benchmark results will be updated at PR #73, the student will complete it in the near future.

fengyuentau · 2022-09-08T02:05:21Z

When we add or update a model in opencv zoo, we always update the average forward latency in the same pull request.

zihaomu · 2022-09-08T02:11:03Z

Ok, let's wait.

fengyuentau · 2022-09-08T02:16:57Z

We dont need to wait for #73 as it is for accuracy. At least we need the speed of this model to merge a pull request of adding or updating a model.

zihaomu · 2022-10-13T00:49:17Z

After this PR is merged, PP-OCR_v3 can be supported and loaded with high-level API. And I think we can only put PP-OCR_v3 in opencv_zoo since it has better accuracy.

asmorkalov · 2022-09-29T11:11:59Z

models/text_detection_ppdetect/demo.py

+    help_msg_backends += "; {:d}: TIMVX"
+    help_msg_targets += "; {:d}: NPU"
+except:
+    print('This version of OpenCV does not support TIM-VX and NPU. Visit https://gist.github.com/fengyuentau/5a7a5ba36328f2b763aea026c43fa45f for more information.')


It looks very strange that sample proposes to use external gist, bug not official documentation or wiki. I propose to use https://github.com/opencv/opencv/wiki/TIM-VX-Backend-For-Running-OpenCV-On-NPU. @fengyuentau Please extend wiki page if something is missing there.

Thanks for the review. IIRC, this link was added before we have the wiki in opencv. Will update this in a separate pull request.

fengyuentau · 2022-10-21T07:11:01Z

@zihaomu Anything else is blocking this PR from merge?

Update: we need to merge with benchmark results. Could you run benchmarking with this model? @zihaomu

zihaomu · 2022-10-21T07:29:05Z

@zihaomu Anything else is blocking this PR from merge?

Update: we need to merge with benchmark results. Could you run benchmarking with this model? @zihaomu

This PR should be closed after the OpenCV support new DB API, since new DB API support pp-ocr V3 and V2.

fengyuentau · 2022-10-22T02:33:20Z

This PR should be closed

Didn't we agree on adding this model (pp-ocr) in place of db? Why are we closing this PR?

zihaomu · 2022-10-25T07:56:51Z

We can directly support ppocr-DB v2 and v3 at new API.

add ppocr-v2 detect

b70acd5

zihaomu self-assigned this Jun 29, 2022

zihaomu added the GSoC Google Summer of Code projected related label Jun 29, 2022

zihaomu changed the title ~~add ppocr-v2 detect~~ add ppocr-v2 detect -WIP Jun 29, 2022

fengyuentau added the add model request to add a new model label Jul 13, 2022

zihaomu reviewed Jul 14, 2022

View reviewed changes

models/text_detection_ppdetect/demo.py Outdated Show resolved Hide resolved

models/text_detection_ppdetect/ppdetect.py Outdated Show resolved Hide resolved

delete the old copyright.

1645d98

zihaomu changed the title ~~add ppocr-v2 detect -WIP~~ OCR Detection: add ppocr-v2 detect -WIP Jul 20, 2022

zihaomu changed the title ~~OCR Detection: add ppocr-v2 detect -WIP~~ Text Detection: add ppocr-v2 detect -WIP Jul 20, 2022

add pp_detect benchmark

630f506

forget upload ppdetect

391a71f

the-star-sea added 2 commits August 5, 2022 10:48

add new ppdetect model and float16

2ce8e76

add new ppdetect model and float16

07c98fd

remove mobile model which is similar to normal version

a1833c6

zihaomu approved these changes Sep 5, 2022

View reviewed changes

fengyuentau reviewed Sep 5, 2022

View reviewed changes

modify init.py

e391930

zihaomu requested a review from fengyuentau September 8, 2022 00:29

the-star-sea mentioned this pull request Sep 12, 2022

[GSOC 2022] Better DB model support based on old API opencv/opencv#22500

Closed

6 tasks

asmorkalov reviewed Oct 13, 2022

View reviewed changes

zihaomu closed this Oct 25, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Text Detection: add ppocr-v2 detect -WIP #66

Text Detection: add ppocr-v2 detect -WIP #66

the-star-sea commented Jun 29, 2022

zihaomu left a comment

zihaomu commented Jul 20, 2022

the-star-sea commented Jul 20, 2022

the-star-sea commented Jul 24, 2022

zihaomu commented Jul 25, 2022 •

edited

Loading

the-star-sea commented Jul 25, 2022

zihaomu commented Jul 25, 2022

the-star-sea commented Jul 25, 2022

zihaomu commented Jul 25, 2022

the-star-sea commented Jul 25, 2022

zihaomu commented Jul 25, 2022

the-star-sea commented Jul 26, 2022

zihaomu commented Aug 30, 2022

the-star-sea commented Aug 31, 2022

zihaomu left a comment

zihaomu Sep 5, 2022

fengyuentau left a comment

fengyuentau Sep 5, 2022

zihaomu commented Sep 5, 2022

fengyuentau commented Sep 6, 2022

fengyuentau commented Sep 8, 2022

zihaomu commented Sep 8, 2022

fengyuentau commented Sep 8, 2022

zihaomu commented Sep 8, 2022

fengyuentau commented Sep 8, 2022

zihaomu commented Oct 13, 2022

asmorkalov Sep 29, 2022

fengyuentau Oct 13, 2022

fengyuentau commented Oct 21, 2022 •

edited

Loading

zihaomu commented Oct 21, 2022

fengyuentau commented Oct 22, 2022

zihaomu commented Oct 25, 2022


		Real-time Scene Text Detection with Differentiable Binarization

		This model is ported from [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR).

Text Detection: add ppocr-v2 detect -WIP #66

Text Detection: add ppocr-v2 detect -WIP #66

Conversation

the-star-sea commented Jun 29, 2022

zihaomu left a comment

Choose a reason for hiding this comment

zihaomu commented Jul 20, 2022

the-star-sea commented Jul 20, 2022

the-star-sea commented Jul 24, 2022

zihaomu commented Jul 25, 2022 • edited Loading

the-star-sea commented Jul 25, 2022

zihaomu commented Jul 25, 2022

the-star-sea commented Jul 25, 2022

zihaomu commented Jul 25, 2022

the-star-sea commented Jul 25, 2022

zihaomu commented Jul 25, 2022

the-star-sea commented Jul 26, 2022

zihaomu commented Aug 30, 2022

the-star-sea commented Aug 31, 2022

zihaomu left a comment

Choose a reason for hiding this comment

zihaomu Sep 5, 2022

Choose a reason for hiding this comment

fengyuentau left a comment

Choose a reason for hiding this comment

fengyuentau Sep 5, 2022

Choose a reason for hiding this comment

zihaomu commented Sep 5, 2022

fengyuentau commented Sep 6, 2022

fengyuentau commented Sep 8, 2022

zihaomu commented Sep 8, 2022

fengyuentau commented Sep 8, 2022

zihaomu commented Sep 8, 2022

fengyuentau commented Sep 8, 2022

zihaomu commented Oct 13, 2022

asmorkalov Sep 29, 2022

Choose a reason for hiding this comment

fengyuentau Oct 13, 2022

Choose a reason for hiding this comment

fengyuentau commented Oct 21, 2022 • edited Loading

zihaomu commented Oct 21, 2022

fengyuentau commented Oct 22, 2022

zihaomu commented Oct 25, 2022

zihaomu commented Jul 25, 2022 •

edited

Loading

fengyuentau commented Oct 21, 2022 •

edited

Loading