Intelligent scissors with OpenCV javascript #2689

bsekachev · 2021-01-20T10:58:17Z

Motivation and context

Integration of OpenCV.js library into CVAT. Now Intelligent Scissors have been added (resolved #336), but we expect more functions in the future.

Threshold maintenance using ctrl + mousewheel

How has this been tested?

Manual testing

Checklist

I submit my changes into the develop branch
I have added description of my changes into CHANGELOG file
I have updated the documentation accordingly
I have added tests to cover my changes
I have linked related issues (read github docs)
I have increased versions of npm packages if it is necessary (cvat-canvas,
cvat-core, cvat-data and cvat-ui)

License

I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
I have updated the license header for each file (see an example below)

cvat/apps/opencv/views.py

nmanovic · 2021-01-28T11:34:48Z

Great!
/cc @dkurt @alvoron @SimpleVlad

cvat/apps/opencv/static/opencv/js/opencv.js

It would be nice to store somewhere some instructions how to update this file in the future

@alalek , do you think we can place information about the integration somewhere on OpenCV site? For example, tutorial or demo page? Let's say a statement like "the feature is integrated into CVAT annotation tool and you can try it online on cvat.org".

nmanovic · 2021-01-28T11:39:30Z

cvat-ui/src/utils/opencv-wrapper/opencv-wrapper.ts

+    }
+
+    public async initialize(onProgress: (percent: number) => void): Promise<void> {
+        const response = await fetch(`${baseURL}/opencv/opencv.js`);


Should we consider it as server API and add into server-proxy?

nmanovic · 2021-01-28T18:01:11Z

@bsekachev , build bs/opencv and got the following error:

nmanovic · 2021-01-28T18:16:04Z

commit b20f5028dfa74b8e63bc7a4180aa064e5ed4dda7 (HEAD -> bs/opencv, origin/bs/opencv)
Author: Boris Sekachev <[email protected]>
Date:   Thu Jan 28 12:22:59 2021 +0300

    Fixed corner case

ActiveChooN · 2021-01-29T04:46:05Z

Most works really fine, but have some thoughts.

If we have no script on related path, click on load button will be completed with success icon.
Threshold can be configurated with consts or state for larger working area for example.
Sometimes i've got an error message when move pointer fast enough. It quite rare and i connot record this case.
Guess we can improve interactivity by starting interactor if only previous interaction has finished. Without it you can only achieve interaction result when mouse pointer stops moving.

bsekachev · 2021-02-01T12:26:14Z

@bsekachev , build bs/opencv and got the following error:

@nmanovic Forgot to setup server & proxy, fixed.

bsekachev · 2021-02-01T12:30:26Z

@ActiveChooN

If we have no script on related path, click on load button will be completed with success icon.

Fixed.

Threshold can be configurated with consts or state for larger working area for example.

Ideally it should be configurated in UI, but it would be difficult to unify UI for all possible thresholds with different tools from OpenCV. Now we also do not have thresholds for automatic annotation, maybe it will be done in the future.

Guess we can improve interactivity by starting interactor if only previous interaction has finished. Without it you can only achieve interaction result when mouse pointer stops moving.

I am not sure I understand you correct

nmanovic · 2021-02-03T11:55:07Z

@bsekachev , an alternative way to implement the feature https://www.photoshopessentials.com/basics/selections/magnetic-lasso-tool/. What do you think?

Looks like it is a polygon feature with "shift" pressed but you don't need move the mouse along the boundary precisely.

nmanovic

Server part looks good for me. I think we can merge it as is for now.

bsekachev · 2021-02-03T16:29:55Z

Okay, let's merge it as an initial version and look how we can improve it in the future patches.

jahaniam · 2021-02-12T20:50:19Z

@bsekachev @nmanovic Hi guys,
I've tested this feature. It looks great although it mostly works well when there is a clear boundary between background and foreground.

I have these two suggestions:

removing points is done using ctrl + clicking. It is not consistent with normal polygon drawing which is right-clicking. It would have been nicer to be consistent.
It would have been great to automatically draw a point when it is at the end of the rectangle (like how magnetic lasso tool does in photoshop. or how it is done in current cvat when holding shift.)

Thanks again for your work.

bsekachev · 2021-02-14T12:53:59Z

@jahaniam

Hi, thanks for your feedback. Please, address your comment to dedicated issue #2800

bsekachev added 23 commits October 14, 2020 13:40

Some UI implementations

b1ac4b1

Added opencv wrapper

b553502

Updated Opencv wrapper

a058c02

Moved initialization stub

28101b4

Added threshold

5c46aaa

Setup interaction with canvas

45c7f96

Fixed couple of issues

516d666

Added threshold, changing size via ctrl

9c83aed

tmp

19d3ce4

Merged develop

7974c6a

Aborted host change

6605cb2

Fixed threshold

fd7a767

Merge branch 'develop' into bs/opencv

0946e53

Aborted host

29e6121

Merge branch 'develop' into bs/opencv

5bcb8b4

Merge branch 'develop' into bs/opencv

9b5d7d5

Merged develop

c6a99b6

Some fixes

9c4f80b

Using ready label selector

be77510

Raw implementation

e296046

Merge branch 'develop' into bs/opencv

f747197

Added additional arguments

afebd50

Fixed some minor issues

9d9c5db

bsekachev added the need test label Jan 20, 2021

bsekachev requested review from vnishukov, ActiveChooN and nmanovic January 20, 2021 10:58

bsekachev requested a review from azhavoro as a code owner January 20, 2021 10:58

nmanovic reviewed Jan 28, 2021

View reviewed changes

cvat/apps/opencv/views.py Show resolved Hide resolved

nmanovic reviewed Jan 28, 2021

View reviewed changes

alalek mentioned this pull request Jan 29, 2021

js(tutorial): add note about CVAT integration opencv/opencv#19422

Merged

bsekachev mentioned this pull request Feb 1, 2021

Adding Magic Wand increases annotation speed in case of not having pre-trained models (enhancement) #2380

Open

2 tasks

Boris Sekachev added 2 commits February 1, 2021 12:36

Merge branch 'develop' into bs/opencv

3ced2b4

Added error handler, opencv version, updated cvat_proxy & mod_wsgi

5fcf469

Boris Sekachev added 3 commits February 1, 2021 15:34

OpenCV returned back

d9eadc1

Using dinamic function instead of script

feb8ad6

Merge branch 'develop' into bs/opencv

53fe4b5

nmanovic previously approved these changes Feb 3, 2021

View reviewed changes

Boris Sekachev added 2 commits February 3, 2021 19:33

Updated changelog & version

2a5e91b

Merge branch 'develop' into bs/opencv

f9c0a34

bsekachev dismissed nmanovic’s stale review via f9c0a34 February 3, 2021 16:35

bsekachev mentioned this pull request Feb 3, 2021

Intelligent Scissors add documentation #2765

Closed

2 tasks

bsekachev merged commit e0fc323 into develop Feb 3, 2021

bsekachev deleted the bs/opencv branch February 9, 2021 07:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Intelligent scissors with OpenCV javascript #2689

Intelligent scissors with OpenCV javascript #2689

bsekachev commented Jan 20, 2021 •

edited

Loading

nmanovic commented Jan 28, 2021

nmanovic Jan 28, 2021

nmanovic Feb 3, 2021

nmanovic commented Jan 28, 2021

nmanovic commented Jan 28, 2021

ActiveChooN commented Jan 29, 2021

bsekachev commented Feb 1, 2021

bsekachev commented Feb 1, 2021

nmanovic commented Feb 3, 2021

nmanovic left a comment

bsekachev commented Feb 3, 2021

jahaniam commented Feb 12, 2021 •

edited

Loading

bsekachev commented Feb 14, 2021

Intelligent scissors with OpenCV javascript #2689

Intelligent scissors with OpenCV javascript #2689

Conversation

bsekachev commented Jan 20, 2021 • edited Loading

Motivation and context

How has this been tested?

Checklist

License

nmanovic commented Jan 28, 2021

nmanovic Jan 28, 2021

Choose a reason for hiding this comment

nmanovic Feb 3, 2021

Choose a reason for hiding this comment

nmanovic commented Jan 28, 2021

nmanovic commented Jan 28, 2021

ActiveChooN commented Jan 29, 2021

bsekachev commented Feb 1, 2021

bsekachev commented Feb 1, 2021

nmanovic commented Feb 3, 2021

nmanovic left a comment

Choose a reason for hiding this comment

bsekachev commented Feb 3, 2021

jahaniam commented Feb 12, 2021 • edited Loading

bsekachev commented Feb 14, 2021

bsekachev commented Jan 20, 2021 •

edited

Loading

jahaniam commented Feb 12, 2021 •

edited

Loading