feat: UI for tracking #1884

savan77 · 2020-07-10T18:30:36Z

This PR adds UI for tracking.

Motivation and context

The older PR for tracking was using older UI and wasn't compatible with new UI. This PR ported UI to the new UI. This is similar to what we had earlier. The only change is for Stop frame (Track Until) we are using a specific frame number rather than the Next Keyframe, etc since it is easier for users to say "track until next 50 or so frames". But its still there in the settings UI in case you want to use it.

Below are some screenshots.

How has this been tested?

It has been tested locally as well as on Onepanel platform.

Checklist

[x ] I submit my changes into the develop branch
I have added description of my changes into CHANGELOG file
I have updated the documentation accordingly
I have added tests to cover my changes
I have linked related issues (read github docs)
I have increased versions of npm packages if it is necessary (cvat-canvas,
cvat-core, cvat-data and cvat-ui)
I will update the documentation once it has been accepted.

License

I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
I have updated the license header for each file (see an example below)

# Copyright (C) 2020 Intel Corporation
#
# SPDX-License-Identifier: MIT

CC: @rushtehrani @nmanovic @TheNaman047

coveralls · 2020-07-10T18:54:21Z

Pull Request Test Coverage Report for Build 6519

0 of 0 changed or added relevant lines in 0 files are covered.
742 unchanged lines in 29 files lost coverage.
Overall coverage decreased (-0.2%) to 64.988%

Files with Coverage Reduction	New Missed Lines	%
cvat/apps/dataset_manager/formats/mask.py	1	89.29%
cvat/apps/dataset_manager/formats/mot.py	1	94.74%
datumaro/datumaro/plugins/datumaro_format/converter.py	2	98.16%
datumaro/datumaro/plugins/tf_detection_api_format/extractor.py	2	88.68%
datumaro/datumaro/plugins/coco_format/importer.py	3	79.49%
cvat/apps/dataset_manager/formats/pascal_voc.py	4	60.0%
datumaro/datumaro/cli/util/project.py	4	26.92%
datumaro/datumaro/plugins/yolo_format/converter.py	4	89.72%
datumaro/datumaro/util/init.py	4	80.85%
datumaro/datumaro/components/converter.py	5	88.46%

Totals
Change from base Build 6382:	-0.2%
Covered Lines:	11065
Relevant Lines:	16620

💛 - Coveralls

nmanovic · 2020-07-10T19:26:18Z

@bsekachev , could you please look at the PR? Probably you want to merge it with minimal changes from @savan77 and after that polish on our side if it is necessary.

bsekachev · 2020-07-11T06:36:42Z

@nmanovic Sure, will do it.

cvat-ui/src/actions/annotation-actions.ts

bsekachev · 2020-07-13T12:50:53Z

...rc/components/annotation-page/standard-workspace/controls-side-bar/track-popover-control.tsx

+const trackTypeOptions = [
+    {
+        label: "OpenCV Boosting",
+        value: "Boosting",
+    },
+    {
+        label: "OpenCV MIL",
+        value: "MIL",
+    },
+    {
+        label: "OpenCV KCF",
+        value: "KCF",
+    },
+    {
+        label: "OpenCV CSRT",
+        value: "CSRT",
+    },
+    {
+        label: "OpenCV MedianFlow",
+        value: "MEDIANFLOW",
+    },
+    {
+        label: "OpenCV TLD",
+        value: "TLD",
+    },
+    {
+        label: "OpenCV MOSSE",
+        value: "MOSSE",
+    },
+    {
+        label: "GOTURN",
+        value: "GOTURN",
+    },
+]


Let's avoid hardcoding of tracker types on client side. I would suggest two options here:

Get these types from the server

Implement UI only for one of tracker as the initial version

In the future we want to be able to work with different trackers, implemented as serverless functions.

That server side function is not implemented yet but I think we should be able to add it easily as we move forward.

@TheNaman047

It should be implemented before we merge this PR, since our users will have questions about new controls in UI, that are useless for them right now, because our server doesn't support tracking yet.

@bsekachev As per our discussion with @nmanovic , this PR was supposed to be for UI only since you guys are implementing a serverless function for tracking. We can update UI assuming there is an API, let's say get_trackers_list, and make appropriate changes in the front-end. I understand we can't merge it until we have the backend. I suggest we merge this PR and PR for the serverless tracking function together, once we have it.

@savan77 Sure, this PR is for UI only. I mean UI shouldn't have hardcoded tracker types and display controls related with tracking by default (at this moment, till we haven't implemented server tracking). I think making temporary stubs (roughly speaking: const supportTracking = async (): boolean => false and const getTrackers = async (): any[] => []) is a suitable option). You do not need even make real server requests.

@bsekachev can you check now? @TheNaman047 pushed a change which checks for at least one tracker to show the tracker settings.

@savan77 Sure, I will. But as I can see, we still have hardcoded trackers on UI (from OpenCV). Is it outdated code?

@bsekachev that's right. @TheNaman047 As we discussed, we should not show any trackers by default. We need to show tracker settings only if we get positive response from that API call. And show that trackers in the list, not the hard-coded once.
Please correct me if I am wrong. @bsekachev

@savan77

Yes, there were two ideas in this thread above. The first is what you mentioned and the second is do not hardcode trackers on client side. UI should get these types from the server. Right now one of trackers is implemented as a nuclio serverless function and we might use it in the implementation.

I understand if it isn't your priority, on our side I am ready to modify the PR next week

Thanks @bsekachev , that would be great since you have some ideas about this new serverless function.

...rc/components/annotation-page/standard-workspace/controls-side-bar/track-popover-control.tsx

cvat-ui/src/containers/annotation-page/standard-workspace/objects-side-bar/object-item.tsx

bsekachev · 2020-07-13T12:57:05Z

@savan77

Hi, thx for the PR, really appreciate it.
There are some minor code issues I found. Will test the PR better tomorrow.
One important issue is that tracker controls in UI are always visible, nevertheless our server doesn't support trackers at all right now. So, we need to ask server whether it supports tracking or not and display these elements dynamically. You can look at deep extreme cut implementation and see how it works in this case.

Please, contact me if you have any questions about our code

savan77 · 2020-07-13T16:34:15Z

Thanks @bsekachev

@TheNaman047 can you take a look at these comments and make appropriate changes?

TheNaman047 · 2020-07-13T16:45:00Z

Yes, already on it.

…

On Mon, 13 Jul, 2020, 22:04 Savan Visalpara, ***@***.***> wrote: Thanks @bsekachev <https://github.com/bsekachev> @TheNaman047 <https://github.com/TheNaman047> can you take a look at these comments and make appropriate changes? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1884 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGDGPYW5KAWWCUHSGGGTUCTR3MZRRANCNFSM4OW2RCSA> .

bsekachev · 2020-07-19T19:29:29Z

@savan77 @TheNaman047

Hi, what are updates on the PR?

TheNaman047 · 2020-07-20T19:01:06Z

Hi @bsekachev, Thanks for reviewing the code. Have corrected the recommended changes.

bsekachev · 2020-07-21T08:43:15Z

@TheNaman047

Thx, what have you decided about dinamic controls displaying? This issue is quite critical IMHO.

nmanovic · 2020-08-05T10:06:31Z

@savan77 , @TheNaman047 , @rushtehrani ,

I have added support for trackers on server: #1988. It is your turn now. Could you please help to finalise the PR and check that it works with SiamMask from the mentioned PR? I'm happy to help and answer questions.

savan77 · 2020-08-10T16:24:07Z

@nmanovic Thanks. We have been busy with one urgent piece.

Anyway, I think PR is good to go except for one change that @bsekachev requested. I will ask @TheNaman047 to take a look as soon as possible.

mhe-wyze · 2020-08-12T05:47:37Z

Any follow up for this? @nmanovic @savan77

savan77 · 2020-08-13T19:29:45Z

We have made required changes. I will be testing and pushing them soon.

nmanovic · 2020-08-16T19:12:45Z

@savan77 , it is great to hear! Do you have any estimation when it will be ready to merge?

savan77 · 2020-08-19T22:54:07Z

@savan77 , it is great to hear! Do you have any estimation when it will be ready to merge?

@nmanovic - @TheNaman047 has pushed required changes.

savan77 · 2020-08-19T23:07:52Z

@rushtehrani Thanks, just corrected those typos.

nmanovic · 2020-08-20T06:37:55Z

@savan77 , I see at least one conflict. Could you please resolve it?

@bsekachev , could you please look at the current patch and comment?

nmanovic · 2020-09-02T19:09:18Z

@savan77 , as I discussed with Rush and Don, my team is going to implement the feature slightly different. Also the PR isn't ready and we have decided to close it. In any case thanks for the contribution. It gave us a lot of ideas and a ready to use prototype.

mhe-wyze · 2020-09-02T20:55:54Z

Hi @nmanovic , thanks for your update, I was recently looking at this branch because the truly object tracking function by AI models would largely help accelerate the annotation speed for our project. Today I saw you closed it, is that meant this function is still ongoing, haven't finished yet, probably your team would work more on the new design later, right?

nmanovic · 2020-09-03T10:23:51Z

@mhe-wyze , it is right. Recently we have added support for the feature as a serverless function. @bsekachev is going to add UI for tracking in coming months (probably significantly early).

savan77 added 3 commits July 10, 2020 14:50

add ui for tracking

96363ec

revert some setting related changes

24f22ae

rever changes in vscode setting

ae7d16e

savan77 requested a review from bsekachev as a code owner July 10, 2020 18:30

bsekachev reviewed Jul 13, 2020

View reviewed changes

tracking ui changes per feedback

0dad4a8

Change - Tracking icon only visible when required

1afa93e

corrected typo in showTrackerSettings

02146c6

nmanovic closed this Sep 2, 2020

rushtehrani deleted the tracking_ui branch January 9, 2021 05:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: UI for tracking #1884

feat: UI for tracking #1884

savan77 commented Jul 10, 2020 •

edited

Loading

coveralls commented Jul 10, 2020 •

edited

Loading

nmanovic commented Jul 10, 2020

bsekachev commented Jul 11, 2020

bsekachev Jul 13, 2020

TheNaman047 Jul 20, 2020

bsekachev Jul 21, 2020

savan77 Jul 21, 2020 •

edited

Loading

bsekachev Jul 21, 2020 •

edited

Loading

savan77 Aug 18, 2020 •

edited

Loading

bsekachev Aug 20, 2020

savan77 Aug 20, 2020

bsekachev Aug 20, 2020

savan77 Aug 24, 2020

bsekachev commented Jul 13, 2020

savan77 commented Jul 13, 2020

TheNaman047 commented Jul 13, 2020 via email

bsekachev commented Jul 19, 2020

TheNaman047 commented Jul 20, 2020

bsekachev commented Jul 21, 2020 •

edited

Loading

nmanovic commented Aug 5, 2020

savan77 commented Aug 10, 2020

mhe-wyze commented Aug 12, 2020 •

edited

Loading

savan77 commented Aug 13, 2020

nmanovic commented Aug 16, 2020

savan77 commented Aug 19, 2020

savan77 commented Aug 19, 2020

nmanovic commented Aug 20, 2020

nmanovic commented Sep 2, 2020

mhe-wyze commented Sep 2, 2020 •

edited

Loading

nmanovic commented Sep 3, 2020 •

edited

Loading

feat: UI for tracking #1884

feat: UI for tracking #1884

Conversation

savan77 commented Jul 10, 2020 • edited Loading

Motivation and context

How has this been tested?

Checklist

License

coveralls commented Jul 10, 2020 • edited Loading

Pull Request Test Coverage Report for Build 6519

💛 - Coveralls

nmanovic commented Jul 10, 2020

bsekachev commented Jul 11, 2020

bsekachev Jul 13, 2020

Choose a reason for hiding this comment

TheNaman047 Jul 20, 2020

Choose a reason for hiding this comment

bsekachev Jul 21, 2020

Choose a reason for hiding this comment

savan77 Jul 21, 2020 • edited Loading

Choose a reason for hiding this comment

bsekachev Jul 21, 2020 • edited Loading

Choose a reason for hiding this comment

savan77 Aug 18, 2020 • edited Loading

Choose a reason for hiding this comment

bsekachev Aug 20, 2020

Choose a reason for hiding this comment

savan77 Aug 20, 2020

Choose a reason for hiding this comment

bsekachev Aug 20, 2020

Choose a reason for hiding this comment

savan77 Aug 24, 2020

Choose a reason for hiding this comment

bsekachev commented Jul 13, 2020

savan77 commented Jul 13, 2020

TheNaman047 commented Jul 13, 2020 via email

bsekachev commented Jul 19, 2020

TheNaman047 commented Jul 20, 2020

bsekachev commented Jul 21, 2020 • edited Loading

nmanovic commented Aug 5, 2020

savan77 commented Aug 10, 2020

mhe-wyze commented Aug 12, 2020 • edited Loading

savan77 commented Aug 13, 2020

nmanovic commented Aug 16, 2020

savan77 commented Aug 19, 2020

savan77 commented Aug 19, 2020

nmanovic commented Aug 20, 2020

nmanovic commented Sep 2, 2020

mhe-wyze commented Sep 2, 2020 • edited Loading

nmanovic commented Sep 3, 2020 • edited Loading

savan77 commented Jul 10, 2020 •

edited

Loading

coveralls commented Jul 10, 2020 •

edited

Loading

savan77 Jul 21, 2020 •

edited

Loading

bsekachev Jul 21, 2020 •

edited

Loading

savan77 Aug 18, 2020 •

edited

Loading

bsekachev commented Jul 21, 2020 •

edited

Loading

mhe-wyze commented Aug 12, 2020 •

edited

Loading

mhe-wyze commented Sep 2, 2020 •

edited

Loading

nmanovic commented Sep 3, 2020 •

edited

Loading