Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed uploading track annotations for multi-segment tasks #1396

Merged
merged 12 commits into from
Apr 30, 2020
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- AttributeError: 'tuple' object has no attribute 'read' in ReID algorithm (https://github.com/opencv/cvat/issues/1403)
- Wrong semi-automatic segmentation near edges of an image (https://github.com/opencv/cvat/issues/1403)
- Git repos paths (https://github.com/opencv/cvat/pull/1400)
- Uploading annotations for tasks with multiple jobs (https://github.com/opencv/cvat/pull/1396)

### Security
-
Expand Down
52 changes: 48 additions & 4 deletions cvat/apps/annotation/annotation.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,13 +77,57 @@ def serialize(self):
if serializer.is_valid(raise_exception=True):
return serializer.data

@staticmethod
def _is_shape_inside(shape, start, stop):
return start <= int(shape['frame']) <= stop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One tricky case here. Should we return True here for shapes with outside == True?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added filter to remove starting shapes with outside==True after processing


@staticmethod
def _is_track_inside(track, start, stop):
# a <= b
def has_overlap(a, b):
return 0 <= min(b, stop) - max(a, start)

prev_shape = None
for shape in track['shapes']:
if prev_shape and not prev_shape['outside'] and \
has_overlap(prev_shape['frame'], shape['frame']):
return True
prev_shape = shape

if not prev_shape["outside"] and prev_shape['frame'] <= stop:
return True

return False

@staticmethod
def _slice_track(track_, start, stop):
track = copy.deepcopy(track_)
segment_shapes = [s for s in track['shapes'] if AnnotationIR._is_shape_inside(s, start, stop)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@azhavoro , if we have a track with only one shape with the frame less than start, segment_shapes will be empty.

Copy link
Contributor Author

@azhavoro azhavoro Apr 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this case will be handled later in 113 line

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potentially in the line we can get first shape with outside == True.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added filter to remove starting shapes with outside==True after processing


if len(segment_shapes) < len(track['shapes']):
interpolated_shapes = TrackManager.get_interpolated_shapes(track, start, stop)

for shape in interpolated_shapes:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@azhavoro , could you please think how to reorganize the code below? It looks messy.

It seems that get_interpolated_shapes doesn't respect start and stop by a reason. Thus let's filter them:

scoped_shapes = [s for s in interpolated_shapes['shapes'] if AnnotationIR._is_shape_inside(s, start, stop)]
if scoped_shapes and scoped_shapes[0]['outside']:
    scoped_shapes.pop(0) 

If scoped_shapes empty then nothing to do. Otherwise if not scoped_shapes[0]['keyframe'], let's add the shape into segment_shapes at the beginning. If not scoped_shapes[-1]['keyframe'], lets' add the shape into segment_shapes at the end.

Let's assign track['interpolated_shapes'] = scoped_shapes.

Of course I didn't check the logic. You need just improve your code and I hope the code above will give you some ideas how to do that.

if shape['frame'] == start and \
(not segment_shapes or segment_shapes[0]['frame'] > start):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that here shape will be with outside == True? Thus we will insert the shape but it looks wrong.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added filter to remove starting shapes with outside==True after processing

segment_shapes.insert(0, shape)
elif shape['frame'] == stop and \
(not segment_shapes or segment_shapes[-1]['frame'] < stop):
segment_shapes.append(shape)
del track['interpolated_shapes']
for shape in segment_shapes:
del shape['keyframe']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why should we delete the property for all segment_shapes? Could you please add a comment?


track['shapes'] = segment_shapes
track['frame'] = track['shapes'][0]['frame']
return track

#makes a data copy from specified frame interval
def slice(self, start, stop):
is_frame_inside = lambda x: (start <= int(x['frame']) <= stop)
splitted_data = AnnotationIR()
splitted_data.tags = copy.deepcopy(list(filter(is_frame_inside, self.tags)))
splitted_data.shapes = copy.deepcopy(list(filter(is_frame_inside, self.shapes)))
splitted_data.tracks = copy.deepcopy(list(filter(lambda y: len(list(filter(is_frame_inside, y['shapes']))), self.tracks)))
splitted_data.tags = [copy.deepcopy(t) for t in self.tags if self._is_shape_inside(t, start, stop)]
splitted_data.shapes = [copy.deepcopy(s) for s in self.shapes if self._is_shape_inside(s, start, stop)]
splitted_data.tracks = [self._slice_track(t, start, stop) for t in self.tracks if self._is_track_inside(t, start, stop)]

return splitted_data

Expand Down
1 change: 0 additions & 1 deletion cvat/apps/engine/annotation.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
from enum import Enum
from collections import OrderedDict
from django.utils import timezone
from PIL import Image

from django.conf import settings
from django.db import transaction
Expand Down
9 changes: 9 additions & 0 deletions cvat/apps/engine/data_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,15 @@ def _modify_unmached_object(obj, end_frame):
shape["frame"] = end_frame
shape["outside"] = True
obj["shapes"].append(shape)
# Need to update cached interpolated shapes
# because key shapes were changed
if "interpolated_shapes" in obj and obj["interpolated_shapes"]:
azhavoro marked this conversation as resolved.
Show resolved Hide resolved
last_interpolated_shape = obj["interpolated_shapes"][-1]
for frame in range(last_interpolated_shape["frame"] + 1, end_frame):
last_interpolated_shape = copy.deepcopy(last_interpolated_shape)
last_interpolated_shape["frame"] = frame
obj["interpolated_shapes"].append(last_interpolated_shape)
obj["interpolated_shapes"].append(shape)

@staticmethod
def normalize_shape(shape):
Expand Down