Add propagate shapes action (#8044)

<!-- Raise an issue to propose your change
(https://github.com/cvat-ai/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/).
-->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
<!-- Why is this change required? What problem does it solve? If it
fixes an open
issue, please link to the issue here. Describe your changes in detail,
add
screenshots. -->
The PR adds new action `Propagate shapes` that allows to propagate all
filtered shapes on the current frame simultaneously

### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->
Manual testing

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have created a changelog fragment <!-- see top comment in
CHANGELOG.md -->
- [ ] I have updated the documentation accordingly
- [ ] I have added tests to cover my changes
- ~~[ ] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~
- [x] I have increased versions of npm packages if it is necessary

([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning),

[cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning)
and

[cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced a new "Propagate Shapes" action to enable simultaneous
propagation of filtered shapes across frames.

- **Updates**
- Updated version numbers for `cvat-core` to `15.0.7` and `cvat-ui` to
`1.63.12` indicating a new release.

- **Enhancements**
- Improved annotation workflows with the addition of frame selection
types and enhanced frame selection handling in the UI.
- Conditional rendering and disabling of frame input fields based on the
active frame selection type.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Boris Sekachev <[email protected]>

changelog.d/20240619_102529_klakhov_propagate_shapes_simultaneously.md

@@ -0,0 +1,4 @@
+### Added
+
+- `Propagate shapes` action to create copies of visible shapes on multiple frames forward or backward
+  (<https://github.com/cvat-ai/cvat/pull/8044>)

cvat-core/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cvat-core",
-  "version": "15.0.6",
+  "version": "15.0.7",
   "type": "module",
   "description": "Part of Computer Vision Tool which presents an interface for client-side integration",
   "main": "src/api.ts",

cvat-core/src/annotations-actions.ts

@@ -1,14 +1,15 @@
-// Copyright (C) 2023 CVAT.ai Corporation
+// Copyright (C) 2023-2024 CVAT.ai Corporation
 //
 // SPDX-License-Identifier: MIT
 
 import { omit, throttle } from 'lodash';
 import { ArgumentError } from './exceptions';
-import { SerializedCollection } from './server-response-types';
+import { SerializedCollection, SerializedShape } from './server-response-types';
 import { Job, Task } from './session';
 import { EventScope, ObjectType } from './enums';
 import ObjectState from './object-state';
 import { getAnnotations, getCollection } from './annotations';
+import { propagateShapes } from './object-utils';
 
 export interface SingleFrameActionInput {
     collection: Omit<SerializedCollection, 'tracks' | 'tags' | 'version'>;
@@ -28,12 +29,20 @@ export enum ActionParameterType {
     NUMBER = 'number',
 }
 
+// For SELECT values should be a list of possible options
+// For NUMBER values should be a list with [min, max, step],
+// or a callback ({ instance }: { instance: Job | Task }) => [min, max, step]
 type ActionParameters = Record<string, {
     type: ActionParameterType;
-    values: string[];
-    defaultValue: string;
+    values: string[] | (({ instance }: { instance: Job | Task }) => string[]);
+    defaultValue: string | (({ instance }: { instance: Job | Task }) => string);
 }>;
 
+export enum FrameSelectionType {
+    SEGMENT = 'segment',
+    CURRENT_FRAME = 'current_frame',
+}
+
 export default class BaseSingleFrameAction {
     /* eslint-disable @typescript-eslint/no-unused-vars */
     public async init(
@@ -58,6 +67,10 @@ export default class BaseSingleFrameAction {
     public get parameters(): ActionParameters | null {
         throw new Error('Method not implemented');
     }
+
+    public get frameSelection(): FrameSelectionType {
+        return FrameSelectionType.SEGMENT;
+    }
 }
 
 class RemoveFilteredShapes extends BaseSingleFrameAction {
@@ -82,6 +95,57 @@ class RemoveFilteredShapes extends BaseSingleFrameAction {
     }
 }
 
+class PropagateShapes extends BaseSingleFrameAction {
+    #targetFrame: number;
+
+    public async init(instance, parameters): Promise<void> {
+        this.#targetFrame = parameters['Target frame'];
+    }
+
+    public async destroy(): Promise<void> {
+        // nothing to destroy
+    }
+
+    public async run(
+        instance,
+        { collection: { shapes }, frameData: { number } },
+    ): Promise<SingleFrameActionOutput> {
+        if (number === this.#targetFrame) {
+            return { collection: { shapes } };
+        }
+        const propagatedShapes = propagateShapes<SerializedShape>(shapes, number, this.#targetFrame);
+        return { collection: { shapes: [...shapes, ...propagatedShapes] } };
+    }
+
+    public get name(): string {
+        return 'Propagate shapes';
+    }
+
+    public get parameters(): ActionParameters | null {
+        return {
+            'Target frame': {
+                type: ActionParameterType.NUMBER,
+                values: ({ instance }) => {
+                    if (instance instanceof Job) {
+                        return [instance.startFrame, instance.stopFrame, 1].map((val) => val.toString());
+                    }
+                    return [0, instance.size - 1, 1].map((val) => val.toString());
+                },
+                defaultValue: ({ instance }) => {
+                    if (instance instanceof Job) {
+                        return instance.stopFrame.toString();
+                    }
+                    return (instance.size - 1).toString();
+                },
+            },
+        };
+    }
+
+    public get frameSelection(): FrameSelectionType {
+        return FrameSelectionType.CURRENT_FRAME;
+    }
+}
+
 const registeredActions: BaseSingleFrameAction[] = [];
 
 export async function listActions(): Promise<BaseSingleFrameAction[]> {
@@ -102,6 +166,7 @@ export async function registerAction(action: BaseSingleFrameAction): Promise<voi
 }
 
 registerAction(new RemoveFilteredShapes());
+registerAction(new PropagateShapes());
 
 async function runSingleFrameChain(
     instance: Job | Task,

cvat-core/src/annotations-objects.ts

@@ -2187,6 +2187,11 @@ export class MaskShape extends Shape {
 
     constructor(data: SerializedShape, clientID: number, color: string, injection: AnnotationInjection) {
         super(data, clientID, color, injection);
+        const [left, top, right, bottom] = this.points.slice(-4);
+        const { width, height } = this.frameMeta[this.frame];
+        if (left >= width || top >= height || right >= width || bottom >= height) {
+            this.points = cropMask(this.points, width, height);
+        }
         [this.left, this.top, this.right, this.bottom] = this.points.splice(-4, 4);
         this.getMasksOnFrame = injection.getMasksOnFrame;
         this.pinned = true;

cvat-core/src/api.ts

@@ -29,7 +29,7 @@ import {
     Exception, ArgumentError, DataError, ScriptingError, ServerError,
 } from './exceptions';
 
-import { mask2Rle, rle2Mask } from './object-utils';
+import { mask2Rle, rle2Mask, propagateShapes } from './object-utils';
 import User from './user';
 import pjson from '../package.json';
 import config from './config';
@@ -397,6 +397,7 @@ function build(): CVATCore {
         utils: {
             mask2Rle,
             rle2Mask,
+            propagateShapes,
         },
     };
 

cvat-core/src/index.ts

@@ -12,7 +12,7 @@ import { AnnotationFormats } from './annotation-formats';
 import logger from './logger';
 import * as enums from './enums';
 import config from './config';
-import { mask2Rle, rle2Mask } from './object-utils';
+import { mask2Rle, rle2Mask, propagateShapes } from './object-utils';
 import User from './user';
 import Project from './project';
 import { Job, Task } from './session';
@@ -201,5 +201,6 @@ export default interface CVATCore {
     utils: {
         mask2Rle: typeof mask2Rle;
         rle2Mask: typeof rle2Mask;
+        propagateShapes: typeof propagateShapes;
     };
 }

cvat-core/src/object-utils.ts

@@ -4,7 +4,9 @@
 
 import { DataError, ArgumentError } from './exceptions';
 import { Attribute } from './labels';
-import { ShapeType, AttributeType } from './enums';
+import { ShapeType, AttributeType, ObjectType } from './enums';
+import { SerializedShape } from './server-response-types';
+import ObjectState, { SerializedData } from './object-state';
 
 export function checkNumberOfPoints(shapeType: ShapeType, points: number[]): void {
     if (shapeType === ShapeType.RECTANGLE) {
@@ -356,3 +358,62 @@ export function rle2Mask(rle: number[], width: number, height: number): number[]
 
     return decoded;
 }
+
+export function propagateShapes<T extends SerializedShape | ObjectState>(
+    shapes: T[], from: number, to: number,
+): T[] {
+    const getCopy = (shape: T): SerializedShape | SerializedData => {
+        if (shape instanceof ObjectState) {
+            return {
+                attributes: shape.attributes,
+                points: shape.shapeType === 'skeleton' ? null : shape.points,
+                occluded: shape.occluded,
+                objectType: shape.objectType !== ObjectType.TRACK ? shape.objectType : ObjectType.SHAPE,
+                shapeType: shape.shapeType,
+                label: shape.label,
+                zOrder: shape.zOrder,
+                rotation: shape.rotation,
+                frame: from,
+                elements: shape.shapeType === 'skeleton' ? shape.elements
+                    .map((element: ObjectState): any => getCopy(element as T)) : [],
+                source: shape.source,
+            };
+        }
+        return {
+            attributes: [...shape.attributes.map((attribute) => ({ ...attribute }))],
+            points: shape.type === 'skeleton' ? null : [...shape.points],
+            occluded: shape.occluded,
+            type: shape.type,
+            label_id: shape.label_id,
+            z_order: shape.z_order,
+            rotation: shape.rotation,
+            frame: from,
+            elements: shape.type === 'skeleton' ? shape.elements
+                .map((element: SerializedShape): SerializedShape => getCopy(element as T) as SerializedShape) : [],
+            source: shape.source,
+            group: 0,
+            outside: false,
+        };
+    };
+
+    const states: T[] = [];
+    const sign = Math.sign(to - from);
+    for (let frame = from + sign; sign > 0 ? frame <= to : frame >= to; frame += sign) {
+        for (const shape of shapes) {
+            const copy = getCopy(shape);
+
+            copy.frame = frame;
+            copy.elements?.forEach((element: Omit<SerializedShape, 'elements'> | SerializedData): void => {
+                element.frame = frame;
+            });
+
+            if (shape instanceof ObjectState) {
+                states.push(new ObjectState(copy as SerializedData) as T);
+            } else {
+                states.push(copy as T);
+            }
+        }
+    }
+
+    return states;
+}

cvat-ui/src/actions/annotation-actions.ts

@@ -473,31 +473,8 @@ export function propagateObjectAsync(from: number, to: number): ThunkAction {
                 throw new Error('There is not an activated object state to be propagated');
             }
 
-            const getCopyFromState = (_objectState: any): any => ({
-                attributes: _objectState.attributes,
-                points: _objectState.shapeType === 'skeleton' ? null : _objectState.points,
-                occluded: _objectState.occluded,
-                objectType: _objectState.objectType !== ObjectType.TRACK ? _objectState.objectType : ObjectType.SHAPE,
-                shapeType: _objectState.shapeType,
-                label: _objectState.label,
-                zOrder: _objectState.zOrder,
-                rotation: _objectState.rotation,
-                frame: from,
-                elements: _objectState.shapeType === 'skeleton' ? _objectState.elements
-                    .map((element: any): any => getCopyFromState(element)) : [],
-                source: _objectState.source,
-            });
-
-            const copy = getCopyFromState(objectState);
             await sessionInstance.logger.log(EventScope.propagateObject, { count: Math.abs(to - from) });
-            const states = [];
-            const sign = Math.sign(to - from);
-            for (let frame = from + sign; sign > 0 ? frame <= to : frame >= to; frame += sign) {
-                copy.frame = frame;
-                copy.elements.forEach((element: any) => { element.frame = frame; });
-                const newState = new cvat.classes.ObjectState(copy);
-                states.push(newState);
-            }
+            const states = cvat.utils.propagateShapes<ObjectState>([objectState], from, to);
 
             await sessionInstance.annotations.put(states);
             const history = await sessionInstance.actions.get();