Merge annotations and dataset_manager apps #1352

zhiltsov-max · 2020-04-03T12:13:12Z

removed annotations module. Its code moved to dataset_manager app.
- Task- and Job- Annotation, moved to dataset_manager
- Annotation, AnnotationIR moved to dataset_manager. Annotation is renamed to TaskData to reflect its actual status.
- data_manager renamed to annotation_manager as no data manipulation took place there
formats:
- dump and dataset formats are made the same, the only difference is image inclusion. All formats are exported as zip archives therefore, but other formats are possible.
- renamed, format names do not include annotation file extension
- are not stored in the database anymore
- have terser definition syntax (decorators), can be both classes and functions
- importers now can accept multiple file formats (e.g. zip or xml)
- datumaro export format for datasets moved to dump annotations
- added datumaro export format for datasets, which includes images in the project's cache
REST API:
- removed dataset/formats
- updated output format of annotation/formats
- formats have different serialized representation (removed the name field)
- dump_annotations and export_dataset have the same logic - only the recent result is cached for N hours (10 by default). This was different for annotations, which were cached indefinitely. Downloading still requires the same several API calls, but now for annotations no file name is specified (i.e. old dataset downloading logic is used).
- updated GET /task/id/annotations: parameters are format (mandatory for export), filename (optional), action (optional)
General:
- updated docs in the landing README and user_manual
- added auto trimming for trailing whitespaces in the workspace config
UI:
- downloaded file name for annotations download in UI changed from task name to task name + update date + format name
fixed frame step bug (Exporting Frame-stepped project in YOLO Format #1294, COCO Dataset Export filename mismatch if frame_step != 1 #1334)
Datumaro:
- added datumaro_project export format
- fixed categories for empty projects with no sources
- added directory removal on error during extract command
- added debug error message on incorrect XPath
- fixed missing self in the cvat format converter (Potential Datumaro Bug ( And Solution ) #1494)

Closes #1334
Closes #1294
Closes #1494

Motivation and context

This PR is part of ongoing efforts on removing deprecated functionality from CVAT. This PR finally removes annotation application in favor of more advanced and generic dataset_manager.

How has this been tested?

Unit tests, manual testing.

Checklist

I have raised an issue to propose this change (required)
My issue has received approval from the maintainers
I've read the CONTRIBUTION guide
I have added description of my changes into CHANGELOG file
I have updated the documentation accordingly
I have added tests to cover my changes
I have linked related issues (read github docs)

License

I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
I have updated the license header for each file (see an example below)

# Copyright (C) 2020 Intel Corporation
#
# SPDX-License-Identifier: MIT

zhiltsov-max · 2020-04-09T09:23:57Z

update format docs
update ui
update rest api tests
update dm tests

…dm-2

CHANGELOG.md

cvat/apps/documentation/user_guide.md

cvat/apps/engine/static/engine/js/3rdparty/Decoder.worker.js

cvat/apps/engine/static/engine/js/download.worker.js

cvat/apps/engine/static/engine/js/unzip_imgs.worker.js

nmanovic · 2020-05-07T15:12:09Z

cvat/apps/tf_annotation/views.py

@@ -203,7 +203,7 @@ def create_thread(tid, labels_mapping, user):
        result = convert_to_cvat_format(result)
        serializer = LabeledDataSerializer(data = result)
        if serializer.is_valid(raise_exception=True):
-            put_task_data(tid, user, result)
+            put_task_data(tid, result)


@zhiltsov-max , could you please explain why we don't need user here anymore? Not all users should be able to run put_task_data function.

This parameter was never used in any of these functions, or in Task- and JobAnnotations classes.

azhavoro · 2020-05-07T17:05:14Z

I found 2 issues:

cannot upload annotation in old UI
cannot upload cvat xml file from the downloaded archive, but uploading zip works fine. xml file looks ok

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/rq/worker.py", line 812, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.5/dist-packages/rq/job.py", line 588, in perform
    self._result = self._execute()
  File "/usr/local/lib/python3.5/dist-packages/rq/job.py", line 594, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/usr/lib/python3.5/contextlib.py", line 30, in inner
    return func(*args, **kwds)
  File "/home/django/cvat/apps/dataset_manager/task.py", line 732, in import_task_annotations
    task.import_annotations(f, importer)
  File "/home/django/cvat/apps/dataset_manager/task.py", line 629, in import_annotations
    importer(src_file, task_data, **options)
  File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 35, in __call__
    f_or_cls(*args, **kwargs)
  File "/home/django/cvat/apps/dataset_manager/formats/cvat.py", line 557, in _import
    load(src_file, task_data)
  File "/home/django/cvat/apps/dataset_manager/formats/cvat.py", line 416, in load
    ev, _ = next(context)
  File "/usr/lib/python3.5/xml/etree/ElementTree.py", line 1306, in __next__
    self._root = self._parser._close_and_return_root()
  File "/usr/lib/python3.5/xml/etree/ElementTree.py", line 1243, in _close_and_return_root
    root = self._parser.close()
  File "/usr/lib/python3.5/xml/etree/ElementTree.py", line 1652, in close
    self._raiseerror(v)
  File "/usr/lib/python3.5/xml/etree/ElementTree.py", line 1544, in _raiseerror
    raise err
  File "<string>", line None
xml.etree.ElementTree.ParseError: no element found: line 1, column 0

nmanovic · 2020-05-07T17:59:27Z

@zhiltsov-max , could you please organise a Skype review of the patch tomorrow?

zhiltsov-max · 2020-05-07T18:10:01Z

@azhavoro, should be fixed now.

zhiltsov-max · 2020-05-08T09:47:07Z

bump cvat-core version

azhavoro · 2020-05-11T08:25:02Z

@nmanovic could you please merge?

* Added button to cancel started automatic annotation (#1198) * [WIP] Cuboid feature user guide (#1218) * Initial cuboid description * Added Gifs * Added gifs to descriptions * Formatting fixes * Codacy Fixes * Az/fix annotation dump upload (#1229) * fixed upload annotation in case of frame step != 1 * fixed upload annotation in case of attribute value is empty * Setup tag forward to the state * React UI: Added shortcuts (#1230) * [Datumaro] Label remapping transform (#1233) * Add label remapping transform * Apply transforms before project saving * Refactor voc converter * [Datumaro] Optimize mask operations (#1232) * Optimize mask to rle * Optimize mask operations * Fix dm format cmdline * Use RLE masks in datumaro format * Added tag support in new UI (without canvas drawing) * merge fix * Fixed copying/pasting actions * Deleted unused objects * [Datumaro] Dataset format auto detection (#1242) * Add dataset format detection * Add auto format detection for import * Split VOC extractor * Some debian package manager tweaks (#1235) * Some debian package manager tweaks By default, Ubuntu or Debian based "apt" or "apt-get" system installs recommended but not suggested packages . By passing "--no-install-recommends" option, the user lets apt-get know not to consider recommended packages as a dependency to install. This results in smaller downloads and installation of packages . Refer to blog at [Ubuntu Blog](https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends) . * doc: fix description of attribute CVAT XML format (#1168) * Fixed security issues in Datumaro (#1244) * Fixed security issues reported by bandit. * Fixed voc_format extractor * Sorted requirements, added a comment, removed nosec for exec. * Fix copying and creating tags * fixed git sync app (#1247) * fixed git sync app * removed shell=True for subprocess call * Fixed tags color changing and hiding * Fixed filters with tags * wip * PR fixed * Styles fixed * PR fixed * Update develop from release-0.6.0 branch (#1266) * temp * React UI: Attribute annotation mode (#1255) * Done main work * Fixed mount/unmount for canvas wrapper * Refactoring, added filters * Added missed file * Removed unnecessary useEffect * Removed extra code * Max 9 attributes, inputNumber -> Input in aam * Added blur * Renamed component * Fixed condition when validate number attribute * Some minor fixes * Fixed hotkeys config * Fixed canvas zoom * Improved behaviour of number & text * Fixed attributes switching order * Fix tags * Fixed interval * Installation issues for development environment (#1280) * Installation issues * Added ffmpeg * Bump acorn from 6.3.0 to 6.4.1 in /cvat-ui (#1270) * Bump acorn from 6.3.0 to 6.4.1 in /cvat-ui Bumps [acorn](https://github.com/acornjs/acorn) from 6.3.0 to 6.4.1. - [Release notes](https://github.com/acornjs/acorn/releases) - [Commits](acornjs/acorn@6.3.0...6.4.1) Signed-off-by: dependabot[bot] <[email protected]> * Updated CHANGELOG.md Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Boris Sekachev <[email protected]> * Bump acorn from 6.2.1 to 6.4.1 in /cvat-canvas (#1281) Bumps [acorn](https://github.com/acornjs/acorn) from 6.2.1 to 6.4.1. - [Release notes](https://github.com/acornjs/acorn/releases) - [Commits](acornjs/acorn@6.2.1...6.4.1) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Use source label map for voc export (#1276) * Use source label map for voc export * Add line to changelog * [Datumaro] Fix frame matching in video annotations import (#1274) * Add extra frame matching way for videos * Add line to changelog * [Datumaro] Allow empty COCO dataset export (#1272) * Allow empty dataset export in coco * Add line to changelog Co-authored-by: Nikita Manovich <[email protected]> * [Datumaro] Fix occluded and z_order attributes export (#1271) * Fix occluded and z_order attributes export * Add line to changelog Co-authored-by: Nikita Manovich <[email protected]> * Fix LabelMe format (#1260) * Fix labelme filenames * Change module path * Add tests for LabelMe * Update test * Fix test * Add line in changelog * Added point deletion context menu * React UI: Added logging (#1288) * OpenVino 2020 (#1269) * added support for OpenVINO 2020 * fixed dextr and tf_annotation Co-authored-by: Andrey Zhavoronkov <[email protected]> * fixed point context menu for rectangles * Add recursive importers (#1290) * [Datumaro] MOT format (#1289) * Add mot format base * Add mot format * Extract common code * [Datumaro] LabelMe format (#1293) * Little refactoring * Add LabelMe format * [Datumaro] Update LabelMe format (#1296) * Little refactoring * Add LabelMe format * Add usernames * Update tests * Add extractor test * Add information about v0.6.1 release. * React UI: Better exception handling (#1297) * Fixed context menu on ubuntu * Fixed deleting of the latest point * fixes * Fix attributes with spaces in names (#1305) * fixed PR * [Datumaro] Fix image merging (#1301) * Always merge images for own dataset * Fix codacy * Validation for frame input value * Fixed UI fail when write characters in auto save interval input * Fixed input numbers in player settings * Fixed ui failing in propagate confirmation * Fixed latest input numbers, removed extra code, fixed typings * Fix navigation * Added undopoint in editing * Fixed: Could not receive frame (after merge on the latest frame) * Removed extra action dispatching * Which -> button property * Fixed: Inconsistent labels between UI and CLI/API * Fixed resize on right mouse button * Fixed create object URL after first save, fixed URL itself * Undo/redo returns frame where was a change (as it was done in previous version) * Fixed unit tests * [Datumaro] Extract common extractor functionality (#1319) * Extract common extractor functionality * Simplify coco extractor * Fix tfrecord * Fix AWS deployment (#1316) * Don't use antd less (big memory consumtion during the build process) * Fix AWS deployment guide * fix a problem with proxy and long domain names * remove sass loader for antd * Removed less and less-loader. * Simplified webpack config. * Data streaming using chunks (#1007) Huge feature (200+ commits from different developers). It completely changes layout of data (please expect very long DB migration process if you have a lot of tasks). The primary idea is to send data as zip chunks (e.g. 36 images in one chunk) or encoded video chunks and decode them on the client side. It helps to solve the problem with latency when you try to view a separate frame in the UI quickly (play mode). Another important feature of the patch is to provide access to the original images. Thus for annotations the client uses compressed chunks but if you want to export a dataset Datumaro will use original chunks (but video will be decoded with original quality and encoded with maximum/optimal quality in any case). * Shortcuts keymaps moved to state * Titles for objects in side menu * Fixed bug in menu * Titles in attribute annotations mode * Controls panel titles * Titles for object list header * Minor fixes * Added tooltips in top bar * Added settings tooltip * Optimized patch * Typos * Fix a problem with known hosts inside git app (cannot clone a repo from github.com) (#1330) * Fixed zOrder range computing in case when there are tags * Small preview and progress (#1331) * Reduce preview size (untested) * Fix tests * Improve media readers (untested) * fixed migration * fixed frame provider * fixed preview save * fixed stop frame * handle duration == None * codacy * added missed import * unified iteration over frames for media readers and fixed corner case when user specify stop_frame = 0 Co-authored-by: Nikita Manovich <[email protected]> * Move annotation formats to dataset manager (#1256) * Move formats to dataset manager * Unify datataset export and anno export implementations * Add track_id to TrackedShape, export tracked shapes * Replace MOT format * Replace LabelMe format * Add new formats to dm * Add dm tests * Extend TrackedShape * Enable dm test in CI * Fix tests * Add import * Fix tests * Fix mot track ids * Fix mot format * Update attribute logic in labelme tests * Use common code in yolo * Put datumaro in path in settings * Expect labels file in MOT next to annotations file * Add MOT format description * Add import * Add labelme format description * Linter fix * Linter fix2 * Compare attributes ordered * Update docs * Update tests * Az/fix migration (#1333) * fixed migration because readers interface changed * fixed tests * Fixed escape in draw * Insert multiple shapes * Added dialog window with some help info about filters * Typos in doc * Add missing information into changelog. * Add information for next release + update version. * Updated changelog * Updated changelog * Special behaviour for the attribute value __undefined__ * Fixed license year * No break space const * Updated changelog * Fixed year in license headers * Implementation of bitmap in client * Updated changelog * Z-layer support * Fixed settings after reopen a job * Do not show invisible objects on bitmap * Fix point interpolation (#1344) * Extend formats tests with different track types * Add unordered list comparison * Skip empty list comparison * fix * fix * Reproduce problem * Fix point interpolation for single point * undo rest api refactor * Added button to reset color settings * Updated changelog * Added option to display shape text always * Updated changelog * Hidden/outside fix * Fixed screen scaling * fixed dump error after moving format files (#1342) * fixed dump error after moving format files * updated changelog * Az/fix dextr (#1348) * Fixed dextr_segmentation app * updated changelog Co-authored-by: Nikita Manovich <[email protected]> * Disabled option by default * Fixed typos in interface * Increase preview size till 256, 256. Previous preview size was not optimal and led to a blurred image due to too small size. * Add a line into CHANGELOG.md about the change. * Refactor frame provider (#1355) * Refactor frame provider * fix * Add CODEOWNERS file (#1360) * Add CODEOWNERS file * Removed the outdated file. See https://github.com/opencv/cvat/graphs/contributors * Change codeowners * Add pull request and issue templates (#1359) * Add initial version of pull request template * Fix links * Fix codacy issues * Slightly improve titles of sections * Add a note about strikethough for the checklist. * Fix progress of a pull request (each checkbox is an issue) * Add the license header, checkboxes about the license. * Updated the license * Update the license to met https://github.com/licensee/licensee/blob/master/vendor/choosealicense.com/_licenses/mit.txt restrictions. * Fix the pull request template name * Make explaination text as comments (it will be visible when you edit the PR message) * Add initial version of the issue template. * Batch of fixes (#1370) * Some margins were change to paddings * Removed extra selected * Fix: added outside shapes when merge polyshapes * Fixed double scroll bars * Updated canvas table * Fixed setup methodf * Disabled change frame during drag, resize and editing * Fixed: hidden points are visible * Fixed: Merge is allowed for points, but clicks on points conflict with frame dragging logic * Fixed: do not filter removed objects * Updated CHANGELOG.md * Couple of headers updated * Added missed fields in exception logs (#1372) * Simplified codeowners file (#1376) * Fixed points visibility when go between frames * React UI: Added message when share is empty or not mounted (#1373) * Added message when share is empty or not mounted * Updated changelog * Update CHANGELOG.md Co-authored-by: Nikita Manovich <[email protected]> * Save full image paths in coco (#1381) * Add chunk iterator cache to frame provider (#1367) * Add chunk iterator cache * fix * Update item "Creating an annotation task" in User Guide (#1363) * Az/fix remote files (#1392) * fixed task creation from remote files * Update CHANGELOG.md * [Datumaro] Fix COCO keypoint export bug (#1388) * Instructions on using HTTPS (#1357) * Fix label comparison in voc format (#1382) * Update item "Interface of the annotation tool" in User Guide (#1386) * React UI: Batch of fixes (#1383) * Fixed: cannot read property 'set' of undefined * Fixed UI failing: save during drag/resize * Fixed multiple saving (shortcut sticking) * Undo/redo fixed * Allowed one interpolated point * Fixed API reaction when repository synchronization is failed * Updated changelog * Update item "Basic navigation" in User Guide and bug fix in user_guide.md (#1395) * Fix git app paths (#1400) * changed paths for the git repos * Update CHANGELOG.md * updated licence header * React UI: Displaying public ssh keys in UI (#1375) * Updated changelog * Typos * Batch of fixes (#1403) * Fixed bug when job cannot be opened * Fixed bug when deactivated shape is still highlighted * Fixed Error: 'AttributeError: 'tuple' object has no attribute 'read' * Fixed: wrong semi-automatic segmentation near edges of an image * Updated changelog Co-authored-by: Nikita Manovich <[email protected]> * React UI: Semi-automatic segmentation (#1398) * implemented checked * Implemented plugin * Added dialog windows * Updated changelo * Added cancel request * React UI: Automatic bordering for polygons and polylines during drawing/editing (#1394) * Fixed: cannot read property 'set' of undefined * Fixed UI failing: save during drag/resize * Fixed multiple saving (shortcut sticking) * Undo/redo fixed * Allowed one interpolated point * Fixed API reaction when repository synchronization is failed * Updated changelog * Auto bordering feature * Some fixes, added shortcuts * Fixed draw when start with one of supporting point * React UI: batch of fixes (#1404) * React UI has become a primary UI * Temporary disabled cuboid in cvat-core * Fixed: Failed to execute removChild.. (#1405) * Update CHANGELOG.md * Updated CHANGELOG (new version will be 1.0.0-beta.1) * Update CHANGELOG.md (next version is 1.0.0-beta.2) * Updated version of CVAT server till beta.2 * Fixed auto annotation, tf annotation and auto segmentation apps (#1409) * fixed code that uses FrameProvider, as the interface has changed * Update CHANGELOG.md * Update item "Types of shapes" (#1401) * React UI: ReID algorithm (#1406) * Initial commit * Connected storage * Added core API method * Done implementation * Removed rule * Removed double cancel * Updated changelog * Fixed: Cannot read property toFixed of undefined * Update CHANGELOG.md * Improve PR template (#1427) * Simplified PR template. * Remove a new line to make codacy happy. * fix: OSError:broken data stream (#1430) * [Datumaro] Fix duplicating keypoints in COCO export (#1435) * React UI: Fixed typos in remove annotations confirmation (#1450) * React UI: Batch of fixes (#1445) * Hide functionality (H) doesn't work * The highlighted attribute doesn't correspond to the chosen attribute in AAM * Inconvinient image shaking while drawing a polygon (hold Alt key during drawing/editing/grouping to drag an image) * Filter property "shape" doesn't work and extra operator in description * Block of text information doesn't disappear after deactivating for locked shapes * Annotation uploading fails in annotation view * UI freezes after canceling pasting with escape Co-authored-by: Nikita Manovich <[email protected]> * Update item "Annotation mode (basics)" in User Guide (#1412) * Update item "annotation mode (basics)" in User Guide * Replacing a gif017 with an image * React UI: Added client versioning (#1448) * Adjusted antd import * Wrapped core and canvas * Added versioning * Updated changelog, adjusted installation guide a bit * Update item "Interpolation mode (basics)" in User Guide (#1455) * Update item interpolation mode in user guide fix typos and contents * Fix a typo * Fix a typo in the filename Co-authored-by: Nikita Manovich <[email protected]> * React UI: batch of fixes (#1462) * CVAT new UI: add arrows on a mouse cursor * Delete point bug (in new UI) * fix auto annotation to not eat all RAM (#1328) Co-authored-by: Nikita Manovich <[email protected]> * Fixed versioning for node 8 (#1466) * Update items from AAM (basics) to Vocabulary in User Guide (#1459) * Update items from AAM (basics) to Vocabulary * Delete unused images and gif * Fix apache startup (#1467) * Fix Network Error after PC Restart (#1035) * Update CHANGELOG.md * Remove deprecated utils (#1477) * removed deprecated convert scripts * updated changelog * Fixed 'Open task' button doesn't work (#1474) * Fixed uploading track annotations for multi-segment tasks (#1396) * fixed uploading annotation for overlapped segments * fixed dump of tracks for multisegment task * Update CHANGELOG.md * fixed comments * fixed comments * Update cvat/apps/engine/data_manager.py Co-Authored-By: Nikita Manovich <[email protected]> * drop start shapes with outside==True for splitted track * code cleanup * fixed typo * fix Co-authored-by: Nikita Manovich <[email protected]> * React UI: cuboids (#1451) * Update changelog. * Update CHANGELOG and the version of CVAT server. * session.annotations.put() returns indexes of added objects (#1493) * session.annotations.put() returns indexes of added objects * Updated changelog * Updated README.md files (added info about versioning) (#1490) * Updated README.md files (added info about versioning) * Typos * Add coverage for python (#1483) * Fix coverage merging (#1504) * Updating instructions to serve Swagger documentation (#1502) * Update items Workspace and Types of shapes in User Guide (#1497) * Merge annotations and dataset_manager apps (#1352) * Fix coverage measurement (#1516) * fixed linter issues and store credentials cookie in the session object (#1526) * fixed issues * fixed issues and stored credentials cookies inside the session * fixed tests * React UI: batch of fixes (#1525) * Update item Settings in User Guide (#1508) * React UI: cookie policy drawer (#1511) * fixed linter issues (#1538) * fixed false tag activation (#1541) * Added item with npm package version increasing to the PR template (#1542) * [Datumaro] Fix coco import conflict with labels (#1548) * [Datumaro] Change alignment in mask parsing (#1547) * Include empty images in exported annotations (#1479) * Update item Bottom panel in User Guide (#1509) * Update item Bottom panel in User Guide rename the "Bottom panel" to the "Top panel" * Fixed typos, remove trailing spaces in user_guide.md * Fix user_guide.md and update image051.jpg * Ability to configure user agreements for the register user form (#1464) * Update item Side panel in User guide (#1513) * Update item Side panel in User guide * Removed trailing spaces in an Object sidebar item * Update cvat/apps/documentation/user_guide.md * fix typo * missing image correction Co-authored-by: Nikita Manovich <[email protected]> * Layout styles fixes * React UI: cuboid interpolation and cuboid drawing from rectangles (#1560) * Added backend cuboid interpolation and cuboid drawing from rectangles * Added CHANELOG.md * Fixed cuboid front edges stroke width * PR fixes * Fixed auto_segmentation app (#1562) * disabled tf eager execution for auto_segmentation * Update CHANGELOG.md * Add VOC grayscale masks test and documentation (#1576) * Add a test for unpainted masks * Update format documentation * [Datumaro] Fix mask to polygons warning (#1581) * Fix message, add test * update changelog * Fix cuboid conversion (#1577) * Fix cuboid conversion * update changelog Co-authored-by: Nikita Manovich <[email protected]> * [Datumaro] Simplify log level setting (#1583) * Simplify loglevel setting * update changelog * Fixed git synchronization (#1582) * fixed git synchronization * Update CHANGELOG.md Co-authored-by: Nikita Manovich <[email protected]> * Fixed dextr, fixed moving of the canvas (#1573) * Fixed dextr, fixed moving of the canvas * Updated CONTRIBUTUNG.md, updated version Co-authored-by: Nikita Manovich <[email protected]> * Added the ability to configure custom pageViewHit (may useful for web analytics) (#1566) * added the ability to configure custom pageViewHit (may useful for web analytics) * updated version and changelog * fixed comments * cvat-ui minor v++ * subscribe on history updates in the root component * updated Online Demo section of Readme (#1588) * updated Online Demo section of Readme * Change content of "online demo" section Co-authored-by: Nikita Manovich <[email protected]> * Fixed task creation for videos with uneven dimensions. (#1594) * used yuv420p format for compressed and original chunks * updated changelog * added settings to reduce access to analytics component (#1592) * added settings to reduce access to analytics component * updated CHANGELOG * fixed typo * Add item Controls sidebar in User Guide (#1510) * Update text, images and gif in user_guide.md (#1558) * Update user_guide.md, images and gif (#1556) Co-authored-by: Nikita Manovich <[email protected]> * update user_guide.md, images and gifs * Delete gif013 * add gif013 with correct name * Use z_order as a class property (#1589) * Use z_order as a class property * Fix z_order use in voc * Update changelog * Update item Annotation with polylines (#1596) * update user_guide.md and images * fix uppercase letters in images path in user_guide.md and remove trailing spaces * delete images and gifs containing uppercase letters in the name * add images with correct names * fix image paths in user_guide.md * Delete image133 * add image133 with correct name * Fix example yaml format (#1603) * delete duplicate item in user_guide.md (#1607) * update user_guide.md and add image (#1604) * update gifs, images and user_guide.md (#1605) * fix analytics permissions (#1608) * Update item Annotation with cuboids (#1598) * Update item Annotation with polygons in User guide v2 (#1612) * update user_guide.md * update images and gifs * Update Filter, Analytics and Shortcuts items in User Guide (#1606) * update user_guide.md and images * fix user_guide.md * delete unused image * delete unused images * Fix duplicate item in User Guide (#1617) * remove item Annotation with Auto Segmentation * fix link in user_guide.md * delete unused images * Slightly improve changelog * Update CVAT version Co-authored-by: Boris Sekachev <[email protected]> Co-authored-by: Tritin Truong <[email protected]> Co-authored-by: Andrey Zhavoronkov <[email protected]> Co-authored-by: Dmitry Kalinin <[email protected]> Co-authored-by: zhiltsov-max <[email protected]> Co-authored-by: Pratik Raj <[email protected]> Co-authored-by: Mathis Chenuet <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Boris Sekachev <[email protected]> Co-authored-by: Ben Hoff <[email protected]> Co-authored-by: Andrey Zhavoronkov <[email protected]> Co-authored-by: TOsmanov <[email protected]> Co-authored-by: ranko r sredojevic <[email protected]> Co-authored-by: Thomas Albrecht <[email protected]> Co-authored-by: Johannes222 <[email protected]> Co-authored-by: Gururaj Jeerge <[email protected]> Co-authored-by: timurx.osmanov <[email protected]> Co-authored-by: YutaYamazaki <[email protected]>

* Added button to cancel started automatic annotation (cvat-ai#1198) * [WIP] Cuboid feature user guide (cvat-ai#1218) * Initial cuboid description * Added Gifs * Added gifs to descriptions * Formatting fixes * Codacy Fixes * Az/fix annotation dump upload (cvat-ai#1229) * fixed upload annotation in case of frame step != 1 * fixed upload annotation in case of attribute value is empty * Setup tag forward to the state * React UI: Added shortcuts (cvat-ai#1230) * [Datumaro] Label remapping transform (cvat-ai#1233) * Add label remapping transform * Apply transforms before project saving * Refactor voc converter * [Datumaro] Optimize mask operations (cvat-ai#1232) * Optimize mask to rle * Optimize mask operations * Fix dm format cmdline * Use RLE masks in datumaro format * Added tag support in new UI (without canvas drawing) * merge fix * Fixed copying/pasting actions * Deleted unused objects * [Datumaro] Dataset format auto detection (cvat-ai#1242) * Add dataset format detection * Add auto format detection for import * Split VOC extractor * Some debian package manager tweaks (cvat-ai#1235) * Some debian package manager tweaks By default, Ubuntu or Debian based "apt" or "apt-get" system installs recommended but not suggested packages . By passing "--no-install-recommends" option, the user lets apt-get know not to consider recommended packages as a dependency to install. This results in smaller downloads and installation of packages . Refer to blog at [Ubuntu Blog](https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends) . * doc: fix description of attribute CVAT XML format (cvat-ai#1168) * Fixed security issues in Datumaro (cvat-ai#1244) * Fixed security issues reported by bandit. * Fixed voc_format extractor * Sorted requirements, added a comment, removed nosec for exec. * Fix copying and creating tags * fixed git sync app (cvat-ai#1247) * fixed git sync app * removed shell=True for subprocess call * Fixed tags color changing and hiding * Fixed filters with tags * wip * PR fixed * Styles fixed * PR fixed * Update develop from release-0.6.0 branch (cvat-ai#1266) * temp * React UI: Attribute annotation mode (cvat-ai#1255) * Done main work * Fixed mount/unmount for canvas wrapper * Refactoring, added filters * Added missed file * Removed unnecessary useEffect * Removed extra code * Max 9 attributes, inputNumber -> Input in aam * Added blur * Renamed component * Fixed condition when validate number attribute * Some minor fixes * Fixed hotkeys config * Fixed canvas zoom * Improved behaviour of number & text * Fixed attributes switching order * Fix tags * Fixed interval * Installation issues for development environment (cvat-ai#1280) * Installation issues * Added ffmpeg * Bump acorn from 6.3.0 to 6.4.1 in /cvat-ui (cvat-ai#1270) * Bump acorn from 6.3.0 to 6.4.1 in /cvat-ui Bumps [acorn](https://github.com/acornjs/acorn) from 6.3.0 to 6.4.1. - [Release notes](https://github.com/acornjs/acorn/releases) - [Commits](acornjs/acorn@6.3.0...6.4.1) Signed-off-by: dependabot[bot] <[email protected]> * Updated CHANGELOG.md Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Boris Sekachev <[email protected]> * Bump acorn from 6.2.1 to 6.4.1 in /cvat-canvas (cvat-ai#1281) Bumps [acorn](https://github.com/acornjs/acorn) from 6.2.1 to 6.4.1. - [Release notes](https://github.com/acornjs/acorn/releases) - [Commits](acornjs/acorn@6.2.1...6.4.1) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Use source label map for voc export (cvat-ai#1276) * Use source label map for voc export * Add line to changelog * [Datumaro] Fix frame matching in video annotations import (cvat-ai#1274) * Add extra frame matching way for videos * Add line to changelog * [Datumaro] Allow empty COCO dataset export (cvat-ai#1272) * Allow empty dataset export in coco * Add line to changelog Co-authored-by: Nikita Manovich <[email protected]> * [Datumaro] Fix occluded and z_order attributes export (cvat-ai#1271) * Fix occluded and z_order attributes export * Add line to changelog Co-authored-by: Nikita Manovich <[email protected]> * Fix LabelMe format (cvat-ai#1260) * Fix labelme filenames * Change module path * Add tests for LabelMe * Update test * Fix test * Add line in changelog * Added point deletion context menu * React UI: Added logging (cvat-ai#1288) * OpenVino 2020 (cvat-ai#1269) * added support for OpenVINO 2020 * fixed dextr and tf_annotation Co-authored-by: Andrey Zhavoronkov <[email protected]> * fixed point context menu for rectangles * Add recursive importers (cvat-ai#1290) * [Datumaro] MOT format (cvat-ai#1289) * Add mot format base * Add mot format * Extract common code * [Datumaro] LabelMe format (cvat-ai#1293) * Little refactoring * Add LabelMe format * [Datumaro] Update LabelMe format (cvat-ai#1296) * Little refactoring * Add LabelMe format * Add usernames * Update tests * Add extractor test * Add information about v0.6.1 release. * React UI: Better exception handling (cvat-ai#1297) * Fixed context menu on ubuntu * Fixed deleting of the latest point * fixes * Fix attributes with spaces in names (cvat-ai#1305) * fixed PR * [Datumaro] Fix image merging (cvat-ai#1301) * Always merge images for own dataset * Fix codacy * Validation for frame input value * Fixed UI fail when write characters in auto save interval input * Fixed input numbers in player settings * Fixed ui failing in propagate confirmation * Fixed latest input numbers, removed extra code, fixed typings * Fix navigation * Added undopoint in editing * Fixed: Could not receive frame (after merge on the latest frame) * Removed extra action dispatching * Which -> button property * Fixed: Inconsistent labels between UI and CLI/API * Fixed resize on right mouse button * Fixed create object URL after first save, fixed URL itself * Undo/redo returns frame where was a change (as it was done in previous version) * Fixed unit tests * [Datumaro] Extract common extractor functionality (cvat-ai#1319) * Extract common extractor functionality * Simplify coco extractor * Fix tfrecord * Fix AWS deployment (cvat-ai#1316) * Don't use antd less (big memory consumtion during the build process) * Fix AWS deployment guide * fix a problem with proxy and long domain names * remove sass loader for antd * Removed less and less-loader. * Simplified webpack config. * Data streaming using chunks (cvat-ai#1007) Huge feature (200+ commits from different developers). It completely changes layout of data (please expect very long DB migration process if you have a lot of tasks). The primary idea is to send data as zip chunks (e.g. 36 images in one chunk) or encoded video chunks and decode them on the client side. It helps to solve the problem with latency when you try to view a separate frame in the UI quickly (play mode). Another important feature of the patch is to provide access to the original images. Thus for annotations the client uses compressed chunks but if you want to export a dataset Datumaro will use original chunks (but video will be decoded with original quality and encoded with maximum/optimal quality in any case). * Shortcuts keymaps moved to state * Titles for objects in side menu * Fixed bug in menu * Titles in attribute annotations mode * Controls panel titles * Titles for object list header * Minor fixes * Added tooltips in top bar * Added settings tooltip * Optimized patch * Typos * Fix a problem with known hosts inside git app (cannot clone a repo from github.com) (cvat-ai#1330) * Fixed zOrder range computing in case when there are tags * Small preview and progress (cvat-ai#1331) * Reduce preview size (untested) * Fix tests * Improve media readers (untested) * fixed migration * fixed frame provider * fixed preview save * fixed stop frame * handle duration == None * codacy * added missed import * unified iteration over frames for media readers and fixed corner case when user specify stop_frame = 0 Co-authored-by: Nikita Manovich <[email protected]> * Move annotation formats to dataset manager (cvat-ai#1256) * Move formats to dataset manager * Unify datataset export and anno export implementations * Add track_id to TrackedShape, export tracked shapes * Replace MOT format * Replace LabelMe format * Add new formats to dm * Add dm tests * Extend TrackedShape * Enable dm test in CI * Fix tests * Add import * Fix tests * Fix mot track ids * Fix mot format * Update attribute logic in labelme tests * Use common code in yolo * Put datumaro in path in settings * Expect labels file in MOT next to annotations file * Add MOT format description * Add import * Add labelme format description * Linter fix * Linter fix2 * Compare attributes ordered * Update docs * Update tests * Az/fix migration (cvat-ai#1333) * fixed migration because readers interface changed * fixed tests * Fixed escape in draw * Insert multiple shapes * Added dialog window with some help info about filters * Typos in doc * Add missing information into changelog. * Add information for next release + update version. * Updated changelog * Updated changelog * Special behaviour for the attribute value __undefined__ * Fixed license year * No break space const * Updated changelog * Fixed year in license headers * Implementation of bitmap in client * Updated changelog * Z-layer support * Fixed settings after reopen a job * Do not show invisible objects on bitmap * Fix point interpolation (cvat-ai#1344) * Extend formats tests with different track types * Add unordered list comparison * Skip empty list comparison * fix * fix * Reproduce problem * Fix point interpolation for single point * undo rest api refactor * Added button to reset color settings * Updated changelog * Added option to display shape text always * Updated changelog * Hidden/outside fix * Fixed screen scaling * fixed dump error after moving format files (cvat-ai#1342) * fixed dump error after moving format files * updated changelog * Az/fix dextr (cvat-ai#1348) * Fixed dextr_segmentation app * updated changelog Co-authored-by: Nikita Manovich <[email protected]> * Disabled option by default * Fixed typos in interface * Increase preview size till 256, 256. Previous preview size was not optimal and led to a blurred image due to too small size. * Add a line into CHANGELOG.md about the change. * Refactor frame provider (cvat-ai#1355) * Refactor frame provider * fix * Add CODEOWNERS file (cvat-ai#1360) * Add CODEOWNERS file * Removed the outdated file. See https://github.com/opencv/cvat/graphs/contributors * Change codeowners * Add pull request and issue templates (cvat-ai#1359) * Add initial version of pull request template * Fix links * Fix codacy issues * Slightly improve titles of sections * Add a note about strikethough for the checklist. * Fix progress of a pull request (each checkbox is an issue) * Add the license header, checkboxes about the license. * Updated the license * Update the license to met https://github.com/licensee/licensee/blob/master/vendor/choosealicense.com/_licenses/mit.txt restrictions. * Fix the pull request template name * Make explaination text as comments (it will be visible when you edit the PR message) * Add initial version of the issue template. * Batch of fixes (cvat-ai#1370) * Some margins were change to paddings * Removed extra selected * Fix: added outside shapes when merge polyshapes * Fixed double scroll bars * Updated canvas table * Fixed setup methodf * Disabled change frame during drag, resize and editing * Fixed: hidden points are visible * Fixed: Merge is allowed for points, but clicks on points conflict with frame dragging logic * Fixed: do not filter removed objects * Updated CHANGELOG.md * Couple of headers updated * Added missed fields in exception logs (cvat-ai#1372) * Simplified codeowners file (cvat-ai#1376) * Fixed points visibility when go between frames * React UI: Added message when share is empty or not mounted (cvat-ai#1373) * Added message when share is empty or not mounted * Updated changelog * Update CHANGELOG.md Co-authored-by: Nikita Manovich <[email protected]> * Save full image paths in coco (cvat-ai#1381) * Add chunk iterator cache to frame provider (cvat-ai#1367) * Add chunk iterator cache * fix * Update item "Creating an annotation task" in User Guide (cvat-ai#1363) * Az/fix remote files (cvat-ai#1392) * fixed task creation from remote files * Update CHANGELOG.md * [Datumaro] Fix COCO keypoint export bug (cvat-ai#1388) * Instructions on using HTTPS (cvat-ai#1357) * Fix label comparison in voc format (cvat-ai#1382) * Update item "Interface of the annotation tool" in User Guide (cvat-ai#1386) * React UI: Batch of fixes (cvat-ai#1383) * Fixed: cannot read property 'set' of undefined * Fixed UI failing: save during drag/resize * Fixed multiple saving (shortcut sticking) * Undo/redo fixed * Allowed one interpolated point * Fixed API reaction when repository synchronization is failed * Updated changelog * Update item "Basic navigation" in User Guide and bug fix in user_guide.md (cvat-ai#1395) * Fix git app paths (cvat-ai#1400) * changed paths for the git repos * Update CHANGELOG.md * updated licence header * React UI: Displaying public ssh keys in UI (cvat-ai#1375) * Updated changelog * Typos * Batch of fixes (cvat-ai#1403) * Fixed bug when job cannot be opened * Fixed bug when deactivated shape is still highlighted * Fixed Error: 'AttributeError: 'tuple' object has no attribute 'read' * Fixed: wrong semi-automatic segmentation near edges of an image * Updated changelog Co-authored-by: Nikita Manovich <[email protected]> * React UI: Semi-automatic segmentation (cvat-ai#1398) * implemented checked * Implemented plugin * Added dialog windows * Updated changelo * Added cancel request * React UI: Automatic bordering for polygons and polylines during drawing/editing (cvat-ai#1394) * Fixed: cannot read property 'set' of undefined * Fixed UI failing: save during drag/resize * Fixed multiple saving (shortcut sticking) * Undo/redo fixed * Allowed one interpolated point * Fixed API reaction when repository synchronization is failed * Updated changelog * Auto bordering feature * Some fixes, added shortcuts * Fixed draw when start with one of supporting point * React UI: batch of fixes (cvat-ai#1404) * React UI has become a primary UI * Temporary disabled cuboid in cvat-core * Fixed: Failed to execute removChild.. (cvat-ai#1405) * Update CHANGELOG.md * Updated CHANGELOG (new version will be 1.0.0-beta.1) * Update CHANGELOG.md (next version is 1.0.0-beta.2) * Updated version of CVAT server till beta.2 * Fixed auto annotation, tf annotation and auto segmentation apps (cvat-ai#1409) * fixed code that uses FrameProvider, as the interface has changed * Update CHANGELOG.md * Update item "Types of shapes" (cvat-ai#1401) * React UI: ReID algorithm (cvat-ai#1406) * Initial commit * Connected storage * Added core API method * Done implementation * Removed rule * Removed double cancel * Updated changelog * Fixed: Cannot read property toFixed of undefined * Update CHANGELOG.md * Improve PR template (cvat-ai#1427) * Simplified PR template. * Remove a new line to make codacy happy. * fix: OSError:broken data stream (cvat-ai#1430) * [Datumaro] Fix duplicating keypoints in COCO export (cvat-ai#1435) * React UI: Fixed typos in remove annotations confirmation (cvat-ai#1450) * React UI: Batch of fixes (cvat-ai#1445) * Hide functionality (H) doesn't work * The highlighted attribute doesn't correspond to the chosen attribute in AAM * Inconvinient image shaking while drawing a polygon (hold Alt key during drawing/editing/grouping to drag an image) * Filter property "shape" doesn't work and extra operator in description * Block of text information doesn't disappear after deactivating for locked shapes * Annotation uploading fails in annotation view * UI freezes after canceling pasting with escape Co-authored-by: Nikita Manovich <[email protected]> * Update item "Annotation mode (basics)" in User Guide (cvat-ai#1412) * Update item "annotation mode (basics)" in User Guide * Replacing a gif017 with an image * React UI: Added client versioning (cvat-ai#1448) * Adjusted antd import * Wrapped core and canvas * Added versioning * Updated changelog, adjusted installation guide a bit * Update item "Interpolation mode (basics)" in User Guide (cvat-ai#1455) * Update item interpolation mode in user guide fix typos and contents * Fix a typo * Fix a typo in the filename Co-authored-by: Nikita Manovich <[email protected]> * React UI: batch of fixes (cvat-ai#1462) * CVAT new UI: add arrows on a mouse cursor * Delete point bug (in new UI) * fix auto annotation to not eat all RAM (cvat-ai#1328) Co-authored-by: Nikita Manovich <[email protected]> * Fixed versioning for node 8 (cvat-ai#1466) * Update items from AAM (basics) to Vocabulary in User Guide (cvat-ai#1459) * Update items from AAM (basics) to Vocabulary * Delete unused images and gif * Fix apache startup (cvat-ai#1467) * Fix Network Error after PC Restart (cvat-ai#1035) * Update CHANGELOG.md * Remove deprecated utils (cvat-ai#1477) * removed deprecated convert scripts * updated changelog * Fixed 'Open task' button doesn't work (cvat-ai#1474) * Fixed uploading track annotations for multi-segment tasks (cvat-ai#1396) * fixed uploading annotation for overlapped segments * fixed dump of tracks for multisegment task * Update CHANGELOG.md * fixed comments * fixed comments * Update cvat/apps/engine/data_manager.py Co-Authored-By: Nikita Manovich <[email protected]> * drop start shapes with outside==True for splitted track * code cleanup * fixed typo * fix Co-authored-by: Nikita Manovich <[email protected]> * React UI: cuboids (cvat-ai#1451) * Update changelog. * Update CHANGELOG and the version of CVAT server. * session.annotations.put() returns indexes of added objects (cvat-ai#1493) * session.annotations.put() returns indexes of added objects * Updated changelog * Updated README.md files (added info about versioning) (cvat-ai#1490) * Updated README.md files (added info about versioning) * Typos * Add coverage for python (cvat-ai#1483) * Fix coverage merging (cvat-ai#1504) * Updating instructions to serve Swagger documentation (cvat-ai#1502) * Update items Workspace and Types of shapes in User Guide (cvat-ai#1497) * Merge annotations and dataset_manager apps (cvat-ai#1352) * Fix coverage measurement (cvat-ai#1516) * fixed linter issues and store credentials cookie in the session object (cvat-ai#1526) * fixed issues * fixed issues and stored credentials cookies inside the session * fixed tests * React UI: batch of fixes (cvat-ai#1525) * Update item Settings in User Guide (cvat-ai#1508) * React UI: cookie policy drawer (cvat-ai#1511) * fixed linter issues (cvat-ai#1538) * fixed false tag activation (cvat-ai#1541) * Added item with npm package version increasing to the PR template (cvat-ai#1542) * [Datumaro] Fix coco import conflict with labels (cvat-ai#1548) * [Datumaro] Change alignment in mask parsing (cvat-ai#1547) * Include empty images in exported annotations (cvat-ai#1479) * Update item Bottom panel in User Guide (cvat-ai#1509) * Update item Bottom panel in User Guide rename the "Bottom panel" to the "Top panel" * Fixed typos, remove trailing spaces in user_guide.md * Fix user_guide.md and update image051.jpg * Ability to configure user agreements for the register user form (cvat-ai#1464) * Update item Side panel in User guide (cvat-ai#1513) * Update item Side panel in User guide * Removed trailing spaces in an Object sidebar item * Update cvat/apps/documentation/user_guide.md * fix typo * missing image correction Co-authored-by: Nikita Manovich <[email protected]> * Layout styles fixes * React UI: cuboid interpolation and cuboid drawing from rectangles (cvat-ai#1560) * Added backend cuboid interpolation and cuboid drawing from rectangles * Added CHANELOG.md * Fixed cuboid front edges stroke width * PR fixes * Fixed auto_segmentation app (cvat-ai#1562) * disabled tf eager execution for auto_segmentation * Update CHANGELOG.md * Add VOC grayscale masks test and documentation (cvat-ai#1576) * Add a test for unpainted masks * Update format documentation * [Datumaro] Fix mask to polygons warning (cvat-ai#1581) * Fix message, add test * update changelog * Fix cuboid conversion (cvat-ai#1577) * Fix cuboid conversion * update changelog Co-authored-by: Nikita Manovich <[email protected]> * [Datumaro] Simplify log level setting (cvat-ai#1583) * Simplify loglevel setting * update changelog * Fixed git synchronization (cvat-ai#1582) * fixed git synchronization * Update CHANGELOG.md Co-authored-by: Nikita Manovich <[email protected]> * Fixed dextr, fixed moving of the canvas (cvat-ai#1573) * Fixed dextr, fixed moving of the canvas * Updated CONTRIBUTUNG.md, updated version Co-authored-by: Nikita Manovich <[email protected]> * Added the ability to configure custom pageViewHit (may useful for web analytics) (cvat-ai#1566) * added the ability to configure custom pageViewHit (may useful for web analytics) * updated version and changelog * fixed comments * cvat-ui minor v++ * subscribe on history updates in the root component * updated Online Demo section of Readme (cvat-ai#1588) * updated Online Demo section of Readme * Change content of "online demo" section Co-authored-by: Nikita Manovich <[email protected]> * Fixed task creation for videos with uneven dimensions. (cvat-ai#1594) * used yuv420p format for compressed and original chunks * updated changelog * added settings to reduce access to analytics component (cvat-ai#1592) * added settings to reduce access to analytics component * updated CHANGELOG * fixed typo * Add item Controls sidebar in User Guide (cvat-ai#1510) * Update text, images and gif in user_guide.md (cvat-ai#1558) * Update user_guide.md, images and gif (cvat-ai#1556) Co-authored-by: Nikita Manovich <[email protected]> * update user_guide.md, images and gifs * Delete gif013 * add gif013 with correct name * Use z_order as a class property (cvat-ai#1589) * Use z_order as a class property * Fix z_order use in voc * Update changelog * Update item Annotation with polylines (cvat-ai#1596) * update user_guide.md and images * fix uppercase letters in images path in user_guide.md and remove trailing spaces * delete images and gifs containing uppercase letters in the name * add images with correct names * fix image paths in user_guide.md * Delete image133 * add image133 with correct name * Fix example yaml format (cvat-ai#1603) * delete duplicate item in user_guide.md (cvat-ai#1607) * update user_guide.md and add image (cvat-ai#1604) * update gifs, images and user_guide.md (cvat-ai#1605) * fix analytics permissions (cvat-ai#1608) * Update item Annotation with cuboids (cvat-ai#1598) * Update item Annotation with polygons in User guide v2 (cvat-ai#1612) * update user_guide.md * update images and gifs * Update Filter, Analytics and Shortcuts items in User Guide (cvat-ai#1606) * update user_guide.md and images * fix user_guide.md * delete unused image * delete unused images * Fix duplicate item in User Guide (cvat-ai#1617) * remove item Annotation with Auto Segmentation * fix link in user_guide.md * delete unused images * Slightly improve changelog * Update CVAT version Co-authored-by: Boris Sekachev <[email protected]> Co-authored-by: Tritin Truong <[email protected]> Co-authored-by: Andrey Zhavoronkov <[email protected]> Co-authored-by: Dmitry Kalinin <[email protected]> Co-authored-by: zhiltsov-max <[email protected]> Co-authored-by: Pratik Raj <[email protected]> Co-authored-by: Mathis Chenuet <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Boris Sekachev <[email protected]> Co-authored-by: Ben Hoff <[email protected]> Co-authored-by: Andrey Zhavoronkov <[email protected]> Co-authored-by: TOsmanov <[email protected]> Co-authored-by: ranko r sredojevic <[email protected]> Co-authored-by: Thomas Albrecht <[email protected]> Co-authored-by: Johannes222 <[email protected]> Co-authored-by: Gururaj Jeerge <[email protected]> Co-authored-by: timurx.osmanov <[email protected]> Co-authored-by: YutaYamazaki <[email protected]>

codewithunknown · 2024-03-03T11:27:36Z

Hello @bsekachev @nmanovic @zhiltsov-max

I encountered an error when attempting to upload annotations with the same file name but different extensions. The error message is: "Item (image1,'default') is repeated in the source sequence".

When trying to upload files with the same name but different extensions, like the example provided in the screenshot

I noticed that the error occurs in the cvat_server logs.

Upon debugging, I found that an exception is being raised in datumaro, specifically in the code link

I am in the process of fixing this issue. I would greatly appreciate your assistance in determining the next steps to resolve it.

Thank you for your help.

2024-03-03 16:47:13 2024-03-03 11:17:13,941 DEBG 'uvicorn-1' stdout output:
2024-03-03 16:47:13 INFO:     172.18.0.1:0 - "POST /api/tasks/4/annotations?org=&use_default_location=true&format=COCO+1.0&filename=anntions.json&conv_mask_to_poly=true HTTP/1.0" 202 Accepted
2024-03-03 16:47:13 
2024-03-03 16:47:14 2024-03-03 11:17:14,004 DEBG 'uvicorn-1' stdout output:
2024-03-03 16:47:14 INFO:     172.18.0.1:0 - "PUT /api/tasks/4/annotations?org=&use_default_location=true&format=COCO+1.0&filename=anntions.json&conv_mask_to_poly=true&rq_id=import:task-4-annotations-by-notimportantupdatesonly HTTP/1.0" 202 Accepted
2024-03-03 16:47:14 
2024-03-03 16:47:17 2024-03-03 11:17:17,085 DEBG 'uvicorn-1' stderr output:
2024-03-03 16:47:17 [2024-03-03 11:17:17,084] ERROR cvat.server.engine: Traceback (most recent call last):
2024-03-03 16:47:17   File "/home/django/cvat/apps/dataset_manager/task.py", line 912, in import_task_annotations
2024-03-03 16:47:17     task.import_annotations(f, importer, conv_mask_to_poly=conv_mask_to_poly)
2024-03-03 16:47:17   File "/home/django/cvat/apps/dataset_manager/task.py", line 796, in import_annotations
2024-03-03 16:47:17     importer(src_file, temp_dir, task_data, **options)
2024-03-03 16:47:17   File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in __call__
2024-03-03 16:47:17     f_or_cls(*args, **kwargs)
2024-03-03 16:47:17   File "/home/django/cvat/apps/dataset_manager/formats/coco.py", line 37, in _import
2024-03-03 16:47:17     import_dm_annotations(dataset, instance_data)
2024-03-03 16:47:17   File "/home/django/cvat/apps/dataset_manager/bindings.py", line 1929, in import_dm_annotations
2024-03-03 16:47:17     if len(dm_dataset) == 0:
2024-03-03 16:47:17   File "/opt/venv/lib/python3.10/site-packages/datumaro/components/dataset.py", line 820, in __len__
2024-03-03 16:47:17     return len(self._data)
2024-03-03 16:47:17   File "/opt/venv/lib/python3.10/site-packages/datumaro/components/dataset.py", line 544, in __len__
2024-03-03 16:47:17     self.init_cache()
2024-03-03 16:47:17   File "/opt/venv/lib/python3.10/site-packages/datumaro/components/dataset.py", line 338, in init_cache
2024-03-03 16:47:17     for _ in self._iter_init_cache():
2024-03-03 16:47:17   File "/opt/venv/lib/python3.10/site-packages/datumaro/components/dataset.py", line 345, in _iter_init_cache
2024-03-03 16:47:17     for item in self._iter_init_cache_unchecked():
2024-03-03 16:47:17   File "/opt/venv/lib/python3.10/site-packages/datumaro/components/dataset.py", line 460, in _iter_init_cache_unchecked
2024-03-03 16:47:17     raise RepeatedItemError(item_id)
2024-03-03 16:47:17 datumaro.components.errors.RepeatedItemError: Item ('image1', 'default') is repeated in the source sequence.
2024-03-03 16:47:17 
2024-03-03 16:47:17 During handling of the above exception, another exception occurred:
2024-03-03 16:47:17 
2024-03-03 16:47:17 Traceback (most recent call last):
2024-03-03 16:47:17   File "/opt/venv/lib/python3.10/site-packages/rq/worker.py", line 1428, in perform_job
2024-03-03 16:47:17     rv = job.perform()
2024-03-03 16:47:17   File "/opt/venv/lib/python3.10/site-packages/rq/job.py", line 1278, in perform
2024-03-03 16:47:17     self._result = self._execute()
2024-03-03 16:47:17   File "/opt/venv/lib/python3.10/site-packages/rq/job.py", line 1315, in _execute
2024-03-03 16:47:17     result = self.func(*self.args, **self.kwargs)
2024-03-03 16:47:17   File "/home/django/cvat/apps/engine/utils.py", line 332, in import_resource_with_clean_up_after
2024-03-03 16:47:17     result = func(filename, *args, **kwargs)
2024-03-03 16:47:17   File "/usr/lib/python3.10/contextlib.py", line 79, in inner
2024-03-03 16:47:17     return func(*args, **kwds)
2024-03-03 16:47:17   File "/home/django/cvat/apps/dataset_manager/task.py", line 914, in import_task_annotations
2024-03-03 16:47:17     raise CvatImportError(str(ex))
2024-03-03 16:47:17 cvat.apps.dataset_manager.bindings.CvatImportError: Item ('image1', 'default') is repeated in the source sequence.
2024-03-03 16:47:17 
2024-03-03 16:47:17

zhiltsov-max requested review from azhavoro and nmanovic as code owners April 6, 2020 13:22

zhiltsov-max requested a review from bsekachev as a code owner April 9, 2020 09:13

zhiltsov-max added 25 commits April 14, 2020 10:54

Move annotations to dm

9c87cdc

Refactor dm

06d6217

Rename data manager

40375f0

Move anno dump and upload functions

8c64a9a

Join server host and port in cvat cli

ce9c0a8

Move export templates dir

cf181ae

add dm project exporter

76c48c7

update mask format support

a2f00f5

Use decorators for formats definition

37a4a03

Update formats

7736039

Update format implementations

ba75d7c

remove parameter

8b0173a

Add dm views

5c770b4

Move annotation components to dm

527123b

restore extension for export formats

a758550

update rest api

81404b1

use serializers, update views

92b1bba

merge develop

ae7974e

Update format names

f8e2663

Update docs

927ca56

Update tests

cc04df1

move test

2d2ca2d

fix import

0263a0e

Extend format tests

ec52ecd

django compatibility for directory access

76f496e

zhiltsov-max added 3 commits May 6, 2020 21:52

Merge remote-tracking branch 'origin/develop' into zm/merge-anno-and-…

4b8016a

…dm-2

fix merge

1d4cb1f

Merge branch 'develop' into zm/merge-anno-and-dm-2

95180eb

nmanovic requested changes May 7, 2020

View reviewed changes

zhiltsov-max added 3 commits May 7, 2020 19:48

remove duplicating lines from changelog

8d3b0ae

fix ci tests

13ea752

revert unnecessary changes in user_guide

c9dfff4

zhiltsov-max added 4 commits May 7, 2020 20:07

revert changes in js libraries

5728cfd

rebuild cvat-core

80e04c6

fix uploading in old ui

9d32c57

fix cvat xml uploading

79e7e9f

Bump cvat-core and cvat-ui versions

ae7c237

nmanovic approved these changes May 10, 2020

View reviewed changes

azhavoro approved these changes May 10, 2020

View reviewed changes

zhiltsov-max mentioned this pull request May 12, 2020

Error when downloading mask #749

Closed

zhiltsov-max linked an issue May 12, 2020 that may be closed by this pull request

Error when downloading mask #749

Closed

nmanovic merged commit 5ab5499 into develop May 13, 2020

azhavoro mentioned this pull request May 14, 2020

fixed linter issues #1538

Merged

7 tasks

nmanovic deleted the zm/merge-anno-and-dm-2 branch May 16, 2020 09:18

zhiltsov-max mentioned this pull request Jun 8, 2020

Empty csv file on "MOT CSV 1.0" #1096

Closed

frndmg pushed a commit to signatrix/cvat that referenced this pull request Aug 5, 2020

Merge annotations and dataset_manager apps (cvat-ai#1352)

3faf3ed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge annotations and dataset_manager apps #1352

Merge annotations and dataset_manager apps #1352

zhiltsov-max commented Apr 3, 2020 •

edited

Loading

zhiltsov-max commented Apr 9, 2020 •

edited

Loading

nmanovic May 7, 2020

zhiltsov-max May 7, 2020

azhavoro commented May 7, 2020

nmanovic commented May 7, 2020

zhiltsov-max commented May 7, 2020

zhiltsov-max commented May 8, 2020 •

edited by nmanovic

Loading

azhavoro commented May 11, 2020

codewithunknown commented Mar 3, 2024 •

edited

Loading

Merge annotations and dataset_manager apps #1352

Merge annotations and dataset_manager apps #1352

Conversation

zhiltsov-max commented Apr 3, 2020 • edited Loading

Motivation and context

How has this been tested?

Checklist

License

zhiltsov-max commented Apr 9, 2020 • edited Loading

nmanovic May 7, 2020

Choose a reason for hiding this comment

zhiltsov-max May 7, 2020

Choose a reason for hiding this comment

azhavoro commented May 7, 2020

nmanovic commented May 7, 2020

zhiltsov-max commented May 7, 2020

zhiltsov-max commented May 8, 2020 • edited by nmanovic Loading

azhavoro commented May 11, 2020

codewithunknown commented Mar 3, 2024 • edited Loading

zhiltsov-max commented Apr 3, 2020 •

edited

Loading

zhiltsov-max commented Apr 9, 2020 •

edited

Loading

zhiltsov-max commented May 8, 2020 •

edited by nmanovic

Loading

codewithunknown commented Mar 3, 2024 •

edited

Loading