Skip to content

Commit

Permalink
Allowed to upload .xml and .json annotation files from UI (#5386)
Browse files Browse the repository at this point in the history
<!-- Raised an issue to propose your change
(https://github.com/cvat-ai/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the
[CONTRIBUTION](https://github.com/cvat-ai/cvat/blob/develop/CONTRIBUTING.md)
guide. -->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
<!-- Why is this change required? What problem does it solve? If it
fixes an open
issue, please link to the issue here. Describe your changes in detail,
add
screenshots. -->
Resolved #5274
+ Fixed typo in tests

### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable by a reason then ~~explicitly
strikethrough~~ the whole
line. If you don't do that github will show an incorrect process for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have added a description of my changes into
[CHANGELOG](https://github.com/cvat-ai/cvat/blob/develop/CHANGELOG.md)
file
- [ ] I have updated the [documentation](
https://github.com/cvat-ai/cvat/blob/develop/README.md#documentation)
accordingly
- [x] I have added tests to cover my changes
- [x] I have linked related issues ([read github docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [x] I have increased versions of npm packages if it is necessary
([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning),
[cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning)
and
[cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.
  • Loading branch information
klakhov authored Nov 30, 2022
1 parent d8c5051 commit 19b2643
Show file tree
Hide file tree
Showing 18 changed files with 135 additions and 44 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ non-ascii paths while adding files from "Connected file share" (issue #4428)
- Missed token with using social account authentication (<https://github.com/opencv/cvat/pull/5344>)
- Fixed FBRS serverless function runtime error on images with alpha channel (<https://github.com/opencv/cvat/pull/5384>)
- Attaching manifest with custom name (<https://github.com/opencv/cvat/pull/5377>)
- Uploading non-zip annotaion files (<https://github.com/opencv/cvat/pull/5386>)

### Security
- TDB
Expand Down
2 changes: 1 addition & 1 deletion cvat-core/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "cvat-core",
"version": "7.2.1",
"version": "7.2.2",
"description": "Part of Computer Vision Tool which presents an interface for client-side integration",
"main": "src/api.ts",
"scripts": {
Expand Down
21 changes: 17 additions & 4 deletions cvat-core/src/annotations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -305,11 +305,24 @@ export function importDataset(
if (!(typeof convMaskToPoly === 'boolean')) {
throw new ArgumentError('Option "convMaskToPoly" must be a boolean');
}
if (typeof file === 'string' && !file.toLowerCase().endsWith('.zip')) {
throw new ArgumentError('File must be file instance with ZIP extension');
const allowedFileExtensions = [
'.zip', '.xml', '.json',
];
const allowedFileExtensionsList = allowedFileExtensions.join(', ');
if (typeof file === 'string' && !(allowedFileExtensions.some((ext) => file.toLowerCase().endsWith(ext)))) {
throw new ArgumentError(
`File must be file instance with one of the following extensions: ${allowedFileExtensionsList}`,
);
}
if (file instanceof File && !(['application/zip', 'application/x-zip-compressed'].includes(file.type))) {
throw new ArgumentError('File must be file instance with ZIP extension');
const allowedMimeTypes = [
'application/zip', 'application/x-zip-compressed',
'application/xml', 'text/xml',
'application/json',
];
if (file instanceof File && !(allowedMimeTypes.includes(file.type))) {
throw new ArgumentError(
`File must be file instance with one of the following extensions: ${allowedFileExtensionsList}`,
);
}

if (instance instanceof Project) {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (C) 2021-2022 Intel Corporation
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

Expand Down Expand Up @@ -39,7 +40,7 @@ context('Export project dataset.', { browser: '!firefox' }, () => {
function checkCounTasksInXML(projectParams, expectedCount) {
cy.exportProject(projectParams);
cy.waitForDownload();
cy.unpackZipArchive(`cypress/fixtures/${projectParams.archiveCustomeName}.zip`);
cy.unpackZipArchive(`cypress/fixtures/${projectParams.archiveCustomName}.zip`);
cy.readFile('cypress/fixtures/annotations.xml').should('exist').then((xml) => {
const tasks = Cypress.$(Cypress.$.parseXML(xml)).find('task').find('name');
expect(tasks.length).to.be.eq(expectedCount);
Expand Down Expand Up @@ -120,7 +121,7 @@ context('Export project dataset.', { browser: '!firefox' }, () => {
as: 'exportAnnotationsRenameArchive',
type: 'annotations',
dumpType: 'CVAT for images',
archiveCustomeName: 'export_project_annotation',
archiveCustomName: 'export_project_annotation',
};
// Check issue 3810
checkCounTasksInXML(exportAnnotationsRenameArchive, 2);
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (C) 2021-2022 Intel Corporation
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

Expand Down Expand Up @@ -86,7 +87,7 @@ context('Export project dataset with 3D task.', { browser: '!firefox' }, () => {
as: 'exportAnnotations3dRenameArchive',
type: 'annotations',
dumpType: 'Datumaro 3D',
archiveCustomeName: 'export_project_3d_annotation',
archiveCustomName: 'export_project_3d_annotation',
};
cy.exportProject(exportAnnotations3dRenameArchive);
cy.waitForDownload();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (C) 2021-2022 Intel Corporation
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

Expand Down Expand Up @@ -36,7 +37,7 @@ context('Dump/Upload annotation.', { browser: '!firefox' }, () => {

const exportFormat = 'CVAT for images';
let annotationArchiveName = '';
let annotationArchiveNameCustomeName = '';
let annotationArchiveNameCustomName = '';

function uploadToTask(toTaskName) {
cy.contains('.cvat-item-task-name', toTaskName)
Expand All @@ -47,8 +48,8 @@ context('Dump/Upload annotation.', { browser: '!firefox' }, () => {
cy.get('.cvat-modal-import-dataset').find('.cvat-modal-import-select').click();
cy.contains('.cvat-modal-import-dataset-option-item', exportFormat.split(' ')[0]).click();
cy.get('.cvat-modal-import-select').should('contain.text', exportFormat.split(' ')[0]);
cy.get('input[type="file"]').attachFile(annotationArchiveNameCustomeName, { subjectType: 'drag-n-drop' });
cy.get(`[title="${annotationArchiveNameCustomeName}"]`).should('be.visible');
cy.get('input[type="file"]').attachFile(annotationArchiveNameCustomName, { subjectType: 'drag-n-drop' });
cy.get(`[title="${annotationArchiveNameCustomName}"]`).should('be.visible');
cy.contains('button', 'OK').click();
}

Expand All @@ -72,12 +73,12 @@ context('Dump/Upload annotation.', { browser: '!firefox' }, () => {
as: 'exportAnnotationsRenameArchive',
type: 'annotations',
format: exportFormat,
archiveCustomeName: 'task_export_annotation_custome_name',
archiveCustomName: 'task_export_annotation_custome_name',
};
cy.exportJob(exportAnnotationRenameArchive);
cy.getDownloadFileName().then((file) => {
annotationArchiveNameCustomeName = file;
cy.verifyDownload(annotationArchiveNameCustomeName);
annotationArchiveNameCustomName = file;
cy.verifyDownload(annotationArchiveNameCustomName);
});
cy.verifyNotification();
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ context('Tests for source and target storage.', () => {
const exportParams = {
type: 'annotations',
format,
archiveCustomeName: 'job_annotations',
archiveCustomName: 'job_annotations',
targetStorage: project.advancedConfiguration.targetStorage,
};
cy.exportJob(exportParams);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ context('Tests for source and target storage.', () => {
const exportParams = {
type: 'annotations',
format,
archiveCustomeName: 'job_annotations',
archiveCustomName: 'job_annotations',
targetStorage: project.advancedConfiguration.targetStorage,
};
cy.exportJob(exportParams);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ context('Import and export annotations: specify source and target storage in mod
const exportParams = {
type: 'annotations',
format,
archiveCustomeName: 'job_annotations',
archiveCustomName: 'job_annotations',
targetStorage: {
location: 'Cloud storage',
cloudStorageId: createdCloudStorageId,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (C) 2021-2022 Intel Corporation
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

Expand Down Expand Up @@ -41,7 +42,7 @@ context('Export task dataset.', () => {
as: 'exportDatasetRenameArchive',
type: 'dataset',
format: exportFormat,
archiveCustomeName: 'job_export_dataset_custome_name',
archiveCustomName: 'job_export_dataset_custome_name',
};
cy.exportJob(exportDataset);
cy.waitForDownload();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (C) 2021-2022 Intel Corporation
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

Expand All @@ -13,7 +14,7 @@ context('Canvas 3D functionality. Dump/upload annotation. "Point Cloud" format',
};
const dumpTypePC = 'Sly Point Cloud Format';
let annotationPCArchiveName = '';
let annotationPCArchiveCustomeName = '';
let annotationPCArchiveCustomName = '';

function confirmUpdate(modalWindowClassName) {
cy.get(modalWindowClassName).should('be.visible').within(() => {
Expand Down Expand Up @@ -63,12 +64,12 @@ context('Canvas 3D functionality. Dump/upload annotation. "Point Cloud" format',
as: 'exportAnnotationsRenameArchive',
type: 'annotations',
format: dumpTypePC,
archiveCustomeName: 'job_export_3d_annotation_custome_name_pc_format',
archiveCustomName: 'job_export_3d_annotation_custome_name_pc_format',
};
cy.exportJob(exportAnnotationRenameArchive);
cy.getDownloadFileName().then((file) => {
annotationPCArchiveCustomeName = file;
cy.verifyDownload(annotationPCArchiveCustomeName);
annotationPCArchiveCustomName = file;
cy.verifyDownload(annotationPCArchiveCustomName);
});
cy.verifyNotification();
cy.removeAnnotations();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (C) 2021-2022 Intel Corporation
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

Expand All @@ -13,7 +14,7 @@ context('Canvas 3D functionality. Dump/upload annotation. "Velodyne Points" form
};
const dumpTypeVC = 'Kitti Raw Format';
let annotationVCArchiveName = '';
let annotationVCArchiveNameCustomeName = '';
let annotationVCArchiveNameCustomName = '';

function confirmUpdate(modalWindowClassName) {
cy.get(modalWindowClassName).should('be.visible').within(() => {
Expand Down Expand Up @@ -63,12 +64,12 @@ context('Canvas 3D functionality. Dump/upload annotation. "Velodyne Points" form
as: 'exportAnnotationsRenameArchive',
type: 'annotations',
format: dumpTypeVC,
archiveCustomeName: 'job_export_3d_annotation_custome_name_vc_format',
archiveCustomName: 'job_export_3d_annotation_custome_name_vc_format',
};
cy.exportJob(exportAnnotationRenameArchive);
cy.getDownloadFileName().then((file) => {
annotationVCArchiveNameCustomeName = file;
cy.verifyDownload(annotationVCArchiveNameCustomeName);
annotationVCArchiveNameCustomName = file;
cy.verifyDownload(annotationVCArchiveNameCustomName);
});
cy.verifyNotification();
cy.removeAnnotations();
Expand Down Expand Up @@ -101,7 +102,7 @@ context('Canvas 3D functionality. Dump/upload annotation. "Velodyne Points" form
cy.contains('Upload annotations').click();
uploadAnnotation(
dumpTypeVC.split(' ')[0],
annotationVCArchiveNameCustomeName,
annotationVCArchiveNameCustomName,
'.cvat-modal-content-load-task-annotation',
);
cy.contains('Annotations have been loaded').should('be.visible');
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Copyright (C) 2021-2022 Intel Corporation
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

Expand Down Expand Up @@ -50,7 +51,7 @@ context('Canvas 3D functionality. Export as a dataset.', () => {
as: 'exportDatasetVCFormatRenameArchive',
type: 'dataset',
format: dumpTypeVC,
archiveCustomeName: 'job_export_3d_dataset_custome_name_vc_format',
archiveCustomName: 'job_export_3d_dataset_custome_name_vc_format',
};
cy.exportJob(exportDatasetVCFormatRenameArchive);
cy.waitForDownload();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

/// <reference types="cypress" />

import { taskName, labelName } from '../../support/const';

context('Upload annotations in different file formats', () => {
const issueId = '5274';
const createRectangleTrack2Points = {
points: 'By 2 Points',
type: 'Track',
labelName,
firstX: 250,
firstY: 350,
secondX: 350,
secondY: 450,
};

const archives = [
{
type: 'annotations',
format: 'CVAT for images',
archiveCustomName: 'issue5274-cvat-for-images',
annotationsPath: 'annotations.xml',
},
{
type: 'annotations',
format: 'COCO',
archiveCustomName: 'issue5274-coco',
annotationsPath: 'annotations/instances_default.json',
},
];

before(() => {
cy.openTaskJob(taskName);
cy.createRectangle(createRectangleTrack2Points);
cy.saveJob('PATCH', 200, 'saveJobDump');
for (const archive of archives) {
cy.exportJob(archive);
cy.waitForDownload();
cy.unpackZipArchive(`cypress/fixtures/${archive.archiveCustomName}.zip`, archive.archiveCustomName);
}
});

describe(`Testing issue "${issueId}"`, () => {
it('Dump annotations. Upload different file formats.', () => {
cy.removeAnnotations();
cy.intercept('GET', '/api/jobs/**/annotations?**').as('uploadAnnotationsGet');
for (const archive of archives) {
cy.interactMenu('Upload annotations');
cy.uploadAnnotations(
archive.format.split(' ')[0],
`${archive.archiveCustomName}/${archive.annotationsPath}`,
'.cvat-modal-content-load-job-annotation',
);

cy.get('.cvat-notification-notice-upload-annotations-fail').should('not.exist');
cy.get('#cvat_canvas_shape_1').should('exist');
cy.get('#cvat-objects-sidebar-state-item-1').should('exist');

cy.removeAnnotations();
}
});
});
});
5 changes: 3 additions & 2 deletions tests/cypress/plugins/unpackZipArchive/addPlugin.js
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
// Copyright (C) 2021-2022 Intel Corporation
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

const path = require('path');
const extract = require('extract-zip');

async function unpackZipArchive(args) {
const { arhivePath } = args;
const { arhivePath, extractPath } = args;
const absolutePath = path.dirname(path.resolve(arhivePath));
await extract(arhivePath, { dir: absolutePath });
await extract(arhivePath, { dir: extractPath ? `${absolutePath}/${extractPath}/` : absolutePath });
return null;
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
// Copyright (C) 2021-2022 Intel Corporation
// Copyright (C) 2022 CVAT.ai Corporation
//
// SPDX-License-Identifier: MIT

Cypress.Commands.add('unpackZipArchive', (arhivePath) => cy.task('unpackZipArchive', {
Cypress.Commands.add('unpackZipArchive', (arhivePath, extractPath) => cy.task('unpackZipArchive', {
arhivePath,
extractPath,
}));
Loading

0 comments on commit 19b2643

Please sign in to comment.