Add REST API GET, POST, PATCH tests for cloud storage #4353

Marishka17 · 2022-02-16T20:55:21Z

Motivation and context

PR contains REST API tests for cloud storage functionality
Resolves #3996

How has this been tested?

Checklist

I submit my changes into the develop branch
I have added a description of my changes into CHANGELOG file
~~- [ ] I have updated the documentation accordingly~~
I have added tests to cover my changes
~~- [ ] I have linked related issues (read github docs)~~
- [ ] I have increased versions of npm packages if it is necessary (cvat-canvas,
cvat-core, cvat-data and cvat-ui)

License

I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
I have updated the license header for each file (see an example below)

# Copyright (C) 2022 Intel Corporation
#
# SPDX-License-Identifier: MIT

nmanovic · 2022-02-18T08:33:54Z

@Marishka17 , could you please merge develop and share your experience about the process? I see that you have conflicts with cvat_data.tar.bz2, cvat_db.sql.

nmanovic · 2022-02-18T08:35:19Z

@kirill-sizov , could you please review the patch and provide your suggestions.

sizov-kirill · 2022-02-18T14:42:57Z

@Marishka17 When I run these tests locally they fails, for example this test failed due to status code is 400

FAILED test_cloud_storages.py::TestPostCloudStorage::test_sandbox_user_create_cloud_storage[user-True]

Am I doing something wrong? Maybe I need to change something in my setup? Have you met this problem?

Marishka17 · 2022-02-18T20:14:09Z

@Marishka17 When I run these tests locally they fails, for example this test failed due to status code is 400
FAILED test_cloud_storages.py::TestPostCloudStorage::test_sandbox_user_create_cloud_storage[user-True]
Am I doing something wrong? Maybe I need to change something in my setup?

You need to export TESTING environment variable and run docker containers with -f docker-compose.testing.yml option.

sizov-kirill · 2022-02-20T20:55:08Z

tests/rest_api/test_cloud_storages.py

+    @pytest.mark.parametrize('org_id', [2])
+    @pytest.mark.parametrize('storage_id', [2])
+    @pytest.mark.parametrize('role, is_owner, is_allow', [
+        #('worker',       True,   True),


Is there any reason to skip this test?

cvat/apps/engine/serializers.py

.github/workflows/main.yml

Marishka17 · 2022-03-16T08:53:31Z

@kirill-sizov, Could you please help and resolve conflicts with assets?

Marishka17 · 2022-03-16T09:17:34Z

@nmanovic, I've changed approach with test cloud storage functionality. Main idea is use Multi-Cloud Object Storage. Now, before running REST API tests MinIO server is started and MinIO client creates several buckets locally that are used in tests.
I'm removing WIP status now because I've mostly done all (maybe I will do only some cosmetics edits) and want to be sure that tests will be passed on CI as locally after resolving conflicts.

sizov-kirill · 2022-03-16T12:02:02Z

@kirill-sizov, Could you please help and resolve conflicts with assets?

Yes sure, I will resolve conflicts.

…oolkit/cvat into mk/cloud_storage_tests

tests/rest_api/setup

tests/rest_api/test_cloud_storages.py

Co-authored-by: Kirill Sizov <[email protected]>

…oolkit/cvat into mk/cloud_storage_tests

sizov-kirill

Looks good to me.

nmanovic

@Marishka17 , great job!

Co-authored-by: kirill.sizov <[email protected]>

Marishka17 added 2 commits February 16, 2022 23:52

Add test

620a6c3

Update assets

9ba023d

Marishka17 requested a review from nmanovic February 16, 2022 20:55

Marishka17 requested a review from azhavoro as a code owner February 16, 2022 20:55

Marishka17 changed the title ~~Add REST API GET, POST, PATCH tests for cloud storage~~ [WIP] Add REST API GET, POST, PATCH tests for cloud storage Feb 17, 2022

Fix env

9bbf0de

Marishka17 changed the title ~~[WIP] Add REST API GET, POST, PATCH tests for cloud storage~~ Add REST API GET, POST, PATCH tests for cloud storage Feb 17, 2022

Merge branch 'develop' into mk/cloud_storage_tests

015dfea

nmanovic requested a review from sizov-kirill February 18, 2022 08:33

Marishka17 added 3 commits February 18, 2022 13:06

Remove extra

3a7012b

Merge branch 'develop'

7d29d3d

Upfdate assets

9b00be4

Update readme

20a746a

sizov-kirill reviewed Feb 20, 2022

View reviewed changes

nmanovic reviewed Feb 21, 2022

View reviewed changes

cvat/apps/engine/serializers.py Outdated Show resolved Hide resolved

nmanovic reviewed Feb 21, 2022

View reviewed changes

.github/workflows/main.yml Outdated Show resolved Hide resolved

Marishka17 changed the title ~~Add REST API GET, POST, PATCH tests for cloud storage~~ [WIP] Add REST API GET, POST, PATCH tests for cloud storage Feb 24, 2022

Marishka17 added 3 commits March 11, 2022 10:32

Add draft support for endpoint url

3a7e0ee

Add MINIO setup

8a1cc7b

Update minio setup

7d69823

Marishka17 requested a review from bsekachev as a code owner March 14, 2022 08:00

Marishka17 added 5 commits March 14, 2022 13:09

Update assets

c13c0b4

Merge branch 'develop' into mk/cloud_storage_tests

0af107c

Update migration dependence

f5f6d39

Update setup

2b3ff87

Remove extra code from previous testing approach

de59044

Update assets

5df9f94

Marishka17 changed the title ~~[WIP] Add REST API GET, POST, PATCH tests for cloud storage~~ Add REST API GET, POST, PATCH tests for cloud storage Mar 16, 2022

Marishka17 added 3 commits March 17, 2022 09:46

Resolve conflicts

413e7c5

Update assets

95db76b

Update workflow & docker-compose conf

0160d1b

Marishka17 force-pushed the mk/cloud_storage_tests branch from ff0087b to 8091cd9 Compare March 18, 2022 08:18

t

4292b34

Marishka17 force-pushed the mk/cloud_storage_tests branch from 8091cd9 to 4292b34 Compare March 18, 2022 08:47

Marishka17 and others added 5 commits March 18, 2022 12:41

fix setting env on github actions

391f4d3

Remove debug info

7d9c635

Merge branch 'develop' into mk/cloud_storage_tests

6252e51

Merge develop into the mk/cloud_storage_tests

a33649c

Merge branch 'mk/cloud_storage_tests' of https://github.com/openvinot…

ba710db

…oolkit/cvat into mk/cloud_storage_tests

sizov-kirill reviewed Mar 22, 2022

View reviewed changes

tests/rest_api/setup Outdated Show resolved Hide resolved

tests/rest_api/setup Outdated Show resolved Hide resolved

tests/rest_api/test_cloud_storages.py Outdated Show resolved Hide resolved

tests/rest_api/test_cloud_storages.py Outdated Show resolved Hide resolved

Marishka17 and others added 5 commits March 23, 2022 11:12

Make a cross platform solution

3e58d0f

Update tests/rest_api/test_cloud_storages.py

992989b

Co-authored-by: Kirill Sizov <[email protected]>

Update tests/rest_api/test_cloud_storages.py

caa0247

Co-authored-by: Kirill Sizov <[email protected]>

fix mistake

88fc98f

Merge branch 'mk/cloud_storage_tests' of https://github.com/openvinot…

c9a5f59

…oolkit/cvat into mk/cloud_storage_tests

sizov-kirill approved these changes Mar 24, 2022

View reviewed changes

nmanovic approved these changes Mar 24, 2022

View reviewed changes

nmanovic merged commit 2a05316 into develop Mar 24, 2022

nmanovic deleted the mk/cloud_storage_tests branch March 24, 2022 12:08

amteusch pushed a commit to amteusch/cvat that referenced this pull request Mar 24, 2022

Add REST API GET, POST, PATCH tests for cloud storage (cvat-ai#4353)

575fd7e

Co-authored-by: kirill.sizov <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add REST API GET, POST, PATCH tests for cloud storage #4353

Add REST API GET, POST, PATCH tests for cloud storage #4353

Marishka17 commented Feb 16, 2022 •

edited

Loading

nmanovic commented Feb 18, 2022

nmanovic commented Feb 18, 2022

sizov-kirill commented Feb 18, 2022 •

edited

Loading

Marishka17 commented Feb 18, 2022

sizov-kirill Feb 20, 2022

Marishka17 commented Mar 16, 2022

Marishka17 commented Mar 16, 2022 •

edited

Loading

sizov-kirill commented Mar 16, 2022

sizov-kirill left a comment

nmanovic left a comment

Add REST API GET, POST, PATCH tests for cloud storage #4353

Add REST API GET, POST, PATCH tests for cloud storage #4353

Conversation

Marishka17 commented Feb 16, 2022 • edited Loading

Motivation and context

How has this been tested?

Checklist

License

nmanovic commented Feb 18, 2022

nmanovic commented Feb 18, 2022

sizov-kirill commented Feb 18, 2022 • edited Loading

Marishka17 commented Feb 18, 2022

sizov-kirill Feb 20, 2022

Choose a reason for hiding this comment

Marishka17 commented Mar 16, 2022

Marishka17 commented Mar 16, 2022 • edited Loading

sizov-kirill commented Mar 16, 2022

sizov-kirill left a comment

Choose a reason for hiding this comment

nmanovic left a comment

Choose a reason for hiding this comment

Marishka17 commented Feb 16, 2022 •

edited

Loading

sizov-kirill commented Feb 18, 2022 •

edited

Loading

Marishka17 commented Mar 16, 2022 •

edited

Loading