increase maximum number of cloud storage attached files

[patrickwasp]

My actions before raising this issue

• [ x] Read/searched the docs
• [x ] Searched past issues

Could not create the task
Error: Could not create the task on the server. 'The maximum number of the cloud storage attached files is 500'.

Steps to Reproduce (for bugs)

1. Create a new task
2. Select "Cloud Storage" tab
3. Select cloud storage
4. Select data source
5. check the folder. it currently has ~100k files. In the future, it will have ~100M.

Expected Behaviour

all files within the cloud storage folder are added into the task

Current Behaviour

A message pops up stating "'The maximum number of the cloud storage attached files is 500'"
https://github.com/opencv/cvat/blob/fd2575c7fba5a449232e574a1a46de7f34591c87/cvat/settings/base.py#L519

Possible Solution

add a environmental variable or setting to increase the maximum number of cloud storage attached files to an arbitrarily large number.

Context

I have many more than 500 images to label. Creating multiple tasks to include all the images is difficult. There is no way to know how many images I selected when checking a folder.

Your Environment

v 2.4.6 release tar, docker compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml up on ubuntu 22.04 x86. Docker version 24.0.2, build cb74dfc

[bsekachev]

#6292

Personally I do not recommend to send 100M images to the server.
Probably task will not be created, better to select a parent directory only.

Related issue #4822

[Pan-will]

I ran into the same need and problem!
Did you fix this problem by changing the value of LOCAL_LOAD_MAX_FILES_COUNT?
#6628 (comment)

[mpaletou]

@Pan-will did you found a workaround by any chances for this issue ?

We experience the same issue and are unable to fix it by settings CLOUD_STORAGE_MAX_FILES_COUNT in environment variable.