Manifest optimization

[Marishka17]

Resolve #2993

Motivation and context

• added progress bar for manifest creating
• reduced manifest creating time

How has this been tested?

Manually, old tests work as expected

Checklist

• I submit my changes into the develop branch
• I have added description of my changes into CHANGELOG file
  - [ ] I have updated the documentation accordingly
  - [ ] I have added tests to cover my changes
• I have linked related issues (read github docs)
  - [ ] I have increased versions of npm packages if it is necessary (cvat-canvas,
  cvat-core, cvat-data and cvat-ui)

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
  - [ ] I have updated the license header for each file (see an example below)

# Copyright (C) 2021 Intel Corporation
#
# SPDX-License-Identifier: MIT

[nmanovic]

@Marishka17 , it will be great to have a line in CHANGELOG. Probably you can estimate some speed up which we should get in some scenarios. It will be helpful as well.

[nmanovic]

@Marishka17 , need to add tqdm module into requirements

Traceback (most recent call last):
  File "./create.py", line 91, in <module>
    from dataset_manifest.core import VideoManifestManager, ImageManifestManager
  File "/mnt/data.ssd1tb/workspace/cvat/utils/dataset_manifest/__init__.py", line 4, in <module>
    from .core import VideoManifestManager, ImageManifestManager
  File "/mnt/data.ssd1tb/workspace/cvat/utils/dataset_manifest/core.py", line 13, in <module>
    from tqdm import tqdm
ModuleNotFoundError: No module named 'tqdm'

[nmanovic]

It is better don't use tqdm inside a library. It is a good solution for create.py. What do you think?

[Marishka17]

I think that need to use tqdm where an iteration directly takes place. It seems to me not a very good solution to raise an iteration to a higher level.
If we will use tqdm in create.py, we should change manifest.create() to for _ in manifest.create(): pass in all places where manifest.create() is used, or expand existing API. What do you think about it?

[nmanovic]

Does it spam CVAT server logs with unnecessary messages?

[nmanovic]

In general I don't advice a solution. I just want to say that the general rule is to use tqdm inside CLI not in a library.

[nmanovic]

You can pass an optional object into the create method. Thus CLI will pass tqdm like object. If the code is imported, the object will be None.