".jpeg" extension trigger an error

[Azraell]

My actions before raising this issue

• Read/searched the docs
• Searched past issues

Expected Behaviour

Exporting annotations shouldn't fail because of image filename. Task creation and image import must failed with precise error.

Current Behaviour

It seems that I've go a task completly messed up by images with ".jpeg" extension in filename, (ex: 1e7ca95c572a8227070dc68badba1e18-79860.241744.jpeg).

Possible Solution

If it's the case, a proper filename check test should prevent this error.
This action prevent people to silently break their collection.

Steps to Reproduce (for bugs)

1. create a task with a picture named "1e7ca95c572a8227070dc68badba1e18-79860.241744.jpeg"
2. do an annotation
3. export as tfexport (to trigger an error)
4. check that the export is broken

Context

We've got a worklow that generate image name with .jpeg extension, trying to export those file trigger error.

Your Environment

• Git hash commit (git log -1):
• Docker version docker version (e.g. Docker 17.0.05):
• Are you using Docker Swarm or Kubernetes?
• Operating System and version (e.g. Linux, Windows, MacOS):
• Code example or link to GitHub repo or gist to reproduce problem:
• Other diagnostic information / logs:
  Logs from `cvat` container
• git log -1
  Author: Andrey Zhavoronkov [email protected]
  Date: Sat Oct 3 07:47:26 2020 +0300
  Rename git app (#2243)
  • renamed application git -> git_manager
  • removed _GitImportFix class
  • fixed cli tests
  • git_manager -> dataset_repo
  • updated changelog
  • fixed dataset manager tests
• docker version is unrelated
• docker local
• linux fedora 32
• related logs :

cvat_proxy    | 172.28.0.1 - - [06/Oct/2020:18:29:16 +0000] "GET /api/v1/tasks/5/dataset?format=TFRecord%201.0 HTTP/1.1" 202 0 "http://localhost:8080/tasks/5/jobs/5" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36"
cvat          | 2020-10-06 18:29:16,972 DEBG 'rqworker_default_0' stderr output:
cvat          | DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 480 seconds.
cvat          | 
cvat          | 2020-10-06 18:29:16,979 DEBG 'rqworker_default_0' stderr output:
cvat          | DEBUG:rq.worker:Sent heartbeat to prevent worker timeout. Next one should arrive within 90 seconds.
cvat          | 
cvat          | 2020-10-06 18:29:17,064 DEBG 'rqworker_default_0' stderr output:
**cvat          | WARNING:root:Item '540d5b824f715a53d994389147ceeb53-80219.032895': can't find format string for the '.jpeg' image extension, the corresponding field will be empty.**
cvat          | 
cvat          | 2020-10-06 18:29:17,508 DEBG 'rqworker_default_0' stderr output:
cvat          | [2020-10-06 18:29:17,507] ERROR cvat.server.task_5: [cvat.apps.dataset_manager.views @ export_task]: exception occurred
cvat          | Traceback (most recent call last):
cvat          |   File "/home/django/cvat/apps/dataset_manager/views.py", line 61, in export_task
cvat          |     task.export_task(task_id, temp_file, dst_format,
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 748, in export_task
cvat          |     task.export(f, exporter, host=server_url, save_images=save_images)
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 630, in export
cvat          |     exporter(dst_file, task_data, **options)
cvat          |   File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in __call__
cvat          |     f_or_cls(*args, **kwargs)
cvat          |   File "/home/django/cvat/apps/dataset_manager/formats/tfrecord.py", line 30, in _export
cvat          |     dm_env.converters.get('tf_detection_api').convert(extractor,
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/components/converter.py", line 37, in convert
cvat          |     return converter.apply()
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/plugins/tf_detection_api_format/converter.py", line 100, in apply
cvat          |     tf_example = self._make_tf_example(item)
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/plugins/tf_detection_api_format/converter.py", line 194, in _make_tf_example
cvat          |     'image/format': bytes_feature(fmt.encode('utf-8')),
cvat          | AttributeError: 'NoneType' object has no attribute 'encode'
cvat          | 
cvat          | 2020-10-06 18:29:17,508 DEBG 'rqworker_default_0' stderr output:
cvat          | ERROR:cvat.server.task_5:[cvat.apps.dataset_manager.views @ export_task]: exception occurred
cvat          | Traceback (most recent call last):
cvat          |   File "/home/django/cvat/apps/dataset_manager/views.py", line 61, in export_task
cvat          |     task.export_task(task_id, temp_file, dst_format,
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 748, in export_task
cvat          |     task.export(f, exporter, host=server_url, save_images=save_images)
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 630, in export
cvat          |     exporter(dst_file, task_data, **options)
cvat          |   File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in __call__
cvat          |     f_or_cls(*args, **kwargs)
cvat          |   File "/home/django/cvat/apps/dataset_manager/formats/tfrecord.py", line 30, in _export
cvat          |     dm_env.converters.get('tf_detection_api').convert(extractor,
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/components/converter.py", line 37, in convert
cvat          |     return converter.apply()
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/plugins/tf_detection_api_format/converter.py", line 100, in apply
cvat          |     tf_example = self._make_tf_example(item)
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/plugins/tf_detection_api_format/converter.py", line 194, in _make_tf_example
cvat          |     'image/format': bytes_feature(fmt.encode('utf-8')),
cvat          | AttributeError: 'NoneType' object has no attribute 'encode'
cvat          | 
cvat          | 2020-10-06 18:29:17,508 DEBG 'rqworker_default_0' stderr output:
cvat          | DEBUG:rq.worker:Handling failed execution of job /api/v1/tasks/5/dataset/TFRecord 1.0
cvat          | 
cvat          | 2020-10-06 18:29:17,510 DEBG 'rqworker_default_0' stderr output:
cvat          | ERROR:rq.worker:Traceback (most recent call last):
cvat          |   File "/usr/local/lib/python3.8/dist-packages/rq/worker.py", line 936, in perform_job
cvat          |     rv = job.perform()
cvat          |   File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 684, in perform
cvat          |     self._result = self._execute()
cvat          |   File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 690, in _execute
cvat          |     return self.func(*self.args, **self.kwargs)
cvat          |   File "/home/django/cvat/apps/dataset_manager/views.py", line 84, in export_task_as_dataset
cvat          |     return export_task(task_id, dst_format, server_url=server_url, save_images=True)
cvat          |   File "/home/django/cvat/apps/dataset_manager/views.py", line 61, in export_task
cvat          |     task.export_task(task_id, temp_file, dst_format,
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 748, in export_task
cvat          |     task.export(f, exporter, host=server_url, save_images=save_images)
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 630, in export
cvat          |     exporter(dst_file, task_data, **options)
cvat          |   File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in __call__
cvat          |     f_or_cls(*args, **kwargs)
cvat          |   File "/home/django/cvat/apps/dataset_manager/formats/tfrecord.py", line 30, in _export
cvat          |     dm_env.converters.get('tf_detection_api').convert(extractor,
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/components/converter.py", line 37, in convert
cvat          |     return converter.apply()
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/plugins/tf_detection_api_format/converter.py", line 100, in apply
cvat          |     tf_example = self._make_tf_example(item)
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/plugins/tf_detection_api_format/converter.py", line 194, in _make_tf_example
cvat          |     'image/format': bytes_feature(fmt.encode('utf-8')),
cvat          | AttributeError: 'NoneType' object has no attribute 'encode'
cvat          | Traceback (most recent call last):
cvat          |   File "/usr/local/lib/python3.8/dist-packages/rq/worker.py", line 936, in perform_job
cvat          |     rv = job.perform()
cvat          |   File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 684, in perform
cvat          |     self._result = self._execute()
cvat          |   File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 690, in _execute
cvat          |     return self.func(*self.args, **self.kwargs)
cvat          |   File "/home/django/cvat/apps/dataset_manager/views.py", line 84, in export_task_as_dataset
cvat          |     return export_task(task_id, dst_format, server_url=server_url, save_images=True)
cvat          |   File "/home/django/cvat/apps/dataset_manager/views.py", line 61, in export_task
cvat          |     task.export_task(task_id, temp_file, dst_format,
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 748, in export_task
cvat          |     task.export(f, exporter, host=server_url, save_images=save_images)
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 630, in export
cvat          |     exporter(dst_file, task_data, **options)
cvat          |   File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in __call__
cvat          |     f_or_cls(*args, **kwargs)
cvat          |   File "/home/django/cvat/apps/dataset_manager/formats/tfrecord.py", line 30, in _export
cvat          |     dm_env.converters.get('tf_detection_api').convert(extractor,
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/components/converter.py", line 37, in convert
cvat          |     return converter.apply()
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/plugins/tf_detection_api_format/converter.py", line 100, in apply
cvat          |     tf_example = self._make_tf_example(item)
cvat          |   File "/usr/local/lib/python3.8/dist-packages/datumaro/plugins/tf_detection_api_format/converter.py", line 194, in _make_tf_example
cvat          |     'image/format': bytes_feature(fmt.encode('utf-8')),
cvat          | AttributeError: 'NoneType' object has no attribute 'encode'
cvat          | 
cvat          | 2020-10-06 18:29:17,511 DEBG 'rqworker_default_0' stderr output:
cvat          | DEBUG:rq.worker:Invoking exception handler <function rq_handler at 0x7f9b36157e50>

Next steps

You may join our Gitter channel for community support.