Couldn't create superuser

[KurochkinVasiliy]

Hello!
I'm trying to install CVAT using "Quick installation guide".
Having troubles on this step:
docker exec -it cvat bash -ic 'python3 ~/manage.py createsuperuser'
Nothing happens if i press enter

Expected Behaviour

I expect that system asks me for username and password of superuser

Current Behaviour

nothing happens.
if i login to cvat
docker exec -it cvat bash
and then run
python3 ~/manage.py createsuperuser
i see "Illegal instruction (core dumped)" error

Your Environment

• Git hash commit (git log -1):
  commit 0982ea3 (HEAD -> develop, origin/develop, origin/HEAD)
• Docker version docker version (e.g. Docker 17.0.05):
  Version: 19.03.8
• Are you using Docker Swarm or Kubernetes?
  No, docker community edition
• Operating System and version (e.g. Linux, Windows, MacOS):
  Ubuntu 19.10

[azhavoro]

This could be an issue with pyhash (it is built with avx support). Could you please check if your CPU supports avx instructions?
Related PR to pyhash repository flier/pyfasthash#45

[KurochkinVasiliy]

Looks like CPU supports it.
grep avx /proc/cpuinfo
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 ht syscall nx lm nopl cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 aes avx hypervisor lahf_lm abm pti smep
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 ht syscall nx lm nopl cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 aes avx hypervisor lahf_lm abm pti smep
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 ht syscall nx lm nopl cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 aes avx hypervisor lahf_lm abm pti smep
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 ht syscall nx lm nopl cpuid tsc_known_freq pni ssse3 cx16 sse4_1 sse4_2 aes avx hypervisor lahf_lm abm pti smep

[azhavoro]

I see all instruction sets are supported except avx2, I think this is root cause.

[KurochkinVasiliy]

Is there any workaround?

[NormanTUD]

Still looking for a workaround here too

[NormanTUD]

Little update:

I ran:

docker exec -it cvat bash
for i in $(cat /opt/venv/lib/python3.8/site-packages/**/*top_level* | sed -e 's#/#.#g'); do python -c "import $i" || echo "Failed on Module $i"; done

Full log:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named '_ffi'
Failed on Module _ffi
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named '_openssl'
Failed on Module _openssl
The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.
Aborted (core dumped)
Failed on Module keras
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/venv/lib/python3.8/site-packages/nbconvert/__init__.py", line 4, in <module>
    from .exporters import *
  File "/opt/venv/lib/python3.8/site-packages/nbconvert/exporters/__init__.py", line 3, in <module>
    from .html import HTMLExporter
  File "/opt/venv/lib/python3.8/site-packages/nbconvert/exporters/html.py", line 11, in <module>
    from jinja2 import contextfilter
ImportError: cannot import name 'contextfilter' from 'jinja2' (/opt/venv/lib/python3.8/site-packages/jinja2/__init__.py)
Failed on Module nbconvert
Illegal instruction (core dumped)
Failed on Module open3d
The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.
Aborted (core dumped)
Failed on Module tensorflow
The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.
Aborted (core dumped)
Failed on Module tensorflow_io_gcs_filesystem
The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.
Aborted (core dumped)
Failed on Module tensorflow_estimator

to look why some modules and got some

The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.

Yes, this is correct. My CPU does not support it.

I removed the lines open3d and tensorflow from cvat/requirements/base.txt.

Trying to recompile now, but this takes some time. docker-compose -f docker-compose.dev.yml up -d --build --force-recreate

UPDATE #1:
Doing this caused some other files, ie. from libgtk2.0-dev, to miss. Still trying to manually fix all dependencies. Will offer a patch here if I can get it to work.

Currently, the site just says "404 page not found" and has a bunch of log errors regarding all kinds of stuff (docker-compose logs --tail=0 --follow), for example:

cvat          | 2022-07-15 10:44:31,001 INFO spawned: 'git_status_updater' with pid 11937
cvat          | 2022-07-15 10:44:31,014 INFO spawned: 'rqworker_low' with pid 11938
cvat          | 2022-07-15 10:44:31,111 DEBG 'git_status_updater' stderr output:
cvat          | wait-for-it.sh: waiting for cvat_redis:6379 without a timeout
cvat          | 
cvat          | 2022-07-15 10:44:31,118 DEBG 'rqworker_low' stderr output:
cvat          | wait-for-it.sh: waiting for cvat_redis:6379 without a timeout
cvat          | 
cvat          | 2022-07-15 10:44:31,191 DEBG 'rqworker_low' stderr output:
cvat          | wait-for-it.sh: cvat_redis:6379 is available after 0 seconds
cvat          | bash: cannot set terminal process group (-1): Inappropriate ioctl for device
cvat          | bash: no job control in this shell
cvat          | 
cvat          | 2022-07-15 10:44:31,197 DEBG 'git_status_updater' stderr output:
cvat          | wait-for-it.sh: cvat_redis:6379 is available after 0 seconds
cvat          | 
cvat          | 2022-07-15 10:44:31,204 DEBG 'git_status_updater' stderr output:
cvat          | bash: cannot set terminal process group (-1): Inappropriate ioctl for device
cvat          | 
cvat          | 2022-07-15 10:44:31,206 DEBG 'git_status_updater' stderr output:
cvat          | bash: no job control in this shell
cvat          | 
cvat          | 2022-07-15 10:44:31,509 DEBG 'rqworker_default_0' stderr output:
cvat          | Traceback (most recent call last):
cvat          |   File "/home/django/manage.py", line 21, in <module>
cvat          |     execute_from_command_line(sys.argv)
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
cvat          |     utility.execute()
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
cvat          |     django.setup()
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
cvat          |     apps.populate(settings.INSTALLED_APPS)
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate
cvat          |     app_config.import_models()
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/apps/config.py", line 301, in import_models
cvat          |     self.models_module = import_module(models_module_name)
cvat          |   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
cvat          |     return _bootstrap._gcd_import(name[level:], package, level)
cvat          |   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
cvat          |   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
cvat          |   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
cvat          |   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
cvat          |   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
cvat          |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
cvat          |   File "/home/django/cvat/apps/engine/models.py", line 16, in <module>
cvat          |     from cvat.apps.engine.utils import parse_specific_attributes
cvat          |   File "/home/django/cvat/apps/engine/utils.py", line 6, in <module>
cvat          |     import cv2 as cv
cvat          |   File "/opt/venv/lib/python3.8/site-packages/cv2/__init__.py", line 181, in <module>
cvat          |     bootstrap()
cvat          |   File "/opt/venv/lib/python3.8/site-packages/cv2/__init__.py", line 153, in bootstrap
cvat          |     native_module = importlib.import_module("cv2")
cvat          |   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
cvat          |     return _bootstrap._gcd_import(name[level:], package, level)
cvat          | ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
cvat          | 
cvat          | 2022-07-15 10:44:32,185 DEBG 'runserver' stderr output:
cvat          | Traceback (most recent call last):
cvat          |   File "/home/django/manage.py", line 21, in <module>
cvat          |     execute_from_command_line(sys.argv)
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
cvat          |     utility.execute()
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
cvat          |     django.setup()
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
cvat          |     apps.populate(settings.INSTALLED_APPS)
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate
cvat          |     app_config.import_models()
cvat          |   File "/opt/venv/lib/python3.8/site-packages/django/apps/config.py", line 301, in import_models
cvat          |     self.models_module = import_module(models_module_name)
cvat          |   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
cvat          |     return _bootstrap._gcd_import(name[level:], package, level)
cvat          |   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
cvat          |   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
cvat          |   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
cvat          |   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
cvat          |   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
cvat          |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
cvat          |   File "/home/django/cvat/apps/engine/models.py", line 16, in <module>
cvat          |     from cvat.apps.engine.utils import parse_specific_attributes
cvat          |   File "/home/django/cvat/apps/engine/utils.py", line 6, in <module>
cvat          |     import cv2 as cv
cvat          |   File "/opt/venv/lib/python3.8/site-packages/cv2/__init__.py", line 181, in <module>
cvat          |     bootstrap()
cvat          |   File "/opt/venv/lib/python3.8/site-packages/cv2/__init__.py", line 153, in bootstrap
cvat          |     native_module = importlib.import_module("cv2")
cvat          |   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
cvat          |     return _bootstrap._gcd_import(name[level:], package, level)
cvat          | ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
cvat          | 
cvat          | 2022-07-15 10:44:32,186 INFO success: git_status_updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
cvat          | 2022-07-15 10:44:32,186 INFO success: rqworker_low entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
cvat          | 2022-07-15 10:44:32,211 DEBG fd 15 closed, stopped monitoring <POutputDispatcher at 140135299185872 for <Subprocess at 140135299016112 with name rqworker_default_1 in state RUNNING> (stdout)>
cvat          | 2022-07-15 10:44:32,211 DEBG fd 30 closed, stopped monitoring <POutputDispatcher at 140135299185344 for <Subprocess at 140135299016112 with name rqworker_default_1 in state RUNNING> (stderr)>
cvat          | 2022-07-15 10:44:32,212 INFO exited: rqworker_default_1 (exit status 1; not expected)
cvat          | 2022-07-15 10:44:32,212 DEBG received SIGCHLD indicating a child quit

Did not work.

I'm trying to install the non-avx tf==2.8.0 version by doing

RUN python3 -m pip install --upgrade pip && \
        python3 -m pip install https://tf.novaal.de/btver1/tensorflow-2.8.0-cp38-cp38-linux_x86_64.whl

in the Dockerfile. Hopefully this helps, trying to compile now...

Update: together with the recipe file from #3982 ( https://gist.githubusercontent.com/bkbilly/cf4f0618391d267eb5d88dd6272bf911/raw/fe53168bed71e0958c4b5c01c02f1f5837e062ab/docker-cvat.yml ) it now works for me.

Diff:

diff --git a/Dockerfile b/Dockerfile
index 1f6410f5..3e89aecf 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -22,6 +22,10 @@ RUN apt-get update && \
         python3-venv && \
     rm -rf /var/lib/apt/lists/*
 
+RUN python3 -m pip install --upgrade pip
+RUN python3 -m pip install wheel
+
+
 # Compile Openh264 and FFmpeg
 ARG PREFIX=/opt/ffmpeg
 ARG PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
@@ -45,6 +49,7 @@ RUN curl -sL https://ffmpeg.org/releases/ffmpeg-${FFMPEG_VERSION}.tar.bz2 --outp
 # Install requirements
 RUN python3 -m venv /opt/venv
 ENV PATH="/opt/venv/bin:${PATH}"
+RUN python3 -m pip install https://tf.novaal.de/btver1/tensorflow-2.8.0-cp38-cp38-linux_x86_64.whl
 RUN python3 -m pip install --no-cache-dir -U pip==22.0.2 setuptools==60.6.0 wheel==0.37.1
 COPY cvat/requirements/ /tmp/requirements/
 RUN DATUMARO_HEADLESS=1 python3 -m pip install --no-cache-dir -r /tmp/requirements/${DJANGO_CONFIGURATION}.txt
diff --git a/cvat/requirements/base.txt b/cvat/requirements/base.txt
index 0caaa0af..f08b8770 100644
--- a/cvat/requirements/base.txt
+++ b/cvat/requirements/base.txt
@@ -37,7 +37,7 @@ h5py==3.6.0
 django-cors-headers==3.5.0
 furl==2.1.0
 av==8.0.2 --no-binary=av
-tensorflow==2.8.0 # Optional requirement of Datumaro. Use tensorflow-macos==2.8.0 for Mac M1
+# tensorflow==2.8.0 # Optional requirement of Datumaro. Use tensorflow-macos==2.8.0 for Mac M1
 # The package is used by pyunpack as a command line tool to support multiple
 # archives. Don't use as a python module because it has GPL license.
 patool==1.12
diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml
index 893d2ae5..c05a62d7 100644
--- a/docker-compose.dev.yml
+++ b/docker-compose.dev.yml
@@ -15,6 +15,12 @@ services:
         socks_proxy:
         CLAM_AV:
         INSTALL_SOURCES:
+    labels:
+      - traefik.enable=true
+      - traefik.http.services.cvat.loadbalancer.server.port=8080
+      - traefik.http.routers.cvat.rule=Host(185.244.195.220) &&
+        PathPrefix(`/api/`, `/git/`, `/opencv/`, `/static/`, `/admin`, `/documentation/`, `/django-rq`)
+      - traefik.http.routers.cvat.entrypoints=web
 
   cvat_ui:
     build:
diff --git a/docker-compose.yml b/docker-compose.yml
index 2ddb9625..940cb5cd 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -102,6 +102,7 @@ services:
           - opa
     volumes:
       - ./cvat/apps/iam/rules:/rules
+      - /var/run/docker.sock:/var/run/docker.sock:ro
     ports:
       - '8181:8181'
     command: run --server --addr :8181 --set=decision_logs.console=true /rules

In my syslog, there's also this:

Jul 17 17:55:18 v220220776251195133 kernel: [60681.347963] traps: python3[12050] trap invalid opcode ip:7ff6093c076e sp:7ffedbc59390 error:0 in pybind.cpython-38-x86_64-linux-gnu.so[7ff600d2f000+a70b000]

I guess it's still because of the TF depenency.

I'm working on a fork with a very basic installer which tries to fix those things.

Still beta, but may be helpful still to some in my situation (no AVX2 on server, but needs some fix to kind of make it work):

https://raw.githubusercontent.com/NormanTUD/cvat/develop/install.sh

[venki-lfc]

Hi I am facing the same issue in creating the superuser.
Could you please tell me the commands you used after making the adjustments to the Dockerfile, base.txt and docker-compose.yml ? Like what command should I give in the CLI in order to build these images successfully?

Did you simply use
docker-compose -f docker-compose.yml -f docker-compose.dev.yml build ?

Thanks in advance!