From 2cfbecfb8ee4d75029625d363ae0cd49b046c606 Mon Sep 17 00:00:00 2001 From: Vladimir Skubriev Date: Mon, 25 May 2020 13:34:21 +0300 Subject: [PATCH] update https install section --- .gitignore | 1 + cvat/apps/documentation/installation.md | 289 ++++++++++-------------- 2 files changed, 116 insertions(+), 174 deletions(-) diff --git a/.gitignore b/.gitignore index 5231aa3d8f38..002d01c48bd0 100644 --- a/.gitignore +++ b/.gitignore @@ -13,6 +13,7 @@ !/ssh/README.md node_modules /Mask_RCNN/ +/letsencrypt-webroot/ # Ignore temporary files docker-compose.override.yml diff --git a/cvat/apps/documentation/installation.md b/cvat/apps/documentation/installation.md index ec59168e2894..0aa5227242b4 100644 --- a/cvat/apps/documentation/installation.md +++ b/cvat/apps/documentation/installation.md @@ -327,29 +327,37 @@ If this is not the case, please complete the steps in the installation manual fi We will go through the following sequence of steps to get CVAT over HTTPS: -- Move Docker Compose CVAT access port to 80/tcp. -- Configure Nginx to pass one of the [ACME challenges](https://letsencrypt.org/docs/challenge-types/). +- Setup containers on default 80/tcp port. Checkin and then down the containers. +- Configure Nginx to pass one of the [ACME challenges](https://letsencrypt.org/docs/challenge-types/) - webroot. - Create the certificate files using [acme.sh](https://github.com/acmesh-official/acme.sh). - Reconfigure Nginx to serve over HTTPS and map CVAT to Docker Compose port 443. #### Step-by-step instructions -##### 1. Move the CVAT access port +##### 1. Make the proxy listen on standard port 80 and prepare nginx for the ACME challenge via webroot method + +> The configuration assumes that on the docker host there will be only one instance of the CVAT site listens for incoming connections on 80 and 443 port. Also redirecting everything that does not concern renewal of certificates to the site via secure HTTPS protocol. Let's assume the server will be at `my-cvat-server.org`. +Point you shell in cvat repository directory, usually `cd $HOME/cvat`: + +Add the following into your `docker-compose.override.yml`, replacing `my-cvat-server.org` with your own IP address. This file lives in the same directory as `docker-compose.yml`. + +Create enough directories for letsencrypt webroot operation and acme folder passthrougth. + + and restart containers with a new configuration updated in `docker-compose.override.yml` + ```bash -# on the server -docker-compose down +# on the docker host -# add docker-compose.override.yml as per instructions below +# this will create ~/.acme.sh directory +curl https://get.acme.sh | sh -docker-compose up -d +# create a subdirs for acme-challenge webroot manually +mkdir -p $HOME/cvat/letsencrypt-webroot/.well-known/acme-challenge ``` -Add the following into your `docker-compose.override.yml`, replacing `my-cvat-server.org` with your own IP address. -This file lives in the same directory as `docker-compose.yml`. - ```yaml # docker-compose.override.yml version: "2.3" @@ -359,177 +367,106 @@ services: environment: CVAT_HOST: my-cvat-server.org ports: - - "80:80" + - "80:80" + - "443:443" + volumes: + - ./letsencrypt-webroot:/var/tmp/letsencrypt-webroot + - /root/.acme.sh:/root/.acme.sh cvat: environment: ALLOWED_HOSTS: '*' ``` -You should now see an unsecured version of CVAT at `http://my-cvat-server.org`. +This will enable serving `http://my-cvat-server.org/.well-known/acme-challenge/` +route from `/var/tmp/letsencrypt-webroot` directory on the container's filesystem which is bind mounted from docker host `$HOME/cvat/letsencrypt-webroot`. That volume needed for issue and renewing certificates only. + +Update a CVAT site proxy template `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` on docker(system) host. Site config updates from this template each time `cvat_proxy` container start. -##### 2. Configure Nginx for the ACME challenge +Add a location to server with `server_name ${CVAT_HOST};` ahead others: -Temporarily, enable serving `http://my-cvat-server.org/.well-known/acme-challenge/` -route from `/letsencrypt` directory on the server's filesystem. +``` + location ^~ /.well-known/acme-challenge/ { + default_type "text/plain"; + root /var/tmp/letsencrypt-webroot; + } +``` You can use the [Nginx quickstart guide](http://nginx.org/en/docs/beginners_guide.html) for reference. + ```bash -# cvat_proxy/conf.d/cvat.conf.template +# on the docker host +docker-compose down +docker-compose up -d +``` -server { - listen 80; - server_name _ default; - return 404; -} +Your server should still be visible (and unsecured) at `http://my-cvat-server.org` +but you won't see any behavior changes. -server { - listen 80; - server_name ${CVAT_HOST}; +At this point your deployment is up and running, ready for run acme-challenge. - # add this temporarily, to pass an acme challenge - location ^~ /.well-known/acme-challenge/ { - allow all; - root /letsencrypt; - } +##### 2. Setting up HTTPS with `acme.sh` helper - location ~* /api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.* { - proxy_pass http://cvat:8080; - proxy_pass_header X-CSRFToken; - proxy_set_header Host $http_host; - proxy_pass_header Set-Cookie; - } +There are multiple approaches. First one is to use helper on docker host. - location / { - # workaround for match location by arguments - error_page 418 = @annotation_ui; +In a our approach +* it is easier to setup automatic certificate updates and (than it can be done in the container). +* leave certificates in safe place on docker host (protect from `docker-compose down` cleanup) +* no unnecessary certificate files copying between container and host. - if ( $query_string ~ "^id=\d+.*" ) { return 418; } +###### Create certificate files using an ACME challenge on docker host - proxy_pass http://cvat_ui; - proxy_pass_header X-CSRFToken; - proxy_set_header Host $http_host; - proxy_pass_header Set-Cookie; - } +**Prepare certificates.** - # old annotation ui, will be removed in the future. - location @annotation_ui { - proxy_pass http://cvat:8080; - proxy_pass_header X-CSRFToken; - proxy_set_header Host $http_host; - proxy_pass_header Set-Cookie; - } -} -``` +Point you shell in cvat repository directory, usually `cd $HOME/cvat` on docker host. -Now create the `/letsencrypt` directory and mount it into `cvat_proxy` container. -Edit your `docker-compose.override.yml` to look like the following: +> Certificate issue and updates should be on docker host in this approach. -```yaml -# docker-compose.override.yml -version: "2.3" +Let’s Encrypt provides rate limits to ensure fair usage by as many people as possible. They recommend utilize their staging environment instead of the production API during testing. So first try to get a test certificate. -services: - cvat_proxy: - environment: - CVAT_HOST: my-cvat-server.org - ports: - - "80:80" - volumes: - - ./letsencrypt:/letsencrypt - - cvat: - environment: - ALLOWED_HOSTS: '*' +``` +~/.acme.sh/acme.sh --issue --staging -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug ``` -Finally, create the directory and restart CVAT. +> Debug note: nginx server logs for cvat_proxy are not saved in container. You shall see it at docker host by with: `docker logs cvat_proxy`. -```bash -# in the same directory where docker-compose.override.yml lives -mkdir -p letsencrypt/.well-known/acme-challenge +If certificates is issued a successful we should test a renew: -docker-compose down -docker-compose up -d ``` +~/.acme.sh/acme.sh --renew --force --staging -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug +``` +If success: -Your server should still be visible (and unsecured) at `http://my-cvat-server.org` -but you won't see any behavior changes. +* remove test certificate +* issue a production certificate +* create a cron job for user (`crontab -e`). -##### 3. Create certificate files using an ACME challenge +``` +~/.acme.sh/acme.sh --remove -d my-cvat-server.org --debug +rm -r /root/.acme.sh/my-cvat-server.org -At this point your deployment is running. +~/.acme.sh/acme.sh --issue -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug -```bash -admin@tempVM:~/cvat$ docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -0a35cd127968 nginx:stable-alpine "/bin/sh -c 'envsubs…" About a minute ago Up About a minute 0.0.0.0:80->80/tcp, 0.0.0.0:8080->80/tcp cvat_proxy -b85497c44836 cvat_cvat_ui "nginx -g 'daemon of…" About a minute ago Up About a minute 80/tcp cvat_ui -d25a00475849 cvat "/usr/bin/supervisord" About a minute ago Up About a minute 8080/tcp, 8443/tcp cvat -6353a43f55c3 redis:4.0-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 6379/tcp cvat_redis -52009636caa8 postgres:10-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 5432/tcp cvat_db +~/.acme.sh/acme.sh --install-cronjob ``` -We will attach `cvat_proxy` container to run `acme.sh` scripts. +Down the cvat_proxy container for setup https with issued certificates. ```bash -admin@tempVM:~/cvat$ docker exec -ti cvat_proxy /bin/sh - -# install some missing software inside cvat_proxy -/ # apk add openssl curl -/ # curl https://get.acme.sh | sh -/ # ~/.acme.sh/acme.sh -h -[... many lines ...] - -/ # ~/.acme.sh/acme.sh --issue -d my-cvat-server.org -w /letsencrypt -[Fri Apr 3 20:49:05 UTC 2020] Create account key ok. -[Fri Apr 3 20:49:05 UTC 2020] Registering account -[Fri Apr 3 20:49:06 UTC 2020] Registered -[Fri Apr 3 20:49:06 UTC 2020] ACCOUNT_THUMBPRINT='tril8-LdJgM8xg6mnN1pMa7vIMdFizVCE0NImNmyZY4' -[Fri Apr 3 20:49:06 UTC 2020] Creating domain key -[ ... many more lines ...] -[Fri Apr 3 20:49:10 UTC 2020] Your cert is in /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer -[Fri Apr 3 20:49:10 UTC 2020] Your cert key is in /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.key -[Fri Apr 3 20:49:10 UTC 2020] The intermediate CA cert is in /root/.acme.sh/my-cvat-server.org/ca.cer -[Fri Apr 3 20:49:10 UTC 2020] And the full chain certs is there: /root/.acme.sh/my-cvat-server.org/fullchain.cer - -/ # cp ~/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer /letsencrypt/certificate.cer -/ # cp ~/.acme.sh/my-cvat-server.org/my-cvat-server.org.key /letsencrypt/certificate.key -/ # cp ~/.acme.sh/my-cvat-server.org/ca.cer /letsencrypt/ca.cer -/ # cp ~/.acme.sh/my-cvat-server.org/fullchain.cer /letsencrypt/fullchain.cer -/ # exit -admin@tempVM:~/cvat$ ls letsencrypt/ -ca.cer certificate.cer certificate.key fullchain.cer -admin@tempVM:~/cvat$ mkdir cert -admin@tempVM:~/cvat$ mv letsencrypt/* ./cert +docker stop cvat_proxy ``` -##### 4. Reconfigure Nginx for HTTPS access - -Update Docker Compose configuration to mount the certificate directory. +**Reconfigure nginx for use certificates.** -```yml -# docker-compose.override.yml -version: "2.3" +Bring the configuration file `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` to the following form: -services: - cvat_proxy: - environment: - CVAT_HOST: my-cvat-server.org - ports: - - "443:443" - volumes: - - ./letsencrypt:/letsencrypt - - ./cert:/cert:ro # this is new - - cvat: - environment: - ALLOWED_HOSTS: '*' -``` +* add location with redirect `return 301` to 80/tcp server. +* change listen to `listen 443 ssl;` in main configururation server +* add ssl certificates options and secure them. -Also, reconfigure Nginx to use `443/tcp` and point it to the new keys. +Example of configuration file: -```bash +``` server { listen 80; server_name _ default; @@ -537,16 +474,47 @@ server { } server { - listen 443 ssl; + listen 80; + server_name ${CVAT_HOST}; + + location ^~ /.well-known/acme-challenge/ { + default_type "text/plain"; + root /var/tmp/letsencrypt; + } + + location / { + return 301 https://$server_name$request_uri; + } +} + +server { + listen 443 ssl; + ssl_certificate /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer; + ssl_certificate_key /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.key; + ssl_trusted_certificate /root/.acme.sh/my-cvat-server.org/fullchain.cer; + + # security options + ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + ssl_prefer_server_ciphers on; + ssl_stapling on; + ssl_session_timeout 24h; + ssl_session_cache shared:SSL:2m; + ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!3DES'; + + # security headers + add_header X-Frame-Options "SAMEORIGIN" always; + add_header X-XSS-Protection "1; mode=block" always; + add_header X-Content-Type-Options "nosniff" always; + add_header Referrer-Policy "no-referrer-when-downgrade" always; + add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always; + add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always; + server_name ${CVAT_HOST}; proxy_pass_header X-CSRFToken; proxy_set_header Host $http_host; proxy_pass_header Set-Cookie; - ssl_certificate /cert/certificate.cer; - ssl_certificate_key /cert/certificate.key; - location ~* /api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.* { proxy_pass http://cvat:8080; } @@ -570,35 +538,8 @@ server { } ``` -Finally, restart your service. +Start cvat_proxy container with https enabled. -```bash -admin@tempVM:~/cvat$ docker-compose down -Stopping cvat_proxy ... done -Stopping cvat_ui ... done -Stopping cvat ... done -Stopping cvat_db ... done -Stopping cvat_redis ... done -Removing cvat_proxy ... done -Removing cvat_ui ... done -Removing cvat ... done -Removing cvat_db ... done -Removing cvat_redis ... done -Removing network cvat_default -admin@tempVM:~/cvat$ docker-compose up -d -Creating network "cvat_default" with the default driver -Creating cvat_db ... done -Creating cvat_redis ... done -Creating cvat ... done -Creating cvat_ui ... done -Creating cvat_proxy ... done -admin@tempVM:~/cvat$ docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -71464aeac87c nginx:stable-alpine "/bin/sh -c 'envsubs…" About a minute ago Up About a minute 0.0.0.0:443->443/tcp, 0.0.0.0:8080->80/tcp cvat_proxy -8428cfbb766e cvat_cvat_ui "nginx -g 'daemon of…" About a minute ago Up About a minute 80/tcp cvat_ui -b5a2f78689da cvat "/usr/bin/supervisord" About a minute ago Up About a minute 8080/tcp, 8443/tcp cvat -ef4a1f47440f redis:4.0-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 6379/tcp cvat_redis -7803bf828d9f postgres:10-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 5432/tcp cvat_db ``` - -Now you can go to `https://my-cvat-server.org/` and verify that you are using an encrypted connection. +docker start cvat_proxy +``` \ No newline at end of file