Skip to content

Commit

Permalink
update https install section
Browse files Browse the repository at this point in the history
  • Loading branch information
Vladimir Skubriev authored and Fernando Martínez González committed Aug 3, 2020
1 parent baa7a94 commit 2cfbecf
Show file tree
Hide file tree
Showing 2 changed files with 116 additions and 174 deletions.
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
!/ssh/README.md
node_modules
/Mask_RCNN/
/letsencrypt-webroot/

# Ignore temporary files
docker-compose.override.yml
Expand Down
289 changes: 115 additions & 174 deletions cvat/apps/documentation/installation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -327,29 +327,37 @@ If this is not the case, please complete the steps in the installation manual fi

We will go through the following sequence of steps to get CVAT over HTTPS:

- Move Docker Compose CVAT access port to 80/tcp.
- Configure Nginx to pass one of the [ACME challenges](https://letsencrypt.org/docs/challenge-types/).
- Setup containers on default 80/tcp port. Checkin and then down the containers.
- Configure Nginx to pass one of the [ACME challenges](https://letsencrypt.org/docs/challenge-types/) - webroot.
- Create the certificate files using [acme.sh](https://github.com/acmesh-official/acme.sh).
- Reconfigure Nginx to serve over HTTPS and map CVAT to Docker Compose port 443.

#### Step-by-step instructions

##### 1. Move the CVAT access port
##### 1. Make the proxy listen on standard port 80 and prepare nginx for the ACME challenge via webroot method

> The configuration assumes that on the docker host there will be only one instance of the CVAT site listens for incoming connections on 80 and 443 port. Also redirecting everything that does not concern renewal of certificates to the site via secure HTTPS protocol.

Let's assume the server will be at `my-cvat-server.org`.
Point you shell in cvat repository directory, usually `cd $HOME/cvat`:
Add the following into your `docker-compose.override.yml`, replacing `my-cvat-server.org` with your own IP address. This file lives in the same directory as `docker-compose.yml`.
Create enough directories for letsencrypt webroot operation and acme folder passthrougth.
and restart containers with a new configuration updated in `docker-compose.override.yml`
```bash
# on the server
docker-compose down
# on the docker host
# add docker-compose.override.yml as per instructions below
# this will create ~/.acme.sh directory
curl https://get.acme.sh | sh
docker-compose up -d
# create a subdirs for acme-challenge webroot manually
mkdir -p $HOME/cvat/letsencrypt-webroot/.well-known/acme-challenge
```
Add the following into your `docker-compose.override.yml`, replacing `my-cvat-server.org` with your own IP address.
This file lives in the same directory as `docker-compose.yml`.
```yaml
# docker-compose.override.yml
version: "2.3"
Expand All @@ -359,194 +367,154 @@ services:
environment:
CVAT_HOST: my-cvat-server.org
ports:
- "80:80"
- "80:80"
- "443:443"
volumes:
- ./letsencrypt-webroot:/var/tmp/letsencrypt-webroot
- /root/.acme.sh:/root/.acme.sh
cvat:
environment:
ALLOWED_HOSTS: '*'
```
You should now see an unsecured version of CVAT at `http://my-cvat-server.org`.
This will enable serving `http://my-cvat-server.org/.well-known/acme-challenge/`
route from `/var/tmp/letsencrypt-webroot` directory on the container's filesystem which is bind mounted from docker host `$HOME/cvat/letsencrypt-webroot`. That volume needed for issue and renewing certificates only.

Update a CVAT site proxy template `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` on docker(system) host. Site config updates from this template each time `cvat_proxy` container start.

##### 2. Configure Nginx for the ACME challenge
Add a location to server with `server_name ${CVAT_HOST};` ahead others:

Temporarily, enable serving `http://my-cvat-server.org/.well-known/acme-challenge/`
route from `/letsencrypt` directory on the server's filesystem.
```
location ^~ /.well-known/acme-challenge/ {
default_type "text/plain";
root /var/tmp/letsencrypt-webroot;
}
```
You can use the [Nginx quickstart guide](http://nginx.org/en/docs/beginners_guide.html) for reference.


```bash
# cvat_proxy/conf.d/cvat.conf.template
# on the docker host
docker-compose down
docker-compose up -d
```

server {
listen 80;
server_name _ default;
return 404;
}
Your server should still be visible (and unsecured) at `http://my-cvat-server.org`
but you won't see any behavior changes.
server {
listen 80;
server_name ${CVAT_HOST};
At this point your deployment is up and running, ready for run acme-challenge.
# add this temporarily, to pass an acme challenge
location ^~ /.well-known/acme-challenge/ {
allow all;
root /letsencrypt;
}
##### 2. Setting up HTTPS with `acme.sh` helper
location ~* /api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.* {
proxy_pass http://cvat:8080;
proxy_pass_header X-CSRFToken;
proxy_set_header Host $http_host;
proxy_pass_header Set-Cookie;
}
There are multiple approaches. First one is to use helper on docker host.
location / {
# workaround for match location by arguments
error_page 418 = @annotation_ui;
In a our approach
* it is easier to setup automatic certificate updates and (than it can be done in the container).
* leave certificates in safe place on docker host (protect from `docker-compose down` cleanup)
* no unnecessary certificate files copying between container and host.
if ( $query_string ~ "^id=\d+.*" ) { return 418; }
###### Create certificate files using an ACME challenge on docker host
proxy_pass http://cvat_ui;
proxy_pass_header X-CSRFToken;
proxy_set_header Host $http_host;
proxy_pass_header Set-Cookie;
}
**Prepare certificates.**
# old annotation ui, will be removed in the future.
location @annotation_ui {
proxy_pass http://cvat:8080;
proxy_pass_header X-CSRFToken;
proxy_set_header Host $http_host;
proxy_pass_header Set-Cookie;
}
}
```
Point you shell in cvat repository directory, usually `cd $HOME/cvat` on docker host.
Now create the `/letsencrypt` directory and mount it into `cvat_proxy` container.
Edit your `docker-compose.override.yml` to look like the following:
> Certificate issue and updates should be on docker host in this approach.
```yaml
# docker-compose.override.yml
version: "2.3"
Let’s Encrypt provides rate limits to ensure fair usage by as many people as possible. They recommend utilize their staging environment instead of the production API during testing. So first try to get a test certificate.
services:
cvat_proxy:
environment:
CVAT_HOST: my-cvat-server.org
ports:
- "80:80"
volumes:
- ./letsencrypt:/letsencrypt
cvat:
environment:
ALLOWED_HOSTS: '*'
```
~/.acme.sh/acme.sh --issue --staging -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug
```
Finally, create the directory and restart CVAT.
> Debug note: nginx server logs for cvat_proxy are not saved in container. You shall see it at docker host by with: `docker logs cvat_proxy`.
```bash
# in the same directory where docker-compose.override.yml lives
mkdir -p letsencrypt/.well-known/acme-challenge
If certificates is issued a successful we should test a renew:
docker-compose down
docker-compose up -d
```
~/.acme.sh/acme.sh --renew --force --staging -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug
```
If success:
Your server should still be visible (and unsecured) at `http://my-cvat-server.org`
but you won't see any behavior changes.
* remove test certificate
* issue a production certificate
* create a cron job for user (`crontab -e`).
##### 3. Create certificate files using an ACME challenge
```
~/.acme.sh/acme.sh --remove -d my-cvat-server.org --debug
rm -r /root/.acme.sh/my-cvat-server.org
At this point your deployment is running.
~/.acme.sh/acme.sh --issue -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug
```bash
admin@tempVM:~/cvat$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0a35cd127968 nginx:stable-alpine "/bin/sh -c 'envsubs…" About a minute ago Up About a minute 0.0.0.0:80->80/tcp, 0.0.0.0:8080->80/tcp cvat_proxy
b85497c44836 cvat_cvat_ui "nginx -g 'daemon of…" About a minute ago Up About a minute 80/tcp cvat_ui
d25a00475849 cvat "/usr/bin/supervisord" About a minute ago Up About a minute 8080/tcp, 8443/tcp cvat
6353a43f55c3 redis:4.0-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 6379/tcp cvat_redis
52009636caa8 postgres:10-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 5432/tcp cvat_db
~/.acme.sh/acme.sh --install-cronjob
```
We will attach `cvat_proxy` container to run `acme.sh` scripts.
Down the cvat_proxy container for setup https with issued certificates.
```bash
admin@tempVM:~/cvat$ docker exec -ti cvat_proxy /bin/sh
# install some missing software inside cvat_proxy
/ # apk add openssl curl
/ # curl https://get.acme.sh | sh
/ # ~/.acme.sh/acme.sh -h
[... many lines ...]
/ # ~/.acme.sh/acme.sh --issue -d my-cvat-server.org -w /letsencrypt
[Fri Apr 3 20:49:05 UTC 2020] Create account key ok.
[Fri Apr 3 20:49:05 UTC 2020] Registering account
[Fri Apr 3 20:49:06 UTC 2020] Registered
[Fri Apr 3 20:49:06 UTC 2020] ACCOUNT_THUMBPRINT='tril8-LdJgM8xg6mnN1pMa7vIMdFizVCE0NImNmyZY4'
[Fri Apr 3 20:49:06 UTC 2020] Creating domain key
[ ... many more lines ...]
[Fri Apr 3 20:49:10 UTC 2020] Your cert is in /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer
[Fri Apr 3 20:49:10 UTC 2020] Your cert key is in /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.key
[Fri Apr 3 20:49:10 UTC 2020] The intermediate CA cert is in /root/.acme.sh/my-cvat-server.org/ca.cer
[Fri Apr 3 20:49:10 UTC 2020] And the full chain certs is there: /root/.acme.sh/my-cvat-server.org/fullchain.cer
/ # cp ~/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer /letsencrypt/certificate.cer
/ # cp ~/.acme.sh/my-cvat-server.org/my-cvat-server.org.key /letsencrypt/certificate.key
/ # cp ~/.acme.sh/my-cvat-server.org/ca.cer /letsencrypt/ca.cer
/ # cp ~/.acme.sh/my-cvat-server.org/fullchain.cer /letsencrypt/fullchain.cer
/ # exit
admin@tempVM:~/cvat$ ls letsencrypt/
ca.cer certificate.cer certificate.key fullchain.cer
admin@tempVM:~/cvat$ mkdir cert
admin@tempVM:~/cvat$ mv letsencrypt/* ./cert
docker stop cvat_proxy
```
##### 4. Reconfigure Nginx for HTTPS access
Update Docker Compose configuration to mount the certificate directory.
**Reconfigure nginx for use certificates.**
```yml
# docker-compose.override.yml
version: "2.3"
Bring the configuration file `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` to the following form:
services:
cvat_proxy:
environment:
CVAT_HOST: my-cvat-server.org
ports:
- "443:443"
volumes:
- ./letsencrypt:/letsencrypt
- ./cert:/cert:ro # this is new
cvat:
environment:
ALLOWED_HOSTS: '*'
```
* add location with redirect `return 301` to 80/tcp server.
* change listen to `listen 443 ssl;` in main configururation server
* add ssl certificates options and secure them.
Also, reconfigure Nginx to use `443/tcp` and point it to the new keys.
Example of configuration file:
```bash
```
server {
listen 80;
server_name _ default;
return 404;
}
server {
listen 443 ssl;
listen 80;
server_name ${CVAT_HOST};
location ^~ /.well-known/acme-challenge/ {
default_type "text/plain";
root /var/tmp/letsencrypt;
}
location / {
return 301 https://$server_name$request_uri;
}
}
server {
listen 443 ssl;
ssl_certificate /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer;
ssl_certificate_key /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.key;
ssl_trusted_certificate /root/.acme.sh/my-cvat-server.org/fullchain.cer;
# security options
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_stapling on;
ssl_session_timeout 24h;
ssl_session_cache shared:SSL:2m;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!3DES';
# security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
server_name ${CVAT_HOST};
proxy_pass_header X-CSRFToken;
proxy_set_header Host $http_host;
proxy_pass_header Set-Cookie;
ssl_certificate /cert/certificate.cer;
ssl_certificate_key /cert/certificate.key;
location ~* /api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.* {
proxy_pass http://cvat:8080;
}
Expand All @@ -570,35 +538,8 @@ server {
}
```
Finally, restart your service.
Start cvat_proxy container with https enabled.
```bash
admin@tempVM:~/cvat$ docker-compose down
Stopping cvat_proxy ... done
Stopping cvat_ui ... done
Stopping cvat ... done
Stopping cvat_db ... done
Stopping cvat_redis ... done
Removing cvat_proxy ... done
Removing cvat_ui ... done
Removing cvat ... done
Removing cvat_db ... done
Removing cvat_redis ... done
Removing network cvat_default
admin@tempVM:~/cvat$ docker-compose up -d
Creating network "cvat_default" with the default driver
Creating cvat_db ... done
Creating cvat_redis ... done
Creating cvat ... done
Creating cvat_ui ... done
Creating cvat_proxy ... done
admin@tempVM:~/cvat$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
71464aeac87c nginx:stable-alpine "/bin/sh -c 'envsubs…" About a minute ago Up About a minute 0.0.0.0:443->443/tcp, 0.0.0.0:8080->80/tcp cvat_proxy
8428cfbb766e cvat_cvat_ui "nginx -g 'daemon of…" About a minute ago Up About a minute 80/tcp cvat_ui
b5a2f78689da cvat "/usr/bin/supervisord" About a minute ago Up About a minute 8080/tcp, 8443/tcp cvat
ef4a1f47440f redis:4.0-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 6379/tcp cvat_redis
7803bf828d9f postgres:10-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 5432/tcp cvat_db
```
Now you can go to `https://my-cvat-server.org/` and verify that you are using an encrypted connection.
docker start cvat_proxy
```

0 comments on commit 2cfbecf

Please sign in to comment.