Installation Guide

This document explains how to deploy a single node instance of the data registry on Ubuntu suitable for test purposes. Depending on the level of service availability required this of course may not be the right approach.

To run a local registry please do not use this document, see here instead: https://fairdatapipeline.github.io/docs/local_registry/

Installing the database

Any database type supported by Django can be used. Here we use PostgreSQL, which can be installed by running:

apt install postgresql postgresql-contrib

Setting up the database

Firstly, connect to the database using psql:

$ sudo -u postgres psql
psql (10.12 (Ubuntu 10.12-0ubuntu0.18.04.1))
Type "help" for help.
postgres=#

Create a database:

postgres=# CREATE DATABASE scrc;
CREATE DATABASE

Create a user (replacing password with the password for your user):

postgres=# CREATE USER scrc WITH PASSWORD 'password';
CREATE ROLE

Give the user the required permissions and exit:

postgres=# ALTER ROLE scrc SET client_encoding TO 'utf8';
ALTER ROLE
postgres=# ALTER ROLE scrc SET default_transaction_isolation TO 'read committed';
ALTER ROLE
postgres=# ALTER ROLE scrc SET timezone TO 'UTC';
ALTER ROLE
postgres=# GRANT ALL PRIVILEGES ON DATABASE scrc TO scrc;
GRANT
postgres=# \q

Save the user password in a .pgpass file in the server user home. This should look something like (with password replaced by the database user password used above):

*:*:scrc:scrc:password

The .pgpass file must be only user-readable. The permissions on the file can be set by:

chmod 0600 ~/.pgpass

Installing the server

To install the server you will need Python3 with the virtual environment python package.

git clone [email protected]:FAIRDataPipeline/data-registry.git
cd data-registry
python -m virtualenv venv
. venv/bin/activate
pip3 install -r requirements.txt

Update the ALLOWED_HOSTS parameter in the settings file drams/settings.py as necessary. This should contain localhost in addition to the external DNS name of the host, e.g.

ALLOWED_HOSTS = ['localhost', 'test.scrc.uk']

By default Django is configured to open a file /home/ubuntu/secret_key.txt containing a secret key. A secret key can be created by running the following Python code:

from django.core.management.utils import get_random_secret_key
get_random_secret_key()

Setting up the server:

cd data-registry
. venv/bin/activate
python3 manage.py makemigrations custom_user
python3 manage.py makemigrations data_management
python3 manage.py migrate

Now create a superuser:

python3 manage.py createsuperuser --username admin

The data schema diagrams can be generated using:

python3 manage.py graph_models data_management --arrow-shape crow -X "BaseModel,DataObject,DataObjectVersion" -E -o schema.dot
dot schema.dot -Tsvg -o static/images/schema.svg
dot schema.dot -Tpng -o static/images/schema.png

Running the server with gunicorn and nginx

Setting up the gunicorn service

As root, create the file /etc/systemd/system/gunicorn.socket containing:

[Unit]
Description=gunicorn socket

[Socket]
ListenStream=/run/gunicorn.sock

[Install]
WantedBy=sockets.target

and create the file /etc/systemd/system/gunicorn.service containing:

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu/scrc
ExecStart=/home/ubuntu/scrc/dataregistry_env/bin/gunicorn \
          --access-logfile - \
          --workers 3 \
          --bind unix:/run/gunicorn.sock \
          dataregistry.wsgi:application

[Install]
WantedBy=multi-user.target

As root, start the gunicorn socket service. The gunicorn service will automatically be started as necessary.

systemctl start gunicorn.socket
systemctl enable gunicorn.socket

Setting up nginx

Create a server block for your domain in /etc/nginx/sites-available, for example /etc/nginx/sites-available/test.scrc.uk.

Example contents:

server {
    server_name test.scrc.uk;

    listen 80 default_server;
    listen [::]:80 default_server;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/ubuntu/data-registry;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn.sock;
    }
}

Create a symbolic link to enable this site in nginx:

ln -s /etc/nginx/sites-available/test.scrc.uk /etc/nginx/sites-enabled/

and start nginx:

systemctl start nginx

Obtain a certificate from Let's Encrypt, e.g.

certbot --nginx -d test.scrc.uk

where the DNS name will have to be adjusted as appropriate. This will update the nginx site config file so it should now look something like:

server {
    server_name test.scrc.uk; # managed by Certbot

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/ubuntu/data-registry;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn.sock;
    }

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/test.scrc.uk/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/test.scrc.uk/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = test.scrc.uk) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    server_name test.scrc.uk;

    listen 80 ;
    listen [::]:80 ;
    return 404; # managed by Certbot
}