Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gms playbook #2340

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions gms_dev.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
# Deploy the Galaxy Media Site to a development server
- hosts: gms_dev_webservers
remote_user: ubuntu
become: true
vars_files:
- group_vars/VAULT
- group_vars/gms.yml
- host_vars/dev-site.gvl.org.au.yml
vars:
sentry_dns_url: null
pre_tasks:
- name: update apt cache
ansible.builtin.apt:
update_cache: yes
- name: install system dependencies
ansible.builtin.package:
name:
- libpq-dev
- python3-pip
roles:
- role: geerlingguy.nginx
tags: init
- role: geerlingguy.postgresql
become: true
tags: init,postgresql
- role: galaxy_media_site
tags: gms
25 changes: 25 additions & 0 deletions gms_prod.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
# Deploy the Galaxy Media Site
- hosts: gms_webservers
remote_user: ubuntu
become: true
vars_files:
- group_vars/VAULT
- group_vars/gms.yml
- host_vars/site.usegalaxy.org.au.yml
pre_tasks:
- name: update apt cache
ansible.builtin.apt:
update_cache: yes
- name: install system dependencies
ansible.builtin.package:
name:
- libpq-dev
roles:
- role: geerlingguy.nginx
tags: init
- role: geerlingguy.postgresql
become: true
tags: init,postgresql
- role: galaxy_media_site
tags: gms
1,084 changes: 558 additions & 526 deletions group_vars/VAULT

Large diffs are not rendered by default.

89 changes: 89 additions & 0 deletions group_vars/gms.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---

# Read from VAULT:
# admin_email: <str>
# admin_password: <str>
# database_password: <str>
# django_secret_key: <str>
# smtp_password: <str>

# If you don't want SSL certs, just flip this:
certbot_ssl: true
certbot_renew_email: c.hyde@qcif.edu.au

# 4 web workers should be plenty - increase for machines with >4 CPUs
gunicorn:
web_workers: 4

# Postgres database conf
postgres_db_name: gms
postgres_db_user: gms

postgresql_locales:
- 'en_US.UTF-8'

postgresql_databases:
- name: "{{ postgres_db_name }}"
port: 5432

postgresql_users:
- name: "{{ postgres_db_user }}"
password: "{{ vault_gms_postgres_db_password }}"
db: "{{ postgres_db_name }}"

postgresql_privs:
- db: "{{ postgres_db_name }}"
roles: "{{ postgres_db_user }}"
objs: ALL_IN_SCHEMA
privs: ALL

postgres_users_no_log: false

# GMS Webapp configuration
# -----------------------------------------------------------------------------

# Will git clone galaxy-content-site to here
project_root: /srv/sites/galaxy-media-site

# This is where service files will be created
server_root: /srv/config

# App will be served from here
django_root: "{{ project_root }}/webapp"

# Virtual environment will be created here
venv_root: "{{ project_root }}/venv"

# Login for django admin portal
admin_user:
first_name: admin
last_name: admin
email: "{{ vault_gms_admin_email }}"
password: "{{ vault_gms_admin_password }}"

smtp:
hostname: mail.smtp2go.com
port: 2525
username: galaxytest
password: "{{ vault_gms_smtp_password }}"
from_address: noreply@genome.edu.au
to_address: help@genome.edu.au
use_tls: true

# skip_database_migration: true

# For error notifications
slack:
errors:
api_key: "{{ vault_gms_slack_error_api_key }}"
channel_id: C038ESFSXLN
mention_user_id: U024MUHM23W
tools:
urls: "{{ vault_gms_slack_api_urls }}" # comma separated list

# For API authentication (requires admin key)
galaxy_api:
url: https://usegalaxy.org.au
key: "{{ vault_gms_galaxy_api_key }}"

sentry_dns_url: "{{ vault_gms_sentry_dns_url }}"
1 change: 1 addition & 0 deletions host_vars/dev-site.gvl.org.au.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
sentry_dns_url: null
Empty file.
6 changes: 6 additions & 0 deletions hosts
Original file line number Diff line number Diff line change
@@ -221,3 +221,9 @@ jenkins ansible_ssh_host=115.146.87.250
ansible_host_key_checking=False
ansible_user=ubuntu
ansible_ssh_extra_args='-o StrictHostKeyChecking=no'

[gms_webservers]
site.usegalaxy.org.au ansible_connection=ssh ansible_host=115.146.85.213 ansible_user=ubuntu

[gms_dev_webservers]
dev-site.gvl.org.au ansible_connection=ssh ansible_host=203.101.227.51 ansible_user=ubuntu
5 changes: 5 additions & 0 deletions requirements.yml
Original file line number Diff line number Diff line change
@@ -86,4 +86,9 @@ roles:
- name: geerlingguy.java
version: 2.5.0
- name: geerlingguy.jenkins

# GMS roles
- name: geerlingguy.nginx
version: 3.2.0
- name: geerlingguy.postgresql
version: 5.2.0
63 changes: 63 additions & 0 deletions roles/galaxy_media_site/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Deployment with Ansible


**For deployment to Ubuntu >20.04 LTS machines**

**Assumptions:**
- You have SSH access to this machine set in your `.ssh/config`
- You have registered a domain name for this site and set the DNS to point at your machine's IP address


### Install Ansible

See Ansible docs for more info:
https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html

```sh
sudo apt install ansible
# ~~~ or ~~~
python -m pip install ansible
```

---

### Application configuration

1. Modify the `hosts` file `webserver` block to match your webserver's hostname

1. Create `host_vars/mysite.com.yml` and add a key for your webserver's hostname (replace mysite.com with your actual hostname):

`hostname: mysite.com`

1. Update the `certbot_renew_email` var in `group_vars/webserver.yml`

1. Update webapp configuration in `group_vars/webserver.yml` to suit:
- Default admin login (please update these for security)
- `smtp` credentials for your mail server
- *Optional* - Host installation paths:
- `project_root` - where this git repository will be cloned
- `server_root` - where server configuration will be saved
- `django_root` - where the application will be served from
- `venv_root` - where the virtual env will be created


5. Create five variables in the file `group_vars/secrets.yml`:
- `admin_email` - login to admin site
- `admin_password` - password for above
- `database_password` - localhost database auth
- `django_secret_key` - for Django to hash database passwords etc
- `smtp_password` - mail server SMTP password (for webforms mail)


6. Encrypt these [Ansible secrets](https://docs.ansible.com/ansible/latest/user_guide/vault.html#encrypting-existing-files) with the ansible vault. Create `~/.vault.pass` if you don't have one already - it should contain a secure and secret password:
```
ansible-vault encrypt group_vars/secrets.yml --vault-password-file ~/.vault.pass
```

---

### Deploy GMS

```sh
ansible-playbook -i hosts playbook.yml --vault-password-file ~/.vault.pass
```
31 changes: 31 additions & 0 deletions roles/galaxy_media_site/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
project_root: /srv/galaxy-media-site
server_root: /srv/config
django_root: "{{ project_root }}/webapp"
venv_root: "{{ project_root }}/venv"

gms:
templates:
- src: "{{ role_path }}/templates/gunicorn.py.j2"
dest: "{{ server_root }}/gunicorn.py"
- src: "{{ role_path }}/templates/gms.service.j2"
dest: /etc/systemd/system/gms.service
- src: "{{ role_path }}/templates/nginx.vhost.j2"
dest: "/etc/nginx/sites-available/{{ inventory_hostname }}.conf"
- src: "{{ role_path }}/templates/.env.j2"
dest: "{{ project_root }}/.env"
- src: "{{ role_path }}/templates/update.sh.j2"
dest: "/home/ubuntu/update.sh"

files:
- src: "{{ role_path }}/files/gms.socket"
dest: /etc/systemd/system/gms.socket

certbot_ssl: true

gunicorn:
web_workers: 4

skip_database_migration: false

admin_user: null
6 changes: 6 additions & 0 deletions roles/galaxy_media_site/files/gms.socket
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[Unit]
Description=Galaxy content site socket
[Socket]
ListenStream=/run/gms.sock
[Install]
WantedBy=sockets.target
12 changes: 12 additions & 0 deletions roles/galaxy_media_site/secrets.yml.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Vault password in Shared Devops

admin_email: <str>
admin_password: <str>
postgres_db_password: <str>
django_secret_key: <str>
smtp_password: <str>
slack_error_api_key: <str>
slack_api_urls: <str>, <str>
recaptcha_secret_key: <str>
recaptcha_site_key: <str>
galaxy_api_key: <str>
Loading