Dag Wieers and Jeroen Hoekx v0.1, Feb 2014
Ansible only talks to hosts you want it to talk to. Those hosts are defined in an inventory file.
[ansible@ws01 workshop]$ cat hosts [vms] vm-master ansible_ssh_host=192.168.122.X vm-web ansible_ssh_host=192.168.122.Y vm-db ansible_ssh_host=192.168.122.Z
Update the IP addresses to correspond with the ones the systems booted with.
The most straightforward usage of Ansible it to run commands on remote hosts. The ping module checks connectivity and correct python setup.
Let’s assume our VM is remote and ping it.
[root@ws01 workshop]$ ansible vm-master -m ping -i hosts
vm-master | success >> {
"changed": false,
"ping": "pong"
}
The syntax is ansible <selector> <options>.
We only want to run it on the local host, so we choose ws01 as selector. The module is selected with the -m switch. The -i hosts switch tells Ansible which list of hosts to select from.
We can run this command on multiple hosts by using the group in the inventory:
[root@ws01 workshop]$ ansible vms -m ping -i hosts
vm-master | success >> {
"changed": false,
"ping": "pong"
}
vm-web| success >> {
"changed": false,
"ping": "pong"
}
vm-db| success >> {
"changed": false,
"ping": "pong"
}
The ping module is useful for testing things, but let’s run a generic command on all systems:
[ansible@ws01 workshop]$ ansible vms -m command -a 'date' -i hosts vm-master | success | rc=0 >> Mon Feb 3 16:31:32 UTC 2014 vm-web | success | rc=0 >> Mon Feb 3 16:31:33 UTC 2014 vm-db | success | rc=0 >> Mon Feb 3 16:31:33 UTC 2014
That uses the command module and gives it arguments with -a.
The goal of this workshop is to deploy a web application. A useful thing to have in that situation is a web server. Let’s use Ansible to make sure Apache is installed and started. This introduces the yum and service modules.
[root@ws01 workshop]# ansible vm-web -m yum -a 'pkg=httpd state=installed' -i hosts
vm-web | success >> {
"changed": false,
"msg": "",
"rc": 0,
"results": [
"httpd-2.2.15-29.el6.centos.i686 providing httpd is already installed"
]
}
Great. Apache was already installed on the system. Notice the arguments to the yum module. The state parameter is used by virtually every Ansible module.
Let’s start Apache.
[root@ws01 workshop]# ansible vm-web -m service -a 'name=httpd state=started enabled=yes' -i hosts
vm-web | success >> {
"changed": false,
"enabled": true,
"name": "httpd",
"state": "started"
}
The service module is one of the two distribution agnostic modules (with user). Package managers for example differ too much in their functionality to have one baseline module.
Another thing to notice is the yes argument to the enabled parameter. Boolean parameters in Ansible are yes and no.
The application we want to deploy is multi-tiered. There is the web server part and the database part. It makes sense to be able to group hosts in functional groups.
The built-in inventory can do that:
[vms] vm-master ansible_ssh_host=192.168.122.X vm-web ansible_ssh_host=192.168.122.Y vm-db ansible_ssh_host=192.168.122.Z [web-servers] vm-web [database-servers] vm-db
Running commands on potentially hundreds of systems is nice, but it does not scale as your infrastructure grows. We need a way to describe the system state. Ansible playbooks provide a declarative way to do just that.
Playbooks are written in YAML. One playbook consists of one or more plays. Each play has a number of tasks. Playbooks are run sequentially. This allows orchestration of actions on multiple system groups.
|
Note
|
Explain YAML here using slides |
Let’s show an example that ensures Apache is installed and running.
- name: Configure Apache
hosts: web-servers
user: root
tasks:
- name: Install Apache
action: yum pkg=httpd state=installed
- name: Start and Enable Apache
action: service name=httpd state=started enabled=yes
This playbook has one play. In that play are two tasks.
A play always indicates on which hosts it will run and as what user. Many more parameters are possible, but this is the minimum. Well, you could skip the user, but then it will run as your local user.
We’ve updated the hosts file to include a second group web-servers. Our vm-web system is in that group.
You can run a playbook like this:
[root@ws01 workshop]# ansible-playbook apache.yml -i hosts PLAY [Configure Apache] ******************************************************* GATHERING FACTS *************************************************************** ok: [vm-web] TASK: [Install Apache] ******************************************************** ok: [vm-web] TASK: [Start and Enable Apache] *********************************************** ok: [vm-web] PLAY RECAP ******************************************************************** vm-web : ok=3 changed=0 unreachable=0 failed=0
You will notice this playbook runs both tasks and has an additional step to gather facts. This step gathers facts from the system. These facts are available as variables in the remainder of the play.
Let’s see which facts are available:
[root@ws01 workshop]# ansible vm-web -m setup -i hosts
vm-web | success >> {
"ansible_facts": {
"ansible_all_ipv4_addresses": [
"192.168.122.21"
],
"ansible_all_ipv6_addresses": [
"fe80::5054:ff:fe23:1614"
],
"ansible_architecture": "i386",
"ansible_bios_date": "01/01/2011",
"ansible_bios_version": "Bochs",
"ansible_cmdline": {
"KEYBOARDTYPE": "pc",
"KEYTABLE": "us",
"LANG": "en_US.UTF-8",
"SYSFONT": "latarcyrheb-sun16",
"quiet": true,
"rd_LVM_LV": "vg_ws01_root/lv_swap",
"rd_NO_DM": true,
"rd_NO_LUKS": true,
"rd_NO_MD": true,
"rhgb": true,
"ro": true,
"root": "/dev/mapper/vg_ws01_root-lv_root"
},
"ansible_date_time": {
"date": "2014-02-03",
"day": "03",
"epoch": "1391415917",
"hour": "08",
"iso8601": "2014-02-03T08:25:17Z",
"iso8601_micro": "2014-02-03T08:25:17.248371Z",
"minute": "25",
"month": "02",
"second": "17",
"time": "08:25:17",
"tz": "UTC",
"tz_offset": "+0000",
"year": "2014"
},
"ansible_default_ipv4": {
"address": "192.168.122.21",
"alias": "eth0",
"gateway": "192.168.122.1",
"interface": "eth0",
"macaddress": "52:54:00:23:16:14",
"mtu": 1500,
"netmask": "255.255.255.0",
"network": "192.168.122.0",
"type": "ether"
},
"ansible_default_ipv6": {},
"ansible_devices": {
"sr0": {
"holders": [],
"host": "",
"model": "QEMU DVD-ROM",
"partitions": {},
"removable": "1",
"rotational": "1",
"scheduler_mode": "cfq",
"sectors": "2097151",
"sectorsize": "512",
"size": "1024.00 MB",
"support_discard": "0",
"vendor": "QEMU"
},
"vda": {
"holders": [],
"host": "",
"model": null,
"partitions": {
"vda1": {
"sectors": "409600",
"sectorsize": 512,
"size": "200.00 MB",
"start": "2048"
},
"vda2": {
"sectors": "5879808",
"sectorsize": 512,
"size": "2.80 GB",
"start": "411648"
}
},
"removable": "0",
"rotational": "1",
"scheduler_mode": "cfq",
"sectors": "6291456",
"sectorsize": "512",
"size": "3.00 GB",
"support_discard": "0",
"vendor": "6900"
}
},
"ansible_distribution": "CentOS",
"ansible_distribution_release": "Final",
"ansible_distribution_version": "6.5",
"ansible_domain": "",
"ansible_env": {
"G_BROKEN_FILENAMES": "1",
"HOME": "/root",
"LANG": "C",
"LESSOPEN": "|/usr/bin/lesspipe.sh %s",
"LOGNAME": "root",
"MAIL": "/var/mail/root",
"PATH": "/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin",
"PWD": "/root",
"SHELL": "/bin/bash",
"SHLVL": "2",
"SSH_CLIENT": "192.168.122.21 37293 22",
"SSH_CONNECTION": "192.168.122.21 37293 192.168.122.21 22",
"USER": "root",
"_": "/usr/bin/python"
},
"ansible_eth0": {
"active": true,
"device": "eth0",
"ipv4": {
"address": "192.168.122.21",
"netmask": "255.255.255.0",
"network": "192.168.122.0"
},
"ipv4_secondaries": [],
"ipv6": [
{
"address": "fe80::5054:ff:fe23:1614",
"prefix": "64",
"scope": "link"
}
],
"macaddress": "52:54:00:23:16:14",
"module": "virtio_net",
"mtu": 1500,
"promisc": false,
"type": "ether"
},
"ansible_form_factor": "Other",
"ansible_fqdn": "ws01",
"ansible_hostname": "ws01",
"ansible_interfaces": [
"lo",
"eth0"
],
"ansible_kernel": "2.6.32-431.el6.i686",
"ansible_lo": {
"active": true,
"device": "lo",
"ipv4": {
"address": "127.0.0.1",
"netmask": "255.0.0.0",
"network": "127.0.0.0"
},
"ipv4_secondaries": [],
"ipv6": [
{
"address": "::1",
"prefix": "128",
"scope": "host"
}
],
"mtu": 16436,
"promisc": false,
"type": "loopback"
},
"ansible_machine": "i686",
"ansible_memfree_mb": 375,
"ansible_memtotal_mb": 498,
"ansible_mounts": [
{
"device": "/dev/mapper/vg_ws01_root-lv_root",
"fstype": "ext4",
"mount": "/",
"options": "rw,noatime",
"size_available": 1166802944,
"size_total": 2113748992
},
{
"device": "/dev/vda1",
"fstype": "ext4",
"mount": "/boot",
"options": "rw,noatime",
"size_available": 165210112,
"size_total": 203097088
}
],
"ansible_os_family": "RedHat",
"ansible_pkg_mgr": "yum",
"ansible_processor": [
"QEMU Virtual CPU version 1.7.0",
"QEMU Virtual CPU version 1.7.0"
],
"ansible_processor_cores": 1,
"ansible_processor_count": 2,
"ansible_processor_threads_per_core": 1,
"ansible_processor_vcpus": 2,
"ansible_product_name": "Bochs",
"ansible_product_serial": "NA",
"ansible_product_uuid": "D17F90A9-2961-401A-ABF2-67A4986DB05A",
"ansible_product_version": "NA",
"ansible_python_version": "2.6.6",
"ansible_selinux": false,
"ansible_ssh_host_key_dsa_public": "AAAAB3NzaC1kc3MAAACBAPEyQcsdKho53k1iGNPovQZcZfWtdKNXszZlhFl64YbSI6hYHmJVLPnQLrsl4wyjrOk49HFsSIM89wAipEHqvr+X3jKOR3PyEIZbeE5B6BKeP8CmDs9Xp7BIyJnwdcJk6w0gOd17nTbjQ4sJDo4FSacTFw5lxTF+8Xq0O9ErJtzBAAAAFQCib5Lo+HBK2iJpHWXLyUqZZzRKQQAAAIEAzHm7CURjGo9geYcvFTsIRvn2GwMqj4TdS2usv4KwhxUl/46a1Jh9/Gp9OZuNNcentUJQGccJf8gIYBVtFA0PcXcuAp6DS4kdk52bhTL+jRARCTb5aXJFUfSZNw5u/E4AhNW9domtiDiu2zEuY+LpMiUDLUSlmRC+BBQkA11dNAsAAACATlZw4Jr1IouATvxdtt+qqG3WrOz82wYcOAVDx8NFAzbFAcUeRswzt9+yWsn1CytTnOtbMwS2psTPIvtYvW8SKDvzlqBrBIUVgJgDVALBRffQ2YIl0XmSNjxZkgz8xvOCbvDFTzBKxbEqgn/0g7XLmfcEQeJvltikyVU4gTc//xU=",
"ansible_ssh_host_key_rsa_public": "AAAAB3NzaC1yc2EAAAABIwAAAQEAqWaeArbN0cZDpm7CX8UnQwF1t8fjm0IDRQIBpbJN21kdvwEppVYCJZw/W+oo66fVyUCRPgI83VeLlqgvObxOQYzUAdfbaEp7yUMelp9wwx1c6PnzbsOWg/lDPW5kXLobKSmECP985cMUFpwb2pA8CjcwhQsS4EIQwZgxIYBqV5MTufLtGTnD3v0qzyRQ7l35srKC4myHL04fk+W4F0+qy2B/gMeY1ViKl/AbIvwO1deZ47Xn1Ier/k7Ou57c9WE6M7z5fl6dBzT6YJzgMEjSdy6U5zV+lYbay5CtSn0rRbiOnKqrhZl8EC0L/+W1DpCeDieikbCKQu0WX7liN3uVJQ==",
"ansible_swapfree_mb": 511,
"ansible_swaptotal_mb": 511,
"ansible_system": "Linux",
"ansible_system_vendor": "Bochs",
"ansible_user_id": "root",
"ansible_userspace_architecture": "i386",
"ansible_userspace_bits": "32",
"ansible_virtualization_role": "guest",
"ansible_virtualization_type": "kvm"
},
"changed": false
}
When the fact you need is not in there, you can also return your own facts.
|
Note
|
explain -vvv |
When managing configuration files you often want to use certain variables in your file. That way one can reuse the same file template for multiple systems.
Let’s configure a virtual host on Apache for the web application we want to deploy.
- name: Configure Apache
hosts: web-servers
user: root
vars:
webapp_dir: /var/www/html
handlers:
- name: restart Apache
action: service name=httpd state=restarted
tasks:
- name: Install Apache
action: yum pkg=httpd state=installed
- name: Add webapp vhost
action: template src=templates/webapp.conf dest=/etc/httpd/conf.d/
notify:
- restart Apache
- name: Start and Enable Apache
action: service name=httpd state=started enabled=yes
Two things are new here. The first one is the template module. This module allows templating a configuration file using the jinja2 templating language (also used in Flask).
The template is simple. It looks like this:
<VirtualHost *:80>
ServerName {{ inventory_hostname }}
DocumentRoot {{ webapp_dir }}
</VirtualHost>
We use our newly defined webapp_dir variable and a magic inventory_hostname variable. The webapp_dir variable is defined in the vars section in the playbook. We’ll look into other ways to define them later.
Of course, if we ever change the configuration again, we want to restart Apache so it picks up the new config. This can be done with handlers and notify. A handler is a normal named task that is run when the task that has it in notify returns changed.
|
Note
|
Add ansible_managed to template |
[root@ws01 workshop]# ansible-playbook apache.yml -i hosts PLAY [Configure Apache] ******************************************************* GATHERING FACTS *************************************************************** ok: [vm-web] TASK: [Install Apache] ******************************************************** ok: [vm-web] TASK: [Add webapp vhost] ****************************************************** changed: [vm-web] TASK: [Start and Enable Apache] *********************************************** changed: [vm-web] NOTIFIED: [restart Apache] **************************************************** changed: [vm-web] PLAY RECAP ******************************************************************** vm-web : ok=5 changed=3 unreachable=0 failed=0
Our web application uses MySQL. For managing MySQL databases with Ansible, the MySQL-Python module is needed. We could use two tasks to install MySQL and the python module, but there is some syntactic sugar that helps.
- name: Configure MySQL
hosts: database-servers
user: root
tasks:
- name: Install MySQL
action: yum pkg={{ item }} state=installed
with_items:
- mysql-server
- MySQL-python
- name: Start MySQL
action: service name=mysqld state=started enabled=yes
The item variable contains the current item.
[root@ws01 workshop]# ansible-playbook mysql.yml -i hosts PLAY [Configure MySQL] ******************************************************** GATHERING FACTS *************************************************************** ok: [vm-db] TASK: [Install MySQL] ********************************************************* ok: [vm-db] => (item=mysql-server,MySQL-python) TASK: [Start MySQL] *********************************************************** ok: [vm-db] PLAY RECAP ******************************************************************** vm-db : ok=3 changed=0 unreachable=0 failed=0
We want to be able to talk to MySQL in the web application we’ll soon write in PHP. Use the with_items syntax to change the Apache installation playbook to install PHP and the MySQL extension.
- name: Install Apache and PHP
action: yum pkg={{ item }} state=installed
with_items:
- httpd
- php-mysql
With the web server and the database up and running, we can start deploying the application.
Let’s first consider the case where the system is fresh. In that case we want to create a database and populate it with initial data. In subsequent reruns, we don’t want to populate the database.
- name: Prepare deployment
hosts: database-servers
user: root
tasks:
- name: Create a temporary directory for database migratons
action: file dest=/tmp/db state=directory
- name: Create the database
action: mysql_db name=ws state=present
register: db_create
- name: Copy initial schema
action: copy src=app/db/initial.sql dest=/tmp/db/
when: db_create|changed
- name: Initialize database
action: mysql_db name=ws state=import target=/tmp/db/initial.sql
when: db_create|changed
- name: Create a ws user
action: mysql_user name=ws host={{ hostvars[item]["ansible_ssh_host"] }} password=''
priv=ws.*:SELECT state=present
with_items:
- groups["web-servers"]
This introduces a few new modules.
We are mostly interested in the first mysql_db task. That task uses the register keyword. This stores the results of that task in the variable db_create.
The last two tasks use the when keyword. A task will only run when the condition in when is true. In this case we use the built-in jinja2 filter changed to only run that task when the task that registered db_create did change the system.
Now we can deploy the application. We will do it in a quick and dirty way, by just copying everything in a particular folder…
- name: Deploy the demo application
hosts: web-servers
user: root
vars:
webapp_dir: /var/www/html
tasks:
- name: Copy all php scripts to the web server
action: copy src={{ item }}
dest={{ webapp_dir }}/
owner=apache group=apache
with_fileglob: app/web/*
This uses the with_fileglob lookup plugin, which provides exactly the functionality you’d expect it to have.
That last play feels wrong. We’ve defined the webapp_dir variable here. We also defined the variable in the Apache playbook. When the webapp location changes we have to remember to change it in two places.
There is a better solution: including variables from files.
- name: Deploy the demo application
hosts: web-servers
user: root
vars_files:
- config.yml
tasks:
- name: Copy all php scripts to the web server
action: template src={{ item }} dest={{ webapp_dir }}/ owner=apache group=apache
with_fileglob: app/web/*
config.yml contains:
webapp_dir: /var/www/html
Application developers have the nasty habit of not being able to come up with a good data model. They require database schema changes with every deployment. This complicates the whole deployment process and makes rollbacks incredibly hard. We will do as every professional PaaS does, or even more, and allow only forward migrations.
This makes our complete deployment playbook:
- name: Prepare deployment
hosts: database-servers
user: root
vars:
db_dir: /tmp/db
tasks:
- name: Create a temporary directory for database migrations
action: file dest={{ db_dir }} state=directory
- name: Create the database
action: mysql_db name=ws state=present
register: db_create
- name: Create a ws user
action: mysql_user name=ws host={{ hostvars[item]["ansible_ssh_host"] }} password=''
priv=ws.*:SELECT state=present
with_items:
- groups["web-servers"]
- name: Copy initial schema
action: copy src=app/db/initial.sql dest={{ db_dir }}/
when: db_create|changed
- name: Initialize database
action: mysql_db name=ws state=import target={{ db_dir }}/initial.sql
when: db_create|changed
- name: Upload database migrations
action: copy src=app/db/migrations.sql dest={{ db_dir }}/
- name: Run database migrations
action: mysql_db name=ws state=import target={{ db_dir }}/migrations.sql
- name: Deploy the demo application
hosts: web-servers
user: root
vars_files:
- config.yml
tasks:
- name: Copy all php scripts to the web server
action: template src={{ item }} dest={{ webapp_dir }}/ owner=apache group=apache
with_fileglob: app/web/*
This playbook has three plays. They are run sequentially.
[root@ws01 workshop]# ansible-playbook deploy.yml -i hosts PLAY [Prepare deployment] ***************************************************** GATHERING FACTS *************************************************************** ok: [ws01] TASK: [Create a temporary directory for database migrations] ******************* ok: [ws01] TASK: [Create the database] *************************************************** ok: [ws01] TASK: [Copy initial schema] *************************************************** skipping: [ws01] TASK: [Initialize database] *************************************************** skipping: [ws01] PLAY [Update the database] **************************************************** TASK: [Create a temporary directory for database migrations] ******************* ok: [ws01] TASK: [Upload database migrations] ******************************************** ok: [ws01] TASK: [Run database migrations] *********************************************** changed: [ws01] PLAY [Deploy the demo application] ******************************************** TASK: [Copy all php scripts to the web server] ******************************** ok: [ws01] => (item=/root/workshop/app/web/index.php) PLAY RECAP ******************************************************************** ws01 : ok=7 changed=1 unreachable=0 failed=0
We can now deploy applications from our management system, which could be you CI system. The typical PaaS allows application deployment through git. That is just a few additional lines of code.
We will add a new directory under /root with a bare git repository (i.e. one without working tree). A git post-receive hook runs the application whenever you push to the repository.
- name: Set up automatic application deployment repository
hosts: vm-master
user: root
tasks:
- name: Clone the workshop repository
action: git repo=/root/workshop
dest=/root/deploy
bare=yes
ignore_errors: yes
- name: Add post-receive hook to deploy
action: copy src=templates/post-receive
dest=/root/deploy/hooks/
mode=0755
Run the new playbook:
[root@ws01 workshop]# ansible-playbook auto-deploy.yml -i hosts PLAY [Set up automatic application deployment repository] ********************* GATHERING FACTS *************************************************************** ok: [vm-master] TASK: [Clone the workshop repository] ***************************************** changed: [vm-master] TASK: [Add post-receive hook to deploy] *************************************** changed: [vm-master] PLAY RECAP ******************************************************************** vm-master : ok=4 changed=2 unreachable=0 failed=0
Try to run it again.
TASK: [Clone the workshop repository] *****************************************
failed: [ws01] => {"cmd": "/usr/bin/git ls-remote origin -h refs/heads/master", "failed": true, "rc": 128}
stderr: fatal: 'origin' does not appear to be a git repository
fatal: The remote end hung up unexpectedly
msg: fatal: 'origin' does not appear to be a git repository
fatal: The remote end hung up unexpectedly
...ignoring
This is something in the Ansible git module that makes incorrect assumptions about bare repositories. It needs to be fixed, but it serves as a good example of the ignore_errors directive.
Add the new repository as a remote of the current repository.
$ git remote add deploy /root/deploy
Now change the title of the application in app/web/index.php, commit and:
root@ws01 workshop]# git add app/web/index.php
[root@ws01 workshop]# git commit -v
[master 8feb5ee] Update title.
Committer: root <root@ws01.(none)>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:
git config --global user.name "Your Name"
git config --global user.email [email protected]
If the identity used for this commit is wrong, you can fix it with:
git commit --amend --author='Your Name <[email protected]>'
1 files changed, 1 insertions(+), 1 deletions(-)
[root@ws01 workshop]# git push deploy master
Counting objects: 9, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 403 bytes, done.
Total 5 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (5/5), done.
remote: Deploying Ansible Workshop Web Application
remote: Initialized empty Git repository in /tmp/ws-deploy/.git/
remote:
remote: PLAY [Prepare deployment] *****************************************************
remote:
remote: GATHERING FACTS ***************************************************************
remote: ok: [vm-db]
remote:
remote: TASK: [Create a temporary directory for database migrations] ******************
remote: ok: [vm-db]
remote:
remote: TASK: [Create the database] ***************************************************
remote: ok: [vm-db]
remote:
remote: TASK: [Copy initial schema] ***************************************************
remote: skipping: [vm-db]
remote:
remote: TASK: [Initialize database] ***************************************************
remote: skipping: [vm-db]
remote:
remote: PLAY [Update the database] ****************************************************
remote:
remote: TASK: [Create a temporary directory for database migrations] ******************
remote: ok: [vm-db]
remote:
remote: TASK: [Upload database migrations] ********************************************
remote: ok: [vm-db]
remote:
remote: TASK: [Run database migrations] ***********************************************
remote: changed: [vm-db]
remote:
remote: PLAY [Deploy the demo application] ********************************************
remote:
remote: TASK: [Copy all php scripts to the web server] ********************************
remote: changed: [vm-web] => (item=/tmp/ws-deploy/app/web/index.php)
remote:
remote: PLAY RECAP ********************************************************************
remote: vm-db : ok=7 changed=2 unreachable=0 failed=0
remote:
To /root/deploy
7874de6..8feb5ee master -> master