ansible

Ansible

is an automation framework that works agentless by leveraging SSH to access remote machines.

Main components:

• Control node: where Ansible is installed (you run commands like ansible, ansible-inventory or ansible-vault from here)
  • Multiple are possible but Ansible does not coordinate across them (see AAP if this is desired)
• Inventory (aka hostfile): one or more lists of managed nodes
• Managed nodes (aka hosts): systems to manage via Ansible

Installation

pip install ansible

Inventory

An inventory manages your nodes (e.g. hosts).

• Nodes are specified by FQDN or IP address
• Make sure you added the SSH key of the control node to all of your managed nodes

The inventory consists of a .ini or .yml file structure:

ini

[someHostGroup]
myhost.lan
102.23.02.01

We'll focus, however, on the yml format.

yml

---
someHostGroup:
  hosts:
    a_host:                                 # Reachable via this name as hostname
    my_host:
      ansible_host: host_alias_fqdn.lan
    another_host:
      ansible_host: 192.168.1.222
      ansible_user: user_used_to_log_in_via_ssh
    webserver:
      ansible_host: 102.23.02.01:5555       # To define a non-standard SSH port
      # or via:
      ansible_port: 5555
      # Some variable for a specific host
      http_port: 443                        # Has precedence over http_port in vars below
    # Variables valid for all hosts in a group
  vars:
      a_group_variable: 1234
      http_port: 443
      http_port: 999                      # Variables can be overwritten (order matters!)

someOtherHostGroup:
  hosts:
    another_host:
      ansible_host: another_host.lan
    www[01:10:2].helloworld.com:             # Matches www01.helloworld.com, www03.helloworld.com, www05.helloworld.com, ... (01 until 10 (inclusive); step size = 2); see also: https://docs.ansible.com/ansible/2.7/user_guide/intro_patterns.html#working-with-patterns
    ~(sftp|portal)\.helloworld\.com:         # Tell ansible you are using a RegEx by prefixing `~`
  
someMetaGroup:
  children:
    someHostGroup:
    someOtherHostGroup:

• 2 spaces per indent level
• ansible_*, hosts are reserved keywords
• More about host patterns
• groups are usually clustered as what (topology, type, ...) / where (geographic location) or when (stage)
• metagroups are used to organise multiple groups
  • children is a reserved keyword to include groups (notice the trailing : after the group names)
  • Circular relationships are not allowed
• default groups created by Ansible
  • all: Every host in the inventory
    • If you want to set variables for the all group inside the inventory file, the all group must be the first entry in the file
  • ungrouped: Any host which does not belong to a group

Inventory commands

Verify the inventory

ansible-inventory -i inventory.ini --list

Run playbook for one inventory

# Run only on region1:
ansible-playbook site.yml -i ./inventory.yml --limit region1
# or on a local host file:
ansible-playbook site.yml --limit @my_local_host_list.txt   # Note the prefix: `@`; file needs to be comma separated

Execute commands on an inventory without a playbook

Ping a inventory group (here someHostGroup):

ansible -u myUsername someHostGroup -m ping -i inventory.ini

-> you only need -u if your user name on the managed node(s) is different from the username from the control node

Inventory structure

└── inventory
    ├── 01-inventory-loaded-first.yml
    ├── 02-inventory-loaded-second.yml
    └── 03-inventory-groups-loaded-third.yml

• Internally Ansible merges all inventories into one
• Load order is based on the ASCII character sequence
• If you cross-file reference hosts or groups make sure they are already loaded (-> check load order)
  • In our example we define groups in the last file (loaded) to ensure that all hosts mentioned in the group definitions are already loaded

Playbooks

= automation blueprints in .yml

Build a playbook

• Play: ordered list of Tasks mapped to nodes in a playbook
  • Roles: Reusable Ansible content (tasks, handlers, variables, plugins, templates and files) to be used inside a Play
• Task: call to an Ansible module (without host information to be called on)
  • Handlers: special form of a Task: executes only when notified by a previous Task (status change to changed)
• Module: code or binary that Ansible runs on the managed nodes
  • Each Module has one particular use
  • Included Collections
  • Collections: Groups Modules
    • Can contain playbooks, roles, modules and plugins
    • Install via Ansible Galaxy
• Plugins: Code that expands Ansible's core capabilities

Plug-ins (executed on the control node) vs. modules (executed on the managed nodes)

Playbooks by example

---
- name: My first play       # Start of Play
  hosts: someHostGroup
  tasks:
  - name: Ping my hosts     # Tasks
    ansible.builtin.ping:   # Doc: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/ping_module.html
  - name: Print message
    ansible.builtin.debug:  # Doc: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/debug_module.html#ansible-collections-ansible-builtin-debug-module
    msg: Hello world
  - name: Print multiple messages
    ansible.builtin.debug:
    msg:
      - "Hello"
      - "Hello again!"
- name: Apt package management
  hosts: someOtherHostGroup
  tasks:
  - name: Run equivalent of `apt-get update` as a separate step
    become: true            # Do something as root (see also: https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_privilege_escalation.html)
    ansible.builtin.apt:    # Doc: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/apt_module.html
      update_cache: yes     # Equivalent of apt-get update
  - block:                  # Use a `block` (reserved keyword) to apply certain parameters to all tasks (here: become)
    - name: Update the repo cache and install the package `pwgen`
    ansible.builtin.apt:
      name: pwgen
      update_cache: yes
    - name: Install the package `pwgen` with a more or less (due to the wildcard) specific version or below and allow downgrade
    ansible.builtin.apt:
      name: pwgen=2.08*
      allow_downgrade: yes    # Allow downgrading the package
      autoremove: true        # Automatically invoke `apt autoremove`
    - name: Install multiple packages
      ansible.builtin.apt:
        pkg:
        - pwgen               # Here attributes like above are not allowed
        - git>=2.4*
    become: true
- name: Change files
  hosts: someOtherHostGroup
  tasks:
    - name: Ensure X11 forwarding is activated
      become: yes
      ansible.builtin.lineinfile:   # Doc: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/lineinfile_module.html
        path: /etc/ssh/sshd_config
        regexp: '^(\s*X11Forwarding\s+)(\w+)$'
        line: '\1yes'         # Use a backreferences in our replacement string (first group: `\1`)
        backrefs: yes         # Use backreferences in regex (basically if you have groups in the regex you can fill in the groups as part of the result)
        validate: 'sshd -t -f %s' # Command to verify the configuration before file replacement (%s provides the temporary file path (must be present))

• Using blocks for error handling

Facts

Get information (=facts) about a host remotely via a task in a playbook:

# ...
  tasks:
    - name: Print all available facts
      ansible.builtin.debug:
        var: ansible_facts
    - name: Print Ansible fact via dict access
      ansible.builtin.debug:
        var: ansible_facts.distribution
    - name: Print Ansible fact via Unsafe Text Access
      ansible.builtin.debug:
        var: ansible_facts["distribution"]
    - name: Print Ansible fact via Unsafe Text and list access
      ansible.builtin.debug:
        var: ansible_facts["all_ipv6_addresses"][1]
    - name: Print Ansible fact type we try to access via type_debug filter
      ansible.builtin.debug:
        var: ansible_facts["all_ipv6_addresses"][1]|type_debug

The syntax is exactly how you would expect it in Python (lists: [2], dict: ansible_facts.memory.mb_real or ["key access"]).

Directly access fact on local host:

ansible <hostname> -m ansible.builtin.setup

Run

ansible-playbook -i ./inventory.yml hello.yml
# Multiple inventories
# ... mentioned individually
ansible-playbook -i ./inventory-eu.yml -i ./inventory-na.yml hello.yml
# ... or a whole folder

ansible-playbook -i ./inventory hello.yml
# Ask for become password
ansible-playbook install.yml -i ./inventory.yml -K              # -K == --ask-become-pass; -k == --ask-pass (connection password)

• More about become

Best practises

Directory Layout

production                # inventory file for production servers
staging                   # inventory file for staging environment

group_vars/
   group1.yml             # here we assign variables to particular groups
   group2.yml
host_vars/
   hostname1.yml          # here we assign variables to particular systems
   hostname2.yml

library/                  # if any custom modules, put them here (optional)
module_utils/             # if any custom module_utils to support modules, put them here (optional)
filter_plugins/           # if any custom filter plugins, put them here (optional)

site.yml                  # master playbook
webservers.yml            # playbook for webserver tier
dbservers.yml             # playbook for dbserver tier

roles/
    common/               # this hierarchy represents a "role"
        tasks/            #
            main.yml      #  <-- tasks file can include smaller files if warranted
        handlers/         #
            main.yml      #  <-- handlers file
        templates/        #  <-- files for use with the template resource
            ntp.conf.j2   #  <------- templates end in .j2
        files/            #
            bar.txt       #  <-- files for use with the copy resource
            foo.sh        #  <-- script files for use with the script resource
        vars/             #
            main.yml      #  <-- variables associated with this role
        defaults/         #
            main.yml      #  <-- default lower priority variables for this role
        meta/             #
            main.yml      #  <-- role dependencies
        library/          # roles can also include custom modules
        module_utils/     # roles can also include custom module_utils
        lookup_plugins/   # or other types of plugins, like lookup in this case

    webtier/              # same kind of structure as "common" was above, done for the webtier role
    monitoring/           # ""
    fooapp/               # ""

(from https://docs.ansible.com/ansible/2.8/user_guide/playbooks_best_practices.html#content-organization)

Misc

• Special variables
• Ansible configuration
  • Special command line options and other things can be configured statically in Ansible via the ansible.cfg file

Troubleshooting

• [WARNING]: Could not match supplied host pattern, ignoring: myhosts
  • The host group mentioned in your playbook might not be in the inventory

yml reference (YAML 1.1)

• YAML 1.1 reference card (as seen below)
• YAML 1.2 specification

Collection indicators

• ?: Key indicator
• :: Value indicator
• -: Nested series entry indicator
• ,: Separate in-line branch entries
• []: Surround in-line series branch
• {}: Surround in-line keyed branch

Scalar indicators

• '' : Surround in-line unescaped scalar ('' escaped ')
• ": Surround in-line escaped scalar (see escape codes below)
• |: Block scalar indicator
• >: Folded scalar indicator
• -: Strip chomp modifier ('|-' or '>-')
• +: Keep chomp modifier ('|+' or '>+')
• 1-9: Explicit indentation modifier ('|1' or '>2') # Modifiers can be combined ('|2-', '>+1')

Alias indicators

• &: Anchor property
• *: Alias indicator

Tag property

Usually unspecified

• none: Unspecified tag (automatically resolved by application)
• !: Non-specific tag (by default, "!!map"/"!!seq"/"!!str")
• !foo: Primary (by convention, means a local "!foo" tag)
• !!foo: Secondary (by convention, means "tag:yaml.org,2002:foo")
• !h!foo: Requires "%TAG !h! " (and then means "foo")
• !<foo>: Verbatim tag (always means "foo")

Document indicators

• %: Directive indicator
• ---: Document header
• ...: Document terminator

Misc indicators

• #: Throwaway comment indicator
• ``@`: Both reserved for future use

Special keys

• =: Default "value" mapping key
• <<: Merge keys from another mapping

Core types

Default automatic tags

• !!map: { Hash table, dictionary, mapping }
• !!seq: { List, array, tuple, vector, sequence }
• !!str: Unicode string

More types

• !!set: { cherries, plums, apples }
• !!omap: [ one: 1, two: 2 ]

Language Independent Scalar types

{ ~, null } : Null (no value) [ 1234, 0x4D2, 02333 ] : [ Decimal int, Hexadecimal int, Octal int ] [ 1_230.15, 12.3015e+02 ]: [ Fixed float, Exponential float ] [ .inf, -.Inf, .NAN ] : [ Infinity (float), Negative, Not a number ] { Y, true, Yes, ON } : Boolean true { n, FALSE, No, off } : Boolean false ? !!binary >: e.g.: R0lG...BADS= : >-: Base 64 binary value

Escape codes

• Numeric : { "\x12": 8-bit, "\u1234": 16-bit, "\U00102030": 32-bit }
• Protective: { "\\": '\', "\"": '"', "\ ": ' ', "\<TAB>": TAB }
• C : { "\0": NUL, "\a": BEL, "\b": BS, "\f": FF, "\n": LF, "\r": CR, "\t": TAB, "\v": VTAB }
• Additional: { "\e": ESC, "\_": NBSP, "\N": NEL, "\L": LS, "\P": PS }
• ...

from https://yaml.org/refcard.html (formatting modified)