In this segment, you will learn how to set up and use Ansible to manage servers efficiently.
A tutorial series on building a complete home lab system from the ground up that is beginner-friendly, versatile, and maintainable.
What is Ansible
Ansible is an administrative tool that aims to make server management easier by using declarative and idempotent configuration files.
Declarative VS Imparative
Declarative means you are expressing the what without the how, this makes things easier to read and make sense of. Below is a contrived example of the difference:
Declarative
- The file
settings.confshould include the linesome_setting=true
Imparative
- Check if
settings.confexists - Check if
settings.confincludes the linesome_setting - If not, add the line
some_setting=true - Else check if the value of
some_settingistrue - If not, change the line to
some_setting=true
Idempotent
For declarative to work, the operations must be idempotent.
In simple terms, idempotent configurations don’t need to know about the current state, they must handle all possible starting states and ensure that whatever state is declared will be the final state upon completion. This means that you can define the state you want, and applying the same configuration again shouldn’t make any changes that aren’t needed.
For example, if we want to ensure that a package is installed, applying the configuration should either install the package if it isn’t already, or do nothing at all.
Note: You may see a warning in the Ansible documentation for some plugins about not being idempotent, this means you have to handle state checking yourself, this is pretty rare but worth mentioning
Benefits
Picture this; something is failing on your server, and you’ve spent hours combing over configuration files and scanning logs trying to figure out why and get it working again. In this case you’re treating your server as a pet, it’s special and irreplaceable. Wouldn’t it be better if you could treat servers as livestock - expendable and easily replaced?
For example; something is failing on your server, so you just re-apply the configuration. It’s fast and easy, and if that won’t cure it, you can put the poor thing down, spin up a new one, and copy over any mutable data. Sure, you still need to investigate, but you’re no longer under pressure because the service is up and running on a new host or virtual machine.
Another benefit is that backups will be faster, cheaper, and easier to manage if you only need to back up your data. There is no need to back up an entire server when all the configuration you need is already tucked away in a git repository. Save all that pricey off-site storage space for your mutable data instead of filling it with the entire contents of a hard drive.
Setup
First, you should install Ansible, on most distributions you can use pip to install it, but there are some
more detailed instructions
if you need them.
Create a folder to use, I went with one called ansible in my home directory, but you can keep it and call it whatever you like, you’ll just have to keep that in mind when creating the Ansible config file.
mkdir ~/ansible && cd ~/ansible
Now we need a configuration file called ansible.cfg, there are a lot of configuration options for Ansible and we’re only using a few here, you should check out the Ansible documentation as well, they did a great job with that.
An enterprise automation platform for the entire IT organization, no matter where you are in your automation journey
# ~/ansible/ansible.cfg
[defaults]
inventory = ~/ansible/inventory.yml
roles_path = ~/ansible/roles
vault_password_file = ~/.ansible_vault
The inventory file will list all of your hosts, we will get into that and roles in a bit, first let’s address that vault password file.
Ansible Vault is a utility for securely storing sensitive information, like passwords and such. The “vault” files are encrypted so they can be safely stored in version control, like GitHub for example.
Create the file ~/.ansible_vault (you can call this something different if you want, just change it in ansible.cfg to match), in this file you should enter a very long random string, save it in your password manager or write it down and keep it somewhere safe.
Once that is done, working with vaults will be seamless, you won’t be asked to enter the password when working with vault files, Ansible will read it from that file automatically.
Inventory
Create the file inventory.yml, here we will define all hosts managed by Ansible, and optionally some global and host-specific variables. Don’t go crazy with variables here, there are better ways to store host-specific variables.
# ~/ansible/inventory.yml
all:
vars:
# These are global variables, they can be overwritten at the host level
ansible_user: the_default_username_for_external_hosts
# We can use variables by putting them in double curly brackets like this
# these variables are assigned in a vault file and read here
ansible_ssh_pass: '{{ host_default_pass }}'
# The "become_pass" is used from sudo commands
ansible_become_pass: '{{ host_default_pass }}'
ansible_python_interpreter: auto_silent
# This is a neat little trick to use short hostnames like myhost
# instead of myhost.mynetwork.local, the domain can be overwritten
# at the host level as well
ansible_host: '{{ inventory_hostname }}.{{ host_domain }}'
host_domain: 'mynetwork.local'
hosts:
# These are hosts on the global level, we can also group hosts as
# seen below under "children"
some_hostname:
another_hostname:
# This host has a variable overwriting the global become_pass variable
third_host:
host_domain: someothernetwork.tld
children:
# These are host groups, we can use groups later to target
# hosts as a group instead of individually
webservers:
hosts:
some_hostname:
firewalls:
hosts:
another_hostname:
virtual_machines:
hosts:
some_hostname:
third_host:
Vaults and Variables
Let’s create a global vault first, if we wanted unencrypted variables we could use the same filename without Ansible Vault, it would work the same way.
mkdir -p ~/ansible/group_vars/all
ansible-vault create ~/ansible/group_vars/all/variables.yml
Now we can add global variables that are secure, we can do the same for other host groups like webservers, etc. in the inventory file.
# ~/ansible/group_vars/all/variables.yml
host_default_pass: super-secret-password
For host variables, we just put the file in a different place.
mkdir -p ~/ansible/host_vars/third_host/
ansible-vault create ~/ansible/host_vars/third_host/variables.yml
Here we can add variables for third_host.
# ~/ansible/host_vars/third_host/variables.yml
secret_value: sensitive-data
mysql_password: super-secret-password
Playbooks
Now we can write playbooks to configure one or more of the hosts in our inventory. Let’s start with third_host as a simple example.
mkdir -p ~/ansible/playbooks/third_host
We’ll start with main.yml, I like to have one main file and import tasks from other files to keep things easy to parse, but you can also put the tasks inline under the tasks key instead.
Note: tags can be used to only run specific sets of tasks, e.g. ansible-playbook ~/ansible/playbooks/third_host/main.yml --tags firewall,some_other_tag. And tasks tagged with never will only run if another tag on that task is specified in the commands tag list.
# ~/ansible/playbooks/third_host/main.yml
- hosts: third_host
tasks:
- import_tasks: ufw.yml
tags: firewall
Then create the task for configuring UFW, this is a good example of looping and list handling in Ansible; anything defined in with_items will be looped over in the rule and each item in the list will be available as {{ item }}.
Note: the line become: true means “run this task with sudo”
# ~/ansible/playbooks/third_host/ufw.yml
- name: Configure UFW
become: true
community.general.ufw: '{{ item }}'
with_items:
- comment: Default deny in
default: deny
direction: incoming
- comment: Default deny forward
default: deny
direction: routed
- comment: Default allow out
default: allow
direction: outgoing
- comment: Allow HTTP from 0.0.0.0/0
rule: allow
port: '80'
proto: tcp
from_ip: 0.0.0.0/0
- comment: Allow HTTPS from 0.0.0.0/0
rule: allow
port: '443'
proto: tcp
from_ip: 0.0.0.0/0
Files
The files directory is for, well, files. These are static files, maybe certificates, or configuration files although it usually makes more sense to use lineinfile to manipulate the configuration file that exists on the remote host, again check out the Ansible documentation, lots of good stuff in there.
Another example of a task, this one ensures the directory /etc/docker exists, and ensures the file daemon.json exists and matches the copy we have locally. The {{ inventory_dir }} variable is one of many built-in variables in Ansible, and points to the inventory file path, in this case that would be ~/ansible/files/third_host/docker/daemon.json.
# ~/ansible/playbooks/traefik_host/install_docker.yml
- name: Ensure Docker config path exists
become: true
file:
path: '/etc/docker'
state: directory
- name: Configure Docker data path
become: true
register: docker_data_path
copy:
src: '{{ inventory_dir }}/files/third_host/docker/daemon.json'
dest: /etc/docker/daemon.json
owner: root
group: root
mode: '0644'
Templates
Templates are for dynamic files, very similar to files, but you can use Ansible variables in the files, and they should have a .j2 (Jinja2) file extension.
Here is an example template for a Traefik config file that uses the variable admin_email.
# ~/ansible/templates/traefik.yml.j2
providers:
docker:
exposedbydefault: false
entryPoints:
http:
address: ':80'
http:
redirections:
entryPoint:
to: https
scheme: https
permanent: true
https:
address: ':443'
forwardedHeaders:
trustedIPs:
- '127.0.0.1/32'
- '172.16.0.0/12'
certificatesResolvers:
letsencrypt:
acme:
email: '{{ admin_email }}'
storage: /etc/traefik/acme.json
httpChallenge:
entryPoint: http
Then we can use that template in a task, and any variables will be replaced by their files in the output file.
# ~/ansible/playbooks/traefik_host/install_traefik.yml
- name: Install Traefik config
become: true
template:
src: '{{ inventor_dir }}/templates/traefik.yml.j2'
dest: /etc/traefik/traefik.yml
owner: root
group: root
mode: '0640'
register: traefik_config
Roles
This is where Ansible gets really powerful really quickly, roles are re-usable sets of playbooks that can be applied to multiple top-level playbooks. There is also a public repository of roles called Ansible Galaxy, where you can share your roles or use roles created by others.
Creating a Role
There is a handy command to scaffold out a new role for Ansible, let’s try that out now.
mkdir ~/ansible/roles
ansible-galaxy init ~/ansible/roles/my_role_name
We now have a bunch of new files ready to go for our new role, these are pretty well documented by Ansible, but here’s a quick overview:
my_role_name/ #
tasks/ #
main.yml # <-- tasks file can include smaller files if warranted
handlers/ #
main.yml # <-- handlers file
templates/ # <-- files for use with the template resource
ntp.conf.j2 # <------- templates end in .j2
files/ #
bar.txt # <-- files for use with the copy resource
foo.sh # <-- script files for use with the script resource
vars/ #
main.yml # <-- variables associated with this role
defaults/ #
main.yml # <-- default lower priority variables for this role
meta/ #
main.yml # <-- role dependencies
library/ # roles can also include custom modules
module_utils/ # roles can also include custom module_utils
lookup_plugins/ # or other types of plugins, like lookup in this case
The variables in the defaults directory can be overwritten in other places like the playbook and host or group variables, as you might expect.
You’ll see a lot of patterns repeated here because a role is kind of like a self-contained playbook of its own.
Handlers
One new directory that is significant here is handlers, these are like tasks, but a handler is run once at the end of the role execution. This is useful for things like restarting services when configuration files change, if three configuration files are changed and they are all tied to the same handler, the service will restart once after all the changes were made.
# my_role_name/handlers/main.yml
- name: restart_sshd
become: true
service:
name: sshd
state: restarted
We can now call that handler from a task using the notify key, in this case we are using lineinfile to make sure certain configuration lines are set the way we want them to be.
# my_role_name/tasks/configure_sshd.yml
- name: Configure sshd
become: true
lineinfile:
path: '/etc/ssh/sshd_config'
regex: '^(#)?{{item.key}}'
line: '{{item.key}} {{item.value}}'
state: present
loop:
- { key: 'StrictModes', value: 'yes' }
- { key: 'PermitRootLogin', value: 'no' }
- { key: 'PasswordAuthentication', value: 'no' }
- { key: 'PermitEmptyPasswords', value: 'no' }
- { key: 'ChallengeResponseAuthentication', value: 'no' }
- { key: 'UsePAM', value: 'no' }
- { key: 'HostbasedAuthentication', value: 'no' }
notify:
- restart_sshd
Assign Roles to a Playbook
Now all we need to do is declare any roles we want to be applied in our playbook.
# ~/ansible/playbooks/third_host/main.yml
- hosts: third_host
roles:
- my_role_name
- some_other_role
tasks:
- import_tasks: ufw.yml
tags: firewall
A Few More Neat Tricks
I’ve only just scratched the surface of what Ansible can do here, and I’ve got a couple more necessities to share, but there is so much more! I encourage you to poke around and experiment!
Register
Register lets you assign a variable dynamically, for example if you wanted to know if a task changed or was already applied.
- name: Install Traefik config
become: true
template:
src: '{{ role_path }}/templates/traefik.yml.j2'
dest: /etc/traefik/traefik.yml
owner: root
group: root
mode: '0640'
register: traefik_config
You can then use that variable in a later task. In this case we want to force recreate the Traefik container if the configuration was changed, otherwise only recreate it if Docker thinks it’s necessary.
- name: Start Traefik
become: true
community.docker.docker_compose:
project_name: traefik
remove_orphans: true
recreate: "{{ 'always' if traefik_config.changed else 'smart' }}"
definition:
version: '3.3'
services:
traefik:
image: 'traefik:v2.7'
container_name: 'traefik'
restart: unless-stopped
labels:
- 'com.centurylinklabs.watchtower.enable="true"'
ports:
- target: 80
published: 80
mode: host
- target: 443
published: 443
mode: host
volumes:
- '/var/run/docker.sock:/var/run/docker.sock:ro'
- '/etc/localtime:/etc/localtime:ro'
- '/etc/traefik/:/etc/traefik/'
networks:
- default
networks:
default:
external:
name: gateway
You can also use these dynamic variables with the when key.
- name: Install Traefik config
become: true
template:
src: '{{ role_path }}/templates/traefik.yml.j2'
dest: /etc/traefik/traefik.yml
owner: root
group: root
mode: '0640'
register: traefik_config
- name: Restart Traefik
community.docker.docker_container:
name: traefik
state: restarted
when: traefik_config.changed
Logging
Arguably the most important thing to know is how to log things out to the console! Fortunately, that’s pretty easy.
- name: Log out data
debug:
msg: Here I am logging some text and a variable - {{ some_variable }}
Summary
Now you’ve seen what Ansible is capable of, or at least a glimpse of it, and how to get started. I hope you found this useful!
If you want to see a working example I’ve set up a demo playbook that runs a Traefik instance for Docker apps, but the catch is you might have to do a little research to get it working, let me know how you do.