MariaDB, MySQL and Ansible: automating database infrastructures

MariaDB, MySQL and Ansible:
automating database infrastructures
Federico Razzoli

$ whoami
Hi, I’m Federico Razzoli from Vettabase Ltd
Database consultant, open source supporter,
long time MariaDB and MySQL user
● vettabase.com
● Federico-Razzoli.com

This talk
● General Ansible concepts
● Opinionated code examples, automating MariaDB/MySQL
● Good and bad practices

What is Ansible
● A tool for deployment and conﬁguration automation
● Infrastructure as code
● Idempotent
● Simple, non-centralised

Ansible Concepts
● Inventory
● Module
● Playbook
● Role
● Play
● Variable
● Facts

Inventory example
[pmm]
pmm-1
[mariadb_main]
mariadb-main-1
mariadb-main-2
mariadb-main-3
[mariadb_main:vars]
pmm_server_host=pmm-1
pmm_server_port=80

Ansible hosts
[pmm]
pmm-1 ansible_host=123.123.123.123
[mariadb_main]
mariadb-main-1 ansible_host=123.0.0.1
[mariadb_main:vars]
pmm_server_host=123.123.123.123
pmm_server_port=80
● Sometimes we cannot use meaningful hostnames

Hierarchical groups
[mariadb_master]
mariadb-master-1
mariadb-master-2
[mariadb_replica]
mariadb-replica-1
mariadb-replica-2
[mariadb:children]
mariadb_master
mariadb_replica
● We have diﬀerent groups for masters and replicas, and a supergroup that
includes them both

Groups goals
● You assign each group some roles
● You can deploy one or more group

Multiple inventories
production_db:
[mariadb_master]
mariadb-master-1 ansible_host=111.0.0.1
staging_db:
[mariadb_master]

Play example
---
- hosts: mariadb_main
roles:
- linux-for-mysql
- mariadb
- hosts: mysql_main
roles:
- linux-for-mysql
- mysql
● You assign roles to groups (or individual hosts)

Applying all roles to a certain host
Call Ansible specifying an inventory and a play:
% ansible-playbook -i staging -l mariadb_main main.yml

Play example
Here is where it can be convenient to have multiple inventories:
% ansible-playbook -i staging -l mariadb_main main.yml
% ansible-playbook -i production -l mariadb_main main.yml

Roles, variables and templates

Example
- name: Create script dir
file:
name: "{{ script_dir }}"
owner: root
group: root
mode: 755
state: directory
● This is a task, but the term is misleading
● A role is a list of tasks
● The name appears on screen
● "ﬁle" is the module
● script_dir is a variable

Deﬁning variables
script_dir: /opt/scripts
base_dir: /usr/local/mysql
...
● /host_vars/<host_name>.yml
● /group_vars/<group_name>.yml
● /roles/<role_name>/defaults/main.yml

Notes on variables
● The same variable can be used by multiple roles
● Roles from Ansible Galaxy can’t use this opportunity
● This is a good reason to make your own roles
● Document what each variable does in README.md

Copying a conﬁguration ﬁle
- name: Copy my.cnf
copy:
src: ./files/my.cnf
dest: "/etc/mysql/{{ inventory_hostname }}.cnf"
● my.cnf is copied to /etc/mysql

Using a template
- name: Copy my.cnf
template:
src: ./templates/my.cnf .j2
dest: /etc/mysql/my.cnf
● This time my.cnf is a Jinja 2 template

Template example
[mysqld]
user = mysql
datadir = {{ mysql_data_dir }}
innodb_buffer_pool_size = {{ innodb_buffer_pool_size }}
● This time my.cnf is a Jinja 2 template

Problems
● If we want to set a variable, we should be able to add it in one place
○ host_vars or group_vars or role's defaults
● We can make a template more dynamic

Dynamic template example
In group_vars we create a list of dictionaries:
(arrays of objects, if you prefer)
group_mysql_variables:
- { name: 'innodb_buffer_pool_size', value: '50G' }
- { name: 'innodb_log_file_size', value: '50G' }
In host_vars we optionally create another list:
host_mysql_variables:
- { name: 'innodb_buffer_pool_size', value: '80G' }

Dynamic template example
We loop over both the lists in the template:
{% for var in group_mysql_variables %}
{{ var.name }} = {{ var.value }}
{% endfor %}
{% if host_mysql_variables is defined %}
{% for var in host_mysql_variables %}
{{ var.name }} = {{ var.value }}
{% endfor %}
{% endif %}

Notes
● We cannot use the same list in group_vars and host_vars, because the
one in host_vars would overwrite the whole list
● Some variables may appear twice in the configuration file
● This is fine: the last occurrence of a variable will override the previous one

Another use for lists
Hosts of a cluster can be a list too:
# print the IPs separated by commas
wsrep_cluster_address = gcomm://{{ private_ips|join(',') }}
# first node is the donor
wsrep_sst_donor = {{ cluster_hosts[0].node_name }}
# Only the first node imports a backup when the cluster is created
- name: Import backup
...
when: cluster_hosts[0] == hostvars[inventory_hostname]['ansible_default_ipv4']['address']

Validate variables
Validate variables for complex roles:
- name: Validate cluster size
assert:
that: cluster_hosts|length > 0
fail_msg: cluster_hosts must contain at least 1 element
success_msg: "cluster_hosts size: {{ cluster_hosts|length }}"
- name: Validate cluster IPs
assert:
that: "'{{ item.public_ip }}'|ipaddr"
fail_msg: "{{ item.public_ip }} is not a valid IP"
success_msg: "{{ item.public_ip }}: OK"
with_items: "{{ cluster_hosts }}"

Conditional tasks
● Certain tasks should only be run for certain servers:
- name: Make server read only
mysql_variables:
variable: read_only
value: 1
when:
is_replica is sameas true
● Sometimes this avoids the need to create multiple roles which are very similar
● Instead of creating mariadb and mariadb_replica roles, we may create
mariadb only, with some conditional tasks
○ Sometimes it makes sense, sometimes it doesn't

Conditional tasks
● We can group optional tasks into separate ﬁles:
- name: Make server read only
include: replica.yml
when:
is_replica is sameas true

Idempotency of Tasks
● Most modules are idempotent
● This means that you can run a task like this multiple times safely:
- name: Start MariaDB
service:
state: started
● If the service is already running, nothing happens

● Sometimes you make non-idempotent tasks on purpose:
- Copy my.cnf
...
- name: Restart MariaDB
service:
state: restarted
● If the service is already running, nothing happens

● Sometimes you'd like a task to be idempotent, but it can't be:
- name: Run a system command
shell: >
rm "{{ mysql_data_dir }}"/*
- name: Run a script
shell: >
"{{ mysql_script_dir }}"/my_script.py
- name: Run some SQL
mysql_query:
query: CREATE OR REPLACE TABLE db.table ( ... );
● Ansible doesn't understand system commands, scripts or queries, so it has not
way to check if they were already run. It runs them every time.

Tags
● Ansible supports tags
● Each task can be assigned a list of tags:
- name: (Re)create users
...
- name: Update timezone info
tags: [ tzinfo-update ]
...
- name: Do something else
...
ansible-playbook -i production_mysql --tag tzinfo-update production.yml

Tags
● Typical tasks I want to have:
- name: Do something
- name: Copy my.cnf
tags: [ mysql-config- static-update ]
- name: Restart MySQL
tags: [ mysql-config- static-update ]
- name: Set variables at runtime
tags: [ mysql-config- dynamic-update ]
- name: Do something else

Tags
● So there is a way to selectively run certain tasks
● But what if we want to selectively exclude certain tasks?
● There is not built-in way, but we can do something like this:
- name: Do something
- name: Restart MySQL
service:
state: restarted
when: {{ mysql_restart }} is defined and {{ mysql_restart }} is sameas true
- name: Do something else mething else
ansible-playbook -i production_mysql
--tag mysql-config-dynamic-update -e 'mysql_no_restart=1' production.yml

Thanks for attending!
Question time :-)

Example
CREATE OR REPLACE TABLE ticket (
id INT PRIMARY KEY NOT NULL AUTO_INCREMENT,
state ENUM('OPEN', 'VERIFIED', 'FIXED', 'INVALID') NOT NULL
DEFAULT 'OPEN',
summary VARCHAR(200) NOT NULL,
description TEXT NOT NULL
)
ENGINE InnoDB
;
● We want to start to track changes to bugs over time

MariaDB, MySQL and Ansible: automating database infrastructures

More Related Content

What's hot (20)

Similar to MariaDB, MySQL and Ansible: automating database infrastructures (20)

More from Federico Razzoli (20)

Recently uploaded (20)

MariaDB, MySQL and Ansible: automating database infrastructures