Ansible fest Presentation slides

Who are we?
Jim Vanns
Aaron Carey
Production Engineers at ILM London

What we’re covering
● Deploying a Mesos cluster in the cloud using
Ansible
● Running services on Mesos using Ansible
● Differences between AWS/GCE modules
● Tips we’ve learnt along the way

Ansible Dockerfile (example follows)
Fix the ansible version
Install a custom dynamic inventory file (or two)
Consider your credentials and security
Install (bake-in a ‘git archive’) your playbooks

Concise Ansible Dockerfile
FROM ubuntu:14.04
RUN apt-get update && apt-get -y install python python-pip python-dev
RUN pip install -U ansible==1.9.4 boto apache-libcloud httplib2
ADD ansible.cfg /etc/ansible/ansible.cfg
ADD ec2.py /etc/ansible/ec2.py
ADD ec2.ini /etc/ansible/ec2.ini
ADD gce.py /etc/ansible/gce.py
ADD gce.ini /etc/ansible/gce.ini
WORKDIR /srv/ansible
ADD ansible .

Requirements
Cloud Agnostic (Work with both AWS and GCE)
Split provisioning and bootstrapping into roles
Need a way to determine which are new hosts
Need a way to group hosts during and after provisioning
Tagging!

Cloud Start Role
# Launch the primary (master/leader) nameserver
- name: Launch a bootstrap consul server
hosts: localhost
connection: local
gather_facts: False
vars:
cloud_provider: ec2
count: 3
service_name: Consul
launch_group: tag_consul_server_{{ consul_domain }}
tags:
consul: server_{{ consul_domain }}
roles:
- cloud_start

Cloud Start Role
---
- include: "{{ cloud_provider }}.yml"

Cloud Start Role
- name: Launch instance
ec2:
key_name: "{{ key_name }}"
instance_type: "{{ instance_type }}"
image: "{{ image }}"
wait: yes
vpc_subnet_id: "{{ vpc_subnet_id }}"
group_id: "{{ security_group_id }}"
region: "{{ region }}"
instance_tags:
ansible: base
count: "{{ count }}"
assign_public_ip: "{{ public_ip }}"
register: ec2
- name: Add instances to launched group
add_host: hostname={{ item.private_ip }} groupname=launched_cloud_default
with_items: ec2.instances
- name: Add instances to parameter group
add_host: hostname={{ item.private_ip }} groupname={{ launch_group }}

Cloud Start Role
- name: Tag instances
ec2_tag: resource={{item.id}} state=present region={{region}}
args:
tags:
"{{ tags }}"
- name: Wait for sshd to come up
wait_for: host={{ item.private_ip }}
port=22
state=started
delay=60
timeout=300
connect_timeout=2

Taking it further...
Parallelise the provisioning
Take advantage of host groups where you can
Use ansible environment variables to target hosts and groups

AWS vs GCE
AWS - Tags are key-value pairs
GCE - Tags are string labels, Metadata are key-value pairs
We used a custom (slightly modified) GCE inventorytags = node.extra['tags']
for t in tags:
tag = 'tag_%s' % t
if groups.has_key(tag): groups[tag].append(name)
else: groups[tag] = [name]
if 'items' in node.extra['metadata']:
for item in node.extra['metadata']['items']:
tag = 'tag_%s_%s' % (item['key'], item['value'])
if groups.has_key(tag): groups[tag].append(name)
else: groups[tag] = [name]

Our Mesos Architecture
Zookeeper*
Mesos Master*
Mesos Agents*
Consul
Marathon (The scheduler’s scheduler)
Mesos-consul

Deploying Zookeeper
- name: Set zookeeper ID facts
gather_facts: True
hosts: tag_zookeeper_server_{{ consul_domain }}
user: ilm-user
tasks:
- set_fact: zkid={{ item.0 | int + 1 }}
when: hostvars[item.1]['ansible_hostname'] == ansible_hostname
with_indexed_items: groups['tag_zookeeper_server_{{ consul_domain }}']
- name: Apply zookeeper role
gather_facts: True
hosts: tag_zookeeper_server_{{ consul_domain }}
sudo: True
user: ilm-user
roles:
- zookeeper

Zookeeper Role
- name: Register zookeeper name with consul
uri: >
url=http://127.0.0.1:8500/v1/agent/service/register
HEADER_Content-Type=application/json
method=PUT
body_format=json
body='{
"Name": "zookeeper",
"Tags": [
"zookeeper",
"{{ zkid }}"
],
"Port": 2181
}'
- name: Register individual zookeeper node with consul
uri: >
url=http://127.0.0.1:8500/v1/agent/service/register
method=PUT
body_format=json
body='{
"Name": "zookeeper-{{ zkid }}",
"Tags": [
"zookeeper",
"{{ zkid }}"
],
"Port": 2181
}'

Zookeeper Role
- name: Run zookeeper container
docker:
name: zookeeper
image: "mesoscloud/zookeeper:3.4.6-ubuntu-14.04"
state: started
net: host
restart_policy: always
volumes:
- /mnt/data/log:/tmp/zookeeper
env:
MYID: "{{ zkid }}"
SERVERS: "zookeeper-1,zookeeper-2,zookeeper-3"

Mesos Containers
- name: Run mesos-master container
docker:
name: mesos-master
image: "mesosphere/mesos-master:{{ img_version }}"
state: started
net: host
restart_policy: always
volumes:
- /mnt/data/log:/var/log
env:
MESOS_ZK: "zk://zookeeper:2181/mesos"
MESOS_CLUSTER: "{{ cluster_name }}"
MESOS_QUORUM: "1"
MESOS_LOG_DIR: "/var/log/mesos"
MESOS_WORK_DIR: "/var/lib/mesos"
MESOS_HOSTNAME: "mesos-master"

Submit tasks to Marathon
- name: Launch docker-registry
hosts: "tag_build_docker_{{ consul_domain }}"
gather_facts: False
tasks:
- name: Submit docker-registry job to marathon
uri: >
url=http://marathon:8080/v2/apps
method=POST
status_code=200,201,409
body_format=json
body='{
"args": [ ],
"container": {
"type": "DOCKER",
"docker": {
"network": "HOST",
"image": "registry:2.2",
"forcePullImage": true,
"parameters":
[
{ "key": "env", "value": "REGISTRY_STORAGE=s3" }
]
}
},
"id": "docker-registry",
"instances": 1,
"cpus": 2,
"mem": 4096
}'

Ansible fest Presentation slides

More Related Content

What's hot (20)

Viewers also liked (11)

Similar to Ansible fest Presentation slides (20)

Recently uploaded (20)

Ansible fest Presentation slides