Using Ansible for Deploying to Cloud Environments

Using Ansible for Deploying
to Cloud Environments
Andrew Hamilton

Who am I?
● Engineering Operations Lead for Prevoty
● Mentor META Lab at CSU, Northridge
● Formerly SRE @Twitter Search and Sys
Admin for Eucalyptus (now HP)

What will we discuss?
● What was the problem to solve?
● Why we chose Ansible
● Tips for using Ansible for deploys
● Working in the cloud and beyond

What was the problem to solve?
We needed a simple and repeatable way to
build services and push them out to properly
configured environments.

Previously used tools wouldn’t cut it
● Puppet
● Capistrano
● Fabric

Multiple languages
● We use:
○ Go
○ Java
○ PHP
● Needed a way to easily build and package
any of these

Moved from EC2 Classic to VPC
Image from: http://docs.aws.amazon.
com/opsworks/latest/userguide/workingstacks-
vpc.html

Moved from EC2 Classic to VPC
● Moved majority of services into private
subnets
● Direct access to instances now limited
○ ELBs are the default access method for pub traffic
○ Bastion host setup in each VPC for SSH access
○ SSH config used for “routing” access to bastions

SSH Config
~/.ssh/config
Host bastion
Hostname <public_dns_hostname_west>
IdentityFile <pem>
Host bastion-east
Hostname <public_dns_hostname_east>
IdentityFile <pem>
Host *.us-west-2.compute.internal
ProxyCommand ssh bastion nc -q 10 -w 10 %h %p 2>/dev/null
Host *.ec2.internal
ProxyCommand ssh bastion-east nc -q 10 -w 10 %h %p 2>/dev/null

Moved towards ephemeral instances
● Nodes are usually rolled between releases
● Use a blue-green deployment process

Why we chose Ansible
We had a developer working on a tool but this
isn’t our core competency so it was better to
move away from the responsibility of building
our own.

Focus on our core competencies
● We’re not in the deployment automation
business
● No need to build a tool if a sufficient one
already exists

Ansible has a simple execution model
● It is easier to understand than the
declarative model used by Puppet
● Execution happens in order

Open source core written in Python
● Easy to extend and update when needed
● Easy to run from HEAD or a branch

YaML is a simple language
● Easy for devs to also add and fix playbooks

SSH based communication
● Don’t need to install anything on new
instances
● Great for the cloud where instances are
created and destroyed often
● No changes needed to security groups
● Respects SSH configs

Simple secret storage
● ansible-vault command
● Integrates automatically with playbooks
● AES encrypted text can sit in version
control

Modules for almost everything
● Makes it super easy to get things done
● From file creation to working with load
balancers and beyond
● Majority are idempotent

Modules can be in any language
● Take in JSON and produce JSON

Ansible is well suited for the cloud
● Dynamic inventories
● Both configuration management and
remote command execution
● Run it when you need it

Use a dynamic inventory
● The cloud is ephemeral
● Standardize on a way to find instances
● ec2.py
○ Uses format tag_<tag_name>_<tag_value>
○ For new hosts: tag_<tag_name>_<tag_value>_new

Configure a dynamic inventory
● Configure it to work for you
● ec2.py and ec2.ini
○ Configured to provide the private DNS even if
public DNS does exist

Break up your playbooks
● Keep playbooks small
● We break ours on verbs:
○ Provision
○ Setup
○ Deploy
○ Promote
○ Terminate

Learn variable hierarchy
● From the Ansible docs
○ extra vars (-e in the command line) always win
○ then comes connection variables defined in inventory
(ansible_ssh_user, etc)
○ then comes "most everything else" (command line switches, vars in
play, included vars, role vars, etc)
○ then comes the rest of the variables defined in inventory
○ then comes facts discovered about a system
○ then "role defaults", which are the most "defaulty" and lose in
priority to everything.

Use common variables when possible
● group_vars/all
● Standardize as much as you can

Examples of what we put in there
group_vars/all
ansible_ssh_private_key_file: ~/.ssh/{{ key_name | default(service_name) }}.pem
ansible_ssh_user: "{{ remote_user }}"
remote_user:"{{aws_config[my_ec2_region]['remote_user'] | default('ec2-user')}}"
my_ec2_region: "{{ lookup('env', 'EC2_REGION') }}"
default_service_dir: /usr/local/prevoty
java_version: 1.8.0_25
java_home: /opt/jre{{ java_version }}
go_version: go1.4.1

Separate specific vars by service
● group_vars/<service_name>
● These will be vars specific to this service
○ ELB
○ VPC Subnet(s)
○ Configuration

Combine secrets by environment
● group_vars/all_<service_env>
● We’ve found that placing all secrets
together to be easier to deal with
● Single simple import
● Decryption happens automatically

Build generic playbooks
● Playbooks can be built on top of variables
● Use the “extra vars” (-e) to specify a service
% ansible-playbook --ask-vault-pass -i <inventory> -e
“service_name=<service> service_env=<env>” deploy.yml

Import var files based on extra vars
● Use the vars passed by the cli to specify
imports
- hosts: my_group
vars_files:
- group_vars/all
- group_vars/all_{{ service_env }}
- group_vars/{{ service_name }}
roles:
- my_role

Specify host groups with vars
● You can reference a host group based on
variables
● Host group can sit inside of a vars file
tag_{{ host_group_key }}_{{ host_group_value }}

Putting it together
deploy.yml
- hosts: tag_{{ host_group_tag }}_{{ host_group_value }}
vars_files:
- group_vars/all
- group_vars/all_{{ service_env }}
- group_vars/{{ service_name }}
role:
- deploy
% ansible-playbook --ask-vault-pass -i <inventory> -e “service_env=<env>
service_name=<service>” deploy.yml

Wrap it up with an old friend
● We use bash to wrap playbooks together
● Easily run a full deploy
● Restart at intermediate steps if needed

Allow hash merging
● Makes it so much easier for cloud
deployments
● Allows you to have one data structure
across files in group_vars that become
more easily accessible at runtime
● Enabled in ansible.cfg

Hash merging example
group_vars/all
aws_config : {
“us-west-1” : {
“ami_id” : “ami-00112233”,
“rds_url” : <west_url>,
},
“us-east-1” : {
“ami_id” : “ami-44556677”,
}
}
group_vars/<service_name>
aws_config : {
“us-west-1” : {
“elb_name” : <west_elb>,
“vpc_subnet” : [<west_subnet>],
},
“us-east-1” : {
“elb_name” : <east_elb>,
“vpc_subnet” : [<east_subnet>],
}
}

Hash merging example cont’d
result
aws_config : {
“us-west-1” : {
“ami_id” : “ami-00112233”,
“elb_name” : <west_elb>,
“vpc_subnet” : [<west_subnet>],
},
“us-east-1” : {
“ami_id” : “ami-44556677”,
result cont’d
“elb_name” : <east_elb>,
“vpc_subnet” : [<east_subnet>],
}
}

Hash merging example cont’d
● Easy access in playbooks based on region
ec2_region: {{ lookup(‘ENV’, ‘EC2_REGION’) }}
- or -
-e “ec2_region=<region>”
● Accessed by:
{{ aws_config[ec2_region][‘elb_name’] }}

Make sure it fails…
● It shouldn’t just fail when there’s an error!
● Don’t run other plays in a playbook if a
prerequisite isn’t met
● Ex: No hosts found in a host group

Test changes from start to finish
● Don’t consider a fix complete until you’ve
run the entire deploy from start to finish
● Commands issued while debugging an issue
can fix that issue without persistence

Working in the cloud and beyond
Our focus is in the cloud but it doesn’t always
work for customers when it comes to their
view of security

A VM is a VM
● We can automatically build a VM with tools
such as Cobbler or packer on VMWare, KVM
or XenServer
● Automated builds of the base OS that is the
same as we run on AWS

Only the endpoints changed
● IP of the VM added to a static inventory
● Same playbooks and roles used for setup of
the OS and build/deploy of the service

An example inventory
/tmp/inventory
[tag_<host_group_name>_<service_a>_new]
<service_a> ansible_ssh_host=10.0.xxx.yyy
[tag_<host_group_name>_<service_b>_new]
<service_b> ansible_ssh_host=10.0.xxx.yyy
[tag_<host_group_name>_<service_c>_new]
<service_c> ansible_ssh_host=10.0.xxx.yyy

Using Ansible for Deploying to Cloud Environments

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Using Ansible for Deploying to Cloud Environments (20)

Recently uploaded (20)

Using Ansible for Deploying to Cloud Environments