Managing Infrastructure as Code
0 likes698 views
The document discusses the evolution of infrastructure management from manual processes to modern Infrastructure as Code (IaC) practices, highlighting various tools such as Ansible, Chef, Puppet, Terraform, and CloudFormation. It emphasizes the benefits of automation, orchestration, and simple syntax for ease of use while also addressing potential drawbacks of each tool. The content also touches on the role of DevOps in facilitating these processes and the importance of effective workflows in leveraging technology for improved infrastructure management.
![Resources
load_balancer "test-elb" do
machines [ "machine1", "machine2" ]
load_balancer_options :listeners => [{
:port => 80,
:protocol => :http,
:instance_port => 80,
:instance_protocol => :http,
}]
end](https://image.slidesharecdn.com/linux-160204020944/85/Managing-Infrastructure-as-Code-16-320.jpg)
![ec2_securitygroup { 'sample-group':
ensure => present,
region => 'us-west-1',
description => 'Group used for testing Puppet AWS module',
}
ec2_instance { 'sample-instance':
ensure => present,
region => 'us-west-1',
availability_zone => 'us-west-1a',
image_id => 'ami-696e652c',
instance_type => 't1.micro',
security_groups => ['sample-group'],
}
Resources](https://image.slidesharecdn.com/linux-160204020944/85/Managing-Infrastructure-as-Code-20-320.jpg)
![Resource - Finalising
ec2_loadbalancer { 'sample-load-balancer':
ensure => present,
region => 'us-west-1',
availability_zones => ['us-west-1a', 'us-west-1b'],
instances => ['sample-instance', 'another-instance'],
security_groups => ['sample-group'],
listeners => [{
protocol => 'tcp',
port => 80,
}],
}](https://image.slidesharecdn.com/linux-160204020944/85/Managing-Infrastructure-as-Code-21-320.jpg)
![Resources - Security Group
"Resources": {
"WebServerSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Minimal Access",
"SecurityGroupIngress": [{
"IpProtocol": "tcp",
"FromPort": "80",
"ToPort": "80",
"CidrIp": "0.0.0.0/0"
}]
}
}
}](https://image.slidesharecdn.com/linux-160204020944/85/Managing-Infrastructure-as-Code-26-320.jpg)
![Resources - Instance
"Resources": {
"WebServer": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"AWS::CloudFormation::Init": {
"install": {
"packages": {
"yum": {
"httpd": []
}
}
}
}
}](https://image.slidesharecdn.com/linux-160204020944/85/Managing-Infrastructure-as-Code-27-320.jpg)
![Tying it together
"Outputs": {
"WebsiteURL": {
"Value": {
"Fn::Join": ["", [
"http://",
{
"Fn::GetAtt": [
"WebServer",
"PublicDnsName"
]},
"/ping"
]]},
"Description": "Website"
}
}](https://image.slidesharecdn.com/linux-160204020944/85/Managing-Infrastructure-as-Code-28-320.jpg)
![Software
resource "aws_instance" "web" {
connection { user = "ubuntu" }
instance_type = "m1.small"
ami = "${lookup(var.aws_amis, var.aws_region)}"
key_name = "${aws_key_pair.auth.id}"
vpc_security_group_ids = ["${aws_security_group.default.id}"]
subnet_id = "${aws_subnet.default.id}"
provisioner "remote-exec" {
inline = [
"sudo apt-get -y update",
"sudo apt-get -y install nginx",
"sudo service nginx start"
]
}
}](https://image.slidesharecdn.com/linux-160204020944/85/Managing-Infrastructure-as-Code-33-320.jpg)
![Resources
resource "aws_elb" "web" {
name = "terraform-example-elb"
subnets = ["${aws_subnet.default.id}"]
security_groups = ["${aws_security_group.elb.id}"]
instances = ["${aws_instance.web.id}"]
listener {
instance_port = 80
instance_protocol = "http"
lb_port = 80
lb_protocol = "http"
}}
resource "aws_key_pair" "auth" {
key_name = "${var.key_name}"
public_key = "${file(var.public_key_path)}"
}](https://image.slidesharecdn.com/linux-160204020944/85/Managing-Infrastructure-as-Code-34-320.jpg)
Managing Infrastructure as Code
- 1. Managing Infrastructure as Code Allan Shone
- 2. Agenda • Look at some legacy concepts • Run through some ideas • Check out a few products • Put together some base requirements • Browse a few modern products • Throw in some more updated concepts • Dive into suitable products • Extra bits, concerns, and ideas
- 3. Back in the day... • Infrastructure was extremely manual • Hosts, meta details, and further information was recorded in text files, or other plain text based systems • Hostnames based on function • Documentation was fragmented • Legacy infrastructure becomes unmanageable and can be forgotten • Nightmare to keep track of
- 4. Tools • Wiki / HTML Tables • Text Files • Shared drives • Internal admin server • Shared FTP • Proprietary and specific software solutions • Shell scripts
- 5. Problems? • Keeping host lists up-to-date • Multiple users managing infrastructure • Recent status indicators • Cumbersome interfaces • Time consuming data interactions • What about software?
- 6. Ideas • Some sort of versioning • Easier interface for collaboration • Provision of host state • Start looking at automation
- 7. Bits and Pieces • Databases • Services • Applications • Cache • Routers
- 8. Software? • First, infrastructure requires orchestration • Then, software dependencies can be pushed within each of those infrastructure items • Bare-metal is different with a separate set of requirements • The premise for both though is still of value to the general topic
- 10. Ansible • Provides inheritance • Allows for variable configuration • Straight-forward to use with automation • Expressive with its syntax using YAML • Playbooks used for grouping of instructions • Playbooks versioned in a DIY fashion • Agentless model for deployment • Templating makes conf files a breeze
- 11. Simple Software # Playbook for Application - hosts: “{{hosts}}” remote_user: root sudo: yes roles: - common - app-server # Directory Structure roles/common/handlers/main.yml roles/common/tasks/main.yml roles/common/templates/ntpd.conf roles/app-server/handlers/main.yml roles/app-server/tasks/main.yml roles/app-server/templates/apache.conf roles/app-server/vars/example.yml
- 12. Provisioning Infrastructure - hosts: localhost connection: local gather_facts: false tasks: - name: Provision instances ec2: key_name: my_key group: test instance_type: t2.micro image: “{{ami_id}}” count_tag: Name: Demo instance_tags: Name: Demo register: ec2 - name: Add Hosts to Host Group add_host: hostname={{ item.public_ip }} groups=ec2hosts with_items: ec2.instances - hosts: ec2hosts name: configuration play user: ec2-user gather_facts: true tasks: - name: Check NTP Service service: name=ntpd state=started
- 13. Drawbacks • Difficult to track created instances • Supplier specific wrapper • Versioning is DIY • Basic in terms of complete solution
- 14. Chef • Builds on Ruby for syntax • Fluent way of pushing modifications • Variable capabilities for ease of automation • Cookbooks used to group instructions • Cookbooks synchronised with the Chef Server • Server to client model
- 15. Simple Instances num_instances = 10 1.upto(num_instances) do |inst| machine "my-machine-#{inst}" do add_machine_options bootstrap_options: { security_group_ids: 'test-sg', subnet_id: 'subnet-1234567', instance_type: 't2.micro' } end end
- 16. Resources load_balancer "test-elb" do machines [ "machine1", "machine2" ] load_balancer_options :listeners => [{ :port => 80, :protocol => :http, :instance_port => 80, :instance_protocol => :http, }] end
- 17. Drawbacks • Dedicated server for management • Uses Ruby natively, which could be a positive if you work with Ruby or don’t mind • What is required for some may not exist unless the necessary Plugin is available for it • OS and Package restrictions for nodes
- 18. Puppet • Simple syntax for configuration • Server model for deployments • Automation readily available • Parameterised configurations for easy environment setup
- 19. Software package { 'apache2': provider=>'apt', ensure=>'installed' } notify { 'Apache2 is installed.': } service { 'apache2': ensure=>'running' } notify { 'Apache2 is running.': }
- 20. ec2_securitygroup { 'sample-group': ensure => present, region => 'us-west-1', description => 'Group used for testing Puppet AWS module', } ec2_instance { 'sample-instance': ensure => present, region => 'us-west-1', availability_zone => 'us-west-1a', image_id => 'ami-696e652c', instance_type => 't1.micro', security_groups => ['sample-group'], } Resources
- 21. Resource - Finalising ec2_loadbalancer { 'sample-load-balancer': ensure => present, region => 'us-west-1', availability_zones => ['us-west-1a', 'us-west-1b'], instances => ['sample-instance', 'another-instance'], security_groups => ['sample-group'], listeners => [{ protocol => 'tcp', port => 80, }], }
- 22. Drawbacks • Requires learning the Puppet specific language for the actual infrastructure code • Complex infrastructure can become quite cumbersome to manage • Dependency based, order of execution can be tricky to control when it is required to be
- 24. CloudFormation • Complete “physical” infrastructure as code • Basic JSON file for definition • Services for usage easily interacted with • Tightly coupled with AWS • Versioned and stored within the console • Ease of automation
- 25. Beginning { "AWSTemplateFormatVersion": "2016-01-01", "Description": "My Template", "Parameters": { "KeyName": { "Description": "EC2 KeyPair", "Type": "AWS::EC2::KeyPair::KeyName", "ConstraintDescription": "must be the name of an existing EC2 KeyPair." } } }
- 26. Resources - Security Group "Resources": { "WebServerSecurityGroup": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": "Minimal Access", "SecurityGroupIngress": [{ "IpProtocol": "tcp", "FromPort": "80", "ToPort": "80", "CidrIp": "0.0.0.0/0" }] } } }
- 27. Resources - Instance "Resources": { "WebServer": { "Type": "AWS::EC2::Instance", "Metadata": { "AWS::CloudFormation::Init": { "install": { "packages": { "yum": { "httpd": [] } } } } }
- 28. Tying it together "Outputs": { "WebsiteURL": { "Value": { "Fn::Join": ["", [ "http://", { "Fn::GetAtt": [ "WebServer", "PublicDnsName" ]}, "/ping" ]]}, "Description": "Website" } }
- 29. Drawbacks • AWS specific • JSON for the configuration can be difficult to create and maintain - No comments • Not idempotent • Templates are very large and can become quite cumbersome to follow • Most functionality can be automated through the command line interface within other tools
- 30. Infrastructure pieces • Software management, host management, resources • A general tool provides one but not the other • Arbitrary scripts can shoehorn this • Duplication and Inconsistencies would become problematic with keeping data sets in different tools
- 31. Combinations • Software dependencies managed • Hosts instantiated or made available on demand • Configurations completed between environments to allow for sand-boxed communication • Entire infrastructures brought up with a single command as replica of production
- 32. TerraForm • Will orchestrate and provision • Syntax is easy to grasp and maintain • Configurations can be quite simple • Parameterised capabilities for ease of scripting with environments
- 33. Software resource "aws_instance" "web" { connection { user = "ubuntu" } instance_type = "m1.small" ami = "${lookup(var.aws_amis, var.aws_region)}" key_name = "${aws_key_pair.auth.id}" vpc_security_group_ids = ["${aws_security_group.default.id}"] subnet_id = "${aws_subnet.default.id}" provisioner "remote-exec" { inline = [ "sudo apt-get -y update", "sudo apt-get -y install nginx", "sudo service nginx start" ] } }
- 34. Resources resource "aws_elb" "web" { name = "terraform-example-elb" subnets = ["${aws_subnet.default.id}"] security_groups = ["${aws_security_group.elb.id}"] instances = ["${aws_instance.web.id}"] listener { instance_port = 80 instance_protocol = "http" lb_port = 80 lb_protocol = "http" }} resource "aws_key_pair" "auth" { key_name = "${var.key_name}" public_key = "${file(var.public_key_path)}" }
- 35. Drawbacks • Tightly integrated with vendors • Learning curve for syntax • Delays with updated services and functionality • Newcomer to the fully managed tool suite, some features are incomplete or in progress
- 36. ManageaCloud • Complete solution, orchestration and provisioning • Simple, re-usable configuration • Built-in versioning for deployments and infrastructure • Open choice of vendor - no requirements • Framework approach for infrastructure management
- 37. Macfile • Configuration template • Complete infrastructure specification • Versioned to allow for ease of use, deployment, and rollback • Simple syntax no vendor specifics
- 38. App Instance roles: demo_app: instance create: configuration: demo_application infrastructures: demo_application_instance: name: demo provider: amazon location: us-east-1 hardware: t1.micro role: demo_app environment: - APP_BRANCH: master
- 39. Resource resources: elastic_load_balancer: create bash: aws elb create-load-balancer --load-balancer-name infrastructure.param.name --listeners infrastructure.param.listeners --availability-zones infrastructure.param.availability-zones --region infrastructure.param.region destroy bash: aws elb delete-load-balancer --load-balancer-name infrastructure.param.name --region infrastructure.param.region
- 40. Resource Instance infrastructures: load balancer 01: resource: elastic_load_balancer params: name: my-demo-load-balancer listeners: Protocol=HTTP,LoadBalancerPort=80,InstanceProtocol=HTTP,InstancePort=80 availability-zones: us-east-1b us-east-1c region: us-east-1
- 41. Associationactions: get_id: ssh: wget -q -O - http://169.254.169.254/latest/meta-data/instance-id register_lb: create bash: aws elb register-instances-with-load-balancer --load-balancer-name infrastructure.param.load-balancer-name --instances infrastructure.param.instances --region infrastructure.param.region infrastructures: register_instance: ready: role.demo_app resource: register_lb params: load-balancer-name: my-demo-load-balancer instances: role.demo_app.get_id
- 42. Drawbacks • Most components are open source, not all at the present time • No unified syntax for providers
- 44. DevOps • Largely, DevOps came about as a hybrid role to help manage and facilitate process change • Automation is a key aspect • Not Operations, but not Development either (but is still both) • Provide an interface between infrastructure and environments and deployments made
- 45. Concepts • Even with automation, humans are still needed • Sanity checking and improving tools • Removing bottlenecks • Increasing developer and wider business productivity • Know the management tools and the details of how the infrastructure functions
- 46. Workflows • Very important to focus on processes • Tools are wonderful, but processes need to be suitable for the tool of choice • Automation will bring down the Op aspects of DevOps • Cross functional efforts to bring the automation to the infrastructure • Size of infrastructure
- 48. Decisions • Situations make decisions difficult • Complete solutions are not always necessary • Preference and team knowledge makes a difference • A product is not specifically good just because others use it
- 49. Options • There’s always more options available than time to discuss - CFengine, Salt, Heat, OneOps • It’s all about automation, and removing bottlenecks in cumbersome processes
- 50. Future • Abilities to share, extend, and work better with infrastructures • Inheritance for roles, resources, and instances • Complete control with automation of infrastructure sets • Simple options for deployment strategies