SlideShare a Scribd company logo

Introduction to Chef

Suresh Paulraj
Suresh Paulraj

This document introduces infrastructure as code and automation using Chef. It discusses how Chef can be used to manage infrastructure complexity through policy-driven configuration management. Chef defines the desired state of systems using recipes, cookbooks and roles. Nodes running the Chef client converge themselves to the desired state defined in policies. Search is used to query nodes and topology data, which can then be passed to templates to dynamically generate configurations. This allows infrastructure to be automated, standardized and maintained as code.

Technology
1 of 69
Downloaded 12 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
Introduction to Infrastructure as Code & Automation / Introduction to Chef Ned Harris, Solution Architect Chef
http://www.flickr.com/photos/michaelheiss/3090102907/ Complexity
Managing Complexity •  SSH, make with the typey typey
Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt
Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt •  Move notes to the wiki
Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt •  Move notes to the wiki •  Custom scripts (setup.sh)
Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt •  Move notes to the wiki •  Custom scripts (setup.sh) •  Golden Images
Golden Images are not the answer • Gold is heavy • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/
Typical Infrastructure Memcache Postgres Slaves Postgres Master NagiosGraphite Jboss App
New Compliance Mandate! Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Move SSH off port 22 • Lets put it on 2022
6 Golden Image Updates Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6
12 Instance Replacements Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually
Done in Maintenance Windows • Don’t break anything! • Bob just got fired =( 5 Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite 1 2 4 5 6 7 8 9 10 11 12 3 12
Different IP Addresses? Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Invalid configs!
Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt •  Move notes to the wiki •  Custom scripts (setup.sh) •  Golden Images •  Policy-driven configuration management
Policies •  Declarations about the state of thing in a system •  applied repeatedly and repair the system when needed •  often change
Repeatable Operations •  Idempotent •  can be applied an infinite number of times and yield the same result every time •  Convergent •  test state and repair if needed
Policy Evolves
Following Policy •  A control loop keeps the system stable and allows for change when policy is updated
Policy Evolves That's great and all, ! but tell me about ! Chef!!
What is Chef? •  Framework for managing complexity •  Infrastructure as code •  a domain-specific language (DSL) for describing convergent operations •  A community of professionals •  A company
The Chef Software Platform Chef Development Kit Cookbook and Policy Authoring Test-Driven Infrastructure Chef Server Management Console Analytics Platform High Availability and Replication Chef Client Nodes Data Center The Cloud
How does Chef work? •  Ensure desired state by continually testing and repairing individual resources in the system •  Compose policies using a series of abstractions
Desired Configuration Node Chef Server What policy should I follow?
Desired Configuration Node Chef Server What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]"
Desired Configuration Chef Server What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]"
Recipes package "apache2"resource one
Recipes package "apache2" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end resource one resource two
Recipes package "apache2" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end service "apache2" do action [:enable,:start] supports :reload => true end resource one resource two resource three
Recipes resource one
Recipes resource one resource two
Recipes resource one resource two resource three
Built-in Resources • package • template • service • cron • directory • mount • user • group • registry_key • remote_directory • route • ...and many more!
Introduction to Chef
Managing Complexity •  Organizations •  Environments •  Roles •  Nodes •  Recipes •  Cookbooks •  Search •  Data
Their Infrastructure Organizations My Infrastructure Your Infrastructure
Environments Development Staging Production
Roles Load Balancers Application Servers DB Cache Database
Nodes
Search •  Search for nodes with Roles •  Find Topology Data •  IP addresses •  Hostnames •  FQDNs http://www.flickr.com/photos/kathycsus/2686740
Search for Nodes pool_members  =  search("node","role:webserver")     template  "/etc/haproxy/haproxy.cfg"  do      source  "haproxy-­‐app_lb.cfg.erb"      owner  "root"      group  "root"      mode  0644      variables  :pool_members  =>  pool_members.uniq      notifies  :restart,  "service[haproxy]"   end  
Webservers HAProxy Configuration HA Proxy
Webservers HAProxy Load Balancer HA ProxyEnterprise Chef pool_members = search("node","role:webserver")
Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver")
Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver")
Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver") { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver") { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver") { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
Search for Nodes pool_members  =  search("node","role:webserver")     template  "/etc/haproxy/haproxy.cfg"  do      source  "haproxy-­‐app_lb.cfg.erb"      owner  "root"      group  "root"      mode  0644      variables  :pool_members  =>  pool_members.uniq      notifies  :restart,  "service[haproxy]"   end  
Pass results into Templates #  Set  up  application  listeners  here.   listen  application  0.0.0.0:80      balance  roundrobin      <%  @pool_members.each  do  |member|  -­‐%>      server  <%=  member[:hostname]  %>  <%=  member[:ipaddress]  %>:>  weight  1  maxconn  1   check      <%  end  -­‐%>   <%  if  node["haproxy"]["enable_admin"]  -­‐%>   listen  admin  0.0.0.0:22002      mode  http      stats  uri  /   <%  end  -­‐%>  
Webservers HAProxy Configuration HA Proxypool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%>
Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check
Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check
Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check
Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check
Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check server web05 10.1.1.5 weight 1 maxconn 1 check
Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check server web05 10.1.1.5 weight 1 maxconn 1 check server web06 10.1.1.1 weight 1 maxconn 1 check pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
So when this… Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite
…becomes this… Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite
…this can happen automaticaly! Memcache Postgres Slaves Postgres Master NagiosGraphite Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite
Count the resources NagiosGraphite NagiosGraphite Memcache Postgres Slaves •  Load balancer config •  Nagios host ping •  Nagios host ssh •  Nagios host HTTP •  Nagios host app health •  Graphite CPU •  Graphite Memory •  Graphite Disk •  Graphite SNMP •  Memcache firewall •  Postgres firewall •  Postgres authZ config • 12+ resource changes for 1 node addition Jboss App
Build Anything •  Simple internal applications •  Complex external applications •  Workstations •  Hadoop clusters •  IaaS infrastructure •  PaaS infrastructure •  SaaS applications •  Storage systems •  You name it http://www.flickr.com/photos/hyku/245010680/
And Manage it Simply •  Automatically reconfigure everything •  Linux, Windows, Unixes, BSDs •  Load balancers •  Metrics collection systems •  Monitoring systems •  Cloud migrations become trivial http://www.flickr.com/photos/helico/404640681/
ChefDK
Increase CHEF adoption through ChefDK ●  ChefDK: CHEF Software Development Kit, fully supported with the Chef Premium Subscription ●  Workflow Definition: Our recommendation on the process to test and verify your infrastructure code before committing it to source control and shipping it to production. ●  Workflow Enhancement: Based on customer feedback and use cases
ChefDK: In the Box First Class Support on Windows, Linux, and OSX for the entire suite of Chef development tools ●  Test Kitchen: Virtualized testing harness ●  Berkshelf: Dependency solver ●  Chef-Vault: Secrets management ●  Rubocop / Foodcritic: Code linting ●  Chefspec: In-memory Unit Testing ●  Chef.bin: New wrapper binary to tie it all together, with new extensible cookbook generators.
The Chef workflow Create new skeleton cookbook. Create a VM environment for cookbook development. Write/debug cookbook recipes (iterative step). Perform acceptance tests. Deploy to production.
Questions?
Introduction to Chef

More Related Content

What's hot (19)

PDF
Altitude SF 2017: Debugging Fastly VCL 101
Fastly
 
PDF
Tips for going fast in a slow world: Michael May at OSCON 2015
Fastly
 
PDF
Rails Caching Secrets from the Edge
Michael May
 
PPTX
Chef Actions: Delightful near real-time activity tracking!
James Casey
 
PDF
Building a better web
Fastly
 
PDF
Chef at WebMD
adamleff
 
PPTX
WordPress + NGINX Best Practices with EasyEngine
NGINX, Inc.
 
PDF
Rebooting a Cloud
Jesse Robbins
 
PPTX
NLIT 2011: Chef & Capistrano
nickblah
 
PPTX
Meetup bangalore aug31st2019
D.Rajesh Kumar
 
PPTX
Meetup bangalore 9_novupdated
D.Rajesh Kumar
 
PDF
Improve Magento Performance
Harald Zeitlhofer
 
PDF
Unlocked Nov 2013: Main Slide Pack
Rackspace Academy
 
PDF
I can't believe it's not a queue: Kafka and Spring
Joe Kutner
 
PPTX
Automated release management with team city & octopusdeploy - NDC 2013
Kristoffer Deinoff
 
PDF
Choosing a Javascript Framework
All Things Open
 
PPTX
NGINX 101 - now with more Docker
Sarah Novotny
 
PDF
Apache Camel: Jetty Component With Example
Amit Aggarwal
 
PPTX
So I Wrote a Manifest
Puppet
 
Altitude SF 2017: Debugging Fastly VCL 101
Fastly
 
Tips for going fast in a slow world: Michael May at OSCON 2015
Fastly
 
Rails Caching Secrets from the Edge
Michael May
 
Chef Actions: Delightful near real-time activity tracking!
James Casey
 
Building a better web
Fastly
 
Chef at WebMD
adamleff
 
WordPress + NGINX Best Practices with EasyEngine
NGINX, Inc.
 
Rebooting a Cloud
Jesse Robbins
 
NLIT 2011: Chef & Capistrano
nickblah
 
Meetup bangalore aug31st2019
D.Rajesh Kumar
 
Meetup bangalore 9_novupdated
D.Rajesh Kumar
 
Improve Magento Performance
Harald Zeitlhofer
 
Unlocked Nov 2013: Main Slide Pack
Rackspace Academy
 
I can't believe it's not a queue: Kafka and Spring
Joe Kutner
 
Automated release management with team city & octopusdeploy - NDC 2013
Kristoffer Deinoff
 
Choosing a Javascript Framework
All Things Open
 
NGINX 101 - now with more Docker
Sarah Novotny
 
Apache Camel: Jetty Component With Example
Amit Aggarwal
 
So I Wrote a Manifest
Puppet
 

Similar to Introduction to Chef (20)

PDF
Introduction to Infrastructure as Code & Automation / Introduction to Chef
Nathen Harvey
 
PDF
Introduction to Infrastructure as Code & Automation / Introduction to Chef
All Things Open
 
PDF
Overview of Chef - Fundamentals Webinar Series Part 1
Chef
 
PDF
Introduction to Chef
kevsmith
 
PDF
Software as a Service workshop / Unlocked: the Hybrid Cloud 12th May 2014
Rackspace Academy
 
PDF
Achieving Infrastructure Portability with Chef
Matt Ray
 
PDF
CouchDB for Web Applications - Erlang Factory London 2009
Jason Davies
 
ODP
Configuration management with Chef
Juan Vicente Herrera Ruiz de Alejo
 
PDF
TXLF: Chef- Software Defined Infrastructure Today & Tomorrow
Matt Ray
 
ODP
DB proxy server test: run tests on tens of virtual machines with Jenkins, Vag...
Timofey Turenko
 
PDF
Building a Dev/Test Cloud with Apache CloudStack
ke4qqq
 
PDF
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Emerson Eduardo Rodrigues Von Staffen
 
ODP
Puppet and Apache CloudStack
Puppet
 
PDF
Introduction to Chef - April 22 2015
Jennifer Davis
 
PPTX
Orchestrating Docker with Terraform and Consul by Mitchell Hashimoto
Docker, Inc.
 
PDF
Managing Infrastructure as Code
Allan Shone
 
PDF
EC2 AMI Factory with Chef, Berkshelf, and Packer
George Miranda
 
PDF
Chef for OpenStack - OpenStack Fall 2012 Summit
Matt Ray
 
PDF
Chef for OpenStack- Fall 2012.pdf
OpenStack Foundation
 
PPTX
Nginx A High Performance Load Balancer, Web Server & Reverse Proxy
Amit Aggarwal
 
Introduction to Infrastructure as Code & Automation / Introduction to Chef
Nathen Harvey
 
Introduction to Infrastructure as Code & Automation / Introduction to Chef
All Things Open
 
Overview of Chef - Fundamentals Webinar Series Part 1
Chef
 
Introduction to Chef
kevsmith
 
Software as a Service workshop / Unlocked: the Hybrid Cloud 12th May 2014
Rackspace Academy
 
Achieving Infrastructure Portability with Chef
Matt Ray
 
CouchDB for Web Applications - Erlang Factory London 2009
Jason Davies
 
Configuration management with Chef
Juan Vicente Herrera Ruiz de Alejo
 
TXLF: Chef- Software Defined Infrastructure Today & Tomorrow
Matt Ray
 
DB proxy server test: run tests on tens of virtual machines with Jenkins, Vag...
Timofey Turenko
 
Building a Dev/Test Cloud with Apache CloudStack
ke4qqq
 
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Emerson Eduardo Rodrigues Von Staffen
 
Puppet and Apache CloudStack
Puppet
 
Introduction to Chef - April 22 2015
Jennifer Davis
 
Orchestrating Docker with Terraform and Consul by Mitchell Hashimoto
Docker, Inc.
 
Managing Infrastructure as Code
Allan Shone
 
EC2 AMI Factory with Chef, Berkshelf, and Packer
George Miranda
 
Chef for OpenStack - OpenStack Fall 2012 Summit
Matt Ray
 
Chef for OpenStack- Fall 2012.pdf
OpenStack Foundation
 
Nginx A High Performance Load Balancer, Web Server & Reverse Proxy
Amit Aggarwal
 
Ad

More from Suresh Paulraj (6)

PDF
Accelerating DevOps Pipelines with AWS
Suresh Paulraj
 
PDF
Introduction to Stacki - World's fastest Linux server provisioning Tool
Suresh Paulraj
 
PDF
2015 08-11-scdo-meetup
Suresh Paulraj
 
PDF
Getting started with salt stack
Suresh Paulraj
 
PDF
Getting started with salt stack
Suresh Paulraj
 
PPTX
SoCalDevOpsUserGroup-PresentationPuppetLabs
Suresh Paulraj
 
Accelerating DevOps Pipelines with AWS
Suresh Paulraj
 
Introduction to Stacki - World's fastest Linux server provisioning Tool
Suresh Paulraj
 
2015 08-11-scdo-meetup
Suresh Paulraj
 
Getting started with salt stack
Suresh Paulraj
 
Getting started with salt stack
Suresh Paulraj
 
SoCalDevOpsUserGroup-PresentationPuppetLabs
Suresh Paulraj
 
Ad

Recently uploaded (20)

PPTX
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
PDF
Smart Trailers 2025 Update with History and Overview
Paul Menig
 
PDF
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
PPT
Interview paper part 3, It is based on Interview Prep
SoumyadeepGhosh39
 
PDF
NewMind AI - Journal 100 Insights After The 100th Issue
NewMind AI
 
PDF
July Patch Tuesday
Ivanti
 
PDF
DevBcn - Building 10x Organizations Using Modern Productivity Metrics
Justin Reock
 
PPTX
✨Unleashing Collaboration: Salesforce Channels & Community Power in Patna!✨
SanjeetMishra29
 
PDF
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PDF
Timothy Rottach - Ramp up on AI Use Cases, from Vector Search to AI Agents wi...
AWS Chicago
 
PDF
Python basic programing language for automation
DanialHabibi2
 
PDF
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
PDF
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
PDF
HubSpot Main Hub: A Unified Growth Platform
Jaswinder Singh
 
PPTX
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
PDF
LLMs.txt: Easily Control How AI Crawls Your Site
Keploy
 
PDF
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
PDF
Building Real-Time Digital Twins with IBM Maximo & ArcGIS Indoors
Safe Software
 
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
Smart Trailers 2025 Update with History and Overview
Paul Menig
 
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
Interview paper part 3, It is based on Interview Prep
SoumyadeepGhosh39
 
NewMind AI - Journal 100 Insights After The 100th Issue
NewMind AI
 
July Patch Tuesday
Ivanti
 
DevBcn - Building 10x Organizations Using Modern Productivity Metrics
Justin Reock
 
✨Unleashing Collaboration: Salesforce Channels & Community Power in Patna!✨
SanjeetMishra29
 
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
Timothy Rottach - Ramp up on AI Use Cases, from Vector Search to AI Agents wi...
AWS Chicago
 
Python basic programing language for automation
DanialHabibi2
 
Fl Studio 24.2.2 Build 4597 Crack for Windows Free Download 2025
faizk77g
 
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
HubSpot Main Hub: A Unified Growth Platform
Jaswinder Singh
 
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
LLMs.txt: Easily Control How AI Crawls Your Site
Keploy
 
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
Building Real-Time Digital Twins with IBM Maximo & ArcGIS Indoors
Safe Software
 

Introduction to Chef

  • 1. Introduction to Infrastructure as Code & Automation / Introduction to Chef Ned Harris, Solution Architect Chef
  • 3. Managing Complexity •  SSH, make with the typey typey
  • 4. Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt
  • 5. Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt •  Move notes to the wiki
  • 6. Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt •  Move notes to the wiki •  Custom scripts (setup.sh)
  • 7. Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt •  Move notes to the wiki •  Custom scripts (setup.sh) •  Golden Images
  • 8. Golden Images are not the answer • Gold is heavy • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/
  • 10. New Compliance Mandate! Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Move SSH off port 22 • Lets put it on 2022
  • 11. 6 Golden Image Updates Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6
  • 12. 12 Instance Replacements Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually
  • 13. Done in Maintenance Windows • Don’t break anything! • Bob just got fired =( 5 Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite 1 2 4 5 6 7 8 9 10 11 12 3 12
  • 14. Different IP Addresses? Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite • Invalid configs!
  • 15. Managing Complexity •  SSH, make with the typey typey •  Keep notes in ~/server.txt •  Move notes to the wiki •  Custom scripts (setup.sh) •  Golden Images •  Policy-driven configuration management
  • 16. Policies •  Declarations about the state of thing in a system •  applied repeatedly and repair the system when needed •  often change
  • 17. Repeatable Operations •  Idempotent •  can be applied an infinite number of times and yield the same result every time •  Convergent •  test state and repair if needed
  • 19. Following Policy •  A control loop keeps the system stable and allows for change when policy is updated
  • 20. Policy Evolves That's great and all, ! but tell me about ! Chef!!
  • 21. What is Chef? •  Framework for managing complexity •  Infrastructure as code •  a domain-specific language (DSL) for describing convergent operations •  A community of professionals •  A company
  • 22. The Chef Software Platform Chef Development Kit Cookbook and Policy Authoring Test-Driven Infrastructure Chef Server Management Console Analytics Platform High Availability and Replication Chef Client Nodes Data Center The Cloud
  • 23. How does Chef work? •  Ensure desired state by continually testing and repairing individual resources in the system •  Compose policies using a series of abstractions
  • 25. Desired Configuration Node Chef Server What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]"
  • 26. Desired Configuration Chef Server What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]"
  • 28. Recipes package "apache2" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end resource one resource two
  • 29. Recipes package "apache2" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end service "apache2" do action [:enable,:start] supports :reload => true end resource one resource two resource three
  • 35. Managing Complexity •  Organizations •  Environments •  Roles •  Nodes •  Recipes •  Cookbooks •  Search •  Data
  • 39. Nodes
  • 40. Search •  Search for nodes with Roles •  Find Topology Data •  IP addresses •  Hostnames •  FQDNs http://www.flickr.com/photos/kathycsus/2686740
  • 41. Search for Nodes pool_members  =  search("node","role:webserver")     template  "/etc/haproxy/haproxy.cfg"  do      source  "haproxy-­‐app_lb.cfg.erb"      owner  "root"      group  "root"      mode  0644      variables  :pool_members  =>  pool_members.uniq      notifies  :restart,  "service[haproxy]"   end  
  • 43. Webservers HAProxy Load Balancer HA ProxyEnterprise Chef pool_members = search("node","role:webserver")
  • 44. Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver")
  • 45. Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver")
  • 46. Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver") { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
  • 47. Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver") { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
  • 48. Webservers HAProxy Load Balancer HA ProxyEnterprise Chef Webservers? pool_members = search("node","role:webserver") { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
  • 49. Search for Nodes pool_members  =  search("node","role:webserver")     template  "/etc/haproxy/haproxy.cfg"  do      source  "haproxy-­‐app_lb.cfg.erb"      owner  "root"      group  "root"      mode  0644      variables  :pool_members  =>  pool_members.uniq      notifies  :restart,  "service[haproxy]"   end  
  • 50. Pass results into Templates #  Set  up  application  listeners  here.   listen  application  0.0.0.0:80      balance  roundrobin      <%  @pool_members.each  do  |member|  -­‐%>      server  <%=  member[:hostname]  %>  <%=  member[:ipaddress]  %>:>  weight  1  maxconn  1   check      <%  end  -­‐%>   <%  if  node["haproxy"]["enable_admin"]  -­‐%>   listen  admin  0.0.0.0:22002      mode  http      stats  uri  /   <%  end  -­‐%>  
  • 51. Webservers HAProxy Configuration HA Proxypool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%>
  • 52. Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check
  • 53. Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check
  • 54. Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check
  • 55. Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check
  • 56. Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } } haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check server web05 10.1.1.5 weight 1 maxconn 1 check
  • 57. Webservers HAProxy Configuration HA Proxy <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check server web05 10.1.1.5 weight 1 maxconn 1 check server web06 10.1.1.1 weight 1 maxconn 1 check pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }, "web05" : { "hostname" : "web05", "ipaddress" : "10.1.1.5" }, "web06" : { "hostname" : "web06", "ipaddress" : "10.1.1.6" } }
  • 58. So when this… Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite
  • 59. …becomes this… Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite
  • 60. …this can happen automaticaly! Memcache Postgres Slaves Postgres Master NagiosGraphite Jboss App Memcache Postgres Slaves Postgres Master NagiosGraphite
  • 61. Count the resources NagiosGraphite NagiosGraphite Memcache Postgres Slaves •  Load balancer config •  Nagios host ping •  Nagios host ssh •  Nagios host HTTP •  Nagios host app health •  Graphite CPU •  Graphite Memory •  Graphite Disk •  Graphite SNMP •  Memcache firewall •  Postgres firewall •  Postgres authZ config • 12+ resource changes for 1 node addition Jboss App
  • 62. Build Anything •  Simple internal applications •  Complex external applications •  Workstations •  Hadoop clusters •  IaaS infrastructure •  PaaS infrastructure •  SaaS applications •  Storage systems •  You name it http://www.flickr.com/photos/hyku/245010680/
  • 63. And Manage it Simply •  Automatically reconfigure everything •  Linux, Windows, Unixes, BSDs •  Load balancers •  Metrics collection systems •  Monitoring systems •  Cloud migrations become trivial http://www.flickr.com/photos/helico/404640681/
  • 65. Increase CHEF adoption through ChefDK ●  ChefDK: CHEF Software Development Kit, fully supported with the Chef Premium Subscription ●  Workflow Definition: Our recommendation on the process to test and verify your infrastructure code before committing it to source control and shipping it to production. ●  Workflow Enhancement: Based on customer feedback and use cases
  • 66. ChefDK: In the Box First Class Support on Windows, Linux, and OSX for the entire suite of Chef development tools ●  Test Kitchen: Virtualized testing harness ●  Berkshelf: Dependency solver ●  Chef-Vault: Secrets management ●  Rubocop / Foodcritic: Code linting ●  Chefspec: In-memory Unit Testing ●  Chef.bin: New wrapper binary to tie it all together, with new extensible cookbook generators.
  • 67. The Chef workflow Create new skeleton cookbook. Create a VM environment for cookbook development. Write/debug cookbook recipes (iterative step). Perform acceptance tests. Deploy to production.