Chef and OpenStack Workshop from ChefConf 2013

Chef and OpenStack
Matt Ray
April 24, 2013

Agenda
• OpenStack Overview
• Chef for OpenStack Overview
• Knife OpenStack
• Lunch 12:15
• Chef for OpenStack Grizzly Roadmap
• Related Technologies
• Code Walkthroughs

Introductions
• Matt Ray
• matt@opscode.com
• mattray IRC, GitHub
• @mattray
http://upload.wikimedia.org/wikipedia/commons/2/27/Hi_How_Are_You_Austin_2005.jpg
Presenter notes: Austin, San Antonio, Santa Clara, Boston, San Francisco, San Diego, Portland
Bexar and on
Mercado Libre, Dell, Rackspace, HP, DreamHost

"To produce the ubiquitous Open Source cloud computing
platform that will meet the needs of public and private
cloud providers regardless of size, by being simple to
implement and massively scalable."
Mission Statement

Why OpenStack?
‣ Control. Open source, no vendor lock in. Apache 2 license.
‣ Flexibility. Modular design integrates legacy and third party technologies.
‣ Emerging Industry Standard. More than 180 technology industry leaders backing it and
major public clouds built on OpenStack.
‣ Proven. Originally built for scale and redundancy at NASA and Rackspace. More than
200 large-scale deployments worldwide.
‣ Compatible and Connected. Enables portability.
Control.
Open
source
means
you’re
never
locked
to
a
proprietary
vendor.
You
always
have
visibility
and
the
ability
to
directly
inﬂuence
the
roadmap
through
the
open
design
process.
Flexibility.
Modular
design
can
integrate
with
legacy
systems
and
third-‐party
technologies,
so
you
don't
have
to
rip-‐and-‐replace
your
exisAng
infrastructure.

Emerging
Industry
Standard.

More
than
170
leading
technology
companies
across
the
globe
are
developing
and
building
tools
for
OpenStack,
including
AT&T,
Cisco,
Dell,
HP,
Intel,
IBM,
MicrosoP
and
Red
Hat,
and
new
OpenStack
clouds
are
coming
online
daily.
Proven
and
Scalable.
OpenStack
was
built
for
scale
and
redundancy.
You
can
run
the
same
soPware
that
today
powers
some
of
the
world's
largest
public
and
private
clouds.
Compa<ble
and
Connected.
CompaAbility
with
public
OpenStack
clouds
means
enterprises
are
prepared
for
the
future—making
it
easy
to
migrate
data
and
applicaAons
to
public
clouds
when
condiAons
are
right.

OpenStack Controls Compute, Storage & Networking

Compute: Nova
‣ Virtual Machines
‣ Provision and manage large pools of on-demand computing resources
(hypervisors & instances)
‣ KVM
‣ Xen
‣ LXC
‣ Hyper-V
‣ VMware
‣ Bare-metal
Compute (codenamed "Nova") provides virtual servers upon demand. Rackspace and HP provide commercial compute services built
on Nova and it is used internally at companies like Mercado Libre, AT&T and NASA (where it originated).

Block Storage: Cinder
‣ Virtual Block Storage Devices
‣ Volumes on commodity storage gear
‣ Drivers for more advanced systems like NetApp, Solidﬁre, Ceph and Nexenta
‣ Released in Folsom Fall 2012
Block Storage (codenamed "Cinder") provides persistent block storage to guest VMs. This project was born from code originally in
Nova (the nova-volume service described below). Please note that this is block storage (or volumes) not ﬁlesystems like NFS or CIFS
share.Cinder is new for the Folsom release.

Networking as a Service: Quantum
‣ Virtualized Networking
‣ Software Deﬁned Networking (SDN)
‣ Automation of hardware & software
‣ Deﬁne network connectivity & addressing used by devices from other services
‣ Drivers for Ryu, Floodlight, Nicira, Midokura, Cisco and many more
‣ Released in Folsom Fall 2012
Network (codenamed "Quantum") provides "network connectivity as a service" between interface devices managed by other
OpenStack services (most likely Nova). The service works by allowing users to create their own networks and then attach interfaces to
them. Quantum has a pluggable architecture to support many popular networking vendors and technologies. Quantum is new in the
Folsom release.

Image Registry: Glance
‣ Multi-format virtual disk image registry & catalog
‣ Delivery of images to Nova Compute
‣ Allows uploads of private and public images in a wide variety of formats
‣ Machine (kernel/ramdisk outside of image, a.k.a. AMI)
‣ qcow2 (Qemu/KVM)
‣ VMDK (VMWare)
‣ OVF (VMWare, others)
‣ And more
Image (codenamed "Glance") provides a catalog and repository for virtual disk images. These disk images are mostly commonly used
in OpenStack Compute. While this service is technically optional, any cloud of size will require it.

Identity: Keystone
‣ Uniﬁes all core projects with common authentication system
‣ Provides authorization for multiple log-in credentials
‣ Username/password
‣ Token-based
‣ AWS-style logins
‣ Integrate with existing systems
Identity (codenamed "Keystone") provides authentication and authorization for all the OpenStack services. It also provides a service
catalog of services within a particular OpenStack cloud.

Object Storage: Swift
‣ Redundant, resilient, horizontally scalable object storage
‣ Petabytes of reliable storage on standard gear
‣ Examples include virtual machine images, photo storage, email storage
and backup archiving
‣ Rackspace Cloud Files
Object Store (codenamed "Swift") allows you to store or retrieve ﬁles (but not mount directories like a ﬁleserver). Several companies
provide commercial storage services based on Swift. These include KT, Rackspace (from which Swift originated) and Internap. Swift is
also used internally at many large companies to store their data.

Web Dashboard: Horizon
‣ Self-service, role-based web interface for users and administrators
‣ Provision cloud-based resources through a self-service portal
‣ Create and manage projects and users, deﬁning resources available to
them
‣ Extensible design makes it easy to plug in and expose third party
products and services
‣ Django application that consumes APIs
Dashboard (codenamed "Horizon") provides a modular web-based user interface for all the OpenStack services. With this web
GUI, you can perform most operations on your cloud like launching an instance, assigning IP addresses and setting access
controls.

OpenStack Community
‣ OpenStack Summits (Spring & Fall)
‣ IRC (irc.freenode.net)
‣ #openstack, #openstack-meetings, #openstack-chef, many more
‣ Mailing Lists (lists.openstack.org)
‣ OpenStack.org
‣ Blog, Docs, Wiki
‣ Twitter @OpenStack

OpenStack Code
‣ GitHub
‣ https://github.com/openstack/
‣ Gerrit
‣ https://review.openstack.org
‣ Jenkins
‣ https://jenkins.openstack.org/
‣ DevStack.org

OpenStack Foundation
Led by Executive Director, Jonathan Bryce, the Foundation is hiring 10-12 employees who, under the strategic direction of the Board, will
help carry out the OpenStack mission. Speciﬁc responsibilities include coordinating the project's infrastructure, such as systems for testing
the software at scale, community building activities, and managing the OpenStack trademark, which was transferred from Rackspace
following the ﬁrst board meeting.

OpenStack Distributions
‣ Linux Distributions
‣ Debian
‣ Fedora
‣ Red Hat
‣ SUSE
‣ Ubuntu
‣ Commercial Offerings
‣ CloudScaling
‣ Mirantis
‣ Nebula
‣ Piston
‣ Rackspace
‣ ...many more

Grizzly Release
‣ 550 Developers
‣ 2500 conference attendees
‣ Red Hat, IBM, Rackspace
‣ Incubating:
‣ Commons: Oslo
‣ Metering: Ceilometer
‣ Orchestration: Heat
• Ceilometer is a metering project. The project offers metering information and the ability to code more ways to know what has happened on an OpenStack cloud.
While it provides metering, it is not a billing project. A full billing solution requires metering, rating, and billing. Metering lets you know what actions have taken
place, rating enables pricing and line items, and billing gathers the line items to create a bill to send to the consumer and collect payment. Ceilometer is available
as a preview.
• Heat provides a REST API to orchestrate multiple cloud applications implementing standards such as AWS CloudFormation.

Why Chef?
I'm here to tell you

Chef is Infrastructure as Code
http://www.flickr.com/photos/louisb/4555295187/
• Programmatically
provision and configure
• Treat like any other code
base
• Reconstruct business from
code repository, data
backup, and bare metal
resources.
When dealing with Chef, need to literally “think outside the box”, by shifting your thinking about configuration away from a
single system, to that of an Application Infrastructure. The concept of an Infrastructure is an abstract one with a specific
technical meaning. When we talk about Infrastructure, we mean..

Declarative Interface to Resources
• Define policy
• Say what, not how
• Pull not Push
http://www.ﬂickr.com/photos/bixentro/2591838509/
Chef gives you declarative interfaces into the Resources on those Nodes.
Being declarative means that you say what you want to do, instead of how to do it.
For example,

you declare that package foobar-1.2.3 should be installed, or that the directory /var/log/foobar should exist.
Chef pulls down policy from the chef-server, ensuring that a node down for maintenance will receive its policy update when it
comes back online.

Ruby!
extra_packages = case node['platform']
when "ubuntu","debian"
%w{
ruby1.8
ruby1.8-dev
rdoc1.8
ri1.8
libopenssl-ruby
}
end
extra_packages.each do |pkg|
package pkg do
action :install
end
end
Because we use a 3GL for the recipe config files, we can use features of ruby like case statements and iterative loops.
Sysadmins don’t need to be afraid of Ruby, they’ve been dealing with sub-standard programming languages like
configuration files for years.
They’re also not limited to what the language tells them they can do.

Recipes and Cookbooks
• Recipes are collections of
Resources
• Cookbooks contain
recipes, templates, files,
custom resources, etc
• Code re-use and
modularity
• Hundreds already on
Community.opscode.com
http://www.flickr.com/photos/shutterhacks/4474421855/

The Chef Community
• Apache License,Version 2.0
• 1300+ Individual contributors
• 200+ Corporate contributors
• Dell, DreamHost, HP, Rackspace,
VMware, SUSE and many more
• 900+ cookbooks
• http://community.opscode.com
Chef is hackable! Permissive Apache2 license, vibrant community of awesome folks.
Community is very important to us.
That's why we're here.

http://www.ﬂickr.com/photos/16339684@N00/2681435235/
Chef and OpenStack?
That's great and all,
but tell me about
OpenStack!
And a basic fact about Infrastructure -- it EVOLVES.

Deploying OpenStack
• Chef ties it all together automatically
• Scaling changes how we deploy
• Interchangeable components
• Configurations shared, supported &
documented
• Licensing makes it available to everyone
We've learned a lot of things.

Overview & Status of
Chef for OpenStack

Chef for OpenStack:Who
• Arista
• AT&T
• Baremetal Cloud
• Calxeda
• Dell
• DreamHost
• HP
• HubSpot
• IBM
• Intel
• Internap
• Mercado Libre
• Mirantis
• NTT
• Nebula
• Nicira
• Piston Cloud
• Rackspace
• SUSE
• TryStack.org
• Voxel
• ...and more
These companies are currently involved to some extent. Some are paying customers that we've done engagements with.

Chef for OpenStack:Why
• Community for the automated deployment
and management of OpenStack
• Reduce fragmentation and encourage
collaboration
• Deploying OpenStack is not "secret sauce"
• Project not a product
• Apache 2 license

Chef for OpenStack:What
• Chef Repository for Deploying OpenStack
• Documentation for Chef for OpenStack
• Cookbooks
• Keystone
• Glance
• Nova
• Horizon
• Swift
• Quantum
• Cinder
• knife-openstack

Chef for OpenStack:Where
• #openstack-chef on irc.freenode.net
• github.com/opscode/openstack-chef-repo
• github.com/opscode-cookbooks/
• keystone, glance, nova, horizon,
swift,quantum,cinder
• github.com/opscode/knife-openstack
• github.com/mattray/openstack-chef-docs
• groups.google.com/group/opscode-chef-
openstack
• @chefopenstack

• Chef repo for Essex/Grizzly
• Operating Systems (Ubuntu 12.04)
• Hypervisors (KVM, LXC)
• Databases (MySQL)
• Nova network FlatDHCP HA & VLAN
• Quantum Nicira plugin available
• Test Kitchen integration
Chef for OpenStack:When (Today)

Chef for OpenStack:When (Tomorrow)
• Grizzly sprint scheduled in 2
weeks
• Merging AT&T, DreamHost,
HubSpot and Rackspace code
• Documentation
(docs.opscode.com)

• Build packages from source
• Continuous integration
• Hypervisors (Hyper-V, bare metal)
• Databases (PostgreSQL)
• Cinder (Ceph)
• Quantum (Midokura)
• Operating Systems (RHEL, Debian, SUSE)
• Documentation (docs.opscode.com)
• HA Configurations
Chef for OpenStack:When (Roadmap)

Resources
Chef for OpenStack
Ecosystem
Mercado Libre
HP
Rackspace
Dell
Trystack
Voxel
OpenStack

AT&T
• github.com/att-cloud/
• primary Folsom merge source
• openstack-common
• cinder (lvm, netapp, rbd)
• lots of support cookbooks

Crowbar
• Hardware provisioning and application
management platform
• crowbar.github.com
• Dell, SUSE, others
• likely Swift cookbook source combined
with Rackspace's

DreamHost
• github.com/dreamhost
• ceph
• ceilometer
• quantum

• Nicira NVP cookbook
• Open vSwitch cookbook
• Development in progress by Opscode
• github.com/gmiranda23/nvp-cookbook
Nicira

Rackspace Private Cloud
• www.rackspace.com/cloud/private/
• github.com/rcbops/chef-cookbooks
• primary Essex merge source
• likely Red Hat source

• Cookbooks reusable outside of
OpenStack
• Test Kitchen
• knife-rackspace/hp
• Crowbar, pxe_dust & Razor
• Arista EOS cookbook
• Berkshelf & Librarian
• Spiceweasel & Sputnik
Chef for OpenStack "Halo Effect"

Why the Cloud?
Why OpenStack?
The solution to this perceived impediment to resources

• Instant infrastructure
• Unlimited capacity
• Autoscaling
• No commitment
• Immediate replacement
Why the Cloud?
Enforces good architecture
No long term commitment
Cloud beneﬁts – instant infrastructure
Immediate replacement, no sparing etc.
Unlimited storage, snapshots (1 TB volume limit)
Provisioning APIs, autoscaling, EU storage, geodist
Public, private, hybrid

• Real Open Source
• Anyone can play
• Choice of features
• Features achieving parity/
accelerating ahead
Why OpenStack?

Know our escape plan
for every infrastructure
provider
"Drink the cloud Kool-aid, but only drink our Kool-aid"
If there are problems that you have with your cloud provider...
Not just the cloud

Chef for Infrastructure Portability
• knife ec2
• knife rackspace
• knife hp
• knife google
• knife azure
• knife cloudstack
• knife openstack
• knife vcloud
• ... and many
others
From EC2 to Rackspace, HP or any other OpenStack provider

• Vagrant
• VMware
• CloudStack
• Eucalyptus
• OpenStack
• bare metal
• AWS
• Rackspace
• HP
• Google
• Azure
• many others
Desktop,Virtualization, Private & Public Clouds
More than 360 individual contributors, over 70 corporate contributors.

• Vagrant
• VMware
• CloudStack
• Eucalyptus
• OpenStack
• bare metal
Desktop,Virtualization, Private & Public Clouds
• AWS
• Rackspace
• HP
• Google
• Azure
• many others
More than 360 individual contributors, over 70 corporate contributors.

Chef for OpenStack TL;DL
• Project, not a product
• Lots of contributors with real
deployments in a vibrant
ecosystem
• Essex works, Grizzly soon
• Features driven by demand
• Documentation with examples
• Do real work with OpenStack
From EC2 to Rackspace, HP or any other OpenStack provider

knife openstack
Knife is our command line tool, literally a swiss army knife of cloud APIs
It talks to the Chef server to manage your infrastructure, but it also talks to APIs like the OpenStack one
So even if you're not managing your OpenStack layer, you have Chef to manage the components on top of it.

knife openstack
$ knife openstack
Available openstack subcommands: (for details, knife SUB-
COMMAND --help)
** OPENSTACK COMMANDS **
knife openstack flavor list (options)
knife openstack group list (options)
knife openstack image list (options)
knife openstack server create (options)
knife openstack server delete SERVER [SERVER] (options)
knife openstack server list (options)
This is a supported knife plugin for Chef, so we have ticket tracking and everything for it.
It has the basics, server creation, deletion and listing available images and servers

knife openstack flavor list
$ knife openstack flavor list
ID Name Virtual CPUs RAM Disk
1 m1.tiny 1 512 MB 0 GB
2 m1.small 1 2048 MB 10 GB
3 m1.medium 2 4096 MB 10 GB
4 m1.large 4 8192 MB 10 GB
5 m1.xlarge 8 16384 MB 10 GB
update your knife.rb with
### Note: If you are not proxying HTTPS to the OpenStack EC2 API port, the scheme
should be HTTP, and the PORT is 8773.
you can get these from "knife node show blahblah -a nova"

knife openstack group list
$ knife openstack group list
Name Protocol From To CIDR Description
default tcp 22 22 0.0.0.0/0 default
default icmp -1 -1 0.0.0.0/0 default
haproxy tcp 22002 22002 0.0.0.0/0 22022

knife openstack image list
$ knife openstack image list
ID Name
4a197431-503d-4b85-b61e-84af21ca8654 cirros-image
f8ebb842-c0c0-4be3-8c4c-f72f48edec50 precise-image

knife openstack server create -h
--bootstrap-version VERSION The version of Chef to install
-N, --node-name NAME The Chef node name for your new node
-s, --server-url URL Chef Server URL
-k, --key KEY API Client Key
--[no-]color Use colored output, defaults to enabled
-c, --config CONFIG The configuration file to use
--defaults Accept default values for all questions
--disable-editing Do not open EDITOR, just accept the data as is
-d, --distro DISTRO Bootstrap a distro using a template; default is 'chef-full'
-e, --editor EDITOR Set the editor to use for interactive commands
-E, --environment ENVIRONMENT Set the Chef environment
-f, --flavor FLAVOR_ID The flavor ID of server (m1.small, m1.medium, etc)
-a, --floating-ip [IP] Request to associate a floating IP address to the new OpenStack node. Assumes IPs
have been allocated to the project. Specific IP is optional.
-F, --format FORMAT Which format to use for output
--[no-]host-key-verify Verify host key, enabled by default
-i IDENTITY_FILE, The SSH identity file used for authentication
--identity-file
-I, --image IMAGE_ID The image ID for the server
-u, --user USER API Client Username
--openstack-api-endpoint ENDPOINT
Your OpenStack API endpoint
--insecure Ignore SSL certificate on the Auth URL
-K, --openstack-password SECRET Your OpenStack Password
-T, --openstack-tenant NAME Your OpenStack Tenant NAME
-A, --openstack-username KEY Your OpenStack Username
--prerelease Install the pre-release chef gems
--print-after Show the data after a destructive operation
--private-network Use the private IP for bootstrapping rather than the public IP
-r, --run-list RUN_LIST Comma separated list of roles/recipes to apply
-G, --groups X,Y,Z The security groups for this server
-S, --ssh-key KEY The OpenStack SSH keypair id
-P, --ssh-password PASSWORD The ssh password
-x, --ssh-user USERNAME The ssh username
--template-file TEMPLATE Full path to location of template to use
-V, --verbose More verbose output. Use twice for max verbosity
-v, --version Show chef version
-y, --yes Say yes to all prompts for confirmation
-h, --help Show this message

$ knife openstack server list
Instance ID Name Public IP Private IP Flavor Image Keypair State
08f2d9f7-eeb0-45e7-8562-63aed8f096cc os-45539345723309377 50.56.12.229 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active
43c6bbf5-b397-4986-8aec-392d955ce5b1 os-9924426691020416 50.56.12.232 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active
c1b9e3df-e566-4378-8a52-ed998b516608 os-553425714287088 50.56.12.230 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active
f3edc5da-ef99-4acb-a141-d957e09809e3 os-07459550287500682 50.56.12.231 2 737969f8-6091-4896-ba9c-f3cf63bd25c5 rs-demo active
knife openstack server list
How did we get to the point where we can build a multi-tiered, monitored infrastructure?

knife openstack server create -a -f 2 -I 737969f8-6091-4896-ba9c-f3cf63bd25c5 -S
rs-demo -i ~/.ssh/rs-demo.pem -x ubuntu -r "role[base]"
knife openstack server create

knife openstack server create
Instance Name: os-45539345723309377
Instance ID: 08f2d9f7-eeb0-45e7-8562-63aed8f096cc
Waiting for server.........
Flavor: 2
Image: 737969f8-6091-4896-ba9c-f3cf63bd25c5
SSH Identity File: /Users/mray/.ssh/rs-demo.pem
SSH Keypair: rs-demo
Public IP Address: 10.241.0.12
Floating IP Address: 50.56.12.229
Waiting for sshd.....done
Bootstrapping Chef on 50.56.12.229
Instance Name: os-45539345723309377
Instance ID: 08f2d9f7-eeb0-45e7-8562-63aed8f096cc
Flavor: 2
Image: 737969f8-6091-4896-ba9c-f3cf63bd25c5
SSH Keypair: rs-demo
Public IP Address: 50.56.12.229
Environment: _default
Run List: role[base]

knife-openstack compatibility
• Uses the OpenStack API
• Diablo, Essex, Folsom, Grizzly
• Cloudscaling
• Crowbar
• DreamHost
• Nebula
• Piston
• Rackspace Private Cloud

knife-openstack Roadmap
• docs.opscode.com/plugin_knife_openstack.html
• tickets.opscode.com/browse/KNIFE/component/
• windows support
• floating IP address management
• quantum network management

Who was there
• AT&T
• Dell
• DreamHost
• HubSpot
• KT
• Midokura
• Opscode
• Rackspace
• SUSE
There had been several days of conversations, these companies were all represented in the meeting. Missing: eNovance, HP, IBM,
KIO, Mirantis

Where
• #openstack-chef on irc.freenode.net
• groups.google.com/group/opscode-chef-
openstack
• github.com/mattray/openstack-chef-docs
• @chefopenstack
The resources we're using

Licensing
• Apache 2
• Opscode CLA/CCLA required
• http://wiki.opscode.com/display/chef/
How+to+Contribute
• http://wiki.opscode.com/display/chef/
Approved+Contributors
Attendees were all covered already

Where on GitHub
• http://github.com/osops
• chef-repo/
• berkshelf, not git submodules
• cookbooks all end in "-cookbook"
• ie. "nova-cookbook"
• "operations" cookbooks outside scope
• ie. logging, monitoring, provisioning
Move to community GitHub repo, not Opscode's. Opscode will upstream from this repo.

Cookbooks
• cinder
• glance
• horizon
• keystone
• nova
• quantum
• swift
• ceilometer & heat eventually
The core OpenStack services

Goal of incorporating into OpenStack
• Get on StackForge
• will provide CI
• which everyone will probably slave
• Gerrit for code reviews
• we'll sort out reviewers once we start
• Testing with TestKitchen initially
• Grenade? Kong? SmokeStack?
We want to go into "mainline" OpenStack

• support alternative package sources
• source-built coming ("VanillaStack")
• packaging recipes before configuration
• ie. "nova/recipes/nova-compute-packages"
Packages
Use distro packages were applicable, but not everyone wants to use them. Build from source will come in eventually.

Chef Style Guide
• Chef 11 target release
• partial search
• partial templates
• Full-stack Chef-client compatible
• Ruby 1.9.x
• Upstream community cookbooks
• Foodcritic as much as possible
Table stakes

Chef Style Guide
• openstack-common instead of osops-utils
• Attribute injection
• attributes may short-circuit search
• few, if any, attributes in roles
• environment-driven attributes
• Chef Solo not actively supported
• platform logic in attributes files
Already using these patterns

• May release "2013.1.0"
• Chef repo for Grizzly
• Operating Systems (Ubuntu 12.04)
• Databases (MySQL)
• Hypervisors (KVM, LXC)
• Nova network FlatDHCP HA & VLAN
Initial osops release
Opscode employee Matt Ray and Chris McClimans are getting together after ChefConf to work on cleaning up Grizzly. Sources
will be AT&T, Dell, HubSpot and Rackspace primarily.

• Operating Systems (RHEL, SUSE)
• Databases (Postgres)
• Hypervisors (Xen, bare metal)
• Cinder (Ceph, LVM, NetApp)
• Quantum (Bridge, Midokura, Nicira, OVS)
• Folsom backport
• HA Configurations may be stretch goal
because of differing implementations
Grizzly Roadmap
SUSE: SLES, OpenSUSE, Postgres
KT: Xen
HubSpot: bare metal
DreamHost: Ceph
AT&T: LVM
Rackspace: Bridge, NetApp, OVS, RHEL
Opscode: Nicira
Midokura: Folsom, MidoNet
AT&T, SUSE, Rackspace different HA setups

knife-openstack v0.7.0
$ knife openstack
Available openstack subcommands: (for details, knife SUB-
COMMAND --help)
** OPENSTACK COMMANDS **
knife openstack flavor list (options)
knife openstack group list (options)
knife openstack image list (options)
knife openstack server delete SERVER [SERVER] (options)
knife openstack server list (options)
Currently supported features.

knife-openstack compatibility
• Uses the OpenStack API
• Diablo, Essex, Folsom, Grizzly
• Cloudscaling
• Crowbar
• DreamHost
• Nebula
• Piston
• Rackspace Private Cloud
Continue to test for compatibility, will build out CI testing for Opscode-supported knife plugins.

knife-openstack Roadmap
• docs.opscode.com/plugin_knife_openstack.html
• tickets.opscode.com/browse/KNIFE/
component/
• Continues to be managed by Opscode
• Test against multiple OpenStack deployments
for compatibility
• next major release v0.8.0 (May)
• floating IP address management
• network assignment on server creates
More features will undoubtedly show up

• Submit talk "Chef for OpenStack Fall 2013
Overview & Status"
• Review this deck
• Report progress
• Submit for developer track session as well
• See you in Hong Kong!
Fall 2013 OpenStack Summit
We'll see what actually happens in November.

Berkshelf
• http://berkshelf.com/
• tool for managing your Chef Cookbooks and
their dependencies
• Community site, Git, local development
• Berksfile managed in version control

Spiceweasel
• https://github.com/mattray/spiceweasel
• manages your Chef repositories and creating
reproducible infrastructure
• nodes, cookbooks, roles, data bags &
environments with a version controlled
manifest
• validates dependencies
• allows extraction and creation of infrastructure
• lightweight orchestration and cluster
management
• Sputnik Cloud Launcher
• fills gap between the documentation and
deployment of your Chef repository &
infrastructure

• http://www.rackspace.com/cloud/private/
• https://github.com/rcbops-cookbooks/
• Session: Deploying OpenStack with Chef
and Operational Tooling

Test Kitchen
• kitchen-openstack
• https://github.com/RoboticCheese/kitchen-openstack
• Session: Test Kitchen: Multi-Platform Integration Testing for the Masses

pxe_dust
• Provisioning solution for hardware
• Initially developed by Matt Ray
• https://github.com/opscode-cookbooks/pxe_dust
• pxe_dust::bootstrap_template
• pxe_dust::installers
• pxe_dust::server

Crowbar
• Initially developed by Dell
• crowbar.github.com
• Barclamps!
• 1.0 vs. 2.0

Razor
• Initially developed by EMC and Puppet
• Open sourced as a Puppet Labs project
• Install using Puppet, Chef, or manual
• Auto-Discovered Real-Time Inventory Data
• Dynamic Image Selection
• Model-Based Provisioning
• APIs and Plug-in Architecture
• Metal-to-Cloud Application Lifecycle Management
• Session: Harnessing the Power of Bare Metal with Razor and Chef Server

OpenStack Baremetal
• https://wiki.openstack.org/wiki/Baremetal
• driver to allow OpenStack Compute to manage hardware directly (Grizzly)
• provisioned via PXE and managed via IPMI
• OpenStack Compute manages them via the Dashboard, CLI and API
• OpenStack on OpenStack (aka "Triple-O")
• authentication, authorization, quotas, a dashboard and an API provided by OpenStack
• roadmap has device discovery, network management and additional hardware features

currently Folsom release (v3.0.1)
open source Chef 11 server embedded
http://www.rackspace.com/cloud/private/
https://github.com/rcbops
https://github.com/rcbops-cookbooks/

github.com/rcbops/chef-cookbooks/tree/v3.0.1
roles/
environments/
cookbooks/

cookbooks/
apache2/
apt/
aws/
build-essential/
cinder/
collectd/
collectd-graphite/
collectd-plugins/
corosync/
database/
dsh/
erlang/
exerstack/
glance/
graphite/
haproxy/
hardware/
horizon/
keepalived/
keystone/
kong/
memcached/
modules/
monit/
monitoring/
mysql/
mysql-openstack/
nova/
nova-network/
ntp/
openssh/
openssl/
openstack-ha/
osops-utils/
postgresql/
rabbitmq/
rabbitmq-openstack/
rsyslog/
runit/
selinux/
sosreport/
swift/
sysctl/
xfs/
yum/

environments/example.json
{
"name": "example_environment",
"description": "Chef-server example environment",
"cookbook_versions": {
},
"json_class": "Chef::Environment",
"chef_type": "environment",
"default_attributes": {
},
"override_attributes": {
"developer_mode": false,
"monitoring" : {
"metric_provider" : "collectd",
"procmon_provider" : "monit"
},
"glance": {
"image_upload": true,
"images": ["cirros", "precise"]
},

environments/example.json"nova": {
"ratelimit" : {
"api" : { "enabled" : true },
"volume" : { "enabled" : true }
},
"libvirt": { "virt_type": "qemu" },
"networks": [
{
"label": "public",
"ipv4_cidr": "10.10.100.0/24",
"num_networks": "1",
"network_size": "255",
"bridge": "br100",
"bridge_dev": "eth0.100",
"dns1": "8.8.8.8",
"dns2": "8.8.4.4"
},
{
"label": "private",
"ipv4_cidr": "172.16.101.0/24",
"num_networks": "1",
"network_size": "255",
"bridge": "br101",
"bridge_dev": "eth0.101",
"dns1": "8.8.8.8",
"dns2": "8.8.4.4"
}
]
},

environments/example.json
"mysql": {
"allow_remote_root": true,
"root_network_acl": "%"
},
"osops_networks": {
"nova": "192.168.1.0/24",
"public": "192.168.1.0/24",
"management": "192.168.1.0/24"
},
"package_component": "folsom"
}
}

roles/
allinone.rb
base.rb
cinder-all.rb
cinder-api.rb
cinder-scheduler.rb
cinder-setup.rb
cinder-volume.rb
collectd-client.rb
collectd-server.rb
glance-api.rb
glance-registry.rb
glance-setup.rb
glance.rb
graphite.rb
ha-controller1.rb
ha-controller2.rb
haproxy.rb
horizon-server.rb
jenkins-allinone.rb
jenkins-compute.rb
jenkins-controller.rb
jenkins-glance.rb
jenkins-mysql-master.rb
jenkins-nova-api.rb
keystone-api.rb
keystone.rb
mysql-master.rb
nova-api-ec2.rb
nova-api-os-compute.rb
nova-api.rb
nova-cert.rb
nova-controller.rb
nova-misc-services.rb
nova-network-compute.rb
nova-network-controller.rb
nova-network.rb
nova-scheduler.rb
nova-setup.rb
nova-vncproxy.rb
nova-volume.rb
openstack-ha.rb
quantum-network-manager.rb
rabbitmq-server.rb
rsyslog-client.rb
rsyslog-server.rb
single-compute.rb
single-controller.rb
swift-account-server.rb
swift-all-in-one.rb
swift-container-server.rb
swift-management-server.rb
swift-object-server.rb
swift-proxy-server.rb
swift-setup.rb

roles/allinone.rb
name "allinone"
description "This will create an all-in-one Openstack
cluster"
run_list(
"role[single-controller]",
"role[single-compute]"
)

expanded run list
osops-utils::packages, openssh, ntp, sosreport, rsyslog::default,
hardware, osops-utils::default, mysql-openstack::server,
erlang::default, rabbitmq-openstack::server, keystone::server,
keystone::keystone-api, glance::setup, glance::registry,
glance::api, nova::nova-setup, nova-network::nova-controller,
nova::scheduler, nova::api-ec2, nova::api-os-compute,
nova::volume, nova::nova-cert, nova::vncproxy, mysql::client,
mysql::ruby, horizon::server, nova-network::nova-compute,
nova::compute

roles/single-controller
name "single-controller"
description "Nova Controller (non-HA)"
run_list(
"role[base]",
"role[mysql-master]",
"role[rabbitmq-server]",
"role[keystone]",
"role[glance-setup]",
"role[glance-registry]",
"role[glance-api]",
"role[nova-setup]",
"role[nova-network-controller]",
"role[nova-scheduler]",
"role[nova-api-ec2]",
"role[nova-api-os-compute]",
"role[nova-volume]",
"role[nova-cert]",
"role[nova-vncproxy]",
"role[horizon-server]"
)

roles/base
name "base"
description "Base role for a server"
run_list(
"recipe[osops-utils::packages]",
"recipe[openssh]",
"recipe[ntp]",
"recipe[sosreport]",
"recipe[rsyslog::default]",
"recipe[hardware]",
"recipe[osops-utils::default]"
)
default_attributes(
"ntp" => {
"servers" => ["0.pool.ntp.org", "1.pool.ntp.org", "2.pool.ntp.org"]
}
)

roles/mysql-master
name "mysql-master"
description "Installs mysql and sets up replication (if 2
nodes with role)"
run_list(
"role[base]",
"recipe[mysql-openstack::server]"
)

roles/rabbitmq-server
name "rabbitmq-server"
description "RabbitMQ Server (non-ha)"
run_list(
"role[base]",
"recipe[erlang::default]",
"recipe[rabbitmq-openstack::server]"
)

roles/keystone
name "keystone"
description "Keystone server"
run_list(
"role[base]",
"recipe[keystone::server]",
"role[keystone-api]"
)

roles/glance-setup
name "glance-setup"
description "sets up glance registry db and passwords"
run_list(
"role[base]",
"recipe[glance::setup]"
)

roles/glance-registry
name "glance-registry"
description "Glance Registry server"
run_list(
"role[base]",
"recipe[glance::registry]"
)

roles/glance-api
name "glance-api"
description "Glance API server"
run_list(
"role[base]",
"recipe[glance::api]"
)

roles/nova-setup
name "nova-setup"
description "Where the setup operations for nova get run"
run_list(
"recipe[nova::nova-setup]"
)

/nova-network-controller
name "nova-network-controller"
description "Setup nova-networking for controller node"
run_list(
"recipe[nova-network::nova-controller]"
)

roles/nova-scheduler
name "nova-scheduler"
description "Nova scheduler"
run_list(
"role[base]",
"recipe[nova::scheduler]"
)

roles/nova-api-ec2
name "nova-api-ec2"
description "Nova API EC2"
run_list(
"role[base]",
"recipe[nova::api-ec2]"
)

/nova-api-os-compute
name "nova-api-os-compute"
description "Nova API for Compute"
run_list(
"role[base]",
"recipe[nova::api-os-compute]"
)

roles/nova-volume
name "nova-volume"
description "Nova Volume Service"
run_list(
"role[base]",
"recipe[nova::volume]"
)

roles/nova-cert
name "nova-cert"
description "Nova Certificate Service"
run_list(
"role[base]",
"recipe[nova::nova-cert]"
)

roles/nova-vncproxy
name "nova-vncproxy"
description "Nova VNC Proxy"
run_list(
"role[base]",
"recipe[nova::vncproxy]"
)

roles/horizon-server
name "horizon-server"
description "Horizon server"
run_list(
"role[base]",
"recipe[mysql::client]",
"recipe[mysql::ruby]",
"recipe[horizon::server]"
)

roles/single-compute
name "single-compute"
description "Nova compute (with non-HA Controller)"
run_list(
"role[base]",
"role[nova-network-compute]",
"recipe[nova::compute]"
)

Thanks!
Matt Ray
matt@opscode.com
@mattray

Chef and OpenStack Workshop from ChefConf 2013

More Related Content

What's hot (19)

Similar to Chef and OpenStack Workshop from ChefConf 2013 (20)

More from Matt Ray (20)

Recently uploaded (20)

Chef and OpenStack Workshop from ChefConf 2013