OpenStack Tutorial

OpenStack Tutorial
IEEE CloudCom 2010

Bret Piatt
Community Stacker
Twitter: @bpiatt

Application Platforms Undergoing A Major Shift

70’s – 80’s 90’s-2000’s 2010-beyond
Mainframe Era Client Server Era Cloud Era

2010 IT budgets aren’t getting cut..
..but CIOs expect their spend to go further.
#1 Priority is Virtualization
#2 is Cloud Computing
[Based on a Gartner Study]

Overview of Rackspace

Founded in 1998
Publicly traded on NYSE: RAX
120,000+ customers

$628m revenue in 2009 across two major businesses
Dedicated Managed Hosting
Cloud Infrastructure & Apps (Servers, Files, Sites, Email)

Primary focus on customer service ("Fanatical Support")
3,000+ employees

9 datacenters in the US, UK and Hong Kong
65,000+ physical servers

Rackspace Cloud: 3 Products with Solid Traction
Compute: Cloud Servers
Virtualized, API-accessible servers with root access
Windows & Linux (many distros)
Sold by the hour (CPU/RAM/HDD) with persistent storage
Launched 2009
Based on Slicehost
Xen & XenServer HVs
Storage: Cloud Files
Launched 2008
Object file store
v2.0 in May 2010
PaaS: Cloud Sites
Launched 2006
Formally Mosso Source: Guy Rosen (http://www.jackofallclouds.com)

Code it & Load it: .Net, PHP, Python apps autoscaled

Active Ecosystem on Rackspace APIs
Open ReST APIs released July 2009 (Creative Commons License)
Included in major API bindings: Libcloud, Simple Cloud, jclouds, σ-cloud
Supported by key cloud vendors and SaaS services
Marketplace: http://tools.rackspacecloud.com

What is OpenStack?
Overview of the project

OpenStack: The Mission

"To produce the ubiquitous Open Source
cloud computing platform that will meet
the needs of public and private cloud
providers regardless of size, by being
simple to implement and massively
scalable."

OpenStack History

2005 2010
March May June July

Rackspace Rackspace Decides NASA Open OpenStack formed Inaugural Design
Cloud to Open Source Sources Nebula b/w Rackspace and Summit in Austin
developed Cloud Software Platform NASA

OpenStack History

2011
July October November February

OpenStack First ‘Austin’ code First public Design Second ‘Bexar’ code
launches with 25+ release with 35+ Summit in San release planned
partners partners Antonio

OpenStack Founding Principles

Apache 2.0 license (OSI), open development process
Open design process, 2x year public Design Summits
Publicly available open source code repository
Open community processes documented and
transparent
Commitment to drive and adopt open standards
Modular design for deployment flexibility via APIs

Community with Broad Commercial
Support

OpenStack Isn't Everything

Consultants
Business Process Automation

Database Engineers
Operating System
Technicians
Systems Security
Professionals
Network Experts

Servers, Firewalls, Load Balancers
Operating Systems
Storage
Management Tools
Virtualization

Data Center
Networking
Power

Software to provision virtual machines on
standard hardware at massive scale

OpenStack Compute

creating open source software to build
public and private clouds

Software to reliably store billions of objects
distributed across standard hardware

OpenStack
Object Storage

OpenStack Release Schedule
Design Summit:
April TBA 2011

Cactus:
April 15, 2011
Community gathers to
plan for next release,
Bexar: likely Fall 2011
February 3, 2011
OpenStack Compute ready for
large service provider scale
deployments
This is the ‘Rackspace-ready’
release; need to communicate
OpenStack Compute ready
Rackspace support and plans
for enterprise private cloud
for deployment
deployments and mid-size
service provider
deployments
Enhanced documentation
Easier to install and deploy

Building an OpenStack Cloud
Datacenter, Hardware, and Process

Cloud Ready Datacenter Requirements

Bootstrapping Your Physical Nodes
Bootstrapping the Host Machines

Object Storage

Zettabyte
1,000 Exabytes
1,000,000 Petabytes
All of the data on Earth today
(150GB of data per person)

Zettabyte
2% OF THE DATA ON EARTH IN 2020

Data Must Be Stored Efficiently
If we stored all of the global data as “an average” enterprise..

ITEM MONTHLY FIGURES
ENTERPRISE AVGERAGE STORAGE COST $1.98 PER GIGABYTE
WORLD GDP $5.13 TRILLION
COST TO STORE A ZETTABYTE $1.98 TRILLION

..it would take..
..38.5% of the World GDP!

Object Storage Key Features

ReST-based API Data distributed evenly
throughout system
Scalable to multiple
petabytes, billions of
objects

Account/Container/Object structure
(not file system, no nesting) plus
Replication (N copies of accounts,
containers, objects)

No central
database
Hardware agnostic: standard
hardware, RAID not required

System Components

The Ring: Mapping of names to entities (accounts,
containers, objects) on disk.
Stores data based on zones, devices, partitions, and replicas
Weights can be used to balance the distribution of partitions
Used by the Proxy Server for many background processes

Proxy Server: Request routing, exposes the public API
Replication: Keep the system consistent, handle failures

Updaters: Process failed or queued updates

Auditors: Verify integrity of objects, containers, and accounts

System Components (Cont.)

Account Server: Handles listing of containers, stores as SQLite DB
Container Server: Handles listing of objects, stores as SQLite DB
Object Server: Blob storage server, metadata kept in xattrs, data in
binary format
Recommended to run on XFS
Object location based on hash of name & timestamp

Software Dependencies

Object Storage should work on most Linux platforms with the following
software (main build target for Austin release is Ubuntu 10.04):
Python 2.6
rsync 3.0

And the following python libraries:
Eventlet 0.9.8
WebOb 0.9.8
Setuptools
Simplejson
Xattr
Nose
Sphinx

Evolution of Object Storage
Architecture
Version 1: Central DB Version 2: Fully Distributed
(Rackspace Cloud Files 2008) (OpenStack Object Storage 2010)

Example Small Scale Deployment

Public Internet Example OpenStack
Object Storage Hardware

Load Balancers (SW)

5 Zones
2 Proxies per 25
Storage Nodes
10 GigE to Proxies
1 GigE to
Storage Nodes
24 x 2TB Drives
per Storage Node

Example Large Scale Deployment -- Many Configs Possible

Compute

OpenStack Compute Key Features

ReST-based API
Asynchronous
eventually consistent
communication

Horizontally and
massively scalable

Hypervisor agnostic:
support for Xen ,XenServer, Hyper-V,
KVM, UML and ESX is coming
Hardware agnostic:
standard hardware, RAID not required

User Manager

Cloud Controller: Global state of
system, talks to LDAP, OpenStack
Object Storage, and node/storage
workers through a queue
ATAoE / iSCSI

API: Receives HTTP requests,
converts commands to/from API
format, and sends requests to cloud
controller

Host Machines: workers
that spawn instances

Glance: HTTP + OpenStack Object
OpenStack Compute Storage for server images

System Components
API Server: Interface module for command and control requests
Designed to be modular to support multiple APIs
In current release: OpenStack API, EC2 Compatibility Module
Approved blueprint: Open Cloud Computing Interface (OCCI)
Message Queue: Broker to handle interactions between services
Currently based on RabbitMQ
Metadata Storage: ORM Layer using SQLAlchemy for datastore
abstraction
In current release: MySQL
In development: PostgreSQL
User Manager: Directory service to store user identities
In current release: OpenLDAP, FakeLDAP (with Redis)
Scheduler: Determines the placement of a new resource
requested via the API
Modular architecture to allow for optimization
Base schedulers included in Austin: Round-robin, Least busy

System Components (Cont.)
Compute Worker: Manage compute hosts through commands
received on the Message Queue via the API
Base features: Run, Terminate, Reboot, Attach/Detach
Volume, Get Console Output
Network Controller: Manage networking resources on compute
hosts through commands received on the Message Queue via the
API
Support for multiple network models
Fixed (Static) IP addresses
VLAN zones with NAT
Volume Worker: Interact with iSCSI Targets to manage volumes
Base features: Create, Delete, Establish
Image Store: Manage and deploy VM images to host machines

Hypervisor Independence
Cloud applications should be designed and packaged abstracted from
the hypervisor, deploy and test for best fit for your workload
Manage application definition and workload, not the machine image
Configuration management
Abstract virtual machine definition

Open Virtualization Format

Network Models
Private VMs on Project VLANs or Public VMs on flat networks

Network Details
Security Group: Named collection of network access rules
Access rules specify which incoming network traffic should be
delivered to all VM instances in the group
Users can modify rules for a group at any time
New rules are automatically enforced for all running
instances and instances launched from then on
Cloudpipe: Per project VPN tunnel to connect users to the cloud
Certificate Authority: Used for Project VPNs and to decrypt
bundled images
Cloudpipe Image: Based on Linux with OpenVPN

Example OpenStack
Compute Hardware
Public Network (other models possible)

Server Groups
Dual Quad Core
RAID 10 Drives
1 GigE Public
1 GigE Private
1 GigE Management Private Network
(intra data center)

Management

Thank You!

Bret Piatt
Email: bret@openstack.org
Twitter: @bpiatt

Questions & Answers

Backup Content
Additional Information

OpenStack Documentation

Project Technical Documentation
Overall: http://wiki.openstack.org
Object Storage (Swift): http://swift.openstack.org
Compute (Nova): http://nova.openstack.org

Project General Documentation
Home Page: http://openstack.org
Announcements: http://openstack.org/blog

OpenStack: Core Open Principles
Open Source: All code will be released under the Apache License
allowing the community to use it freely.

Open Design: Every 6 months the development community will hold a
design summit to gather requirements and write specifications for the
upcoming release.

Open Development: We will maintain a publicly available source code
repository through the entire development process. This will be hosted
on Launchpad, the same community used by 100s of projects
including the Ubuntu Linux distribution.

Open Community: Our core goal is to produce a healthy, vibrant
development and user community. Most decisions will be made using
a lazy consensus model. All processes will be documented, open and
transparent.

Backup Content
Bootstrapping a cloud

Hardware Selection

OpenStack is designed to run on industry
standard hardware, with flexible configurations

Compute
x86 Server (Hardware Virt. recommended)
Storage flexible (Local, SAN, NAS)

Object Storage
x86 Server (other architectures possible)
Do not deploy with RAID (can use controller for cache)

Server Vendor Support

Find out how much configuration your
hardware vendor can provide
Basic needs
BIOS settings
Network boot
IP on IPMI card

Advanced support
Host OS installation
Still get management network IP via DHCP

Network Device Configuration

Build in a manner that requires minimal change
Lay out addressing in a block based model
Go to L3 from the top of rack uplink
Keep configuration simple
More bandwidth is better than advanced QoS
Let the compute host machines create logical zones

Host Networking

DHCP for the management network
Infinite leases
Base DNS on IP
Ex. nh-pod-a-10-241-61-8.example.org
OpenStack Compute handles IP provisioning
for all guest instances – Cloud deployment tools
only need to setup management IPs

Host OS Seed Installation

BOOTP / TFTP – Simple to configure
Security must be handled outside of TFTP
Host node must be able to reach management
system via broadcast request
Top of rack router can be configured to forward
GPXE
Not all hardware supports
Better concurrent install capability than TFTP

Host OS Installation

Building a configuration based on a scripted
installation is better than a monolithic
“golden image”
Preseed for Ubuntu / Debian hosts
Kickstart for Fedora / CentOS / RHEL hosts
YaST for SUSE / SLES hosts
Remote bootstrapping for XenServer / Hyper-V hosts
Scripted configuration allows for incremental
updates with less effort

Post OS Configuration

Utilize a configuration management solution
Puppet / Chef / Cfengine
Create roles to scale out controller infrastructure
Queue
Database
Controller
Automate registration of new host machines
Base the configuration to run on management net IP

OpenStack Tutorial

Recommended

More Related Content

What's hot (20)

Viewers also liked (16)

Similar to OpenStack Tutorial (20)

More from Bret Piatt (7)

OpenStack Tutorial