Testing Your Automation Code (Vagrant Version)

v0.0.1
Testing Your
Automation Code
Mischa Taylor (@misheska)
misheska@getchef.com
Copyright (C) 2014 Chef Software, Inc.
1
1

3
You’ve used and want to
bulletproof
your so that
you are with
Spec
3

In this class
• We’ll add tests to the apache cookbook from the
Fundamentals Course
• We’ll show you how to run cookbooks in a sandbox
environment mirroring production with Test
Kitchen
• We’ll show you how to detect suspicious cookbook
code with Foodcritic & RuboCop
• We’ll show you how to produce runnable
documentation with ChefSpec
4
4

Using Chef is half the battle
5
“Chef is like....
tests for your infrastructure”
-Ezra Zygmuntowicz, Co-Founder EngineYard
http://www.akitaonrails.com/2008/6/5/railsconf-2008-brazil-rails-podcast-special-edition#.U0HﬁF7Ed-8
5

Chef makes things more testable
• Chef automates infrastructure in a repeatable
fashion
6
6

What’s the other half of the battle?
7
“Have a plan”
-Adam Jacob, Co-Founder Chef
7

There’s no more magic to testing
8
http://www.flickr.com/photos/dkeats/4128747046/sizes/s/in/photostream/
8

You Are The Testing A-Team When...
9
9

Recommended plan
• Build in quality and robustness up front
10
https://ﬂic.kr/p/8W67ZC
10

Otherwise you could...
• Verify and validate just before going to production
until time runs out. But time always runs out
11
http://mrg.bz/iEr1oj
11

Waiting to test when it’s “done”
12
Intention:
Reality:
Build Test Deploy
Build
T
es
De
ploy
We’re late
no time
to test!
12

Bake testing in earlier
15
Shorter cycles,
to start testing early as possible:
Build Test Deploy Build Test Deploy Build Test Deploy
15

Penny saved with testing
16
Up Front Testing Saves Money
https://www.ﬂickr.com/people/68751915@N05/
16

Test arrangement
• Arrange tests to get feedback fast - at the earliest
possible time
18
seconds
minutes
hours
Foodcritic/Rubocop
ChefSpec
Serverspec
18

Reason for multiple tools
• Finding a bug in something that you can’t execute
is freaking hard!
• While fixing bugs before writing code is cheap,
finding them is expensive
19
19

The Tools
• Each tool is specialized to give feedback as early
as possible during the cookbook authoring process
20
20

What each tool does
• In your text editor when you type in cookbook code:
• Foodcritic analyzes your Chef style
• RuboCop analyzes your Ruby coding technique
• Before you deploy to a test node:
• ChefSpec helps you document and organize your
code
• After you deploy to a test node:
• Serverspec verifies a cookbook behaves as intended
21
21

Legend: Do I run that command on my workstation?
$ whoami
i-am-a-workstation
This is an example of a command to run on your workstation
user@hostname:~$ whoami
i-am-a-chef-node
This is an example of a command to run on your target node via SSH.
23
23

$ ifconfig
Legend: Example Terminal Command and Output
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
! options=3<RXCSUM,TXCSUM>
! inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
! inet 127.0.0.1 netmask 0xff000000
! inet6 ::1 prefixlen 128
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280
stf0: flags=0<> mtu 1280
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
! ether 28:cf:e9:1f:79:a3
! inet6 fe80::2acf:e9ff:fe1f:79a3%en0 prefixlen 64 scopeid 0x4
! inet 10.100.0.84 netmask 0xffffff00 broadcast 10.100.0.255
! media: autoselect
! status: active
p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
! ether 0a:cf:e9:1f:79:a3
! media: autoselect
! status: inactive
24
24

OPEN IN EDITOR:
SAVE FILE!
~/hello_world
Hi!
I am a friendly file.
Legend: Example of editing a file on your workstation
25
25

v2.0.0
Workstation Setup - Linux and
Mac OS X (VirtualBox)
Getting started
26
26

Install Chef
• Install Chef (if not already installed)
http://www.getchef.com/chef/install
27
27

$ curl -L http://www.getchef.com/chef/install.sh | sudo bash
Workstation Setup - Mac OS X / Linux
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13347 100 13347 0 0 12147 0 0:00:01 0:00:01 --:--:-- 12155
Downloading Chef for mac_os_x...
Installing Chef
installing with sh...
Verifying archive integrity... All good.
Uncompressing The full stack of chef...
Thank you for installing Chef!
Checksum compare with shasum succeeded.
Installing Chef
installing with sh...
Verifying archive integrity... All good.
Uncompressing The full stack of chef
31
31

What just happened?
• Chef and all of its dependencies installed via an
operating system-specific package ("omnibus installer")
• Installation includes
• The Ruby language - used by Chef
• knife - Command line tool for administrators
• chef-client - Client application
• ohai - System profiler
• ...and more
32
32

OPEN IN EDITOR: $HOME/.bash_profile
export PATH="/opt/chef/embedded/bin:$PATH"
Add Chef Client to PATH
33
33

$ source $HOME/.bash_profile
Make the profile active
34
34

$ which ruby
Check the PATH setting
/opt/chef/embedded/bin/ruby
35
35

$ chef-client --version
Verify Install
Chef: 11.12.2
36
36

Install Developer Tools
• Install Developer Tools to build native extensions
37
http://patshaughnessy.net/2011/10/31/dont-be-terriﬁed-of-building-native-extensions
37

Install Developer Tools
• Use build-essential cookbook:
https://github.com/opscode-cookbooks/build-
essential
38
38

$ mkdir /tmp/cookbooks
Temporary area for cookbook downloads
39
39

$ cd /tmp/cookbooks
Easiest to go to cookbooks dir
40
40

$ knife cookbook site download build-essential
$ tar xvf build-essential-*.tar.gz
Download build-essential cookbook
41
41

$ cd /tmp
$ sudo chef-client -z -o build-essential
Use local mode to run cookbook
42
42

$ gcc --version
Verify Install
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)
43
43

Mac OS X will prompt - ignore
44
44

$ cd /tmp/cookbooks
45
45

$ knife cookbook site download git
$ knife cookbook site download dmg
$ knife cookbook site download windows
$ knife cookbook site download runit
$ knife cookbook site download yum
$ knife cookbook site download yum-epel
$ knife cookbook site download chef_handler
Download git cookbook + dependencies
46
46

$ tar xvf git*.tar.gz
$ tar xvf dmg*.tar.gz
$ tar xvf windows*.tar.gz
$ tar xvf runit*.tar.gz
$ tar xvf yum-3*.tar.gz
$ tar xvf yum-epel*.tar.gz
$ tar xvf chef_handler*.tar.gz
Extract cookbook archives
47
47

$ cd /tmp
$ sudo chef-client -z -o git
48
48

$ git --version
Verify Install
git version 1.8.2.1
49
49

$ rm -rf /tmp/cookbooks
Keep it neat
50
50

v2.0.0
Workstation Setup - Windows
(VirtualBox)
Getting started
53
53

Install Chef
• Install Chef (if not already installed)
http://www.getchef.com/chef/install
54
54

> where ruby
58
PS> get-command ruby
58

> C:opscodechefembeddedbinruby.exe
59
Output:
59

> chef-client --version
Verify Install
Chef: 11.12.2
60
60

Developer Tools - Windows
• build-essential cookbook does not currently support
Windows.
• All the necessary developer tools come packaged
with the Omnibus Installer.
61
61

> mkdir %TEMP%cookbooks
Temporary area for cookbook downloads
62
PS> mkdir $env:tempcookbooks
62

> cd %TEMP%cookbooks
63
PS> cd $env:tempcookbooks
63

> knife cookbook site download build-essential
> knife cookbook site download git
> knife cookbook site download dmg
> knife cookbook site download windows
> knife cookbook site download runit
> knife cookbook site download yum
> knife cookbook site download yum-epel
> knife cookbook site download chef_handler
Download git cookbook + dependencies
64
64

> tar xvf build-essential*.tar.gz
> tar xvf git*.tar.gz
> tar xvf dmg*.tar.gz
> tar xvf windows*.tar.gz
> tar xvf runit*.tar.gz
> tar xvf yum-3*.tar.gz
> tar xvf yum-epel*.tar.gz
> tar xvf chef_handler*.tar.gz
Extract cookbook archives
65
65

> cd %TEMP%
> chef-client -z -o git
66
PS> cd $env:temp
PS> chef-client -z -o git
Run As
Administrator
Run As
Administrator
66

> git --version
Verify Install
git version 1.8.1.msysgit.1
67
67

> rmdir /s %TEMP%cookbooks
Keep it neat
68
PS> rm $env:tempcookbooks -Recurse -Force
Run As
Administrator
68

Install Sublime/Notepad++
69
69

chef-fundamentals-repo
• We’re going to build on the chef-fundamentals-
repo created in the Chef Fundamentals training,
adding tests:
https://github.com/learnchef/chef-fundamentals-repo/
tree/master/cookbooks/apache
73
73

$ cd $HOME
Home directory - great place for source
74
$ cd %USERPROFILE%
74

$ git clone https://github.com/learnchef/chef-fundamentals-repo.git
Grab the chef-fundamentals repo from GitHub
Cloning into 'chef-fundamentals-repo'...
remote: Reusing existing pack: 247, done.
remote: Total 247 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (247/247), 149.52 KiB | 184.00 KiB/s,
done.
Resolving deltas: 100% (45/45), done.
Checking connectivity... done.
75
75

$ cd chef-fundamentals-repo
76
!"" Berksfile
!"" cookbooks/
!"" data_bags/
!"" environments/
!"" .git/
!"" README.md
!"" roles/
#"" Vagrantfile
76

v2.0.0
Reviewing Cookbooks with
Test Kitchen (Vagrant Version)
77
77

Apache Clowns & Bears
• We’ll be focusing on the Apache Clowns and
Bears cookbook in:
chef-‐fundamentals-‐repo/cookbooks/apache
78
78

Cookbook Review with Test Kitchen
• Test Kitchen is a great environment in which to
develop and review cookbooks
79
79

Back in my day...
• Running a cookbook
involved a lot of setup:
• Configure workstation
• Configure Chef Server
• Bootstrap node
80
80

No longer a chore
• Test Kitchen lets you set up sandbox environments
in which to run cookbooks right on your Chef
development workstation
81
81

Supported Environments
• Test Kitchen supports:
• Virtual Machines
• Cloud Instances
• Metal - Physical Servers
• Containers (Docker, LXC, etc.)
82
82

Sandbox Benefits
• A sandbox environment:
• Is a safe place to make mistakes
• Easily reset to a clean config
• Can simulate production
83
83

Virtual Machines vs. Containers
84
Hypervisor/Host OS
Guest OS
Core OS
Guest OS Guest OS
App App App
App App App
Virtual Machines Containers
84

85
Test Kitchen is
packaged as a
Ruby Gem
85

Ruby Gem
• A gem is a application or supporting library
written in ruby, installable with:
86
gem
install
86

$ sudo gem install test-kitchen --no-ri --no-rdoc
Install Test Kitchen
87
$ sudo env "PATH=$PATH"
gem install test-kitchen --no-ri --no-rdoc
Run As
Administrator
> gem install test-kitchen --no-ri --no-rdoc
87

Install Test Kitchen
88
Fetching: net-scp-1.1.2.gem (100%)
Fetching: safe_yaml-1.0.2.gem (100%)
Fetching: thor-0.19.1.gem (100%)
Fetching: test-kitchen-1.2.1.gem (100%)
Successfully installed net-scp-1.1.2
Successfully installed safe_yaml-1.0.2
Successfully installed thor-0.19.1
Successfully installed test-kitchen-1.2.1
4 gems installed
Text
Output:
88

OPEN IN EDITOR: $HOME/.gemrc
gem: --no-ri --no-rdoc
Automatically add --no-ri & --no-rdoc
89
or %USERPROFILE%/.gemrc
89

$ cd chef-fundamentals-repo/cookbooks/apache
Go to Apache cookbook dir
90
90

91
We’re going to
need a lot of gems
91

Gemfile
• A Gemfile can be used to list all the gems you need
for cookbook testing
92
92

Gemfile template
• Test Kitchen can generate a Gemfile template
(among other things)
93
93

$ kitchen init --create-gemfile
create .kitchen.yml
create test/integration/default
create Gemfile
append Gemfile
append Gemfile
You must run ‘bundle install’ to fetch any new
gems.
Create Gemfile template
94
94

Kitchen output on Windows
• The strange <-‐[0m<-‐33m characters in the Windows
output are ANSI escape sequences.
95
95

Kitchen output on Windows
• ANSICon also adds support for ANSI escape
sequences to Windows as a workaround:
https://github.com/adoxa/ansicon
Download link:
http://adoxa.hostmyway.net/ansicon/
96
96

chef-fundamentals-repo/cookbooks/apache/Gemfile
source 'https://rubygems.org'
gem 'test-kitchen'
gem 'kitchen-vagrant'
97
97

$ kitchen init --create-gemfile
create .kitchen.yml
create test/integration/default
create Gemfile
append Gemfile
append Gemfile
You must run ‘bundle install’ to fetch any new
gems.
Create Gemfile template
98
98

100
Gemﬁle is read by:
bundle
install
to install gems
100

101
No need to install bundler
• Chef install includes the bundler gem
• Outside the Chef install, bundler can be installed via:
•gem
install
bundler
101

$ bundle install --path vendor/bundle
Do what Test Kitchen tells you
Fetching gem metadata from https://rubygems.org/..........
Fetching gem metadata from https://rubygems.org/..
Installing mixlib-shellout (1.3.0)
Installing net-ssh (2.8.0)
Installing net-scp (1.1.2)
Installing safe_yaml (1.0.1)
Installing thor (0.19.1)
Installing test-kitchen (1.2.1)
Installing kitchen-vagrant (0.14.0)
Using bundler (1.1.5)
Your bundle is complete! It was installed into ./vendor/bundle
102
102

103
Vendor Everything
• Gems change frequently and sometimes conflict
• -‐-‐path option passed to bundle
install
overrides system gems by installing Gemfile gems
locally
• Bundler docs recommend using vendor/bundle
103

104
If you vendor gems use:
bundle
exec
to read gems in
vendor/bunde
104

105
Test Kitchen commands
• Let’s cover a few essential Test Kitchen commands
105

106
kitchen list
• kitchen
list prints out a list of sandbox instances
defined in .kitchen.yml
106

$ bundle exec kitchen list
kitchen list
Instance Driver Provisioner Last Action
default-ubuntu-1204 Vagrant ChefSolo <Not Created>
default-centos-64 Vagrant ChefSolo <Not Created>
107
107

108
kitchen create
• kitchen
create
<instance_name> spins up a
sandbox instance
108

$ bundle exec kitchen create default-centos-64
kitchen create
-----> Starting Kitchen (v1.2.1)
-----> Creating <default-centos-64>...
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'opscode-centos-6.4'...
==> default: Matching MAC address for NAT networking..
...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Setting hostname...
Vagrant instance <default-centos-64> created.
Finished creating <default-centos-64> (0m35.40s).
-----> Kitchen is finished. (0m35.65s)
109
109

110
kitchen login
• kitchen
login
<instance_name> uses ssh to
login to the sandbox instance
110

kitchen login on Windows
• Test Kitchen requires ssh to login to guest VMs
• Easiest way to get ssh is to use the Unix command
line tools packaged with Git for Windows
111
111

kitchen login on Windows
• Add the Unix tools to your path
• 64-bit: C:Program
Files
(x86)Gitbin
• 32-bit: C:Program
FilesGitbin
112
112

$ bundle exec kitchen login default-centos-64
kitchen login
Last login: Mon Nov 25 07:00:52 2013 from 10.0.2.2
[vagrant@default-centos-64 ~]$ cat /etc/redhat-release
CentOS release 6.4 (Final)
[vagrant@default-centos-64 ~]$ exit
logout
Connection to 127.0.0.1 closed.
113
113

114
kitchen destroy
• kitchen
destroy
<instance_name> shuts down a
sandbox instance and destroys and virtual resources
allocated
114

$ bundle exec kitchen destroy default-centos-64
kitchen destroy
-----> Destroying <default-centos-64>...
==> default: Forcing shutdown of VM...
==> default: Destroying VM and associated drives...
Vagrant instance <default-centos-64> destroyed.
Finished destroying <default-centos-64> (0m3.07s).
115
115

116
kitchen
converge
performs a Chef run
in the sandbox instance
116

$ bundle exec kitchen converge default-centos-64
Perform Chef run
==> default: Box 'opscode-centos-6.4' could not be found. Attempting
to find and install...
...
[2014-03-30T09:09:59+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 11.10.4
[2014-03-30T09:09:59+00:00] INFO: *** Chef 11.10.4 ***
[2014-03-30T09:09:59+00:00] INFO: Chef-client pid: 2542
[2014-03-30T09:09:59+00:00] INFO: Setting the run_list to
["recipe[apache::default]"] from JSON
....
117
117

Perform Chef run
==> default: Box 'opscode-centos-6.4' could not be found. Attempting to find and install...
...
[2014-03-30T09:09:59+00:00] INFO: *** Chef 11.10.4 ***
[2014-03-30T09:09:59+00:00] INFO: Setting the run_list to ["recipe[apache::default]"] from JSON
....
[2014-04-07T02:32:50-04:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully
(exit code 1)
>>>>>> Converge failed on instance <default-centos-64>.
>>>>>> Please see .kitchen/logs/default-centos-64.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sudo -E chef-solo --config /tmp/kitchen/solo.rb --json-attributes /
tmp/kitchen/dna.json --log_level info]
>>>>>> ----------------------
118
FAIL
118

119
Our ﬁrst bit of feedback!
119

121
Undefined attribute
• When you see undefined
method
‘[]’
for

nil:NilClass it oftentimes means you have an
undefined attribute
• Let’s see if node["motd"]["company"] is being set
121

cookbooks/apache/attributes/default.rb
122
default["apache"]["indexfile"] = "index1.html"
default["apache"]["sites"]["clowns"] = { "port"
=> 80 }
default["apache"]["sites"]["bears"] = { "port"
=> 81 }
NOPE
122

123
Undefined attribute
• How is ["motd"]["company"] set in the motd
cookbook?
123

cookbooks/motd/attributes/default.rb
124
default["motd"]["company"] = "Chef"
124

Good cookbooks
• Good cookbooks can be used in isolation
• They set reasonable defaults for all attributes used
• Test Kitchen is designed to run a cookbook in
isolation to give you feedback on attribute use
125
125

Test attributes
• We won’t fix the motd cookbook in this class
• Test Kitchen supports injection of test attributes
• We’ll supply the correct attribute in
the .kitchen.yml configuration file
126
126

Default .kitchen.yml
127
---
driver:
name: vagrant
provisioner:
name: chef_solo
platforms:
- name: ubuntu-12.04
- name: centos-6.4
suites:
- name: default
run_list:
- recipe[apache::default]
attributes:
127

---
driver:
name: vagrant
provisioner:
name: chef_solo
platforms:
- name: ubuntu-12.04
- name: centos-6.4
suites:
- name: default
run_list:
attributes:
motd: {company: Chef}
cookbooks/apache/.kitchen.yml
Set node[“motd”][“company”] (vagrant)
128
OPEN IN EDITOR:
128

Perform Chef run
...
[2014-03-30T09:09:59+00:00] INFO: *** Chef 11.10.4 ***
....
129
129

Perform Chef run
==> default: Box 'opscode-centos-6.4' could not be found. Attempting to find and
install...
...
[2014-03-30T09:09:59+00:00] INFO: *** Chef 11.10.4 ***
[2014-03-30T09:09:59+00:00] INFO: Setting the run_list to ["recipe[apache::default]"]
from JSON
....
[2014-04-07T02:40:03-04:00] INFO: Report handlers complete
Chef Client finished, 5/10 resources updated in 2.52587913 seconds
Finished converging <default-centos-64> (0m4.15s).
130
WIN
130

Where to go next
• Learning Chef book excerpt
was sent to you are part of
the class registration.
• Chapter 1 covers Test
Kitchen and .kitchen.yml
format in more detail.
• Appendix provides
sample .kitchen.yml configs
131
131

v2.0.0
Automated Verification in
Test Kitchen with Serverspec
(Vagrant Version)
132
132

133
Let’s verify that the Apache cookbook
actually works by
conﬁguring Test Kitchen to allow
web browser access by
the Chef Development workstation
133

OPEN IN EDITOR: cookbooks/apache/.kitchen.yml
---
driver:
name: vagrant
provisioner:
name: chef_solo
platforms:
- name: centos-6.4
driver_config:
network:
- ["private_network", {ip: "33.33.33.10"}]
suites:
- name: default
run_list:
attributes:
Network configuration
134
134

$ bundle exec kitchen destroy default-centos-64
Network config requires destory
==> default: Forcing shutdown of VM...
==> default: Destroying VM and associated drives...
Vagrant instance <default-centos-64> destroyed.
135
135

Perform Chef run
...
[2014-03-30T09:09:59+00:00] INFO: *** Chef 11.10.4 ***
....
136
136

Private network set up
137
137

Clowns & Bears in your web browser
138
138

Serverspec author
• Written by Gosuke Miyashita
141
141

OPEN IN EDITOR: cookbooks/apache/Gemfile
gem 'test-kitchen'
gem 'serverspec', '~> 1.1'
Add serverspec to Gemfile
142
142

gem 'test-kitchen'
gem ‘serverspec’, ‘~> 1.1’
Add serverspec to Gemfile
143
PessimisticVersion
Constraint
143

Version Constraint
144
•If a gem properly follows semantic
versioning with its versioning scheme.
You can take advantage of this to
choose a version constraint to lock
down the gem in your application.
http://guides.rubygems.org/patterns/#declaring_dependencies
144

Semantic Versioning
145
Given a version number MAJOR.MINOR.PATCH, increment
the:
1.MAJOR version when you make incompatible API
changes,
2.MINOR version when you add functionality in a backwards-
compatible manner, and
3.PATCH version when you make backwards-compatible bug
fixes.
Additional labels for pre-release and build metadata are
available as extensions to the MAJOR.MINOR.PATCH format.
http://guides.rubygems.org/patterns/#semantic_versioning
145

Versioning Example
146
Let’s say the following releases of a gem exist:
■Version 2.1.0 — Baseline
■Version 2.2.0 — Introduced some new (backward
compatible) features.
■Version 2.2.1 — Removed some bugs
■Version 2.2.2 — Streamlined your code
■Version 2.3.0 — More new features (but still backwards
compatible).
■Version 3.0.0 — Reworked the interface. Code written to
version 2.x might not work.
http://guides.rubygems.org/patterns/#semantic_versioning
146

Version Constraint
147
gem 'library', '= 2.2.0'
Only use version 2.2.0
147

Optimistic Version Constraint
148
gem 'library', '>= 2.2.0'
Assume all changes from 2.x on will
work, including 3.0.0 and higher
148

Pessimistic Version Constraint
149
gem 'library', '>= 2.2.0', ‘< 3.0’
Explicitly exclude any versions that might
break your code
149

Pessimistic Version Constraint
150
gem 'library', '>= 2.2.0', ‘< 3.0’
Shorthand for:
Using the twiddle-wakka:
gem 'library', '~> 2.2'
150

$ bundle install
Install Serverspec
151
$ CONFIGURE_ARGS="--with-ldflags=
'-Wno-error=unused-command-line-argument-hard-error-in-
future'" bundle install
clang 5.1 Workaround
151

Documentation form
153
describe
‘<entity>’
do

<descriptions
here>
end
153

Documentation example
154
describe
‘clowns
site’
do

...
end
154

Description form
155
describe
‘<entity>’
do

it
‘<description>’
do

...

end
end
155

Description example
156
describe
‘clowns
site’
do

it
‘responds
on
port
80’
do

...

end
end
156

Expectation form
157
describe
‘<entity>’
do

it
‘<description>’
do

expect(thing).to
eq
result

end
end
157

Expectation form
158
describe
‘<entity>’
do

it
‘<description>’
do

expect(thing).to
eq
result

end
end
Matcher
158

Expectation example
159
describe
‘clowns
site’
do

it
‘responds
on
port
80’
do

expect(port
80).to
be_listening
‘tcp’

end
end
159

Expectation form
160
describe
‘<entity>’
do

it
‘<description>’
do

expect(thing).to
eq
result

end
end
This is new syntax
160

Should vs Expect
161
describe
‘clowns
site’
do

it
‘responds
on
port
80’
do

expect(port
80).to
be_listening
‘tcp’

end
end
Expect Form
One-Liner Should Form
describe
‘clowns
site’
do

describe
port(80)
do

it
{
should
be_listening.with(‘tcp’)
}

end
end
161

Expect vs. Should
162
Debate on whether or not to use expect vs. should is
epic:
http://myronmars.to/n/dev-blog/2012/06/rspecs-new-
expectation-syntax
...and pointless. Use whatever makes the most sense
to you. There are some technical limitations to the
‘should’ form, but if you stick to the “one-liner should”
syntax, they are essentially interchangeable.
162

We use Expect Form
163
Because all the ChefSpec examples are in the expect
form
163

164
Let’s write some serverspec
tests!
164

165
Default location for tests
• By default, Test Kitchen will look in the test/
integration directory for test-related files
• For convenience, Test Kitchen creates this directory
when you run kitchen
init
165

166
.
!"" attributes/
!"" .bundle/
!"" CHANGELOG.md
!"" files/
!"" Gemfile
!"" Gemfile.lock
!"" .kitchen/
!"" .kitchen.yml
!"" metadata.rb
!"" README.md
!"" recipes/
!"" templates/
!"" test/
# $"" integration/
$"" vendor/
166

167
Suite subdirectory
• Test Kitchen requires a few more directories underneath
test/integration
• First directory name underneath test/integration should
match the suite name:
└──
test/

└──
integration/

└──
<suite_name>/
167

OPEN IN EDITOR: cookbooks/apache/.kitchen.yml
---
driver:
name: vagrant
provisioner:
name: chef_solo
platforms:
- name: centos-6.4
driver_config:
network:
- ["private_network", {ip: "33.33.33.10"}]
suites:
- name: default
run_list:
attributes:
Network configuration
168
Suite name
168

169
Suite subdirectory
• Our suite name is default
└──
test/

└──
integration/

└──
default/
169

170
Busser directory
• The next directory level denotes the test plugin, as
Test Kitchen many different kinds of test plugins. A test
plugin is called a busser. We’ll be using the busser
directory called serverspec.
└──
test/

└──
integration/

└──
default/

└──
serverspec/
170

171
Hostname directory
• Serverspec supports testing via SSH, so it requires yet
another directory level to denote the hostname. We
won’t be using this capability, so it should be localhost
└──
test/

└──
integration/

└──
default/

└──
serverspec/

└──
localhost/
171

172
Hostname directory
• NOTE: Use of the localhost directory is optional
172

$ mkdir -p test/integration/default/serverspec
Create test directory structure
173
> mkdir testintegrationdefaultserverspec
173

174
*_spec.rb files
• By convention, Test Kitchen expects files with tests
to end in _spec.rb
174

Serverspec expectation form
• Every specialized RSpec-based testing library like
serverspec has their own special twist on the basic
RSpec expectation form
175
175

Generic Expectation Form
176
describe
‘<entity>’
do

it
‘<description>’
do

expect(thing).to
eq
result

end
end
176

Serverspec Command
177
describe
‘<entity>’
do

it
‘<description>’
do

expect(command).to
eq
result

end
end
thing to expect
is called a command
in serverspec
177

Serverspec commands & matchers
• Serverspec has provides a wide variety of matchers
for each command
• Serverspec commands are well-documented: http://
serverspec.org/resource_types.html
178
178

Port resource/be_listening matcher
180
180

181
Writing your first test
• Let’s create a serverspec test checking to make sure
the clowns web site is active on port 80
• Let’s use the port resource and the be_listening
matcher
181

Spec for clowns
182
require 'serverspec'
include Serverspec::Helper::Exec
describe 'clowns site' do
it 'responds on port 80' do
expect(port 80).to be_listening 'tcp'
end
end
OPEN IN EDITOR:
apache/test/integration/default/serverspec/clown_spec.rb
182

183
kitchen setup
• Before running tests you need to run kitchen

setup
• kitchen
setup loads and configures the file
necessary to run test plugins on the node
• The component that manages Test Kitchen plugins
is called Busser
183

$ bundle exec kitchen setup default-centos-64
kitchen setup
-----> Setting up <default-centos-64>...
-----> Setting up Busser
Creating BUSSER_ROOT in /tmp/busser
Creating busser binstub
Plugin serverspec installed (version 0.2.6)
-----> Running postinstall for serverspec plugin
Finished setting up <default-centos-64> (0m13.03s).
184
184

185
kitchen verify
• The kitchen
verify command will run the tests in
your *_spec.rb files in the test/integration tree
185

$ bundle exec kitchen verify default-centos-64
kitchen verify
-----> Verifying <default-centos-64>...
Removing /tmp/busser/suites/serverspec
Uploading /tmp/busser/suites/serverspec/localhost/clown_spec.rb (mode=0644)
-----> Running serverspec test suite
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/
embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/clown_spec.rb --color --
format documentation
clowns site
responds on port 80
Finished in 0.02071 seconds
1 example, 0 failures
Finished verifying <default-centos-64> (0m1.17s).
186
186

187
Verifying that the tests work
• Did the test actually do anything? Let’s verify this
by changing the port to a known incorrect value.
187

Replace port 80 to 85
188
end
end
OPEN IN EDITOR:
188

This should fail
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/
busser/suites/serverspec/localhost/clown_spec.rb --color --format documentation
clowns site
responds on port 80 (FAILED - 1)
Failures:
1) clowns site responds on port 80
Failure/Error: expect(port 85).to be_listening 'tcp'
netstat -tunl | grep -- :85
expected Port "85" to be listening "tcp"
# /tmp/busser/suites/serverspec/localhost/clown_spec.rb:7:in `block (2 levels) in <top
(required)>'
1 example, 1 failure
...
189
189

This should fail
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/
busser/suites/serverspec/localhost/clown_spec.rb --color --format documentation
clowns site
responds on port 80 (FAILED - 1)
Failures:
1) clowns site responds on port 80
Failure/Error: expect(port 85).to be_listening 'tcp'
netstat -tunl | grep -- :85
expected Port "85" to be listening "tcp"
# /tmp/busser/suites/serverspec/localhost/clown_spec.rb:7:in `block (2 levels) in <top
(required)>'
...
190
WIN FAIL
190

191
Back to success
• Remember to reset the tests back to the original port
value so they succeed again!
191

Reset back to port 80
192
end
end
OPEN IN EDITOR:
192

kitchen verify
-----> Verifying <default-centos-64>...
Removing /tmp/busser/suites/serverspec
Uploading /tmp/busser/suites/serverspec/localhost/clown_spec.rb (mode=0644)
embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/clown_spec.rb --color --
clowns site
responds on port 80
193
193

194
Testing bears
• Let’s add another test to do a similar check for the
bears port
194

Spec for bears
195
describe 'bears site' do
end
end
OPEN IN EDITOR:
apache/test/integration/default/serverspec/bear_spec.rb
195

196
No need to run kitchen setup
• You only need to run kitchen
setup once per
node. (Though it doesn’t hurt to run it more than
once).
196

...
----> Running serverspec test suite
embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/bear_spec.rb /tmp/busser/
suites/serverspec/localhost/clown_spec.rb --color --format documentation
bears site
response on port 81
clowns site
responds on port 80
2 examples, 0 failures
...
197
197

198
Code cleanup
• Common code can be moved to a file called
spec_helper.rb in test/integration/default/
serverspec
198

199
Code cleanup
• Let’s move common code between clowns & bears
to spec_helper.rb
199

spec_helper.rb
200
OPEN IN EDITOR:
apache/test/integration/default/serverspec/spec_helper.rb
200

require spec_helper clowns
201
require 'spec_helper'
end
end
OPEN IN EDITOR:
201

require spec_helper bears
202
end
end
OPEN IN EDITOR:
202

Testing clowns and bears w/spec_helper.rb
...
bears site
response on port 81
clowns site
responds on port 80
...
203
203

Testing clowns and beras
...
bears site
response on port 81
clowns site
responds on port 80
...
204
WIN
204

205
Are the web sites really working?
• While we’ve added checks to verify that the test
node is listening on ports 80 and 81, we haven’t
verified that users see the right content when they
visit these sites.
• Let’s use the command resource with the
return_stdout matcher to do a simple check with
curl to verify that port 80 is clowns and port 81 is
bears.
205

command resource/return_stdout matcher
206
206

Check clown content
207
end
it 'returns clowns in the HTML body' do
expect(command 'curl localhost:80').to
return_stdout(/clowns/)
end
end
OPEN IN EDITOR:
207

Check bear content
209
end
it 'returns bears in the HTML body' do
expect(command 'curl localhost:81').to
return_stdout(/bears/)
end
end
OPEN IN EDITOR:
209

Testing for content
...
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/
suites/serverspec/localhost/bear_spec.rb /tmp/busser/suites/serverspec/localhost/clown_spec.rb --color --
bears site
responds on port 81
returns bears in the HTML body
clowns site
responds on port 80
returns clowns in the HTML body
...
210
WIN
210

211
Detecting the target OS
• Many of the resources require that Serverspec
detect the OS so it can run the correct command for
your platform
expect(package
'httpd').to
be_installed
• You’ll need to add an extra Helper to spec_helper.rb
211

spec_helper.rb
212
include Serverspec::Helper::DetectOS
OPEN IN EDITOR:
apache/test/integration/default/serverspec/spec_helper.rb
212

Check httpd package
213
describe 'server' do
it 'has apache installed' do
expect(package 'httpd').to be_installed
end
end
OPEN IN EDITOR:
apache/test/integration/default/serverspec/default_spec.rb
213

Testing for httpd
...
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/
suites/serverspec/localhost/bear_spec.rb /tmp/busser/suites/serverspec/localhost/clown_spec.rb --color --
bears site
responds on port 81
returns bears in the HTML body
clowns site
responds on port 80
returns clowns in the HTML body
...
214
WIN
214

215
kitchen test
• The kitchen
test command will automate all the
previous actions you’ve learned so far into one command.
It runs the following commands in sequence:
• kitchen
destroy (if necessary)
•kitchen
create
•kitchen
converge
•kitchen
setup
•kitchen
verify
•kitchen
destroy
215

216
kitchen test
• The kitchen
test command is intended to be used
as a final check on a fresh image before committing
changes to source control and/or to be used in a
Continuous Integration environment like Jenkins.
216

$ bundle exec kitchen test default-centos-64
kitchen test
-----> Cleaning up any prior instances of <default-centos-64>
2c46b1a4609dc6a2beaf44e1134638b0a8ac47c9c5a02baee0bdb3df64e7bcdf
2c46b1a4609dc6a2beaf44e1134638b0a8ac47c9c5a02baee0bdb3df64e7bcdf
-----> Testing <default-centos-64>
...
d22a8c4db8505f89f7f7e65bca26492f58d5637f9a88763d5eb919d860dade4e
d22a8c4db8505f89f7f7e65bca26492f58d5637f9a88763d5eb919d860dade4e
Finished testing <default-centos-64> (0m39.78s).
217
217

Where to go next
• Jenkins cookbook is chock full of advanced
Serverspec techniques:
https://github.com/opscode-cookbooks/jenkins
218
218

Where to go next
• jenkins/test/shared/support contains
examples for implementing custom Serverspec
matchers used in jenkins/test/integration:
•describe
jenkins_job('my-‐project')
do

it
{
should
be_a_jenkins_job
}
end
219
219

Where to go next
• test/fixtures contains mini-cookbooks to
exercise resource providers
• Aliased in Berksfile:
•cookbook
'smoke',
path:
'test/fixtures/
cookbooks/smoke'
• Run via Rakefile
220
220

Where to go next
• Uses data/path directive in .kitchen.yml to share
test data between serverspec suites
• Directory specified in data/path is copied to /tmp/
kitchen/data on guest
• Reason for weird require_relative directive in tests
that use custom Serverspec matchers:
require_relative
'../../../kitchen/data/
spec_helper'
221
221

RECAP: Why Test?
• It’s important to find bugs fast
222
222

Better, Faster, Stronger
• Test Kitchen is an invaluable tool for managing
sandbox environments and truly verifying that a
cookbook produces the intended results
• But it does require spinning up an instance and
performing a full Chef converge, which can take a
long time
• Use Test Kitchen judiciously. The other tools can
provide more limited forms of feedback faster.
223
223

v2.0.0
Detect Suspicious Cookbook
Code with Foodcritic
224
224

Feedback on Chef Coding Style
• Foodcritic provides feedback on your Chef coding
style
• It is designed to be used as you are writing Chef
code - how’s that for freaking fast!
• Written by Andrew Crump
http://acrmp.github.com/footcritic
225
225

• Let’s install Foodcritic on your development
workstation so you can give it a spin
• Add Foodcritic to your Gemfile
• Install the app with bundle
install
226
226

gem 'test-kitchen'
gem 'foodcritic', '~> 3.0'
Add foodcritic to Gemfile
227
227

$ sudo yum install -y libxslt-devel libxml2-devel
Install Prerequisites
228
$ sudo apt-get install -y libxslt-dev libxml2-dev
228

$ bundle install
Install Foodcritic
229
229

$ bundle exec foodcritic --version
Verify install
foodcritic 3.0.3
230
230

$ bundle exec foodcritic .
Run Foodcritic on your cookbook
231
FC003: Check whether you are running with chef
server before using server-specific features:
cookbooks/apache/recipes/ip-logger.rb:1
FC008: Generated cookbook metadata needs
updating: cookbooks/apache/metadata.rb:2
FC008: Generated cookbook metadata needs
updating: cookbooks/apache/metadata.rb:3
231

• Foodcritic comes with a set of checks called rules
• Foodcritic rules are documented at http://
acrmp.github.io/foodcritic/
• The default rules are a good start, and you can add
new rules of your own easily
234
234

236
What about the
FC003 & FC008 issues
in the apache cookbook?
236

FC008 - Generated cookbook metadata needs updating
237
237

cookbooks/apache/metadata.rb
239
name 'apache'
maintainer 'YOUR_COMPANY_NAME'
maintainer_email 'YOUR_EMAIL'
license 'All rights reserved'
description 'Installs/Configures apache'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '0.2.0'
239

OPEN IN EDITOR: cookbooks/apache/metadata.rb
name 'apache'
maintainer 'Mischa Taylor'
maintainer_email 'misheska@getchef.com'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '0.2.0'
240
Addressing FC008
240

$ bundle exec foodcritic .
Rerun foodcritic
241
FC003: Check whether you are running
with chef server before using server-
specific features: ./recipes/ip-
logger.rb:1
241

FC003 - Check for chef server before using server-specific features
243
243

244
Ignoring FC003
• Let’s say, for now, we don’t want to fix ip-‐
logger.rb, and we’d like to squelch the FC003
check
244

245
--tags parameter
• The -‐-‐tags
<TAGS> parameter can be used to
specify a list of rules for foodcritic to use
•foodcritic
-‐-‐tags
FC001,FC002,FC008
• The tilde (~) modifier can be used to ignore specific
rules
•foodcritic
-‐-‐tags
~FC003
245

$ bundle exec foodcritic --tags ~FC003 .
Ignore FC003
246
246

$ bundle exec foodcritic --tags ~FC003 .
Ignore FC003
247
WIN
247

248
Custom rules
• Etsy created some custom Foodcritic rules to check
for issues that caused production outages/
performance degradation.
• Good example for how to create your own custom
rules
• Documented here:
https://github.com/etsy/foodcritic-rules
248

Etsy Foodcritic Rules
• ETSY001 - Package or yum_package resource used
with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource deﬁned without conditional or
action :nothing
• ETSY005 - Action :restart sent to a core service
• ETSY006 - Execute resource used to run chef-provided command
• ETSY007 - Package or yum_package resource used to install
core package without speciﬁc version number
249
249

$ git clone https://github.com/etsy/
foodcritic-rules ../../foodcritic/etsy
$ rm -rf ../../foodcritic/etsy/.git
Installing new rules
250
250

251
--include parameter
• The -‐-‐include
<PATH> parameter species
additional paths to load rules (shortened with -I)
251

$ bundle exec foodcritic -t ~FC003 -I ../../
foodcritic .
Including Custom Rules
ETSY005: Action :restart sent to a core
service: ./recipes/default.rb:19
ETSY005: Action :restart sent to a core
service: ./recipes/default.rb:32
ETSY007: Package or yum_package resource used to
install core package without specific version
number: ./recipes/default.rb:10
252
252

253
Editor support
• Many popular editors can be configured to run
Foodcritic inside the editor (including Vim, GNU
Emacs and Sublime Text). So you can get feedback
even faster.
253

v2.0.0
Detect Suspicious Ruby Code
with RuboCop
254
254

RuboCop - Feedback on Ruby Style
• Many people new to Ruby would like some guidance
on how to write idiomatic Ruby
• Get the same kind of feedback for Ruby using
RuboCop that you get for Chef Code using
Foodcritic (Chef code is Ruby)
255
255

RuboCop Author
• Written by Bozhidar Batsov:
https://github.com/bbatsov/rubocop
256
256

• Follows community-driven style guide:
https://github.com/bbatsov/ruby-style-guide
• Looks at cookbooks for Ruby best practices, not the
Chef DSL - that’s Foodcritic
257
257

• Let’s install RuboCop on your development
• Add RuboCop to your Gemfile
install
258
258

gem 'test-kitchen'
gem 'rubocop', '~> 0.20'
Add rubocop to Gemfile
259
259

$ bundle install
Install RuboCop
260
260

$ bundle exec rubocop --version
Verify Install
0.20.1
261
261

Running RuboCop
• Just run the rubocop command - it recursively
checks all the *.rb files in all subdirectories
underneath the current directory (excluding
vendor/)
262
262

$ bundle exec rubocop
Run RuboCop on your cookbook
attributes/default.rb:3:19: C: Prefer single-quoted strings when you don't
need string interpolation or special symbols.
default["apache"]["sites"]["bears"] = { "port" => 81 }
^^^^^^^
^^^^^^^
^^^^^^
7 files inspected, 52 offenses detected
263
263

rubocop-todo.yml via --auto-gen-config
• rubocop-todo.yml will help generate TODOs for each
item on the offense list
• It also shows you what config setting can be used to
mask each offense, which we’ll need to do for some
of these, because Chef code conventions vary
slightly from the Rubocop community standards
264
264

$ bundle exec rubocop --auto-gen-config
Generate rubocop-todo.yml
attributes/default.rb:3:28: C: Prefer single-quoted strings when you
don't need string interpolation or special symbols.
^^^^^^^
attributes/default.rb:3:41: C: Prefer single-quoted strings when you
don't need string interpolation or special symbols.
^^^^^^
Created rubocop-todo.yml.
Run `rubocop --config rubocop-todo.yml`, or
add inherit_from: rubocop-todo.yml in a .rubocop.yml file.
265
265

.rubocop.yml Configures RuboCop
• .rubocop.yml can be used to configure RuboCop
(similar to .kitchen.yml in Test Kitchen)
• We’ll add a settings to ignore things, similar to what
we did for Foodcritic, that don’t make as much sense
for Chef.
• Settings are documented in the RuboCop README:
https://github.com/bbatsov/rubocop/blob/master/
README.md
• Cop is the RuboCop equivalent of a rule
266
266

OPEN IN EDITOR: cookbooks/apache/.rubocop.yml
inherit_from:
rubocop-‐todo.yml
Include rubocop-todo.yml
267
267

$ bundle exec rubocop
Run RuboCop on your cookbook
Inspecting 7 files
.......
7 files inspected, no offenses detected
268
268

Easy peasy
• Wow, now more offenses...not really
269
269

Match Chef community standards
• First, we’ll move some of the Cops from rubocop-
todo.yml to .rubocop.yml for things that match Chef
community standards (as opposed to the Ruby
community standards)
270
270

Missing utf-8 encoding comment
271
271

inherit_from:
rubocop-‐todo.yml
Encoding:

Enabled:
false
Chef does not (yet) support encoding comment
272
272

inherit_from:
rubocop-‐todo.yml
Encoding:

Enabled:
false
LineLength:

Max:
200
Relax line limit
274
274

Use the new Ruby 1.9 hash syntax
275
275

inherit_from:
rubocop-‐todo.yml
Encoding:

Enabled:
false
LineLength:

Max:
200
HashSyntax:

EnforcedStyle:
hash_rockets
Some cookbooks try to be Ruby 1.8 compatible
276
276

Prefer single-quoted strings
277
277

inherit_from:
rubocop-‐todo.yml
Encoding:

Enabled:
false
LineLength:

Max:
200
HashSyntax:

EnforcedStyle:
hash_rockets
StringLiterals:

Enabled:
false
Conflicts w/decision to relax FC002
278
278

$ bundle exec rubocop --auto-gen-config
Regenerate rubocop-todo.yml
metadata.rb:2:11: C: Put one space between the method name and the first argument.
maintainer 'Mischa Taylor'
^^^^^^^
^^^^^^^^^^
^^^^^^
version '0.2.0'
^^^^^^^^^^
279
279

Rubocop Workflow
• Uncomment lines in rubocop-todo.yml
• Fix offenses
• git commit
• Repeat
280
280

Only really critical issue
281
281

Trailing whitespace & Git
• Whitespace differences make diffs longer and
diverts focus from more important changes
• Even with Git, trailing whitespace can make merge
conflicts more difficult to resolve
282
282

283
Editor support
• Many popular editors can be configured to run
RuboCop inside the editor (including Vim, GNU
Emacs and Sublime Text). So you can get feedback
even faster.
• RuboCop includes great docs on editor configuration
(which work for Foodcritic as well):
https://github.com/bbatsov/rubocop#editor-
integration
283

v2.0.0
Runnable documentation with
ChefSpec
284
284

RECAP: Why Not Begin With Testing?
• Finding a bug in something that you can’t execute
is freaking hard!
• While fixing bugs before writing code is cheap,
finding them is expensive
285
285

What is ChefSpec?
• ChefSpec helps produce runnable documentation.
Its primary purpose is to help document and
organize your code.
• As a side effect, you’ll end up with a set of tests
which can also be used to uncover bugs when
changes are made.
• Plus, your cookbook code will be improved when it
is guided by tests.
286
286

ChefSpec Authors
• Written by Andrew Crump and Seth Vargo
287
287

ChefSpec - Runnable Documentation
• Let’s install ChefSpec on your development
• Add ChefSpec to your Gemfile
install
288
288

OPEN IN EDITOR: chef-fundamentals-repo/Gemfile
gem 'test-kitchen'
gem 'chefspec', '~> 3.4'
Gemfile
289
289

$ bundle install
Install ChefSpec
290
290

ChefSpec builds on RSpec
• ChefSpec uses the RSpec description form to create
runnable documentation (in a similar vein to
serverspec)
291
291

Documentation form
292
describe
‘<recipe_name>’
do

<perform
in-‐memory
Chef
run>

<descriptions
here>
end
292

Documentation example
293
describe
‘apache::default’
do

...
end
293

In-Memory Chef Run Form
294
require
‘chefspec’
describe
‘<recipe_name>’
do

chef_run
=
ChefSpec::Runner.new.converge(<recipe_name>)

<descriptions
here>
end
294

In-Memory Chef Run Example
295
require
‘chefspec’
describe
'apache::default'
do

chef_run
=
ChefSpec::Runner.new.converge('apache::default')

<descriptions
here>
end
295

Expectation form
296
describe
‘<recipe_name>’
do

<perform
in-‐memory
Chef
run>

it
‘<description>’
do

expect(<chef_run>).to
eq
result

end
end
296

Expectation form
297
describe
‘<recipe_name>’
do

<perform
in-‐memory
Chef
run>

it
‘<description>’
do

expect(<chef_run>).to
eq
result

end
end
Matcher
297

Expectation Example
298
require
‘chefspec’
describe
'apache::default'
do

chef_run
=
ChefSpec::Runner.new.converge('apache::default')

it
‘installs
apache2’
do

expect(chef_run).to
install_package(‘httpd’)

end
end
298

Runnable Documentation
• expect statement does not actually perform the
httpd package installation
• It just verifies the cookbook syntax that it instructs
Chef to install the package
• Good enough for well-tested primitives like the
package resource
299
299

300
Let’s write some ChefSpec
tests!
300

301
Default location for tests
• By default, ChefSpec will look in the spec/directory
for ChefSpec test-related files
301

302
.
!"" .bundle/
!"" .kitchen/
!"" .kitchen.yml
!"" CHANGELOG.md
!"" Gemfile
!"" Gemfile.lock
!"" README.md
!"" attributes/
!"" files/
!"" metadata.rb
!"" recipes/
!"" spec/
!"" templates/
!"" test/
# $"" integration/
$"" vendor/
$"" bundle/
302

$ mkdir spec
Create spec directory in cookbooks/apache
303
> mkdir spec
303

304
*_spec.rb files
• By convention, ChefSpec expects files with tests to
end in _spec.rb
304

ChefSpec Matcher Documentation
305
http://rubydoc.info/github/acrmp/chefspec/frames
305

Each API has an example
307
307

Let’s use the install_package matcher
308
308

install_package Example
309
309

OPEN IN EDITOR: spec/default_spec.rb
require 'chefspec'
describe 'apache::default' do
chef_run = ChefSpec::Runner.new.converge('apache::default')
it 'installs apache2' do
expect(chef_run).to install_package('httpd')
end
end
Test apache::default recipe
310
310

311
Rspec runs ChefSpec
• There’s no separate chefspec command.
• Just run rspec to run ChefSpec tests.
311

$ bundle exec rspec --color
Run ChefSpec on your cookbok
.
312
312

require 'chefspec'
expect(chef_run).to install_package('badhttpd')
end
end
Did it really check anything?
313
313

$ bundle exec rspec
F
Failures:
1) apache::default installs apache2
Failure/Error: expect(chef_run).to install_package('badhttpd')
expected "package[badhttpd]" with action :install to be in Chef run. Other
package resources:
package[httpd]
# ./spec/default_spec.rb:7:in `block (2 levels) in <top (required)>'
314
314

require 'chefspec'
end
end
Restore back to working
315
315

.
316
316

Lazy evaluation with let
317
require 'chefspec'
let(:chef_run)
{ ChefSpec::Runner.new.converge(described_recipe) }
end
end
Lazy evaluation
317

318
described_recipe
• let blocks aren’t evaluated until the first time they
are called
• Also allows ChefSpec to run the described_recipe
macro to evaluate the recipe name
318

require 'chefspec'
let(:chef_run)
end
end
Lazy evaluation
319
319

.
320
320

321
ChefSpec Resource Report
• ChefSpec Resource Report can help guide writing
your tests
321

require 'chefspec'
at_exit { ChefSpec::Coverage.report! }
let (:chef_run)
end
end
Adding resource report
322
322

.
ChefSpec Coverage report generated...
Total Resources: 9
Touched Resources: 1
Touch Coverage: 11.11%
Untouched Resources:
service[httpd] /recipes/default.rb:14
execute[mv /etc/httpd/conf.d/welcome.conf /etc/httpd/conf.d/welcome.conf.disabled] /recipes/default.rb:19
template[/etc/httpd/conf.d/clowns.conf] /recipes/default.rb:32
directory[/srv/apache/clowns] /recipes/default.rb:43
template[/srv/apache/clowns/index.html] /recipes/default.rb:49
template[/etc/httpd/conf.d/bears.conf] /recipes/default.rb:32
directory[/srv/apache/bears] /recipes/default.rb:43
template[/srv/apache/bears/index.html] /recipes/default.rb:49
323
323

324
create_file matcher
• Let’s verify that the clowns.conf file gets created
with the create_file matcher
324

require 'chefspec'
let(:chef_run)
...
it 'creates clowns.conf' do
expect(chef_run).to
create_file('/etc/httpd/conf.d/clowns.conf')
end
end
Checking clowns.conf files
326
326

F
Failures:
1) apache::default creates clowns.conf
Failure/Error: expect(chef_run).to create_file('/srv/apache/clowns')
expected "file[/srv/apache/clowns]" with action :create to be in Chef run. Other file resources:
2 examples, 1 failure
Failed examples:
rspec ./spec/default_spec.rb:12 # apache::default creates clowns.conf
327
327

F
Failures:
1) apache::default creates clowns.conf
Failure/Error: expect(chef_run).to create_file('/srv/apache/clowns')
expected "file[/srv/apache/clowns]" with action :create to be in Chef run. Other file resources:
2 examples, 1 failure
Failed examples:
rspec ./spec/default_spec.rb:12 # apache::default creates clowns.conf
328
FAIL
328

329
ChefSpec == Runnable Documentation
• Remember: ChefSpec is just runnable
documentation
• It isn’t actually performing a Chef run to verify that
clowns.conf was created
• Instead it is just verifying that you told Chef to
create the clowns.conf via the file resource, which
you never did - you used the template resource
329

require 'chefspec'
let(:chef_run)
...
expect(chef_run).to
create_template('/etc/httpd/conf.d/clowns.conf')
end
end
Checking clowns.conf file
331
331

.
332
332

.
333
WIN
333

334
spec_helper.rb
• Similar to Serverspec, common code can be moved
to a file called spec_helper.rb with ChefSpec
334

OPEN IN EDITOR: spec/spec_helper.rb
require 'chefspec'
335
335

336
RSpec recurses through spec/*
• RSpec recurses through the spec/ subtree, looking
for tests, so you can create any directory structure
you like underneath
• We’ll move default_spec.rb to spec/recipes
336

$ mkdir spec/recipes
$ mv spec/default_spec.rb spec/recipes
Move default_spec.rb
337
337

OPEN IN EDITOR: spec/recipes/default_spec.rb
let (:chef_run)
end
expect(chef_run).to
create_template('/etc/httpd/conf.d/clowns.conf')
end
end
338
338

.
339
339

.
340
WIN
340

341
Where to go next
• There’s a lot of ChefSpec written for the community
cookbooks. Check out the spec/ directory your
favorites.
341

v2.0.0
Continuous Validation with
Guard
342
342

What is Guard?
• A tool that monitors for filesystem changes and
performs actions (like launching rake tasks)
• Written by Thibaud Guillaume-Gentil
343
343

Guard install
• Let’s install Guard on your development
• Add guard to your Gemfile
install
344
344

gem 'test-kitchen'
gem 'guard', '~> 2.6'
Gemfile
345
345

$ bundle install
Install Guard
346
346

gem 'test-kitchen'
gem 'guard', '~> 2.6'
gem 'guard-rubocop', '~> 1.1'
Gemfile
347
347

$ bundle install
Install guard-rubocop
348
348

$ bundle exec guard init
Create Guardfile
02:39:58 - INFO - Writing new Guardfile to /home/vagrant/
chef-fundamentals-repo/cookbooks/apache/Guardfile
02:45:32 - INFO - rubocop guard added to Guardfile, feel
free to edit it
349
349

cookbooks/apache/Guardfile
350
# A sample Guardfile
# More info at https://github.com/guard/guard#readme
guard :rubocop do
watch(%r{.+.rb$})
watch(%r{(?:.+/)?.rubocop.yml$}) { |m| File.dirname(m[0]) }
end
350

$ bundle exec guard
Run Guard
02:48:54 - INFO - Guard is now watching at '/home/vagrant/
chef-fundamentals-repo/cookbooks/apache'
[1] guard(main)>
351
351

$ cd $HOME/chef-fundamentals-repo/cookbooks/apache
In another session
354
And edit some .rb ﬁle - upon save, rubocop is
launched!
354

Stopping guard
[1] guard(main)> quit
19:23:42 - INFO - Bye bye...
355
355

356
Where to go next
Michael Goetz blog posts:
https://micgo.net/check-yo-self-before-you-wreck-yo-self-with-
foodcritic-chefspec/
Foodcritic and Guard:
Serverspec and Guard:
https://micgo.net/serverspec-guard-and-test-kitchen-testing-
servers-like-a-boss/
356

357
Where to go next
Michael Goetz blog posts:
ChefSpec and Guard:
https://micgo.net/continuous-chefspec-validation-with-guard/
357

v2.0.0
Repeating Test Steps with
Rake
358
358

What is Rake?
• Rake includes a language for expressing the
command line steps needed to create an app
• Perfect for capturing all the commands you’ve
learned in this class so others can run them easily,
or in your continuous integration system (Jenkins,
Bamboo, TeamCity, etc.)
359
359

Rake Author
• Written by Jim Weirich:
http://rake.rubyforge.org/
360
360

Rake - Repeatable Test Commands
• Let’s install Rake on your development workstation
so you can give it a spin
• Add rake to your Gemfile
install
361
361

OPEN IN EDITOR: chef-fundamentals-repo/Gemfile
gem 'test-kitchen'
gem 'rake'
gem 'guard', '~> 2.6'
gem 'guard-rubocop', '~> 1.1'
Gemfile
362
362

$ bundle install
Install Rake
363
363

Rake - Repeatable Tasks
• Task - expresses command line actions to perform
364
364

Rake Task Form
365
task :<task_name> do
<action>
<action>
end
365

Rake - Repeatable Tasks
• Configuration file for rake is a Rakefile
366
366

Rake - Actions
• Actions are expressed in Ruby syntax
• sh
“<command>”
runs a shell command:
sh “bundle
exec
rspec”
367
367

OPEN IN EDITOR: cookbooks/apache/Rakefile
task :rubocop do
sh 'bundle exec rubocop'
end
Rubocop Task
368
368

$ bundle exec rake rubocop
Execute Rake Task
Inspecting 9 files
.......
369
369

Task Description
• Every task should have a description which
documents what the task does
• rake
-‐-‐tasks prints out tasks with descriptions
370
370

desc 'Run Ruby style checks with Rubocop'
task :rubocop do
end
Rubocop Task
371
371

$ bundle exec rake --tasks
Execute Rake Task
rake rubocop # Run Ruby style checks with Rubocop
372
372

Adding Foodcritic
• Let’s add a task for Foocritic next
373
373

task :rubocop do
end
desc 'Run Chef style checks with Foodcritic'
task :foodcritic do
sh 'bundle exec foodcritic -t ~FC003 .'
end
Foodcritic Task
374
374

$ bundle exec rake foodcritic
Execute Rake Task
bundle exec foodcritic -t ~FC003 .
FC011: Missing README in markdown format: spec/README.md:1
FC031: Cookbook without metadata file: spec/metadata.rb:1
FC045: Consider setting cookbook name in metadata: spec/
metadata.rb:1
375
375

$ bundle exec rake foodcritic
Execute Rake Task
bundle exec foodcritic -t ~FC003 .
FC011: Missing README in markdown format: spec/README.md:1
FC031: Cookbook without metadata file: spec/metadata.rb:1
FC045: Consider setting cookbook name in metadata: spec/
metadata.rb:1
376
WAT?
376

Foodcritic 3.0.3 issue
• Foodcritic is checking spec/ subtree when it
shouldn’t
• Does not expose command line option to exclude
directories:
https://github.com/acrmp/foodcritic/issues/148
• When fixed, this should work:
bundle
exec
foodcritic
-‐X
spec
-‐t
~FC003
.
377
377

task :rubocop do
end
require 'foodcritic'
FoodCritic::Rake::LintTask.new(:foodcritic) do |t|
t.options = {
tags: ['~FC003'],
excludes: ['test', 'spec', 'features']
}
end
Workaround - Use Ruby
378
378

Default task
• Rake supports a special task name called default
• default runs when no parameters are supplied to
rake
• default (as well as any other task) can point to a
list of other task names to execute
task
:default
=>
[:foodcritic]
379
379

task :default => [:rubocop, :foodcritic]
task :rubocop do
end
require 'foodcritic'
FoodCritic::Rake::LintTask.new(:foodcritic) do |t|
t.options = {
tags: ['~FC003'],
excludes: ['test', 'spec', 'features' ]
}
end
Foodcritic Task
380
380

$ bundle exec rake
Execute Rake Task
bundle exec rubocop
Inspecting 9 files
.......
381
381

382
Where to go next
Rake Boot Camp
http://cloud.github.com/downloads/jimweirich/RakePresentations/PowerRake.key.pdf
http://www.confreaks.com/videos/899-railsconf2012-basic-rake
Go to http://confreaks.com
Search for “Basic Rake”
382

383
Where to go next
Rake Tasks can have tests
http://blog.jayﬁelds.com/2006/11/ruby-testing-rake-tasks.html
383

What is Jenkins?
• Jenkins is a commonly used, open source
continuous integration system used to build early
and often
• Written by Kohsuke Kawaguchi
385
385

$ sudo yum install -y libxslt-devel libxml2-devel
Install Prerequisites
386
$ sudo apt-get install -y libxslt-dev libxml2-dev
386

$ cd $HOME
Home directory - great place for source
387
$ cd %USERPROFILE%
387

Jenkins cookbook
• Jenkins cookbook - https://github.com/opscode-
cookbooks/jenkins
• Jenkins cookbook is library cookbook
388
388

Library cookbook
• Popularized by Bryan Berry’s blog post How to Write
Resuable Chef Cookbooks, Gangnam Style
389
389

Jenkins wrapper cookbook
• Start of our wrapper cookbook:
https://github.com/misheska/test-class-jenkins
390
390

$ git clone https://github.com/misheska/test-class-jenkins
Grab test-class-jenkins from Github
Cloning into 'test-class-jenkins'...
remote: Counting objects: 19, done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 19 (delta 0), reused 19 (delta 0)
Unpacking objects: 100% (19/19), done.
391
391

$ cd test-class-jenkins
test-class-jenkins
392
392

$ bundle install --path vendor/bundle
Install gems vendored
Fetching gem metadata from https://rubygems.org/.......
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Installing rake (10.2.2)
Installing addressable (2.3.6)
Installing ast (1.1.0)
...
Installing powerpack (0.0.9)
Installing rainbow (2.0.0)
Installing ruby-progressbar (1.4.2)
Installing rubocop (0.20.1)
Using bundler (1.5.3)
Your bundle is complete!
It was installed into ./vendor/bundle
393
393

test-class-jenkins/recipes/default.rb
396
include_recipe 'jenkins::java'
include_recipe 'jenkins::master'
396

test-class-jenkins/attributes/default.rb
397
include_attribute 'jenkins::master'
397

test-class-jenkins/Gemfile
398
gem 'test-kitchen'
gem 'kitchen-docker'
gem 'rake'
gem 'berkshelf', '~> 3.0.0.rc'
398

test-class-jenkins/.kitchen.yml
399
---
driver:
name: docker
provisioner:
name: chef_solo
platforms:
- name: centos-6.4
driver_config:
forward:
- 8080:8080
suites:
- name: default
run_list:
- recipe[test-class-jenkins::default]
399

$ bundle exec kitchen converge
Perform Chef run of Jenkins wrapper
Step 0 : FROM centos:6.4
...
----> Converging <default-centos-64>...
Preparing files for transfer
Resolving cookbook dependencies with Berkshelf
3.0.0.rc1...
...
400
400

http://<EXTERNAL_ADDRESS>:8080
401
401

jenkins_plugin resource
404
404

OPEN IN EDITOR: test-class-jenkins/recipes/default.rb
include_recipe 'jenkins::java'
include_recipe 'jenkins::master'
# Install version 1.13 of the greenballs plugin
jenkins_plugin 'greenballs' do
version '1.13'
end
Install greenballs plugin
405
405

...
3.0.0.rc1...
...
406
406

jenkins_script resource
408
408

OPEN IN EDITOR: test-class-jenkins/recipes/default.rb
...
jenkins_script
'configure-‐mailer'
do

command
<<-‐GROOVY.gsub(/^
{4}/,
'')

jenkins
=
jenkins.model.Jenkins.getInstance()

mailer
=
jenkins.getDescriptorByType(hudson.tasks.Mailer.DescriptorImpl)

mailer.setSmtpHost("smtp.gmail.com")

mailer.setUseSsl(true)

mailer.setSmtpAuth("smtp",
"password")

mailer.setReplyToAddress("jenkins@my.com")

mailer.save()

GROOVY
end
Configure E-mail Notification
409
409

...
3.0.0.rc1...
...
411
411

Testing Your Automation Code (Vagrant Version)

Recommended

More Related Content

What's hot (20)

Similar to Testing Your Automation Code (Vagrant Version) (20)

Recently uploaded (20)

Testing Your Automation Code (Vagrant Version)