How I hack on puppet modules

How I hack Puppet
and made my life soo much easier
by
Kris Buytaert

Kris Buytaert
● I used to be a Dev,
● Then Became an Op
● Chief Trolling Officer and Open Source
Consultant @inuits.eu
● Everything is an effing DNS Problem
● Building Clouds since before the bookstore
● Some books, some papers, some blogs
● Evangelizing devops
● But mostly, trying to be good at my job

Today

• Vagrant
• Style
• Testing Puppet
• Modules vs Modules
• Jenkins
• Demo ?

Typical Environments
For Devs For Ops
● Scrum ● Kanban
● Version Control ● Version Control
● Automated Build ● Automated Build
● Bugtracking ● Bugtracking
● Continous integration ● Continous integration
● Integrated testing ● Integrated testing
● Automated ● Automated
deployment deployment

Everybody is a developer
● Yes we write code also
● httpd.conf, squid.conf, my.cnf
● Just crappy languages :)
● shell, perl, ruby, python, puppet
● So those rules apply for Everyone

Vagrant
● Abstraction layer for
VirtualBox
● Integrates well with
Puppet/Chef
● Project =
● Vagrantfile
● Manifests /
Cookbooks
● Portable, Small ,
Versionable

Veewee
● Vabgrantbox.es
● Use veewee to build your boxen
•gem install veewee
•veewee templates
•veewee init natty ubuntu-11.04-server-amd64
•vagrant basebox build natty
•vagrant box add 'natty' 'natty.box'

A Vagrant project

[sdog@mine vagrant-graphite]$ ls
manifests modules README TODO Vagrantfile
[sdog@mine vagrant-graphite]$ tree -dL 2
.
├── manifests
│ └── hosts
└── modules
├── apache
├── collectd
├── graphite
├── jmxtrans
├── logster
├── statsd
└── tattle
10 directories

A Vagrantfile
Vagrant::Config.run do |config|
# All Vagrant configuration is done here. The most common configuration
# options are documented and commented below. For a complete reference,
# please see the online documentation at vagrantup.com.

config.vm.define :mongo1 do |mongo1_config|
mongo1_config.ssh.max_tries = 100
mongo1_config.vm.box = "MyCentOS2"
mongo1_config.vm.network("192.168.99.101")
mongo1_config.vm.host_name = "mongo1"
mongo1_config.vm.provision :puppet do |mongo1_puppet|
mongo1_puppet.pp_path = "/tmp/vagrant-puppet"
mongo1_puppet.manifests_path = "manifests"
mongo1_puppet.module_path = "modules"
mongo1_puppet.manifest_file = "site.pp"
end
end
config.vm.define :mongo2 do |mongo2_config|
mongo2_config.ssh.max_tries = 100
mongo2_config.vm.box = "MyCentOS2"
mongo2_config.vm.network("192.168.99.102")
mongo2_config.vm.host_name = "mongo2"
mongo2_config.vm.provision :puppet do |mongo2_puppet|
mongo2_puppet.pp_path = "/tmp/vagrant-puppet"
mongo2_puppet.manifests_path = "manifests"
mongo2_puppet.module_path = "modules"
mongo2_puppet.manifest_file = "site.pp"
end
end

Vagrant Rocks
● Vagrant init
● Vagrant up
● Vagrant provision
● Vagrant down
● Vagrant destroy

Booting Up
[sdog@stillmine vagrant-graphite]$ vagrant up
[default] VM already created. Booting if its not already running...
[default] Preparing host only network...
[default] Clearing any previously set forwarded ports...
[default] Forwarding ports...
[default] -- carbon: 2003 => 2021 (adapter 1)
[default] -- http2: 8080 => 50051 (adapter 1)
[default] -- http: 80 => 50050 (adapter 1)
[default] -- ssh: 22 => 2222 (adapter 1)
[default] Cleaning previously set shared folders...
[default] Creating shared folders metadata...
[default] Running any VM customizations...
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.
[default] VM booted and ready for use!
[default] Enabling host only network...
[default] Setting host name...
[default] Mounting shared folders...
[default] -- v-root: /vagrant
[default] -- manifests: /tmp/vagrant-puppet/manifests
[default] -- v-pp-m0: /tmp/vagrant-puppet/modules-0
[default] Running provisioner: Vagrant::Provisioners::Puppet...
[default] Running Puppet with site.pp...
[default] notice: /Stage[main]/Default-repo/Yumrepo[epel]/descr: descr changed 'Extra Packages for Enterprise Linux 6.x' to 'Extra Packages for Enterpris
'
[default]
[default] notice: /Stage[main]/Up-graph::Document-throughput/File[/usr/local/up-graph/document-throughput/graph.sh]/content: content changed '{md5}
[default]
[default] notice: /Stage[main]/Collectd/Service[collectd]/ensure: ensure changed 'stopped' to 'running'
[default]
[default] notice: Finished catalog run in 12.45 seconds
[default]

Failed Provisioning
[sdog@stillmine vagrant-graphite]$ vagrant provision
[default] Running provisioner: Vagrant::Provisioners::Puppet...
[default] Running Puppet with site.pp...
[default] Could not parse for environment production: Syntax error at 'import'; expected '}' at /tmp/vagrant-puppet/manifests/site.pp:5 on node graphite.lan
[default]
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

cd /tmp/vagrant-puppet/manifests
puppet apply --modulepath '/tmp/vagrant-puppet/modules-0' site.pp

The output of the command prior to failing is outputted below:

[no output]

Vagrant Tips

Set up a puppetmaster node
Stored configs
Hiera
Mount Symlink /etc/puppet dirs
/vagrant is your host folder

Handover to devs
● Gitrepo
•Vagrantfile
•A Box definition
•Puppet Manifests
•Puppet modules
•Their code
=> A reproducable environment easy to
migrate to test/uat/prod.

Style
● I never cared
● Jan Coworkers complained
● http://docs.puppetlabs.com/guides/style_guide

node 'nova' {
Before
yumrepo {

'epel':
baseurl => $operatingsystemrelease ? {
'6.0' => "http://mirror.eurid.eu/epel/6/$hardwaremodel/",
'*' => "http://mirror.eurid.eu/epel/5/$hardwaremodel/",
},
descr => $operatingsystemrelease ? {
'6.0' => 'Extra Packages for Enterprise Linux 6.x
',
'*' => 'Extra Packages for Enterprise Linux 5.x',
},
gpgcheck => 0,
enabled => 1;

'openstack':
baseurl => "http://yum.griddynamics.net/yum/diablo-centos",
descr => "openstack diablo at griddynamics",
gpgcheck => 0,
enabled => 1;

}

service { "iptables": ensure => "stopped", enable => "false";
}

package {
"centos-release-cr":
ensure => "present";
"openstack-nova":
}

node 'nova' {
After
yumrepo {
'epel':
baseurl => $operatingsystemrelease ? {
'6.0' => "http://mirror.eurid.eu/epel/6/$hardwaremodel/",
'*' => "http://mirror.eurid.eu/epel/5/$hardwaremodel/",
},
descr => $operatingsystemrelease ? {
'6.0' => 'Extra Packages for Enterprise Linux 6.x',
'*' => 'Extra Packages for Enterprise Linux 5.x',
},
gpgcheck => 0,
enabled => 1;

'openstack':
baseurl => "http://yum.griddynamics.net/yum/diablo-centos",
descr => "openstack diablo at griddynamics",
gpgcheck => 0,
enabled => 1;
}

service {
"iptables":
ensure => "stopped",
enable => "false";
}

package {
"centos-release-cr":
"openstack-nova":
}

Puppet Lint
puppet-lint nova.pp
WARNING: double quoted string containing no variables on line 18
WARNING: => on line 47 isn't aligned with the previous line

Note: Buy Tim Sharpe more beer ...

Testing Puppet
● puppet --parseonly test.pp
● puppet parser validate test.pp
● erb -x -T '-' foreman-vhost.conf.erb | ruby -c
● Cucumber-puppet
● rspec-puppet

Editor plugins
● Vim : https://github.com/rodjek/vim-puppet
● Emacs etc ..

Puppet-module
● Generates a clean structure
● and unused code
● Does stuff with forge (Does anyone actually use forge ? )
● + shell script also create class stubs

Classes vs Modules
● Module :
● Abstract definition on configuring a
service
● Class :
● Specific implementation of your use case
of such a module
•e.g usernames / passwords / hosts do not
belong in modules

Services and Modules
Not in my cluster service {

please ! 'blah':

Ensure => running,

}

class autofs::disable inherits autofs {

Service['autofs'] {

ensure => 'stopped' ,

enable => false,

}

}

GitHub

The Nr 1 place for your modules
(Till forge.puppetlabs.org is revamped)

A github project

[sdog@mine vagrant-graphite]$ ls
manifests modules README TODO Vagrantfile
[sdog@mine vagrant-graphite]$ tree -dL 2
.
├── manifests
│ └── hosts
└── modules
├── apache
├── collectd
├── graphite
├── jmxtrans
├── logster
├── statsd
└── tattle
10 directories

Submodules
git submodule

-d132d2ba4b8c4848acee3ee8e6144b68e86bedc3 modules/apache

6a44fb1179e23ff42f476233d9c65b545be76470 modules/collectd
(heads/master)

+24560116dd498033d81a444f5e326ba9b8b7aefc modules/graphite
(heads/master)

9b5c07afcf4e0b18f48346093053a7d46222a19a modules/jmxtrans
(heads/master)

1a200f9dfae4f346b7f10f7b8f33076d99670ef8 modules/logster (heads/master)

-7f801701898dec12b578aafa35f7df8fa64f2c05 modules/statsd

2d506cac88a731d142b434a99646926c015af6a3 modules/tattle (heads/master)

Git Flow
● git flow feature start blah
● git flow feature finish blah

● git flow release start blah
● git flow release finish blah

Jenkins
● Pull git
● “Build”
•puppet-syntax-
check.sh
•puppet-lint-check.sh
•Generate-puppet-
doc.sh
● Package & deploy

Demo
$git clone git@github.com:KrisBuytaert/vagrant-
graphite.git
$cd vagrant-graphite
$git submodule init
$git submodule update
There's a submodule in a submodule !

$vagrant up
$vagrant provision
$vagrant provision (or fix your ordering)

My code sucked, but I try to make it suck less
step by step,
patches welcome

Homework:
● puppet-rspec
● cucumber-puppet
● guard
● jenkins vagrant plugin

Contact
Kris Buytaert
Kris.Buytaert@inuits.eu

Further Reading
@krisbuytaert
http://krisbuytaert.be/blog/
http://www.inuits.eu/

Inuits
Duboistraat 50
2060 Antwerpen
Belgium
891.514.231

+32 475 961221

How I hack on puppet modules

Recommended

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to How I hack on puppet modules (20)

More from Kris Buytaert (20)

Recently uploaded (20)

How I hack on puppet modules