Drupal Continuous Integration with Jenkins - The Basics

Drupal Continuous Integration
With Jenkins

Classic Graphics – Charlotte, NC

By: John W Smith
License: Creative Commons CC0

Description / Purpose

1. In the following slides, we will build an integrated Jenkins CI
environment that can be used for testing of the Drupal CMS
system and modules.
2. Since we have not fully integrated Jenkins into our development
environment, we’ll explain here our current implementation and
will continue to add to this document as we improve upon it in the
future.
3. The purpose of this document is to provide the procedures and
criteria to install, configure and verify Drupal code commits via a
Jenkins Continuous Integration Server environment. Depending
on the complexity of your environment, and the amount of
integration needed for existing infrastructure, you should be able
to have a basic Jenkins system capable of running Drupal
automated testing within a few hours.

Current Setup

1. As of now, we have a Jenkins server setup that performs
a “Build” in response to a “Push” to our Git repository.
2. Immediately following the “Push”, our post-receive script
accesses a URL on the Jenkins server, passing it the name
of the Git repo, the branch that was updated and the
Key/Token for authentication.
3. With this information, Jenkins clones the repository and
then kicks off a BASH script that switches the cloned
repo to the “commit” branch and then utilizes Drush to
install a new site and perform automated SimpleTest
testing of the committed code.

Future Plans

1. Currently, we have plans to email our developers if any of
their committed code causes a build to fail, email
successes to the deployment team, etc.
2. Integration with a headless Selenium Web Based testing
server setup.

The percentage of our current test coverage is very limited at
best. This is a far cry from test-driven development and total
Continuous Integration, but it's a start.

Disclaimer

This document does NOT go into the complexities of
securing your system from internal or external attackers,
please ensure your system is secure whether or not it is
accessible from the internet.
Also, please consult with your organizations security
and system administrators before making any changes to
existing systems configuration or authentication models,
and before introducing new systems into development and
/ or production server environments.

System Requirements

2GHz+ Multi-Core CPU
4GB+ RAM
250GB or more depending on the number of concurrent builds,
build history, artifacts, etc that you will be wanting to keep
around for archival / review purposes.
Software needed:
 Jenkins 1.xxx (I used 1.433.x)
 Java JDK/JRE 1.x
 MySQL 5.x
 PHP 5.x
 Drush (latest version)
 Drupal 6.x Core SimpleTest patch file (included with
SimpleTest Drupal Module)

References

1. The following references have direct applicability to the system.
Ubuntu Server Community Documentation.
(https://help.ubuntu.com/10.04/serverguide/C/index.html)
Jenkins Continuous Server Documentation.
(https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins)
Getting Hudson (Jenkins) Up and Running with Drupal
(http://thinkshout.com/blog/2010/09/sean/beginners-guide-using-hudson-
continuous-integration-drupal)
2. Points of Contact
The Points of Contact (POC) for this documentation are:
 Technical Lead: David Norman
(davidn@knowclassic.com)
 Project Lead: John W Smith
(johns@knowclassic.com) or (JSmith@i1Technologies.com)

Scope

This document will cover the installation of a basic
Ubuntu 10.04 LTS Server and other software associated
with running a Jenkins CI environment capable of
testing Drupal source code commits. We assume for
this tutorial / documents purpose, that the server is
running a local MySQL Server and ClamAV daemons.

Welcome!

Jenkins is here to help. Specifically, Jenkins is an "extensible continuous integration (CI)
server". From a 10,000ft view, Jenkins is a tool for offloading the deployment and
automated testing of a software application. You write your code, push it into version
control, and Jenkins will take over the task of grabbing that code, running an installation /
deployment process, testing the application (if you provide it with a test environment),
and reporting back to you those test results.
~ (http://thinkshout.com/blog/2010/09/sean/beginners-guide-using-hudson-continuous-
integration-drupal)

There are a number of different CI tools out there, but we (Classic Graphics) have chosen
Jenkins as it seems to be the leader in the Drupal world. Jenkins, for us, provides a web-
based user interface, easy integration via LDAP to our Windows AD Domain, a friendly
way to run scripts, integration with CVS / Subversion / Git repositories, and of course it
works seamlessly with multiple projects / builds simultaneously. Also, as an added bonus,
Jenkins has a Debian repository that we can use to install it on our Ubuntu servers
without worrying about clobbering other libraries and avoid any dependency nightmares.

Installing Ubuntu 10.04 LTS

1. Insert the Ubuntu Server CD into your system and boot from it. Select your language:

2. Select “Install Ubuntu Server” from the following screen:

3. Choose your installation language again.

4. Select the location of your server installation.

5. Choose a keyboard layout (if you select “Yes” here to detect keyboard layout, the installation will ask you to
press a series of keys and try to detect your keyboard layout).

6. The installer will now check your hardware and configure the network with
DHCP if there is a DHCP server on the network.

7. Now we’ll need to enter a hostname for the server.

8. Setup will now attempt to detect your timezone, if it successfully detects the
correct timezone, select “Yes” to continue. Otherwise, select “No” to manually
select the appropriate timezone from the list.

9. Your hard disk drive will now be detected, and the partition setup wizard will be
launched. Select “Guided – use entire disk and set up LVM” or the type
recommended by your IT department.

Write partition information to disk before creating LVM setup:

Confirm writing changes to disk.

10. The installation will now install the base Ubuntu 10.04 system.

11. Once the base system has been installed, you will need to create a standard user
account for the system. Enter the users Full Name at the prompt.

Next you’ll need to enter the user login name, a suggested name will be supplied.

Next you will be asked for a password for this user.

Select whether or not to encrypt your home directory based your organizations IT
policy and procedures.

12. Setup Package Manager proxy settings.

Setup will now configure “apt” and download the appropriate files and repository
data files.

Next you will be asked how you would like to handle updates, select the option
based on your organizations IT policy and procedures.

13. Next select the following 2 software groups to install on your new server.

Drupal Continuous Integration with Jenkins - The Basics

15. The selected software will be downloaded and installed on the server system.

16. When asked if you would like to install the GRUB boot loader on your system,
select “Yes”.

17. We are now finished with the basic Ubuntu 10.04 system install.

System Configuration

1. Login to your system using the account we created during the installation.
Once logged in, we will use the “sudo” command to switch to the root user.
user@jenkins:~$ sudo su -
[sudo] password for user:
root@jenkins:~#
2. Edit the network configuration to assign a static IP address instead of obtaining
an DHCP assigned address.

We will be modifying the “eth0” portion of this file so that it looks like the
following (substituting your network specific IP settings).

root@jenkins:~# vim /etc/network/interfaces

We will be modifying the “eth0” portion of this file so that it looks like
the following (substituting your network specific IP settings).
# The primary network interface
auto eth0
iface eth0 inet static
address 192.168.0.100
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.1

Restart the network sub-system.
root@jenkins:~# /etc/init.d/networking restart
Next we need to edit the “hosts” file, remember to use your network
specific IP address.
root@jenkins:~# vim /etc/hosts

127.0.0.1 localhost.localdomain localhost 192.168.0.100 jenkins.ourdomain.com
jenkins

# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Setup the “hostname” file.
root@jenkins:~# echo jenkins.ourdomain.com > /etc/hostname
root@jenkins:~# /etc/init.d/hostname restart

Additional Software

Here we will install additional software needed to be installed before we
configure Jenkins. We’ll take care of the rest of the software configuration
after Jenkins is up and running.
1. Just to cover the most popular and supported repository software, we’ll install CVS,
Subversion and Git. We will also install the ClamAV daemon to allow Jenkins to scan our
project artifacts for virus signatures.
a. Install the software packages.
root@jenkins:~# aptitude install git-core subversion cvs clamav-daemon ant openjdk-6-jdk
b. Configure clamd for TCP operation.
root@jenkins:~# vim /etc/clamav/clamd.conf
c. We need to add the following 2 lines to the configuration file.
TCPSocket 3310
TCPAddr 127.0.0.1

2. For our purpose in the use of Jenkins for Drupal testing, we will also
need to install Drush. The version available through the Ubuntu 10.04
repo is quite old however, so we will first download the latest
released version (7.x-4.5) from http://drupal.org/project/drush
(http://ftp.drupal.org/files/projects/drush-7.x-4.5.tar.gz). Follow the
instructions in the included INSTALL.txt to install, my personal
preference is to extract the tarball into /usr/share (or /usr/local/share)
and then symlink /usr/share/drush/drush (/usr/local/share/drush/drush)
to /usr/bin/drush (/usr/local/bin/drush), just to make things simple.
root@jenkins:~# wget http://ftp.drupal.org/files/projects/drush-7.x-4.5.tar.gz
root@jenkins:~# tar –C /usr/share –xzf drush-7.x-4.5.tar.gz
root@jenkins:~# ln –s /usr/share/drush/drush /usr/bin

3. Now we’ll update the system to download any security updates
released since our installation CD was created. After the upgraded
packages have been installed, we will need to reboot the system
before proceeding.

root@jenkins:~# aptitude update
root@jenkins:~# aptitude full-upgrade
-root@jenkins:~# shutdown –r now

4. All done, the system should be ready for installing Jenkins when it
comes back up.

Install Jenkins

Jenkins can be easily installed on Debian based systems using the
repository provided by the kind folks over at jenkins-ci.org. Simply follow
these instructions to add the Jenkins repository to your system and install
Jenkins along with its dependencies.
1. Setup Jenkins CI Repository.
a. First we will need to add the repository signing key from jenkins-ci to our key
chain.
root@jenkins:~# wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -

b. Next we create the jenkins.list repo file for jenkins-ci.org.
root@jenkins:~# echo “deb http://pkg.jenkins-ci.org/debian binary/” > /etc/apt/sources.list.d/jenkins.list

c. Now update the repository data.
root@jenkins:~# aptitude update

2. Install Jenkins and its dependencies using aptitude.
root@jenkins:~# aptitude install jenkins

3. That’s it for getting Jenkins installed, the rest of the
configuration happens from the web interface.

Configure Jenkins Server

1. Point your favorite web browser
(FireFox of course) to http://{ip of
server}:8080 (or if you have DNS entries
for the system already in place),
http://jenkins.ourdomain.com:8080.

You should now see the Jenkins
Dashboard.

2. The first thing we’ll need to do is secure
our Jenkins install. (using this guide,
we’ll just enable Jenkins own internal
user management for simplicity).
a. Click the “Manage Jenkins” link on
the left-hand side of the page.

b. Click the “Configure System” link
in the right-hand content portion
of the page.
c. Scroll down until you see the
“Enable Security” checkbox and
check it. Then select “Jenkin’s
own user database” under
“Security Realm”, leave “Allow
users to sign up” checked for
now. Scroll to the bottom and
click the “Save” button.

d. Add yourself as a user by clicking the
“sign up” link now located in the upper
right-hand corner of the webpage.

And then enter your user / login
information on the form provided, and
then clicking the “Sign up” button.

e. Now we will modify the security configuration to
disabled user account creation and provide a finer
grained security model. Click “Manage Jenkins” ->
“Configure System” again.
f. Scroll down to “Security Realm” -> “Jenkin’s own
user database” and remove the checkmark from
“Allow users to sign up”.
g. Scroll down and under “Authorization” select
“Project-based Matrix Authorization Strategy”.
Enter the username used above in the
“User/group to add:” text box and click “Add”.
Select the permissions for the Anonymous user
and “All” permissions for the “Administrative”
user we just created.
h. We should now have a Jenkins install that will
only allow anonymous users to see jobs that have
been created by authorized users. Additional
Users may be added to the system by going to
“Manage Jenkins” -> “Manage Users”.

3. Next we need to apply any updates to
Jenkins plugins that are available and
restart Jenkins. Click “Manage
Jenkins” -> “Manage Plugins”. If there
are any available updates, select “All”
and click “Install”.

Click the “Restart Jenkins when …”
checkbox at the bottom of the page.

4. After the plugins have been updated, Jenkins will restart and you will be directed to the
“Login” page. Enter your credentials and click “Login”.

5. Click “Manage Jenkins” -> “Manage Plugins”. Click on the “Available” tab to display a list
of modules available for install. We will be select the following modules (Audit Trail
Plugin, Create Job Advanced Plugin, Blame Upstream Committers Plugin, Email-ext Plugin,
Status Monitor Plugin, All Changes Plugin, Checkstyle Plugin, DRY Plugin, Log Parser Plugin,
PMD Plugin, Violations, Workspace Cleanup Plugin, ClamAV Plugin, AnsiColor Plugin, Git
Plugin) which may be listed in multiple locations, we only need to check it once on the
page, not every location . Not all of these are required, but most should be useful for
Drupal builds, feel free to leave some out or add additional modules as required. Once
complete, click the “Install” button at the bottom of the page. Jenkins will restart when
finished installing the modules, and you will be presented with the login screen.

6. After logging into the Jenkins system
again, click “Manage Jenkins” ->
“Configure System”.
a. Scroll down to the “Create Job
Advanced” section. We want to
check the first 3 checkboxes.

b. Next we will add the path to our
Java JDK installation. Enter a name
for the install, then uncheck the
“Install Automatically” and enter
the correct path for JAVA_HOME
(should point to /usr/lib/jvm/java-6-
openjdk/ on Ubuntu systems using
the standard repository packages).

c. Next we want to make sure the Git:
configuration for the source code
management programs is correct,
or enter the correct values as
shown below.

Git Username / email:

CVS/Subversion:

d. Next we need to configure the
location of our Apache Ant
installation. Enter a name for the
install, and then uncheck the
“Install Automatically” checkbox,
then enter the correct path to our
Apache Ant installation
(/usr/share/ant if using standard
Ubuntu repository).

e. Set /bin/bash as our shell.

f. Configure Jenkins email settings.

g. Configure the ClamAV scanner and
then click the “Save” button.

That’s it for the basic configuration of Jenkins. Later on we’ll use the web interface
to configure a sample job that will perform the following tasks:
Download source code from the repository
Use drush to setup / install the Drupal instance
Enable the Coder module to perform Drupal Style analysis of the given modules
Enable the SimpleTest module and run available SimpleTests on the module(s) /
Module Group(s)
Run PHP Mess Detector against internally developed modules
Run PHP Copy and Paste Detector against internally developed modules
Collect XML output from above tests and create trend graphs to track internal
modules
Notify developers when committed code breaks build
Scan source code for viruses

System Configuration

Next we’ll be modifying the default configuration of Apache and MySQL. Since we won’t be
using any live data for these tests, standard configuration is find for the most part.

1. Apache
a. We need to install PHP5 for both apache and command line use. At a minimum, the modules
included in the command below should also be installed. You may install other PHP5 modules if
your drupal installation requires them to be installed.
root@jenkins:~# aptitude install php5 php5-cli php5-gd php5-curl php5-dev php-pear

NOTICE: If you installed either the php5-imagick or php5-mcrypt modules above, you will need to edit their
configuration files to avoid errors when running SimpleTest tests. Comments beginning with ‘#’in
configuration files have been deprecated, all comments should start with the ‘;’ character. (See Ubuntu
bugs 573436 and 556469)

i. Edit the /etc/php5/cli/php.ini configuration file as follows
root@jenkins:~# vim /etc/php5/cli/php.ini
Set the following configuration variables, everything else can remain
as is in the configuration file. Memory limit for CLI environment has
been set to unlimited (-1).
short_open_tag = Off
max_execution_time = 600
memory_limit = -1
ii. Make the same edits to /etc/php5/apache2/php.ini, with the
exception of the memory_limit variable which should be set to a
minimum of 128MB.
memory_limit = 128M

b. We need to enable the rewrite module.
root@jenkins:~# a2enmod rewrite

c. Next we’ll need to modify the standard default site configuration to
allow overrides.
root@jenkins:~# cd /etc/apache2/sites-available/
root@jenkins:/etc/apache2/sites-available# vim default
The finished file should look like the following:
<VirtualHost *:80>
ServerAdmin webmaster@localhost
DocumentRoot /var/www

<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride All

Order allow,deny
allow from all
</Directory>

ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/

<Directory "/usr/lib/cgi-bin">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>

ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice,
warn, error, crit,
# alert, emerg.
LogLevel info CustomLog /var/log/apache2/access.log combined
Alias /doc/ "/usr/share/doc/“
<Directory "/usr/share/doc/">
Options Indexes MultiViews FollowSymLinks
AllowOverride None

Order deny,allow
Deny from all
Allow from 127.0.0.0/255.0.0.0 ::1/128
</Directory>
</VirtualHost>
d. Now we need to restart apache so the configuration changes will be
applied.
root@jenkins:~# /etc/init.d/apache2 restart

2. MySQL
a. Create a MySQL user to use for testing.
root@jenkins:~# mysql -u root –p mysql
Enter password:
mysql> CREATE USER 'testuser'@'localhost' IDENTIFIED BY 'testuser';
mysql> GRANT ALL ON *.* TO 'testuser'@'localhost';
mysql> quit;

b. That should do it for MySQL.

3. Now we will install some additional PHP tools.
a. First let’s upgrade pear to the latest version.
root@jenkins:~# pear channel-update pear.php.net
root@jenkins:~# pear upgrade-all
b. PHP Mess Detector (phpmd)
i. First we need to add the required pear channels.
root@jenkins:~# pear channel-discover pear.phpmd.org
root@jenkins:~# pear channel-discover pear.pdepend.org
ii. Install some dependencies for phpmd / pdepend with aptitude.
root@jenkins:~# aptitude install imagemagick libmagickwand-dev

iii. Now we can install phpmd.
root@jenkins:~# pear install --alldeps phpmd/PHP_PMD
root@jenkins:~# if [ ! -e /etc/php5/conf.d/imagick.ini ]
 then
 echo 'extension=imagick.so' > /etc/php5/conf.d/imagick.ini
 fi
root@jenkins:~#

iv. This following step is optional. However, if you want to integrate
phpmd with ant to check your builds, you’ll need to do the
following additional step. We need to download and install the ant-
phpmd jar file, and place it into the ant lib directory.
root@jenkins:~# wget http://static.phpmd.org/ant-phpmd/0.1.2/ant-phpmd.jar -O
/usr/share/java/ant-phpmd-0.1.2.jar
root@jenkins:~# ln -s /usr/share/java/ant-phpmd-0.1.2.jar /usr/share/java/ant-phpmd.jar
c. PHP Copy Paste Detector
i. First we need to add the required pear channels.
root@jenkins:~# pear channel-discover pear.phpunit.de
root@jenkins:~# pear channel-discover components.ez.no
ii. Now we can install phpcpd.
root@jenkins:~# pear install --alldeps phpunit/phpcpd

4. We’re done…. Simple right!?

Create a Drupal Job in Jenkins

Here, we will be creating a simple / basic job to test Drupal, and any custom
modules just to make sure everything is in working order. Since every environment
and required testing is different, we will leave the advanced job creation and
deployment scripting to the end-user of this document.

The Jenkins job setup in this tutorial makes a couple of assumptions:
1. You have setup pub-key based auth over ssh to your git repo, depending
on your configuration this type of setup may or may not work for you.
2. You have Drupal core, all required modules, and all custom modules
located within your git repository. However, the script could be easily
modified to retrieve the latest released version of Drupal and any other
required modules using Drush if this is not the case for you organization.

1. Open up our favorite web browser
and point it to our new Jenkins server
(http://jenkins.ourdomain.com:8080).
Click the “log in” link in the upper
right-hand corner of the browser
window.

2. Enter your login credentials and click
“log in”.

3. Click the “New Job” link in the menu.
Enter “Drupal Test” and select “Build a
free-style software project”. Click “Ok”
when done.

NOTE: When creating these jobs, in order to
simplify the creation of the “git” post-receive bash
script to launch a build remotely, you should
consider naming your jobs the same as the name of
your repository/repo-directory name. So, if you
have your repos on your git server in the
/var/git/reponame.git directory,then this job would
be called reponame. When viewing the post-
receive-jenkins script below, it will become
apparent why this makes things easier.

4. Now we need to fill in some basic
configuration for our newly created
Drupal-Test job as shown in the
following screenshots.
a. Enter a description for our job (not required)
b. If you would like to setup automatic deletion
of old builds, check the “Discard Old Builds”
checkbox. Additional options will be
displayed.
c. You will probably want to leave project based
security enabled, add any additional users and
set permissions as required.
d. Next check the box labeled “This build is
parameterized” and add parameters as
needed. (optional) I have added a parameter
for branch below, this will allow you to include
the parameter in a URL from a post-commit to
build a specific branch of the code, or leave it
out to build the master branch.
e. We’ll probably also want to be able to have
concurrent builds going at the same time, just
in case you have multiple developers that
could be submitting changes at the same time.

f. Next we need to setup our repository
information. As stated earlier, I have
setup pub-key auth for accessing our
remote repository.
g. At some point you’ll want to be able to
trigger builds remotely, so we’ll check
the “Trigger builds remotely” option,
and fill in the token field with a string
used to uniquely identify and verify the
build. This ensures that only people
knowing the correct “Token” are able to
kickoff a build remotely.
h. Check the “Delete workspace before
build starts” to ensure you are starting
from a clean slate with each build.

i. Check the “Color ANSI Console Output”
to capture any color output from your
scripts.
j. Next we need to create a build script.
Scroll down to the “Build” section, and
click the “Add build step” and select
“Execute Shell” from the dropdown
menu.

#!/bin/sh -x
# First we need to create an empty DB for drupal using our testuser account
mysqladmin -u testuser -ptestuser create ${BUILD_TAG}

# Change to our working directory
cd ${WORKSPACE}

# Switch to the specified repository branch
#(this is where our custom BRANCH parameter comes into play)
git checkout ${BRANCH}

# apply the drupal core patch for drupal 6.x
# assumes you have the simpletest module installed already
patch -p0 < sites/all/modules/simpletest/D6-core-simpletest.patch

# use drush to install a basic site with defaults enabled using our empty db
drush si --yes --db-url="mysql://testuser:testuser@localhost/${BUILD_TAG}"
k. You will then be able to add a script (BASH)
# just check the status to make sure we are good
in the provided textbox to execute our drush --uri=http://customsite.example.com/ status

Drupal build tests. I have included a sample # Change to our multi-site directory and link the default settings file here
cd sites/customsite.example.com
script here that runs PHP Mess Detector and ln -sf ../default/settings.php

PHP Copy and Paste Detector on our custom # go back to the working directory so Drush can find things
cd ${WORKSPACE}
Drupal modules, the Coder module to check
# create the directory to put our logs in
our code against Drupal coding standards mkdir logs
and SimpleTest(s) where available. We’ll # enable the coder module and run the coder based tests then disable it
drush --yes --uri=http://customsite.example.com/ pm-enable coder,simpletest
pipe the output of these checks into a
# enable simpletest and other (required) modules that we will be testing.
“logs” directory that we create in the script, drush --yes --uri=http://customsite.example.com/ pm-enable simpletest,{comma separated
list of modules to enable, not including the standard/required Drupal modules}
and use Jenkins plugins PMD, Checkstyle
and DRY to parse and track the build history. # Run coder against our custom modules
for MOD in {space separated list of modules to run coder against}; do
drush --uri=http://customsite.example.com/ coder checkstyle ${MOD} > logs/coder-
${MOD}.xml 2>/dev/null
done

# Run the SimpleTest tests for our Custom module
for GROUP in {space separated list of module(s) and/or module group(s) to test}; do
drush --uri=http://customsite.example.com/ test-run $GROUP} 2>/dev/null
done

# Lets run the PHP Mess Detector on our custom modules
phpmd sites/customesite.example.com/ xml codesize,unusedcode --reportfile logs/phpmd-
drupal.xml 2>/dev/null

# Run the PHP Copy Paste Detector on our code
phpcpd --log-pmd logs/phpcpd-drupal.xml sites/customsite.example.com 2>/dev/null

# drop this build database
mysqladmin -f -u testuser -ptestuser drop ${BUILD_TAG}

l. Under “Post Build Actions” we will select
“Publish Checkstyle analysis results” and enter
the “Checkstyle results” as “logs/coder-*.xml”
(or a pattern matching your script if different).
m. Select “Publish PMD analysis results” and enter
the “PMD results” as “logs/phpmd-drupal.xml”
(or a pattern matching your script if different).
n. Select “Publish duplicate code analysis results”
and enter the “Duplicate code results” as
“logs/phpcpd-drupal.xml” (or a pattern
matching your script if different).
o. We want to save our artifacts (code snapshot).
So check the “Archive the artifacts” checkbox.
p. Yeah, it’s PHP code, but someone could have
embedded something nasty in an image file,
DB insert, etc. Let’s check it just to be on the
safe side! Enable the ClamAV scanner by
checking the “Check for viruses” option.

q. We want to parse the output from our
script, and mark builds accordingly, so
we’ll check “Console output parsing” and
both sub-options “Mark build Unstable
on Warning” and “Mark build Failed on
Error”.
r. Select “Record fingerprints of files to
track usage” and “Fingerprint all
archived artifacts”.
s. Select “Email Notification” enter your
team email address (or mailing list, etc).
Optionally select one or both of the sub-
options for email notification.
t. And the last option we will select for this
build is the “Status Monitor” option.
u. To save the configuration, click on the
“Save” button at the bottom of the
page.

1. Now we’re ready for a test run, click “Build Now” on
the left-hand side of the page.

2. How did we do? Did it work? How did your dev
team do? Any warnings or errors in the code?

Git Post-Receive Processing

#!/bin/bash
1. We need to create a post-receive script while read oldrev newrev refname; do
for git (assuming git is being used) to break;
done;
remotely trigger the Jenkins job each
# find the branch of the push
time an update is pushed to the git BRANCH_NAME=${refname##refs/heads/}
repository. This script should be placed
# we also need to find what our repository name is, this
in a directory that is accessible by all # script assume that all repositories are stored in
git users (perms 755). This is an # /some/directory/tree/reponame.git
if [ $(git rev-parse --is-bare-repository) = true ]; then
example script that provides basic REPONAME=$(basename $PWD)
else
functionality. We’ll expand upon it a REPONAME=$(basename $(readlink -nf "$PWD"/..))
little later to provide useful services to fi
# strip off the .git part if present
our build process: REPONAME=${REPONAME%.git}
# Now we just do a simple http request to the jenkins server
wget -q -O - -t 1
"http://jenkins.ourdomain.com:8080/job/${REPONAME}/buildWithParameters?to
ken=YOUR-TOKEN&BRANCH=${BRANCH_NAME}" > /dev/null

2. Next we will need to modify each of the existing repositories .git/hooks/post-receive
script as follows:
#!/bin/sh
#
# see contrib/hooks/ for a sample, or uncomment the next line and
# rename the file to "post-receive".
#. /usr/share/doc/git-core/contrib/hooks/post-receive-email
# IF we uncomment the line above, since post-receive arguments are passed through
# <STDIN>, the above script will pull those off <STDIN> and we have no way of
# grabbing them. There are several solutions to this, the one I have selected below
# serves our purpose nicely.
# get the <STDIN> input into a variableINPUT=$(cat)
# now we can just pipe those args to any number of scripts in the same fashion,
# without having to rewrite them to accept command line arguments
echo ${INPUT} | /usr/share/doc/git-core/contrib/hooks/post-receive-email

# and of course our new jenkins script
echo ${INPUT} | /usr/share/doc/git-core/contrib/hooks/post-receive-jenkins

3. Next we’ll modify our original simple script (Item #1 above) showing how the post-receive
script could be used to perform different actions based on the type of action that was
performed on the git repository. This could possibly be a different Jenkins job, or the same
job with more parameters passed to modify how the job script functions, or what
procedures it uses to accomplish the selected task(s). Our script below creates an archive
of the repository, and moves it to an archival location.
a. First we will need to determine the type of change happening
b. Then we need to know what type of revision is going on
c. Finally we add some logic to act on the 2 pieces of additional information we have
gathered from our git repo and passed parameters.
d. We can easily add more cases that we could act on, or do different things based on
any predetermined conditions that we can meet or be extracted from the
information contained within the git push/transaction.
Our modified script follows on the next few slides.

#!/bin/bash
#
# Copyright (c) 2011 Classic Graphics
# Released under GPLv2
# read variables / parameters from <STDIN>
while read oldrev newrev refname
do
break
done

# determine the type of change submitted
# borrowed / copied from the post-receive-email script
if expr "$oldrev" : '0*$' >/dev/null
then
change_type="create"
else
if expr "$newrev" : '0*$' >/dev/null
then
change_type="delete"
else
change_type="update"

fi
fi

# Next we need to determine the revision type we are dealing with and act accordingly
# borrowed / copied from the post-receive-email script
newrev_type=$(git cat-file -t $newrev 2> /dev/null)
oldrev_type=$(git cat-file -t "$oldrev" 2> /dev/null)

case "$change_type" in
create|update)
rev="$newrev"
rev_type="$newrev_type"
;;
delete)
rev="$oldrev"
rev_type="$oldrev_type"
;;
*)
rev=""
rev_type="“
esac

# we also need to find what our repository name is, this script assumes that all
# repositories are stored in /some/directory/tree/reponame.git
if [ $(git rev-parse --is-bare-repository) = true ]
then
REPONAME=$(basename $PWD)
else
REPONAME=$(basename $(readlink -nf "$PWD"/..))
fi

# strip off the .git part if present
REPONAME=${REPONAME%.git}

# now that we know what, let's do something useful with it
case "$refname","$rev_type" in
refs/tags/live-*,commit)
# we'll create a tarball of the tag here, and then move it to long term storage where it
# can be archived and or moved to an Escrow area.
TAG_NAME=${refname##refs/tags/}
TMP_DIR="/tmp/${TAG_NAME}"

# create our temp directory for repo
mkdir -p ${TMP_DIR}

# clone the repo into the temporary directory
git clone /var/git/${REPONAME}.git ${TMP_DIR} 2>/dev/null 1>/dev/null

# change into the tmp directory
cd ${TMP_DIR}

# checkout the tag we are going to archive
git checkout tags/${TAG_NAME} 2>/dev/null 1>/dev/null

# The following code assumes there is a directory for the archives
# @ /var/git/archives and that the git user has permissions to
# write/create files and directories there.
# make sure our path for the generated archive exists
mkdir -p /var/git/archives/${REPONAME}.git

# now we'll create the archive of our tag in a safe location
tar cjf /var/git/archives/${REPONAME}.git/${TAG_NAME}.tbz --exclude-vcs . 2>/dev/null 1>/dev/null

# remove our temporary directory
cd

rm -rf ${TMP_DIR}
;;
refs/heads/*,commit)
# find the branch of the push
BRANCH_NAME=${refname##refs/heads/}

# Now we just do a simple http request to the Jenkins server
wget -q -O - -t 1
"http://localhost:8080/job/${REPONAME}/buildWithParameters?token=DrupalTestBuild123456&BRANCH=${BRANCH_NAME}"
> /dev/null
;;
*)
# anything else we simply ignore for now, other scenarios
# and / or options can be added to the script as needed.
;;
esac

# exit
exit 0

4. This could be extended further, to say deploy the live-{date} tagged items to the production
server. Or possibly deploy tags like “staging-{date}” to a testing server (Part 2? / Deploy) or a
Full Blown integrated testing environment that runs Selenium automated testing for QA
Purposes (Part 3? / Testing). The possibilities are truly endless. I would also like to point out an
excellent tutorial on getting Selinium + Jenkins setup on a headless linux system for running
automated tests:

http://centripetal.ca/blog/2011/02/07/getting-started-with-selenium-and-jenkins/

Hopefully this has helped someone in the community in setting up there own Drupal CI
environment, or at least given you a trotting start !

Enjoy, and feel free to contact us with any suggestions, corrections or mistakes we may have
made or that inadvertently slipped between the cracks.

John W Smith (johns@knowclassic.com) or (JSmith@i1Technologies.com)

Drupal Continuous Integration with Jenkins - The Basics

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Drupal Continuous Integration with Jenkins - The Basics (20)

Recently uploaded (20)

Drupal Continuous Integration with Jenkins - The Basics