Manoj Kolhe - Setup GitHub with Jenkins on Amazon Cloud - End-to-end Automation

This document highlights the steps wise
procedure for setting up Continuous
Integration with Jenkins and GitHub
repository
Jenkins
Integration
with GitHub
Continuous Integration Test
Environment
Manoj Kolhe

© www.ManojKolhe.com
Page 1 of 45

Page 2 of 45
Introduction .................................................................................................................................................................................................. 3
Approach....................................................................................................................................................................................................... 3
Hardware Requirements............................................................................................................................................................................... 5
Software Requirements................................................................................................................................................................................ 5
Configure Git Plugin with Eclipse IDE............................................................................................................................................................ 6
Create Repository.................................................................................................................................................................................... 7
Track Changes ........................................................................................................................................................................................ 10
Inspect History....................................................................................................................................................................................... 12
GitHub Tutorial....................................................................................................................................................................................... 15
Create Local Repository..................................................................................................................................................................... 15
Create Repositoryat GitHub ............................................................................................................................................................. 15
Eclipse SSH Configuration.................................................................................................................................................................. 16
Push Upstream.................................................................................................................................................................................. 17
Downloadand Install Java........................................................................................................................................................................... 21
Configure Apache Tomcat........................................................................................................................................................................... 22
Changing the Tomcat Default 8080 port ............................................................................................................................................... 24
Configure Apache Maven............................................................................................................................................................................ 25
Configure Jenkins........................................................................................................................................................................................ 26
Download GitHub related plugins.......................................................................................................................................................... 28
Configure System................................................................................................................................................................................... 29
Create a new jobin Jenkins.................................................................................................................................................................... 30
Adding GitHub Credentials..................................................................................................................................................................... 31
To generate SSH key for GitHub ............................................................................................................................................................ 31
Step 1: Check Existing keys................................................................................................................................................................ 31
Step 2:Generate a new SSH key....................................................................................................................................................... 31
Step 3:Add your SSH key to GitHub.................................................................................................................................................. 32
Step 4:Test everything out............................................................................................................................................................... 35
Test GitHub Jenkins Integration............................................................................................................................................................. 36
Create Userin Jenkins for Remote Job Builds ....................................................................................................................................... 38
Initial Steps:....................................................................................................................................................................................... 38
Push GitHub Changes to Jenkins................................................................................................................................................................. 41
Showing Jenkins Build Status on GitHub..................................................................................................................................................... 43
Queries........................................................................................................................................................................................................ 44

Page 3 of 45
Jenkins Integration with GitHub
A Step by step guide for the setup and configuration
Introduction
enkins is used for Continuous Integration Testing of builds. Jenkins provides continuous integration
services for software development. It is a server-based system running in a servlet container such as
Apache Tomcat. It supports SCM tools including AccuRev, CVS, Subversion, Git, Mercurial, Perforce,
Clearcase and RTC, and can execute Apache Ant and Apache Maven based projects as well as arbitrary
shell scripts and Windows batch commands. The primary developer of Jenkins is Kohsuke Kawaguchi.
Released under the MIT License, Jenkins is free software.
Builds can be started by various means, including being triggered by commit in a version control system,
scheduling via a cron-like mechanism, building when other builds have completed, and by requesting a
specific build URL.
Approach
1. Configure Jenkins to communicate with GitHub by means of available plugins
2. Jenkins would contain the target repository URL to pull the repo contents
3. Build would happen locally referring to Maven pom.xml
4. Build status would be published and embedded on GitHub readme.md
J

Page 4 of 45
DevelopmentCodeCheck-in
at Local repository
Eclipse Git plugin pushes
local changes to Git
repository
Git Repository uses
WebHook APIto push
notification to Jenkinsfor
any changes observed
Jenkinstriggers build for the
changes notification and
pullsthe changes from Git
repository
Jenkinsbuild status is
published back to Git
repository 'readme.md' for
instant notification of build
success / failure

Page 5 of 45
Hardware Requirements
 4 GB RAM recommended
 3 GHz CPU recommended
Software Requirements
No Software URL
1 Apache
Tomcat v7.0
http://tomcat.apache.org/download-70.cgi
2 Java Runtime
Environment
6.0
http://www.oracle.com/technetwork/java/javase/downloads/jre6downloads-
1902815.html
3 Apache
Maven 3.0
http://maven.apache.org/download.cgi
4 Jenkins
Distributable
WAR
http://mirrors.jenkins-ci.org/war/latest/jenkins.war
5 GitHub
Repository
http://www.github.com
6 Git
Installation
http://git-scm.com/download/win
7 Eclipse Git
Plugin
www.eclipse.org/egit/download

Page 6 of 45
Configure Git Pluginwith Eclipse IDE
Whenever the history of the repository is changed (technically, whenever a commit is created), Git keeps
track of the user who created that commit. The identification consists of a name (typically a person's
name) and an e-mail address. This information is stored in file ~/.gitconfig under dedicated keys.
EGit will ask you for this information when you create your first commit. By default, this dialog is shown
only once until you create a new workspace or tick the checkbox "Show initial configuration dialog" on
the Git Preference page:
Youcan also un-tick "Don'tshow this dialog again" if you want to see it again later.
Instead of using this dialog, youcan always change this information using the Git configuration:
 Click Preferences>Team> Git > Configuration
 Click NewEntry and enter the key value pairs user.email and user.name

Page 7 of 45
Create Repository
 Create a new Java project HelloWorld. (In this case, the project was built outside of your Eclipse
Workspace.)

Page 8 of 45
 Select the project,click File > Team> ShareProject.
 Select repository type Git and click Next.
 To configurethe Git repository select the new project HelloWorld.

Page 9 of 45
 Click Create Repository to initialize a new Git repository for the HelloWorld project. If your project
already resides in the working tree of an existing Git repository the repository is chosen
automatically.
 Click Finish to close the wizard.
 The decorator text "[master]" behind the project shows that this project is tracked in a repository on
themaster branch and the question mark decorators show that the .classpath and .project and
the .settingsfiles are not yet under version control.

Page 10 of 45
Track Changes
 Click Team > Add on the project node. (This menu item may read Add to Index on recent versions
of Egit.)
 The + decorators show that now the project's files have been added to version control.
 Mark the "bin" folder as "ignored by Git", either by right-clicking on it and selecting Team >
Ignore or by creating a file .gitignore in the project folder with the following content:
/bin
 This excludes the bin folder from Git's list of tracked files.
 Add .gitignore to version control (Team > Add):
 You may have to set your Package Explorer filters in order to see .gitignore displayed in the Package
Explorer. To access filters, select the down arrow on the right edge of the Package Explorer tab to
display View Menu.
 Select Filters... from the View Menu and you will be presented with the Java Element Filters dialog.
Unselect the top entry to display files that begin with . (period) such as .gitignore.

Page 11 of 45
 Click Team > Commit in the project context menu.
 Enter a commit message explaining your change, the first line (followed by an empty line) will
become the short log for this commit. By default the author and committer are taken from
the .gitconfig file in your home directory.
 You may click Add Signed-off-by to add a Signed-off-by: tag.
 If you are committing the change of another author you may alter the author field to give the name
and email address of the author.
 Click Commit to commit your first change.
 Note that the decorators of the committed files have changed as a result of your commit.

Page 12 of 45
Inspect History
 Click Team > Show in History from the context menu to inspect the history of a resource:
 Create a new Java class Hello.java and implement it.
 Add it to version control and commit your change.
 Improve your implementation and commit the improved class.
 The resource history should now show 2 commits for this class.

Page 13 of 45
 Click the Compare Mode toggle button in the History View.
 Double click src/Hello.java in the Resource list of the History View to open your last committed
change in the Compare View.

Page 14 of 45
Congratulations, you just have mastered your first project using Git!!!

Page 15 of 45
GitHub Tutorial
Create Local Repository
 Follow EGit/User Guide/Getting Started to create a new local repository (with your content instead
of the demo project).
Create Repository at GitHub
 create a new repository at GitHub:
On the next screen you can see the URLs you may use to access your fresh new repository:
 click SSH to choose the SSH protocol. It can be used for read and write access.
 click HTTP to choose the HTTP protocol. It can also be used for read and write access.
 clickGit Read-Onlyto choose the anonymous git protocol for cloning. It's the most efficient protocol
git supports. Since the git protocol doesn't support authentication it's usually used to provide
efficient read-only access to public repositories.

Page 16 of 45
Eclipse SSH Configuration
 Open the Eclipse Preferences and ensure that your SSH2 home is configured correctly (usually this
is~/.ssh) and contains your SSH2 keys:

Page 17 of 45
 if you don't have SSH keys yet you may generate them on the second tab of this dialog (Key
Management). Use a good pass phrase to protect your private key, for more details see "working
with key passphrases".
 upload your public SSH key to your GitHub account settings.
PushUpstream
 Click Team > Remote > Push... and copy and paste the SSH URL of your new GitHub repository.
 If you are behind a firewall which doesn't allow SSH traffic, use the GitHub HTTPS URL instead and
provide your GitHub user and password instead of using the uploaded public SSH key. To store your
credentials into the Eclipse secure store click Store in Secure Store.
 Note: many HTTP proxies are configured to block HTTP URLs containing a user name, since
disclosing a user name in an HTTP URL is considered a security risk. In that case remove the user
name from the HTTP URL and only provide it in the user field. It will be sent as an HTTP header.

Page 18 of 45
 Click Next and on first connection accept GitHub's host key.
 Enter your SSH key's passphrase and click OK.
 On the next wizard page click Add all branches spec to map your local branch names to the same
branch names in the destination repository (on a one-to-one basis).
 Click Next. The push confirmation dialog will show a preview of the changes that will be pushed to
the destination repository.

Page 19 of 45
 Click Finish to confirm that you want to push these changes.
 The next dialog reports the result of the push operation.

Page 20 of 45
 Point your browser at your GitHub repository to see that your new repository content has arrived.

Page 21 of 45
Download and Install Java
1. Download and install java using the setup
2. Confirm Java installation by running command ‘java -version’ (without quotes )on the command
prompt

Page 22 of 45
Configure Apache Tomcat
Installing Tomcat on Windows can be done easily using the Windows installer. Its interface and
functionality is similar to other wizard based installers, with only a few items of interest.
 Installation as a service: Tomcat will be installed as a Windows service no matter what setting
is selected. Using the checkbox on the component page sets the service as "auto" startup, so that
Tomcat is automatically started when Windows starts. For optimal security, the service should
be run as a separate user, with reduced permissions (see the Windows Services administration
tool and its documentation).
 Java location: The installer will provide a default JRE to use to run the service. The installer uses
the registry to determine the base path of a Java 6 or later JRE, including the JRE installed as part
of the full JDK. When running on a 64-bit operating system, the installer will first look for a 64-
bit JRE and only look for a 32-bit JRE if a 64-bit JRE is not found. It is not mandatory to use the
default JRE detected by the installer. Any installed Java 6 or later JRE (32-bit or 64-bit) may be
used.
 Tray icon: When Tomcat is run as a service, there will not be any tray icon present when Tomcat
is running. Note that when choosing to run Tomcat at the end of installation, the tray icon will be
used even if Tomcat was installed as a service.
 Refer to the Windows Service HOW-TO for information on how to manage Tomcat as a Windows
service.
The installer will create shortcuts allowing starting and configuring Tomcat. It is important to note that
the Tomcat administration web application can only be used when Tomcat is running.
Further help: https://tomcat.apache.org/tomcat-7.0-doc/setup.html

Page 23 of 45
For manual startup of Tomcat, browse to <TOMCAT_DIRECTORY>binstartup.bat
On successful configuration of Tomcat, open browser and type http://localhost:8080 in address bar,
enter.

Page 24 of 45
Changing the Tomcat Default 8080 port
In order to change default 8080 port of tomcat, please navigate to
<TOMCAT_DIRECTORY>confserver.xml
Modify default 8080 port to other port.
NOTE:
 Ensure no other changes are done in Server.xml.
 Tomcat restart would be required for reflecting port changes
Ensure CATALINA_HOME is present in Environment Properties
My Computer -> Properties -> Advanced -> Environment Variables -> User Variables

Page 25 of 45
Configure Apache Maven
Maven is not installable. Unzip the download in some directory and update M3_HOME environment
variable in System.
Confirm Maven Installation by running ‘mvn -version’ at command prompt:

Page 26 of 45
Configure Jenkins
Place downloaded archive ‘Jenkins.war’ in <TOMCAT>webapps folder and start the Tomcat server.
Browser to URL: http://localhost:8080/jenkins

Page 27 of 45
Go to Manage Jenkins -> Manage Plugins -> Available Plugins and in filter box type ‘git’ and download
below plugins:

Page 28 of 45
Download GitHub related plugins
Restart Jenkins to install the plugins and see further configuration.

Page 29 of 45
Configure System
Go to Manage Jenkins -> Configure System
Specify JDK, Maven Installation and Git installation

Page 30 of 45
Create a new job in Jenkins
Go to URL: http://localhost:8080/jenkins and click on create a job
Select Build a maven2/3 project and specify Item name as below:
Once project created specify the GitHub project location:
Specify the Source Code for GitHub and provide credentials

Page 31 of 45
Adding GitHub Credentials
To generate SSH key for GitHub
Step 1: Check Existing keys
First, we need to check for existing SSH keys on your computer. Open up your Git Bash and type:
Check the directory listing to see if you have files named either id_rsa.pub or id_dsa.pub. If you don't
have either of those files go to step 2. Otherwise, you can skip to step 3.
Step 2: Generate a new SSH key
To generate a new SSH key, copy and paste the text below, making sure to substitute in your email. The
default settings are preferred, so when you're asked to "enter a file in which to save the key,"" just press
enter to continue.

Page 32 of 45
Next, you'll be asked to enter a passphrase.
Which should give you something like this:
Then add your new key to the ssh-agent:
Step 3: Add your SSH key to GitHub
Run the following code to copy the key to your clipboard.
Alternatively, using your favorite text editor, you can open the ~/.ssh/id_rsa.pub file and copy the
contents of the file manually
Now that you have the key copied, it's time to add it into GitHub:
1.
2. In the user bar in the top-right corner of any page, click Account Settings.

Page 33 of 45
3. Click SSH Keys in the left sidebar.
4.
Click Add SSH key.
5. In the Title field, add a descriptive label for the new key. For example, if you're using a personal
Mac, you might call this key "Personal MacBook Air".
6.
Paste your key into the "Key" field.

Page 34 of 45
7. Click Add key.
8. Confirm the action by entering your GitHub password.

Page 35 of 45
Step 4: Test everything out
To make sure everything is working, you'll now try SSHing to GitHub. When you do this, you will be
asked to authenticate this action using your password, which for the passphrase you created earlier.
Open up your Git Bash and type:
You may see this warning:
Type “yes”

Page 36 of 45
Test GitHub Jenkins Integration
Check Console Output forsuccessful execution

Page 37 of 45
Build, if successful will be denoted by BLUE dot whereas failed build would be denoted by RED dot.
You can see the build status in the project home page.

Page 38 of 45
Create User in Jenkins for Remote Job Builds
To automate the process of building, create users in Jenkins
Initial Steps:
1. Go to the Configure Global Security screen (http://server/jenkins/configureSecurity/) and
choose "enable security". An alternate URL to try is http://server:8080/configureSecurity.
2. Select "Jenkins's own user database" as the security realm
3. Place a check mark next to "Allow users to sign up"
4. Select "Matrix-based security" as the authorization
5. Give anonymous user the read access
6. In the text box below the table, type in your user name (you'd be creating this later) and click
"add"
7. Give yourself a full access by checking the entire row for your user name
8. Scroll all the way to the bottom, click "save"
The configuration should look like the picture below:
At this point, you'll be taken back to the top page, and Jenkins is successfully secured.
Restart Jenkins (service jenkins restart)
Now you need to create an user account for yourself.

Page 39 of 45
1. Click "login" link at the top right portion of the page
2. Choose "create an account"
3. Use the user name you've used in the above step, and fill in the rest.
If everything works smoothly, you are now logged on as yourself with full permissions.
If something goes wrong, follow below steps:
One may accidentally set up security realm / authorization in such a way that you may no longer able to
reconfigure Jenkins.
When this happens, you can fix this by the following steps:
1. Stop Jenkins (the easiest way to do this is to kill the servlet container.)
2. Go to $JENKINS_HOME in the file system and find config.xml file.
3. Open this file in the editor.
4. Look for the <useSecurity>true</useSecurity> element in this file.
5. Replace true with false
6. Remove the elements authorizationStrategy and securityRealm
7. Start Jenkins
When Jenkins comes back, it's in the unsecured mode where everyone gets full access to the system.
If this is still not working, trying renaming or deleting config.xml.
We have created user github/github for GitHub to push Jenkins the push information.
In Project Home click on Project Based Security to enable this feature and sample configuration is as
shown below:

Page 40 of 45
In Build Triggers add below informationonProjectHomepage inJenkins

Page 41 of 45
Push GitHub Changes to Jenkins
In order to push any changes such as code commit, branch created those events GitHub can push to
Jenkins.
Go to GitHub project settings:

Page 42 of 45
Add Webhook URL as below format:
http://github:github@<JENKINS_HOST>:<PORT>/jenkins/job/<JOB_NAME>/build?token=<LONG_RAN
DOM_TOKEN_HERE>
 JenkinsHost:Host IP where Jenkins is hosted (should be accessible overinternet)
 Port: Portof Web Server
 Job_Name:Jenkins JobName
 LONG_RANDOM_TOKEN_HERE:Authenticationtokenstring
Here, when any new comment is pushed in GitHub / new branch created, above URL is used to push
information to remote Jenkins.
LONG_RANDOM_TOKEN_HERE: This is the secret token that we have set in above step
We can simulate the setup by committing single change in GitHub.

Page 43 of 45
Showing Jenkins Build Status on GitHub
DownloadJenkins plugin EmbeddableBuildStatus andrestart Jenkins.
Simply clickon the link and screen like below would be shown
Youcan copy MARKDOWNUNPROTECTED linkcontentsand in GitHub create new file called
‘readme.md’ and paste this content.
To get IP address for Jenkins Hosted machine:
(Make sure localhostis to be replaced with host IP)
Host IP can be obtained by command -> ipconfig/all
Check IPv4 Address in console output.

Page 44 of 45
Queries
For any queries or suggestions, please get in touch:
Me@ManojKolhe.com / ManojGKolhe@Gmail.com

Page 45 of 45
THANK YOU

Manoj Kolhe - Setup GitHub with Jenkins on Amazon Cloud - End-to-end Automation

Recommended

More Related Content

What's hot (19)

Similar to Manoj Kolhe - Setup GitHub with Jenkins on Amazon Cloud - End-to-end Automation (20)

More from Manoj Kolhe (14)

Recently uploaded (20)

Manoj Kolhe - Setup GitHub with Jenkins on Amazon Cloud - End-to-end Automation