Jenkins Pipeline Concepts
Jenkins Pipeline Concepts
Pipeline As Code
Topics
• About Me
• Introduction
• Pipeline Basic
• Variables
• parameters
• option sets
• trigger builds
• schedule jobs
• parallel
• post jobs
• tools
• conditional and loop statements
• other examples
• sample maven build
• archive artifacts and finger prints
• uses of credentials option
• checkos_andexecutesteps
• input section
• scm git
• when
• sample local deployment
About Me
Hi there👋
I am Kalaiarasan (GitHub), a DevOps Consultant from 🇮🇳 with 6+ years of experience focusing
majorly on Continous Integration, Continous Deployment, Continous Monitoring and process
development which includes code compilation, packaging , deployment/release and monitoring.🎯
I mostly work with CI/CD Setup, Cloud Services, Developing Automation & SRE tools and
AI/MLops.🚀
Introduction
Pipeline script
• Another way of job configurartion with help of code
Advantages:
• Can divide the jobs into parts (build /test /deploy/..) & each part can run in each agent.
• Parallel execution of stages are easy configure so that we can save time
• Build can hold for user input(specific user can eneter, explain LDAP concept)
Types of Pipeline
• Declarative
• scripted
pipeline {
agent any
stages {
stage('Build') {
steps {
//
}
}
stage('Test') {
steps {
//
}
}
stage('Deploy') {
steps {
//
}
}
}
}
node {
stage('Build') {
//
}
stage('Test') {
//
}
stage('Deploy') {
//
}
}
PIPELINE BASIC
Steps
• We need to write step inside the stage directive
• steps contains (command/scripts) that we used in the build
• One steps directive should be there in stage directive
Stage
stages
• contains sequence of stages
• atleast one stage has tobe there
Agent
• where (master/slave/container..)we need to run our pipeline script
Stage colors
• White (stage is not executed )
• Green (stage is success)
• Blue lines (stage is executing)
• Redlines or red line (stage is failed)
• Red (fews stage success, any one is failed, few remain sucess stage will show red )
Multiline comment: /*
==============
*/
pipeline{
agent any
stages{
stage('Hello_world'){
steps{
echo 'Hello world'
}
}
}
}
O/P
Batch commands
O/P
pipeline{
agent any
stages{
stage('batch'){
steps{
bat """
dir
cd
ping www.google.com
"""
}
}
}
}
O/P
What is variable?
Predefined:
http://localhost:8080/env-vars.html
pipeline{
agent any
stages{
stage('pre'){
steps{
echo " predefined variable $BUILD_NUMBER $WORKSPACE "
}
}
}
}
pipeline{
agent any
environment{
MYHOME="Chennai"
}
stages{
stage('User'){
steps{
echo " userdefined variable $MYHOME "
}
}
}
}
• Global level
• stage level
• script level
Global level
pipeline{
agent any
environment{
MYHOME="Chennai"
}
stages{
stage('User'){
steps{
echo " userdefined variable $MYHOME "
}
}
}
}
Stage level
pipeline{
agent any
stages{
stage('User'){
environment{
MYHOME="Chennai"
}
steps{
echo " userdefined variable $MYHOME "
}
}
}
}
stage('User'){
steps{
script{
MYHOME="Chennai"
}
echo " userdefined variable $MYHOME "
}
}
}
}
pipeline{
agent any
stages{
stage('User'){
steps{
script{
MYHOME="Chennai"
echo " userdefined variable $MYHOME "
}
}
}
}
}
Scope of the Variables : priority order first (script),second(stage),third(global or root)
if you defined the same varaible in global ,stage and , it will pick up stage.
pipeline{
agent any
environment{
MYHOME="Chennai"
}
stages{
stage('User'){
environment{
MYHOME="thiruverkadu"
O/P
if you defined diff values in variable , we can call above stage variable by ${env.variablename}
pipeline{
agent any
environment{
MYHOME="Chennai"
}
stages{
stage('User'){
environment{
MYHOME="thiruverkadu"
}
steps{
script{
MYHOME="chen"
}
echo " userdefined variable $MYHOME previous variable ${env.MYHOME} "
}
}
}
}
pipeline{
agent any
environment{
BUILD_NUMBER="Chennai"
}
stages{
stage('User'){
environment{
MYHOME="thiruverkadu"
}
steps{
script{
MYHOME="chen"
}
echo " used predefined variable $BUILD_NUMBER "
}
}
}
}
Concatenate
process of combining two or more string by '+' operator in jenkins
pipeline{
agent any
environment{
name1="kalai"
name2='arasan'
}
stages{
stage('concatenate'){
steps{
script{
Name= name1 + name2
}
echo " concatenate $Name"
}
}
}
}
Syntax:
stages{
stage('string'){
steps{
echo " string $DEPLOY_ENV"
}
}
stage('text'){
steps{
echo " text $DEPLOY_TEXT"
}
}
stage('booleanParam'){
steps{
script{
if(TOGGLE){
echo " now execute, booleann is true"
}else{
echo " Dont execute, boolean is true"
}
}
}
}
stage('choice'){
steps{
script{
if(DEPLOY_ENV=='staging'){
echo " choice $CHOICE"
}
}
}
}
stage('file'){
steps{
echo " file $FILE"
}
}
stage('password'){
steps{
echo " password $PASSWORD"
}
}
}
}
O/P
Dryrun is mainly used for first time of parameter build, before getting build with parameter.
OPTION SET
}
steps {
sh 'echo hello'
sleep(10)
}
}
}
}
pipeline {
agent any
stages {
stage('Deploy') {
steps {
retry(3) {
sh 'echo hello'
}
}
}
}
}
pipeline {
agent any
options{
retry(3)
}
stages {
stage('Deploy') {
sh 'echo hello'
}
}
}
}
buildDiscarder
pipeline {
agent any
options { buildDiscarder(logRotator(numToKeepStr: '5')) }
stages {
stage('Deploy') {
steps {
sh 'echo hello'
}
}
}
}
disableConcurrentBuilds
if execute the build if it takes time to complete again paralley , we trigger b4 complete the previous build,
again build get start to execute, due to this job will get conflicts with nodes.
pipeline {
agent any
options {
buildDiscarder(logRotator(numToKeepStr: '5'))
disableConcurrentBuilds()
}
stages {
stage('Deploy') {
steps {
sh 'echo hello'
sleep(10)
}
}
}
}
Syntax
pipeline {
agent any
options {
buildDiscarder(logRotator(numToKeepStr: '5'))
disableConcurrentBuilds()
timeout(time: 5, unit: 'SECONDS')
}
stages {
stage('Deploy') {
steps {
sh 'echo hello'
sleep(10)
}
}
}
}
pipeline {
agent any
stages {
stage('Deploy') {
options {
retry(3)
}
steps {
sh 'echo hello'
sleep(10)
}
}
}
}
Timestamp:
pipeline {
agent any
options {
buildDiscarder(logRotator(numToKeepStr: '5'))
disableConcurrentBuilds()
timestamps()
}
stages {
stage('Deploy') {
steps {
sh 'echo hello'
sleep(2)
sh 'echo hi'
sleep(2)
With timestamp
Without timestamp
syntax
pipeline {
agent any
stages {
stage('triggerjob') {
steps {
build('job1')
build('job2')
}
}
}
}
O/P
if we triggering two job, if first job got failed, it wont trigger the second job.so we gng say propagate:false,
so even though job failed, second job will get trigger.
pipeline {
agent any
stages {
stage('triggerjob') {
steps {
build(job:'job1', propagate:false)
build('job2')
}
}
}
}
while using the below function, it will store the status in jobresult, now eventhough job failed, it will run
triffer both job, but it will show unstable result status
syntax
pipeline {
agent any
stages {
stage('triggerjob') {
steps {
script{
jobresult = build(job:'job1', propagate:false).result
build('job2')
if(jobresult=='FAILURE'){
currentBuild.result='UNSTABLE'
}
}
}
O/P
pipeline {
agent any
parameters {
choice(
name: 'Nodes',
choices:"Linux\nMac",
description: "Choose Node!")
choice(
name: 'Versions',
choices:"3.4\n4.4",
description: "Build for which version?" )
string(
name: 'Path',
defaultValue:"/home/pencillr/builds/",
description: "Where to put the build!")
}
stages {
stage("build") {
steps {
script {
echo "$Nodes"
echo "Versions"
echo "Path"
}
}
}
}
}
pipeline {
agent any
stages {
stage("build") {
steps {
script {
build(job: "builder-job",
parameters:
[string(name: 'Nodes', value: "Linux"),
string(name: 'Versions', value: "3.4"),
string(name: 'Path', value: "/home/pencillr/builds/}")])
}
}
}
}
}
Schedule Jobs
pipeline {
agent any
options{
timestamps()
}
triggers{
cron('* * * * *')
}
stages {
stage("cron") {
steps {
Poll SCM- will trigger the job depends up the changes in code, if there is no commit it wont run.
pipeline {
agent any
options{
timestamps()
}
triggers{
pollSCM('* * * * *')
}
stages {
stage("cron") {
steps {
echo "heloo"
git url:"https://github.com/kalaiarasan33/public.git"
}
}
}
}
stages {
stage("cron") {
steps {
echo "step1"
}
steps {
echo "step2"
}
}
}
pipeline {
stages {
stage("build") {
parallel{
stage('job1'){
steps{
echo "job1"
}
}
stage('job2'){
steps{
echo "job2"
}
}
}
}
}
}
Parallel stages:
pipeline {
agent any
options{
timestamps()
}
stages {
stage("stage1") {
parallel{
stage('stage1job1'){
steps{
echo "stage1job1"
sleep(10)
}
}
stage('stage1job2'){
}
}
stage("stage2") {
parallel{
stage('stage2job1'){
steps{
echo "stage2job1"
sleep(5)
}
}
stage('stage2job2'){
steps{
echo "stage2job2"
sleep(5)
}
}
}
}
}
}
failFast
In parallel, eventhough any job is failed, it wont stop, it will execute other job. If we want any job is failed, it
should stop the other build means we need to use failFast
stages {
stage("stage1") {
failFast true
parallel{
stage('stage1job1'){
steps{
echo "stage1job1"
sleep(10)
}
}
stage('stage1job2'){
steps{
eecho "stage1job2"
sleep(10)
}
}
stage('stage2job1'){
steps{
echo "stage2job1"
sleep(5)
}
}
stage('stage2job2'){
steps{
echo "stage2job2"
sleep(5)
}
}
}
}
}
}
POST JOBS
post will execute after the completion of pipeline's stages section contains the following blocks
changed: Runs only if current build status is changed when compare to previous
cleanup : like always, will execute at every time in the last ( if you want to delete any workspace and cleaup
any folder , we can use this)
stages {
stage("stage1") {
steps{
sh "ls -l"
}
post{
always{
echo " action always "
}
changed{
echo " action always Changed from previous state"
}
fixed{
echo " action Fixed when previous state is failure"
}
regression{
echo " action when current state is fail/unstable/aborted , previous state is
success"
}
aborted{
echo " action always aborted"
}
failure{
echo " action always failure"
}
success{
echo " action always success"
}
unstable{
echo " action unstable"
}
cleanup{
echo " action similar like always , it is using to cleanup folder or
workspace"
}
}
}
So Always , change (changes in state from previous state ), fixed (previous buil failed, current passed), all
executed
If you want to run specific version of tools to use in pipeline for specific job, so we using tools.
pipeline{
agent any
tools{
maven 'Maven3.6.1'
}
stages{
stage('tools_version'){
steps{
sh 'mvn --version'
}
}
}
}
pipeline{
agent any
tools{
maven 'Maven3.5.0'
}
stages{
stage('tools_version'){
steps{
sh 'mvn --version'
}
}
}
}
pipeline{
agent any
tools{
maven 'Maven3.6.1'
}
stages{
stage('tools_version'){
steps{
sh 'mvn --version'
}
}
stage('diff_version_stage_level'){
tools{
maven 'Maven3.5.0'
}
steps{
echo "stage level"
sh 'mvn --version'
}
}
}
}
IF Condition:
pipeline{
agent any
environment{
Tools='Jenkins'
}
stages{
stage('conditions'){
steps{
script{
if(Tools == 'Jenkins'){
echo 'Tools is jenkins'
}else{
echo 'Tools is not jenkins '
}
}
}
pipeline{
agent any
environment{
Tools='Jenkins'
}
stages{
stage('conditions'){
steps{
script{
int buildno="$BUILD_NUMBER"
if(buildno %2 == 0){
echo 'builno is even'
}else{
echo 'buildno is odd'
}
}
}
}
}
}
pipeline{
agent any
environment{
Tools='Jenkins'
}
stages{
stage('conditions'){
steps{
script{
for(i=0;i<=5;i++){
println i
}
int buildno="$BUILD_NUMBER"
if(buildno %2 == 0){
echo 'builno is even'
}else{
echo 'buildno is odd'
}
}
}
}
}
}
Other Example
we need to install the plugin first , then set the ansi in configuration , jenkins foreground
pipeline{
agent any
stages{
stage('ansi'){
steps{
ansiColor('xterm') {
echo 'something that outputs ansi colored stuff'
}
}
}
stage('non_ansi'){
steps{
echo 'non_ansi'
}
}
}
}
This is used to define the name for the job and description.
pipeline{
agent any
stages{
stage('buid_name'){
steps{
script{
currentBuild.displayName = "Deployment"
currentBuild.description = "This is deployment build"
}
echo 'build name changing'
}
}
}
}
O/P
create folder inside workspace -> job name -> (creating folder) -> job output is here
pipeline{
agent any
stages{
stage('cleanWS'){
steps{
dir('build_one'){
script{
currentBuild.displayName = "Deployment"
currentBuild.description = "This is deployment build"
}
sh "echo dir creation and delete WS > hello.txt"
}
}
}
}
Creating output in workspace -> jobname -> build_one --> outputfiles --> deleted job workspace
kalai@jenlinux:~/jenkins_home/workspace/Example/delete_WS/build_one/..$ cd ..
kalai@jenlinux:~/jenkins_home/workspace/Example$ ls
change_build_name change_build_name@tmp
kalai@jenlinux:~/jenkins_home/workspace/Example$
pipeline{
agent any
stages{
stage('cleanWS'){
steps{
dir('build_one'){
script{
currentBuild.displayName = "Deployment"
currentBuild.description = "This is deployment build"
}
sh "echo build name changing > hello.txt"
}
cleanWs()
}
}
}
pipeline{
agent any
stages{
stage('write_file'){
steps{
}
}
}
pipeline{
agent any
tools{
maven 'Maven3.6.1'
}
stages{
stage('maven_project'){
steps{
git url:"https://github.com/nkarthikraju/MavenExamples.git"
sh "mvn clean install"
}
}
}
}
without moving into the directory , it will show error , no pom file
pipeline{
agent any
tools{
maven 'Maven3.6.1'
}
stages{
stage('maven_project'){
steps{
git url:"https://github.com/nkarthikraju/MavenExamples.git"
dir('MavenHelloWorldProject'){
sh "mvn clean install"
}
}
}
}
}
pipeline{
agent any
tools{
maven 'Maven3.6.1'
}
stages{
stage('maven_project'){
steps{
Fingerprint:
if we execute the same job 10 times or etc, it will give same name output , for the identificaton or record the
artifacts with fingerprint it will create checksum with build.
pipeline{
agent any
tools{
maven 'Maven3.6.1'
}
stages{
stage('maven_project'){
steps{
git url:"https://github.com/nkarthikraju/MavenExamples.git"
dir('MavenHelloWorldProject'){
sh "mvn clean install"
}
}
post{
success{
archiveArtifacts artifacts:"MavenHelloWorldProject/target/*.jar" , fingerprint:true
}
}
}
}
}
O/P
pipeline{
agent any
environment{
pass="jobs123"
}
stages{
stage('passwd'){
steps{
echo "password $pass"
}
}
}
}
pipeline{
agent any
environment{
pass="jobs123"
}
parameters{
password(name:'entry_password')
}
stages{
stage('passwd'){
steps{
echo "password $pass, password parameter $entry_password"
}
}
}
}
O/P
pipeline{
agent any
environment{
pass="jobs123"
passwod=credentials('DB_PASSWORD')
}
parameters{
password(name:'entry_password')
}
stages{
stage('passwd'){
steps{
echo "password $pass, parameter password $entry_password , credential funtion password
$passwod"
}
}
}
}
CheckOS_AndExecuteSteps
if we execute like below it will show error, so we need to check the os type with of function
stages{
stage('os_type'){
steps{
sh "ls"
bat "dir"
}
}
}
}
pipeline{
agent any
stages{
stage('os_type'){
steps{
script{
if(isUnix()){
sh "ls"
}else{
bat "dir"
}
}
}
}
}
}
pipeline{
agent any
environment{
tools='jenkins'
}
stages{
stage('trimming_string'){
steps{
script{
t1=tools[0..6] //jenkins
t2=tools[3..5] //kin
echo "$t1 , $t2"
}
}
}
}
InPut Section
stages {
stage('build user') {
steps {
wrap([$class: 'BuildUser']) {
sh 'echo "${BUILD_USER}"'
}
}
}
}
}
pipeline{
agent any
stages{
stage('user_input'){
steps{
wrap([$class: 'BuildUser']){
script{
def name1="${BUILD_USER}"
echo "${BUILD_USER}, $name1"
if(name1=='kalai'){
echo "only kalai can able to build"
}else{
echo "others cant able to build"
}
}
}
}
}
}
}
pipeline{
agent any
stages{
stage('user_input'){
steps{
}
}
}
}
we can give list of user permission to proceed, other cant give proceed and get specific string from submitter.
pipeline{
agent any
stages{
stage('user_input'){
input{
message "press ok to continue"
submitter "kalai,lead"
parameters{
string(name:'username1',description: "only kalai and lead has permission")
}
}
steps{
}
}
}
}
SCM GIT
We can check out the jenkinsfile in scm whether it is git or SVN.and can maintain with version control.
We can generate the pipeline syntax from passing the parameter to plugin , then with generate option
This script will read the file and write and push to the git
pipeline{
agent any
stages{
stage('commitcode'){
steps{
cleanWs()
dir('comit_from_jenkins'){
git 'https://github.com/kalaiarasan33/jenkins_commit.git'
script{
oldv=readFile('file.txt')
newv=oldv.tpInteger() + 1
}
writeFile file:"file.txt", text:"$newv"
sh """
git add file.txt
git commit -m "files commited"
git push
"""
}
}
}
}
}
pipeline{
agent any
stages{
stage('commitcode'){
steps{
git 'https://github.com/kalaiarasan33/jenkins_commit.git'
dir('tag_jenkins'){
}
}
}
}
Save Build Numbers which are used for Deployment in the file
pipeline{
agent any
stages{
stage('builnum'){
steps{
git 'https://github.com/kalaiarasan33/jenkins_commit.git'
sh "echo $BUILD_NUMBER >> Deployment.txt"
dir('save_builnum_jenkins'){
sh """
git add file.txt
git commit -m "updating with build num commited"
git push
"""
}
}
}
}
}
O/P
pipeline{
agent any
stages{
stage('svn'){
when{
changelog 'build'
}
steps{
sh """
}
}
}
}
}
WHEN
When is similar to If condition, but its more adavanced with in-built condition
pipeline{
agent any
environment{
Tool="Jenkins"
}
stages{
stage('When_equals'){
when{
equals expected:'Docker' , actual: "$Tool"
}
steps{
sh """
echo " if when equal "
"""
}
}
stage('When_no_equals'){
when{
not {
environment name:Tool , value:"Jenkins"
}
}
steps{
sh """
echo " if when not equal "
"""
}
}
}
}
pipeline{
agent any
environment{
Tool="Jenkins"
}
stages{
stage('When_equals'){
when{
equals expected:'Jenkins' , actual: "$Tool"
}
steps{
sh """
echo " if when equal "
"""
}
}
stage('When_no_equals'){
when{
not {
environment name:Tool , value:"Jenkins"
}
}
steps{
sh """
echo " if when not equal "
"""
}
}
}
}
sh """
echo " hello_world "
"""
}
}
stage('based_on_previous'){
when{
expression {
currentBuild.getPreviousBuild().result =='SUCCESS'
}
}
steps{
sh """
echo " previous build is failled, now sucess"
"""
}
}
}
}
sh """
echo " hello_world "
"""
}
}
stage('based_on_previous'){
when{
expression {
currentBuild.getPreviousBuild().result =='FAILURE'
}
}
steps{
sh """
echo " previous build is failled, now sucess"
"""
}
}
}
}
pipeline{
agent any
stages{
stage('svn'){
when{
changelog 'build'
}
steps{
sh """
echo "found build keyword in the commit, so proceeding futher satge "
"""
}
}
}
}
}
pipeline{
agent any
stages{
stage('pythonfile'){
when{
changeset '*.py'
}
steps{
sh """
echo "commit files contains only for python file "
"""
}
}
sh """
echo " commit files contains only for xml file"
"""
}
}
}
}
}
Allof, Anyof
pipeline{
agent any
environment{
Tool="jenkins"
envv="PRD"
}
stages{
stage('allof'){
when{
allOf{
equals expected: "jenkins" , actual: "$Tool"
equals expected: "PRD" , actual: "$envv"
}
steps{
sh """
echo " when both condition is pass "
"""
}
}
sh """
echo " when any one condition is pass"
"""
}
}
}
}
pipeline{
agent any
stages{
stage('string_in_file'){
when{
expression{ return readFile('C:\\Users\\user\\Desktop\\cloudguru_sysops\\files.txt').contains('truncated')}
}
steps{
sh """
echo " string in file "
"""
}
}
}
}
stages{
stage('string_in_file'){
when{
return false
expression{ return readFile("C:\\Users\\user\\Desktop\\cloudguru_sysops\\files.txt").contains('truncated')}
}
steps{
sh """
echo " string in file "
"""
}
}
}
}
SHELL
pipeline{
agent any
stages{
stage('shell'){
steps{
sh """ ls -l
pwd
"""
}
}
}
}
pipeline{
}
}
}
}
pipeline{
agent any
stages{
stage('html'){
steps{
bat """
echo hello this is my HTML >> "D:\\"
"""
}
}
stage('copy_location'){
steps{
bat """
copy *.html D:\\tomcat\\
"""
}
}
}
}
}