Continuous Integration Made Easy:

An Introduction with GitLab

Koundinya N V S S, Senior Data Scientist
Agenda

 GitLab CI Basics
 Overview of Continuous Integration (CI)
 Building Jobs in GitLab CI
 Creating and configuring GitLab CI jobs
 Triggering builds and managing dependencies
 GitLab CI Pipeline
 Introduction to GitLab CI Pipeline
 Creating and managing GitLab CI Pipeline jobs
Understanding DevOps

 DevOps – Development + Operations

 A set of practices and cultural values that ensures collaboration between development and operations teams.
 Why?
 To make software production and deployment faster in a repeatable and automated way in a cross-functional
environment

Source: https://www.simplilearn.com/tutorials/devops-tutorial/continuous-integration
Understanding DevOps – Continuous Methodologies

 How?
 Using Agile – an iterative approach
 Continuous Methodologies

 Understanding Continuous Methodologies

 Continuous Integration (CI) – Automated testing and artifact creation
 Continuous Delivery (CD) – Automated deployment to test and staging environments; Manual deployment to production
 Continuous Deployment (CD) – Automated deployment to production
 Continuous Monitoring – Automated monitoring and alert mechanisms
Continuous Integration

 For every change pushed to the repository, the application is built, tested, and deployed for review
automatically and continuously.
Continuous Integration

Why?
 Detect development problems soon
 Reduce risks of cost, schedule, and budget
 Find and remove bugs earlier
 Deliver new features and get user feedback more rapidly
How?
 Maintain a single source repository
 Automate the build
 Make your build self-testing
 Keep the build on the CI machine
 Pace up the build with periodic commits
 Test in a clone of the production (staging) environment
 Make it easy for everyone to get the latest executable
 Make the process transparent for everyone
Continuous Delivery/Deployment

 CI/CD: For every change pushed to the repository, the application is built, tested, and deployed for
production automatically and continuously.
Continuous Delivery/Deployment

Why?
 Reduce deployment risks
 Change the version in production more rapidly
 Get the feedback earlier

How?
 Continuously integrating the software done by the development team
 Run automated tests
 Push build to the production-like environment
 Can release one version at the push of the button
Understanding DevOps – Summary of Continuous Methodologies

Source: https://biplus.com.vn/ci-cd-devops/
Software/Packages for Continuous Integration

 GitLab GitLab Jenkins

 Jenkins  Pros  Pros
 Circle CI  Parallel builds  Highly customizable

 Travis CI  Native Docker support  Community. A lot of resources and tutorials

 Fully Integrated in Gitlab  Supports multiple version control systems
 Bamboo
 Configurability per branch already on jobs-level  Cross-platform
 Spinnaker
 Permission inheritance (Repository Manager)  Build-in time based execution
 AWS Code Pipeline
 Multi-language  UI and Dashboard
 Azure DevOps
 One application  Report integration
 Cons  Cons
 Integrated only in Gitlab  Poor quality of some plug-ins
 High overhead (setup and host)
Why GitLab? GitLab Features

Source: https://about.gitlab.com/blog/2023/07/27/gitlab-flow-duo/
CI/CD Life Cycle

Source: https://www.inviggo.com/blog/an-insiders-guide-to-ci-cd-for-mobile-app-development-tools-expert-tips
GitLab CI/CD Common Terms

 The `.gitlab-ci.yml` file

 Jobs and Stages
 Runners
 Pipelines
 CI/CD variables
 CI/CD components
The `.gitlab-ci.yml` File

 contains the configuration for your CI/CD pipeline

Example
 usually present at the root of your project
 can name this file anything you want, but `.gitlab-ci.yml` is the most common name
 In this file, you can define:
 The tasks you want to complete, for example, test and deploy your application
 Other configuration files and templates you want to include
 Dependencies and caches
 The commands you want to run in sequence and those you want to run in parallel
 The location to deploy your application to
 Whether you want to run the scripts automatically or trigger any of them manually
Jobs and Stages
Jobs Example
Jobs define “what to do”
 Building blocks of GitLab CI Pipelines
 Executed by Runners
 Can be configured with scripts, commands, and artifacts
 must contain at least the ‘script’ clause
Stages define “when to run”
 Help organize jobs sequentially to create GitLab CI Pipelines
Stages Example
 represent different phases of the software delivery process
 Multiple jobs in the same stage are executed in parallel
 If all jobs in a stage succeed, the pipeline moves on to the next stage
 If any job in a stage fails, the next stage is not (usually) executed and the pipeline ends early
Why multiple jobs?
 The smaller a job the easier parallelization across multiple runners is
 Smaller jobs are easier to diagnose when they break
Pipelines

 Collections of jobs and stages that define the software delivery How to trigger pipelines?
process
 Manually: Trigger on demand for specific
 Each pipeline represents a complete build, test, and deployment builds or deployments.
cycle
 Automatically: Trigger on push/merge
 Executed automatically when triggered according to defined events, schedule, or API calls.
conditions
 Conditional triggering: Use rules to
specify when to trigger pipelines based on
specific conditions.

Types of pipelines
 Basic  Merged results
 Directed Acyclic  Merge trains
Graph (DAG)
 Parent-child
 Merge request
 Multi-project
CI/CD Variables

 Control the behavior of jobs and pipelines Example

 Store values you want to re-use
 Avoid hard-coding values (such as credentials, API keys, or other
secrets) in your `.gitlab-ci.yml` file
Why CI/CD variables?
 Easy configuration changes without modifying pipeline code
 Enhance security and maintainability by centralizing access control
 Dynamic configuration based on environment or user needs
 Simplified collaboration and sharing between team members

To make variables more secure, consider

using external secrets and file type variables

Image Source: https://about.gitlab.com/blog/2021/04/09/demystifying-ci-cd-variables/

CI/CD Components

 Reusable units of pipeline configuration that encapsulate specific functionalities

 Promote modularity, reusability, and maintainability in complex pipelines
 Can be shared across projects and repositories within the GitLab instance
 Can take input parameters for dynamic behavior and customization

Why CI/CD Components?

 Reduce code duplication and boilerplate code in pipelines
 Encourage collaboration and sharing of reusable components within teams
 Enable centralized management and updates for commonly used functionalities
 Simplify complex pipelines by breaking them down into smaller, manageable units
Runners

 Software agents that execute CI/CD jobs on the infrastructure

 Runners build, test, and deploy jobs as instructed by GitLab CI
 Can be self-hosted on your servers, cloud VMs, or containers
 An executor is a tool for the runner that determines the environment each job runs in

Runner Installations Supported Executors

• SSH • Docker
• Shell • Docker
• Parallels Autoscaler (Beta)
• VirtualBox • Docker Machine
• Kubernetes (auto-scaling)
• Instance (Beta) • Custom
CI/CD YAML Syntax Reference

GitLab CI/CD pipeline configuration includes

 Global Keywords – configures pipeline behaviour
 Header Keywords – define specifications for
external configuration files
 Job Keywords – define the behaviour of individual
jobs within a pipeline

How to set up a GitLab CI/CD Pipeline?

Step 1: Define what to run
Step 2: Define where to run
Step 3: Give it a go!
CI/CD Basic Pipeline Example

Source: https://docs.gitlab.com/ee/ci/pipelines/pipeline_architectures.html#basic-pipelines