FOSDEM 2017: GitLab CI

GitLab CI
Open Source Continuous Integration made easy

Who am I?
Walter Heck
CTO at OlinData since 2008
Background: Delphi, MySQL, SysOps
Love: Automation, DevOps, FOSS

Who are you?
Anyone using GitLab self hosted?
Github Enterprise
Atlassian Bitbucket / Bamboo?
Jenkins?

What is GitLab
GitLab is a web-based Git repository manager with wiki and
issue tracking features, using an open source license,
developed by GitLab Inc.

What is GitLab CI
Gitlab CI is a Continuous Integration service fully integrated with GitLab. Due to
it’s tight integration many tasks related to CI and CD become much easier to
manage.
GitLab CI is fully open source and included in any GitLab CE installation. Check in
a .gitlab-ci.yaml file to get started :)

GitLab CI Runners
In GitLab CI, Runners run your yaml. A runner is an isolated (virtual) machine that
picks up builds through the coordinator API of GitLab CI.
A runner can be specific to a certain project or serve any project in GitLab CI. A
runner that serves all projects is called a shared runner.

Docker runner
Runs GitLab CI builds inside one or more docker containers
● Easiest to get started (works out of the box)
○ For production usage look into docker in production (in particular storage drivers)
● ‘Abuses’ containers as throw-away VMs
○ Feels strange at first, perfect after you get used to it

Starting .gitlab-ci.yaml
#.gitlab-ci.yaml
image: ruby:2.2
test:
script:
- bundle install
- bundle exec rake lint
- bundle exec rake syntax
● Choose a docker image
(https://hub.docker.com/_/ruby)
● Simple job called test
○ Runs two rake tasks
● Runs on every push + MR on every
branch that contains a .gitlab-ci.yaml
○ May be different between branches

Jobs
Jobs can be run:
● locally
● using Docker containers
● using Docker containers and executing job over
SSH
● using Docker containers with autoscaling on
different clouds and virtualization hypervisors
● connecting to remote SSH server
Why multiple jobs?
● The smaller a job the easier parallelization across
multiple runners is
● Smaller jobs are easier to diagnose when they
break
image: ruby:2.2
before_script:
- bundle install
job:test:syntax:
script:
job:test:lint:
script:

Artifacts/dependencies
Allows for transferring data/files from one job to
the next within the same build
● Why run bundle install every time?
● Artifacts are passed between jobs
○ Make sure to set expiration (default = 1
day)
● Dependencies determine a job can only
run after some other job
image: ruby:2.2
job:build:artifacts:
script:
- bundle install --deployment
artifacts:
expire_in: 1 day
paths:
- vendor/
job:test:syntax:
script:
dependencies:
- job:build:artifacts
job:test:lint:
script:
dependencies:

Stages
The ordering of elements in stages defines the ordering of
builds' execution:
● Jobs of the same stage are run in parallel.
● Jobs of the next stage are run after the jobs from
the previous stage complete successfully.
image: ruby:2.2
stages:
- build
- test
- deploy
job:build:artifacts:
stage: build
script:
artifacts:
expire_in: 1 day
paths:
- vendor/
job:test:syntax:
stage: test
script:
dependencies:
job:test:lint:
stage: test
script:
dependencies:

Limiting builds
● Sometimes we don’t want certain jobs to
always run
○ Usually when they take lots of resources
(time, compute)
○ Or when they do things like deploy to
staging/production
● Use only
○ Defines a list of git refs for which build is
created
● Or use except
○ Defines a list of git refs for which build is not
created
job_name:
script:
- rake spec
- coverage
stage: test
only:
- master
except:
- develop
tags:
- ruby
- postgres
allow_failure: true

Selecting specific runners
tags is used to select specific Runners from the list of all Runners that are allowed to run this project.
During the registration of a Runner, you can specify the Runner's tags, for example ruby, postgres, development.
tags allow you to run builds with Runners that have the specified tags assigned to them:
job:
tags:
- ruby
- postgres
The specification above, will make sure that job is built by a Runner that has both ruby AND postgres tags defined.

Manual build
Introduced in GitLab 8.10.
Manual actions are a special type of job that are not executed automatically; they need to be explicitly started by a user. Manual
actions can be started from pipeline, build, environment, and deployment views. You can execute the same manual action multiple
times.
An example usage of manual actions is deployment to production.

Secret variables
GitLab CI allows you to define per-project secret variables that are set in the build environment. The secret variables are stored out of
the repository (.gitlab-ci.yml) and are securely passed to GitLab Runner making them available in the build environment. It's the
recommended method to use for storing things like passwords, secret keys and credentials.
Secret variables can be added by going to your project's Settings ➔ Variables ➔ Add variable.
Once you set them, they will be available for all subsequent builds.
Be aware that secret variables are not masked,
and their values can be shown in the build logs if
explicitly asked to do so. If your project is public
or internal, you can set the pipelines private from
your project's Pipelines settings. Follow the
discussion in issue #13784 for masking the
secret variables.

GitLab currently doesn't have built-in support for managing
SSH keys in a build environment.
The SSH keys can be useful when:
1. You want to checkout internal submodules
2. You want to download private packages using
your package manager (eg. bundler)
3. You want to deploy your application to eg. Heroku
or your own server
4. You want to execute SSH commands from the
build server to the remote server
5. You want to rsync files from your build server to
the remote server
Private repos, deploy servers, SSH Keys
How it works
1. Create a new SSH key pair with ssh-keygen
2. Add the private key as a Secret Variable to the project
3. Run the ssh-agent during build to load the private key.
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *ntStrictHostKeyChecking nonn" >
~/.ssh/config'
Test SSH:
script:
- ssh git@gitlab.com
- git clone git@gitlab.com:gitlab-examples/ssh-private-key.git

YAML anchors
YAML also has a handy feature called 'anchors', which
let you easily duplicate content across your document.
.job_template: &job_definition # Hidden key that defines an anchor
named 'job_definition'
image: ruby:2.1
services:
- postgres
- redis
test1:
<<: *job_definition # Merge the contents of the
'job_definition' alias
script:
- test1 project
test2:
<<: *job_definition # Merge the contents of the
'job_definition' alias
script:
- test2 project
.job_template:
image: ruby:2.1
services:
- postgres
- redis
test1:
image: ruby:2.1
services:
- postgres
- redis
script:
- test1 project
test2:
image: ruby:2.1
services:
- postgres
- redis
script:
- test2 project

Questions?
@olindata / @walterheck
walterheck@olindata.com
www.linkedin.com/in/walterheck

FOSDEM 2017: GitLab CI

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to FOSDEM 2017: GitLab CI (20)

More from OlinData (20)

Recently uploaded (20)

FOSDEM 2017: GitLab CI