A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker

Paul Bonaud
@paulrb_r
Maxime Visonneau
@mvisonneau
A
A
A SMOOTH
SMOOTH
SMOOTH MIGRATION TO
MIGRATION TO
MIGRATION TO
DOCKER
DOCKER
DOCKER FOCUSING ON
FOCUSING ON
FOCUSING ON
BUILD PIPELINES
BUILD PIPELINES
BUILD PIPELINES

Paul Bonaud
⌨ + ⚙ +
▶ Infra & Ops Engineer
⏸ Software Developer
Maxime Visonneau
⌨ + +
Infrastructure Engineer

Meet Maurice
Meet Maurice
Meet Maurice

Meet Maurice
Meet Maurice
Meet Maurice
aka
aka
aka build2
build2
build2

Building all projects
Android, JS (Ember), C++, Ruby

Deploying everything
Nginx, apps, infrastructure

Single architecture (Debian 7)

Version controlled
Dockerfile

Dockerfile
FROM ruby:2.4-jessie

Dockerfile
RUN apt-get update
RUN apt-get -y install libpq-dev ghostscript

Dockerfile
RUN apt-get update &&
apt-get -y install libpq-dev ghostscript &&
rm -rf /var/lib/apt/lists/*

Dockerfile
WORKDIR /opt/app
COPY Gemfile Gemfile.lock /opt/app
RUN bundle install

Dockerfile
VOLUME /opt/app
CMD [ "make", "run" ]

Dockerfile
FROM ruby:2.4-jessie
RUN apt-get update &&
apt-get -y install libpq-dev ghostscript &&
rm -rf /var/lib/apt/lists/*
WORKDIR /opt/app
COPY Gemfile Gemfile.lock /opt/app
RUN bundle install
VOLUME /opt/app
CMD [ "make", "run" ]

Integrated Docker registry!
registry.gitlab.com

Version controlled
.gitlab-ci.yml

.gitlab-ci.yml
stages:
- build
- test
- package
- deploy

.gitlab-ci.yml
variables:
IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
LATEST_IMAGE: ${CI_REGISTRY}/${CI_PROJECT_PATH}:latest

.gitlab-ci.yml
build:docker:
stage: build

.gitlab-ci.yml
tags: [ privileged ]
image: "docker:latest"

.gitlab-ci.yml
before_script:
- docker login -u gitlab-ci-token
-p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${LATEST_IMAGE} || true
- docker build --cache-from ${LATEST_IMAGE}
-t ${IMAGE} -t ${LATEST_IMAGE} .
- docker push ${IMAGE}
- docker push ${LATEST_IMAGE}

.gitlab-ci.yml
image: ${IMAGE}

.gitlab-ci.yml
test:
stage: test
script:
- make test

.gitlab-ci.yml
package:
stage: package
script:
- make package > release.tar.gz
artifacts:
paths: [ release.tar.gz ]

.gitlab-ci.yml
deploy:integration:
stage: deploy
image: ${CI_REGISTRY}/infra/ansible:latest
variables:
ENV: integration
script:
- make deploy file=release.tar.gz env=${ENV}

.gitlab-ci.yml
deploy:integration: &deploy
stage: deploy
variables:
ENV: integration
script:
- make deploy file=release.tar.gz env=${ENV}
deploy:production:
<<: *deploy
when: manual
variables:
ENV: production

WHAT ABOUT EASY
WHAT ABOUT EASY
WHAT ABOUT EASY OS
OS
OS TESTING AND UPGRADES?!
TESTING AND UPGRADES?!
TESTING AND UPGRADES?!

Dockerfile
FROM ruby:2.4-stretch

.gitlab-ci.yml
variables:
IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
# (...)

.gitlab-ci.yml
variables:
IMAGE: ${CI_REGISTRY_IMAGE}:${OS}_${CI_COMMIT_SHA}
# (...)

.gitlab-ci.yml
variables:
# (...)
test:jessie:
stage: test
variables:
OS: jessie
script:
- make test

.gitlab-ci.yml
variables:
# (...)
test:jessie: &test
stage: test
variables:
OS: jessie
script:
- make test
test:stretch:
<<: *test
variables:
OS: stretch

.gitlab-ci.yml
package:jessie: &package
stage: package
variables:
OS: jessie
script:
- make package > release-${OS}.tar.gz
artifacts:
paths: [ release-${OS}.tar.gz ]
package:stretch:
<<: *package
variables:
OS: stretch

.gitlab-ci.yml
deploy:integration: &deploy
stage: deploy
variables:
OS: stretch
ENV: integration
script:
- make deploy file=release-${OS}.tar.gz env=${ENV}
deploy:production:
<<: *deploy
when: manual
variables:
OS: jessie
ENV: production

Different Executors / Multiple OSes
Shell
Docker
VirtualBox
Parallels
SSH
Kubernetes
Linux
Windows
Mac OS X
FreeBSD

Easily configurable and maintainable

---
classes:
- docker
- gitlab::ci
gitlab::ci::runners:
"%{::fqdn}_docker":
executor: docker
docker-image: docker:latest

---
classes:
- docker
- gitlab::ci
"%{::fqdn}_docker":
executor: docker
docker-image: docker:latest
docker-volumes: "/var/run/docker.sock:/var/run/docker.sock"
docker-privileged: true

terraform.tf
resource "aws_instance" "gitlab_runner" {
count = 10
ami = "ami-6b2cd712"
instance_type = "m4.large"
[..]
}

MISSINGDEP
MISSINGDEP
MISSINGDEP
Who had any clue this system library was required?

MISSINGDEP
MISSINGDEP
MISSINGDEP
Update Dockerfiles with missing dependencies

ENOSPC
ENOSPC
ENOSPC
No space left on device

ENOSPC
ENOSPC
ENOSPC
Increase space / Automate cleanup of old containers and
images

UNAUTH_DEPLOY
UNAUTH_DEPLOY
UNAUTH_DEPLOY
Where are my SSH keys for Ansible?

UNAUTH_DEPLOY
UNAUTH_DEPLOY
UNAUTH_DEPLOY
Enhance security by creating individual keys per
project/env

SHM_OOM
SHM_OOM
SHM_OOM
Less than 64MB of free space in temporary directory for
shared memory files

SHM_OOM
SHM_OOM
SHM_OOM
Configure max SHM size

IOWAIT
IOWAIT
IOWAIT
On heavy load tests

IOWAIT
IOWAIT
IOWAIT
Bigger disks
Provisioned IOPS
Leverage tmpfs

terraform.tf
resource "aws_autoscaling_group" "gitlab_runner" {
desired_capacity = 10
min_size = 4
max_size = 24
launch_configuration = "${aws_launch_configuration.runner.name}"
[..]
}

We can do better and easier!

Ourselves
Ourselves
Ourselves
with an interesting challenge

Software engineers
Software engineers
Software engineers
with a reliable CI implementation

Finance
Finance
Finance
by making some savings

docker-machine integrated
out-of-the-box with gitlab-runner
scheduling jobs on spot instances

'aws_spot_docker_machine':

executor: docker+machine

limit: 24

limit: 24
machine-IdleCount: 4
machine-IdleTime: 600

limit: 24
machine-OffPeakPeriods:
- "* * 0-8,18-23 * * mon-fri *"
- "* * * * * sat,sun *"
machine-OffPeakIdleCount: 0
machine-OffPeakIdleTime: 600

limit: 24
- "* * 0-8,18-23 * * mon-fri *"
- "* * * * * sat,sun *"
machine-MaxBuilds: 30

limit: 24
- "* * 0-8,18-23 * * mon-fri *"
- "* * * * * sat,sun *"
machine-MaxBuilds: 30
machine-MachineDriver: amazonec2
machine-MachineOptions:
- amazonec2-instance-type=m4.large
- amazonec2-request-spot-instance=true
- amazonec2-spot-price=0.10
[..]

A small drawback..
A small drawback..
A small drawback..

Scheduling
Scheduling
Scheduling

Kubernetes
Kubernetes
Kubernetes

---
apiVersion: v1
kind: ConfigMap
metadata:
name: gitlab-runner
namespace: gitlab-ci
data:
config.toml: |
[[runners]]
name = "k8s_runner"
url = "https://gitlab.example.com/"
executor = "kubernetes"
[runners.kubernetes]
cpu_limit = "1"
memory_limit = "256Mi"
[..]

~$ kubectl -n gitlab-ci get po
NAME READY STATUS RESTARTS AGE
runner-315e4d80-0qj79g 2/2 Running 0 3m
runner-315e4d80-9kdkee 2/2 Running 0 2m
runner-315e4d80-kdfdfe 2/2 Running 0 1m
runner-315e4d80-ldorpk 2/2 Running 0 1m
runner-315e4d80-pleofi 2/2 Running 0 20s

Definitely promising

Still a work in progress for us

QUESTIONS?
QUESTIONS?
QUESTIONS?

REFERENCES
REFERENCES
REFERENCES
· Dockerfile documentation
· .gitlab-ci.yml documentation
· Slides
https://docs.docker.com/engine/reference/builder/
https://gitlab.com/help/ci/yaml/README.md
https://paulrbr.gitlab.io/ci-migration-slides/

A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker

Recommended

More Related Content

What's hot (19)

Similar to A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker (20)

More from The Incredible Automation Day (20)

Recently uploaded (20)

A smooth migration to Docker focusing on build pipelines - TIAD Camp Docker