DCEU 18: Building Your Development Pipeline
1 like1,589 views
This document discusses building a development pipeline using containers. It outlines using containers for building images, automated testing, security scanning, and deploying to production. Containers make environments consistent and reproducible. The pipeline includes building images, testing, security scanning, and promoting images to production. Methods discussed include using multi-stage builds to optimize images, leveraging Buildkit for faster builds, and parallel testing across containers. Automated tools are available to implement rolling updates and rollbacks during deployments.
![Building Performance - MultiStage Builds
Multi-stage builds in your Dockerfiles. Introduced in Docker CE 17.05.
FROM golang:1.7.3 AS BUILDER
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app
.
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=BUILDER /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]](https://image.slidesharecdn.com/dceu18buildingyourdevelopmentpipeline-181214184701/85/DCEU-18-Building-Your-Development-Pipeline-19-320.jpg)
![In practice: integration testing patterns
docker-compose.yml docker-compose.test.yml
version: '3'
services:
vote:
build: ../vote/
ports: ["80"]
depends_on:
- redis
- db
networks:
- front-tier
- back-tier
result:
...
worker:
...
redis:
...
db:
...
version: '3'
services:
test:
build: ./tests/
depends_on:
- vote
- result
- worker
vote:
...
result:
...
worker:
...
redis:
...
db:
...
Create new one-off application
environment
Create service to run integration tests](https://image.slidesharecdn.com/dceu18buildingyourdevelopmentpipeline-181214184701/85/DCEU-18-Building-Your-Development-Pipeline-29-320.jpg)
![How do I control what runs in my cluster??
How do I stop my developers running unknown images from the internet on my
production clusters?
$ kubectl apply -f exampleapp.yaml
Error from server (Forbidden): error when creating "exampleapp.yaml": pods
"nginx" is forbidden: one or more container images do not meet the required
signing policy: [nginx: image did not meet required signing policy]
$ docker run nginx:latest
docker: Error response from daemon: image did not meet required signing
policy.](https://image.slidesharecdn.com/dceu18buildingyourdevelopmentpipeline-181214184701/85/DCEU-18-Building-Your-Development-Pipeline-43-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to DCEU 18: Building Your Development Pipeline (20)
Ad
More from Docker, Inc. (20)
Ad
Recently uploaded (20)
DCEU 18: Building Your Development Pipeline
- 3. Agenda ● Why use containers for your pipeline ● Building images in build agents ● Automated testing with containers ● Security and scanning ● Deploying to production
- 4. What is a pipeline? A set of processes to make software development more efficient, secure, and high-quality. We want to deploy a containerized image. Our pipeline includes: ● Building the image ● Automated testing ● Security scanning ● Promoting the image and deploying to production
- 6. Why use containers for your pipeline?
- 7. Containers make it easy to create consistent, reproducible environments because your environment is declared in a Dockerfile. You know exactly what’s running, where, and can modify and reproduce environments easily It also allows for efficiency by sharing some artifacts between dev, test, and prod. It’s about what’s INSIDE the container...
- 8. Since containers are lightweight, isolated, and fast to boot, they enable different workflows that are a great fit for your pipelines ● Fanning out to run large tasks across multiple containers ● Parallelizing workflows You also have the extra benefit of using common tooling to set up tests, which reduces the cognitive overhead and allows developers to be more autonomous. ...And what goes on OUTSIDE in systems and workflows
- 9. But there are still no shortcuts! Certain things will be made easier, but Docker can’t do the work for you. It’s still up to you to: ● Follow 12-factor app guidelines like pinning dependencies ● Pay attention to size of images and understand what’s in them ● Perform security and vulnerability scans
- 10. Building Images in a Pipeline
- 11. • We size our build agents for that 1 job that requires a lot of CPU, the rest of the time they are pretty idle… • We don’t standardize our tools, so I need EVERYTHING on EVERY build agent… Build Agents Today...
- 12. • We size our build agents for that 1 job that requires a lot of CPU, the rest of the time they are pretty idle… On Demand Build Agents • We don’t standardize our tools, so I need EVERYTHING on EVERY build agent… A Build Agent Container Image for everyone! :) But is this Secure? Build Agents Today...
- 13. Containerised Build Agents - Docker in Docker $ docker run --privileged --name dind -d docker:18.09.0-dind $ docker run -it --rm --link dind:docker docker:18.09.0 sh / # docker images REPOSITORY TAG IMAGE ID CREATED SIZE / # docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
- 14. $ docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock docker:18.09.0 sh / # docker images REPOSITORY TAG IMAGE ID CREATED SIZE olly/jenkins-slave 1 fae1591c6584 3 hours ago 628MB nginx latest 62f816a209e6 6 days ago 109MB openjdk 8-stretch 954739b8bdfb 2 weeks ago 624MB golang latest 45e48f60e268 4 weeks ago 777MB / # docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b2e07edbd47e jenkins/slave:3.27-1 "sh" 2 hours ago Up 2 hours optimistic_chandrasekhar 86f77c2b67f0 openjdk:8-stretch "sh" 3 hours ago Up 3 hours distracted_mcnulty Containerised Build Agents - Mounted Socket
- 15. $ docker run -it --rm -v .pipedocker_engine:.pipedocker_engine docker:18.09.0 cmd C:>docker images REPOSITORY TAG IMAGE ID CREATED SIZE openjdk 1803 2f91c65915d9 2 weeks ago 5.2GB dtr.az.olly.dtcntr.net/admin/openjdk 1803 2f91c65915d9 2 weeks ago 5.2GB docker 18.09.0 629e0258a222 2 weeks ago 5.11GB microsoft/aspnet 4.7.2-windowsservercore-1803 cbdbd42e5a14 7 weeks ago 5.46GB mcr.microsoft.com/windows/servercore 1803 1a4a9d0fd8af 7 weeks ago 4.93GB C:>docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a7911d0ff315 docker:18.09.0 "cmd" 9 hours ago Up 9 seconds priceless_franklin Containerised Build Agents - Mounted Pipe
- 16. Docker in Docker - “--privileged” - The Container can do almost everything the host can do :( Mounted sock - “-v /var/run/docker.sock:/var/run/docker.sock” - The container controls your docker host. Any security applied to the socket has just been bypassed :( Containerised Build Agents - Is this Secure?
- 17. Docker in Docker - “--privileged” - The Container can do almost everything the host can do :( • Rootless Docker? • Standalone building daemon? Mounted sock - “-v /var/run/docker.sock:/var/run/docker.sock” - The container controls your docker host. Any security applied to the socket has just been bypassed :( • Dedicated Build Host? • Dedicated Build Cluster? Containerised Build Agents - Is this Secure?
- 18. • We size our build agents for that 1 job that requires a lot of CPU, the rest of the time they are pretty idle… On Demand Build Agents • We don’t standardize our tools, so I need EVERYTHING on EVERY build agent… A Build Agent Container Image for everyone! :) How do I optimise my Containerised Builds Agents? Build Agents Today...
- 19. Building Performance - MultiStage Builds Multi-stage builds in your Dockerfiles. Introduced in Docker CE 17.05. FROM golang:1.7.3 AS BUILDER WORKDIR /go/src/github.com/alexellis/href-counter/ RUN go get -d -v golang.org/x/net/html COPY app.go . RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app . FROM alpine:latest RUN apk --no-cache add ca-certificates WORKDIR /root/ COPY --from=BUILDER /go/src/github.com/alexellis/href-counter/app . CMD ["./app"]
- 20. Building Performance - MultiStage Builds Multi-stage builds in your Dockerfiles. Introduced in Docker CE 17.05. • Dramatically Reducing the Size of your Container Images • Hardening your Container Images as all unnecessary tools are removed for the Runtime Image • The whole Build process defined in 1 Dockerfile. Check out Tibor and Sebastiaans’s “Dockerfile Best Practices” 5:25pm Today, Tuesday the 4th
- 21. Building Performance - Buildkit Buildkit - A Brand new Container builder introduced in Docker CE 18.06, now available to everyone in Docker CE / EE 18.09. No side-effects Quality, performance, extensibility Parallel requests support Concurrent solver Easy storage management (GC) Custom language support Custom Dockerfile features Incremental build context sending Persistent cache checksums Secure git cache roots HTTP dependency caching Multiple inputs/outputs Nested invocations Multi-format export OpenTracing support Signable image manifests Automatic build context detection Remote build cache Skip unused stages Per-host registry authentication
- 22. Building Performance - Buildkit Based on github.com/moby/moby Dockerfile, master branch. Smaller is better. Time for full build from empty state 2.0x faster Measured on DO 4vcpu droplet
- 23. Repeated build with matching cache 7.2x faster Measured on DO 4vcpu droplet Building Performance - Buildkit Based on github.com/moby/moby Dockerfile, master branch. Smaller is better.
- 24. Checkout Tonis and Ian’s “Supercharged Docker Build with BuildKit” 12:00pm Wednesday the 5th Building Performance - Buildkit Based on github.com/moby/moby Dockerfile, master branch. Smaller is better. Repeated build with matching cache 7.2x faster Measured on DO 4vcpu droplet
- 27. Build once, use everywhere Treat your Dockerfile as a shared artifact that can before different types of testing during all phases in your development process ● local testing during development ● standalone unit tests ● browser and integration tests ● promotion to staging or QA environments
- 28. Maintain flexibility in each environment ● Dev and test environments can be very different ○ Reuse the Dockerfile (or a shared base image, if that makes sense) ○ Create a specific docker-compose.yml file with your testing environment ○ Long(er)-running dev environment can coexist alongside ephemeral test environments
- 29. In practice: integration testing patterns docker-compose.yml docker-compose.test.yml version: '3' services: vote: build: ../vote/ ports: ["80"] depends_on: - redis - db networks: - front-tier - back-tier result: ... worker: ... redis: ... db: ... version: '3' services: test: build: ./tests/ depends_on: - vote - result - worker vote: ... result: ... worker: ... redis: ... db: ... Create new one-off application environment Create service to run integration tests
- 30. Configurations can be reused with many tools OSS Jenkins Jenkins X Hosted SaaS Circle CI CodeShip Travis CI Azure DevOps Supported on-prem CloudBees Core (Jenkins) Circle CI Bamboo TeamCity GitLab and plenty more!
- 31. Parallel Testing with Docker ● Theory: employ task parallelism to split work across parallel computers (containers) ○ Think of your container as just one process, and split testing loads across processes ○ Improve performance on-demand by adding more containers ○ Manage environments simply with Docker ecosystem tools
- 32. Parallel Testing with Docker ● In practice: most CI tools do this for you i.e. declarative pipelines in Jenkinsfile, CodeShip steps, GitLab ● Use cases ○ Test against a matrix of versions ○ Cross-compile on Linux and Windows ○ Run integration tests against different browsers ● Caution: parallelism is great for testing, but not deploying.
- 33. Example: Windows & Linux Builds in Jenkins pipeline { agent none stages { stage("build and deploy on Windows and Linux") { parallel { stage("windows") { agent { label "windows" } stages { stage("build") {} stage("deploy") {} } } stage("linux") { agent { label "linux" } stages { stage("build") {} stage("deploy") {} } } } } } }
- 34. Example: Selenium Grid Selenium Hub Firefox Ubuntu Chrome MacOS Safari iOS Safari MacOS Chrome Windows
- 36. Security question Q. I’ve downloaded all these containers, how do I know what's inside them? What happens if there is out of data packages in there? How do I know what all the Vulnerabilities are exposed? A. It's fine….. They came from the DockerHub and the Dockerfile looks ok…..
- 37. The Old World Host Operating System, Kernel…. Java, Python, .NET App1 App3App2Devs Ops
- 38. The New World Host Operating System, Kernel…. App1 App3App2 Devs Java .NetPython Ops Who’s giving this TLC?
- 39. Maybe we can check the Dockerfile? # Pull base image FROM oracle/serverjre:8 # Maintainer LABEL MAINTAINER=”[email protected]” ENV ORACLE_HOME=/u01/oracle USER_MEM_ARGS="-Djava.security.egd=file:/ dev/./urandom" PATH=$PATH:/usr/java/default/bin:/u01/ora cle/oracle_common/common/bin RUN mkdir -p /u01 && ... Oracle Weblogic 12.1.3 Image FROM oraclelinux:7-slim LABEL MAINTAINER=”[email protected]” ENV JAVA_PKG=server-jre-8u*-linux-x64.tar. gz JAVA_HOME=/usr/java/default ... oracle/serverjre:8 FROM scratch LABEL MAINTAINER=”[email protected]” ADD oraclelinux-7-slim-rootfs.tar.xz / oraclelinux:7-slim
- 40. Maybe we can check the Dockerfile? # Pull base image FROM oracle/serverjre:8 # Maintainer LABEL MAINTAINER=”[email protected]” ENV ORACLE_HOME=/u01/oracle USER_MEM_ARGS="-Djava.security.egd=file:/ dev/./urandom" PATH=$PATH:/usr/java/default/bin:/u01/ora cle/oracle_common/common/bin RUN mkdir -p /u01 && ... FROM scratch LABEL MAINTAINER=”[email protected]” ADD oraclelinux-7-slim-rootfs.tar.xz / FROM oraclelinux:7-slim LABEL MAINTAINER=”[email protected]” ENV JAVA_PKG=server-jre-8u*-linux-x64.tar. gz JAVA_HOME=/usr/java/default ... Oracle Weblogic 12.1.3 Image oracle/serverjre:8 oraclelinux:7-slim
- 41. Image Vulnerability Scanning Docker Trusted Registry
- 42. Automating your Pipeline with a Private Registry Automated Image Promotion dev/example qa/example Developer Pushes an Image to DTR Promotion Policy verifies that the image has no vulnerabilities. Webhooks at every step
- 43. How do I control what runs in my cluster?? How do I stop my developers running unknown images from the internet on my production clusters? $ kubectl apply -f exampleapp.yaml Error from server (Forbidden): error when creating "exampleapp.yaml": pods "nginx" is forbidden: one or more container images do not meet the required signing policy: [nginx: image did not meet required signing policy] $ docker run nginx:latest docker: Error response from daemon: image did not meet required signing policy.
- 45. The Software Supply Chain New Code Lands in SCM Jenkins - Builds new Image from SCM Image Uploaded to Registry If Image has no vulnerabilities. Move to testing. Jenkins Pipeline runs QA on the Image. Security Team, sign off on image. Moves to Production. New Image lands in Production Jenkins Pipeline Creates the Deployment from Templates App Successfully Deployed to Docker EE Cluster. Continuous Integration Continuous Deployment ???
- 46. Out-of-the-Box Features Modern orchestration systems come pre-baked with many deployment features. The work is already done for you! This means that your pipeline needs to monitor for status updates, but not implement the functionality.
- 47. Deployment Strategies Rolling Update Update containers one-by-one (or in groups), so that the application has no downtime. It’s possible for two versions of the software to be deployed at the same time.
- 52. In practice: Rolling Updates and Auto-Rollbacks Swarm and Kubernetes handle this for you -- no need to custom build service: build: myapp/myservice image: ${REGISTRY-127.0.0.1:5000}/myservice:${TAG-latest} deploy: replicas: 7 update_config: delay: 5s failure_action: rollback max_failure_ratio: .5 monitor: 5s parallelism: 1 docker stack
- 53. In practice: Rolling Updates and Auto-Rollbacks ... strategy: type: RollingUpdate #this is the default rollingUpdate: maxSurge: 1 maxUnavailable: 1 Swarm and Kubernetes handle this for you -- no need to custom build kubernetes deployment
- 54. The Software Supply Chain New Code Lands in SCM Jenkins - Builds new Image from SCM Image Uploaded to Registry If Image has no vulnerabilities. Move to testing. Jenkins Pipeline runs QA on the Image. Security Team, sign off on image. Moves to Production. New Image lands in Production Jenkins Pipeline Creates the Deployment from Templates App Successfully Deployed to Docker EE Cluster. Continuous Integration Continuous Deployment
- 55. The Software Supply Chain
- 56. Thank You!
- 57. Take A Breakout Survey Access your session and/or workshop surveys for the conference at any time by tapping the Sessions link on the navigation menu or block on the home screen. Find the session/workshop you attended and tap on it to view the session details. On this page, you will find a link to the survey.