SlideShare a Scribd company logo

ExpoQA 2017 Docker and CI

ElasTest Project
ElasTest Project

In the 2017 edition of ExpoQA we presented our experiences developing end-to-end automated tests for WebRTC applications. We talked about the architecture and the infrastructure used, and how leveraging the use of Docker containers eased the path towards a full suite of tests for real-time video over the Internet.

1 of 34
Download to read offline
ExpoQA 2017 Docker and CI
CI & Docker An experience building complex CI environments Patxi Gortázar francisco.gortazar@urjc.es @fgortazar
Bio Project Coordinator at @elastestio EU project Devops @ Kurento Teaching Distributed Systems @ URJC @fgortazar https://es.linkedin.com/in/franciscogortazar
Consultancy / Training Cloud Computing Distributed Systems Web Technologies Extreme Programming Testing / Git / Jenkins Software Architectures Concurrent Programming Open source elastic platform for end to end testing http://codeurjc.es http://elastest.io Advanced log management Test orchestration Test execution comparison Web and Mobile testing Security testing IoT testing Performance testing
Why A media server for real time video communication The software under test
WebRTC
Software under test
Software under test
Software under test So Kurento is… • A media server • APIs for managing connections in a WebRTC communication • Tutorials • Applications • Testing – Under many scenarios – Even more testing in different environments
A soup of technologies
Some numbers • 30 code repositories • ~400 Jenkins jobs • +1,000 tests • +20 different environments to test • +80 artifacts to be deployed at release time
Feasible infrastructures Cloud • Public or private • 20 environments = 20 different VMs • Cost increases with each new environment!! • Effort increases with each new environment to configure (ops)!! • Time-to-market increases also!!
Kurento’s CI Infrastructure Building Debian packages • 2 Ubuntu Trusty VMs • 4 Ubuntu Xenial VMs • 2 Ubuntu latest VMs Building clients • 1 node mvn & jdk 7 & node 4 • 1 node mvn & jdk 7 & node 6 • 1 node mvn & jdk 8 & node 4 • 1 node mvn & jdk 8 & node 6
Further problems • Developers are pushing hard towards ops/devops to include changes in infrastructure • Changes can hardly be reverted (possible, but…) • Hard to test locally – Works in my machine effect • Wasted resources • And at the same time, insufficient resources Can we do better?
Simplifying CI with Docker
How • Leverage Jenkins Pipeline – CI configuration hosted within code repositories – Docker support out-of-the-box – Jenkins jobs launch containers on top of VMs – VMs have all the same minimum configuration: Docker
Current Infrastructure 1 VM Jenkins Master N VMs Jenkins Workers (running Docker) 2 VM configurations
Where are my environments? • The different environments are encapsulated as Docker images • Docker images can be used in any of the workers • Docker images can be used in your laptop • Docker images are versioned, easy to maintain, easy to build • Any version can be used • Several environments can be run in parallel in the same VM: isolation
Which environments do I need? You need images for your builds You need images for your applications You need images for your tests
Images for builds • Heavy images (> 400Mb) • One image per tool version: jdk7, jdk8 • Download ahead of time – As opposed to downloading at build time – E.g., downloading images in a nightly job
Images for builds With Docker it’s easier to tame matrix envs • Consider building a project using… – Ubuntu • Trusty • Xenial • Zesty – Java • JDK 7 • JDK 8 • JDK 9
Images for builds What if we need to mix… • JDK • Maven • Node • Npm • Bower … in the same environment?
Images for applications You're ready to ship your application as a container (optional) • Choose lightweight containers – Docker is focusing on lightweight containers leveraging Alpine Linux • Include just what your application needs – Leverage new multi-stage build in Docker • Think about dependencies (databases, caching...)
Images for testing • You can run browsers within a container – https://github.com/SeleniumHQ/docker-selenium • Available images – selenium/hub – selenium/node-chrome – selenium/node-firefox – selenium/node-chrome-debug – selenium/node-firefox-debug
Images for testing • Unfortunately they're a bit out of date – See https://github.com/SeleniumHQ/docker- selenium/issues/229#issuecomment-253334224 • Don't worry – It's easy to build them with the latest browser – We can test against latest release, beta and canary • Alternatives – TestContainers: https://www.testcontainers.org/
Testing with multiple containers Orchestration... • docker compose – Require to know the number of containers in advance • Otherwise, docker run • Cleaning containers and images with spotify-gc
Testing with multiple containers Orchestration... • End-to-end tests – 1-5 browsers – Test scenario is run several times with different configurations • Chrome (stable, beta, canary) • Firefox (stable, beta) • …
Testing with multiple containers Orchestration... • Tests can include a Docker client for flexibility: – Start & stop containers – Container inspection – Gathering logs – Managing Volumes – Managing environment variables
Testing with multiple containers Orchestration... • Network topologies – Specific network plugins – Handcrafted networks • docker run --net=none …  Ugly hack!! • Better use network plugins
Testing with multiple containers
Challenges • Gathering data – Logs from several containers – Metrics (CPU, memory, network…) – Files generated (browser recording) • Integrating data with test execution – Comparing some data across several executions – Data integration
Challenges • Reproducing real-world conditions – Failing nodes – Failing networks – Network bandwidth (4G, 5G, DSL) – Network topologies (NAT traversal, firewalls, …) – High loads (bursting CPUs)
Consultancy / Training Cloud Computing Distributed Systems Web Technologies Extreme Programming Testing / Git / Jenkins Software Architectures Concurrent Programming Open source elastic platform for end to end testing http://codeurjc.es http://elastest.io Advanced log management Test orchestration Test execution comparison Web and Mobile testing Security testing IoT testing Performance testing
Bio Coordinator http://elastest.io/ Devops @ Kurento Teaching Distributed Systems @ URJC @fgortazar https://es.linkedin.com/in/franciscogortazar

Recommended

ExpoQA 2017 Using docker to build and test in your laptop and Jenkins
ExpoQA 2017 Using docker to build and test in your laptop and JenkinsExpoQA 2017 Using docker to build and test in your laptop and Jenkins
ExpoQA 2017 Using docker to build and test in your laptop and Jenkins
ElasTest Project
 
JOSA TechTalk: Introduction to docker
JOSA TechTalk: Introduction to dockerJOSA TechTalk: Introduction to docker
JOSA TechTalk: Introduction to docker
Jordan Open Source Association
 
Docker 101: An Introduction
Docker 101: An IntroductionDocker 101: An Introduction
Docker 101: An Introduction
POSSCON
 
Docker Continuous Delivery Workshop
Docker Continuous Delivery WorkshopDocker Continuous Delivery Workshop
Docker Continuous Delivery Workshop
Jirayut Nimsaeng
 
Docker and the Linux Kernel
Docker and the Linux KernelDocker and the Linux Kernel
Docker and the Linux Kernel
Docker, Inc.
 
Docker Basics
Docker BasicsDocker Basics
Docker Basics
DuckDuckGo
 
Docker Presentation at the OpenStack Austin Meetup | 2013-09-12
Docker Presentation at the OpenStack Austin Meetup | 2013-09-12Docker Presentation at the OpenStack Austin Meetup | 2013-09-12
Docker Presentation at the OpenStack Austin Meetup | 2013-09-12
dotCloud
 
Introduction to Docker and deployment and Azure
Introduction to Docker and deployment and AzureIntroduction to Docker and deployment and Azure
Introduction to Docker and deployment and Azure
Jérôme Petazzoni
 
Introduction to Docker at SF Peninsula Software Development Meetup @Guidewire
Introduction to Docker at SF Peninsula Software Development Meetup @GuidewireIntroduction to Docker at SF Peninsula Software Development Meetup @Guidewire
Introduction to Docker at SF Peninsula Software Development Meetup @Guidewire
dotCloud
 
Learning Docker with Thomas
Learning Docker with ThomasLearning Docker with Thomas
Learning Docker with Thomas
Thomas Tong, FRM, PMP
 
Containerization and Docker
Containerization and DockerContainerization and Docker
Containerization and Docker
Megha Bansal
 
Docker and the Container Ecosystem
Docker and the Container EcosystemDocker and the Container Ecosystem
Docker and the Container Ecosystem
psconnolly
 
Docker
DockerDocker
Docker
Mutlu Okuducu
 
Shipping Applications to Production in Containers with Docker
Shipping Applications to Production in Containers with DockerShipping Applications to Production in Containers with Docker
Shipping Applications to Production in Containers with Docker
Jérôme Petazzoni
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
MANAOUIL Karim
 
docker installation and basics
docker installation and basicsdocker installation and basics
docker installation and basics
Walid Ashraf
 
Docker in real life
Docker in real lifeDocker in real life
Docker in real life
Nguyen Van Vuong
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
Alan Forbes
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
Jeffrey Ellin
 
JOSA TechTalk: Taking Docker to Production
JOSA TechTalk: Taking Docker to ProductionJOSA TechTalk: Taking Docker to Production
JOSA TechTalk: Taking Docker to Production
Jordan Open Source Association
 
Docker 101 - Intro to Docker
Docker 101 - Intro to DockerDocker 101 - Intro to Docker
Docker 101 - Intro to Docker
Adrian Otto
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
Sparkbit
 
Docker based-pipelines
Docker based-pipelinesDocker based-pipelines
Docker based-pipelines
DevOps.com
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
Jian Wu
 
Architecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based DeploymentsArchitecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based Deployments
Ben Hall
 
Docker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken CochraneDocker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken Cochrane
dotCloud
 
Virtual Machines and Docker
Virtual Machines and DockerVirtual Machines and Docker
Virtual Machines and Docker
Danish Khakwani
 
Introduction To Docker
Introduction To DockerIntroduction To Docker
Introduction To Docker
Dr. Syed Hassan Amin
 
DCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development PipelineDCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development Pipeline
Docker, Inc.
 
Exploring Docker in CI/CD
Exploring Docker in CI/CDExploring Docker in CI/CD
Exploring Docker in CI/CD
Henry Huang
 

More Related Content

What's hot (20)

Introduction to Docker at SF Peninsula Software Development Meetup @Guidewire
Introduction to Docker at SF Peninsula Software Development Meetup @GuidewireIntroduction to Docker at SF Peninsula Software Development Meetup @Guidewire
Introduction to Docker at SF Peninsula Software Development Meetup @Guidewire
dotCloud
 
Learning Docker with Thomas
Learning Docker with ThomasLearning Docker with Thomas
Learning Docker with Thomas
Thomas Tong, FRM, PMP
 
Containerization and Docker
Containerization and DockerContainerization and Docker
Containerization and Docker
Megha Bansal
 
Docker and the Container Ecosystem
Docker and the Container EcosystemDocker and the Container Ecosystem
Docker and the Container Ecosystem
psconnolly
 
Docker
DockerDocker
Docker
Mutlu Okuducu
 
Shipping Applications to Production in Containers with Docker
Shipping Applications to Production in Containers with DockerShipping Applications to Production in Containers with Docker
Shipping Applications to Production in Containers with Docker
Jérôme Petazzoni
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
MANAOUIL Karim
 
docker installation and basics
docker installation and basicsdocker installation and basics
docker installation and basics
Walid Ashraf
 
Docker in real life
Docker in real lifeDocker in real life
Docker in real life
Nguyen Van Vuong
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
Alan Forbes
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
Jeffrey Ellin
 
JOSA TechTalk: Taking Docker to Production
JOSA TechTalk: Taking Docker to ProductionJOSA TechTalk: Taking Docker to Production
JOSA TechTalk: Taking Docker to Production
Jordan Open Source Association
 
Docker 101 - Intro to Docker
Docker 101 - Intro to DockerDocker 101 - Intro to Docker
Docker 101 - Intro to Docker
Adrian Otto
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
Sparkbit
 
Docker based-pipelines
Docker based-pipelinesDocker based-pipelines
Docker based-pipelines
DevOps.com
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
Jian Wu
 
Architecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based DeploymentsArchitecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based Deployments
Ben Hall
 
Docker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken CochraneDocker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken Cochrane
dotCloud
 
Virtual Machines and Docker
Virtual Machines and DockerVirtual Machines and Docker
Virtual Machines and Docker
Danish Khakwani
 
Introduction To Docker
Introduction To DockerIntroduction To Docker
Introduction To Docker
Dr. Syed Hassan Amin
 
Introduction to Docker at SF Peninsula Software Development Meetup @Guidewire
Introduction to Docker at SF Peninsula Software Development Meetup @GuidewireIntroduction to Docker at SF Peninsula Software Development Meetup @Guidewire
Introduction to Docker at SF Peninsula Software Development Meetup @Guidewire
dotCloud
 
Learning Docker with Thomas
Learning Docker with ThomasLearning Docker with Thomas
Learning Docker with Thomas
Thomas Tong, FRM, PMP
 
Containerization and Docker
Containerization and DockerContainerization and Docker
Containerization and Docker
Megha Bansal
 
Docker and the Container Ecosystem
Docker and the Container EcosystemDocker and the Container Ecosystem
Docker and the Container Ecosystem
psconnolly
 
Docker
DockerDocker
Docker
Mutlu Okuducu
 
Shipping Applications to Production in Containers with Docker
Shipping Applications to Production in Containers with DockerShipping Applications to Production in Containers with Docker
Shipping Applications to Production in Containers with Docker
Jérôme Petazzoni
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
MANAOUIL Karim
 
docker installation and basics
docker installation and basicsdocker installation and basics
docker installation and basics
Walid Ashraf
 
Docker in real life
Docker in real lifeDocker in real life
Docker in real life
Nguyen Van Vuong
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
Alan Forbes
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
Jeffrey Ellin
 
JOSA TechTalk: Taking Docker to Production
JOSA TechTalk: Taking Docker to ProductionJOSA TechTalk: Taking Docker to Production
JOSA TechTalk: Taking Docker to Production
Jordan Open Source Association
 
Docker 101 - Intro to Docker
Docker 101 - Intro to DockerDocker 101 - Intro to Docker
Docker 101 - Intro to Docker
Adrian Otto
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
Sparkbit
 
Docker based-pipelines
Docker based-pipelinesDocker based-pipelines
Docker based-pipelines
DevOps.com
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
Jian Wu
 
Architecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based DeploymentsArchitecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based Deployments
Ben Hall
 
Docker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken CochraneDocker at Djangocon 2013 | Talk by Ken Cochrane
Docker at Djangocon 2013 | Talk by Ken Cochrane
dotCloud
 
Virtual Machines and Docker
Virtual Machines and DockerVirtual Machines and Docker
Virtual Machines and Docker
Danish Khakwani
 
Introduction To Docker
Introduction To DockerIntroduction To Docker
Introduction To Docker
Dr. Syed Hassan Amin
 

Similar to ExpoQA 2017 Docker and CI (20)

DCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development PipelineDCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development Pipeline
Docker, Inc.
 
Exploring Docker in CI/CD
Exploring Docker in CI/CDExploring Docker in CI/CD
Exploring Docker in CI/CD
Henry Huang
 
Dockerize the World
Dockerize the WorldDockerize the World
Dockerize the World
damovsky
 
ma-formation-en-Docker-jlklk,nknkjn.pptx
ma-formation-en-Docker-jlklk,nknkjn.pptxma-formation-en-Docker-jlklk,nknkjn.pptx
ma-formation-en-Docker-jlklk,nknkjn.pptx
imenhamada17
 
Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetes
Dongwon Kim
 
Django and Docker
Django and DockerDjango and Docker
Django and Docker
Docker, Inc.
 
Docker - Portable Deployment
Docker - Portable DeploymentDocker - Portable Deployment
Docker - Portable Deployment
javaonfly
 
Using Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and JenkinsUsing Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and Jenkins
Micael Gallego
 
Intro docker and demo monitor on docker
Intro docker and demo monitor on dockerIntro docker and demo monitor on docker
Intro docker and demo monitor on docker
Watcharin Yang-Ngam
 
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
E. Camden Fisher
 
Docker for the new Era: Introducing Docker,its components and tools
Docker for the new Era: Introducing Docker,its components and toolsDocker for the new Era: Introducing Docker,its components and tools
Docker for the new Era: Introducing Docker,its components and tools
Ramit Surana
 
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
dotCloud
 
Docker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationDocker and Puppet for Continuous Integration
Docker and Puppet for Continuous Integration
Giacomo Vacca
 
Docker.pptx
Docker.pptxDocker.pptx
Docker.pptx
balaji257
 
Lightweight Virtualization Docker in Practice
Lightweight Virtualization Docker in PracticeLightweight Virtualization Docker in Practice
Lightweight Virtualization Docker in Practice
Docker, Inc.
 
vodQA(Pune) 2018 - Docker in Testing
vodQA(Pune) 2018 - Docker in TestingvodQA(Pune) 2018 - Docker in Testing
vodQA(Pune) 2018 - Docker in Testing
vodQA
 
IBM WebSphere Application Server traditional and Docker
IBM WebSphere Application Server traditional and DockerIBM WebSphere Application Server traditional and Docker
IBM WebSphere Application Server traditional and Docker
David Currie
 
CI, CD with Docker, Jenkins and Tutum
CI, CD with Docker, Jenkins and TutumCI, CD with Docker, Jenkins and Tutum
CI, CD with Docker, Jenkins and Tutum
Sreenivas Makam
 
DockerCon EU 2015 Barcelona
DockerCon EU 2015 BarcelonaDockerCon EU 2015 Barcelona
DockerCon EU 2015 Barcelona
Roman Dembitsky
 
Containerization using docker and its applications
Containerization using docker and its applicationsContainerization using docker and its applications
Containerization using docker and its applications
Puneet Kumar Bhatia (MBA, ITIL V3 Certified)
 
DCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development PipelineDCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development Pipeline
Docker, Inc.
 
Exploring Docker in CI/CD
Exploring Docker in CI/CDExploring Docker in CI/CD
Exploring Docker in CI/CD
Henry Huang
 
Dockerize the World
Dockerize the WorldDockerize the World
Dockerize the World
damovsky
 
ma-formation-en-Docker-jlklk,nknkjn.pptx
ma-formation-en-Docker-jlklk,nknkjn.pptxma-formation-en-Docker-jlklk,nknkjn.pptx
ma-formation-en-Docker-jlklk,nknkjn.pptx
imenhamada17
 
Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetes
Dongwon Kim
 
Django and Docker
Django and DockerDjango and Docker
Django and Docker
Docker, Inc.
 
Docker - Portable Deployment
Docker - Portable DeploymentDocker - Portable Deployment
Docker - Portable Deployment
javaonfly
 
Using Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and JenkinsUsing Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and Jenkins
Micael Gallego
 
Intro docker and demo monitor on docker
Intro docker and demo monitor on dockerIntro docker and demo monitor on docker
Intro docker and demo monitor on docker
Watcharin Yang-Ngam
 
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
E. Camden Fisher
 
Docker for the new Era: Introducing Docker,its components and tools
Docker for the new Era: Introducing Docker,its components and toolsDocker for the new Era: Introducing Docker,its components and tools
Docker for the new Era: Introducing Docker,its components and tools
Ramit Surana
 
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013
dotCloud
 
Docker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationDocker and Puppet for Continuous Integration
Docker and Puppet for Continuous Integration
Giacomo Vacca
 
Docker.pptx
Docker.pptxDocker.pptx
Docker.pptx
balaji257
 
Lightweight Virtualization Docker in Practice
Lightweight Virtualization Docker in PracticeLightweight Virtualization Docker in Practice
Lightweight Virtualization Docker in Practice
Docker, Inc.
 
vodQA(Pune) 2018 - Docker in Testing
vodQA(Pune) 2018 - Docker in TestingvodQA(Pune) 2018 - Docker in Testing
vodQA(Pune) 2018 - Docker in Testing
vodQA
 
IBM WebSphere Application Server traditional and Docker
IBM WebSphere Application Server traditional and DockerIBM WebSphere Application Server traditional and Docker
IBM WebSphere Application Server traditional and Docker
David Currie
 
CI, CD with Docker, Jenkins and Tutum
CI, CD with Docker, Jenkins and TutumCI, CD with Docker, Jenkins and Tutum
CI, CD with Docker, Jenkins and Tutum
Sreenivas Makam
 
DockerCon EU 2015 Barcelona
DockerCon EU 2015 BarcelonaDockerCon EU 2015 Barcelona
DockerCon EU 2015 Barcelona
Roman Dembitsky
 
Containerization using docker and its applications
Containerization using docker and its applicationsContainerization using docker and its applications
Containerization using docker and its applications
Puneet Kumar Bhatia (MBA, ITIL V3 Certified)
 

More from ElasTest Project (12)

ElasTest: quality for cloud native applications
ElasTest: quality for cloud native applicationsElasTest: quality for cloud native applications
ElasTest: quality for cloud native applications
ElasTest Project
 
How to plan and define your CI-CD pipeline
How to plan and define your CI-CD pipelineHow to plan and define your CI-CD pipeline
How to plan and define your CI-CD pipeline
ElasTest Project
 
Bringing observability to your testing environments
Bringing observability to your testing environmentsBringing observability to your testing environments
Bringing observability to your testing environments
ElasTest Project
 
Life-cycle is too short not to use ElasTest
Life-cycle is too short not to use ElasTestLife-cycle is too short not to use ElasTest
Life-cycle is too short not to use ElasTest
ElasTest Project
 
ElasTest Webinar
ElasTest WebinarElasTest Webinar
ElasTest Webinar
ElasTest Project
 
ElasTest presentation in MadridJUG (Madrid December 2017)
ElasTest presentation in MadridJUG (Madrid December 2017)ElasTest presentation in MadridJUG (Madrid December 2017)
ElasTest presentation in MadridJUG (Madrid December 2017)
ElasTest Project
 
ElasTest presentation in Panel Sistemas company (Madrid December 2017)
ElasTest presentation in Panel Sistemas company (Madrid December 2017)ElasTest presentation in Panel Sistemas company (Madrid December 2017)
ElasTest presentation in Panel Sistemas company (Madrid December 2017)
ElasTest Project
 
ElasTest presentation in VLCTesting Conference (Valencia Novemeber 2017)
ElasTest presentation in VLCTesting Conference (Valencia Novemeber 2017)ElasTest presentation in VLCTesting Conference (Valencia Novemeber 2017)
ElasTest presentation in VLCTesting Conference (Valencia Novemeber 2017)
ElasTest Project
 
ElasTest - Testing in the large
ElasTest - Testing in the largeElasTest - Testing in the large
ElasTest - Testing in the large
ElasTest Project
 
ElasTest ICSOFT 2017 - Panel H2020
ElasTest ICSOFT 2017 - Panel H2020ElasTest ICSOFT 2017 - Panel H2020
ElasTest ICSOFT 2017 - Panel H2020
ElasTest Project
 
Tarot 2017
Tarot 2017Tarot 2017
Tarot 2017
ElasTest Project
 
ElasTest technical presentation
ElasTest technical presentationElasTest technical presentation
ElasTest technical presentation
ElasTest Project
 
ElasTest: quality for cloud native applications
ElasTest: quality for cloud native applicationsElasTest: quality for cloud native applications
ElasTest: quality for cloud native applications
ElasTest Project
 
How to plan and define your CI-CD pipeline
How to plan and define your CI-CD pipelineHow to plan and define your CI-CD pipeline
How to plan and define your CI-CD pipeline
ElasTest Project
 
Bringing observability to your testing environments
Bringing observability to your testing environmentsBringing observability to your testing environments
Bringing observability to your testing environments
ElasTest Project
 
Life-cycle is too short not to use ElasTest
Life-cycle is too short not to use ElasTestLife-cycle is too short not to use ElasTest
Life-cycle is too short not to use ElasTest
ElasTest Project
 
ElasTest Webinar
ElasTest WebinarElasTest Webinar
ElasTest Webinar
ElasTest Project
 
ElasTest presentation in MadridJUG (Madrid December 2017)
ElasTest presentation in MadridJUG (Madrid December 2017)ElasTest presentation in MadridJUG (Madrid December 2017)
ElasTest presentation in MadridJUG (Madrid December 2017)
ElasTest Project
 
ElasTest presentation in Panel Sistemas company (Madrid December 2017)
ElasTest presentation in Panel Sistemas company (Madrid December 2017)ElasTest presentation in Panel Sistemas company (Madrid December 2017)
ElasTest presentation in Panel Sistemas company (Madrid December 2017)
ElasTest Project
 
ElasTest presentation in VLCTesting Conference (Valencia Novemeber 2017)
ElasTest presentation in VLCTesting Conference (Valencia Novemeber 2017)ElasTest presentation in VLCTesting Conference (Valencia Novemeber 2017)
ElasTest presentation in VLCTesting Conference (Valencia Novemeber 2017)
ElasTest Project
 
ElasTest - Testing in the large
ElasTest - Testing in the largeElasTest - Testing in the large
ElasTest - Testing in the large
ElasTest Project
 
ElasTest ICSOFT 2017 - Panel H2020
ElasTest ICSOFT 2017 - Panel H2020ElasTest ICSOFT 2017 - Panel H2020
ElasTest ICSOFT 2017 - Panel H2020
ElasTest Project
 
Tarot 2017
Tarot 2017Tarot 2017
Tarot 2017
ElasTest Project
 
ElasTest technical presentation
ElasTest technical presentationElasTest technical presentation
ElasTest technical presentation
ElasTest Project
 

Recently uploaded (20)

Designing AI-Powered APIs on Azure: Best Practices& Considerations
Designing AI-Powered APIs on Azure: Best Practices& ConsiderationsDesigning AI-Powered APIs on Azure: Best Practices& Considerations
Designing AI-Powered APIs on Azure: Best Practices& Considerations
Dinusha Kumarasiri
 
Revolutionizing Residential Wi-Fi PPT.pptx
Revolutionizing Residential Wi-Fi PPT.pptxRevolutionizing Residential Wi-Fi PPT.pptx
Revolutionizing Residential Wi-Fi PPT.pptx
nidhisingh691197
 
The Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdfThe Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdf
drewplanas10
 
Avast Premium Security Crack FREE Latest Version 2025
Avast Premium Security Crack FREE Latest Version 2025Avast Premium Security Crack FREE Latest Version 2025
Avast Premium Security Crack FREE Latest Version 2025
mu394968
 
Automation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath CertificateAutomation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath Certificate
VICTOR MAESTRE RAMIREZ
 
Douwan Crack 2025 new verson+ License code
Douwan Crack 2025 new verson+ License codeDouwan Crack 2025 new verson+ License code
Douwan Crack 2025 new verson+ License code
aneelaramzan63
 
Landscape of Requirements Engineering for/by AI through Literature Review
Landscape of Requirements Engineering for/by AI through Literature ReviewLandscape of Requirements Engineering for/by AI through Literature Review
Landscape of Requirements Engineering for/by AI through Literature Review
Hironori Washizaki
 
Download Wondershare Filmora Crack [2025] With Latest
Download Wondershare Filmora Crack [2025] With LatestDownload Wondershare Filmora Crack [2025] With Latest
Download Wondershare Filmora Crack [2025] With Latest
tahirabibi60507
 
Maxon CINEMA 4D 2025 Crack FREE Download LINK
Maxon CINEMA 4D 2025 Crack FREE Download LINKMaxon CINEMA 4D 2025 Crack FREE Download LINK
Maxon CINEMA 4D 2025 Crack FREE Download LINK
younisnoman75
 
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Eric D. Schabell
 
Adobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest VersionAdobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest Version
kashifyounis067
 
Societal challenges of AI: biases, multilinguism and sustainability
Societal challenges of AI: biases, multilinguism and sustainabilitySocietal challenges of AI: biases, multilinguism and sustainability
Societal challenges of AI: biases, multilinguism and sustainability
Jordi Cabot
 
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AIScaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
danshalev
 
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
AxisTechnolabs
 
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdfMicrosoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
TechSoup
 
Exploring Wayland: A Modern Display Server for the Future
Exploring Wayland: A Modern Display Server for the FutureExploring Wayland: A Modern Display Server for the Future
Exploring Wayland: A Modern Display Server for the Future
ICS
 
Adobe Lightroom Classic Crack FREE Latest link 2025
Adobe Lightroom Classic Crack FREE Latest link 2025Adobe Lightroom Classic Crack FREE Latest link 2025
Adobe Lightroom Classic Crack FREE Latest link 2025
kashifyounis067
 
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRYLEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
NidaFarooq10
 
Exploring Code Comprehension in Scientific Programming: Preliminary Insight...
Exploring Code Comprehension in Scientific Programming: Preliminary Insight...Exploring Code Comprehension in Scientific Programming: Preliminary Insight...
Exploring Code Comprehension in Scientific Programming: Preliminary Insight...
University of Hawai‘i at Mānoa
 
How can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptxHow can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptx
laravinson24
 
Designing AI-Powered APIs on Azure: Best Practices& Considerations
Designing AI-Powered APIs on Azure: Best Practices& ConsiderationsDesigning AI-Powered APIs on Azure: Best Practices& Considerations
Designing AI-Powered APIs on Azure: Best Practices& Considerations
Dinusha Kumarasiri
 
Revolutionizing Residential Wi-Fi PPT.pptx
Revolutionizing Residential Wi-Fi PPT.pptxRevolutionizing Residential Wi-Fi PPT.pptx
Revolutionizing Residential Wi-Fi PPT.pptx
nidhisingh691197
 
The Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdfThe Significance of Hardware in Information Systems.pdf
The Significance of Hardware in Information Systems.pdf
drewplanas10
 
Avast Premium Security Crack FREE Latest Version 2025
Avast Premium Security Crack FREE Latest Version 2025Avast Premium Security Crack FREE Latest Version 2025
Avast Premium Security Crack FREE Latest Version 2025
mu394968
 
Automation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath CertificateAutomation Techniques in RPA - UiPath Certificate
Automation Techniques in RPA - UiPath Certificate
VICTOR MAESTRE RAMIREZ
 
Douwan Crack 2025 new verson+ License code
Douwan Crack 2025 new verson+ License codeDouwan Crack 2025 new verson+ License code
Douwan Crack 2025 new verson+ License code
aneelaramzan63
 
Landscape of Requirements Engineering for/by AI through Literature Review
Landscape of Requirements Engineering for/by AI through Literature ReviewLandscape of Requirements Engineering for/by AI through Literature Review
Landscape of Requirements Engineering for/by AI through Literature Review
Hironori Washizaki
 
Download Wondershare Filmora Crack [2025] With Latest
Download Wondershare Filmora Crack [2025] With LatestDownload Wondershare Filmora Crack [2025] With Latest
Download Wondershare Filmora Crack [2025] With Latest
tahirabibi60507
 
Maxon CINEMA 4D 2025 Crack FREE Download LINK
Maxon CINEMA 4D 2025 Crack FREE Download LINKMaxon CINEMA 4D 2025 Crack FREE Download LINK
Maxon CINEMA 4D 2025 Crack FREE Download LINK
younisnoman75
 
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Mastering Fluent Bit: Ultimate Guide to Integrating Telemetry Pipelines with ...
Eric D. Schabell
 
Adobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest VersionAdobe Illustrator Crack FREE Download 2025 Latest Version
Adobe Illustrator Crack FREE Download 2025 Latest Version
kashifyounis067
 
Societal challenges of AI: biases, multilinguism and sustainability
Societal challenges of AI: biases, multilinguism and sustainabilitySocietal challenges of AI: biases, multilinguism and sustainability
Societal challenges of AI: biases, multilinguism and sustainability
Jordi Cabot
 
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AIScaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
Scaling GraphRAG: Efficient Knowledge Retrieval for Enterprise AI
danshalev
 
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
Interactive odoo dashboards for sales, CRM , Inventory, Invoice, Purchase, Pr...
AxisTechnolabs
 
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdfMicrosoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
Microsoft AI Nonprofit Use Cases and Live Demo_2025.04.30.pdf
TechSoup
 
Exploring Wayland: A Modern Display Server for the Future
Exploring Wayland: A Modern Display Server for the FutureExploring Wayland: A Modern Display Server for the Future
Exploring Wayland: A Modern Display Server for the Future
ICS
 
Adobe Lightroom Classic Crack FREE Latest link 2025
Adobe Lightroom Classic Crack FREE Latest link 2025Adobe Lightroom Classic Crack FREE Latest link 2025
Adobe Lightroom Classic Crack FREE Latest link 2025
kashifyounis067
 
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRYLEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
LEARN SEO AND INCREASE YOUR KNOWLDGE IN SOFTWARE INDUSTRY
NidaFarooq10
 
Exploring Code Comprehension in Scientific Programming: Preliminary Insight...
Exploring Code Comprehension in Scientific Programming: Preliminary Insight...Exploring Code Comprehension in Scientific Programming: Preliminary Insight...
Exploring Code Comprehension in Scientific Programming: Preliminary Insight...
University of Hawai‘i at Mānoa
 
How can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptxHow can one start with crypto wallet development.pptx
How can one start with crypto wallet development.pptx
laravinson24
 

ExpoQA 2017 Docker and CI

  • 2. CI & Docker An experience building complex CI environments Patxi Gortázar [email protected] @fgortazar
  • 3. Bio Project Coordinator at @elastestio EU project Devops @ Kurento Teaching Distributed Systems @ URJC @fgortazar https://es.linkedin.com/in/franciscogortazar
  • 4. Consultancy / Training Cloud Computing Distributed Systems Web Technologies Extreme Programming Testing / Git / Jenkins Software Architectures Concurrent Programming Open source elastic platform for end to end testing http://codeurjc.es http://elastest.io Advanced log management Test orchestration Test execution comparison Web and Mobile testing Security testing IoT testing Performance testing
  • 5. Why A media server for real time video communication The software under test
  • 9. Software under test So Kurento is… • A media server • APIs for managing connections in a WebRTC communication • Tutorials • Applications • Testing – Under many scenarios – Even more testing in different environments
  • 10. A soup of technologies
  • 11. Some numbers • 30 code repositories • ~400 Jenkins jobs • +1,000 tests • +20 different environments to test • +80 artifacts to be deployed at release time
  • 12. Feasible infrastructures Cloud • Public or private • 20 environments = 20 different VMs • Cost increases with each new environment!! • Effort increases with each new environment to configure (ops)!! • Time-to-market increases also!!
  • 13. Kurento’s CI Infrastructure Building Debian packages • 2 Ubuntu Trusty VMs • 4 Ubuntu Xenial VMs • 2 Ubuntu latest VMs Building clients • 1 node mvn & jdk 7 & node 4 • 1 node mvn & jdk 7 & node 6 • 1 node mvn & jdk 8 & node 4 • 1 node mvn & jdk 8 & node 6
  • 14. Further problems • Developers are pushing hard towards ops/devops to include changes in infrastructure • Changes can hardly be reverted (possible, but…) • Hard to test locally – Works in my machine effect • Wasted resources • And at the same time, insufficient resources Can we do better?
  • 16. How • Leverage Jenkins Pipeline – CI configuration hosted within code repositories – Docker support out-of-the-box – Jenkins jobs launch containers on top of VMs – VMs have all the same minimum configuration: Docker
  • 17. Current Infrastructure 1 VM Jenkins Master N VMs Jenkins Workers (running Docker) 2 VM configurations
  • 18. Where are my environments? • The different environments are encapsulated as Docker images • Docker images can be used in any of the workers • Docker images can be used in your laptop • Docker images are versioned, easy to maintain, easy to build • Any version can be used • Several environments can be run in parallel in the same VM: isolation
  • 19. Which environments do I need? You need images for your builds You need images for your applications You need images for your tests
  • 20. Images for builds • Heavy images (> 400Mb) • One image per tool version: jdk7, jdk8 • Download ahead of time – As opposed to downloading at build time – E.g., downloading images in a nightly job
  • 21. Images for builds With Docker it’s easier to tame matrix envs • Consider building a project using… – Ubuntu • Trusty • Xenial • Zesty – Java • JDK 7 • JDK 8 • JDK 9
  • 22. Images for builds What if we need to mix… • JDK • Maven • Node • Npm • Bower … in the same environment?
  • 23. Images for applications You're ready to ship your application as a container (optional) • Choose lightweight containers – Docker is focusing on lightweight containers leveraging Alpine Linux • Include just what your application needs – Leverage new multi-stage build in Docker • Think about dependencies (databases, caching...)
  • 24. Images for testing • You can run browsers within a container – https://github.com/SeleniumHQ/docker-selenium • Available images – selenium/hub – selenium/node-chrome – selenium/node-firefox – selenium/node-chrome-debug – selenium/node-firefox-debug
  • 25. Images for testing • Unfortunately they're a bit out of date – See https://github.com/SeleniumHQ/docker- selenium/issues/229#issuecomment-253334224 • Don't worry – It's easy to build them with the latest browser – We can test against latest release, beta and canary • Alternatives – TestContainers: https://www.testcontainers.org/
  • 26. Testing with multiple containers Orchestration... • docker compose – Require to know the number of containers in advance • Otherwise, docker run • Cleaning containers and images with spotify-gc
  • 27. Testing with multiple containers Orchestration... • End-to-end tests – 1-5 browsers – Test scenario is run several times with different configurations • Chrome (stable, beta, canary) • Firefox (stable, beta) • …
  • 28. Testing with multiple containers Orchestration... • Tests can include a Docker client for flexibility: – Start & stop containers – Container inspection – Gathering logs – Managing Volumes – Managing environment variables
  • 29. Testing with multiple containers Orchestration... • Network topologies – Specific network plugins – Handcrafted networks • docker run --net=none …  Ugly hack!! • Better use network plugins
  • 30. Testing with multiple containers
  • 31. Challenges • Gathering data – Logs from several containers – Metrics (CPU, memory, network…) – Files generated (browser recording) • Integrating data with test execution – Comparing some data across several executions – Data integration
  • 32. Challenges • Reproducing real-world conditions – Failing nodes – Failing networks – Network bandwidth (4G, 5G, DSL) – Network topologies (NAT traversal, firewalls, …) – High loads (bursting CPUs)
  • 33. Consultancy / Training Cloud Computing Distributed Systems Web Technologies Extreme Programming Testing / Git / Jenkins Software Architectures Concurrent Programming Open source elastic platform for end to end testing http://codeurjc.es http://elastest.io Advanced log management Test orchestration Test execution comparison Web and Mobile testing Security testing IoT testing Performance testing
  • 34. Bio Coordinator http://elastest.io/ Devops @ Kurento Teaching Distributed Systems @ URJC @fgortazar https://es.linkedin.com/in/franciscogortazar