Container Orchestration @Docker Meetup Hamburg

Sep 9, 201420 likes3,424 views

This document discusses container orchestration and provides an overview of various container orchestration tools and concepts. It describes schedulers that manage resource allocation and deployment of containers across clusters as well as tools for configuration management, service discovery, and maintaining a consistent cluster state. Examples of specific container orchestration systems like Mesos, Marathon, CoreOS, Kubernetes, and Docker libswarm are outlined.

Container Orchestration
Timo Derstappen
@teemow

Why Orchestration?
● Microservice Architectures
● Polyglot Persistence
● Fault tolerance / High Availability
● Scalability
● Utilization

Container Orchestration @Docker Meetup Hamburg

Overview
● Schedulers
● Configuration Management
● Service Discovery

Scheduler
● Resource management
● Decide which job runs where
● Monitor jobs
● Dependency resolution

Scheduler Concepts
● Google Borg (Monolithic)
● Apache Mesos (Two Level)
● Hadoop on Demand (Two Level)
● Google Omega (Shared State)
Read: “Omega: flexible, scalable schedulers for
large compute clusters”
http://research.google.com/pubs/pub41684.html

Static Partitioning
http://cto.vmware.com/wp-content/uploads/2012/10/mesos_workloads.png

Configuration Management
● State which container runs where
● Configure your containers
● Basis for service discovery

Consistent Cluster State
● Etcd (Raft)
● Consul (Raft)
● Doozer (Paxos)
● Zookeeper (Paxos)
● Spanner (Paxos)
http://raftconsensus.github.io/

Service Discovery
● Ambassador Pattern
● Consul (DNS)
● Etcd
● Haproxy / confd

Docker libchan
● Link containers from everywhere
● Unix Socket
● HTTP2 / TLS
● Messaging / Raw TCP
● Like Go Channels over the network

Many Choices
● CoreOS
● Kubernetes
● Apache Mesos
● Consul / Serf
● Project Atomic / Geard
● Panamax

Docker libswarm
● Primitives of container orchestration
● With adaptors to actual implementations
● Docker Remote API abstraction
● Not yet ready for prime time

Mesos
● Primitives for distributed scheduling
● Meta Scheduler
● Writing your own scheduler is “easy”
● Docker Containerizer (since 0.20.0)

Marathon
● Framework for long running processes
● REST API / Webinterface
● Meta Framework

Marathon
● Constraints
● Health Checks
● “Service Discovery” / Load balancing
● Rolling update

CoreOS
● Small linux distribution to manage container
clusters
● Automatic updates
● Using docker for packet management

Fleet
● Systemd for the cluster
● Simple scheduling via Metadata
● Resolves service dependencies
● Health Check / Failover

Kubernetes
● Container cluster management
● Created by Google

Kubernetes Concepts
● Pods
○ A group of containers on the same host
○ They share the same network namespace
○ Eg. presence, ambassador, logging, monitoring,
caching, configuring sidekicks
● Labels
● Service
● Replication Controller

Inception anyone?
● Kubernetes runs on CoreOS
● Mesosphere works on Kubernetes
integration
● Conair - run CoreOS/Kubernetes/Panamax
components in containers

Thanks for listening!
Timo Derstappen
@teemow
http://giantswarm.io
We’re
hiring!

1. Kubernetes and Docker Swarm are container orchestrators that ensure applications have the required number of running instances and provide automatic failover. 2. Kubernetes uses a master-node architecture and deploys configurations declaratively using YAML files. It ensures configurations are consistent and provides built-in health checks. 3. Docker Swarm manages nodes in a cluster using Docker APIs. It provides container placement using pluggable schedulers with strategies like bin packing and spread. It also supports resource management and affinity/anti-affinity filters. 4. Both orchestrators have limitations like complicated deployment and a lack of automatic horizontal scaling. Kubernetes has more advanced functionality for application deployments and health checks.

AtlasCamp 2015: The age of orchestration: From Docker basics to cluster manag...Atlassian

Nicola Paolucci, Atlassian Containers hit the collective developer mind with great force the past two years and created a space of fervent innovation. Now work is moving towards orchestration. In this session we'll cover an overview of the container orchestration landscape, give an introduction to Docker's own tools - machine, swarm and compose - and show a (semi)live demo of how they work in practice.

Building and deploying a distributed application with Docker, Mesos and MarathonJulia Mateo

Platform as a Service with Kubernetes and Mesos Miguel Zuniga

Mesos ♥ DockerAish Fenton

Mesos and Kubernetes ecosystem overviewKrishna-Kumar

Scaling Jenkins with Docker: Swarm, Kubernetes or Mesos?Carlos Sanchez

The Jenkins platform can be dynamically scaled by using several Docker cluster and orchestration platforms, using containers to run slaves and jobs and also isolating job execution. But which cluster technology should be used? Docker Swarm? Apache Mesos? Kubernetes? How do they compare? All of them can be used to dynamically run jobs inside containers. This talk will cover these main container clusters, outlining the pros and cons of each, the current state of the art of the technologies and Jenkins support.

Introduction to Apache MesosJoe Stein

Docker Madison, Introduction to KubernetesTimothy St. Clair

Kubernetes is an open-source system for managing containerized applications across multiple hosts. It provides mechanisms for deploying, maintaining, and scaling applications. Kubernetes uses declarative APIs and controllers to maintain the desired state of applications. The document then discusses key Kubernetes concepts like pods, containers, services, labels, replication controllers, and selectors. It explains how Kubernetes operations work through components like the API server, scheduler, controller manager, kubelet, and proxy.

Kubernetes Introduction & Whats new in Kubernetes 1.6Opcito Technologies

virtualization-vs-containerization-paasrajdeep

Musings on Mesos: Docker, Kubernetes, and Beyond.Timothy St. Clair

Crossing the Streams Mesos <> KubernetesTimothy St. Clair

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that should be co-located into pods and manages replication and rollouts of pods across a cluster of machines. Kubernetes uses controllers to maintain the desired state by rectifying any discrepancies. It provides labels to map structures onto objects and services for load balancing pods behind a single IP address. Mesos and Kubernetes both aim to orchestrate containers but take different approaches, with Mesos focusing on resource isolation and sharing and Kubernetes focusing on deployment and management of applications.

Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)Docker, Inc.

Operating apps at web scale has become the new normal, but has been out of reach for most companies. Join us as we show you how to deploy and manage your Docker containers at scale. See how easy it is to build highly-available, fault-tolerant web scale apps using Docker with the Mesos cluster scheduler. Docker plus Mesos is a new way to scale applications. Together they give you capabilities similar to Google’s Borg, the Googleplex’s secret weapon of scalability and fault tolerance.

Docker and kubernetes_introductionJason Hu

Kubernetes Architecture and Introduction – Paris Kubernetes MeetupStefan Schimanski

The document provides an overview of Kubernetes architecture and introduces how to deploy Kubernetes clusters on different platforms like Mesosphere's DCOS, Google Container Engine, and Mesos/Docker. It discusses the core components of Kubernetes including the API server, scheduler, controller manager and kubelet. It also demonstrates how to interact with Kubernetes using kubectl and view cluster state.

Apache Mesos: a simple explanation of basicsGladson Manuel

Mesos is a distributed systems kernel that allows sharing of computing resources across multiple distributed applications or frameworks. It provides scalability, fault tolerance, and resource isolation. Mesos runs on every machine in a distributed cluster and acts as a mediator between systems and applications. The Mesos architecture uses a master-slave design where the master manages resource offers from slaves to frameworks. Frameworks use schedulers to launch and manage tasks on slaves based on resource availability.

Play Support in Cloud Foundryrajdeep

- Play 2.0 is a web framework for Java and Scala that simplifies development by embracing HTTP rather than fighting it - It takes a new approach to building web apps in Java by not being built on top of servlet APIs and using an asynchronous programming model - Developing, testing, and deploying a Play app locally and to CloudFoundry involves creating a project, running it locally, and pushing the compiled code to CloudFoundry which automatically detects and supports Play apps

HA Kubernetes on Mesos / MarathonCobus Bernard

This document discusses Docker containers and orchestration tools like Kubernetes, Mesos, and Marathon. It summarizes setting up Docker containers on Kubernetes and other orchestration platforms, the challenges of service discovery and load balancing, and difficulties experienced in setting up Kubernetes in a development environment. Overall it provides an overview of Docker and various orchestration tools while recounting lessons learned from a hands-on setup.

Introduction to KubernetesVishal Biyani

Docker, Mesos, Spark Qiang Wang

The document discusses integrating Docker, Mesos, Spark, Marathon, and Chronos into a unified big data platform. Docker provides containerization capabilities, while Mesos is a distributed resource manager that supports running Docker containers. Spark can run natively on Mesos by running Spark as a Docker container within Mesos. Marathon and Chronos help manage long-running services and cron jobs on Mesos. The author will demonstrate how to put these technologies together into an integrated system and address running Spark on Mesos in Docker containers.

An Introduction to KubernetesRohman Muhamad

Kubernetes intro public - kubernetes meetup 4-21-2015Rohit Jnagal

This document introduces Kubernetes, an open-source system for automating deployment, scaling, and management of containerized applications. It was developed at Google based on their 15+ years of running production workloads in containers. Kubernetes can manage applications running on virtual machines, bare metal, public or private cloud providers. It uses a declarative model where users specify the desired state and Kubernetes ensures the actual state matches it. Key concepts include pods, replication controllers, services, labels/selectors, and monitoring/logging addons.

KubeCon 2017: Kubernetes from Dev to ProdSubhas Dandapani

Kubernetes from Dev to Prod summarizes GoEuro's transition from a legacy environment to using Kubernetes and CI/CD pipelines from development to production. Key points: - GoEuro transitioned 50+ services across 150 engineers from separate development and operations teams to using Kubernetes, Docker, and CI/CD pipelines in 4 months. - They developed "hyper-vm" single-node Kubernetes VMs for local development and testing and "y8s" for sharing Kubernetes configurations across environments from development through production. - CI/CD pipelines were automated using GitLab CI and custom implementations running jobs on "hyper-vm" agents to deploy to environments from preview through production. - Additional services

runC: The little engine that could (run Docker containers) by Docker Captain ...Docker, Inc.

With the announcement of the OCI by Solomon Hykes at last summer's DockerCon, a Docker-contributed reference implementation of the OCI spec, called runC, was born. While some of you may have tried runC or have a history of poking at the OS layer integration library to Linux namespaces, cgroups and the like (known as libcontainer), many of you may not know what runC offers. In this talk Phil Estes, Docker engine maintainer who has also contributed to libcontainer and runC, will show what's possible using runC as a lightweight and fast runtime environment to experiment with lower-level features of the container runtime. Phil will introduce a conversion tool called "riddler", which can inspect and convert container configurations from Docker into the proper OCI configuration bundle for easy conversion between the two environments. He'll also demonstrate how to make custom configurations for trying out security features like user namespaces and seccomp profiles.

Kubernetes on Top of Mesos on Top of DCOSStefan Schimanski

Containers, cluster management, microservices, Kubernetes and many other buzzwords are flying around us all the time. Our team is building solutions that make it easy to cope with all the complexity around cluster infrastructure. In this talk we present the project we are working on, namely running Kubernetes on top of the Mesos cluster scheduler. Furthermore we show DCOS which makes it easy to deploy and run Kubernetes with a single command.

Docker at Shopify: From This-Looks-Fun to Production by Simon Eskildsen (Shop...Docker, Inc.

Since July 2014 Shopify's been serving thousands of requests per second of production web traffic from Docker containers. This was an 8 month effort, with multiple pivots of direction from the team—and we're only getting started. This talk covers the lessons learned through the trial and error of an in-flight architecture redesign, spanning hundreds of hosts, as well as the technical vision of the future of our platform.

Batch Applications for the Java PlatformSivakumar Thyagarajan

Cologne Web Performance Optimization Group Web - VarnishD

More Related Content

What's hot (20)

Introduction to Apache MesosJoe Stein

Docker Madison, Introduction to KubernetesTimothy St. Clair

Kubernetes Introduction & Whats new in Kubernetes 1.6Opcito Technologies

virtualization-vs-containerization-paasrajdeep

Musings on Mesos: Docker, Kubernetes, and Beyond.Timothy St. Clair

Crossing the Streams Mesos <> KubernetesTimothy St. Clair

Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)Docker, Inc.

Docker and kubernetes_introductionJason Hu

Kubernetes Architecture and Introduction – Paris Kubernetes MeetupStefan Schimanski

Apache Mesos: a simple explanation of basicsGladson Manuel

Play Support in Cloud Foundryrajdeep

HA Kubernetes on Mesos / MarathonCobus Bernard

Introduction to KubernetesVishal Biyani

Docker, Mesos, Spark Qiang Wang

An Introduction to KubernetesRohman Muhamad

Kubernetes intro public - kubernetes meetup 4-21-2015Rohit Jnagal

KubeCon 2017: Kubernetes from Dev to ProdSubhas Dandapani

runC: The little engine that could (run Docker containers) by Docker Captain ...Docker, Inc.

Kubernetes on Top of Mesos on Top of DCOSStefan Schimanski

Docker at Shopify: From This-Looks-Fun to Production by Simon Eskildsen (Shop...Docker, Inc.

Introduction to Apache MesosJoe Stein

Docker Madison, Introduction to KubernetesTimothy St. Clair

Kubernetes Introduction & Whats new in Kubernetes 1.6Opcito Technologies

virtualization-vs-containerization-paasrajdeep

Musings on Mesos: Docker, Kubernetes, and Beyond.Timothy St. Clair

Crossing the Streams Mesos <> KubernetesTimothy St. Clair

Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)Docker, Inc.

Docker and kubernetes_introductionJason Hu

Kubernetes Architecture and Introduction – Paris Kubernetes MeetupStefan Schimanski

Apache Mesos: a simple explanation of basicsGladson Manuel

Play Support in Cloud Foundryrajdeep

HA Kubernetes on Mesos / MarathonCobus Bernard

Introduction to KubernetesVishal Biyani

Docker, Mesos, Spark Qiang Wang

An Introduction to KubernetesRohman Muhamad

Kubernetes intro public - kubernetes meetup 4-21-2015Rohit Jnagal

KubeCon 2017: Kubernetes from Dev to ProdSubhas Dandapani

runC: The little engine that could (run Docker containers) by Docker Captain ...Docker, Inc.

Kubernetes on Top of Mesos on Top of DCOSStefan Schimanski

Docker at Shopify: From This-Looks-Fun to Production by Simon Eskildsen (Shop...Docker, Inc.

Viewers also liked (19)

Batch Applications for the Java PlatformSivakumar Thyagarajan

Cologne Web Performance Optimization Group Web - VarnishD

Things Your Mother Didnt Tell You About Bundle Configurations - Symfony Live…D

symfony live 2010 - Using symfony events to create clean class interfacesD

The document discusses using events in Symfony applications to decouple classes and increase flexibility, describing how to connect events to listeners, dispatch events, and pass data, as well as some potential issues like lack of notification ordering and increased debugging complexity with events. Examples are provided of using events for actions on state changes, file system operations, and loose coupling of Symfony plugins.

Microservices Docker @Bonn AgileTimo Derstappen

The document discusses microservices and how Docker can be used with microservices. Some key points about microservices are that they are small, focused services that are highly decoupled and independent. Docker is well-suited for microservices because it allows for lightweight, fast containers that are composable. The document also covers how Docker can be used for development, testing, continuous delivery and production of microservices-based applications.

VarnaConf - Blue/Green Deployments with Docker, haproxy and Consulzeridon

Giant Swarm @Devhouse fridayTimo Derstappen

The document introduces Giant Swarm, which addresses challenges of service discovery, scheduling, deployment, and operations for microservices and stateless web applications. It discusses motivations like polyglot persistence and independently deployable services. An example template demonstrates declaring services, their dependencies, and deployment configuration for automated and scalable operations on multiple servers.

Introduction to Docker & CoreOS - Symfony User Group CologneD

This document provides an introduction and overview of Docker and CoreOS. It describes Docker as a tool for isolation processes in lightweight Linux containers and discusses CoreOS, a minimal Linux distribution focused on running modern infrastructure stacks. CoreOS utilizes Docker containers and tools like Etcd for service discovery, Locksmith for updates, Cloud Config for initialization, Flannel for networking, and Fleet for cluster management.

CoreOS @Codetalks HamburgTimo Derstappen

This document introduces CoreOS, an open source operating system focused on automation, security, and scalability. It provides automatic updates, uses Docker containers, and includes tools like Etcd for service discovery and configuration. CoreOS is based on Gentoo Linux and uses systemd. It focuses on immutable infrastructure with atomic updates and rollbacks. The document describes CoreOS tools like Etcd, Locksmith, Cloud Config, Flannel and Fleet for cluster management.

Consul: Service-oriented at ScaleC4Media

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1A8pJF6. Armon Dadgar presents Consul, a distributed control plane for the datacenter. Armon demonstrates how Consul can be used to build, configure, monitor, and orchestrate distributed systems. Filmed at qconsf.com. Armon Dadgar has a passion for distributed systems and their application to real world problems. He is currently the CTO of HashiCorp, where he brings distributed systems into the world of DevOps tooling.

Dockerizing Symfony Applications - Symfony Live Berlin 2014D

This document discusses Dockerizing Symfony applications. It introduces Docker containers as a lightweight virtualization method using Linux namespaces and union filesystems. It then demonstrates how to containerize the components of a sample Symfony app - Redis, PHP-FPM, Nginx - connecting them using Docker links and volumes. Fig is introduced as a tool to define and run multi-container Docker environments using a YAML configuration file, making management of the Symfony app containers easier. Hosting challenges are mentioned along with an invitation to join Giantswarm's private beta for container management.

Prometheus meets Consul -- Consul Casual TalksSatoshi Suzuki

Satoshi Suzuki gave a presentation on integrating Consul and Prometheus. He discussed Consul's ACL system for restricting access and the need to configure permissions for the anonymous token. He explained Prometheus is a monitoring system that pulls metrics and has native integration with Consul using Consul service discovery configurations. The integration allows Prometheus to retrieve scrape targets from Consul's catalog. Targets are removed from Prometheus when the Consul agent changes state to "left" or when the target is removed from Consul service discovery.

Integrating Spark and Solr-(Timothy Potter, Lucidworks)Spark Summit

This document discusses integrating Solr and Spark. It provides an example of using Solr as a sink for streaming data from Spark Streaming. It also describes reading data from Solr into Spark using SolrRDD and exposing it as a Spark SQL DataFrame. Additional capabilities covered include querying Solr from the Spark shell, document matching using stored queries, and reading term vectors from Solr for machine learning with MLLib.

ContainerDays NYC 2015: "Container Orchestration Compared: Kubernetes and Doc...DynamicInfraDays

Container Orchestration WarsKarl Isenberg

Video: https://youtu.be/C_u4_l84ED8 Karl Isenberg reviews the history of distributed computing, clarifies terminology for layers in the container stack, and does a head to head comparison of several tools in the space, including Kubernetes, Marathon, and Docker Swarm. Learn which features and qualities are critical for container orchestration and how you can apply this knowledge when evaluating platforms.

Choosing PaaS: Cisco and Open Source Options: an overviewCisco DevNet

This document discusses container platforms and PaaS. It provides context on containers and supporting technologies like Docker. It describes how containers are limited when confined to a single host, and how schedulers can distribute containers across multiple hosts. It outlines common production tools used with containers like configuration management, monitoring, and logging. It compares PaaS and containers, noting how PaaS consumed containers before they were widely known, and how the lines between the two are blurring as container platforms provide more services. It introduces Mantl as Cisco's container stack designed to run container workloads and big data applications across clouds.

A Gentle Introduction To Docker And All Things ContainersJérôme Petazzoni

Docker is a runtime for Linux Containers. It enables "separation of concern" between devs and ops, and solves the "matrix from hell" of software deployment. This presentation explains it all! It also explains the role of the storage backend and compares the various backends available. It gives multiple recipes to build Docker images, including integration with configuration management software like Chef, Puppet, Salt, Ansible. If you already watched other Docker presentations, this is an actualized version (as of mid-November 2013) of the thing!

Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs...Nati Shalom

Video recording: https://www.youtube.com/watch?v=tGlIgUeoGz8 It’s no news that containers represent a portable unit of deployment, and OpenStack has proven an ideal environment for running container workloads. However, where it usually becomes more complex is that many times an application is often built out of multiple containers. What’s more, setting up a cluster of container images can be fairly cumbersome because you need to make one container aware of another and expose intimate details that are required for them to communicate which is not trivial especially if they’re not on the same host. These scenarios have instigated the demand for some kind of orchestrator. The list of container orchestrators is growing fairly fast. This session will compare the different orchestation projects out there - from Heat to Kubernetes to TOSCA - and help you choose the right tool for the job. Session link from teh summit: https://openstacksummitmay2015vancouver.sched.org/event/abd484e0dedcb9774edda1548ad47518#.VV5eh5NViko

Cluster SchedulersPietro Michiardi

Batch Applications for the Java PlatformSivakumar Thyagarajan

Cologne Web Performance Optimization Group Web - VarnishD

Things Your Mother Didnt Tell You About Bundle Configurations - Symfony Live…D

symfony live 2010 - Using symfony events to create clean class interfacesD

Microservices Docker @Bonn AgileTimo Derstappen

VarnaConf - Blue/Green Deployments with Docker, haproxy and Consulzeridon

Giant Swarm @Devhouse fridayTimo Derstappen

Introduction to Docker & CoreOS - Symfony User Group CologneD

CoreOS @Codetalks HamburgTimo Derstappen

Consul: Service-oriented at ScaleC4Media

Dockerizing Symfony Applications - Symfony Live Berlin 2014D

Prometheus meets Consul -- Consul Casual TalksSatoshi Suzuki

Integrating Spark and Solr-(Timothy Potter, Lucidworks)Spark Summit

ContainerDays NYC 2015: "Container Orchestration Compared: Kubernetes and Doc...DynamicInfraDays

Container Orchestration WarsKarl Isenberg

Choosing PaaS: Cisco and Open Source Options: an overviewCisco DevNet

A Gentle Introduction To Docker And All Things ContainersJérôme Petazzoni

Orchestration tool roundup kubernetes vs. docker vs. heat vs. terra form vs...Nati Shalom

Cluster SchedulersPietro Michiardi

Similar to Container Orchestration @Docker Meetup Hamburg (20)

The journey to container adoption in enterpriseIgor Moochnick

This document discusses the journey to container adoption in enterprises. It begins by describing traditional monolithic architectures and then discusses how container technologies like Docker and Mesos enable new paradigms like microservices that emphasize speed, agility and loose coupling. It covers challenges around deployment, testing, monitoring and failure handling with containers and discusses emerging tools and approaches to address these challenges. Finally, it considers future directions like Kubernetes and stream processing architectures.

Intro to cluster scheduler for Linux containersKumar Gaurav

Cluster schedulers help manage resources and schedule jobs across multiple nodes in a container cluster. The main types are monolithic, two-level, and shared-state schedulers. Popular cluster schedulers for containers include Mesos, Docker Swarm, and Kubernetes. Each takes a different approach to scheduling but aim to provide high resource utilization, placement constraints, rapid scheduling, and fairness.

Cncf meetup-rookJuraj Hantak

Rook.io is an open source cloud-native storage orchestrator for Kubernetes that supports various storage providers like Ceph and allows provisioning of block storage, object storage, and shared file systems using CRDs; it provides a framework to deploy and manage different storage systems on Kubernetes including Ceph which is a highly scalable distributed storage solution for production environments. The presenter demonstrates configuring and deploying Rook operator for Ceph which manages the Ceph components and allows dynamically growing or replacing the underlying storage infrastructure.

Cncf meetup-rookJuraj Hantak

Rook.io is an open source cloud-native storage orchestrator for Kubernetes that supports various storage providers like Ceph and allows provisioning of block storage, object storage, and shared file systems using CRDs; it provides a framework to deploy and manage different storage systems on Kubernetes including Ceph which is a highly scalable distributed storage solution for block storage, object storage, and file systems that can be run using Kubernetes primitives. The presenter demonstrates configuring and deploying Rook operator for Ceph which manages the Ceph cluster through CRDs to provide storage for applications like MySQL.

7. Cloud Native Computing - Kubernetes - Bratislava - Rook.ioDávid Kőszeghy

A Storage Orchestrator for Kubernetes Rook turns distributed storage systems into self-managing, self-scaling, self-healing storage services. It automates the tasks of a storage administrator: deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management. Rook uses the power of the Kubernetes platform to deliver its services: cloud-native container management, scheduling, and orchestration.

Deploying WSO2 Middleware on MesosImesh Gunaratne

This document summarizes deploying WSO2 middleware on Apache Mesos. It discusses using Mesos and Docker to deploy WSO2 products as containers in a clustered and scalable way. Key points include: Mesos architecture and features for scheduling resources and containers; using WSO2 Docker images with Mesos; a reference Carbon architecture on Mesos using Marathon load balancers; and the deployment workflow of building Docker images, importing them to a registry, and deploying Mesos artifacts.

An Introduction to KubernetesImesh Gunaratne

Traditional virtualization technologies have been used by cloud infrastructure providers for many years in providing isolated environments for hosting applications. These technologies make use of full-blown operating system images for creating virtual machines (VMs). According to this architecture, each VM needs its own guest operating system to run application processes. More recently, with the introduction of the Docker project, the Linux Container (LXC) virtualization technology became popular and attracted the attention. Unlike VMs, containers do not need a dedicated guest operating system for providing OS-level isolation, rather they can provide the same level of isolation on top of a single operating system instance. An enterprise application may need to run a server cluster to handle high request volumes. Running an entire server cluster on Docker containers, on a single Docker host could introduce the risk of single point of failure. Google started a project called Kubernetes to solve this problem. Kubernetes provides a cluster of Docker hosts for managing Docker containers in a clustered environment. It provides an API on top of Docker API for managing docker containers on multiple Docker hosts with many more features.

Craig Box (Google) - The road to Kubernetes 1.0Outlyer

Apache Cassandra Lunch #41: Cassandra on Kubernetes - Docker/Kubernetes/Helm ...Anant Corporation

In Apache Cassandra Lunch #41: Apache Cassandra Lunch #41: Cassandra on Kubernetes - Docker/Kubernetes/Helm Part 1, we discuss Cassandra on Kubernetes and give an introduction to Docker, Kubernetes, and Helm. Accompanying Blog: https://blog.anant.us/apache-cassandra-lunch-41-cassandra-on-kubernetes-docker-kubernetes-helm-part-1/ Accompanying YouTube: https://youtu.be/-I8cKQO_Qr0 Sign Up For Our Newsletter: http://eepurl.com/grdMkn Join Cassandra Lunch Weekly at 12 PM EST Every Wednesday: https://www.meetup.com/Cassandra-DataStax-DC/events/ Cassandra.Link: https://cassandra.link/ Follow Us and Reach Us At: Anant: https://www.anant.us/ Awesome Cassandra: https://github.com/Anant/awesome-cassandra Cassandra.Lunch: https://github.com/Anant/Cassandra.Lunch Email: [email protected] LinkedIn: https://www.linkedin.com/company/anant/ Twitter: https://twitter.com/anantcorp Eventbrite: https://www.eventbrite.com/o/anant-1072927283 Facebook: https://www.facebook.com/AnantCorp/

Containerization - The DevOps RevolutionYulian Slobodyan

Containerization allows for standardized and isolated application deployment across infrastructure through container runtimes like Docker. Key features that enable containers include namespaces which isolate processes and their views of the system, cgroups which control resource usage, and copy-on-write storage for efficient application packaging. A container orchestration system like Mesos provides scalability, fault tolerance, and unified resource management across clusters. This allows maximizing infrastructure utilization through flexible scheduling of containerized applications and services on shared clusters.

Apache Mesos Overview and IntegrationAlex Baretto

Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications running on a shared pool of nodes. It provides APIs for resource management and scheduling and can run applications like Hadoop, Spark, and Kafka across large compute clusters. Mesos was created at Berkeley and adopted by companies like Twitter, Airbnb, and Hubspot to improve resource utilization, allow dynamic scaling of services, and reduce the operational complexity of managing large fleets of servers. Mesos also integrates with Docker and Kubernetes to enable containerized workloads and services to be deployed and orchestrated on Mesos clusters.

On Prem Container Cloud - Lessons LearnedCodeOps Technologies LLP

This document discusses best practices for setting up a Docker and Kubernetes environment. It covers topics like Docker storage drivers, Docker networking, Kubernetes authentication and authorization, exposing services, integrating logging and monitoring. The key recommendations are to use OverlayFS or Devicemapper storage, Calico networking, Kubernetes security policies, namespaces and RBAC, and Ingress for exposing services. ETCD backup and fault tolerance is also important.

Docker. Does it matter for Java developer ?Izzet Mustafaiev

OSDC 2015: Bernd Mathiske | Why the Datacenter Needs an Operating SystemNETWAYS

Developers are moving away from their host-based patterns and adopting a new mindset around the idea that the datacenter is the computer. It?s quickly becoming a mainstream model that you can view a warehouse full of servers as a single computer (with terabytes of memory and tens of thousands of cores). There is a key missing piece, which is an operating system for the datacenter (DCOS), which would provide the same OS functionality and core OS abstractions across thousands of machines that an OS provides on a single machine today. In this session, we will discuss: How the abstraction of an OS has evolved over time and can cleanly scale to spand thousands of machines in a datacenter. How key open source technologies like the Apache Mesos distributed systems kernel provide the key underpinnings for a DCOS. How developers can layer core system services on top of a distributed systems kernel, including an init system (Marathon), cron (Chronos), service discovery (DNS), and storage (HDFS) What would the interface to the DCOS look like? How would you use it? How you would install and operate datacenter services, including Apache Spark, Apache Cassandra, Apache Kafka, Apache Hadoop, Apache YARN, Apache HDFS, and Google's Kubernetes. How will developers build datacenter-scale apps, programmed against the datacenter OS like it?s a single machine?

Monitoring with prometheus at scaleJuraj Hantak

Prometheus is an open-source monitoring system that can scale from a single server to large clusters. While a single Prometheus instance is sufficient for many deployments, tools like Trickster, sharding, and Thanos/Cortex can help scale Prometheus further. Trickster caches query results to reduce load, sharding splits data across servers, and Thanos provides high availability and long term storage across multiple clusters.

Monitoring with prometheus at scaleAdam Hamsik

Introduction to mesosOmid Vahdaty

Mesos is a cluster manager that provides efficient resource sharing across distributed applications. It sits between applications and the operating system to make deploying and managing applications in large clusters more efficient. Mesos introduces distributed two-level scheduling where it decides how to allocate resources to frameworks, which then decide how to use those resources. It provides features like fault tolerance, scalability, resource isolation, and APIs for building distributed apps. Common uses include running Hadoop, Spark, Storm, Jenkins, and Docker on Mesos clusters.

Introduction to containers a practical session using core os and dockerAlessandro Martellone

The document provides an introduction to containers using CoreOS and Docker. It discusses cloud computing stacks including IaaS, PaaS, and SaaS and container solutions such as Docker, CoreOS, and Kubernetes. It then focuses on CoreOS, describing its components including systemd, etcd, and fleetd. The document also discusses microservices architectures and concludes with information about doing a practical lab using CoreOS and Docker.

Containers kuberenetescsegayan

Recent momentum around the evolution of Containers are gradually increase in last two years.Containers virtualize an OS and applications running in each container believe that they have full access to their very own copy of that OS. This is analogous to what VMs do when they virtualize at a lower level, the hardware. In the case of containers, it’s the OS that does the virtualization and maintains the illusion.

Integration in the age of DevOpsAlbert Wong

In an increasingly competitive marketplace, speed and business agility are paramount. And integration between customer-facing systems and back-end applications is more crucial than ever. At this event, you'll learn how open source software built by communities, like Apache Camel, Docker, Kubernetes, OpenShift Origin, and Fabric8, can help organizations integrate services and establish effective continuous integration and delivery (CI/CD) pipelines.

The journey to container adoption in enterpriseIgor Moochnick

Intro to cluster scheduler for Linux containersKumar Gaurav

Cncf meetup-rookJuraj Hantak

7. Cloud Native Computing - Kubernetes - Bratislava - Rook.ioDávid Kőszeghy

Deploying WSO2 Middleware on MesosImesh Gunaratne

An Introduction to KubernetesImesh Gunaratne

Craig Box (Google) - The road to Kubernetes 1.0Outlyer

Apache Cassandra Lunch #41: Cassandra on Kubernetes - Docker/Kubernetes/Helm ...Anant Corporation

Containerization - The DevOps RevolutionYulian Slobodyan

Apache Mesos Overview and IntegrationAlex Baretto

On Prem Container Cloud - Lessons LearnedCodeOps Technologies LLP

Docker. Does it matter for Java developer ?Izzet Mustafaiev

OSDC 2015: Bernd Mathiske | Why the Datacenter Needs an Operating SystemNETWAYS

Monitoring with prometheus at scaleJuraj Hantak

Monitoring with prometheus at scaleAdam Hamsik

Introduction to mesosOmid Vahdaty

Introduction to containers a practical session using core os and dockerAlessandro Martellone

Containers kuberenetescsegayan

Integration in the age of DevOpsAlbert Wong

Recently uploaded (20)

The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfAbi john

TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc

Most consumers believe they’re making informed decisions about their personal data—adjusting privacy settings, blocking trackers, and opting out where they can. However, our new research reveals that while awareness is high, taking meaningful action is still lacking. On the corporate side, many organizations report strong policies for managing third-party data and consumer consent yet fall short when it comes to consistency, accountability and transparency. This session will explore the research findings from TrustArc’s Privacy Pulse Survey, examining consumer attitudes toward personal data collection and practical suggestions for corporate practices around purchasing third-party data. Attendees will learn: - Consumer awareness around data brokers and what consumers are doing to limit data collection - How businesses assess third-party vendors and their consent management operations - Where business preparedness needs improvement - What these trends mean for the future of privacy governance and public trust This discussion is essential for privacy, risk, and compliance professionals who want to ground their strategies in current data and prepare for what’s next in the privacy landscape.

Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan

This is a Quick Research Guide (QRG). QRGs include the following: - A brief, high-level overview of the QRG topic. - A milestone timeline for the QRG topic. - Links to various free online resource materials to provide a deeper dive into the QRG topic. - Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic. QRGs planned for the series: - Artificial Intelligence QRG - Quantum Computing QRG - Big Data Analytics QRG - Spacecraft Guidance, Navigation & Control QRG (coming 2026) - UK Home Computing & The Birth of ARM QRG (coming 2027) Any questions or comments? - Please contact Arthur Morgan at [email protected]. 100% human made.

TrsLabs - Fintech Product & Business ConsultingTrs Labs

Hybrid Growth Mandate Model with TrsLabs Strategic Investments, Inorganic Growth, Business Model Pivoting are critical activities that business don't do/change everyday. In cases like this, it may benefit your business to choose a temporary external consultant. An unbiased plan driven by clearcut deliverables, market dynamics and without the influence of your internal office equations empower business leaders to make right choices. Getting things done within a budget within a timeframe is key to Growing Business - No matter whether you are a start-up or a big company Talk to us & Unlock the competitive advantage

Special Meetup Edition - TDX Bengaluru Meetup #52.pptxshyamraj55

Build Your Own Copilot & Agents For DevsBrian McKeiver

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada

Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next. Link to recording, transcript, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/ Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.

Drupalcamp Finland – Measuring Front-end Energy ConsumptionExove

How Can I use the AI Hype in my Business Context?Daniel Lehner

𝙄𝙨 𝘼𝙄 𝙟𝙪𝙨𝙩 𝙝𝙮𝙥𝙚? 𝙊𝙧 𝙞𝙨 𝙞𝙩 𝙩𝙝𝙚 𝙜𝙖𝙢𝙚 𝙘𝙝𝙖𝙣𝙜𝙚𝙧 𝙮𝙤𝙪𝙧 𝙗𝙪𝙨𝙞𝙣𝙚𝙨𝙨 𝙣𝙚𝙚𝙙𝙨? Everyone’s talking about AI but is anyone really using it to create real value? Most companies want to leverage AI. Few know 𝗵𝗼𝘄. ✅ What exactly should you ask to find real AI opportunities? ✅ Which AI techniques actually fit your business? ✅ Is your data even ready for AI? If you’re not sure, you’re not alone. This is a condensed version of the slides I presented at a Linkedin webinar for Tecnovy on 28.04.2025.

Mobile App Development Company in Saudi ArabiaSteve Jonas

EmizenTech is a globally recognized software development company, proudly serving businesses since 2013. With over 11+ years of industry experience and a team of 200+ skilled professionals, we have successfully delivered 1200+ projects across various sectors. As a leading Mobile App Development Company In Saudi Arabia we offer end-to-end solutions for iOS, Android, and cross-platform applications. Our apps are known for their user-friendly interfaces, scalability, high performance, and strong security features. We tailor each mobile application to meet the unique needs of different industries, ensuring a seamless user experience. EmizenTech is committed to turning your vision into a powerful digital product that drives growth, innovation, and long-term success in the competitive mobile landscape of Saudi Arabia.

Splunk Security Update | Public Sector Summit Germany 2025Splunk

HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungenpanagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-und-verwaltung-von-multiuser-umgebungen/ HCL Nomad Web wird als die nächste Generation des HCL Notes-Clients gefeiert und bietet zahlreiche Vorteile, wie die Beseitigung des Bedarfs an Paketierung, Verteilung und Installation. Nomad Web-Client-Updates werden “automatisch” im Hintergrund installiert, was den administrativen Aufwand im Vergleich zu traditionellen HCL Notes-Clients erheblich reduziert. Allerdings stellt die Fehlerbehebung in Nomad Web im Vergleich zum Notes-Client einzigartige Herausforderungen dar. Begleiten Sie Christoph und Marc, während sie demonstrieren, wie der Fehlerbehebungsprozess in HCL Nomad Web vereinfacht werden kann, um eine reibungslose und effiziente Benutzererfahrung zu gewährleisten. In diesem Webinar werden wir effektive Strategien zur Diagnose und Lösung häufiger Probleme in HCL Nomad Web untersuchen, einschließlich - Zugriff auf die Konsole - Auffinden und Interpretieren von Protokolldateien - Zugriff auf den Datenordner im Cache des Browsers (unter Verwendung von OPFS) - Verständnis der Unterschiede zwischen Einzel- und Mehrbenutzerszenarien - Nutzung der Client Clocking-Funktion

Procurement Insights Cost To Value Guide.pptxJon Hansen

What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat

The MCP (Model Context Protocol) is a framework designed to manage context and interaction within complex systems. This SlideShare presentation will provide a detailed overview of the MCP Model, its applications, and how it plays a crucial role in improving communication and decision-making in distributed systems. We will explore the key concepts behind the protocol, including the importance of context, data management, and how this model enhances system adaptability and responsiveness. Ideal for software developers, system architects, and IT professionals, this presentation will offer valuable insights into how the MCP Model can streamline workflows, improve efficiency, and create more intuitive systems for a wide range of use cases.

Semantic Cultivators : The Critical Future Role to Enable AIartmondano

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Impelsys Inc.

Dev Dives: Automate and orchestrate your processes with UiPath MaestroUiPathCommunity

This session is designed to equip developers with the skills needed to build mission-critical, end-to-end processes that seamlessly orchestrate agents, people, and robots. 📕 Here's what you can expect: - Modeling: Build end-to-end processes using BPMN. - Implementing: Integrate agentic tasks, RPA, APIs, and advanced decisioning into processes. - Operating: Control process instances with rewind, replay, pause, and stop functions. - Monitoring: Use dashboards and embedded analytics for real-time insights into process instances. This webinar is a must-attend for developers looking to enhance their agentic automation skills and orchestrate robust, mission-critical processes. 👨‍🏫 Speaker: Andrei Vintila, Principal Product Manager @UiPath This session streamed live on April 29, 2025, 16:00 CET. Check out all our upcoming Dev Dives sessions at https://community.uipath.com/dev-dives-automation-developer-2025/.

tecnologias de las primeras civilizaciones.pdffjgm517

The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfAbi john

TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc

Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan

TrsLabs - Fintech Product & Business ConsultingTrs Labs

Special Meetup Edition - TDX Bengaluru Meetup #52.pptxshyamraj55

Build Your Own Copilot & Agents For DevsBrian McKeiver

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada

Drupalcamp Finland – Measuring Front-end Energy ConsumptionExove

How Can I use the AI Hype in my Business Context?Daniel Lehner

Mobile App Development Company in Saudi ArabiaSteve Jonas

Splunk Security Update | Public Sector Summit Germany 2025Splunk

HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungenpanagenda

Procurement Insights Cost To Value Guide.pptxJon Hansen

What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat

Semantic Cultivators : The Critical Future Role to Enable AIartmondano

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Impelsys Inc.

Dev Dives: Automate and orchestrate your processes with UiPath MaestroUiPathCommunity

tecnologias de las primeras civilizaciones.pdffjgm517

Container Orchestration @Docker Meetup Hamburg

1. Container Orchestration Timo Derstappen @teemow

2. Why Orchestration? ● Microservice Architectures ● Polyglot Persistence ● Fault tolerance / High Availability ● Scalability ● Utilization

3. One Single Computer

5. The Beginning

6. Overview ● Schedulers ● Configuration Management ● Service Discovery

7. Scheduler ● Resource management ● Decide which job runs where ● Monitor jobs ● Dependency resolution

8. Scheduler Concepts ● Google Borg (Monolithic) ● Apache Mesos (Two Level) ● Hadoop on Demand (Two Level) ● Google Omega (Shared State) Read: “Omega: flexible, scalable schedulers for large compute clusters” http://research.google.com/pubs/pub41684.html

9. Static Partitioning

10. Static Partitioning http://cto.vmware.com/wp-content/uploads/2012/10/mesos_workloads.png

11. Configuration Management ● State which container runs where ● Configure your containers ● Basis for service discovery

12. Consistent Cluster State ● Etcd (Raft) ● Consul (Raft) ● Doozer (Paxos) ● Zookeeper (Paxos) ● Spanner (Paxos) http://raftconsensus.github.io/

13. Service Discovery ● Ambassador Pattern ● Consul (DNS) ● Etcd ● Haproxy / confd

14. Ambassador Pattern

15. Docker libchan ● Link containers from everywhere ● Unix Socket ● HTTP2 / TLS ● Messaging / Raw TCP ● Like Go Channels over the network

16. Many Choices ● CoreOS ● Kubernetes ● Apache Mesos ● Consul / Serf ● Project Atomic / Geard ● Panamax

18. Docker libswarm ● Primitives of container orchestration ● With adaptors to actual implementations ● Docker Remote API abstraction ● Not yet ready for prime time

19. Mesos ● Primitives for distributed scheduling ● Meta Scheduler ● Writing your own scheduler is “easy” ● Docker Containerizer (since 0.20.0)

21. Mesos Apps

22. Marathon ● Framework for long running processes ● REST API / Webinterface ● Meta Framework

23. Marathon ● Constraints ● Health Checks ● “Service Discovery” / Load balancing ● Rolling update

24. CoreOS ● Small linux distribution to manage container clusters ● Automatic updates ● Using docker for packet management

25. CoreOS ● Fleet ● Etcd

26. Fleet ● Systemd for the cluster ● Simple scheduling via Metadata ● Resolves service dependencies ● Health Check / Failover

27. Kubernetes ● Container cluster management ● Created by Google

28. Kubernetes Concepts ● Pods ○ A group of containers on the same host ○ They share the same network namespace ○ Eg. presence, ambassador, logging, monitoring, caching, configuring sidekicks ● Labels ● Service ● Replication Controller

29. Kubernetes Example

30. Inception anyone? ● Kubernetes runs on CoreOS ● Mesosphere works on Kubernetes integration ● Conair - run CoreOS/Kubernetes/Panamax components in containers

31. Thanks for listening! Timo Derstappen @teemow http://giantswarm.io We’re hiring!

Container Orchestration @Docker Meetup Hamburg

Recommended

More Related Content

What's hot (20)

Viewers also liked (19)

Similar to Container Orchestration @Docker Meetup Hamburg (20)

Recently uploaded (20)

Container Orchestration @Docker Meetup Hamburg