Scaling Jenkins with Kubernetes by Ami Mahloof
Download as PPTX, PDF1 like961 views
This document discusses scaling Jenkins with Kubernetes. Previously, Jenkins was run on single EC2 instances with issues like underutilization and port collisions. The new approach runs Jenkins on Kubernetes, allowing on-demand PODs for each build with complete isolation. A modified Kubernetes plugin was created to support multiple containers and persistent volumes per POD. Builds are queued and run on Kubernetes nodes, improving scalability. Demostrating this approach saves 70-90% over running Jenkins on always-on EC2 instances by using spot instances for Kubernetes nodes.
1 of 14
Downloaded 15 times
Ad
Recommended
Kubernetes @ Nanit by Chen Fisher
Kubernetes @ Nanit by Chen FisherDoiT International This document summarizes Chen Fisher's presentation on Kubernetes at nanit. The key points are:
1) Nanit uses Kubernetes for service orchestration, taking advantage of its built-in service discovery, high availability features, and port management capabilities.
2) Kubernetes is preferred over Amazon ECS due to ECS lacking service discovery and only supporting basic health checks.
3) Nanit runs two Kubernetes clusters for staging and production, with over 100 pods across more than 20 instances in production.
4) Nanit uses Kubernetes for deploying microservices from development to production, with Jenkins doing automated testing and deployment via Slack notifications.
Scaling jenkins with kubernetes
Scaling jenkins with kubernetesAmi Mahloof This document discusses scaling Jenkins with Kubernetes. Previously, Jenkins was run on single EC2 instances with issues like underutilization and port collisions. The new approach runs Jenkins on Kubernetes, allowing on-demand PODs for each build with complete isolation. A modified Kubernetes plugin was created to support multiple containers and persistent volumes per POD. Builds now have isolated environments, persistent storage, and can scale across Kubernetes nodes. This saves costs by using spot instances for Kubernetes nodes managed by Spot Fleets.
Running Production-Grade Kubernetes on AWS
Running Production-Grade Kubernetes on AWSDoiT International Kubernetes has been a key component for many companies to reduce technical debt in infrastructure by:
• Fostering the Adoption of Docker
• Simplifying Container Management
• Onboarding Developers On Infrastructure
• Unlocking Continuous Integration and Delivery
During this meetup we are going to discuss the following topics and share some best practices
• What's new with Kubernetes 1.3
• Generate Cluster Configuration using CloudFormation
• Deploy Kubernetes Clusters on AWS
• Scaling the Cluster
• Integrating Ingress with Elastic Load Balancer
• Using Internal ELB's as Kubernetes' Service
• Using EBS for persistent volumes
• Integrating Route53
CI Implementation with Kubernetes at LivePerson by Saar Demri
CI Implementation with Kubernetes at LivePerson by Saar DemriDoiT International How LivePerson have built their Continuous Integration flow with Kubernetes to streamline development to production rollouts.
Continuous Deployment with Jenkins on Kubernetes
Continuous Deployment with Jenkins on KubernetesMatt Baldwin Google Senior Software Engineer Evan Brown's presentation from the March 18, 2016 Seattle Kubernetes meetup hosted by StackPointCloud. Evan shows how you deploy Jenkins into Kubernetes, then takes us through CD and canary deployments. Join us in Seattle: http://www.meetup.com/Seattle-Kubernetes-Meetup/
Deploying apps with Docker and Kubernetes
Deploying apps with Docker and KubernetesDaniel Fenton This was a demo of deploying a Koop (https://koopjs.github.io) onto Kubernetes using a Docker container.
Container Orchestration using Kubernetes
Container Orchestration using KubernetesHesham Amin This document provides an overview of Kubernetes concepts including:
- Docker containers are individual processes that make up applications, not complete applications themselves.
- Kubernetes manages container deployments through services that define relationships between containers and provide load balancing, discovery, scaling, updates and rollbacks.
- Kubernetes was created by Google and is supported on platforms like Google Kubernetes Engine, Azure Container Service, and others.
- Kubernetes manages clusters of nodes that host containers through masters nodes that control workers.
- The basic building block is the Pod, which contains one or more containers that share resources and networking.
Scaling Docker Containers using Kubernetes and Azure Container Service
Scaling Docker Containers using Kubernetes and Azure Container ServiceBen Hall This document discusses scaling Docker containers using Kubernetes and Azure Container Service. It begins with an introduction to containers and Docker, including how containers improve dependency and configuration management. It then demonstrates building and deploying containerized applications using Docker and discusses how to optimize Docker images. Finally, it introduces Kubernetes as a tool for orchestrating containers at scale and provides an example of deploying a containerized application on Kubernetes in Azure.
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka 
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka Mario Ishara Fernando This document discusses microservices and containers. It provides an overview of microservices architecture compared to monolithic architecture, highlighting that microservices are composed of many small, independent services with separate deployments and databases. It then discusses containers and how Docker is used to package and run applications in isolated containers. Finally, it introduces Kubernetes as a container orchestration system to manage and scale multiple containerized applications across a cluster of machines.
Effective Building your Platform with Kubernetes == Keep it Simple 
Effective Building your Platform with Kubernetes == Keep it Simple Wojciech Barczyński Effective Kubernetes is a continuous deployment process that the team understands. Keep it Simple. Think twice before going for more complex solutions.
Source: https://github.com/wojciech12/talk_effective_kubernetes
Presented at Cloud Native Talks #2 (Online Meetup) - https://www.meetup.com/Cloud-Native-Kubernetes-Warsaw/events/257125529/
Kubernetes HA @ AppDirect - Montreal Kubernetes Meetup
Kubernetes HA @ AppDirect - Montreal Kubernetes Meetupalexgervais HA Kubernetes Deployment by Alexandre Gervais, Senior Software Developper, AppDirect
* How AppDirect deploys HA Kubernetes clusters using a multi-master setup
* Kubernetes upgrades and lifecycle
Hands on docker
Hands on dockerinovex GmbH inovex Meetup: Let´s talk about docker!
Speaker: Johannes Scheuermann, inovex GmbH
Karlsruhe, 18.12.2014
Mehr Meetups:
http://www.meetup.com/inovex-karlsruhe
http://www.meetup.com/inovex-cologne
http://www.meetup.com/inovex-munich
Containerizing a REST API and Deploying to Kubernetes
Containerizing a REST API and Deploying to KubernetesAshley Roach This document discusses containerizing a REST microservice and deploying it to Kubernetes. It begins by explaining why to build a REST API using Swagger and containerization. It then demonstrates containerizing a sample REST API created with Swagger-node. Finally, it covers deploying the containerized REST API to Kubernetes, including using Kubernetes templates for the deployment and service, and deploying manually or through a CI system.
Net core, mssql, container und kubernetes
Net core, mssql, container und kubernetesThomas Fricke This document discusses Google Cloud Platform and related technologies like .NET Core, SQL Server, containers, and Kubernetes. It provides an agenda for a user group meeting that will cover containers, Kubernetes, CoreOS, creating and running a .NET application in a Docker container, container registries, setting up a local Kubernetes cluster, and using pods and services. There is also information about Endocode, a company that provides software solutions and open source projects using technologies like these.
Lifecycle of a pod
Lifecycle of a podHarshal Shah Pods are the smallest deployable units in Kubernetes. A Pod contains one or more containers that share resources and can communicate through localhost. The lifecycle of a Pod includes phases like Pending, Running, Succeeded, Failed, and Unknown. Probes like liveness and readiness probes are used to check the health of containers. Init containers allow running pre-startup actions before app containers are started.
Kubernetes - Starting with 1.2
Kubernetes - Starting with 1.2William Stewart This talk will focus on a brief history, including a demo and overview of how we at Superbalist use Kubernetes, and how Kubernetes uses Docker, does load balancing, deployments, and data migrations.
Talk from Cape Town DevOps meetup on Jun 21, 2016:
https://www.meetup.com/Cape-Town-DevOps/events/231530172/
Code: https://github.com/zoidbergwill/kubernetes-examples
Slides as markdown: http://www.zoidbergwill.com/presentations/2016/kubernetes-1.2-and-spread/index.md
Kubernetes101 - Pune Kubernetes Meetup 6
Kubernetes101 - Pune Kubernetes Meetup 6Harshal Shah This document provides an overview and agenda for a hands-on Kubernetes workshop. The workshop will cover Kubernetes concepts like pods, deployments, services, labels and selectors. It will demonstrate how to set up a Kubernetes cluster on Google Cloud and on a local laptop. Attendees will get hands-on experience with deploying applications and performing rolling updates using Kubernetes primitives.
Docker storage designing a platform for persistent data
Docker storage designing a platform for persistent dataDocker, Inc. This document summarizes a presentation about Docker storage and persistent data. It discusses immutable Docker images, applications with persistent data requirements, and different approaches to persistent storage with Docker including per-container storage, shared storage on the same host, multi-host shared storage, Docker volume plugins, and orchestrating storage with Swarm and Kubernetes. Key takeaways are that shared storage allows for smaller images, efficient usage of repetitive data, and decoupling of applications and data, and that running databases in containers is acceptable if requirements for latency, IOPS, bandwidth, and security are met.
Kubernetes deep dive - - Huawei 2015-10
Kubernetes deep dive - - Huawei 2015-10Vishnu Kannan Kubernetes is an open-source container orchestration system that automates deployment, scaling, and management of containerized applications. It was originally designed by Google based on years of experience running containers internally. Kubernetes runs containerized applications across multiple machines, dynamically allocating resources and balancing load. It supports both public and private cloud environments as well as bare metal servers. The system aims to simplify container operations while providing portability and scalability.
Kubernetes Architecture and Introduction
Kubernetes Architecture and IntroductionStefan Schimanski - Archeology: before and without Kubernetes
- Deployment: kube-up, DCOS, GKE
- Core Architecture: the apiserver, the kubelet and the scheduler
- Compute Model: the pod, the service and the controller
Bitbucket Pipelines - Powered by Kubernetes
Bitbucket Pipelines - Powered by KubernetesNathan Burrell This talk covers how pipelines uses Kubernetes to power its builder infrastructure and shares some tips on running Kubernetes at scale in a secure way.
This presentation was presented to the sydney Kubernetes meetup on the 3rd of August 2017.
Kubernetes 101
Kubernetes 101Jacopo Nardiello This document provides an introduction and overview of Kubernetes for deploying and managing containerized applications at scale. It discusses Kubernetes' key features like self-healing, dynamic scaling, networking and efficient resource usage. It then demonstrates setting up a Kubernetes cluster on AWS and deploying a sample application using pods, deployments and services. While Kubernetes provides many benefits, the document notes it requires battle-testing to be production-ready and other topics like logging, monitoring and custom autoscaling solutions would need separate discussions.
Kubernetes Introduction
Kubernetes IntroductionRed Hat Developers This document discusses Kubernetes and container orchestration. It begins with an introduction to containers and their advantages. It then introduces Kubernetes as an open source container orchestration system and discusses some of its key concepts like pods, replication controllers, services, labels, and persistent volumes. It also provides an overview of a sample application and Kubernetes lab for hands-on learning.
Kubernetes persistence 101
Kubernetes persistence 101Kublr In this meetup, Oleg, CTO at Kublr, walks you through the basics of K8s persistence management functionality and how it can be used to simplify managing persistent applications across different environments - in the cloud or on premise. Oleg will use a demo environment with clusters in different clouds to show K8s persistence in practice.
We will cover:
• Persistent data abstractions in K8s: persistent volumes (PV) and their attributes
• PV specifics in different clouds
• Using PV in K8s: persistent volume claims (PVC) and storage classes (SC)
• Automatic volume provisioning
• Persistence and scheduling interrelationships
• Practical examples
Kubernetes (K8s) is a powerful and flexible open source container orchestration system. The power of K8s comes from its modularity and simplicity of basic concepts. Each of these basic concepts build on the other and, from the most basic elements to more advanced ones, each is responsible for its own well-defined logic and behavior.
Continuous Delivery to Kubernetes Using Helm
Continuous Delivery to Kubernetes Using HelmAdnan Abdulhussein This document discusses using Helm and continuous delivery to deploy applications to Kubernetes. It provides an overview of CI/CD pipelines for Kubernetes and how Helm can be used to package and deploy Kubernetes applications and their dependencies. The document includes an example Helm deployment pipeline with stages for building, testing, releasing container images, and deploying applications to staging and production environments in Kubernetes.
Kubernetes extensibility
Kubernetes extensibilityDocker, Inc. Kubernetes is designed to be an extensible system. But what is the vision for Kubernetes Extensibility? Do you know the difference between webhooks and cloud providers, or between CRI, CSI, and CNI? In this talk we will explore what extension points exist, how they have evolved, and how to use them to make the system do new and interesting things. We’ll give our vision for how they will probably evolve in the future, and talk about the sorts of things we expect the broader Kubernetes ecosystem to build with them.
Deep dive in container service discovery
Deep dive in container service discoveryDocker, Inc. This document discusses service discovery and load balancing in Kubernetes. It begins by defining service discovery and explaining why it is important. It then demonstrates how Kubernetes implements service discovery using Deployments, Services, and Endpoints. It explains how kube-proxy performs load balancing using different modes like iptables and IPVS. It also covers topics like hairpin traffic, persistence, and alternatives to kube-proxy. Overall, the document provides an in-depth look at how service discovery and load balancing work under the hood in Kubernetes.
Kubernetes Introduction
Kubernetes IntroductionMartin Danielsson A basic introductory slide set on Kubernetes: What does Kubernetes do, what does Kubernetes not do, which terms are used (Containers, Pods, Services, Replica Sets, Deployments, etc...) and how basic interaction with a Kubernetes cluster is done.
Kubernetes - State of the Union (Q1-2016)
Kubernetes - State of the Union (Q1-2016)DoiT International Kubernetes is making the promise of changing the datacenter from being a group of computer to "a computer" itself. This presentation outlines the new features in K8S with 1.1 and 1.2 release.
Kubernetes kubecon-roundup
Kubernetes kubecon-roundupSebastien Goasguen Kubecon saw significant growth in attendance from 350 people in March 2016 to 1000 people in November 2016. The Kubernetes project also continues to grow rapidly with over 1000 contributors now and the highest velocity GitHub project. Key announcements included Kubernetes support on Azure Container Service, new projects in the Kubernetes incubator, and upcoming features like StatefulSets in version 1.5. Overall, the conference highlighted Kubernetes' transition to an established ecosystem for cloud native applications.
Ad
More Related Content
What's hot (20)
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka Mario Ishara Fernando This document discusses microservices and containers. It provides an overview of microservices architecture compared to monolithic architecture, highlighting that microservices are composed of many small, independent services with separate deployments and databases. It then discusses containers and how Docker is used to package and run applications in isolated containers. Finally, it introduces Kubernetes as a container orchestration system to manage and scale multiple containerized applications across a cluster of machines.
Effective Building your Platform with Kubernetes == Keep it Simple
Effective Building your Platform with Kubernetes == Keep it Simple Wojciech Barczyński Effective Kubernetes is a continuous deployment process that the team understands. Keep it Simple. Think twice before going for more complex solutions.
Source: https://github.com/wojciech12/talk_effective_kubernetes
Presented at Cloud Native Talks #2 (Online Meetup) - https://www.meetup.com/Cloud-Native-Kubernetes-Warsaw/events/257125529/
Kubernetes HA @ AppDirect - Montreal Kubernetes Meetup
Kubernetes HA @ AppDirect - Montreal Kubernetes Meetupalexgervais HA Kubernetes Deployment by Alexandre Gervais, Senior Software Developper, AppDirect
* How AppDirect deploys HA Kubernetes clusters using a multi-master setup
* Kubernetes upgrades and lifecycle
Hands on docker
Hands on dockerinovex GmbH inovex Meetup: Let´s talk about docker!
Speaker: Johannes Scheuermann, inovex GmbH
Karlsruhe, 18.12.2014
Mehr Meetups:
http://www.meetup.com/inovex-karlsruhe
http://www.meetup.com/inovex-cologne
http://www.meetup.com/inovex-munich
Containerizing a REST API and Deploying to Kubernetes
Containerizing a REST API and Deploying to KubernetesAshley Roach This document discusses containerizing a REST microservice and deploying it to Kubernetes. It begins by explaining why to build a REST API using Swagger and containerization. It then demonstrates containerizing a sample REST API created with Swagger-node. Finally, it covers deploying the containerized REST API to Kubernetes, including using Kubernetes templates for the deployment and service, and deploying manually or through a CI system.
Net core, mssql, container und kubernetes
Net core, mssql, container und kubernetesThomas Fricke This document discusses Google Cloud Platform and related technologies like .NET Core, SQL Server, containers, and Kubernetes. It provides an agenda for a user group meeting that will cover containers, Kubernetes, CoreOS, creating and running a .NET application in a Docker container, container registries, setting up a local Kubernetes cluster, and using pods and services. There is also information about Endocode, a company that provides software solutions and open source projects using technologies like these.
Lifecycle of a pod
Lifecycle of a podHarshal Shah Pods are the smallest deployable units in Kubernetes. A Pod contains one or more containers that share resources and can communicate through localhost. The lifecycle of a Pod includes phases like Pending, Running, Succeeded, Failed, and Unknown. Probes like liveness and readiness probes are used to check the health of containers. Init containers allow running pre-startup actions before app containers are started.
Kubernetes - Starting with 1.2
Kubernetes - Starting with 1.2William Stewart This talk will focus on a brief history, including a demo and overview of how we at Superbalist use Kubernetes, and how Kubernetes uses Docker, does load balancing, deployments, and data migrations.
Talk from Cape Town DevOps meetup on Jun 21, 2016:
https://www.meetup.com/Cape-Town-DevOps/events/231530172/
Code: https://github.com/zoidbergwill/kubernetes-examples
Slides as markdown: http://www.zoidbergwill.com/presentations/2016/kubernetes-1.2-and-spread/index.md
Kubernetes101 - Pune Kubernetes Meetup 6
Kubernetes101 - Pune Kubernetes Meetup 6Harshal Shah This document provides an overview and agenda for a hands-on Kubernetes workshop. The workshop will cover Kubernetes concepts like pods, deployments, services, labels and selectors. It will demonstrate how to set up a Kubernetes cluster on Google Cloud and on a local laptop. Attendees will get hands-on experience with deploying applications and performing rolling updates using Kubernetes primitives.
Docker storage designing a platform for persistent data
Docker storage designing a platform for persistent dataDocker, Inc. This document summarizes a presentation about Docker storage and persistent data. It discusses immutable Docker images, applications with persistent data requirements, and different approaches to persistent storage with Docker including per-container storage, shared storage on the same host, multi-host shared storage, Docker volume plugins, and orchestrating storage with Swarm and Kubernetes. Key takeaways are that shared storage allows for smaller images, efficient usage of repetitive data, and decoupling of applications and data, and that running databases in containers is acceptable if requirements for latency, IOPS, bandwidth, and security are met.
Kubernetes deep dive - - Huawei 2015-10
Kubernetes deep dive - - Huawei 2015-10Vishnu Kannan Kubernetes is an open-source container orchestration system that automates deployment, scaling, and management of containerized applications. It was originally designed by Google based on years of experience running containers internally. Kubernetes runs containerized applications across multiple machines, dynamically allocating resources and balancing load. It supports both public and private cloud environments as well as bare metal servers. The system aims to simplify container operations while providing portability and scalability.
Kubernetes Architecture and Introduction
Kubernetes Architecture and IntroductionStefan Schimanski - Archeology: before and without Kubernetes
- Deployment: kube-up, DCOS, GKE
- Core Architecture: the apiserver, the kubelet and the scheduler
- Compute Model: the pod, the service and the controller
Bitbucket Pipelines - Powered by Kubernetes
Bitbucket Pipelines - Powered by KubernetesNathan Burrell This talk covers how pipelines uses Kubernetes to power its builder infrastructure and shares some tips on running Kubernetes at scale in a secure way.
This presentation was presented to the sydney Kubernetes meetup on the 3rd of August 2017.
Kubernetes 101
Kubernetes 101Jacopo Nardiello This document provides an introduction and overview of Kubernetes for deploying and managing containerized applications at scale. It discusses Kubernetes' key features like self-healing, dynamic scaling, networking and efficient resource usage. It then demonstrates setting up a Kubernetes cluster on AWS and deploying a sample application using pods, deployments and services. While Kubernetes provides many benefits, the document notes it requires battle-testing to be production-ready and other topics like logging, monitoring and custom autoscaling solutions would need separate discussions.
Kubernetes Introduction
Kubernetes IntroductionRed Hat Developers This document discusses Kubernetes and container orchestration. It begins with an introduction to containers and their advantages. It then introduces Kubernetes as an open source container orchestration system and discusses some of its key concepts like pods, replication controllers, services, labels, and persistent volumes. It also provides an overview of a sample application and Kubernetes lab for hands-on learning.
Kubernetes persistence 101
Kubernetes persistence 101Kublr In this meetup, Oleg, CTO at Kublr, walks you through the basics of K8s persistence management functionality and how it can be used to simplify managing persistent applications across different environments - in the cloud or on premise. Oleg will use a demo environment with clusters in different clouds to show K8s persistence in practice.
We will cover:
• Persistent data abstractions in K8s: persistent volumes (PV) and their attributes
• PV specifics in different clouds
• Using PV in K8s: persistent volume claims (PVC) and storage classes (SC)
• Automatic volume provisioning
• Persistence and scheduling interrelationships
• Practical examples
Kubernetes (K8s) is a powerful and flexible open source container orchestration system. The power of K8s comes from its modularity and simplicity of basic concepts. Each of these basic concepts build on the other and, from the most basic elements to more advanced ones, each is responsible for its own well-defined logic and behavior.
Continuous Delivery to Kubernetes Using Helm
Continuous Delivery to Kubernetes Using HelmAdnan Abdulhussein This document discusses using Helm and continuous delivery to deploy applications to Kubernetes. It provides an overview of CI/CD pipelines for Kubernetes and how Helm can be used to package and deploy Kubernetes applications and their dependencies. The document includes an example Helm deployment pipeline with stages for building, testing, releasing container images, and deploying applications to staging and production environments in Kubernetes.
Kubernetes extensibility
Kubernetes extensibilityDocker, Inc. Kubernetes is designed to be an extensible system. But what is the vision for Kubernetes Extensibility? Do you know the difference between webhooks and cloud providers, or between CRI, CSI, and CNI? In this talk we will explore what extension points exist, how they have evolved, and how to use them to make the system do new and interesting things. We’ll give our vision for how they will probably evolve in the future, and talk about the sorts of things we expect the broader Kubernetes ecosystem to build with them.
Deep dive in container service discovery
Deep dive in container service discoveryDocker, Inc. This document discusses service discovery and load balancing in Kubernetes. It begins by defining service discovery and explaining why it is important. It then demonstrates how Kubernetes implements service discovery using Deployments, Services, and Endpoints. It explains how kube-proxy performs load balancing using different modes like iptables and IPVS. It also covers topics like hairpin traffic, persistence, and alternatives to kube-proxy. Overall, the document provides an in-depth look at how service discovery and load balancing work under the hood in Kubernetes.
Kubernetes Introduction
Kubernetes IntroductionMartin Danielsson A basic introductory slide set on Kubernetes: What does Kubernetes do, what does Kubernetes not do, which terms are used (Containers, Pods, Services, Replica Sets, Deployments, etc...) and how basic interaction with a Kubernetes cluster is done.
Viewers also liked (20)
Kubernetes - State of the Union (Q1-2016)
Kubernetes - State of the Union (Q1-2016)DoiT International Kubernetes is making the promise of changing the datacenter from being a group of computer to "a computer" itself. This presentation outlines the new features in K8S with 1.1 and 1.2 release.
Kubernetes kubecon-roundup
Kubernetes kubecon-roundupSebastien Goasguen Kubecon saw significant growth in attendance from 350 people in March 2016 to 1000 people in November 2016. The Kubernetes project also continues to grow rapidly with over 1000 contributors now and the highest velocity GitHub project. Key announcements included Kubernetes support on Azure Container Service, new projects in the Kubernetes incubator, and upcoming features like StatefulSets in version 1.5. Overall, the conference highlighted Kubernetes' transition to an established ecosystem for cloud native applications.
Evolução das arquiteturas
Evolução das arquiteturasBruno Nunes Trassante Time line simplista do passo a passo da evolução das tecnologias e modelo das arquiteturas de software nos últimos 20 anos
Web scale infrastructures with kubernetes and flannel
Web scale infrastructures with kubernetes and flannelpurpleocean La capacità di rispondere in poche frazioni di secondo alle richieste degli utenti - indipendentemente dal loro numero - è un fattore determinante per il successo dei servizi sul web. Secondo Amazon, bastano 100 millisecondi di latenza nella risposta per generare una perdita economica di circa l'1% sul
fatturato [1]. In base alle statistiche di Google AdWords, inoltre, il 2015 ha sancito l’ufficiale superamento del numero di interazioni mobile rispetto a quelle desktop [2], con la conseguente riduzione della durata media delle sessioni di navigazione web.
In uno scenario di questo tipo, la razionalizzazione dell’utilizzo delle risorse hardware e la capacità di scalare rispetto al numero di utenti sono fattori determinanti per il successo del business.
In questo talk racconteremo la nostra esperienza di migrazione di soluzioni e-commerce di tipo enterprise in Magento da un’architettura basata su VM tradizionali ad una di tipo software-defined basata su Kubernetes, Flannel e Docker. Discuteremo, quindi, delle reali difficoltà da noi incontrate nel porting su container di soluzioni in produzione e daremo evidenza di come, alla fine di questo lungo viaggio, i nostri sforzi siano stati concretamente premiati dall’aumento di resilienza, affidabilità e automazione della soluzione finale.
A supporto della conversazione, mostreremo i risultati dei benchmark da noi condotti per valutare la scalabilità della nuova architettura presentando delle evidenze delle reali capacità di Kubernetes come strumento di orchestrazione di servizi erogati in Docker container.
Concluderemo l’intervento presentando il nostro progetto di distribuzione geografica dei nodi master di Kubernetes facendo uso di reti SD-WAN per garantire performance e continuità di servizio della soluzione.
KubeCon EU 2016: Heroku to Kubernetes
KubeCon EU 2016: Heroku to KubernetesKubeAcademy Arkena's video-on-demand platform is used as backend by major european channels (TF1 / beIN SPORTS / Elisa) to propose a non-linear experience to their customers.
Previously hosted on Heroku, the number of our users is increasing constantly. In order to optimize resources we decided to move on a bare metal infrastructure powered by Kubernetes.
We'll share thoughts, feedbacks and technical details about this successful transition.
Sched Link:
Kube-AWS
Kube-AWSCoreOS This document discusses Kube-AWS, which is a tool for deploying Kubernetes clusters on AWS. It outlines the design goals of creating artifacts that are secure, reproducible, and auditable. It then demonstrates "under the hood" how Kube-AWS works by initializing a cluster configuration, rendering assets, deploying the cluster, exporting the deployment details, and making changes to reproduce the cluster. Recent work is noted along with future plans.
2016 08-30 Kubernetes talk for Waterloo DevOps
2016 08-30 Kubernetes talk for Waterloo DevOpscraigbox This document discusses Kubernetes and container orchestration on Google Cloud Platform. It provides an overview of Kubernetes and how it allows users to manage applications and deploy containers across clusters. Key points include that Kubernetes was created at Google and is now open source, it provides tools for scheduling, load balancing and ensuring availability of containerized applications, and that adoption is growing rapidly across startups and enterprises due to benefits like portability and ease of updating clusters.
From dev to prod: Kubernetes on AWS (short ver.)
From dev to prod: Kubernetes on AWS (short ver.)佑介 九岡 The document discusses Yusuke Kuoka's recommendations and experience for deploying Kubernetes on AWS from development to production, including tools for bootstrapping Kubernetes clusters on AWS, running local development environments, automating the deployment process, and implementing logging and monitoring across environments using DaemonSets and Concourse CI for continuous integration. It also notes challenges around achieving high availability with Kubernetes and etcd in the Tokyo region and ideas for further improvements.
Building .NET Microservices
Building .NET MicroservicesVMware Tanzu SpringOne Platform 2016
Speakers: Kevin Hoffman; Advisory Solutions Architect, Pivotal & Chris Umbel; Advisory Architect, Pivotal
With the advent of ASP.NET Core, developers can now build cross-platform microservices in .NET. We can build services on the Mac, Windows, or Linux and deploy anywhere--most importantly to the cloud.
In this session we'll talk about Cloud Native .NET, building .NET microservices, and deploying them to the cloud. We'll build services that participate in a robust ecosystem by consuming OSS servers such as Spring Cloud Configuration Server and Eureka. We'll also show how these .NET microservices can take advantage of circuit breakers and be automatically deployed to the cloud via CI/CD pipelines.
Micro Services in .NET Core and Docker
Micro Services in .NET Core and Dockercjmyers This document discusses using .NET Core and Docker for microservices. It begins with an overview of why Docker and microservices are useful. It then discusses why .NET Core and Microsoft technologies are good choices for building microservices. The document demonstrates creating a simple .NET Core app as a Docker container. It also discusses microservices patterns like having a database per service and isolating service instances. The document concludes with information about prerequisites for the demos and asking if there are any questions.
A brief study on Kubernetes and its components
A brief study on Kubernetes and its componentsRamit Surana Kubernetes is an open source orchestration system for Docker containers. It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions. Using the concepts of "labels" and "pods", it groups the containers which make up an application into logical units for easy management and discovery.
Kubernetes Networking - Giragadurai Vallirajan
Kubernetes Networking - Giragadurai VallirajanNeependra Khare The document discusses Docker networking and Kubernetes networking concepts. It provides an overview of Docker networking and how containers on the same host can communicate. It then summarizes key Kubernetes concepts like pods, replication controllers, services and networking. It demonstrates how to set up a sample application topology in Kubernetes using replication controllers and services. It also discusses exposing services externally and additional resources for learning about Docker and Kubernetes.
Tutorial on using CoreOS Flannel for Docker networking
Tutorial on using CoreOS Flannel for Docker networkingLorisPack Project Flannel is an overlay based networking technique for networking Docker containers on CoreOS platforms. This tutorial explains the theory, setup instructions and limtations of the mechanism.
Container Network Interface: Network Plugins for Kubernetes and beyond
Container Network Interface: Network Plugins for Kubernetes and beyondKubeAcademy With the rise of modern containers comes new problems to solve – especially in networking. Numerous container SDN solutions have recently entered the market, each best suited for a particular environment. Combined with multiple container runtimes and orchestrators available today, there exists a need for a common layer to allow interoperability between them and the network solutions.
As different environments demand different networking solutions, multiple vendors and viewpoints look to a specification to help guide interoperability. Container Network Interface (CNI) is a specification started by CoreOS with the input from the wider open source community aimed to make network plugins interoperable between container execution engines. It aims to be as common and vendor-neutral as possible to support a wide variety of networking options — from MACVLAN to modern SDNs such as Weave and flannel.
CNI is growing in popularity. It got its start as a network plugin layer for rkt, a container runtime from CoreOS. Today rkt ships with multiple CNI plugins allowing users to take advantage of virtual switching, MACVLAN and IPVLAN as well as multiple IP management strategies, including DHCP. CNI is getting even wider adoption with Kubernetes adding support for it. Kubernetes accelerates development cycles while simplifying operations, and with support for CNI is taking the next step toward a common ground for networking. For continued success toward interoperability, Kubernetes users can come to this session to learn the CNI basics.
This talk will cover the CNI interface, including an example of how to build a simple plugin. It will also show Kubernetes users how CNI can be used to solve their networking challenges and how they can get involved.
KubeCon schedule link: http://sched.co/4VAo
Google BigQuery 101 & What’s New
Google BigQuery 101 & What’s NewDoiT International Vadim Solovey is a CTO of DoiT International has helped to implement Google BigQuery as a cloud data warehouse for many medium and large sized data and analytics initiatives. BigQuery’s serverless architecture had redefined what it means to be fully managed for hundreds of Israeli's startups.
Recently, Google announced an update to BigQuery that dramatically advances cloud data analytics for large-scale businesses such as BigQuery now support Standard SQL, implementing the SQL 2011 standard as well as new ODBC drivers making it possible to use BigQuery with a number of tools ranging from Microsoft Excel to traditional business intelligence systems such as Microstrategy and Qlik.
Agenda:
• Partitioned tables
• The ability to update, delete rows and columns using SQL
• Integration with IAM for fine-grained security policies
• Monitoring w/ StackDriver to track performance and usage
• Query sharing via links, to foster knowledge within orgs
• Cost optimisation strategies
AWS Athena vs. Google BigQuery for interactive SQL Queries
AWS Athena vs. Google BigQuery for interactive SQL QueriesDoiT International During the re:Invent 2016, AWS has released the Amazon Athena - an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.
We took a look on AWS Athena and compared it to the Google BigQuery - another player of serverless interactive data analysis.
Would you like to know which one is the right tool for you? Join us for this meetup to learn AWS Athena and for the test drive of querying exactly the same dataset using AWS Athena and Google BigQuery to see where each one shines (or totally blows it).
Tectonic Summit 2016: Kubernetes 1.5 and Beyond
Tectonic Summit 2016: Kubernetes 1.5 and BeyondCoreOS Kubernetes 1.5 introduces several new features to simplify cluster setup and improve scheduling. It provides an easy way to initialize a Kubernetes cluster with a single command using kubeadm. Multiple clusters can also be easily federated together using kubefed. Additionally, Kubernetes 1.5 enhances scheduling capabilities with taints and tolerations, which allow pods to be selectively scheduled to nodes based on hardware requirements like GPUs. This helps optimize workload placement on large, heterogeneous clusters.
Scaling Docker with Kubernetes
Scaling Docker with KubernetesCarlos Sanchez DevoxxFR 2015 Talk http://cfp.devoxx.fr/2015/talk/WXY-1157/Scaling_Docker_with_Kubernetes
Kubernetes is an open source project to manage a cluster of Linux containers as a single system, managing and running Docker containers across multiple Docker hosts, offering co-location of containers, service discovery and replication control. It was started by Google and now it is supported by Microsoft, RedHat, IBM and Docker Inc amongst others.
Once you are using Docker containers the next question is how to scale and start containers across multiple Docker hosts, balancing the containers across them. Kubernetes also adds a higher level API to define how containers are logically grouped, allowing to define pools of containers, load balancing and affinity.
Kubernetes networking in AWS
Kubernetes networking in AWSZvika Gazit The attached is a summary of terms, description of constructs, integration alternatives and more in the networking world of Kubernetes, Openshift and AWS
Kubernetes Basics
Kubernetes BasicsEueung Mulyana This document provides an overview of Kubernetes including:
1) Kubernetes is an open-source platform for automating deployment, scaling, and operations of containerized applications. It provides container-centric infrastructure and allows for quickly deploying and scaling applications.
2) The main components of Kubernetes include Pods (groups of containers), Services (abstract access to pods), ReplicationControllers (maintain pod replicas), and a master node running key components like etcd, API server, scheduler, and controller manager.
3) The document demonstrates getting started with Kubernetes by enabling the master on one node and a worker on another node, then deploying and exposing a sample nginx application across the cluster.
Ad
Similar to Scaling Jenkins with Kubernetes by Ami Mahloof (20)
Pipeline as code - new feature in Jenkins 2
Pipeline as code - new feature in Jenkins 2Michal Ziarnik What is pipeline as code in continuous delivery/continuous deployment environment.
How to set up Multibranch pipeline to fully benefit from pipeline features.
Jenkins master-node concept in Kubernetes cluster.
Developer Experience Cloud Native - From Code Gen to Git Commit without a CI/...
Developer Experience Cloud Native - From Code Gen to Git Commit without a CI/...Michael Hofmann Developing cloud native applications bring in a lot of complexities for developers. Without using tools to compensate these complexities, you will not become very efficient. Additional, cloud developers often suffer a rising frustration, by fighting these problems.
Before I push my code into Git, I want to test different things in my cloud environment. Therefore it is essential to have a fast and easy round trip. A classic round trip starts by writing or generating code, create a Docker image, deploy it into Kubernetes and test or remote debug the application in Docker or in Kubernetes. Without some elementary tools, this round trip will not be very fast or simple and therefore error prone.
This Lab will show you some open source tools, making your live as a developer more easy. Short demos will demonstrate the simple handling of these tools. Starting point is the generation of a MicroProfile and a SpringBoot application. By using the different tools (e.g. Helm, Shell completion, kubectl cp, Ksync, Stern, Kubefwd, Telepresence, …) on these applications, the complete round trip will be shown. Most of these tools can also be used with other programming languages. Every tool works on its own which makes it easy to switch between these tools.
Finally you will get an evaluation of these tools and I will show you an outlook on tools which are more focused on larger developer teams.
Jenkins as a Service - Code all the way down
Jenkins as a Service - Code all the way downSteve Mactaggart This presentation walks through a Jenkins as Code approach that aims to fully automate and describe the creation of Infrastructure, Application and Configuration as Code.
We treat our applications with a strong 'as code' approach, but often forget about the critical operational tools. This presentation shows how it is possible to create a code first approach to creating and managing a Jenkins Service.
Working code repository is available at https://bitbucket.org/stevemac/dockerfiles
Docker for .NET Developers - Michele Leroux Bustamante, Solliance
Docker for .NET Developers - Michele Leroux Bustamante, SollianceDocker, Inc. This document discusses how Docker can help .NET developers by allowing them to containerize their .NET applications. It provides benefits like running applications across Windows and Linux containers, improving development workflows through features of Docker for Windows like Linux containers on Windows, and facilitating continuous integration and deployment of containerized applications through automation. The document provides examples of Dockerfiles for different types of .NET applications and demonstrates how to add Docker support and containerize existing applications using Visual Studio tools.
CIbox - OpenSource solution for making your #devops better
CIbox - OpenSource solution for making your #devops betterAndrii Podanenko This document describes an old and new development workflow for code reviews and continuous integration. The old workflow involved directly committing code to a shared master branch and deploying to a development server, while the new workflow uses feature branches, pull requests, and local virtual environments for development. It also introduces CIBox, an open source project that provides tools and automation to implement the new workflow, including provisioning a CI server and setting up initial project files.
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipeline
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipelineKubeAcademy One of the most underrated features of Kubernetes is namespaces. In the market, instead of using this feature, people are still stuck with having different clusters for their environments. This talk will try to break this approach, and will introduce how we end up using ephemeral namespaces within our CI/CD pipeline. It will cover the architecture of our system for running the user acceptance tests on isolated ephemeral namespaces with every bits and pieces running within pods. While doing this, we will set up our CI/CD pipeline on top of TravisCI, GoCD, and Selenium that is controlled by Nightwatch.js.
Sched Link: http://sched.co/6Bcb
Deploying Windows Apps to Kubernetes with Draft and Helm
Deploying Windows Apps to Kubernetes with Draft and HelmJessica Deen This document provides an overview and summary of a presentation about deploying Windows applications with Kubernetes, Draft, and Helm. It begins with introductions and disclaimers about the hands-on lab. It then discusses the history of building Kubernetes on Windows and mixed clusters. The presentation demonstrates deploying applications across Windows and Linux nodes using kubectl and shows how to consider Windows-specific aspects like resource limits and node selection. It also covers Helm charts for application deployment and management and using Draft to simplify and automate application development and deployment to Kubernetes.
Docker Enterprise Workshop - Technical
Docker Enterprise Workshop - TechnicalPatrick Chanezon Docker Enterprise Workshop, more technical presentation:
- Docker 101
- Kubernetes in Docker
- Windows containers
Fabric8: Better Software Faster with Docker, Kubernetes, Jenkins
Fabric8: Better Software Faster with Docker, Kubernetes, JenkinsBurr Sutter These days time to value is critical. We all need to develop better software and to get it to customers faster. The quicker the software gets to customers, the more iterations and feedback then the more value we create. Continuous Delivery of Containerized Microservices is the best approach today for developing better software by iterating fast, capturing user feedback and iterating again, continuously.
In this talk, we'll introduce the Fabric8 Microservices Platform which is open source, Apache licensed and built on top of Docker, Kubernetes, OpenShift and Jenkins. We'll demonstrate how to create microservices easily, have them automatically built, tested and staged with Continuous Delivery and deal with the issues of supporting microservices architectures like service discovery, high availability, auto scaling, rolling upgrades, canary deployments, circuit breakers, tracing, logging and metrics.
Kubernetes: training micro-dragons for a serious battle
Kubernetes: training micro-dragons for a serious battleAmir Moghimi My second version of how to use kubernetes effectively as a platform for developing microservice applications.
Kubernetes Node Deep Dive
Kubernetes Node Deep DiveLei (Harry) Zhang Kubernetes nodes provide the infrastructure for containers to run by implementing container-centric features, networking, volumes, and interfacing with container runtimes. The node is an unsung hero that bridges the Kubernetes control plane with containers. Pods are the atomic scheduling unit in Kubernetes and are implemented at the node level, allowing for complex workloads using multiple cooperating containers with shared resources. Kubelets run the sync loop that manages pods and containers on each node by communicating with the API server and container runtime.
Deploying your web application with AWS ElasticBeanstalk
Deploying your web application with AWS ElasticBeanstalkJulien SIMON This document summarizes AWS Elastic Beanstalk, which is a PaaS that allows developers to deploy and manage applications in the AWS cloud. It supports many programming languages and frameworks out of the box. Elastic Beanstalk handles the infrastructure management including provisioning servers, load balancing, auto-scaling and monitoring. Developers can focus on their code without worrying about managing the underlying infrastructure. The document provides an example of deploying a Ruby on Rails application to Elastic Beanstalk and accessing other AWS services.
Kubernetes
KubernetesLinjith Kunnon This document provides an overview of container orchestration with Kubernetes. It begins with recapping container and Docker concepts like namespaces, cgroups, and union filesystems. It then introduces Kubernetes architecture including components like kube-apiserver, kubelet and kube-proxy. Common Kubernetes objects like pods, services, replica sets and deployments are described. The document also covers Kubernetes networking with options like NodePort, LoadBalancer and Ingress. Additional topics include service discovery, logging/monitoring and persistent storage.
Revolutionizing WSO2 PaaS with Kubernetes & App Factory
Revolutionizing WSO2 PaaS with Kubernetes & App FactoryImesh Gunaratne WSO2 Con USA 2015 tutorial session on running WSO2 products on Kubernetes and the features of WSO2 App Factory.
An Introduction to Kubernetes and Continuous Delivery Fundamentals
An Introduction to Kubernetes and Continuous Delivery FundamentalsAll Things Open Presented at All Things Open RTP Meetup
Presented by Brad Topol
Title: An Introduction to Kubernetes and Continuous Delivery Fundamentals
Abstract: Kubernetes is a cloud infrastructure that has emerged as the de facto standard platform for managing, orchestrating, and provisioning container-based cloud native computing applications. Cloud native computing applications are built from a collection of smaller services and take advantage of the speed of development and scalability cloud computing environments provide. In this talk, we provide an overview of the fundamentals of Kubernetes. We begin with a short introduction to the concept of containers and describe the Kubernetes architecture. We then present several core features provided by Kubernetes such as Pods, ReplicaSets, Deployments, Service objects, and autoscaling capabilities. We conclude with a discussion of Kubernetes continuous delivery fundamentals and tools, including how to do small batch changes, source control, and developer access to production-like environments.
Using Containers for Continuous Integration and Continuous Delivery
Using Containers for Continuous Integration and Continuous DeliveryCarlos Sanchez Building and testing is a great use case for containers, both due to the dynamic and isolation aspects, but it increases complexity when scaling to multiple nodes and clusters.
However, the Kubernetes project provides a container orchestration solution that greatly simplifies app deployments in large clusters, and allows to dynamically run any containerized workload. Jenkins is an example of an application that can take advantage of such technology to run Continuous Integration and Continuous Delivery workloads.
The Jenkins Kubernetes plugin can transparently use on demand containers to run build agents and jobs, and isolate job execution. It also supports CI/CD-as-code using Jenkins Pipelines.
The presentation will allow a better understanding of Kubernetes, and how to use Jenkins on Kubernetes for container based large scale, showing also the challenges of running distributed applications (particularly JVM apps).
Using containers for continuous integration and continuous delivery - Carlos ...
Using containers for continuous integration and continuous delivery - Carlos ...Paris Container Day This document discusses using containers for continuous integration and delivery with Jenkins. It covers running Jenkins in Docker containers, using Docker plugins with Jenkins, and running Jenkins pipelines in Kubernetes. Scaling Jenkins with more build agents or masters is discussed. When resource limits are exceeded in containers, the JVM or CPU is restricted instead of the container being killed. The Jenkins Kubernetes plugin allows dynamically running Jenkins agents as Kubernetes pods.
Dockerizing a Symfony2 application
Dockerizing a Symfony2 applicationRoman Rodomansky Dockerizing Symfony2 application. Why Docker is so cool And what is Docker? And what are Containers? How they works? What are the ecosystem of Docker? And how to dockerize your web application (can be based on Symfony2 framework)?
KVM and docker LXC Benchmarking with OpenStack
KVM and docker LXC Benchmarking with OpenStackBoden Russell Passive benchmarking with docker LXC and KVM using OpenStack hosted in SoftLayer. These results provide initial incite as to why LXC as a technology choice offers benefits over traditional VMs and seek to provide answers as to the typical initial LXC question -- "why would I consider Linux Containers over VMs" from a performance perspective.
Results here provide insight as to:
- Cloudy ops times (start, stop, reboot) using OpenStack.
- Guest micro benchmark performance (I/O, network, memory, CPU).
- Guest micro benchmark performance of MySQL; OLTP read, read / write complex and indexed insertion.
- Compute node resource consumption; VM / Container density factors.
- Lessons learned during benchmarking.
The tests here were performed using OpenStack Rally to drive the OpenStack cloudy tests and various other linux tools to test the guest performance on a "micro level". The nova docker virt driver was used in the Cloud scenario to realize VMs as docker LXC containers and compared to the nova virt driver for libvirt KVM.
Please read the disclaimers in the presentation as this is only intended to be the "chip of the ice burg".
Docker and containers - Presentation Slides by Priyadarshini Anand
Docker and containers - Presentation Slides by Priyadarshini AnandPRIYADARSHINI ANAND The document provides an overview of Docker containers and how to get started with Docker. It discusses what containers are, how Docker works, the differences between containers and VMs, and how to use basic Docker commands. It also covers creating Docker images using Dockerfiles and provides examples of common Dockerfile commands.
Ad
More from DoiT International (13)
Terraform Modules Restructured
Terraform Modules RestructuredDoiT International Terraform modules provide reusable, composable infrastructure components. The document discusses restructuring infrastructure code into modules to make it more reusable, testable, and maintainable. Key points include:
- Modules should be structured in a three-tier hierarchy from primitive resources to generic services to specific environments.
- Testing modules individually increases confidence in changes.
- Storing module code and versions in Git provides versioning and collaboration.
- Remote state allows infrastructure to be shared between modules and deployments.
GAN training with Tensorflow and Tensor Cores
GAN training with Tensorflow and Tensor CoresDoiT International Training Generative Adversarial Networks is a gentle process. Recent advances in GANs research resulted in incredible results of generated images. However, training GANs still remains tricky and slow.
With TensorFlow and NVidia’s new Volta architecture it is possible to reduce training time by up to 50% using simplified code.
Orchestrating Redis & K8s Operators
Orchestrating Redis & K8s OperatorsDoiT International The document discusses an agenda for a meetup about Redis Labs and Kubernetes operators. Key points:
- An introduction to Redis Enterprise architecture and Redis Labs products.
- A discussion of "double orchestration" using Kubernetes and PKS to manage Redis clusters for performance and resource management.
- An overview of Redis Labs' Kubernetes solution using StatefulSets, services, and a custom controller.
- An introduction to operators, how they provide lifecycle management and simplify deployments compared to static YAML files or Helm.
- Details on Redis Labs' operator development process and challenges in building idempotent APIs and handling state changes and validation in the reconciliation loop.
K8s best practices from the field!
K8s best practices from the field!DoiT International Learn from the dozens of large-scale deployments how to get the most out of your Kubernetes environment:
- Container images optimization
- Organizing namespaces
- Readiness and Liveness probes
- Resource requests and limits
- Failing with grace
- Mapping external services
- Upgrading clusters with zero downtime
An Open-Source Platform to Connect, Manage, and Secure Microservices
An Open-Source Platform to Connect, Manage, and Secure MicroservicesDoiT International Services are at the core of modern software architecture. Deploying a series of modular, small (micro-)services rather than big monoliths gives developers the flexibility to work in different languages, technologies and release cadence across the system; resulting in higher productivity and velocity, especially for larger teams.
With the adoption of microservices, however, new problems emerge due to the sheer number of services that exist in a larger system. Problems that had to be solved once for a monolith, like security, load balancing, monitoring, and rate limiting need to be handled for each service.
Istio, announced at GlueCon 2017, addresses these problems in a fundamental way through a service mesh framework. With Istio, developers can implement the core logic for the microservices, and let the framework take care of the rest – traffic management, discovery, service identity and security, and policy enforcement. Better yet, this can be also done for existing microservices without rewriting or recompiling any of their parts. Istio uses Envoy as its runtime proxy component and provides an extensible intermediation layer which allows global cross-cutting policy enforcement and telemetry collection.
Is your Elastic Cluster Stable and Production Ready?
Is your Elastic Cluster Stable and Production Ready?DoiT International So you are deployed to production (or soon to be) with Elasticsearch running and powering important application features. Or maybe used for centralized logging for effective debugging.
Was your Elastic cluster deployed correctly? Is it stable? Can it hold the throughput you expect it to?
How did you do capacity planning? How to tell if the cluster is healthy and what to monitor? How to apply effective multi-tenancy? and what would be an ideal cluster topology and data ingestion architecture?
Applying ML for Log Analysis
Applying ML for Log AnalysisDoiT International We already trust artificial intelligence to drive our car, but we still configure thresholds and thrift through logs manually. In this talk, Ronny Lehmann, Loom CTO will discuss how he spent months analyzing modern-ops work, until he finally was able to extract the common-basis practices; and how we used this understanding to build a machine that complements ops teams, automating much of the work which is more suitable for machines - leaving for "humans" just the parts which require humans. What we built saves you time spent on parsers, on configuring and tuning rules and alerts, on conducting root-cause analysis and triage - and finally - on figuring out what to do.
GCP for AWS Professionals
GCP for AWS ProfessionalsDoiT International This meeting we'll host a discussion on Google Cloud Platform and Amazon Web Services to bring light to similarities and differences between platforms. If you have questions about how our platforms compare this is the meeting to attend!
Cloud Dataflow - A Unified Model for Batch and Streaming Data Processing
Cloud Dataflow - A Unified Model for Batch and Streaming Data ProcessingDoiT International Dataflow is a unified programming model and a managed service for developing and executing a wide range of data processing patterns including ETL, batch computation, and continuous computation. Cloud Dataflow frees you from operational tasks like resource management and performance optimization.
AWS Cyber Security Best Practices
AWS Cyber Security Best PracticesDoiT International We believe that security *IS* a shared responsibility, - when we give developers the power to create infrastructure, security became their responsibility, too.
During this meetup, we'd like to share our experience with implementing security best practices, to be implemented directly by development teams to build more robust and secure cloud environments. Make cloud security your team's sport!
Google Cloud Spanner Preview
Google Cloud Spanner PreviewDoiT International Cloud Spanner is the first and only relational database service that is both strongly consistent and horizontally scalable. With Cloud Spanner you enjoy all the traditional benefits of a relational database: ACID transactions, relational schemas (and schema changes without downtime), SQL queries, high performance, and high availability. But unlike any other relational database service, Cloud Spanner scales horizontally, to hundreds or thousands of servers, so it can handle the highest of transactional workloads.
Amazon Athena Hands-On Workshop
Amazon Athena Hands-On WorkshopDoiT International This 1-day course provides hands-on skills in ingesting, analyzing, transforming and visualizing data using AWS Athena and getting the best performance when using it at scale.
Audience:
This class is intended for data engineers, analysts and data scientists responsible for: analyzing and visualizing big data, implementing cloud-based big data solutions, deploying or migrating big data applications to the public cloud, implementing and maintaining large-scale data storage environments, and transforming/processing big data.
Dataflow - A Unified Model for Batch and Streaming Data Processing
Dataflow - A Unified Model for Batch and Streaming Data ProcessingDoiT International Batch and Streaming Data Processing and Vizualize 300Tb in 5 Seconds meetup on April 18th, 2016 (http://www.meetup.com/Big-things-are-happening-here/events/229532500)
Recently uploaded (20)
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
Mobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi ArabiaSteve Jonas EmizenTech is a globally recognized software development company, proudly serving businesses since 2013. With over 11+ years of industry experience and a team of 200+ skilled professionals, we have successfully delivered 1200+ projects across various sectors. As a leading Mobile App Development Company In Saudi Arabia we offer end-to-end solutions for iOS, Android, and cross-platform applications. Our apps are known for their user-friendly interfaces, scalability, high performance, and strong security features. We tailor each mobile application to meet the unique needs of different industries, ensuring a seamless user experience. EmizenTech is committed to turning your vision into a powerful digital product that drives growth, innovation, and long-term success in the competitive mobile landscape of Saudi Arabia.
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next.
Link to recording, transcript, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/
Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts!
📕 Agenda
Welcome & Introductions
Orchestrator API Overview
Exploring the Swagger Interface
Test Manager API Highlights
Streamlining Automation & Testing with APIs (Demo)
Q&A and Open Discussion
Perfect for developers, testers, and automation enthusiasts!
👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/
This session streamed live on April 29, 2025, 18:00 CET.
Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB Want to learn practical tips for designing systems that can scale efficiently without compromising speed?
Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development.
As you explore key principles of designing low-latency systems with Rust, you will learn how to:
- Create and compile a real-world app with Rust
- Connect the application to ScyllaDB (NoSQL data store)
- Negotiate tradeoffs related to data modeling and querying
- Manage and monitor the database for consistently low latencies
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungenpanagenda Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-und-verwaltung-von-multiuser-umgebungen/
HCL Nomad Web wird als die nächste Generation des HCL Notes-Clients gefeiert und bietet zahlreiche Vorteile, wie die Beseitigung des Bedarfs an Paketierung, Verteilung und Installation. Nomad Web-Client-Updates werden “automatisch” im Hintergrund installiert, was den administrativen Aufwand im Vergleich zu traditionellen HCL Notes-Clients erheblich reduziert. Allerdings stellt die Fehlerbehebung in Nomad Web im Vergleich zum Notes-Client einzigartige Herausforderungen dar.
Begleiten Sie Christoph und Marc, während sie demonstrieren, wie der Fehlerbehebungsprozess in HCL Nomad Web vereinfacht werden kann, um eine reibungslose und effiziente Benutzererfahrung zu gewährleisten.
In diesem Webinar werden wir effektive Strategien zur Diagnose und Lösung häufiger Probleme in HCL Nomad Web untersuchen, einschließlich
- Zugriff auf die Konsole
- Auffinden und Interpretieren von Protokolldateien
- Zugriff auf den Datenordner im Cache des Browsers (unter Verwendung von OPFS)
- Verständnis der Unterschiede zwischen Einzel- und Mehrbenutzerszenarien
- Nutzung der Client Clocking-Funktion
Build Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For DevsBrian McKeiver May 2nd, 2025 talk at StirTrek 2025 Conference.
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxAnoop Ashok In today's fast-paced retail environment, efficiency is key. Every minute counts, and every penny matters. One tool that can significantly boost your store's efficiency is a well-executed planogram. These visual merchandising blueprints not only enhance store layouts but also save time and money in the process.
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell With expertise in data architecture, performance tracking, and revenue forecasting, Andrew Marnell plays a vital role in aligning business strategies with data insights. Andrew Marnell’s ability to lead cross-functional teams ensures businesses achieve sustainable growth and operational excellence.
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AIartmondano By 2026, AI agents will consume 10x more enterprise data than humans, but with none of the contextual understanding that prevents catastrophic misinterpretations.
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroUiPathCommunity This session is designed to equip developers with the skills needed to build mission-critical, end-to-end processes that seamlessly orchestrate agents, people, and robots.
📕 Here's what you can expect:
- Modeling: Build end-to-end processes using BPMN.
- Implementing: Integrate agentic tasks, RPA, APIs, and advanced decisioning into processes.
- Operating: Control process instances with rewind, replay, pause, and stop functions.
- Monitoring: Use dashboards and embedded analytics for real-time insights into process instances.
This webinar is a must-attend for developers looking to enhance their agentic automation skills and orchestrate robust, mission-critical processes.
👨🏫 Speaker:
Andrei Vintila, Principal Product Manager @UiPath
This session streamed live on April 29, 2025, 16:00 CET.
Check out all our upcoming Dev Dives sessions at https://community.uipath.com/dev-dives-automation-developer-2025/.
How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?Daniel Lehner 𝙄𝙨 𝘼𝙄 𝙟𝙪𝙨𝙩 𝙝𝙮𝙥𝙚? 𝙊𝙧 𝙞𝙨 𝙞𝙩 𝙩𝙝𝙚 𝙜𝙖𝙢𝙚 𝙘𝙝𝙖𝙣𝙜𝙚𝙧 𝙮𝙤𝙪𝙧 𝙗𝙪𝙨𝙞𝙣𝙚𝙨𝙨 𝙣𝙚𝙚𝙙𝙨?
Everyone’s talking about AI but is anyone really using it to create real value?
Most companies want to leverage AI. Few know 𝗵𝗼𝘄.
✅ What exactly should you ask to find real AI opportunities?
✅ Which AI techniques actually fit your business?
✅ Is your data even ready for AI?
If you’re not sure, you’re not alone. This is a condensed version of the slides I presented at a Linkedin webinar for Tecnovy on 28.04.2025.
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrs Labs Hybrid Growth Mandate Model with TrsLabs
Strategic Investments, Inorganic Growth, Business Model Pivoting are critical activities that business don't do/change everyday. In cases like this, it may benefit your business to choose a temporary external consultant.
An unbiased plan driven by clearcut deliverables, market dynamics and without the influence of your internal office equations empower business leaders to make right choices.
Getting things done within a budget within a timeframe is key to Growing Business - No matter whether you are a start-up or a big company
Talk to us & Unlock the competitive advantage
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan This is a Quick Research Guide (QRG).
QRGs include the following:
- A brief, high-level overview of the QRG topic.
- A milestone timeline for the QRG topic.
- Links to various free online resource materials to provide a deeper dive into the QRG topic.
- Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic.
QRGs planned for the series:
- Artificial Intelligence QRG
- Quantum Computing QRG
- Big Data Analytics QRG
- Spacecraft Guidance, Navigation & Control QRG (coming 2026)
- UK Home Computing & The Birth of ARM QRG (coming 2027)
Any questions or comments?
- Please contact Arthur Morgan at [email protected].
100% human made.
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsInData Labs In this infographic, we explore how businesses can implement effective governance frameworks to address AI data privacy. Understanding it is crucial for developing effective strategies that ensure compliance, safeguard customer trust, and leverage AI responsibly. Equip yourself with insights that can drive informed decision-making and position your organization for success in the future of data privacy.
This infographic contains:
-AI and data privacy: Key findings
-Statistics on AI data privacy in the today’s world
-Tips on how to overcome data privacy challenges
-Benefits of AI data security investments.
Keep up-to-date on how AI is reshaping privacy standards and what this entails for both individuals and organizations.
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...SOFTTECHHUB I started my online journey with several hosting services before stumbling upon Ai EngineHost. At first, the idea of paying one fee and getting lifetime access seemed too good to pass up. The platform is built on reliable US-based servers, ensuring your projects run at high speeds and remain safe. Let me take you step by step through its benefits and features as I explain why this hosting solution is a perfect fit for digital entrepreneurs.
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next.
Link to recording, presentation slides, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/
Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Impelsys Inc. Impelsys provided a robust testing solution, leveraging a risk-based and requirement-mapped approach to validate ICU Connect and CritiXpert. A well-defined test suite was developed to assess data communication, clinical data collection, transformation, and visualization across integrated devices.
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat The MCP (Model Context Protocol) is a framework designed to manage context and interaction within complex systems. This SlideShare presentation will provide a detailed overview of the MCP Model, its applications, and how it plays a crucial role in improving communication and decision-making in distributed systems. We will explore the key concepts behind the protocol, including the importance of context, data management, and how this model enhances system adaptability and responsiveness. Ideal for software developers, system architects, and IT professionals, this presentation will offer valuable insights into how the MCP Model can streamline workflows, improve efficiency, and create more intuitive systems for a wide range of use cases.
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...Vishnu Singh Chundawat
Scaling Jenkins with Kubernetes by Ami Mahloof
- 1. Scaling Jenkins with Kubernetes 1.2
- 2. About me Senior DevOps Engineer @ Glide rubygems.org/profiles/amimahloof Contribution to OpenSource via RubyGems and GitHub: github.com/innovia linkedin.com/in/amimahloof
- 3. Glide environment Multiple Redis Servers MySQL DynamoDB Multiple Background Queue Processors App Server Multiple Web Socket Servers CloudSearch
- 4. Previous Jenkins setup Single Jenkins Master machine for Server code builds Single Jenkins Master Machine for Android client builds Dedicated MySQL database per build Dedicated port per build service Each build environment encapsulated and managed by Eye - process monitoring https://github.com/kostya/eye
- 5. Previous Jenkins Issues Under-utilized instances running 24/7 (expensive!) Port collisions Out of memory issues - build failures Databases needed to be created and wiped per build Debugging failed tests was extremely hard Updating Jenkins ec2 image especially for Android was a challenge
- 6. NFS-SERVER POD EBS Volume Scaling Jenkins with Kubernetes Jenkins Master BackEnd RC Jenkins Master Android RC Backend POD Jenkins Ingress Controller POD nginx configmap Jenkins Slave POD With KubeCtl Backend Service (IP) single elb for both CI’s addresses
- 7. Current Jenkins Kubernetes Plugin https://github.com/jenkinsci/kubernetes-plugin Support single docker image (jenkins-slave docker image - it create’s a POD with that image) Does not support multiple containers in a POD Does not support Persistent Volumes
- 8. Modified Jenkins Kubernetes Plugin https://github.com/innovia/kubernetes-plugin Support reading and parsing a POD template file Support multiple containers in a POD Support Persistant Volumes, EBS, EmptyDir, HostPath Volum Does not support POD template per JOB (System global config)
- 9. Read POD Template Start Build Send POD Object Jenkins Slave POD download JNLP file from Jenkins Master Connect to Jenkins Master Slave Port Build Flow Part 1 Pull Request
- 10. Build Flow Part 2 Jenkins Master Jenkins Slave Start Job on Slave git fetch and merge server code create service JOB_NAME-BUILD_NUMBER create PV - PVC JOB_NAME-BUILD_NUMBER start tests teardown POD, SERVICE, PV, PVC clean-up temp location submit result to bitbucket create POD JOB_NAME-BUILD_NUMBER wait for setup complete file (check dependancies)
- 11. Advantages On Demand PODs Complete test isolation Source controlled by developers flexible POD template Jobs queued up in Kubernetes Scheduler until node resources are available (running builds finish) Scalable Kubernetes Nodes Plugin keeps track of POD - if POD dies in the middle it launches up a new slave for that job Persistant Storage (EBS via NFS) Spot instances using node selectors
- 12. Demo
- 13. Kubernetes on Spot Instances Save 70-90% over On Demand Managed by Spot Fleet to reduce downtime Different fleets for different types of Kubernetes Nodes, allowing dynamic pod allocation using Node Selector Until implemented in Kubernetes 1.3, we have to create our own Fleets eg. https://github.com/kubernetes/kubernetes/issues/24472#issuecommen t-211975112