Stateful applications on kubernetes

Feb 18, 2018Download as ODP, PDF0 likes335 views

This document discusses stateful and stateless applications on Kubernetes, data persistence, volumes, claims, and StatefulSets. It explains that stateless applications like nginx work best with containers and use Deployments, while stateful applications require unique identities, stable storage, and ordered deployment/scaling. Data persistence in Kubernetes is achieved through volumes, volume types, persistent volumes, persistent volume claims, and storage classes. StatefulSets provide identity, storage, and ordering guarantees for stateful applications.

Stateful Applications on Kubernetes
Akash Agrawal
February’17 2018

Stateless Applications on
Kubernetes
● Containers works best with Stateless Applications
● Example:
– nginx (web front end/reverse proxy)
● Use “Deployment” workload object to
deploy/scale a Stateless Application

Data Persistence on Kubernetes
● Containers are ephermal in nature, so they
loose the data during container’s destroy/create
or restart cycles.
● Data Persistence: Any persistent data must be
stored outside of container.

Data Persistence on
Kubernetes....cont.
● Data Persistence Medium
Any persistent data (outside of the container) can
be stored at:
Pod Level
Host Level (Local)
Remote Level
NFS
Cloud Storage
Third Party Storage Solutions

Data Persistence on
Kubernetes....cont.
● Docker achieves the Data Persistence with
Volumes & Volume Plugins
● Kubernetes achieves/extend the same by:
– Volumes (Kubernetes Volumes) & Volume
Types/Plugins
– Persistent Volumes (PV)
– Persistent Volume Claims (PVC)
– Persistent Volume Types/Plugins
– Storage Classes

Data Persistence on
Kubernetes....cont.
● Lifecycle of Volume & Claims:
 Provisioning (Static or Dynamic)
 Binding
 Using
 Reclaiming (Reclaim Policies)

StatefulSets
● StatefulSets are workload objects intended to
be used with stateful applications.
● Stateful Applications may have certain
requirements like:
 Stable, unique network identifiers.
 Stable, persistent storage.
 Ordered, graceful deployment and scaling.
 Ordered, graceful deletion and termination.

StatefulSets....cont.
● StatefulSet Pods have a unique identity that is comprised of an
ordinal, a stable network identity, and stable storage. The
identity sticks to the Pod, regardless of which node it’s
(re)scheduled on.
● PersistentVolumes associated with the Pod’s PersistentVolume
Claims are not deleted when the Pods, or StatefulSet are
deleted. This must be done manually. It ensures data safety.
● StatefulSets currently require a Headless Service to be
responsible for the network identity of the Pods.

StatefulSets....cont.
● Deployment & Scaling Guarantees:
 For a StatefulSet with N replicas, when Pods are being
deployed, they are created sequentially, in order from
{0..N-1}.
 When Pods are being deleted, they are terminated in
reverse order, from {N-1..0}.
 Before a scaling operation is applied to a Pod, all of its
predecessors must be Running and Ready.
 Before a Pod is terminated, all of its successors must
be completely shutdown.

This document discusses using microservice architecture with Kubernetes. It begins with an overview of microservice architecture compared to traditional monolithic architecture and its benefits. It then covers Kubernetes and containers, the Kubernetes architecture and components, and managed Kubernetes services like Google Kubernetes Engine. It proposes combining microservices, containers, Kubernetes, and services like Istio and GKE to build microservices applications at scale. The presenter is introduced as an expert with experience implementing such architectures for clients.

Kubernetes for machine learningAkash Agrawal

This document discusses machine learning on Kubernetes and Google Cloud Platform. It provides an overview of machine learning, discusses machine learning in production environments, and describes how to use Google Cloud Platform and Kubernetes Engine (GKE) for machine learning workloads. It also introduces Kubeflow, an open source project that makes it easy to deploy and manage machine learning pipelines on Kubernetes. The document outlines best practices for designing machine learning systems on Kubernetes and provides additional resources.

Kubernetes best practices.odfAkash Agrawal

This document outlines several Kubernetes best practices including: - Using small, secure container images to improve performance and security - Running containers as non-root users for an extra layer of security - Organizing workloads into namespaces for isolation, resource utilization, and security - Implementing liveness and readiness probes to ensure instances are ready to receive traffic - Setting resource requests and limits to control node resource allocation - Ensuring applications terminate gracefully through termination grace periods and pre-stop hooks - Implementing administrative boundaries through namespaces, RBAC, quotas, and policies

MicroServices with Containers, Kubernetes & ServiceMeshAkash Agrawal

This document discusses microservices, containers, Kubernetes, Google Kubernetes Engine (GKE), and service meshes. It provides an overview of each technology and how they can be combined for a microservices architecture. Specifically, it explains how containers package applications and dependencies, Kubernetes provides orchestration of containers, GKE manages Kubernetes clusters on Google Cloud, and service meshes like Istio provide traffic management and visibility across microservices.

Cloud Native and CNCFAkash Agrawal

The document discusses Cloud Native computing and the Cloud Native Computing Foundation (CNCF). It defines Cloud Native as a new paradigm for developing, deploying, and running applications using open source software like microservices, containers, and container orchestration. The CNCF is responsible for building sustainable ecosystems of Cloud Native software and serves as the home for many fastest-growing open source projects like Kubernetes, Prometheus, and Envoy. The CNCF landscape includes graduated, incubated, and sandbox projects and drives adoption of the Cloud Native paradigm.

Google Kubernetes Engine (GKE) deep diveAkash Agrawal

This document provides an overview of Google Kubernetes Engine (GKE) including its architecture and request flow. It discusses GKE features like node pools, cluster scoping, auto repair, and integration with other Google Cloud services. The presenter then covers getting started with GKE, best practices for production usage, and the broader GKE ecosystem which includes GKE, GKE On-Prem, Anthos, and Cloud Run.

Google cloud infrastructure workshopAkash Agrawal

This document outlines an agenda for a Google Cloud Infrastructure workshop. The workshop will provide an overview of Google Cloud Platform, including its core computing and storage services. It will also cover hands-on labs and quests using Qwiklabs to practice deploying and developing applications on Google Cloud. The instructor, Akash Agrawal, will discuss his background and provide resources for learning more about Google Cloud Platform.

Google Cloud platform: GKE with CI/CD using CircleCI and Fluxkomaldevg

This document provides an overview of using Google Kubernetes Engine (GKE) with continuous integration/continuous delivery (CI/CD) workflows using CircleCI and Flux. It discusses basic Kubernetes concepts, GKE features like node pools and the components it manages. It also outlines the CI/CD workflow with CircleCI and how Flux can be used to automatically synchronize changes from a Git repository to a Kubernetes cluster. Key steps include adding Flux to the cluster, configuring it to watch a Git repository, and adding annotations to enable automated deployments from images in the repository.

Google kubernetes Engine with Google CI/CD ImplementationPiyushKamboj6

GKE allows users to deploy and manage containerized applications on Google Cloud. It provides scalability, availability, and integration with services like Calico for network policies. The document demonstrates deploying a Node.js app to GKE including building a Docker image, creating deployment and service files, and implementing CI/CD using Cloud Build to automate building and deploying on container cluster updates.

Get started with Kubernetes on GKEZachary Russell

Scalable Clusters On DemandBogdan Kyryliuk

At Opendoor, we do a lot of big data processing, and use Spark and Dask clusters for the computations. Our machine learning platform is written in Dask and we are actively moving data ingestion pipelines and geo computations to PySpark. The biggest challenge is that jobs vary in memory, cpu needs, and the load in not evenly distributed over time, which causes our workers and clusters to be over-provisioned. In addition to this, we need to enable data scientists and engineers run their code without having to upgrade the cluster for every request and deal with the dependency hell. To solve all of these problems, we introduce a lightweight integration across some popular tools like Kubernetes, Docker, Airflow and Spark. Using a combination of these tools, we are able to spin up on-demand Spark and Dask clusters for our computing jobs, bring down the cost using autoscaling and spot pricing, unify DAGs across many teams with different stacks on the single Airflow instance, and all of it at minimal cost.

Google container engine (GKE)Md. Sadhan Sarker

This document provides an overview of Google Container Engine (now known as Google Kubernetes Engine or GKE). It begins with introducing Google Cloud and its services, then defines virtualization and containerization. It explains the need for container orchestration using Kubernetes and compares Kubernetes to GKE. Finally, it lists some resource references and concludes with contact information for the author.

Introducing MagnetoDB, a key-value storage sevice for OpenStackMirantis

MagnetoDB is an open source implementation of the Amazon DynamoDB API for OpenStack. It provides a key-value database service for storing unlimited data with scalability and predictable performance. MagnetoDB's API is compatible with existing DynamoDB clients, allowing applications using DynamoDB storage to run on OpenStack. The pilot implementation provides basic CRUD operations for items and tables and is available on GitHub under an Apache 2 license.

Web App Prototypes with Google App EngineVlad Filippov

This document discusses using Google App Engine and its Python SDK for quickly prototyping web applications. Some key points covered include: - App Engine allows developers to build and host web apps and supports Python runtimes. It offers scalable hosting and APIs for common tasks like user auth and data storage. - The Python SDK makes it easy to quickly create prototypes by providing a local development server, a scalable datastore, and APIs that handle common tasks so developers can focus on unique features. - Examples of prototyping tasks demonstrated include adding entities to the datastore, building basic views and filters, and testing the prototype. Frameworks like webapp2 and appengine-rest-server can further accelerate the prototyping process

[WSO2Con USA 2018] Deploying Applications in K8S and DockerWSO2

OSMC 2018 | Visualization of your distributed infrastructure by Nicolai BuchwitzNETWAYS

In times of industrial IoT devices and cloud providers like AWS, which allow small and medium-sized companies to distribute their infrastructure around the world, it is becoming increasingly important to keep an overview. Here a few instances in Amsterdam, there a few in Tokyo and not to forget the satellites in Moscow. The Map Addon with its filters and dashboards helps to keep the overview in this constantly changing landscape and to recognize patterns and anomalies at an early stage.

Netflix Data Benchmark @ HPTS 2017Ioannis Papapanagiotou

Cloud database vendors tend to report performance numbers for the sweet spot or running in highly optimized hardware with specific workload parameters. Moreover, many of these systems are not tested under different failure scenarios that may appear in the public cloud. At Netflix, as a cloud-native enterprise, our focus is on high availability. We achieve high availability by deploying at multiple regions. Hence, our data store system performance is highly affected by our global deployment model, instance types and workload patterns. Hence, we were interested in a Cloud database benchmark tool that could be deployed in a loosely-coupled fashion, as a microservice, and with the ability to dynamically change the configurations parameters at run time. In this paper, we present Netflix Data Benchmark (NDBench). NDBench offers pluggable patterns and loads and support for different client APIs. It offers the ability to deploy, manage and monitor multiple instances from a single point. NDBench was designed to run for infinite time. This nature offered us with the ability to test long-running database maintenance jobs, test database systems under conditions that affect the performance, such as compactions, repairs etc., and also gives us a view on client side issues like memory leaks and heap pressure. We have been running NDBench for almost 3 years having validated multiple database versions, tested numerous NoSQL systems running on the Cloud, and tested new functionalities. NDBench is major component of our testing and validation pipelines.

Kubernetes in Production: Lessons LearntArunvel Sriram

Webinar: Building a multi-cloud Kubernetes storage on GitLabMayaData Inc

Michael Hall [InfluxData] | InfluxDB Community Update | InfluxDays EMEA 2021InfluxData

Java Support On Google App EngineXebia IT Architects

Google App Engine is a platform that allows developers to build and host Java web applications on Google's infrastructure. It handles tasks like scaling and maintenance automatically. Developers can focus on coding without worrying about managing hardware, servers, or other infrastructure. The platform provides APIs that allow access to Google services like Datastore, Memcache, and others. It also includes tools for deploying, monitoring, and managing applications.

OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...NETWAYS

The evolving container landscapeNilesh Trivedi

This document discusses the evolving container landscape and how applications are becoming cloud native. It outlines how companies are increasingly relying on technology and how software development has focused on continuous development, integration, and deployment. Open source adoption is growing as the default choice for developers. The document then examines the container technology stack from infrastructure to applications and services. It analyzes challenges around enterprise adoption and providing standard comparisons. Finally, it discusses how containers need to address security, storage, networking and other issues to become truly enterprise-grade.

Ryan Betts [InfluxData] | InfluxDB Platform Performance | InfluxDays Virtual ...InfluxData

The document discusses updates and new features for InfluxDB Platform in year 1, including new AWS regions for InfluxDB Cloud, pricing and signup improvements, expanded Telegraf plugin and SDK support, InfluxDB templates for sharing configurations, performance improvements for querying and visualizing data, alerting capabilities, and open sourcing InfluxDB 2.0 with a single binary. It also advertises upcoming talks on integrating OSS at the edge and Cloud at the core, and seamlessly migrating from InfluxDB 1.x to 2.0.

Cncf storage-final-filipJuraj Hantak

This document provides an overview of cloud native storage. It discusses how storage is a key component of cloud native reference architectures and how container-based applications require persistent storage volumes. It introduces the concept of out-of-tree storage plugins that allow various storage platforms to integrate with container orchestrators. The document also outlines common cloud native storage patterns, such as giving containers persistent volumes, and how this enables portability across infrastructure providers. Finally, it provides examples of how storage classes, persistent volumes, and persistent volume claims can be used to provision storage for pods running in containers.

Kubernetes session by Priyadarshini AnandPRIYADARSHINI ANAND

This document provides an overview of Kubernetes (K8s): - K8s is a platform for automating deployment, scaling, and operations of application containers across clusters of nodes. It adds value by reducing costs and downtime. - Key features include automatic deployment, self-healing, service discovery, load balancing, scaling, rollouts/rollbacks, storage/networking orchestration. - The architecture includes users, masters, minions/nodes. Components include pods, controllers, services. - Pods are the smallest work units, which are collections of containers that share resources and are scheduled together. Services provide load balancing to abstract accessing pods.

GIS on Rails by Oleksandr Kychun Pivorak MeetUp

This document discusses integrating geospatial data and functionality into a Ruby on Rails application using the OpenGeo Suite. It provides an overview of key concepts like raster vs vector data, and describes how to store geospatial data from shapefiles in PostGIS, serve it with GeoServer, and display it in maps using OpenLayers. It also outlines the typical steps to publish a geospatial layer and introduces various APIs for GeoServer configuration. Finally, it lists requirements and describes a process for importing shapefiles into PostGIS through a background job to create layers in GeoServer.

New web service oriented ARCFerenc Szalai

The document summarizes the new web service oriented ARC framework called HED. HED provides a lightweight and modular hosting environment for developing distributed services. It uses a message-chain architecture with common libraries to handle tasks like XML manipulation, logging, security and interfacing with grid resources. Example services described include job management, storage services and information indexing services. Future plans include a production release, improved configuration and documentation and additional security modules.

Autopilot : Securing Cloud Native StorageSF Bay Cloud Native Open Infra Meetup

As enterprises adopt cloud native infrastructure to run their applications, data security and compliance is becoming a crucial area of interest. When you run your containers in a public cloud, you want to make sure that the data being accessed is secure and that there are no bread crumbs left behind once the container exits. A common mistake many people make is to host-mount a volume directly inside a container, which leaves the container's data behind (directly on the host.) In this session, we focus on the best practices for ensuring the security and compliance of your applications’ persistent volumes. But ensuring security is an on-going exercise. Ideally you would deploy intelligent software that can constantly monitor and audit the application environment for security holes and breaches. Autopilot is an automated application runtime management engine built for Kubernetes, and is an open source project sponsored by Portworx: https://github.com/libopenstorage/autopilot Presented by Gunjan Patel, Gou Rao, and Aditya Dani, January 2019. More details here: https://www.meetup.com/openstack/events/258284618/

Pluggable Infrastructure with CI/CD and DockerBob Killen

More Related Content

What's hot (20)

Google kubernetes Engine with Google CI/CD ImplementationPiyushKamboj6

Get started with Kubernetes on GKEZachary Russell

Scalable Clusters On DemandBogdan Kyryliuk

Google container engine (GKE)Md. Sadhan Sarker

Introducing MagnetoDB, a key-value storage sevice for OpenStackMirantis

Web App Prototypes with Google App EngineVlad Filippov

[WSO2Con USA 2018] Deploying Applications in K8S and DockerWSO2

OSMC 2018 | Visualization of your distributed infrastructure by Nicolai BuchwitzNETWAYS

Netflix Data Benchmark @ HPTS 2017Ioannis Papapanagiotou

Kubernetes in Production: Lessons LearntArunvel Sriram

Webinar: Building a multi-cloud Kubernetes storage on GitLabMayaData Inc

Michael Hall [InfluxData] | InfluxDB Community Update | InfluxDays EMEA 2021InfluxData

Java Support On Google App EngineXebia IT Architects

OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...NETWAYS

The evolving container landscapeNilesh Trivedi

Ryan Betts [InfluxData] | InfluxDB Platform Performance | InfluxDays Virtual ...InfluxData

Cncf storage-final-filipJuraj Hantak

Kubernetes session by Priyadarshini AnandPRIYADARSHINI ANAND

GIS on Rails by Oleksandr Kychun Pivorak MeetUp

New web service oriented ARCFerenc Szalai

Google kubernetes Engine with Google CI/CD ImplementationPiyushKamboj6

Get started with Kubernetes on GKEZachary Russell

Scalable Clusters On DemandBogdan Kyryliuk

Google container engine (GKE)Md. Sadhan Sarker

Introducing MagnetoDB, a key-value storage sevice for OpenStackMirantis

Web App Prototypes with Google App EngineVlad Filippov

[WSO2Con USA 2018] Deploying Applications in K8S and DockerWSO2

OSMC 2018 | Visualization of your distributed infrastructure by Nicolai BuchwitzNETWAYS

Netflix Data Benchmark @ HPTS 2017Ioannis Papapanagiotou

Kubernetes in Production: Lessons LearntArunvel Sriram

Webinar: Building a multi-cloud Kubernetes storage on GitLabMayaData Inc

Michael Hall [InfluxData] | InfluxDB Community Update | InfluxDays EMEA 2021InfluxData

Java Support On Google App EngineXebia IT Architects

OSMC 2018 | Stream connector: Easily sending events and/or metrics from the C...NETWAYS

The evolving container landscapeNilesh Trivedi

Ryan Betts [InfluxData] | InfluxDB Platform Performance | InfluxDays Virtual ...InfluxData

Cncf storage-final-filipJuraj Hantak

Kubernetes session by Priyadarshini AnandPRIYADARSHINI ANAND

GIS on Rails by Oleksandr Kychun Pivorak MeetUp

New web service oriented ARCFerenc Szalai

Similar to Stateful applications on kubernetes (20)

Autopilot : Securing Cloud Native StorageSF Bay Cloud Native Open Infra Meetup

Pluggable Infrastructure with CI/CD and DockerBob Killen

Introduction to Container Storage Interface (CSI)Idan Atias

Among the cool stuff we do at Silk, my colleagues and I develop the Silk CSI Plugin for customers who use our system as the storage layer for their Kubernetes workloads. Before deep diving into the code and as part of my ramp-up on this subject I prepared some slides that cover some basic and important information on this topic. These slides start by recapping some basic storage principals in containers and Kubernetes, continues with some more advanced use cases (including an "offline demo" of persisting Redis data on EBS volumes), and ends with a detailed information on the CSI solution itself. IMHO, reviewing these slides can improve your understanding on this matter and can get you started implementing your own CSI plugin. The main sources of information I used for preparing these slides are: * Official CSI docs * Kubernetes Storage Lingo 101 - Saad Ali, Google * Container Storage Interface: Present and Future - Jie Yu, Mesosphere, Inc.

[WSO2Con EU 2018] Architecting for a Container Native EnvironmentWSO2

Containers are becoming the standard way to package and deploy applications. Going beyond just containers, enterprise apps increasingly depend on a large set of ecosystem projects providing rich features such as high availability, sidecar architecture patterns for security/monitoring, and distributed tracing. Running an existing application on a container doesn't automatically make it container native. This discussion explores architectural choices for making applications and integration services first class citizens in a container native environment.

Rise of the machines: Continuous Delivery at SEEK - YOW! Night Summary SlidesDiUS

The virtues of continuous delivery are widely understood and accepted by organisations which value fast feedback cycles, reduced risk through incremental delivery of smaller changes and the ability to respond quickly to external factors. Furthermore if microservices are part of your architecture, then the ability to rapidly deploy multiple components of a system become increasingly important. The foundations of scripting, automation and more recently containers made *nix-based systems the first target for automated deployments and subsequently continuous delivery. With the advent of some new tooling and a bit of courage these principles can now be applied to more heterogeneous environments including those from Redmond. Using their backgrounds in automating large-scale ruby and java-based deployments, Warner and Matt embarked on a journey with SEEK to increase their agility by enabling continuous delivery – typically multiple times per day. This is their story.

QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemonsaspyker

Disenchantment is a Netflix show following the medieval misadventures of a hard-drinking princess, her feisty elf, and her personal demon. In this talk, we will follow the story of Netflix’s container management platform, Titus, which powers critical aspects of the Netflix business (video encoding & streaming, big data, recommendations & machine learning, and other workloads). We’ll cover the challenges growing Titus from 10’s to 1000’s of workloads. We’ll talk about our feisty team’s work across container runtimes, scheduling & control plane, and cloud infrastructure integration. We’ll talk about the demons we’ve found on this journey covering operability, security, reliability and performance.

Things You MUST Know Before Deploying OpenStack: Bruno Lago, Catalyst ITOpenStack

Audience: Advanced About: Real world lessons and war stories about Catalyst IT’s experience in rolling out an OpenStack based public cloud in New Zealand. This presentation will provide tips and advice that may save you a lot of time, money and nights of sleep if you are planning to run OpenStack in the future. It may also bring some insights to people that are already running OpenStack in production. Topics covered will include: selection of hardware for optimal costs, techniques that drive quality and service levels up, common deployment mistakes, in place upgrades, how to identify the maturity level of each project and decide what is ready for production, and much more! Speaker Bio: Bruno Lago – Entrepreneur, Catalyst IT Limited Bruno Lago is a solutions architect that has been involved with the Catalyst Cloud (New Zealand’s first public cloud based on OpenStack) from its inception. He is passionate about open source software, cloud computing and disruptive technologies. OpenStack Australia Day - Sydney 2016 https://events.aptira.com/openstack-australia-day-sydney-2016/

Neutron-to-Neutron: interconnecting multiple OpenStack deploymentsThomas Morin

Provide network interconnections between Openstack clouds/regions ? Neutron offers floating IPs and IPSec VPNaaS. However this is not always good enough: sometimes network isolation is needed, but without the the overhead of IPSec encryption. How to avoid putting the burden on an orchestrator ? Solutions exist to create interconnections in ways specific to each overlay technology or SDN backends, but they require central coordination via an orchestrator (not always easy), and sometimes also the provisioing of network devices (not always simple). "Neutron talking to Neutron" This presentation exposes a solution developed in the Neutron project, where tenants define their network interconnection needs across regions or clouds, and Neutron components in the different regions coordinate together to setup these private isolated interconnections. Without orchestration nor network device configuration.

The road to enterprise ready open stack storage as serviceSean Cohen

The document summarizes updates and improvements to OpenStack storage services. Key points include: - Efforts to improve high availability of Cinder APIs and services, including multipath support and active/active deployments. - Enhancements for volume management such as attaching single volumes to multiple hosts, improved volume migration, and retype functionality. - Updates to backup services including incremental backups, support for additional targets like NFS/POSIX, and improved integration with Swift. - Disaster recovery features like consistency group enhancements and planned import/export of snapshots between Cinder installations. - Work on deployment and rolling upgrades, including database cleanup utilities, object-based communication, and standardized driver

OpenStack Neutron-Neutron interconnectionsThomas Morin

[ lightning talk done during the OpenStack Summit, Sydney Nov. 2018 ] Provide network interconnections between Openstack clouds ? between regions ? DC pods ? Neutron today offers floating IPs and IPSec VPNaaS. However these are not always good enough: sometimes private addressing and network isolation is needed, but avoiding the overhead of IPSec encryption would be preferable. How to avoid the overhead of adding an orchestrator ? Solutions also exists to create interconnections in ways specific to each overlay technology or SDN backends, but they will require central coordination via an orchestrator (not always possible), and sometimes also the provisioing of network devices (not always simple). "Neutron talking to Neutron" This talk exposes and showcases a solution where Openstack projects define their network interconnection needs across regions or clouds, and Neutron endpoints in the different regions coordinate together in a simple way to setup these private isolated interconnections. Without orchestration nor network device configuration.

Introduction openstack-meetup-nov-28Sadique Puthen

Red Hat Openstack and Ceph Meetup, Pune | 28th NOV 2015 Sadique Puthen, Principal Technical Support Engineer at Red Hat, Inc., gave an introduction to Red Hat Openstack (RDO) and its components. He discussed how Openstack provides infrastructure services like compute (Nova), storage (Cinder, Swift), networking (Neutron), and database (Trove) as a service. He also covered Openstack deployment options like Packstack, TripleO, and Ironic for bare metal provisioning. The meetup aimed to introduce Openstack components and services and their role in providing infrastructure as a service through a cloud platform.

Kubernetes IntroAntonio Ojea Garcia

This document provides an overview of Kubernetes. It begins by introducing Kubernetes and its origins at Google. It then discusses key Kubernetes concepts like pods, deployments, services, ingress, namespaces, labels, and storage. It explains how Kubernetes provides automation of application deployment, scaling, and management using containers. It also discusses how Kubernetes helps enable continuous integration, continuous deployment, and digital transformation. The document concludes by mentioning the Kubernetes ecosystem and Cloud Native Computing Foundation.

A New Way of Thinking | NATS 2.0 & ConnectivityNATS

NATS 2.0 is the largest feature release since the original code base for the server was released. NATS 2.0 was created to allow a new way of thinking about NATS as a shared utility, solving problems at scale through distributed security, multi-tenancy, larger networks, and secure sharing of data. In this presentation, Derek discusses the motives behind the newest features of NATS and how to leverage them to reduce total cost of ownership, decrease time to value, support extremely large scale deployments, and decentralize security to create secure and easy to manage modern distributed systems.

Kubernetes basics, Nodes, Pods, Containers, DeploymentsBeroza Paul

Kubernetes, also known as K8s for short, is an open-source system for automating the deployment, scaling, and management of containerized applications. Open-source: Anyone can use and contribute to it, making it a popular choice for many organizations. Container orchestration: It groups containers (which are essentially miniaturized applications) into logical units for easy management and discovery. Automated: It takes care of tasks like deploying new versions of your application, scaling it up or down based on demand, and restarting crashed containers.

The Highs and Lows of Stateful ContainersC4Media

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2BubBkw. Alex Robinson walks through his experiences trying to reliably run a distributed database on Kubernetes, optimize its performance, and help others do the same in their heterogeneous environments. He looks at what kinds of stateful applications can most easily be run in containers, and a number of pitfalls he encountered along the way. Filmed at qconsf.com. Alex Robinson is a member of the technical staff at Cockroach Labs, where he works on CockroachDB's core transactional storage layer and leads all integrations with orchestration systems. Previously, he was a senior software engineer at Google, where he spent his last two years as a core early developer of Kubernetes and GKE.

TechDay - Toronto 2016 - Hyperconvergence and OpenNebulaOpenNebula Project

Hyperconvergence integrates compute, storage, networking and virtualization resources from scratch in a commodity hardware box supported by a single vendor. It offers scalability, performance, centralized management, reliability and is software-focused. StorPool is a storage software that can be installed on servers to pool and aggregate the capacity and performance of drives. It provides standard block devices and replicates data across drives and servers for redundancy. StorPool integrates fully with Opennebula to provide a robust hyperconverged infrastructure on commodity hardware using distributed storage.

How to deliver High Performance OpenStack Cloud: Christoph Dwertmann, Vault S...OpenStack

Securing Openstack in Line with the Government ISM and PSPF controls and how to deliver High Performance OpenStack Cloud to address Government Legacy Systems Audience: Intermediate/Advanced Topic: Security, Infrastructure, Performance Abstract: As the CTO of Vault Systems, Christoph will take us through the challenges of implementing ASD’s ISM controls within Vault’s OpenStack cloud to create a Protected Certified OpenStack Platform and give a technical account of some of the optimizations he has done around Ceph on NVMe Storage to deliver High Performance Storage. Speaker Bio: Christoph Dwertmann, Vault Systems Christoph is a full stack engineer with four years of experience in deploying and securing Openstack. Fully automated software deployment and self-healing microservice containers are amongst his current interests. As the CTO of Vault Systems he recently deployed the world’s first pure NVMe Ceph cluster into production. From his previous work in network research for the National Science Foundation (NSF) he gathered in-depth knowledge spanning software-defined networks across continents. OpenStack Australia Day Government - Canberra 2016 https://events.aptira.com/openstack-australia-day-canberra-2016/

WP VERITAS InfoScale Storage and Dockers Intro - v8Rajagopal Vaideeswaran

This document discusses using Veritas InfoScale Storage with Docker containers to provide persistent storage. It provides instructions on setting up an InfoScale Storage cluster with three servers, configuring storage pools and volumes, deploying a MySQL container with a mapped persistent volume, populating the database, and running the container on different hosts while maintaining access to the persistent data volume. This allows stateful containers to have their data available across any server in the cluster for high availability and mobility.

Vector Search at Scale - Pro Tips - Stephen BatifolZilliz

Openstack devops challenges openstackindia

Openstack DevOps Challenges outlines the journey of CloudRX, a fictitious company, to setup a production-grade Openstack cloud using DevOps practices. It discusses challenges faced in implementing continuous integration/delivery pipelines for Openstack and its heterogeneous components, managing configurations, automated testing of environments, packaging applications, and baremetal server management.

Autopilot : Securing Cloud Native StorageSF Bay Cloud Native Open Infra Meetup

Pluggable Infrastructure with CI/CD and DockerBob Killen

Introduction to Container Storage Interface (CSI)Idan Atias

[WSO2Con EU 2018] Architecting for a Container Native EnvironmentWSO2

Rise of the machines: Continuous Delivery at SEEK - YOW! Night Summary SlidesDiUS

QConSF18 - Disenchantment: Netflix Titus, its Feisty Team, and Daemonsaspyker

Things You MUST Know Before Deploying OpenStack: Bruno Lago, Catalyst ITOpenStack

Neutron-to-Neutron: interconnecting multiple OpenStack deploymentsThomas Morin

The road to enterprise ready open stack storage as serviceSean Cohen

OpenStack Neutron-Neutron interconnectionsThomas Morin

Introduction openstack-meetup-nov-28Sadique Puthen

Kubernetes IntroAntonio Ojea Garcia

A New Way of Thinking | NATS 2.0 & ConnectivityNATS

Kubernetes basics, Nodes, Pods, Containers, DeploymentsBeroza Paul

The Highs and Lows of Stateful ContainersC4Media

TechDay - Toronto 2016 - Hyperconvergence and OpenNebulaOpenNebula Project

How to deliver High Performance OpenStack Cloud: Christoph Dwertmann, Vault S...OpenStack

WP VERITAS InfoScale Storage and Dockers Intro - v8Rajagopal Vaideeswaran

Vector Search at Scale - Pro Tips - Stephen BatifolZilliz

Openstack devops challenges openstackindia

Recently uploaded (20)

2025-05-Q4-2024-Investor-Presentation.pptxSamuele Fogagnolo

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Talk at the final event of Data Fusion Dynamics: A Collaborative UK-Saudi Initiative in Cybersecurity and Artificial Intelligence funded by the British Council UK-Saudi Challenge Fund 2024, Cardiff Metropolitan University, 29th April 2025 https://alandix.com/academic/talks/CMet2025-AI-Changes-Everything/ Is AI just another technology, or does it fundamentally change the way we live and think? Every technology has a direct impact with micro-ethical consequences, some good, some bad. However more profound are the ways in which some technologies reshape the very fabric of society with macro-ethical impacts. The invention of the stirrup revolutionised mounted combat, but as a side effect gave rise to the feudal system, which still shapes politics today. The internal combustion engine offers personal freedom and creates pollution, but has also transformed the nature of urban planning and international trade. When we look at AI the micro-ethical issues, such as bias, are most obvious, but the macro-ethical challenges may be greater. At a micro-ethical level AI has the potential to deepen social, ethnic and gender bias, issues I have warned about since the early 1990s! It is also being used increasingly on the battlefield. However, it also offers amazing opportunities in health and educations, as the recent Nobel prizes for the developers of AlphaFold illustrate. More radically, the need to encode ethics acts as a mirror to surface essential ethical problems and conflicts. At the macro-ethical level, by the early 2000s digital technology had already begun to undermine sovereignty (e.g. gambling), market economics (through network effects and emergent monopolies), and the very meaning of money. Modern AI is the child of big data, big computation and ultimately big business, intensifying the inherent tendency of digital technology to concentrate power. AI is already unravelling the fundamentals of the social, political and economic world around us, but this is a world that needs radical reimagining to overcome the global environmental and human challenges that confront us. Our challenge is whether to let the threads fall as they may, or to use them to weave a better future.

Heap, Types of Heap, Insertion and DeletionJaydeep Kale

#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada

Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next. Link to recording, transcript, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/ Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.

Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma

DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxJustin Reock

Building 10x Organizations with Modern Productivity Metrics 10x developers may be a myth, but 10x organizations are very real, as proven by the influential study performed in the 1980s, ‘The Coding War Games.’ Right now, here in early 2025, we seem to be experiencing YAPP (Yet Another Productivity Philosophy), and that philosophy is converging on developer experience. It seems that with every new method we invent for the delivery of products, whether physical or virtual, we reinvent productivity philosophies to go alongside them. But which of these approaches actually work? DORA? SPACE? DevEx? What should we invest in and create urgency behind today, so that we don’t find ourselves having the same discussion again in a decade?

AI and Data Privacy in 2025: Global TrendsInData Labs

In this infographic, we explore how businesses can implement effective governance frameworks to address AI data privacy. Understanding it is crucial for developing effective strategies that ensure compliance, safeguard customer trust, and leverage AI responsibly. Equip yourself with insights that can drive informed decision-making and position your organization for success in the future of data privacy. This infographic contains: -AI and data privacy: Key findings -Statistics on AI data privacy in the today’s world -Tips on how to overcome data privacy challenges -Benefits of AI data security investments. Keep up-to-date on how AI is reshaping privacy standards and what this entails for both individuals and organizations.

ThousandEyes Partner Innovation Updates for May 2025ThousandEyes

Mobile App Development Company in Saudi ArabiaSteve Jonas

EmizenTech is a globally recognized software development company, proudly serving businesses since 2013. With over 11+ years of industry experience and a team of 200+ skilled professionals, we have successfully delivered 1200+ projects across various sectors. As a leading Mobile App Development Company In Saudi Arabia we offer end-to-end solutions for iOS, Android, and cross-platform applications. Our apps are known for their user-friendly interfaces, scalability, high performance, and strong security features. We tailor each mobile application to meet the unique needs of different industries, ensuring a seamless user experience. EmizenTech is committed to turning your vision into a powerful digital product that drives growth, innovation, and long-term success in the competitive mobile landscape of Saudi Arabia.

Drupalcamp Finland – Measuring Front-end Energy ConsumptionExove

Role of Data Annotation Services in AI-Powered ManufacturingAndrew Leo

Procurement Insights Cost To Value Guide.pptxJon Hansen

Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada

Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next. Link to recording, presentation slides, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/ Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.

Technology Trends in 2025: AI and Big Data AnalyticsInData Labs

At InData Labs, we have been keeping an ear to the ground, looking out for AI-enabled digital transformation trends coming our way in 2025. Our report will provide a look into the technology landscape of the future, including: -Artificial Intelligence Market Overview -Strategies for AI Adoption in 2025 -Anticipated drivers of AI adoption and transformative technologies -Benefits of AI and Big data for your business -Tips on how to prepare your business for innovation -AI and data privacy: Strategies for securing data privacy in AI models, etc. Download your free copy nowand implement the key findings to improve your business.

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Splunk Security Update | Public Sector Summit Germany 2025Splunk

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts! 📕 Agenda Welcome & Introductions Orchestrator API Overview Exploring the Swagger Interface Test Manager API Highlights Streamlining Automation & Testing with APIs (Demo) Q&A and Open Discussion Perfect for developers, testers, and automation enthusiasts! 👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/ This session streamed live on April 29, 2025, 18:00 CET. Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.

Complete Guide to Advanced Logistics Management Software in Riyadh.pdfSoftware Company

Explore the benefits and features of advanced logistics management software for businesses in Riyadh. This guide delves into the latest technologies, from real-time tracking and route optimization to warehouse management and inventory control, helping businesses streamline their logistics operations and reduce costs. Learn how implementing the right software solution can enhance efficiency, improve customer satisfaction, and provide a competitive edge in the growing logistics sector of Riyadh.

Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan

This is a Quick Research Guide (QRG). QRGs include the following: - A brief, high-level overview of the QRG topic. - A milestone timeline for the QRG topic. - Links to various free online resource materials to provide a deeper dive into the QRG topic. - Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic. QRGs planned for the series: - Artificial Intelligence QRG - Quantum Computing QRG - Big Data Analytics QRG - Spacecraft Guidance, Navigation & Control QRG (coming 2026) - UK Home Computing & The Birth of ARM QRG (coming 2027) Any questions or comments? - Please contact Arthur Morgan at [email protected]. 100% human made.

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

2025-05-Q4-2024-Investor-Presentation.pptxSamuele Fogagnolo

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Heap, Types of Heap, Insertion and DeletionJaydeep Kale

#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025BookNet Canada

Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma

DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxJustin Reock

AI and Data Privacy in 2025: Global TrendsInData Labs

ThousandEyes Partner Innovation Updates for May 2025ThousandEyes

Mobile App Development Company in Saudi ArabiaSteve Jonas

Drupalcamp Finland – Measuring Front-end Energy ConsumptionExove

Role of Data Annotation Services in AI-Powered ManufacturingAndrew Leo

Procurement Insights Cost To Value Guide.pptxJon Hansen

Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada

Technology Trends in 2025: AI and Big Data AnalyticsInData Labs

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Splunk Security Update | Public Sector Summit Germany 2025Splunk

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Complete Guide to Advanced Logistics Management Software in Riyadh.pdfSoftware Company

Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

Stateful applications on kubernetes

1. Stateful Applications on Kubernetes Akash Agrawal February’17 2018

2. Stateless Applications on Kubernetes ● Containers works best with Stateless Applications ● Example: – nginx (web front end/reverse proxy) ● Use “Deployment” workload object to deploy/scale a Stateless Application

3. Data Persistence on Kubernetes ● Containers are ephermal in nature, so they loose the data during container’s destroy/create or restart cycles. ● Data Persistence: Any persistent data must be stored outside of container.

4. Data Persistence on Kubernetes....cont. ● Data Persistence Medium Any persistent data (outside of the container) can be stored at: Pod Level Host Level (Local) Remote Level NFS Cloud Storage Third Party Storage Solutions

5. Data Persistence on Kubernetes....cont. ● Docker achieves the Data Persistence with Volumes & Volume Plugins ● Kubernetes achieves/extend the same by: – Volumes (Kubernetes Volumes) & Volume Types/Plugins – Persistent Volumes (PV) – Persistent Volume Claims (PVC) – Persistent Volume Types/Plugins – Storage Classes

6. Data Persistence on Kubernetes....cont. ● Lifecycle of Volume & Claims:  Provisioning (Static or Dynamic)  Binding  Using  Reclaiming (Reclaim Policies)

7. StatefulSets ● StatefulSets are workload objects intended to be used with stateful applications. ● Stateful Applications may have certain requirements like:  Stable, unique network identifiers.  Stable, persistent storage.  Ordered, graceful deployment and scaling.  Ordered, graceful deletion and termination.

8. StatefulSets....cont. ● StatefulSet Pods have a unique identity that is comprised of an ordinal, a stable network identity, and stable storage. The identity sticks to the Pod, regardless of which node it’s (re)scheduled on. ● PersistentVolumes associated with the Pod’s PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted. This must be done manually. It ensures data safety. ● StatefulSets currently require a Headless Service to be responsible for the network identity of the Pods.

9. StatefulSets....cont. ● Deployment & Scaling Guarantees:  For a StatefulSet with N replicas, when Pods are being deployed, they are created sequentially, in order from {0..N-1}.  When Pods are being deleted, they are terminated in reverse order, from {N-1..0}.  Before a scaling operation is applied to a Pod, all of its predecessors must be Running and Ready.  Before a Pod is terminated, all of its successors must be completely shutdown.

10. Thanks

Stateful applications on kubernetes

Recommended

More Related Content

What's hot (20)

Similar to Stateful applications on kubernetes (20)

Recently uploaded (20)

Stateful applications on kubernetes