Autoscaling in Kubernetes (K8s)

Jun 20, 20220 likes61 views

Autoscaling in Kubernetes allows applications to automatically scale resources up or down based on metrics like CPU usage. It addresses challenges with traditional autoscaling approaches by being platform independent and scaling pods quickly using the Horizontal Pod Autoscaler. The document outlines an architecture that sets autoscaling to increase application pods when CPU usage crosses 50%, with a minimum of 1 and maximum of 3 pods. It then demonstrates this through scenarios of idle and heavy loads, ramping up users over 10 seconds to test the autoscaling capabilities.

Autoscaling – Challenges & Advantages
• Why?
• Traditional autoscaling in cloud
• Challenges with this approach
• Autoscaling in K8s
• Advantages of this approach -
• Auto-scales and provides results quicker
• Platform independent, HPA is a core component of any K8s distribution
and will work out of the box without any changes to code

For testing purposes, we set autoscaling to increase application pod replicas
when they cross over 50% of their allotted CPU usage. We have a minimum replica
count of 1, and a maximum of 3.
Scenario 1: Idle load
• Number of users: 100
• Requests per user: 10
• Ramp up time: 10s
A total of 1000 requests,
by 100 users, within 10 seconds
Test Scenarios
Scenario 2: Heavy load
• Number of users: 1000
• Requests per user: 10
• Ramp up time: 10s
A total of 10000 requests, by 100 users,
within 10 seconds

The document provides an overview of a presentation on cloud performance testing. The presentation agenda includes cloud 101 concepts, cloud offerings and deployment models, challenges of cloud computing, and tools for cloud performance testing. It also summarizes a proof of concept that was conducted to compare the performance and costs of using a commercial tool versus an open source tool for load testing on cloud infrastructure. The results showed comparable response times between the tools and significantly lower costs when using the cloud versus maintaining physical infrastructure.

Jolt: Distributed, fault-tolerant test running at scale using MesosMesosphere Inc.

This document summarizes Yelp's new test parallelization system called Jolt. It discusses how Jolt bundles tests into groups to run in parallel on Mesos clusters. This improves upon their previous system called Seagull which had complex custom components. Jolt uses Task Processing to interface with Mesos in a composable way. The document also describes Yelp's autoscaling system called Clusterman which adjusts Mesos cluster sizes based on resource usage and spot instance pricing. It notes how these systems have significantly reduced Yelp's infrastructure costs while improving their ability to run tests and deploy code.

BATbern53 ETHZ Rethinking Cluster State Management for Lightweight Function a...BATbern

Serverless computing raises the level abstraction to the cloud, making the cloud easier to use and enabling the cloud platform to optimize for performance and energy efficiency under the hood. Although the serverless paradigm holds great promise, the system software that serverless platforms are built on today is still rooted in the very different, more traditional execution model of long-running containers or virtual machines. Today’s Functions as a Service (FaaS) platforms orchestrate function sanboxes with systems based on conventional cluster managers like Kubnernetes. We find that Kubernetes-based cluster managers add significant overhead in FaaS environments because the way that they manage, persist, and update cluster state is not designed for high churn or short-lived sandboxes. Rather than retrofitting existing cluster managers, we propose Dirigent, a clean-slate system tailored for FaaS that supports orders of magnitude higher scheduling throughput than Kubernetes-based systems. Dirigent simplifies cluster manger abstractions to optimize state management, eliminates persistent updates on the critical path of function invocations by recognizing that serverless allows for relaxing state reconstruction guarantees, and runs monolithic control and data planes to minimize internal communication between cluster manager components to maximize throughput.

Kubernetes: Reducing Infrastructure Cost & ComplexityDevOps.com

goto; London: Keeping your Cloud Footprint in CheckCoburn Watson

Melbourne Big Data Meetup Talk: Scaling a Real-Time Anomaly Detection Applica...Paul Brebner

Apache Kafka, Apache Cassandra and Kubernetes are open source big data technologies enabling applications and business operations to scale massively and rapidly. While Kafka and Cassandra underpins the data layer of the stack providing capability to stream, disseminate, store and retrieve data at very low latency, Kubernetes is a container orchestration technology that helps in automated application deployment and scaling of application clusters. In this presentation, Paul will reveal how he architected a massive scale deployment of a streaming data pipeline with Kafka and Cassandra to cater to an example Anomaly detection application running on a Kubernetes cluster and generating and processing massive amount of events. Anomaly detection is a method used to detect unusual events in an event stream. It is widely used in a range of applications such as financial fraud detection, security, threat detection, website user analytics, sensors, IoT, system health monitoring, etc. When such applications operate at massive scale generating millions or billions of events, they impose significant computational, performance and scalability challenges to anomaly detection algorithms and data layer technologies. Paul will demonstrate the scalability, performance and cost effectiveness of Apache Kafka, Cassandra and Kubernetes, with results from his experiments allowing the Anomaly detection application to scale to 19 Billion anomaly checks per day. Melbourne Big Data Meetup, March 5 2020 https://www.eventbrite.com/e/melbourne-big-data-meetup-realtime-anomaly-detection-with-cassandra-kafka-tickets-93028445585

Scaling Open Source Big Data Cloud Applications is Easy/HardPaul Brebner

In the last decade, the development of modern horizontally scalable open-source Big Data technologies such as Apache Cassandra (for data storage), and Apache Kafka (for data streaming) enabled cost-effective, highly scalable, reliable, low-latency applications, and made these technologies increasingly ubiquitous. To enable reliable horizontal scalability, both Cassandra and Kafka utilize partitioning (for concurrency) and replication (for reliability and availability) across clustered servers. But building scalable applications isn’t as easy as just throwing more servers at the clusters, and unexpected speed humps are common. Consequently, you also need to understand the performance impact of partitions, replication, and clusters; monitor the correct metrics to have an end-to-end view of applications and clusters; conduct careful benchmarking, and scale and tune iteratively to take into account performance insights and optimizations. In this presentation, I will explore some of the performance goals, challenges, solutions, and results I discovered over the last 5 years building multiple realistic demonstration applications. The examples will include trade-offs with elastic Cassandra auto-scaling, scaling a Cassandra and Kafka anomaly detection application to 19 Billion checks per day, and building low-latency streaming data pipelines using Kafka Connect for multiple heterogeneous source and sink systems. Invited keynote for 5th Workshop on Hot Topics in Cloud Computing Performance (HotCloudPerf 2022) https://hotcloudperf.spec.org/ at ICPE 2022 https://icpe2022.spec.org/

IEEE 2014 JAVA CLOUD COMPUTING PROJECTS Automatic scaling of internet applica...IEEEGLOBALSOFTSTUDENTPROJECTS

Cray HPC Environments for Leading Edge Simulationsinside-BigData.com

This presentation is from the PBS User Group 2013, as presented by Greg Clifford. "Cray scalable solutions are optimized for complex simulations, where parallel and massively concurrent applications need access to data, fast. This presentation will cover recent updates from Cray, showing how Cray continues to deliver the largest production supercomputers leveraging Altair PBS Works and HyperWorks software solutions." Learn more: http://www.altairatc.com/page.aspx?region=na&name=agenda Watch the presentation video: http://insidehpc.com/2013/10/04/cray-hpc-environments-leading-edge-simulation/

CeiloscaFabio Giannetti

Anton Boyko, "The evolution of microservices platform or marketing gibberish"Sigma Software

This document discusses Azure Functions and serverless computing. It provides an overview of serverless benefits like manage less, micro-pricing, and ease of scale. It also compares common serverless offerings from Microsoft Azure, AWS, and Google Cloud. The document demonstrates consumption scale, durable functions, and addressing issues like capacity planning, durability, and performance for serverless applications.

Anton Boyko "The future of serverless computing"Fwdays

Are you still planning your servers capacity in CPU and RAM? Do you enjoy answering questions like “what amount of servers we need to buy to serve 10K and 10M users”? Do you enjoy paying for those servers while they are sitting idle? In my session I will talk about the current state and the future of serverless computing. I will show several demos that show how to start using serverless computing, what are the application scenarios for it, how it can scale, what it will cost, etc. This session will be useful for developers and architects who are seeking a way to create an app utilizing all your previous skills, but make it scalable, reliable, fast and cost efficient.

Robotics technical Presentationklepsydratechnologie

HIGH PERFORMANCE EDGE DATA PROCESSING SOFTWARE FOR ROBOTIC APPLICATIONS Robotic edge autonomous systems is a fast growing market with as fast growing challenges: Sophisticated applications, Increasing demand in complexity: Communications with other systems, Heavy algorithms: AI, vision navigation, etc. Hardware has improved substantially: - Better embedded computers - Improved sensors (more and faster data) https://klepsydra.com/demos-media/#earth-observation-throughput

Surge 2013: Maximizing Scalability, Resiliency, and Engineering Velocity in t...Coburn Watson

Kubernetes at NU.nl (Kubernetes meetup 2019-09-05)Tibo Beijen

The Need of Cloud-Native ApplicationEmiliano Pecis

Taming the cost of your first cloud - CCCEU 2014Tim Mackey

Today everyone is talking about clouds, and a few are building them, but far fewer are operating successful clouds. In this session we'll examine a variety of paradigm shifts IT makes when moving from a traditional virtualization and management mindset to operating a successful cloud. For most organizations, without careful planning the hype of a cloud solution can quickly overcome its capabilities and pre-existing best practices can combine to create the worst possible cloud scenario -- a cloud which isn't economical to operate, and which is more cumbersome to manage than a traditional virtualization farm. Key topics covered include: - Successful transition of operational and management paradigm - How the VM density of clouds change Ops - What it means to monitor the network in a cloud environment, at hyper-dense virtualization levels - Preventing storage costs from outpacing delivery costs

Building Reactive applications with AkkaKnoldus Inc.

The document discusses building reactive applications using Akka. Akka allows for building applications that are message-driven, elastic, resilient and able to handle failures. It enables building applications as systems of actors that communicate asynchronously by message passing. This model allows the applications to scale horizontally across clusters of machines and to recover from failures through supervision. The document provides examples of how to model parts of an application, such as a flight search service, as actor systems.

Serverless on AWS : Understanding the hard parts at Froscon 2019Vadym Kazulkin

In unserem Vortrag tauchen wir tiefer in die Serverless-Welt ein und zeigen wie eine produktionsreife Serverless-Anwendung mithilfe von AWS-Cloud mit dem Technologie-Stack API Gateway, SNS, Lambda and DynamoDB aufgebaut werden kann. Dabei gehen wir auf die Herausforderungen der jeweiligen Services ein, wie "cold start" bei Lamda oder "provisioned throughput" und "adaptice capacity" bei DynamoDB. Dabei zeigen wir, welche Strategien und Wege es gibt, damit umzugehen. Außerdem behandeln wir solche Themen wie Implementierung von Aggregationslogik und (Scheduled) Auto Scaling bei DynamoDB. Am Ende werfen wir einen Blick in die Zukunft und sprechen über die erste relationale serverless Datenbank "Aurora Serverless"

Univa Presentation at DAC 2020 Univa, an Altair Company

kubernetesNotes_ggggggbggggggggggggggg.pdfTinotendaChagaka

Intel Atom Processor Pre-Silicon Verification ExperienceDVClub

This document discusses the verification methodology and results for the Intel Atom processor. It describes the challenges of verifying a new microarchitecture with power management features on an aggressive schedule. The methodology involved cluster-level validation with functional coverage, architectural validation using an instruction set generator, and power management validation. Verification metrics like coverage and bug rates were tracked. The results included booting Windows and Linux 10 hours after receiving silicon, with few functional bugs found post-silicon that weren't corner cases. Debug and survivability features helped reduce escapes.

Making Apache Kafka Even Faster And More ScalablePaulBrebner2

Intel open stack-summit-session-nov13-finalDeepak Mane

- Intel is a major contributor to OpenStack and open source projects, contributing across every layer of the OpenStack stack. As the #2 Linux kernel contributor, Intel helps improve performance, stability, and efficiency. - Intel enables OpenStack cloud deployments through contributions to OpenStack projects, open source tools, and optimizations. Intel IT also uses OpenStack in their own private cloud. - Intel is working on technologies to address challenges in datacenters, including security and compliance, cost reduction, and business uptime. Technologies include trusted compute pools, erasure coding, and enhanced platform awareness.

Consolidate your enterprise workloads without creating new performance constr...NetApp

In this presentation by SolidFire's vExpert Jeramiah Dooley, you will learn about how consolidating workloads onto a single platform can be the crucial missing step as VMware customers transition from a legacy architecture into a Software Defined Data Center. Storage has become the boat anchor of the modern, virtualized data center. Up until recently it has been slow, unpredictable, inflexible, required far too much operational overhead and was sized, deployed and managed using methods created in the 1980s. Worst of all, storage capacity and performance are often still locked together in fixed ratios, making it inevitable that even the best run storage environment will be wasteful and inefficient as you consolidate your applications. But it doesn’t have to be this way. The technology and know-how exists to have storage be a predictable, flexible, and easily managed part of an overall virtualized platform designed to provide the performance and availability that today’s end user workloads demand.

Diameter progress, popular use cases, roadmap - Mobicents Summit 2011telestax

[AWS Dev Day] 실습워크샵 | Amazon EKS 핸즈온 워크샵Amazon Web Services Korea

This document provides an overview of an Amazon EKS hands-on workshop. It introduces the workshop agenda which includes deploying example microservices, logging with Elasticsearch Fluentd and Kibana, monitoring with Prometheus and Grafana, and continuous integration/continuous delivery using GitOps with Weave Flux. Key concepts covered are Kubernetes pods, services, deployments, container networking with CNI plugins, observability tools, and CI/CD approaches.

#lspe Q1 2013 dynamically scaling netflix in the cloudCoburn Watson

CloudOps_tool.pptxAshnikbiz

Webinar_CloudOps final.pptxAshnikbiz

Database automation tools are needed to automate repetitive tasks, reduce risks from manual errors, improve alignment between business and IT, and allow organizations to move faster. They help keep systems running smoothly through monitoring, provisioning, backup/restore, maintenance, security, and more. When choosing a tool, organizations should consider ease of implementation, breadth of use cases covered, ability to work on-premises and in the cloud, long-term costs, customizability, learning curve, and do a trial run.

More Related Content

Similar to Autoscaling in Kubernetes (K8s) (20)

Cray HPC Environments for Leading Edge Simulationsinside-BigData.com

CeiloscaFabio Giannetti

Anton Boyko, "The evolution of microservices platform or marketing gibberish"Sigma Software

Anton Boyko "The future of serverless computing"Fwdays

Robotics technical Presentationklepsydratechnologie

Surge 2013: Maximizing Scalability, Resiliency, and Engineering Velocity in t...Coburn Watson

Kubernetes at NU.nl (Kubernetes meetup 2019-09-05)Tibo Beijen

The Need of Cloud-Native ApplicationEmiliano Pecis

Taming the cost of your first cloud - CCCEU 2014Tim Mackey

Building Reactive applications with AkkaKnoldus Inc.

Serverless on AWS : Understanding the hard parts at Froscon 2019Vadym Kazulkin

Univa Presentation at DAC 2020 Univa, an Altair Company

kubernetesNotes_ggggggbggggggggggggggg.pdfTinotendaChagaka

Intel Atom Processor Pre-Silicon Verification ExperienceDVClub

Making Apache Kafka Even Faster And More ScalablePaulBrebner2

Intel open stack-summit-session-nov13-finalDeepak Mane

Consolidate your enterprise workloads without creating new performance constr...NetApp

Diameter progress, popular use cases, roadmap - Mobicents Summit 2011telestax

[AWS Dev Day] 실습워크샵 | Amazon EKS 핸즈온 워크샵Amazon Web Services Korea

#lspe Q1 2013 dynamically scaling netflix in the cloudCoburn Watson

Cray HPC Environments for Leading Edge Simulationsinside-BigData.com

CeiloscaFabio Giannetti

Anton Boyko, "The evolution of microservices platform or marketing gibberish"Sigma Software

Anton Boyko "The future of serverless computing"Fwdays

Robotics technical Presentationklepsydratechnologie

Surge 2013: Maximizing Scalability, Resiliency, and Engineering Velocity in t...Coburn Watson

Kubernetes at NU.nl (Kubernetes meetup 2019-09-05)Tibo Beijen

The Need of Cloud-Native ApplicationEmiliano Pecis

Taming the cost of your first cloud - CCCEU 2014Tim Mackey

Building Reactive applications with AkkaKnoldus Inc.

Serverless on AWS : Understanding the hard parts at Froscon 2019Vadym Kazulkin

Univa Presentation at DAC 2020 Univa, an Altair Company

kubernetesNotes_ggggggbggggggggggggggg.pdfTinotendaChagaka

Intel Atom Processor Pre-Silicon Verification ExperienceDVClub

Making Apache Kafka Even Faster And More ScalablePaulBrebner2

Intel open stack-summit-session-nov13-finalDeepak Mane

Consolidate your enterprise workloads without creating new performance constr...NetApp

Diameter progress, popular use cases, roadmap - Mobicents Summit 2011telestax

[AWS Dev Day] 실습워크샵 | Amazon EKS 핸즈온 워크샵Amazon Web Services Korea

#lspe Q1 2013 dynamically scaling netflix in the cloudCoburn Watson

More from Ashnikbiz (20)

CloudOps_tool.pptxAshnikbiz

Webinar_CloudOps final.pptxAshnikbiz

Why and how to use Kubernetes for scaling of your multi-tier (n-tier) appli...Ashnikbiz

Kubernetes can be used to scale multi-tier applications by providing tools for container orchestration, including service discovery, load balancing, storage orchestration, and self-healing capabilities. It addresses challenges with traditional monolithic architectures by allowing microservices that are isolated, declarative, and can autoscale horizontally and vertically through features like horizontal pod autoscaling, vertical pod autoscaling, and cluster autoscaling. This allows applications to dynamically add or remove pods and nodes as needed to meet changing workload demands.

Zero trust in a multi tenant environment Ashnikbiz

Vault provides secure multi-tenancy by allowing the creation of namespaces that isolate "Vaults within a Vault". Each namespace can have independent authentication methods, secret engines, policies, identities and access management. Vault also enables API-driven encryption through secret engines like Transit and unified identities across multiple environments through its identity system. These capabilities allow Vault to securely store, restrict access to, and manage encryption of secrets and keys for multi-tenant infrastructure.

Deploy and automate ‘Secrets Management’ for a multi-cloud environmentAshnikbiz

Over the years, there has been a massive transition from on-premise environments to hybrid or multi-cloud, resulting in a significant increase in the adoption of cloud-native practices and technologies. However, while cloud-native methodologies offer growing benefits and are instrumental to digitalization, they can pose considerable challenges in managing secrets. Secrets management aims to solve a lack of visibility and control on handling these highly-trusted credentials.

Deploy, move and manage Postgres across cloud platformsAshnikbiz

Running applications in a hybrid set-up creates complexities that can increase downtime and maintenance. PostgreSQL runs across virtual, cloud, and container environments; minimizing complexity without sacrificing the performance, so you can take control. Being today’s undisputed leader of relational databases for new and modern applications, Postgres’ tools and features will enable you to swiftly deploy, move and manage your database across platforms.

Deploy, move and manage Postgres across cloud platformsAshnikbiz

Running applications in a hybrid setup creates complexities that can increase downtime and maintenance. PostgreSQL runs across virtual, cloud, and container environments; minimizing complexity without sacrificing the performance, so you can take control. Being today’s undisputed leader of relational databases for new and modern applications, Postgres’ tools and features will enable you to swiftly deploy, move and manage your database across platforms. Webinar Covers: Multi-cloud strategy and trends How EDB Postgres can pillar your cloud platform Use cases: Postgres and its tools on-premises and multi-cloud platforms Demo: Using Postgres tools on-premises and for diverse cloud platforms – handling back-up, monitoring, and ensuring Business Continuity Process (BCP)

The Best Approach For Multi-cloud Infrastructure Provisioning-2Ashnikbiz

The webinar covers the best approach for multi-cloud infrastructure provisioning. It discusses the complexities of multi-cloud environments and how Terraform can help with adoption. The webinar features a demo of provisioning cloud infrastructure with Terraform. It also addresses the business drivers for multi-cloud, including improving customer experience, and the challenges of multi-cloud such as the need for multiple skills and complex deployments.

The Best Approach For Multi-cloud Infrastructure ProvisioningAshnikbiz

This document discusses challenges of cloud computing and how HashiCorp's products address them. It introduces Cloud 2.0 as needing a unified control plane across networking, security, operations, and development to manage applications across private and public clouds. HashiCorp's products like Terraform for infrastructure as code, Vault for secrets management, and Consul for service discovery provide a full stack to operate in modern, dynamic cloud environments.

Which PostgreSQL is right for your multi cloud strategy? P2Ashnikbiz

Which PostgreSQL is right for your multi cloud strategy? P1Ashnikbiz

This webinar discusses strategies for using PostgreSQL in a multi-cloud environment. It will cover the different PostgreSQL deployment options for multi-cloud, including PostgreSQL-as-a-service, containerized PostgreSQL, and PostgreSQL on infrastructure-as-a-service. The webinar will also demonstrate how to automate deploying and scaling PostgreSQL. Key considerations for choosing a PostgreSQL option for multi-cloud include manageability, transportability, and automation.

Reduce the complexities of managing Kubernetes clusters anywhere 2Ashnikbiz

Reduce the complexities of managing Kubernetes clusters anywhereAshnikbiz

Enhance your multi-cloud application performance using Redis Enterprise P2Ashnikbiz

This document provides an overview of Redis Enterprise. It discusses how Redis Enterprise is an in-memory multi-model database built on open source Redis that supports high-performance operational, analytics, and hybrid use cases. It offers deployment options including cloud, on-premises, and Kubernetes and supports a wide range of modern use cases including caching, transactions, streaming, messaging, and analytics workloads. Redis Enterprise provides features like high availability, security, and support for Redis modules to extend its capabilities.

Enhance your multi-cloud application performance using Redis Enterprise P1Ashnikbiz

Redis Enterprise can help enhance performance for multi-cloud applications. The webinar covered challenges of multi-cloud environments and how Redis is used, including a demo of setting up Redis across different platforms. It discussed how 72% of customers in Southeast Asia and India are adopting multi-cloud and the business benefits like improved experience, flexibility and reduced time to launch products. Challenges of multi-cloud like complex architecture and application performance were also reviewed.

Gain multi-cloud versatility with software load balancing designed for cloud-...Ashnikbiz

Enterprise-class security with PostgreSQL - 1Ashnikbiz

Enterprise-class security with PostgreSQL - 2Ashnikbiz

Securing Your CI Pipeline with HashiCorp Vault - P2Ashnikbiz

Today, CI/CD is becoming a practice for optimum software delivery in almost every organization. What is key is how you manage the secrets in your pipeline, especially in a large organization with multiple projects, across several teams. Hashicorp Vault helps organizations to centrally manage secrets even in your CI/CD pipelines. WEBINAR COVERS: Why is it critical to secure your pipeline which needs to access a lot of important secrets in order to provision and deploy How Vault provides a unified interface to any secret while providing tight access control and recording a detailed audit log Customer use cases and scenarios Demo: How to secure your CI pipeline with Vault Watch on demand: https://bit.ly/35QCq0u