SlideShare a Scribd company logo
Scalable Performance


Building enterprise-scale web applications that perform
Scalability vs. Performance



 • Ratio of the increase in          • Serving a single request in the
   throughput to an increase in        shortest amount of time
   resources                         • Inverse of latency
 • Support additional users at the
   least incremental cost
 • Predictability of application
   behavior as users are added
Scalable Performance



 • Number of requests that can be concurrently served
   (throughput) while meeting a minimum level of service
   (response time)
 • Measuring:
     o   Resource utilization
     o   Throughput
     o   Response time
Horizontal vs. Vertical Scaling



 • Increase the hardware resources       • Improve hardware capabilities
 • Separate types of processing into       (cpu, RAM, storage, etc…)
   tiers                                 • No network bottleneck
 • Commodity hardware is a               • Becomes increasingly expensive
   predictable cost per user             • Practical and financial limitations
 • Limitations to scaling are dictated     to the ability to scale
   by application architecture           • Typically increases performance
 • Can degrade performance
Horizontal vs. Vertical Scaling
General Observations



 • Performance decreases in each later tier (LB > web > app >
   DB) due to increasing complexity
     o   Service requests in the earliest possible tier
 • Costs of scalability increases in each later tier (LB < web <
   app < DB)
     o   Architect bottlenecks in the earliest possible tier
 • Scalable performance is ultimately limited by the operations
   that do not scale linearly
 • Ideally, each request that makes it to the database tier
   would have its own connection
     o   Realistically, this means serving requests in earlier tiers because of
         constraints to db scaling
JEE Request Processing
Application Bottlenecks



 •   Thread starvation
 •   Thread contention
 •   IO contention
 •   IO performance
 •   Memory limitations
 •   Data access
Resource Utilization
Resource Capacity Settings



 •   Database CPUs
 •   Database connection pool
 •   Application server CPUs
 •   Application server thread pool
 •   JVM Heap settings
 •   Web server thread pool
Resource Capacity Settings



 • Walk through the application architecture and identify the
   points where a request could potentially wait.
 • Open all wait points.
 • Generate balanced and representative load against the
   environment.
 • Identify the limiting wait point’s saturation point.
 • Tighten all wait points to facilitate only the maximum load of
   the limiting wait point.
 • Force all pending requests to wait at the Web server.
 • Add more resources.
Profiling



 •   Long running http requests
 •   Long running methods
 •   Memory leaks
 •   Deadlocks
 •   Long running queries
Database Tier



 • System of record
 • Difficult to scale horizontally and expensive to scale
   vertically
 • Keep connections limited to what the server will support
     o   Block at the app tier
 • Perform data processing on staging server
 • Each database has its own optimization techniques
     o   Explain plan to locate and eliminate full table scans
     o   Query and table caches
     o   Buffer sizes
Application Tier



 • Generally cannot be stateless due to security and business
   requirements
 • Sticky vs. clustered sessions
     o   Use the HTTPSession sparingly
     o   If the app does not need to be HA, it may not require clean failover
         and can drop sessions
 • Scaling horizontally could potentially put more load on the
   DB
     o   Caching can be used to offset the load
Application Caching



 • Read-only data can be cached like static data in the web
   tier
 • Write-able data can be cached but will impose limitations on
   clustering
     o   Will probably either need to be in-sync across the cluster or turned off
         completely
 • Filters can provide caching of dynamic, secure data at the
   earliest point in this tier
     o   Caches entire response
     o   Not recommended for user-specific data
 • Service layer or data access caches provide a simple way
   to stop requests from continuing to the database
     o   Transparent to the calling code
     o   Cache interceptors
Tomcat Tuning



 • maxThreads controls actively served connections
 • backlog controls the number of connections that can be
   queued
 • maxThreads + backlog = total accepted connections
 • connectionTimeout can be used to drop faulty connections
 • bufferSize is by default set to -1, no buffering of output
 • Keep heap size manageable, < 2GB
Web Tier



 • Clustering is easiest in the web tier because they are
   generally stateless
 • Web tier clustering can provide super-linear scaling (IO
   contention, context switching costs)
 • The web tier should serve all static data (images, static
   html)
 • The web tier can serve dynamic requests by caching non-
   secure data that only depends upon url parameters
     o   Squid reverse proxy
     o   Apache mod cache
     o   Memcached
Apache Tuning



 • Limit connections in the web tier to prevent overloading
   later tiers (MaxClients)
 • ServerLimit x Memory per process < RAM available to limit
   swapping
 • Avg connections = ThreadsPerChild x Apache hosts / App
   Server hosts
 • ProxyPass max = ThreadsPerChild
Other



 • Grid Caches
        o   GigaSpaces
        o   Coherence
 • CDN
        o   Akamai
 • Compute Appliances
        o   Azul Systems
References



 • http://www.theserverside.com/tt/articles/content/JIApresent
   ations/JIA-HASP.pdf
 • http://www.mnot.net/cache_docs/
 • http://httpd.apache.org/docs/2.2/misc/perf-tuning.html
 • http://www.yourkit.com/overview/index.jsp
 • http://dev2dev.bea.com/pub/a/2006/05/declarative-
   caching.html
 • http://azulsystems.com/
 • http://www.theserverside.com/tt/knowledgecenter/knowledg
   ecenter.tss?l=ProJavaEE_Ch06

More Related Content

What's hot (20)

PPTX
Cloud Design Pattern part2
Masashi Narumoto
 
PPTX
Hadoop introduction
Dong Ngoc
 
PPT
The experiences of migrating a large scale, high performance healthcare network
george.james
 
PPT
Scalable Web Architectures and Infrastructure
george.james
 
PDF
Datasheet weblogic midvisionextensionforibmraf
MidVision
 
PDF
HBaseCon2017 Apache HBase at Didi
HBaseCon
 
PDF
Restful风格ž„web服务架构
Benjamin Tan
 
PPTX
Server load balancer ppt
Shilpi Tandon
 
PDF
MariaDB High Availability Webinar
MariaDB plc
 
PPTX
Resource loading, prioritization, HTTP/2 - oh my!
Patrick Meenan
 
PDF
Http2 in practice
Patrick Meenan
 
PPTX
Change Data Capture using Kafka
Akash Vacher
 
PDF
Benchmarking Performance and Scalability with Web Stress
InterSystems Corporation
 
PPT
Weblogic-clustering-failover-and-load-balancing-training
Unmesh Baile
 
KEY
Seda与Java并行编程点滴
Benjamin Tan
 
PDF
SAP on Azure Web Dispatcher High Availability
Gary Jackson MBCS
 
PPTX
System design for video streaming service
Nirmik Kale
 
PPTX
Peer Cache for Configuration Manager clients
Karthick Jokirathinam
 
PPT
Load Balancing
nashniv
 
PPTX
Building WebLogic Domains With WLST
C2B2 Consulting
 
Cloud Design Pattern part2
Masashi Narumoto
 
Hadoop introduction
Dong Ngoc
 
The experiences of migrating a large scale, high performance healthcare network
george.james
 
Scalable Web Architectures and Infrastructure
george.james
 
Datasheet weblogic midvisionextensionforibmraf
MidVision
 
HBaseCon2017 Apache HBase at Didi
HBaseCon
 
Restful风格ž„web服务架构
Benjamin Tan
 
Server load balancer ppt
Shilpi Tandon
 
MariaDB High Availability Webinar
MariaDB plc
 
Resource loading, prioritization, HTTP/2 - oh my!
Patrick Meenan
 
Http2 in practice
Patrick Meenan
 
Change Data Capture using Kafka
Akash Vacher
 
Benchmarking Performance and Scalability with Web Stress
InterSystems Corporation
 
Weblogic-clustering-failover-and-load-balancing-training
Unmesh Baile
 
Seda与Java并行编程点滴
Benjamin Tan
 
SAP on Azure Web Dispatcher High Availability
Gary Jackson MBCS
 
System design for video streaming service
Nirmik Kale
 
Peer Cache for Configuration Manager clients
Karthick Jokirathinam
 
Load Balancing
nashniv
 
Building WebLogic Domains With WLST
C2B2 Consulting
 

Viewers also liked (18)

PPTX
Presentasi Message Confidentiality (Kerahasiaan Pesan)
Uliel Azmie
 
PDF
4 sem pathology - 2005 to 2010 1
Siam Weng Loong
 
PPT
Ppa6 lecture ch_13
Chris Auld
 
PPT
Electric charge and electric field
Chris Auld
 
PPTX
Presentati Sisteam Antrian Real-Time
Uliel Azmie
 
PPT
Chapter 3 motion in two d 2
Chris Auld
 
PPT
чил презентация
Doltz
 
PPTX
Presentasi Implementasi Algoritma ID3
Uliel Azmie
 
PDF
IoT: from zero to hero for web developers - GDG DevFest Nantes 2016
Jan Jongboom
 
PDF
Building IoT devices for fun and profit - Mobile Era 2016
Jan Jongboom
 
PPT
Chapter 3 motion in two d
Chris Auld
 
PPT
Chapter 15 thermodyanamic
Chris Auld
 
PPT
Motion in one direction
Chris Auld
 
PPTX
Implementasi Metode AHP (Kasus : Smartphone)
Uliel Azmie
 
PPT
Electric charge and electric field
Chris Auld
 
PDF
Run your JavaScript app for years on a coin cell - JSConf.asia 2016
Jan Jongboom
 
PDF
Qué son los estilos de aprendizaje
javier ramos moreno
 
Presentasi Message Confidentiality (Kerahasiaan Pesan)
Uliel Azmie
 
4 sem pathology - 2005 to 2010 1
Siam Weng Loong
 
Ppa6 lecture ch_13
Chris Auld
 
Electric charge and electric field
Chris Auld
 
Presentati Sisteam Antrian Real-Time
Uliel Azmie
 
Chapter 3 motion in two d 2
Chris Auld
 
чил презентация
Doltz
 
Presentasi Implementasi Algoritma ID3
Uliel Azmie
 
IoT: from zero to hero for web developers - GDG DevFest Nantes 2016
Jan Jongboom
 
Building IoT devices for fun and profit - Mobile Era 2016
Jan Jongboom
 
Chapter 3 motion in two d
Chris Auld
 
Chapter 15 thermodyanamic
Chris Auld
 
Motion in one direction
Chris Auld
 
Implementasi Metode AHP (Kasus : Smartphone)
Uliel Azmie
 
Electric charge and electric field
Chris Auld
 
Run your JavaScript app for years on a coin cell - JSConf.asia 2016
Jan Jongboom
 
Qué son los estilos de aprendizaje
javier ramos moreno
 
Ad

Similar to Performance and Scalability Tuning (20)

PPT
Large-scale projects development (scaling LAMP)
Alexey Rybak
 
PDF
Scalability broad strokes
Gagan Bajpai
 
PPS
Web20expo Scalable Web Arch
mclee
 
PPS
Web20expo Scalable Web Arch
guest18a0f1
 
PPS
Web20expo Scalable Web Arch
royans
 
PPT
Silicon India Java Conference: Building Scalable Solutions For Commerce Silic...
Kalaiselvan (Selvan)
 
PPS
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Cal Henderson
 
PPTX
How to Build Scalable Websites in the Cloud
RightScale
 
PDF
Scalability designprinciples-v2-130718023602-phpapp02 (1)
Minal Patil
 
PDF
Scalability Design Principles - Internal Session
Sachin Sancheti - Microsoft Azure Architect
 
PPTX
Scalable Web Architecture and Distributed Systems
hyun soomyung
 
PPTX
Building Scalable Websites for the Cloud
RightScale
 
PDF
Bottlenecks exposed web app db servers
Upender Dravidum
 
PDF
Build A Scalable Mobile App
Mohamed Aboul-Fotouh
 
PPTX
Building-Scalable-Web-Applications.Presentation
Ozias Rondon
 
PDF
Best Practices for Building Scalable Web Applications.pdf
Isabella Barry
 
PPTX
Application architecture for cloud
Marco Parenzan
 
PPTX
05. performance-concepts
Muhammad Ahad
 
PPTX
Java Colombo: Developing Highly Scalable Apps
Afkham Azeez
 
PDF
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...
IndicThreads
 
Large-scale projects development (scaling LAMP)
Alexey Rybak
 
Scalability broad strokes
Gagan Bajpai
 
Web20expo Scalable Web Arch
mclee
 
Web20expo Scalable Web Arch
guest18a0f1
 
Web20expo Scalable Web Arch
royans
 
Silicon India Java Conference: Building Scalable Solutions For Commerce Silic...
Kalaiselvan (Selvan)
 
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Cal Henderson
 
How to Build Scalable Websites in the Cloud
RightScale
 
Scalability designprinciples-v2-130718023602-phpapp02 (1)
Minal Patil
 
Scalability Design Principles - Internal Session
Sachin Sancheti - Microsoft Azure Architect
 
Scalable Web Architecture and Distributed Systems
hyun soomyung
 
Building Scalable Websites for the Cloud
RightScale
 
Bottlenecks exposed web app db servers
Upender Dravidum
 
Build A Scalable Mobile App
Mohamed Aboul-Fotouh
 
Building-Scalable-Web-Applications.Presentation
Ozias Rondon
 
Best Practices for Building Scalable Web Applications.pdf
Isabella Barry
 
Application architecture for cloud
Marco Parenzan
 
05. performance-concepts
Muhammad Ahad
 
Java Colombo: Developing Highly Scalable Apps
Afkham Azeez
 
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...
IndicThreads
 
Ad

Recently uploaded (20)

PDF
HydITEx corporation Booklet 2025 English
Георгий Феодориди
 
PDF
Sustainable and comertially viable mining process.pdf
Avijit Kumar Roy
 
PDF
Are there government-backed agri-software initiatives in Limerick.pdf
giselawagner2
 
PPTX
Darren Mills The Migration Modernization Balancing Act: Navigating Risks and...
AWS Chicago
 
PDF
Bitcoin+ Escalando sin concesiones - Parte 1
Fernando Paredes García
 
PPTX
UI5Con 2025 - Get to Know Your UI5 Tooling
Wouter Lemaire
 
PDF
Shuen Mei Parth Sharma Boost Productivity, Innovation and Efficiency wit...
AWS Chicago
 
PPTX
The Yotta x CloudStack Advantage: Scalable, India-First Cloud
ShapeBlue
 
PDF
NewMind AI Journal - Weekly Chronicles - July'25 Week II
NewMind AI
 
PDF
CloudStack GPU Integration - Rohit Yadav
ShapeBlue
 
PDF
Wojciech Ciemski for Top Cyber News MAGAZINE. June 2025
Dr. Ludmila Morozova-Buss
 
PDF
Market Wrap for 18th July 2025 by CIFDAQ
CIFDAQ
 
PDF
HR agent at Mediq: Lessons learned on Agent Builder & Maestro by Tacstone Tec...
UiPathCommunity
 
PDF
SFWelly Summer 25 Release Highlights July 2025
Anna Loughnan Colquhoun
 
PPTX
Lifting and Rigging Safety AQG-2025-2.pptx
farrukhkhan658034
 
PDF
Novus-Safe Pro: Brochure-What is Novus Safe Pro?.pdf
Novus Hi-Tech
 
PDF
Human-centred design in online workplace learning and relationship to engagem...
Tracy Tang
 
PDF
Ampere Offers Energy-Efficient Future For AI And Cloud
ShapeBlue
 
PDF
Upgrading to z_OS V2R4 Part 01 of 02.pdf
Flavio787771
 
PDF
Meetup Kickoff & Welcome - Rohit Yadav, CSIUG Chairman
ShapeBlue
 
HydITEx corporation Booklet 2025 English
Георгий Феодориди
 
Sustainable and comertially viable mining process.pdf
Avijit Kumar Roy
 
Are there government-backed agri-software initiatives in Limerick.pdf
giselawagner2
 
Darren Mills The Migration Modernization Balancing Act: Navigating Risks and...
AWS Chicago
 
Bitcoin+ Escalando sin concesiones - Parte 1
Fernando Paredes García
 
UI5Con 2025 - Get to Know Your UI5 Tooling
Wouter Lemaire
 
Shuen Mei Parth Sharma Boost Productivity, Innovation and Efficiency wit...
AWS Chicago
 
The Yotta x CloudStack Advantage: Scalable, India-First Cloud
ShapeBlue
 
NewMind AI Journal - Weekly Chronicles - July'25 Week II
NewMind AI
 
CloudStack GPU Integration - Rohit Yadav
ShapeBlue
 
Wojciech Ciemski for Top Cyber News MAGAZINE. June 2025
Dr. Ludmila Morozova-Buss
 
Market Wrap for 18th July 2025 by CIFDAQ
CIFDAQ
 
HR agent at Mediq: Lessons learned on Agent Builder & Maestro by Tacstone Tec...
UiPathCommunity
 
SFWelly Summer 25 Release Highlights July 2025
Anna Loughnan Colquhoun
 
Lifting and Rigging Safety AQG-2025-2.pptx
farrukhkhan658034
 
Novus-Safe Pro: Brochure-What is Novus Safe Pro?.pdf
Novus Hi-Tech
 
Human-centred design in online workplace learning and relationship to engagem...
Tracy Tang
 
Ampere Offers Energy-Efficient Future For AI And Cloud
ShapeBlue
 
Upgrading to z_OS V2R4 Part 01 of 02.pdf
Flavio787771
 
Meetup Kickoff & Welcome - Rohit Yadav, CSIUG Chairman
ShapeBlue
 

Performance and Scalability Tuning

  • 1. Scalable Performance Building enterprise-scale web applications that perform
  • 2. Scalability vs. Performance • Ratio of the increase in • Serving a single request in the throughput to an increase in shortest amount of time resources • Inverse of latency • Support additional users at the least incremental cost • Predictability of application behavior as users are added
  • 3. Scalable Performance • Number of requests that can be concurrently served (throughput) while meeting a minimum level of service (response time) • Measuring: o Resource utilization o Throughput o Response time
  • 4. Horizontal vs. Vertical Scaling • Increase the hardware resources • Improve hardware capabilities • Separate types of processing into (cpu, RAM, storage, etc…) tiers • No network bottleneck • Commodity hardware is a • Becomes increasingly expensive predictable cost per user • Practical and financial limitations • Limitations to scaling are dictated to the ability to scale by application architecture • Typically increases performance • Can degrade performance
  • 6. General Observations • Performance decreases in each later tier (LB > web > app > DB) due to increasing complexity o Service requests in the earliest possible tier • Costs of scalability increases in each later tier (LB < web < app < DB) o Architect bottlenecks in the earliest possible tier • Scalable performance is ultimately limited by the operations that do not scale linearly • Ideally, each request that makes it to the database tier would have its own connection o Realistically, this means serving requests in earlier tiers because of constraints to db scaling
  • 8. Application Bottlenecks • Thread starvation • Thread contention • IO contention • IO performance • Memory limitations • Data access
  • 10. Resource Capacity Settings • Database CPUs • Database connection pool • Application server CPUs • Application server thread pool • JVM Heap settings • Web server thread pool
  • 11. Resource Capacity Settings • Walk through the application architecture and identify the points where a request could potentially wait. • Open all wait points. • Generate balanced and representative load against the environment. • Identify the limiting wait point’s saturation point. • Tighten all wait points to facilitate only the maximum load of the limiting wait point. • Force all pending requests to wait at the Web server. • Add more resources.
  • 12. Profiling • Long running http requests • Long running methods • Memory leaks • Deadlocks • Long running queries
  • 13. Database Tier • System of record • Difficult to scale horizontally and expensive to scale vertically • Keep connections limited to what the server will support o Block at the app tier • Perform data processing on staging server • Each database has its own optimization techniques o Explain plan to locate and eliminate full table scans o Query and table caches o Buffer sizes
  • 14. Application Tier • Generally cannot be stateless due to security and business requirements • Sticky vs. clustered sessions o Use the HTTPSession sparingly o If the app does not need to be HA, it may not require clean failover and can drop sessions • Scaling horizontally could potentially put more load on the DB o Caching can be used to offset the load
  • 15. Application Caching • Read-only data can be cached like static data in the web tier • Write-able data can be cached but will impose limitations on clustering o Will probably either need to be in-sync across the cluster or turned off completely • Filters can provide caching of dynamic, secure data at the earliest point in this tier o Caches entire response o Not recommended for user-specific data • Service layer or data access caches provide a simple way to stop requests from continuing to the database o Transparent to the calling code o Cache interceptors
  • 16. Tomcat Tuning • maxThreads controls actively served connections • backlog controls the number of connections that can be queued • maxThreads + backlog = total accepted connections • connectionTimeout can be used to drop faulty connections • bufferSize is by default set to -1, no buffering of output • Keep heap size manageable, < 2GB
  • 17. Web Tier • Clustering is easiest in the web tier because they are generally stateless • Web tier clustering can provide super-linear scaling (IO contention, context switching costs) • The web tier should serve all static data (images, static html) • The web tier can serve dynamic requests by caching non- secure data that only depends upon url parameters o Squid reverse proxy o Apache mod cache o Memcached
  • 18. Apache Tuning • Limit connections in the web tier to prevent overloading later tiers (MaxClients) • ServerLimit x Memory per process < RAM available to limit swapping • Avg connections = ThreadsPerChild x Apache hosts / App Server hosts • ProxyPass max = ThreadsPerChild
  • 19. Other • Grid Caches o GigaSpaces o Coherence • CDN o Akamai • Compute Appliances o Azul Systems
  • 20. References • http://www.theserverside.com/tt/articles/content/JIApresent ations/JIA-HASP.pdf • http://www.mnot.net/cache_docs/ • http://httpd.apache.org/docs/2.2/misc/perf-tuning.html • http://www.yourkit.com/overview/index.jsp • http://dev2dev.bea.com/pub/a/2006/05/declarative- caching.html • http://azulsystems.com/ • http://www.theserverside.com/tt/knowledgecenter/knowledg ecenter.tss?l=ProJavaEE_Ch06