Build Your Custom Performance Testing Framework
- 1. T17 Performance Testing 10/16/2014 1:30:00 PM Build Your Custom Performance Testing Framework Presented by: Prashant Suri Rackspace Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com
- 2. Prashant Suri Rackspace Prashant Suri is a senior member of the performance engineering team at Rackspace, the open cloud company. In his role, he helps performance test and benchmark Rackspace's cloud products and applications, writing his own test frameworks to help. In his early years, Prashant developed a number of products on Java/J2EE stack but was fascinated by the vastness of performance engineering. He enjoys improving products’ scaling and robustness. Through the years, he has helped products, developed and used by Fortune 500, scale better. A presenter at international testing conferences, Prashant can be reached at [email protected].
- 3. Build your custom Performance Testing Framework Prashant Suri RACKSPACE® HOSTING | WWW.RACKSPACE.COM Agenda 2 • Evolution of the web and growing complexity of products. • Essentials of performance engineering. • Popular performance test tools. • Holistic performance test framework. • Components of a performance test framework. • Deep dive on an example: performance testing at RAX • Advantages of building your own framework.
- 4. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Evolution of the Web 3 RACKSPACE® HOSTING | WWW.RACKSPACE.COM Essentials of Performance Engineering • Performance objectives – Latency, Throughput, Utilization, Workload. • Performance Modeling – Scenarios, Objectives, Requirements, Budgets, Metrics. • Architecture and Design Guidelines – Principles, Practices and Patterns • Performance and Scalability Frame – Coupling and cohesion, Communication, Concurrency, Resource Management, Caching, State Management, Data Structures/Algorithms. • Measurement, Tuning and Analysis. 4
- 5. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Popular Performance Test Tools 5 RACKSPACE® HOSTING | WWW.RACKSPACE.COM There’s No Silver Bullet 6 • Performance testing is more than simulating load. • Accurate and intricate work load models need to be implemented. • Traffic to be simulated from world geographies. • Testing in service oriented architectures comes with its own challenges. • Reporting and pattern analysis is key for successful performance engineering. • Performance monitors and profiling equally important.
- 6. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Holistic Performance Test Framework 7 RACKSPACE® HOSTING | WWW.RACKSPACE.COM Components of the framework 8 Controller Agents Database & Graphing Engine SimulatingLoads • Brain behind the complete orchestration. Parses and executes the data script. • Picks up threads from queues, and hit application under test. • Repository for metrics of response times, throughputs, errors etc. • Can scale dynamically at run time, to add load generation capacity. • Maintains the test details, workload model details and objectives of any performance test. Capable of distributing load from different data centers. • Distributed across geographies to simulate identical world traffic. • Optimal database tuning for handling 1000+ transactions per second test. Monitoringand Debugging Could be integrated to application monitors and profilers. • Feeds all metrics data into the Database. • Correlates and consolidates all test data. Updated information on runtime activity from database is read and published. • Logging for every granular transaction with application under test. • Historical comparisons and pattern analysis. Could be leveraged for analysis and data based decision making. • Distributed agents through way of logging are consolidated for complete test. • Profiling and application under test data, for analyzing deviation and anomalies.
- 7. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Adobe CQMicrosoft® SharePointMicrosoft® ExchangeRackspace Email Case Study – Capabilities galore! Single F/W to simulate on all Products RACKSPACE® HOSTING | WWW.RACKSPACE.COM Sample Script – Design workload models! 10
- 8. RACKSPACE® HOSTING | WWW.RACKSPACE.COM 11 Execution – Distribute across Agents through a queue RACKSPACE® HOSTING | WWW.RACKSPACE.COM 12 Run time statistics of Performance Test
- 9. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Consolidated Reports 13 RACKSPACE® HOSTING | WWW.RACKSPACE.COM Consolidated Reports 14
- 10. RACKSPACE® HOSTING | WWW.RACKSPACE.COM Advantages of Building your own frameworks 15 • Cost savings on commercial off-the-shelf tools. • Flexibility to simulate complex workload at any scale (literally!) • Simplicity in designing complex workload models. • No constraints and capability of leveraging as per product needs. • Ease of integration with CI/CD Pipeline. • Load simulator and application resource monitors in a single framework setup. • Efficient and inclusive engagement for Performance Engineering in overall product development lifecycle. RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM Questions…
- 11. RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM [email protected]