Less11 3 e_loadmodule_1
Download as PPT, PDF0 likes536 views
The document introduces performance testing basics and methodology using Oracle Application Testing Suite. It covers types of performance testing like load testing, stress testing, and volume testing. It emphasizes the importance of setting up realistic user scenarios and test scripts. The testing environment should replicate production and use dedicated agent machines to generate load. Performance testing helps identify bottlenecks and determine scalability.
Ad
More Related Content
What's hot (20)
Similar to Less11 3 e_loadmodule_1 (20)
Ad
More from Suresh Mishra (15)
Ad
Less11 3 e_loadmodule_1
- 1. e-Load Introduction to Oracle Application Testing Suite
- 2. What This Class Module Will Cover Performance testing basics Methodology Planning/Preparation Using e-Load Configuring/Optimizing Load testing
- 3. What This Class Module Will Cover Using Serverstats Configuring Resource monitoring Using e-Reporter Analyzing test data Drawing Conclusions
- 4. Performance Testing Basics Introduction to Oracle Application Testing Suite: Testing Concepts
- 5. What is performance testing? For the class, we will use the term “performance testing” as a generic term for all types of web application scalability testing. Testing in order to determine the performance characteristics of an application Performed under varying levels of concurrent simulated users - or load
- 6. Why is performance testing necessary? To determine whether an application meets the business requirements To determine if the application can handle the expected load Some examples of load metrics are: Concurrent users Throughput (KB/sec) Pages/sec
- 7. Why is performance testing necessary? To evaluate whether the system resources are being utilized efficiently To test system robustness and capability to recover from errors To test across different configurations and versions To identify bottlenecks in order to ensure the application scales
- 8. Why is performance testing necessary? Performance Testing helps determine: When to add components to architecture What components to add to architecture What configuration changes to make What changes to make in the software
- 9. Why is performance testing necessary? Tangible Lost Revenue Lost Customers Increased expenses associated with solving performance problems Intangible Damaged business reputation/brand Diminished customer loyalty Diminished employee morale
- 10. Performance Testing Methodology Types of “Performance” Testing Capacity Planning vs. Performance Tuning Common Factors of Load Testing Measurements of Success
- 11. Types of “Performance” Testing There are four different types of testing Load Testing Performance Testing Stress Testing Volume Testing Each has different goals and metrics Each is related to system performance
- 12. Load Testing Goal: To verify that the performance requirements of the application have been achieved at a various and worst case load levels. This is accomplished by determining response times, transaction rates, and other time sensitive requirements under normal conditions.
- 13. Load Testing Load testing will identify system bottlenecks that occur below normal and maximum anticipated load volume Retest with identical setup after making changes that correct the bottleneck(s) Testing is repeated until all bottlenecks are eliminated and/or system performance is acceptable.
- 14. Performance Testing Goal: To evaluate the system’s ability to continue to function properly under different workloads against a baseline An information gathering and analyzing process Used to determine effective enhancement strategies for maintaining acceptable system performance. Performance testing will help determine how the application can be modified to handle an increased number of users and improve performance.
- 15. Performance Testing There are two common methods of scalability: Vertical Scalability – refers to system’s ability to increase performance as server machines are enhanced with additional hardware or software changes (web server or application server) Horizontal Scalability – refers to system’s ability to distribute the increased load as more server machines are added to the system.
- 16. Stress Testing Goal: Determine the behavior of the system when it reaches its operational limits. Determine what components, if any, will fail when system resources are pushed to the limit. Stress tests should be designed to push system resources to the point where the weak links are exposed.
- 17. Stress Testing Stress testing will help identify bottlenecks that may hamper robustness such as: Memory leaks throughout the system Unexpected behavior when maximum number of application server sessions is reached Unexpected behavior when maximum number of SSL connections to the load balancer is exceeded
- 18. Volume Testing Goal: Testing for the amount of data that a system is able to push Subjecting the system to a series of tests where the volume of data being processed is the subject of the test Volume testing determines whether the physical and logical limits of the system at capacity are acceptable to meet the business requirements.
- 19. Capacity Planning vs. Performance Tuning Capacity Planning – Executed BEFORE the site goes live to determine the scalability of the application Performance Tuning – Gain incremental improvements in the application AFTER the site is ready for production Both are used to help avoid re-architecting and redeveloping the current application
- 20. Setting up a Test Plan Define the performance and scalability requirements Write a test plan What defines a successful test? # of users, pages/sec, error rate?
- 21. Setting up a Test Plan Create user scenarios Based on application’s most important functionality and user actions What transactions are used most frequently? What transactions are integral to the business?
- 22. Setting up a Test Plan Create test scripts Scripts are transaction specific Keep scripts short and modular Base scripts on documented test cases
- 23. Pointers to Keep in Mind Be realistic when creating your user scenario Know your web site visitors Create test scenarios as dynamic as you expect from your visitors Understand the cost of poor website performance Schedule regular checkups of the application
- 24. Pointers to Keep in Mind Be realistic when creating your user scenario Know your web site visitors Create test scenarios as dynamic as you expect from your visitors Understand the cost of poor website performance Schedule regular checkups of the application
- 25. Pointers to Keep in Mind Test early and test often Respond to constant change Simplify test scripts for easy maintenance Use conclusive evidence to highlight potential or probable bottlenecks Use conclusive evidence to justify modifications to the application
- 26. Review 1 Why is it necessary to load test? What is volume testing? What is stress testing? What is the difference between horizontal scalability and vertical scalability? What transactions should be load tested for a given application? What are some issues load testing can help identify? Why should users be ramped up rather than just starting all at once?
- 27. Testing Environment Setup Test Environment Considerations Test Environment should be controlled Isolated network No other testing being performed Network Switch Application Hardware Load Generation Hardware
- 28. Testing Environment Setup Reproduce the production environment Application hardware – use production equivalent equipment. Software – configure with same settings as in production environment Other considerations Database Data – same as production (amount and type) Links to external sites should be disabled 3rd party vendors – should be disconnected or notified of the testing
- 29. Configuring the Load Test Hardware Agent 2 Agent 3 Agent n Agent 1 e-Load ServerStats
- 30. Configuring the Load Test Hardware Determine number and performance of agent machines How much RAM will each machine need? Thick Client uses ~10 MB RAM per VU Thin Client uses ~.4 MB RAM per VU RAM per virtual user MAY VARY depending on script size, think time, and other memory intensive factors
- 31. Configuring the Load Test Hardware Setup of Agent Machines Have full local administrative rights Login with same username/password Install e-Load agent software Don’t install or run any other software Disable unnecessary services Agents should have full and uncontested control of the workstations
- 32. Configuring the Load Test Hardware Setup of Agent Machines Have full local administrative rights Login with same username/password Install e-Load agent software Don’t install or run any other software Disable unnecessary services Agents should have full and uncontested control of the workstations
Editor's Notes
- #3: Oracle Application Testing Suite: Introduction 11 - What This Class Module Will Cover Methodology - types of performance testing Planning – Setting up and utilizing a performance test plan Configuring/Optimizing - e-Load setup and configuration settings Load testing – running the load test, things to do in real time Instructor Note Intro to class. Inquire regarding the students’ background in load testing, their use with other tools, and so on.
- #4: Oracle Application Testing Suite: Introduction 11 - What This Class Module Will Cover (continued) Configuring – Setting up and connecting to backend device counters Resource Monitoring – what to look for during the load test. Analyzing test data – creating e-Reporter graphs Drawing Conclusions – making correlations with the captured data Instructor Note Intro to class continued.
- #6: Oracle Application Testing Suite: Introduction 11 - What is performance testing? Many people use the terms “load testing,” “performance testing,” and “scalability testing” interchangeably to describe this type of testing. We’d like to establish a “blanket” term for use during the class as to not confuse those folks who may be accustomed to using different terms.
- #7: Oracle Application Testing Suite: Introduction 11 - Why is performance testing necessary? Business requirements are typically defined in the project proposal for the application. Expected load is often determined by Marketing using a tool such as Webtrends
- #8: Oracle Application Testing Suite: Introduction 11 - Why is performance testing necessary? (continued) Monitor backend counters to make sure system resources are distributed properly, efficiently, and so on. Bottlenecks item leads into next slide.
- #9: Oracle Application Testing Suite: Introduction 11 - Why is performance testing necessary? (continued) Identifying bottlenecks allow us to make these determinations. If something is a bottleneck but not currently serious enough to affect the application, it will most likely be the first thing looked to add scalability later.
- #10: Oracle Application Testing Suite: Introduction 11 -
- #11: Oracle Application Testing Suite: Introduction 11 -
- #12: Oracle Application Testing Suite: Introduction 11 - Types of “Performance” Testing Each type is different and used to obtain different relevant information about the system under test.
- #13: Oracle Application Testing Suite: Introduction 11 - Load Testing The performance requirements will be stated in the test plan and development documentation. Determining the application’s characteristic response times will yield data to be compared to these requirements
- #14: Oracle Application Testing Suite: Introduction 11 - Load Testing (continued) Performance testing will, in effect, determine system architecture weaknesses at anticipated user volume
- #15: Oracle Application Testing Suite: Introduction 11 - Performance Testing Enhancement strategies: “Will I have to rewrite code to increase performance?” or “Can I just add another Web server or even just another processor within?”
- #16: Oracle Application Testing Suite: Introduction 11 - Performance Testing (continued) This will determine which method will work best depending on what platform the application is built on. Some platforms do not have a significant increase in performance by just adding another CPU to the app or web server, but require another complete machine with the business logic. Other simply need more horsepower to do server-side processing or request responses. This, of course, depends on the business logic and content (images, and so on) of the application.
- #17: Oracle Application Testing Suite: Introduction 11 - Stress Testing Example: a bottleneck may be a memory leak or when the max number of connections on the application server is reached. Or a the max number of SSL connections to the load balancer (like an F5 Big IP) is reached.
- #18: Oracle Application Testing Suite: Introduction 11 -
- #19: Oracle Application Testing Suite: Introduction 11 - Volume Testing Setting up a distinct test where transactions using/requiring the most data (both on the client side and back end) are used to test the application/system.
- #20: Oracle Application Testing Suite: Introduction 11 - Capacity Planning vs. Performance Tuning Capacity planning and performance tuning is performed in order to avoid Re-architecting and to validate the current architecture. Instructor Note This slide is to introduce these concepts the audience, not define or teach how to do each.
- #21: Oracle Application Testing Suite: Introduction 11 - Setting up a Test Plan Define the performance and scalability requirements: Derived from business requirements Concurrent users, hits/day, pages/day, and so on Write a test plan What functions will be tested? How will the test be controlled? Who will do the testing, monitoring, and so on?
- #22: Oracle Application Testing Suite: Introduction 11 - Setting up a Test Plan Tie the test cases to individual business functions Keep the tests short and modular Keep in mind what the back-end effects are when recording or developing a test. For example: a search might exercise the database, but not some external system connection Know what parts of the system are affected by each of the tests so that proper monitoring can be put in place
- #23: Oracle Application Testing Suite: Introduction 11 -
- #24: Oracle Application Testing Suite: Introduction 11 - Pointers to Keep in Mind Be realistic: If the load generated on your web site during your load test isn't realistic, the results are useless at best and dangerously misleading at worst. Make sure your load test uses a real-world methodology that accurately reflects the rate and pattern that your visitors arrive and leave your web site. Know your Web site visitors: Not all your web site visitors are alike. Your load test should account for different types of visitor behavior on your web site. For example, visitors trying to execute a stock transaction will have very different tolerance and tenacity thresholds than will a user trying to view a weather update. Your load test should accommodate variable online behaviors for different types of actions on your web site. Test scenarios as dynamic as your visitors: Your visitors can interact with your web site in a variety of ways. They can often navigate through a multitude of paths to get to the same endpoint or enter a variety of data inputs to generate dynamic content. Make sure that your web site can accommodate these dynamic patterns without tedious, hard-coding of test scripts and huge resource commitments.
- #25: Oracle Application Testing Suite: Introduction 11 - Pointers to Keep in Mind (continued) Understand the cost of poor web site performance: Suppose you know that your web site is performing 5 seconds slower on average than it was a month before. What conclusions can you draw? How many users abandoned your site due to poor performance? At what point did they abandon the site? How much potential revenue was lost as a result? Schedule regular Web site health checkups: If you only load test your site during high-volume periods such as Christmas or Valentine's Day, you may be losing loyal customers to your competitors during the remainder of the year. The key to a high-performing Web site involves more than just running a pre-deployment load test. You must also perform periodic tests throughout the life span of your Web site.
- #26: Oracle Application Testing Suite: Introduction 11 - Pointers to Keep in Mind (continued) Test early, and test often: Always plan ahead to make sure your site can handle major site launches, seasonal promotions and marketing initiatives. However, don't ignore day-to-day content changes and modifications to your Web site. What may appear as a minor change to your Web site can negatively impact your Web site's performance and availability. Respond to constant change: Changes within organizations are often unpredictable and difficult to plan. A last minute change to your infrastructure - such as an application upgrade, network configuration change, load-balancer setting or database modification - can cause performance variations. Prepare your Web site by testing these Web site changes with a full-service, turnkey load test and a flexible, self-service testing solution. Simplify Test Scripts for Easy Maintenance: Simplify the administration of your load tests. Your business environment is dynamic - often necessitating last-minute changes to your web site. Your solution should give you the flexibility to schedule your tests at a moment's notice and to change test scenarios on the fly so you won't have to worry about how even the most minor changes will impact your users.
- #27: Oracle Application Testing Suite: Introduction 11 -
- #28: Oracle Application Testing Suite: Introduction 11 - Testing Environment Setup Define what the “test environment” entails: load test hardware, application hardware, network hardware, and other devices. Note: Isolating the test environment ensures the accuracy of the test results. It’s important to be sure that the performance characteristics are specifically of the application.
- #29: Oracle Application Testing Suite: Introduction 11 - Testing Environment Setup (continued) Application Hardware: If identical hardware can not be obtained, the environment should be made as close as possible: Web servers Application servers Database servers Firewalls, load balancers, and so on Reproducing the production environment is not always a reality as it’s often not in the budget. A scaled down version is usually much more attainable, however it’s important to keep in mind that creating a scaled down version of the application will not provide accurate insights into the overall performance of the actually production environment. By adding hardware or software, a system’s performance characteristics can be significantly altered. Testing a scaled down version of production will help provide some insights but not all into the overall performance characteristics of the production environment.
- #30: Oracle Application Testing Suite: Introduction 11 - Configuring the Load Test Hardware For a large load test, we will have more than one agent machine. It will be necessary to configure them. Instructor Note Briefly discuss what is included in “load test hardware” (controller, console, agent machines).
- #31: Oracle Application Testing Suite: Introduction 11 - Instructor Note Mention that mileage may vary depending on factors like script length and content (number of forms, and so on)
- #32: Oracle Application Testing Suite: Introduction 11 - Configuring the Load Test Hardware Verifying network access to agent workstations Once you have the e-Load controller and agent software installed on the individual workstations, you should verify network access between the controller workstation and each agent workstation. The following provides basic tips and techniques to make sure the e-Load controller workstation can successfully communicate with each agent workstation. 1. Make sure that you have the e-Load agent software loaded on the agent workstation(s) and that it is the same version as the Oracle Application Testing Suite software that is loaded on the e-Load controller workstation. The workstations you plan to use as agents must have either the e-Load agent software or the full Oracle Application Testing Suite installed to work as agents. 2. Make sure you can successfully ping all of the agent workstations from the e-Load controller workstation. The names you use to ping the workstations are the same names that you will specify for the agent workstations in the e-Load controller.
- #33: Oracle Application Testing Suite: Introduction 11 - Configuring the Load Test Hardware (continued) 3. Make sure that the same user is logged on both the e-Load controller workstation and all of the agent workstations. All of the agent workstations must be logged in to be controlled by the controller workstation. You may be able to log in as a different user on the agent workstations as long as the user login has the same administrative privileges as the user logged in on the controller workstation. 4. If the e-Load controller and agent workstations are not participating in the domain security model, both the e-Load controller and the agent workstations must log in as administrator and have the exact same password. 5. In the e-Load controller, add a visual script to the scenario profiles list. Enter the machine name or IP address of the Agent workstation where you want to run the visual script into the workstation field on the scenario tab of e-Load.