Performance Testing Tutorial - Learn With Its Types and Examples
Performance Testing Tutorial - Learn With Its Types and Examples
Some enhancements have been made to this existing chat client like Remote PC control, PC diagnosis,
Repair tools, Online chat, etc., so this performance Test strategy is a sample of such applications.
For this application let’s assume that the project team has decided to use JMeter for Performance Testing
and JIRA for defect tracking.
The first page of the Performance Test Strategy document should contain Title of the Document
and the Copyrights of the Company.
The second page should contain Document Control which includes, Document Version history,
Reviewers & Approvers list and Contributors list.
The third page should contain the Table of contents, followed by the below topics.
#1) Introduction
The purpose of this document is to define/explain how Performance Testing will be performed on the ABC
chat application for the current and future state.
ABC chat application is an in-house remote support Agent workbench. This workbench will be used to
fulfill customer requests. This Workbench has capabilities such as Online chat, Customer Identification,
Remote PC control, PC diagnosis, and repair tools.
Objective
The key objectives of Performance Testing are as follows:
To gain the confidence that the changes to the existing chat application are in line with the defined
Service Level Agreement.
To ensure that the application performance, service availability, and the stability of the application
are not impacted as a result of the new enhancements.
Transaction Response Times remain within the acceptable tolerance over the increasing Load
profile.
JVMs show stable memory usage over the increasing load profiles.
The below picture clearly explains Performance Testing & Optimization process:
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 1/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
Architecture
You need to incorporate the architecture diagram of your project in this session.
#2) Scope
In Scope
Below is the Performance Testing scope for ABC chat workbench:
Knowledge acquisition of the key business transactions and build load distribution after a detailed
study of the system.
Identify the critical scenarios for performance testing with assistance from different project tracks.
Use the previous release results as a baseline for future releases.
Verify and validate the performance test environment and the Performance/Load test tool
infrastructure for any additional Agent Machines.
Preparation of performance test scripts using JMeter for the identified scenarios that mimic the
identified peak load.
Setup performance monitoring on the servers for monitoring the test in order to identify the
bottlenecks during the test execution phase.
Publish Performance test results.
Coordinate with various stakeholders to resolve the identified performance issues.
Baseline the performance level for future releases.
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 2/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
Out of Scope
Functional Testing, UAT, System Testing & Security Testing.
Performance testing/monitoring of any third-party interfaces.
Performance Tuning. (Most of the times Tuning is done by a different team, if in case you have
performance engineers to tune the system then you can add this in the Inscope).
Code profiling / Hardware sizing / Capacity planning.
Security / Vulnerability testing / UAT/ White box testing.
Data generation for Performance Testing.
Non-functional tests (For Example, failover, disaster recovery, back-up, usability) other than the
performance tests.
Testing of any mobile solution.
Third-Party Application Performance Testing & Tuning.
Realization of performance recommendations, Application code changes and the vendor-
supported products/server configuration changes will be out of scope from the Performance Team
perspective.
Infrastructure Support / Build Deployment/ Environment Readiness/ Database Restore/ Network
Support etc.
#3) Approach
Performance testing for ABC chat will be conducted using Jmeter by writing custom XMPP plugins that
use a smack library for XMPP connections. These libraries are used to set up connections, login and
send chat messages to the XMPP server.
These libraries are bundled into a jar file that is deployed into the Jmeter and is designed based on the
scenarios to be tested. The Jmeter Work Bench is installed in the local machine which connects to the
JMeter server which has the Load Generators to generate the required load on the Chat server system to
monitor the system behavior.
The test scenario will be scripted using the JMeter tool. The scripts would be customized as required. The
schedule will be created with the required ramp-up to simulate the real-world scenarios.
The Test Scenario would be broken up and measured in the below aspects:
a) Baseline Test: To run each scenario with 1 Vuser and multiple iterations in order to identify whether
the application performance meets the business Service Level Agreement or not.
b) Base Load Test: To meet the Business Benchmark under load test, the Performance Testing team will
perform a baseload test which will help to identify any system performance issues with increasing load
and creates the baseline for the next level of performance testing.
c) Peak Load / Scalability Test: Performance Testing team will perform multiple tests with increasing
Vusers to meet the expected load and also to measure the application performance to establish the
performance curve and identify whether the deployment can support the service level agreements under
the peak user load.
It helps in tuning or capacity planning of the individual Java virtual machines (JVM), the total number of
required JVMs, and the processors. This will be achieved by increasing the no of Vusers to 50%, 75%,
100% and 125% of peak capacity.
d) Endurance Test: Performance Testing team will run this test for a period of 8 Hours / 16 Hours /24
hours to identify memory leaks, performance issues over time, and overall system stability. During
endurance tests, the Performance Testing team monitors the key performance indicators, such as
transaction response times and the stability of memory usage.
System resources like CPU, Memory, and IO need to be monitored with the help of the project team.
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 3/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
The Performance test environment is assumed to be a replica of the production environment. The tests
will be run with an incremental load to identify where the application fails.
For Example,
Scenario 1: To validate the Agent and customer chat for X no. of concurrent sessions.
Baseline Establish the best performance under specific volumes which will be used as a
Test reference for subsequent measurements.
Load Test Measure the system performance under anticipated peak production load.
Endurance Measuring the system stability under high volume for an extended period.
Test
Performance Metrics
Client-side Metrics
2 Throughput The amount of data that the VUsers received from Graph
the server over time
5 Transaction Error Rate The Percentage of transactions that failed during Graph
the test execution
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 4/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 5/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
Major A serious problem was detected for which the workaround exists that might not be
clear to all the users, however, product should not be released without fixing
Medium Problem exists with easy/simple work around but this type of defect may be
released upon approval by Business and/or Project Manager
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 6/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
Low Cosmetic issues that do not interfere with business functionality or other
intermittent problems that are not reproducible every time
Tools Purpose
Jmeter To verify the Load and Performance of the ABC Chat application.
Test Data not available Testing cannot proceed. Test Data ready
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 7/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
Mitigation
S.No Risk Probability Impact Owner
Plan
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 8/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
Mitigation
S.No Risk Probability Impact Owner
Plan
#12) Assumptions
Performance Test Environment will be a replica of the product architecture landscape. (i.e. correct
Hardware, Software, Interfaces, Integration Layers, etc).
Performance scripts will be designed based on the critical flows for which the usage is high.
All Infrastructure Issues should be resolved before the beginning of Performance testing. Any
system configuration changes made later will invalidate the test results.
An application is stable and ready to use in the Performance test environment.
Necessary hardware and software resources (like load generator machines/software,
controller/agent machines) are made available.
Any changes to the scope will go through a change control process and the performance testing
team will assess the impact of timelines and resources.
Respective Servers are expected to handle the load.
Application trace logs have to be enabled for the supporting systems for monitoring purposes.
#13) Dependencies
Availability of the Performance test environment which is a replica of the product architecture
landscape.
Support required from various Functional, Development, Database and Infrastructure teams during
the test preparation and execution stages.
No code changes are implemented during the entire Performance testing phase as time is very
limited.
In the event of unforeseen issues that lead to restrictions within the timelines, if timelines do not
allow for all the test scopes to be met within the original milestone dates support is available from
the Release Managers, to provide a scoping and prioritization decision.
Application Business Users / Subject Matter Experts will be made available for functional
clarifications, and business transactions sign-off.
ABC chat Program Manager will review and sign-off.
#14) Abbreviations
Abbreviation Description
DB Database
QA Quality Assurance
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 9/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
By now you must have clearly understood how to write an effective Performance Test Strategy for a
Messaging application.
Let’s take a look at each stage in detail in order to conduct Performance Testing effectively.
#1) Planning
Try to identify the most common workflows i.e the business scenarios which have to be tested. If
the application is an existing one, then check the server logs to understand the most frequently
accessed scenarios. If the application is new than talk to the project management team to
understand the major business flow.
Plan the Load test in such a way that you cover a broad range of workflows like light usage,
medium usage and peak loads.
You need to perform many cycles of the Load Test, so try to create a framework so that you can
use the same scripts again and again. Also, try to have a backup of the scripts.
Try to analyze how long a test has to run, is it one hour? 8 hours? A day or a week? Usually, long-
duration tests will uncover many major defects such as OS bugs, Memory leaks, etc.
If your organization is using any APM (Application Monitoring Tool), then you can include it during
the test runs so that you can easily identify the performance issues and identify the root cause
more easily.
#2) Development
While developing the scripts i.e recording, try to give a more meaningful transaction name based
on the business flow names that are mentioned in the plan.
Don’t record any third-party applications and if it gets recorded, try to filter it out while enhancing
the scripts.
Not all the dynamic values can be correlated using Autocorrelation feature in the tool, so try to do a
manual correlation to avoid errors.
Try to design your performance tests in such a way that you are hitting the backend of the
application and not just the cache server.
#3) Execution
Make sure to run the tests in a production-like environment, including factors like SSL, Load
Balancer, and Firewalls. This is necessary to simulate a realistic load on the system.
Try to create a workload which is very realistic, you can get this by checking the server logs if it is
an existing application and if it is a new application you need to get this info from the business
team. Remember that workload is very important for conducting successful performance tests.
Never come to a conclusion by running tests with half the production size environment, it is always
advised to conduct tests in an environment which is just the same as production.
While executing long-run tests try to watch the run at frequent intervals in order to make sure that
the test is running smoothly.
#4) Analysis
Try to analyze the application by adding a few important counters first, when a bottleneck is found
then try to add additional counters with respect to the bottleneck. This, in turn, will help in finding
the issue more easily.
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 10/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples
An application can fail for many reasons like it can fail to respond to a request, respond with an
error code, fail your validation logic or responding too slowly. So try to look into all these before
coming to a conclusion.
Conclusion
I’m sure that this tutorial would have given you immense knowledge on Performance tests and how to
write a Performance Test Strategy Document with detailed examples.
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 11/11