0% found this document useful (0 votes)
74 views

Performance Testing Tutorial - Learn With Its Types and Examples

Uploaded by

GTS SS DRSC
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
74 views

Performance Testing Tutorial - Learn With Its Types and Examples

Uploaded by

GTS SS DRSC
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

Sample Performance Test Strategy Template


About ABC chat Application – Let’s assume that this is a chat workbench that is used in a company by
their customer support agent, this chat application uses XMPP protocol i.e, Extensible Messaging and
Presence Protocol and Open fire server for sending and receiving Instant messages.

Some enhancements have been made to this existing chat client like Remote PC control, PC diagnosis,
Repair tools, Online chat, etc., so this performance Test strategy is a sample of such applications.

For this application let’s assume that the project team has decided to use JMeter for Performance Testing
and JIRA for defect tracking.

The first page of the Performance Test Strategy document should contain Title of the Document
and the Copyrights of the Company.

The second page should contain Document Control which includes, Document Version history,
Reviewers & Approvers list and Contributors list.

The third page should contain the Table of contents, followed by the below topics.

#1) Introduction
The purpose of this document is to define/explain how Performance Testing will be performed on the ABC
chat application for the current and future state.

ABC chat application is an in-house remote support Agent workbench. This workbench will be used to
fulfill customer requests. This Workbench has capabilities such as Online chat, Customer Identification,
Remote PC control, PC diagnosis, and repair tools.

Objective
The key objectives of Performance Testing are as follows:

To gain the confidence that the changes to the existing chat application are in line with the defined
Service Level Agreement.
To ensure that the application performance, service availability, and the stability of the application
are not impacted as a result of the new enhancements.
Transaction Response Times remain within the acceptable tolerance over the increasing Load
profile.
JVMs show stable memory usage over the increasing load profiles.

The below picture clearly explains Performance Testing & Optimization process:

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 1/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

Architecture
You need to incorporate the architecture diagram of your project in this session.

#2) Scope

In Scope
Below is the Performance Testing scope for ABC chat workbench:

Knowledge acquisition of the key business transactions and build load distribution after a detailed
study of the system.
Identify the critical scenarios for performance testing with assistance from different project tracks.
Use the previous release results as a baseline for future releases.
Verify and validate the performance test environment and the Performance/Load test tool
infrastructure for any additional Agent Machines.
Preparation of performance test scripts using JMeter for the identified scenarios that mimic the
identified peak load.
Setup performance monitoring on the servers for monitoring the test in order to identify the
bottlenecks during the test execution phase.
Publish Performance test results.
Coordinate with various stakeholders to resolve the identified performance issues.
Baseline the performance level for future releases.
https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 2/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

Out of Scope
Functional Testing, UAT, System Testing & Security Testing.
Performance testing/monitoring of any third-party interfaces.
Performance Tuning. (Most of the times Tuning is done by a different team, if in case you have
performance engineers to tune the system then you can add this in the Inscope).
Code profiling / Hardware sizing / Capacity planning.
Security / Vulnerability testing / UAT/ White box testing.
Data generation for Performance Testing.
Non-functional tests (For Example, failover, disaster recovery, back-up, usability) other than the
performance tests.
Testing of any mobile solution.
Third-Party Application Performance Testing & Tuning.
Realization of performance recommendations, Application code changes and the vendor-
supported products/server configuration changes will be out of scope from the Performance Team
perspective.
Infrastructure Support / Build Deployment/ Environment Readiness/ Database Restore/ Network
Support etc.

#3) Approach
Performance testing for ABC chat will be conducted using Jmeter by writing custom XMPP plugins that
use a smack library for XMPP connections. These libraries are used to set up connections, login and
send chat messages to the XMPP server.

These libraries are bundled into a jar file that is deployed into the Jmeter and is designed based on the
scenarios to be tested. The Jmeter Work Bench is installed in the local machine which connects to the
JMeter server which has the Load Generators to generate the required load on the Chat server system to
monitor the system behavior.

The test scenario will be scripted using the JMeter tool. The scripts would be customized as required. The
schedule will be created with the required ramp-up to simulate the real-world scenarios.

The Test Scenario would be broken up and measured in the below aspects:

a) Baseline Test: To run each scenario with 1 Vuser and multiple iterations in order to identify whether
the application performance meets the business Service Level Agreement or not.

b) Base Load Test: To meet the Business Benchmark under load test, the Performance Testing team will
perform a baseload test which will help to identify any system performance issues with increasing load
and creates the baseline for the next level of performance testing.

c) Peak Load / Scalability Test: Performance Testing team will perform multiple tests with increasing
Vusers to meet the expected load and also to measure the application performance to establish the
performance curve and identify whether the deployment can support the service level agreements under
the peak user load.

It helps in tuning or capacity planning of the individual Java virtual machines (JVM), the total number of
required JVMs, and the processors. This will be achieved by increasing the no of Vusers to 50%, 75%,
100% and 125% of peak capacity.

d) Endurance Test: Performance Testing team will run this test for a period of 8 Hours / 16 Hours /24
hours to identify memory leaks, performance issues over time, and overall system stability. During
endurance tests, the Performance Testing team monitors the key performance indicators, such as
transaction response times and the stability of memory usage.

System resources like CPU, Memory, and IO need to be monitored with the help of the project team.

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 3/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

The Performance test environment is assumed to be a replica of the production environment. The tests
will be run with an incremental load to identify where the application fails.

Performance Test Scenarios


Include the excel with the set of scenarios.

For Example,

Scenario 1: To validate the Agent and customer chat for X no. of concurrent sessions.

Performance Test Types


The table given below explains the various types of Performance Tests along with their objectives.

Test Type Objective

Baseline Establish the best performance under specific volumes which will be used as a
Test reference for subsequent measurements.

Load Test Measure the system performance under anticipated peak production load.

Endurance Measuring the system stability under high volume for an extended period.
Test

Stress Measure the system performance under unfavorable conditions.


Test

Performance Metrics
Client-side Metrics

S.No Metric Description Format

1 Transaction Response Response time of pages during the steady state of Graph


Time the performance test

2 Throughput The amount of data that the VUsers received from Graph
the server over time

3 Hits/second The number of HTTP requests made by VUsers to Graph


the Web server during the scenario run

4 Number of Total number of transactions that Passed and Excel


Passed/Failed Failed during the test execution
Transactions

5 Transaction Error Rate The Percentage of transactions that failed during Graph
the test execution

System & Network Performance Metrics

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 4/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

Performance Testing Activities & Deliverables

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 5/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

#4) Test Data


It is being assumed that the Performance environment data will be a copy of the production data and the
required test data will be provided by the project team.

#5) Entry & Exit Criteria


Access to all the applications in the environment.
Environment readiness complete.
Performance Test Data readiness.

#6) Defect Management


The Defect Management module in JIRA will be used in the project for defect logging and for
tracking to closure.
Identification of defects that are found during the test execution phase will be captured in JIRA and
these defects will be resolved by the development team according to the below severities.
Defect review meetings would be held on a daily basis with the participation from the testing,
development, Quality Analysts, and business teams.
The criteria to fix defects would get stringent as the project approaches the Go Live date.
Guidelines for defect fix criteria to be published in defect review meetings.

Defect Severity Definition


The definitions of severity codes are as follows:

Severity Description for Development and Enhancement Problems

Blocker System error, show stopper, Network issues

Critical System errors, no clear workaround, interruption or missing business functionality


Major A serious problem was detected for which the workaround exists that might not be
clear to all the users, however, product should not be released without fixing

Medium Problem exists with easy/simple work around but this type of defect may be
released upon approval by Business and/or Project Manager

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 6/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

Severity Description for Development and Enhancement Problems

Low Cosmetic issues that do not interfere with business functionality or other
intermittent problems that are not reproducible every time

#7) Testing Tools & Techniques

Tools Purpose

Jmeter To verify the Load and Performance of the ABC Chat application.

#8) Suspension and Resumption Criteria


Given below are the Critical Suspension and Resumption criteria which will impact the testing
activities:

Suspension Impact Resumption

Environment not set up Testing cannot proceed Environment readiness.


Application found to be unstable Testing cannot proceed. Issue resolved


Test Data not available Testing cannot proceed. Test Data ready

#9) Test Deliverables


The Performance Test Deliverables include:

Performance Testing Strategy


Performance Requirements Document
Performance Test Scenario Document
Performance Test Scripts
Performance Test Results

#10) Roles & Responsibilities


Roles & Responsibilities are clearly explained in the table given below.

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 7/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

#11) Potential Risks & Mitigation Plan

Mitigation
S.No Risk Probability Impact Owner
Plan

1 Test Data unavailability for H H Estimated --


performance load test executions dates for the
performance
test executions
should be
reviewed and
updated.
Functional/Dev
team support
required for
data gathering.

2 Environmental Issues L M Re prioritize --


Deliverables

3 Change in Functionality/design M H This requires --


during performance test execution rework on the
performance
test scenarios

4 Extra performance runs to M H Performance --


troubleshoot performance issues testing
schedules
would be
modified and
updated to the
product team.

5 Estimations are prepared based H H Re prioritize --


on 1 bug fix build for performance. the
Multiple bug fix builds will delay performance
test cycles and eventually it test execution
depends on when the next build cycles.
will be available for rerun.

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 8/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

Mitigation
S.No Risk Probability Impact Owner
Plan

6 Hardware Availability M H Schedule start --


date would be
moved
accordingly.

#12) Assumptions
Performance Test Environment will be a replica of the product architecture landscape. (i.e. correct
Hardware, Software, Interfaces, Integration Layers, etc).
Performance scripts will be designed based on the critical flows for which the usage is high.
All Infrastructure Issues should be resolved before the beginning of Performance testing. Any
system configuration changes made later will invalidate the test results.
An application is stable and ready to use in the Performance test environment.
Necessary hardware and software resources (like load generator machines/software,
controller/agent machines) are made available.
Any changes to the scope will go through a change control process and the performance testing
team will assess the impact of timelines and resources.
Respective Servers are expected to handle the load.
Application trace logs have to be enabled for the supporting systems for monitoring purposes.

#13) Dependencies
Availability of the Performance test environment which is a replica of the product architecture
landscape.
Support required from various Functional, Development, Database and Infrastructure teams during
the test preparation and execution stages.
No code changes are implemented during the entire Performance testing phase as time is very
limited.
In the event of unforeseen issues that lead to restrictions within the timelines, if timelines do not
allow for all the test scopes to be met within the original milestone dates support is available from
the Release Managers, to provide a scoping and prioritization decision.
Application Business Users / Subject Matter Experts will be made available for functional
clarifications, and business transactions sign-off.
ABC chat Program Manager will review and sign-off.

#14) Abbreviations

Abbreviation Description

DB Database

Http Hyper Text Transfer Protocol

JDBC Java Database Connectivity

QA Quality Assurance

SLA Service Level Agreement

SME Subject Matter Expert

UAT User Acceptance Testing

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 9/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

By now you must have clearly understood how to write an effective Performance Test Strategy for a
Messaging application.

Best Practices For Realistic Performance Testing


In order to complete a Performance Test project successfully, we need to ensure that we are doing it in
the right way from the planning stage ie planning, development, execution and analysis.

Let’s take a look at each stage in detail in order to conduct Performance Testing effectively.

#1) Planning
Try to identify the most common workflows i.e the business scenarios which have to be tested. If
the application is an existing one, then check the server logs to understand the most frequently
accessed scenarios. If the application is new than talk to the project management team to
understand the major business flow.
Plan the Load test in such a way that you cover a broad range of workflows like light usage,
medium usage and peak loads.
 You need to perform many cycles of the Load Test, so try to create a framework so that you can
use the same scripts again and again. Also, try to have a backup of the scripts.
Try to analyze how long a test has to run, is it one hour? 8 hours? A day or a week? Usually, long-
duration tests will uncover many major defects such as OS bugs, Memory leaks, etc.
If your organization is using any APM (Application Monitoring Tool), then you can include it during
the test runs so that you can easily identify the performance issues and identify the root cause
more easily.

#2) Development
While developing the scripts i.e recording, try to give a more meaningful transaction name based
on the business flow names that are mentioned in the plan.
Don’t record any third-party applications and if it gets recorded, try to filter it out while enhancing
the scripts.
Not all the dynamic values can be correlated using Autocorrelation feature in the tool, so try to do a
manual correlation to avoid errors.
Try to design your performance tests in such a way that you are hitting the backend of the
application and not just the cache server.

#3) Execution
Make sure to run the tests in a production-like environment, including factors like SSL, Load
Balancer, and Firewalls. This is necessary to simulate a realistic load on the system.
Try to create a workload which is very realistic, you can get this by checking the server logs if it is
an existing application and if it is a new application you need to get this info from the business
team. Remember that workload is very important for conducting successful performance tests.
Never come to a conclusion by running tests with half the production size environment, it is always
advised to conduct tests in an environment which is just the same as production.
While executing long-run tests try to watch the run at frequent intervals in order to make sure that
the test is running smoothly.

#4) Analysis
Try to analyze the application by adding a few important counters first, when a bottleneck is found
then try to add additional counters with respect to the bottleneck. This, in turn, will help in finding
the issue more easily.

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 10/11
6/23/2021 Performance Testing Tutorial: Learn With Its Types And Examples

An application can fail for many reasons like it can fail to respond to a request, respond with an
error code, fail your validation logic or responding too slowly. So try to look into all these before
coming to a conclusion.

Conclusion
I’m sure that this tutorial would have given you immense knowledge on Performance tests and how to
write a Performance Test Strategy Document with detailed examples.

https://www.softwaretestinghelp.com/introduction-to-performance-testing-loadrunner-training-tutorial-part-1/ 11/11

You might also like