Chapter 7

Test Tools and Automation

1
Contents

• Introduction to Automated Software Testing

• Process of Automated Testing

• Advantages & Disadvantages

• Choosing Automation Tools

2
 Introduction to automated software
testing

• “Software Testing is usually the part of the project which makes everybody

weep”

• Ever-shrinking schedule and minimal resources

• It involves automating a manual process of testing

3
Definition of Automation

“Automation is the use of tools and strategies that reduce human

involvement or interaction in unskilled, repetitive or

redundant tasks”

4
1. Decision to Automate Test: Overcoming False Expectations

REALITY
EXPECTATION
• No tool can create automatically a comprehensive
• Automatic test plan test plan
generation
• No single tool can support all operating systems
• Test tool fits all environments and programming languages
• Imminent test effort • Initial use of an automated test tool can actually
reduction increase the test effort
• Tool ease of use • Using an automated tool requires new skills,
additional training is required.
• Universal application • Not all the tests required for a project can be
of test automation automated (e.g. some test are physically
impossible; time or cost limitation)
• 100% test coverage • It is impossible to perform an exhaustive testing of
all the possible inputs (simple or combination) to a
system
5
 Test selection guidelines for automation

• Test cases should be automated only if there is a clear economic benefit

over manual execution. We can use automation, If the test cases are:

Fig 1:Test selection guideline for automation.

6
The automated test lifecycle methodology

1. Decision to
Automate
Testing

5. Execution and
2. Test Tool
Management of
Acquisition
Tests

3. Automated
4. Test Planning,
Testing
Design, and
Introduction
Development
Process

7
 1. Decision to Automate Test: Benefits of Automated
Testing
Production of a reliable system Improved requirements definition
Improved performance/load/stress testing
Improved partnership with development team
Improved system development life cycle
Improvement of the quality of the Improved build verification testing (smoke testing)
test effort.
Improved regression testing
Improved multiplatform/software compatibility testing
Improved execution of the repetitive tests
Improved focus on advanced test issues
Execution of tests that manual testing can't accomplish
Ability to reproduce software defects
After-hours testing
Reduction of test effort and minimization of schedule
8
2. Test Tool Acquisition Business analysis business modeling tools
phase configuration management tools
defect tracking tools
• Identify which of the various tool types suit
the organization system environment, requirements management tools
Requirements
considering: definition phase requirements verifiers tools
- the group/department that will use the
tool; Analysis and database design tools
- the budget allocated for the tool design phase application design tools
acquisition; syntax checkers/debuggers
Programming
- the most/least important functions of phase memory leak and run-time error detection tools
the tool etc.
source code or unit testing tools
• Choose the tool type according to the stage static and dynamic analyzers
of the software testing life cycle
• Evaluate different tools from the selected tool
Testing phase test management tools
category
network testing tools
• Hands-on tool estimation – request product
GUI testing tools (capture/playback)
demonstration (evaluation copy)
non-GUI test drivers
• Following the conclusion of the evaluation
process, an evaluation report should be load/performance testing tools
prepared environment testing tools
Testing Tool Acquisition
• Tool or No Tool?

• Tool Selection Team

• Testing Tool Strategy

• Preparation of a Business Case

• Identification of Tool Requirements

• Buy, Open-Source, or Do-It-Yourself

• Preparation of a Shortlist of Candidates

• Detailed Evaluation

• Performance of Competitive Trials

10
3. Automated Testing Introduction Process – Test Process Analysis

Process Test process characteristics (goals, strategies, methodologies) have been defined and they are
review compatible with automated testing
Schedule and budget allows process implementation
The test team is involved from the beginning of SDLC
Test goals Increase the probability that application under test will behave correctly under all circumstances
Increase the probability that application meets all the defined requirements
Execute a complete test of the application within a short time frame
Test objectives Ensure that the system complies with defined client and server response times
Ensure that the most critical end-user paths through the system perform correctly
Incorporate the use of automated test tools whenever feasible
Perform test activities that support both defect prevention and defect detection
Incorporate the use of automated test design and development standards to create reusable and
maintanable scripts
Test strategies Defect prevention (early test involvement, use of process standards, inspection and walkthroughs)
Defect detection (use of automated test tools, unit/integration/system/acceptance test phase)
4. Test Planning
• The test planning element of the ATLM(Automated Test
Lifecycle Methodology ) incorporates the review of all
activities required in the test program
• It ensures that testing processes, methodologies,
techniques, people, tools, schedule and equipment are
organized and applied in an efficient way
• Key elements: planning associated with project milestone
events, test program activities and test program-related
documentation.
• The following must be accomplished:
- the technical approach for these elements is developed;
- personnel are assigned
- performance timelines are specified in the test program
schedule.
• Test planning is not a single event, but rather a process.
• It is the document that guides test execution through
to a conclusion, and it needs to be updated
frequently to reflect any changes.
 Test Design

• Test program model definition – identifies the

test techniques that apply to the test program

• Test architecture definition – involves the

selection of a test architecture/planning model and
population of model attributes.

• Test procedure definition – groups logically the

test procedures/prioritization.

• Automated/manual test mapping (decision what to

automate)
Characteristics of automated test cases
• The characteristics of good automated test cases are:

 Data value independent

 Failure independent
 Final state independent
Test Development
Development Description
Guideline Topics
Design-to- Specify how design and setup activities will be translated
Development into test development action
Transition
Reusable Test Test procedures need to be reusable for highest test
Procedures program return on investment
Data Reusable or Avoid hard-coding data values into scripts
Capture/Playback Outlines on how to apply the use of capture/playback
recording
Maintainable Test A test procedure whose defects are easy to remove and
Procedures can easily be adapted to meet new requirements
Test Script Test script documentation is important for test procedure
Documentation maintainability
Naming Standards Defines the standard naming convention for test
procedures
Modularity Guidelines for creating modular test scripts
Global Files Globally declared functions are available to any
procedure
Constants Use of constants in order to support maintainable
procedure
5. Test Execution  Before initiating system testing:
• How many times are the test cases
executed and when?
• What does one do with the failed test
cases?
• What happens when too many test
cases fail?
• In what order are the test cases
executed?
• Which test cases are to be run just
before ending the system testing phase?
16
Structure of an automated test case

Six major steps in automated test case.

17
o Tools may be classified to get a better overview of the tools available. There are
different classification

• Verification and Validation of the requirements and organisation

• The test activity they support

• The test level the tools primarily support

• The types of failures or defects they can find

• The test approach or test technique they support

• The purpose they have

• The domain to which they are applied

• Who the primary users of the tools are

18
Tool support exists for the following primary users:

• All testers
 Test management tools, including configuration
management tools
• Test analysts and technical test analysts
 Test design tools
 Test data generation tools
 Test oracles
 Simulation/emulation tools
 Test execution tools
 Keyword-driven automation tools
 Comparison tools
 Fault-seeding and fault-injection tools
 Web tools
19
Cont…

• Technical test analysts only

 Static analysis tools

 Dynamic analysis tools

 Performance testing tools

• Programmers (or technical test analysts writing

and maintaining test scripts)

 Debugging, tracing, and troubleshooting tools

20
Components of automation infrastructure

21
Advantages & Disadvantages of Automated Testing

Advantages Disadvantages
• High investment is needed in the tools and
• Reliable: eliminating human
training
error
• High man power requirement for test
• Reusable
preparations
• Better Quality Software
• A lot of testing areas left uncovered
• Fast

• Cost Reduction

22
Choosing Automation Tools

 Ease of integration

 Compatibility

 Performance

 Types of tests

 Maintainability

 Affordability/inexpensive

23
 …Automated

◦ Now days we can get lots of software testing tools in the market.
◦ Selection of tools is totally based on the project requirements
◦ Proprietary/commercial tools or
◦ Free tools/open source tools
◦ Off course, free testing tools may have some limitation in the features
list of the product, so it’s totally based on what are you looking for & is
that your requirement fulfill in free version or go for paid software
testing tools.

24
 …Automated
◦ The tools are divided into different categories as follows:
1. Test Management Tools

2. Functional Testing Tools

3. Non-Functional Testing Tools

◦ Performance/Load

25
 Test Management Tools
Open Source Tools Proprietary/Commercial Tools
• HP Quality Center/ALM
• TET (Test Environment Toolkit) • QA Complete
• TETware • T-Plan Professional
• Automated Test Designer (ATD)
• Test Manager
• Testuff
• RTH • SMARTS
• QAS.TCS (Test Case Studio)
• PractiTest
• Test Manager Adaptors
• SpiraTest
• TestLog
• ApTest Manager
• DevTest

26
Functional Testing Tools
Proprietary/Commercial Tools
Open Source Tools
• QuickTest Pro
• Selenium • Imprimatur • Rational Robot
• Sahi
• Soapui • SAMIE • SoapTest
• Watir • Swete • Badboy
• Test Complete
• HTTP::Recorder • ITP • QAWizard
• Netvantage Functional Tester
• WatiN • WET • PesterCat
• Canoo WebTest • WebInject • AppsWatch
• Squish
• Webcorder • Katalon Studio • actiWATE
• liSA
• Solex • vTest
• Internet Macros
• Ranorex

27
 Performance/Load Testing Tools
• Proprietary/Commercial Tools
Open Source Tools • WebLOAD Professional
◦ Jmeter • HP LoadRunner
◦ FunkLoad • LoadStorm
• NeoLoad
• Loadtracer
• Forecast
• ANTS – Advanced .NET Testing System
• vPerformer
• Webserver Stress Tool
• preVue-ASCII
• Load Impact

28
 1. Selenium
• It is the most widely used automated testing tool among all web application
testing tools.

• Selenium can be executed in multiple browsers and operating systems.

• It is compatible with several programming languages and automation

testing frameworks.

• With selenium, you can come up with very powerful browser- centered
automation test scripts which are scalable across different environments.

• You can also create scripts using Selenium that is of great help for prompt
reproduction of bugs, regression testing, and exploratory testing.
29
 2. Unified Functional Testing (UFT) QTP
◦ Unified Functional Testing (UFT) tool given by Hewlett-Packard
Enterprise is one of the best automation testing software for
functional testing.
It was previously known as QuickTest Professional (QTP).
◦ It brings developers & testers coming together under one umbrella
and provides high-quality automation testing solutions.
◦ It makes functional testing less complex and cost-friendly.
◦ Its top features include Cross browser & multi-platform
compatibility, Optimized distributed testing, multiple testing
solutions, image-based object recognition and canvas – visual test flows.

30
 3. H P Quality Center

•It is basically an integrated IT quality management

software.
•Automated testing is one of its key features which
constantly allows you to test earlier and quicker.
•Asset sharing and reusability allows QC to deliver bug-free
and reliable applications.

31

36
 4. Testim.io

◦It leverages machine learning for the authoring, execution, and

maintenance of automated test cases.

◦We use dynamic locators and learn with every execution.

◦The outcome is super fast authoring and stable tests that
learn, thus eliminating the need to continually maintain
tests with every code change.

32
 5.TestComplete

◦ It allows all level of users to quickly create powerful, reusable and time-saving

GUI automation tests for the web, mobile, and desktop applications.

◦ It lets you combine the recorded scripts and tests into a single framework
which reduces the training cost and testing time.

◦ The best part of the tool is TestComplete Visualizer, which is a screenshot

based feature allows you to modify previously recorded tests, quickly update
your assertions and checkpoint and provide a visual recording of your test – all
in one area.

33
 6. Telerik Test Studio
◦ Telerik test studio is a comprehensive test automation
solution.
◦ It is well suited for GUI, performance, load and API testing.
◦ It allows you to test desktop, mobile and web applications.
◦ Its main features include Point-and-click test recorder, support
for coding languages like C# and VB.NET, central object
repository and continuous integration with source control.

34

39
 7. Katalon Studio

• Katalon Studio is a powerful test automation solution for mobile, Web, and API
testing. And it is completely FREE!

• It provides a comprehensive set of features for test automation, including

recording actions, creating test cases, generating test scripts, executing tests,
reporting results, and integrating with many other tools in the software
development lifecycle.

• Katalon Studio runs on both Windows and MacOS, supporting automated testing of
iOS and Android apps, web applications on all modern browsers, and API services.

• It can integrate with tools such as JIRA, qTest, Kobiton, Git, and Slack.

35
????

36