6.

Testing Tools and Measurements

Q. State limitations of manual testing. Write any four.

(Any 4 appropriate limitations of manual testing - 4Marks; 1 Mark each)
Ans:
Limitations of Manual Testing are as given below:
i. Manual testing is slow and costly.
ii. It is very labor intensive; it takes a long time to complete tests.
iii. Manual tests don’t scale well. As the complexity of the software increases the
complexity of the testing problem grows exponentially. This leads to an increase in
total time devoted to testing as well as total cost of testing.
iv. Manual testing is not consistent or repeatable. Variations in how the tests are
performed as inevitable, for various reasons. One tester may approach and perform a
certain test differently from another, resulting in different results on the same test,
because the tests are not being performed identically.
v. Lack of training is the common problem, although not unique to manual software
testing.
vi. GUI objects size difference and color combinations are not easy to find in manual
testing.
vii. Not suitable for large scale projects and time bound projects.
Batch testing is not possible, for each and every test execution Human user
interaction is mandatory.
viii. Comparing large amount of data is impractical.
ix. Processing change requests during software maintenance takes more time.

Q. Which different benefits help to recommend automated testing? Write advantages of

switching to automated testing.(Need / Benefit – 2 Marks, Advantages – 2 Marks)
Ans:
NEED of automated testing
i. An automated testing tool is able to playback pre-recorded and predefined actions,
compare the results to the expected behavior and report the success or failure of these
manual tests to a test engineer.
ii. Once automated tests are created they can easily be repeated and they can be extended
to perform tasks impossible with manual testing.
iii. Because of this, savvy managers have found that automated software testing is an
essential component of successful development projects.

1) Automated Software Testing Saves Time and Money

i. Software tests have to be repeated often during development cycles to ensure quality.
Every time source code is modified software tests should be repeated.
ii. For each release of the software it may be tested on all supported operating systems
and hardware configurations.
iii. Manually repeating these tests is costly and time consuming. Once created, automated
tests can be run over and over again at no additional cost and they are much faster than
manual tests.
iv. Automated software testing can reduce the time to run repetitive tests from days to
hours.
v. A time savings that translates directly into cost savings.

STE Page 1
 6. Testing Tools and Measurements

2) Testing Improves Accuracy

i. Even the most conscientious tester will make mistakes during monotonous
manual testing.
ii. Automated tests perform the same steps precisely every time they are executed
and never forget to record detailed results.

3) Increase Test Coverage

i. Automated software testing can increase the depth and scope of tests to help
improve software quality.
ii. Lengthy tests that are often avoided during manual testing can be run
unattended.
iii. They can even be run on multiple computers with different configurations.
iv. Automated software testing can look inside an application and see memory
contents, data tables, file contents, and internal program states to determine if
the product is behaving as expected.
v. Automated software tests can easily execute thousands of different complex test
cases during every test run providing coverage that is impossible with manual
tests.
vi. Testers freed from repetitive manual tests have more time to create new
automated software tests and deal with complex features.

Q. Explain static and dynamic testing tools in details.

(Static tools-2 Marks, Dynamic tools- 2 Marks)
Ans:
Static testing tools are used during static analysis of a system. Static testing tools
are used throughout a software development life cycle, e.g , tools used for verification
purposes. There are many varieties of static testing tools used by different people as
per the type of system being developed.
Code complexity measurement tools can be used to measure the complexity of a
given code. Similarly, data-profiling tools can be used to optimize a database. Code-
profiling tools can be used to optimize code. Test-generators are used for generating a
test plan form code. Syntax-checking tools are used to verify correctness of code.
Dynamic testing tools are used at different levels of testing starting from unit
testing & which may go up to system testing & performance testing. These tools are
generally used by tester. There are many different tools used for dynamic testing.
Some of the areas covered by testing tools are:
1. Regression testing using automated tools.
2. Defect tracking and communication systems used by tracking &
communication. Performance, Load, stress-testing tools.

STE Page 2
 6. Testing Tools and Measurements

Q. Elaborate the advantages (any four) of using the test automation tools.
(Any four appropriate advantages of the test automation tools - 4 marks; 1 mark
each)
Ans: Advantages of using the test automation tools are as given below:
1. Speed. The automation tools tests the software under tests with the very faster
speed. There‘s a vast difference between the speed of user entering the data and
the automated tools generating and entering the data required for the testing of
the software. Speed of these software also completes the work faster.
2. Efficiency. While testers are busy running test cases, testers can't be doing
anything else. If the tester have a test tool that reduces the time it takes for him
to run his tests, he has more time for test planning and thinking up new tests.
3. Accuracy and Precision. After trying a few hundred cases, tester‘s attention
span will wane and he may start to make mistakes. A test tool will perform the
same test and check the results perfectly, each and every time.
4. Resource Reduction. Sometimes it can be physically impossible to perform a
certain test case. The number of people or the amount of equipment required to
create the test condition could be prohibitive. A test tool can be used to simulate
the real world and greatly reduce the physical resources necessary to perform
the testing.
5. Simulation and Emulation. Test tools are often used to replace hardware or
software that would normally interface to your product. This "fake" device or
application can then be used to drive or respond to your software in ways that
you choose and ways that might otherwise be difficult to achieve.
6. Relentlessness. Test tools and automation never tire or give up. they can keep
going and going and on and on without any problem; whereas the tester gets
tired to test again and again.

STE Page 3
 6. Testing Tools and Measurements

Q. Enlist any four benefits of automation testing.

(Any four benefits, Each benefit -1 Mark)
Ans:
1. Save Time /Speed: Due to advanced computing facilities, automation test tools
prevail in speed of processing the tests. Automation saves time as software can
execute test cases faster than human.
2. Reduces the tester’s involvement in executing tests: It relieves the testers to
do some other work.
3. Repeatability/Consistency: The same tests can be re-run in exactly the same
manner eliminating the risk of human errors such as testers forgetting their exact
actions, intentionally omitting steps from the test scripts, missing out steps from
the test script, all of which can result in either defects not being identified or the
reporting of invalid bugs (which can again, be time consuming for both
developers and testers to reproduce)
4. Simulated Testing: Automated tools can create many concurrent virtual
users/data and effectively test the project in the test environment before
releasing the product.
5. Test case design: Automated tools can be used to design test cases also.
Through automation, better coverage can be guaranteed than if done manually.
6. Reusable: The automated tests can be reused on different versions of the
software, even if the interface changes.
7. Avoids human mistakes: Manually executing the test cases may incorporate
errors. But this can be avoided in automation testing.
8. Internal Testing:Testing may require testing for memory leakage or checking
the coverage of testing. Automation can done this easily.
9. Cost Reduction:If testing time increases cost of the software also increases.
Due to testing tools time and therefore cost is reduced.

STE Page 4
 6. Testing Tools and Measurements

Q. Enlist factors considered for selecting a testing tool for test automation.
(Any four Factors- 4 Marks)
[Note: Criteria or guidelines can be considered as an answer]
Ans:
Criteria for Selecting Test Tools:
The Criteria's for selecting Test Tools are,
2. Meeting requirements;
3. Technology expectations;
4. Training/skills;
5. Management aspects.
1. Meeting requirements-
There are plenty of tools available in the market but rarely do they meet all the
requirements of a given product or a given organization. Evaluating different
tools for different requirements involve significant effort, money, and time.
Given of the plethora of choice available, huge delay is involved in selecting
and implementing test tools.
2. Technology expectations-
Test tools in general may not allow test developers to extends/modify the
functionality of the framework. So extending the functionality requires going
back to the tool vendor and involves additional cost and effort. A good number
of test tools require their libraries to be linked with product binaries.
3. Training/skills-
While test tools require plenty of training, very few vendors provide the training
to the required level. Organization level training is needed to deploy the test
tools, as the user of the test suite are not only the test team but also the
development team and other areas like configuration management.
4. Management aspects-
A test tool increases the system requirement and requires the hardware and
software to be upgraded. This increases the cost of the already- expensive test
tool.
OR

Guidelines for selecting a tool:

1. The tool must match its intended use. Wrong selection of a tool can lead to
problems like lower efficiency and effectiveness of testing may be lost.
2. Different phases of a life cycle have different quality-factor requirements. Tools
required at each stage may differ significantly.
3. Matching a tool with the skills of testers is also essential. If the testers do not
have proper training and skill then they may not be able to work effectively.
4. Select affordable tools. Cost and benefits of various tools must be compared
before making final decision.
5. Backdoor entry of tools must be prevented. Unauthorized entry results into
failure of tool and creates a negative environment for new tool introduction.

STE Page 5
 6. Testing Tools and Measurements

Q. How to select a testing tool? Explain in detail.

(Explanation – 4 Marks)
[Note: Criteria or guidelines can be considered as an answer]
Ans: Criteria for Selecting Test Tools:
The Categories for selecting Test Tools are,
1. Meeting requirements;
2. Technology expectations;
3. Training/skills;
4. Management aspects.

1. Meeting requirements-
There are plenty of tools available in the market but rarely do they meet all the
requirements of a given product or a given organization. Evaluating different
tools for different requirements involve significant effort, money, and time.
Given of the plethora of choice available, huge delay is involved in selecting
and implementing test tools.
2. Technology expectations-
Test tools in general may not allow test developers to extends/modify the
functionality of the framework. So extending the functionality requires going
back to the tool vendor and involves additional cost and effort. A good number
of test tools require their libraries to be linked with product binaries.
3. Training/skills-
While test tools require plenty of training, very few vendors provide the training
to the required level. Organization level training is needed to deploy the test
tools, as the user of the test suite are not only the test team but also the
development team and other areas like configuration management.
4. Management aspects-
A test tool increases the system requirement and requires the hardware and
software to be upgraded. This increases the cost of the already- expensive test
tool.
OR
Guidelines for selecting a tool:
1. The tool must match its intended use. Wrong selection of a tool can lead to
problems like lower efficiency and effectiveness of testing may be lost.
2. Different phases of a life cycle have different quality-factor requirements. Tools
required at each stage may differ significantly.
3. Matching a tool with the skills of testers is also essential. If the testers do not
have proper training and skill then they may not be able to work effectively.
4. Select affordable tools. Cost and benefits of various tools must be compared
before making final decision.
5. Backdoor entry of tools must be prevented. Unauthorized entry results into
failure of tool and creates a negative environment for new tool introduction.

STE Page 6
 6. Testing Tools and Measurements

Q. Which are features for selecting static test tools? Also list any two available
test tools (static). (Any 3 features for selecting static test tools - 3 Marks, listing of
two available tools - 1 Mark)
Ans:
Features for selecting static test tools:
i. Assessment of the organization’s maturity (e.g. readiness for change);
ii. Identification of the areas within the organization where tool support will
help to improve testing processes;
iii. Evaluation of tools against clear requirements and objective criteria;
iv. Proof-of-concept to see whether the product works as desired and meets the
requirements and objectives defined for it;
v. Evaluation of the vendor (training, support and other commercial aspects) or
open-source network of support;
vi. Identifying and planning internal implementation (including coaching and
mentoring for those new to the use of the tool).
Available static test tools are:
1. code coverage analyzer
2. Interface Analyzer

Q. Which types of test are first candidates for test automation? Why?
(Stating candidates - 1 mark; explanation - 3 marks)
Ans.
Stress, reliability, scalability and performance testing:
These types of testing require the test case to be run from a large number of different
machines for an extended period of time, such as 24 hours, 48 hours, and so on. It is
just not possible to have hundreds of users trying out the product they may be not
willing to perform the repetitive tasks, nor will it be possible to find that many people
with the required skill sets. Test cases belonging to these testing types become the first
candidates for automation.
Regression tests:
Regression tests are repetitive in nature .These test cases are executed multiple times
during the product development phase. Given the repetitive nature of test cases,
automation will save significant time and effort in the long run. The time thus gained
can be effectively utilized for other tests.
Functional tests:
These kinds of tests may require a complex set up and thus require specialized skill,
which may not be available on an ongoing basis. Automating these once, using the
expert skill sets, can enable using less-skilled people to run these test on an ongoing
basis.

STE Page 7
 6. Testing Tools and Measurements

Q. What is software test automation? State types of test automation tools.

(Test automation - 2 marks; Types – 2 marks)
Ans:
Test automation is the use of special software to control the execution of tests and the
comparison of actual outcomes with predicted outcomes. The objective of automated
testing is to simplify as much of the testing effort as possible with a minimum set of
scripts. Test automation can automate some repetitive but necessary tasks in a
formalized testing process already in place, or add additional testing that would be
difficult to perform manually.
Types of test automation tools:
 Static automation tools: These tools do not involve actual input and output.
Rather, they take a symbolic approach to testing, i.e. they do not test the actual
execution of the software. e.g. Flow analyzers, Coverage analyzers, Interface
analyzer
 Dynamic automation tools: These tools test the software system with live data.
e.g. Test driver, Test beds, Emulators

Q. Define metrics and measurements. Explain need of software measurement.

(Definition-2 Marks, Any two Need-2 Marks)
Ans:
Metrics & measurement:
Metrics is a relative measurement of status of process or product in terms of two or
more entities taken together for comparison.
Measurements are key element for controlling software engineering processes.

Need of software measurements:

1. Understanding: Metrics can help in making the aspects of process more visible,
thereby giving a better understanding of the relationship among the activities
and entities they affect.
2. Control: Using baselines, goals and an understanding of the relationships, we
can predict what is likely to happen and correspondingly, make appropriate
changes in the process to help meet the goals.
3. Improvement: By taking corrective actions and making appropriate changes,
we can improve a product. Similarly, based on the analysis of a project, a
process can also be improved.

STE Page 8
 6. Testing Tools and Measurements

Q. State the different metrics types with its classification.

(Stating metrics - 1 mark; Classification with explanation - 3 marks)
OR
Q. State the different metrics types with its classification.
(Stating metrics - 1 mark; Classification with explanation - 3 marks)
Ans:

Metrics are basically classified as:

1. Product Metrics: Product metrics are measures of software product at any stage of its
development, from requirements to installed system.
2. Process Metrics: Process metrics are measures of the software development process such
as the overall development time, type of methodology used or the average level of
experience of the programming staff.

Product Metrics is classified as

 Project Metrics: A set of metrics that indicates how the project is planned and executed.
 Progress: A set of metrics that tracks how the different activities of the project are
progressing.
Progress Metrics is classified as 1. Test defect metrics 2. Development defect metrics
1. Test defect metrics: help the testing team in analysis of product quality and testing
2. Development defect metrics: help the development team in analysis of development
activities.
 Productivity: A set of metrics that takes into account various productivity numbers that
can be collected and used for planning and tracking testing activities.

OR

Other type of classification is:

1. Product vs. Process Metrics
2. Objective vs. Subjective Metrics
3. Primitive vs. Computed Metrics
4. Private vs. Public Metrics

STE Page 9