Question Answers

Module 4
1 mark

1. What is Test Plan?

A Test Plan is a detailed document that describes the test strategy, objectives,
schedule, estimation, deliverables, and resources required to perform testing for a
software product.

2. What is Test Management?

Test management, process of managing the tests. Test management tools allow
automatic generation of the requirement test matrix (RTM), which is an indication of
functional coverage of the application under test (SUT).
3. What is test report?
Test Report is a document which contains a summary of all test activities and final
test results of a testing project.

4. What is Test Automation?

Software Test automation makes use of specialized tools to control the execution of
tests and compares the actual results against the expected result.
2 marks

1. What are Test Management Responsibilities?

1. Test Management has a clear set of roles and responsibilities for improving the
quality of the product.

2. Test management helps the development and maintenance of product metrics

during the course of project.

3. Test management enables developers to make sure that there are fewer design or
coding faults.
2. What are the Criteria for Tool Selection in automation?
For automating any application, the following parameters should be considered:
1. Data driven capabilities
2. Debugging and logging capabilities
3. Platform independence
4. Extensibility and Customizability
 5. E-mail Notifications
6. Version control friendly
7. Support unattended test runs
4 marks

1. What does a test report contain?

1. Project Information
All information of the project such as the project name, product name, and version
should be described in the test report.

2. Test Objective
As mentioned in Test Planning tutorial, Test Report should include the objective
of each round of testing, such as Unit Test, Performance Test, System Test …Etc.

3. Test Summary
This section includes the summary of testing activity in general. Information
detailed here includes

1. The number of test cases executed

2. The numbers of test cases pass
3. The numbers of test cases fail
4. Pass percentage
5. Fail percentage
6. Comments

This information should be displayed visually by using color

indicator, graph, and highlighted table.

4. Defect
One of the most important information in Test Report is defect. The report should
contain following information

1. Total number of bugs

2. Status of bugs (open, closed, responding)
3. Number of bugs open, resolved, closed
4. Breakdown by severity and priority

2. What are the basic types of test case reporting?

There are three basic types of test case reporting in software testing:
1. Test incident report
2. Test cycle report
3. Test summary report
Test incident report gives a bird’s eye view of the project, including issues or any
incident that occurs during the life cycle; test cycle report gives information about a
certain test that is run; and thirdly, test summary report summarizes everything related
to the product cycle and explains the stability of the product.
1. Test Incident Report

This report communicates the defects or issues as they occur during the life
cycle. Defect and issue management involve multiple people from various teams; for
instance, the nature of the bug or issue may require the tester to get in touch with the
developers, or there may be issues in deployment, or simply a technical flaw in the
design and propagation. This is the stage where the report is generated regarding these
issues and bugs to take the relevant teams on board.

The issues most often refer to bugs but that is not always the case. Any incident that
may occur unexpectedly during the testing life cycle is reported in the Test Incident
Report. The defects or incidents reported in this report are added to a repository with a
unique ID for an easier inquiry into the issue. Here, the issues and incidents are
categorized according to the priority as well. The high priority ones are highlighted
for preferred redressal. It is advisable to add the persons assigned with the task in the
incident report as well, for better management.

Besides the obvious objective of this report, which is to highlight all the bugs and
issues, and hence, take consequent action, this process also cultivates transparency
amongst the software teams and enhances correspondence between them. This
atmosphere of transparency leads to lesser chances of miscommunication between the
team members and other participants involved in the project.

2. Test Cycle Report

Test Cycle Report refers to running various tests in the testing life cycle. It includes
the planning and execution of certain tests, the priority of those tests, and the severity
of issues to be addressed. Test Cycle Report is generated for each cycle and a separate
cycle is executed for each separate build of the product. The idea is to run multiple
cycles on a product if needed, and to ensure that the product emerges stable and
error-free at the end of the rigorous test cycles.

The Test Cycle Report ideally explains an overall situation of the cycle with the
various defects that emerged during the cycle, their severity, and the impact those
defects had on the overall product test cycle. It should also be able to bridge the
previous cycles with the latest in an attempt to explain the flow of events. An
important thing about a Test Cycle Report is that it entails the new defects and
apprehends the issues that may arise as the product sets on to maturity. All of the
issues that surfaced but were not addressed in the latest cycle are also included in
 the Test Cycle Report so that the redressal gets picked up right where the last cycle
ended.

3. Test Summary Report

The test summary report is the crux of the test cycle that recommends that
the product is mature and ready for release. It can be considered as the final word
regarding the entire cycle, finishing it with a conclusive remark. It indicates how
effectively the testing project was carried out

A test summary report can be generated at the end of each phase of the cycle
separately, as well as at the end of the entire life cycle of the product. A phase-specific
test summary report helps identify the targeted areas in the product life cycle for a
referral. A conclusive test summary is a certificate of fitness for the product.

This type of report is usually prepared by the Test Manager. It should be crisp and to
the point—with all necessary details included.

The test summary report should be able to explain the test items with their IDs, the
variations that may have occurred during the cycle and how they were
resolved, and a clear-cut summary of all the results including their assessment. It
should distinctly mention that the product is fit for release along with a formal
recommendation for product release.

3. What Types of Frameworks used in test automation?

Typically, there are 4 test automation frameworks that are adopted while automating the
applications:

1. Data Driven Automation Framework

2. Keyword Driven Automation Framework

3. Modular Automation Framework

4. Hybrid Automation Framework

Popular Tools that are used for Functional automation

Product Vendor URL

Quick Test HP www.hp.com/go/qtp

Professional

Rational Robot IBM http://www-03.ibm.com/software/products/us/e

n/robot/
 Coded UI Microsoft http://msdn.microsoft.com/en-us/library/dd286
726.aspx

Selenium Open http://docs.seleniumhq.org/

Source

Auto IT Open http://www.autoitscript.com/site/

Source
Popular Tools that are used for Non Functional automation

Product Vendor URL

Load Runner HP www.hp.com/go/LoadRunner

Jmeter Apache jmeter.apache.org/

Burp Suite PortSwigger http://portswigger.net/burp/

Acunetix Acunetix http://www.acunetix.com/

4. What are Software Testing Tools?

Software testing tools are required for the betterment of the application or software. That's
why we have so many tools available in the market where some are open-source and paid
tools.

The significant difference between open-source and the paid tool is that the open-source tools
have limited features, whereas paid tool or commercial tools have no limitation for the
features. The selection of tools depends on the user's requirements, whether it is paid or free.

The software testing tools can be categorized, depending on the licensing (paid or
commercial, open-source), technology usage, type of testing, and so on.

With the help of testing tools, we can improve our software performance, deliver a
high-quality product, and reduce the duration of testing, which is spent on manual efforts.

The software testing tools can be divided into the following:

1. Test management tool

2. Bug tracking tool
3. Automated testing tool
4. Performance testing tool
5. Cross-browser testing tool
 6. Integration testing tool
7. Unit testing tool
8. Mobile/android testing tool
9. GUI testing tool
10. Security testing tool

Test management tool

Test management tools are used to keep track of all the testing activity, fast data analysis,
manage manual and automation test cases, various environments, and plan and
maintain manual testing as well.

Bug tracking tool

The defect tracking tool is used to keep track of the bug fixes and ensure the delivery of a
quality product. This tool can help us to find the bugs in the testing stage so that we can get
the defect-free data in the production server. With the help of these tools, the end-users can
allow reporting the bugs and issues directly on their applications.

Automation testing tool

This type of tool is used to enhance the productivity of the product and improve the accuracy.
We can reduce the time and cost of the application by writing some test scripts in any
programming language.

Performance testing tool

Performance or Load testing tools are used to check the load, stability, and scalability of the
application. When n-number of the users using the application at the same time, and if the
application gets crashed because of the immense load, to get through this type of issue, we
need load testing tools.

Cross-browser testing tool

This type of tool is used when we need to compare a web application in the various web
browser platforms. It is an important part when we are developing a project. With the help of
these tools, we will ensure the consistent behavior of the application in multiple devices,
browsers, and platforms.

Integration testing tool

This type of tool is used to test the interface between modules and find the critical bugs that
are happened because of the different modules and ensuring that all the modules are working
as per the client requirements.

Unit testing tool

This testing tool is used to help the programmers to improve their code quality, and with the
help of these tools, they can reduce the time of code and the overall cost of the software.

Mobile/android testing tool

We can use this type of tool when we are testing any mobile application. Some of the tools
are open-source, and some of the tools are licensed. Each tool has its functionality and
features.

GUI testing tool

GUI testing tool is used to test the User interface of the application because a proper GUI
(graphical user interface) is always useful to grab the user's attention. These type of tools will
help to find the loopholes in the application's design and makes its better.

Security testing tool

The security testing tool is used to ensure the security of the software and check for the
security leakage. If any security loophole is there, it could be fixed at the early stage of the
product. We need this type of the tool when the software has encoded the security code which
is not accessible by the unauthorized users.

15 Marks

1. How to write a Test Plan.

You already know that making a Test Plan is the most important task of Test
Management Process. Follow the seven steps below to create a test plan as per IEEE 829

1. Analyze the product

2. Design the Test Strategy
3. Define the Test Objectives
4. Define Test Criteria
5. Resource Planning
6. Plan Test Environment
7. Schedule and Estimation
8. Determine Test Deliverables
Step 1) Analyze the product
How can you test a product without any information about it? The answer
is Impossible. You must learn a product thoroughly before testing it.

You should research clients and the end users to know their needs and expectations from
the application

1. Who will use the website?

2. What is it used for?
3. How will it work?
4. What are software/ hardware the product uses?

Step 2) Design the Test Strategy

Test Strategy is a critical step in making a Test Plan in Software Testing. A Test Strategy
document, is a high-level document, which is usually developed by Test Manager. This
document defines:

1. The project’s testing objectives and the means to achieve them

2. Determines testing effort and costs

Step 3) Define Test Objective

Test Objective is the overall goal and achievement of the test execution. The objective of
the testing is finding as many software defects as possible; ensure that the software under
test is bug free before release.

To define the test objectives, you should do 2 following steps

1. List all the software features (functionality, performance, GUI…) which may need
to test.
2. Define the target or the goal of the test based on above features

You can choose the ‘TOP-DOWN’ method to find the website’s features which may need
to test. In this method, you break down the application under test
to component and sub-component.

Step 4) Define Test Criteria

Test Criteria is a standard or rule on which a test procedure or test judgment can be based.
There’re 2 types of test criteria as following

Suspension Criteria

Specify the critical suspension criteria for a test. If the suspension criteria are met during
testing, the active test cycle will be suspended until the criteria are resolved.
Test Plan Example: If your team members report that there are 40% of test cases failed,
you should suspend testing until the development team fixes all the failed cases.
Exit Criteria
It specifies the criteria that denote a successful completion of a test phase. The exit
criteria are the targeted results of the test and are necessary before proceeding to the next
phase of development. Example: 95% of all critical test cases must pass.

Step 5) Resource Planning

Resource plan is a detailed summary of all types of resources required to complete

project task. Resource could be human, system needed to complete a project

The resource planning is important factor of the test planning because helps
in determining the number of resources (employee, system…) to be used for the project.
Therefore, the Test Manager can make the correct schedule and estimation for the project.

This section represents the recommended resources for your project.

Human Resource

The following table represents various members in your project team

No. Member Tasks

Manage the whole project

1. Test Manager Define project directions
Acquire appropriate resources
Identifying and describing appropriate test
techniques/tools/automation architecture
Verify and assess the Test Approach
Execute the tests, Log results, Report the
defects.
2. Tester
Tester could be in-sourced or out-sourced
members, base on the project budget
For the task which required low skill, I
recommend you choose outsourced members
to save project cost.
Developer in Implement the test cases, test program, test
3.
Test suite etc.
Test Builds up and ensures Test Environment and
4.
Administrator assets are managed and maintained
 Support Tester to use the test environment for
test execution
Take in charge of quality assurance
SQA
5. Check to confirm whether the testing process is
members
meeting specified requirements

System Resource
For testing, a web application, you should plan the resources as following tables:

Resource
No. Descriptions
s

Install the web application under test

1. Server
This includes a separate web server, database server,
and application server if applicable
The testing tool is to automate the testing, simulate
the user operation, generate the test results
2. Test tool
There are tons of test tools you can use for this
project such as Selenium, QTP…etc.
You need a Network include LAN and Internet to
3. Network
simulate the real business and user environment
The PC which users often use to connect the web
4. Computer
server

Step 6) Plan Test Environment

What is the Test Environment
A testing environment is a setup of software and hardware on which the testing team is
going to execute test cases.

How to setup the Test Environment

To finish this task, you need a strong cooperation between Test Team and Development
Team

You should ask the developer some questions to understand the web application under
test clearly. Here’re some recommended questions. Of course, you can ask the other
questions if you need.

1. What is the maximum user connection which this website can handle at the same
time?
 2. What are hardware/software requirements to install this website?
3. Does the user’s computer need any particular setting to browse the website?

Step 7) Schedule and Estimation

In the Test Estimation phase, suppose you break out the whole project into small tasks
and add the estimation for each task as below

Estimate
Task Members
effort

170
Create the test specification Test Designer
man-hour
Perform Test Execution Tester, Test Administrator 80 man-hour
Test Report Tester 10 man-hour
Test Delivery 20 man-hour
280
Total
man-hour
Then you create the schedule to complete these tasks.

Making schedule is a common term in project management. By creating a solid schedule

in the Test Planning, the Test Manager can use it as tool for monitoring the project
progress, control the cost overruns.

To create the project schedule, the Test Manager needs several types of input as below:

1. Employee and project deadline: The working days, the project deadline,
resource availability are the factors which affected to the schedule
2. Project estimation: Base on the estimation, the Test Manager knows how long it
takes to complete the project. So he can make the appropriate project schedule
3. Project Risk : Understanding the risk helps Test Manager add enough extra time
to the project schedule to deal with the risks

Step 8) Determine Test Deliverables

Test Deliverables is a list of all the documents, tools and other components that has to be
developed and maintained in support of the testing effort.

There are different test deliverables at every phase of the software development lifecycle.

Test deliverables are provided before testing phase.

1. Test plans document.

2. Test cases documents
3. Test Design specifications.
 Test deliverables are provided during the testing
1. Test Scripts
2. Simulators.
3. Test Data
4. Test Traceability Matrix
5. Error logs and execution logs.

Test deliverables are provided after the testing cycles is over.

1. Test Results/reports
2. Defect Report
3. Installation/ Test procedures guidelines
4. Release notes

2. What are the basic steps of testing?

The activities of testing can be divided into the following basic steps:

1. Planning and Control

2. Analysis and Design
3. Implementation and Execution
4. Evaluating exit criteria and Reporting
5. Test Closure activities

1) Planning and Control

Test Planning

Test planning involves producing a document that describes an overall approach and
test objectives. It involves reviewing the test basis, identifying the test conditions
based on analysis of test items, writing test cases and Designing the test environment.
Completion or exit criteria must be specified so that we know when testing (at any
stage) is complete.
Purpose

1. To determine the scope and risks and identify the objectives of testing.
2. To determine the required test resources like people, test environments etc.
3. To schedule test analysis and design tasks, test implementation, execution and
evaluation.

Control
This is the activity of comparing actual progress against the plan, and reporting the
status, including deviations from the plan. It involves taking actions necessary to meet
the mission and objectives of the project.

2) Analysis and Design

Test analysis and Test Design has the following major tasks:

1. To review the test basis. The test basis is the information on which test cases
are based, such as requirements, design specifications, product risk
analysis, architecture and interfaces
2. To identify test conditions
3. To design the tests
4. To design the test environment set-up and identify the required infrastructure
and tools

3) Implementation and Execution

Test execution involves actually running the specified test on a computer system
either manually or by using an automated test tool. It is a Fundamental Test Process in
which actual work is done.

Test implementation has the following major task:

1. To develop and prioritize test cases by using techniques and create test data for
those tests.
2. To create test suites from the test cases for efficient test execution. Test suite is
a collection of test cases that are used to test a software program
3. To re-execute the tests that previously failed in order to confirm a fix.
4. To log the outcome of the test execution. A test log is the status of the test case
(pass/fail).
5. To compare actual results with expected results.

4) Evaluating Exit criteria and Reporting

Evaluating exit criteria is a process defining when to stop testing. It depends on
coverage of code, functionality or risk. Basically it also depends on business risk, cost
and time and vary from project to project. Exit criteria come into picture, when:

1. Maximum test cases are executed with certain pass percentage

2. Bug rate falls below certain level
3. When we achieve the deadlines

Evaluating exit criteria has the following major tasks:

 1. To assess if more test are needed or if the exit criteria specified should be
changed
2. To write a test summary report for stakeholders

5) Test Closure activities:

Test closure activities are done when software is ready to be delivered. The testing
can be closed for the other reasons also like:

1. When a project is cancelled

2. When some target is achieved
3. When a maintenance release or update is done

Test closure activities have the following major tasks:

1. To check which planned deliverables are actually delivered and to ensure that
all incident reports have been resolved
2. To finalize and archive testware such as scripts, test environments, etc. for
later reuse
3. To handover the testware to the maintenance organization. They will give
support to the software
4. To evaluate how the testing went and learn lessons for future releases and
projects