6.

Software Testing
6.1. Software Testing Fundamentals
6.1.1. Causes of Software Defects
6.1.2. Seven Testing principles
6.1.3. Fundamental Test Process
6.1.4. Testing Throughout the Software Lifecycle
6.1.5. Test Levels
6.1.6. Test Types
6.1.7. Tool Support for Testing
6.1.8. Test Tool Classification
6.1.9. The Psychology of Testing
6.1.10. Code of Ethics
6.2. Test Techniques
6.2.1. Categories of Test Design Techniques
6.2.2. Specification-based or Black-box Techniques
6.2.3. Structure-based or White-box Techniques
6.2.4. Experience-based Techniques
6.3. Defect Management
6.3.1. Reporting Defects
6.3.2. Defect Life cycle
6.3.3. Severity and Priority
6.4. Test Management
6.4.1. Test Organization and Independence
6.4.2. Tasks of the Test Leader and Tester
6.4.3. Test Planning and Estimation
6.4.4. Strategy and Approach
6.4.5. Test Progress Monitoring and Control
6.4.6. Risk and Testing
6.5. Testing Tools
6.5.1. Manual and Automation testing
6.5.2. Automation Testing Tools
6.5.3. Selenium IDE
6.5.3.1. Introduction to Selenium IDE
6.5.3.2. Creating your First Selenium IDE script
6.5.3.3 Selenium Commands Selenese
6.5.3.4. Locators in Selenium IDE
6.5.3.5. Selenium IDE Troubleshooting

6. Software Testing

6.1. Software Testing Fundamentals

Testing is the process of evaluating a system or its component(s) with the intent to find whether it
satisfies the specified requirements or not. In simple words, testing is executing a system in order to
identify any gaps, errors, or missing requirements in contrary to the actual requirements.
Software testing is often used in association with the terms verification and validation.
Validation: Are we doing the right job?
Verification: Are we doing the job right?
Verification is the checking or testing of items, including software, for conformance and
consistency with an associated specification.
Validation is the process of checking that what has been specified is what the user actually wanted.
6.1.1. Causes of Software Defects
The human makes an error (called a mistake) and it is done unintentionally. The mistake introduces
a defect, sometimes called a fault or bug in the code or in a document.
If a defect in code is executed, the system may produce unintended results, causing a failure.
Error
An error is a human mistake which can lead to a chain of other negative effects. The reason for the
mistake could for example be:

time pressure - tight deadlines in the project

complex code - code constructions difficult to understand and maintain
changing technologies - moving from Java to .Net development
many system interactions - large scale systems with many dependencies to other systems

Defect
A defect in software, systems or documents is a result of an error. Defects are also known as faults.
Failure
A failure is defined as a deviation of the component or system from its expected delivery, service or
result. There are many possible causes of a failure, such as:

Software defect
Hardware malfunction
Environmental conditions, for example radiation or magnetic interference

6.1.2. Seven Testing principles

1. Testing shows presence of defects
Testing can never prove correctness. It can show that defects are present, but we cannot prove that
we have found all of them.
This mean that you must think about what you would like to focus the testing on and how it should
be carried because time and resources are limited.
2. Exhaustive testing is impossible
Testing everything (all combinations of inputs and preconditions) is called exhaustive testing. In
practice, it is not possible to do exhaustive testing. Most systems are too complex for this to be
possible in any realistic time frame. Instead, we use risk analysis and priorities to focus testing
efforts according to objectives.
3. Early testing
Testing activities should start as early as possible in the software or system development life cycle.
The testing shall be focused on defined objectives. The sooner testing begins, the easier it is to find
defects and cheaper to fix them.
4. Defect clustering
Defect Clustering in Software Testing means that the majority of the defects are caused by a small
number of modules, i.e. the distribution of defects are not across the application but rather
centralized in limited sections of the application.
5. Pesticide paradox
If the same tests are repeated over and over again, eventually the same set of test cases will no
longer find any new defects. To overcome this paradox, the test cases need to be regularly reviewed
and revised, and new and different tests need to be written to exercise different parts of the software
or system to potentially find more defects.
6. Testing is context dependent
Testing is executed differently in different contexts. For example, safety-critical software is tested
differently from software that is less critical.
7. Absence of errors fallacy
Finding and fixing defects does not help if the system built is unusable and does not fulfil the users
needs and expectations.
6.1.3.

Fundamental Test Process

The most visible part of testing is test execution but there are other activities as well. Test plans
should also include time to be spent on planning the tests, designing test cases, preparing for

execution and evaluating results to be as effective and efficient as possible.

The fundamental test process consist of the following main activities:

planning and control

analysis and design
implementation and execution
evaluating exit criteria and reporting
test closure activities

The activities in the process may overlap and take places concurrently. Tailoring these activities
within the context of the system and the project is usually required.
6.1.4. Testing Throughout the Software Lifecycle
What are the SDLC Phases? Software
Development Life Cycle, or SDLC is a process
used to develop software. There are different
stages or phases within the Software
Development Life Cycle and in each phase,
different activities take place.
SDLC creates a structure for the development
teams to be able to design, create and deliver
high quality software by defining various tasks
that need to happen. The life cycle defines a
methodology for improving the quality of
software and the overall development process.
The methodology within the SDLC process can vary across organizations, but standards such as
ISO/IEC 12207 represent processes that establish a life cycle for software, and provide a standard
for building and maintaining software.
The intent of a SDLC process it to help produce a product that is cost-efficient, effective, and of
high quality.
Some general rules apply to good testing in any kind of software development model:

All development activities have a corresponding testing activity

Each test level has test objectives specific to that level
The analysis and design of tests for a test level begins during the corresponding
development level
Testers should review development documents for each level as soon as drafts are
available

Test levels can be combined or reorganized depending on the nature of the project or the system
architecture.

6.1.5. Test Levels

Component Testing
Component testing, also known as module or unit testing, is the process of searching for defects and
verify functionality in the smallest separately testable part of a software system. A unit could for
example be a software module, object or a class.
Component testing may include functional, non-functional (e.g. finding memory leaks) and
structural testing (e.g. statement and decision coverage) and my be done separately from the system
using stubs, simulators or drivers.
Integration Testing
Integration testing aims at tests of interfaces between components and systems as well as
interactions with different parts of a system, e.g. operating system, file system and hardware.
There may be more than one level of integration testing e.g. integration between components within
a system or integration between two or more separate systems e.g. business process may involve a
series of systems.
Integration strategies may be based on the system architecture (such as top-down and bottom-up),
functional tasks or transaction processing sequences.
Integration testing can be functional, non-functional and structural (white-box testing).
System Testing
System testing is concerned with the behaviour of a whole system or product. The scope of system
testing shall be clearly documented in the test plan related to the system test level.
In system testing, the test environment should correspond to the final target or production
environment as much as possible in order to minimize the risk of environment-specific failures not
being found in testing.
The purpose of system testing is to investigate functional and non-functional behavior.
Acceptance Testing
The goal of acceptance testing is to establish confidence in the system, either the whole system or
specific non-functional characteristics of the system. Finding defects is not the main focus.
Acceptance testing may assess if the system is ready for deployment and use, although it is not
necessarily the final level of testing. For example, a large-scale system integration test may come
after the acceptance test for a system.

User acceptance testing is a typical form of acceptance test. This to verify the fitness of the system
by real business users.
Contract acceptance testing is performed against a contract's acceptance criteria for producing
custom-developed software.
Regulation acceptance testing is performed against any regulations that must be adhered to, such
as governmental, legal or safety regulations.
Alpha testing is performed at the developing organization's site. Beta testing or field testing, is
performed by people at their own locations. Both are tested by customers or potential customers,
not the developers of the product.
6.1.6. Test Types
Testing of Function
The functions are "what" the system does. Functional tests are based on functions and features and
their interoperability with specific systems.
Functional testing considers how the software behaves during normal usage for a user i.e. black box
testing.
Testing of Non-functional Software Characteristics
Non-functional testing is when we test the software characteristics, i.e. "how" the system works.
These test are broader and could include:
Usability
Maintainability
Portability
Reliability
Security
Performance
Scalability
Localization
Non-functional testing may be performed at all test levels. It considers the external behaviour and
the most common test design technique is black-box testing.
Load Testing
It is a process of testing the behavior of a software by applying maximum load in terms of software
accessing and manipulating large input data. It can be done at both normal and peak load
conditions. This type of testing identifies the maximum capacity of software and its behavior at
peak time.

Stress Testing
Stress testing includes testing the behavior of a software under abnormal conditions. For example, it
may include taking away some resources or applying a load beyond the actual load limit.
The aim of stress testing is to test the software by applying the load to the system and taking over
the resources used by the software to identify the breaking point.

Testing of Software Structure/Architecture

Structural testing is when we target the overall software architecture. It is referred to as white-box
testing.
Structural testing helps measure the thoroughness of testing by investigating the coverage of a type
of structure. It means studying the system and asking: "How much of this structure has been
tested?"
Testing Related to Changes: Re-testing and Regression Testing
Re- testing
When testing uncovers a defect in software, a developer is assigned to fix it. The action of locating
and fixing the defect is called debugging. It is a development activity, not a testing activity.
When the developer has fixed the bug, the software should be retested exactly as before, to confirm
that the defect has been successfully removed. This is called re-testing.
Regression Testing
Regression testing is performed after a change or modification to already tested system. The
purpose of regression tests is to repeat testing of an already tested program to confirm that no new
defects have been introduced or revealed in related or unrelated software areas.
The extent of regression testing is based on the risk not finding any defects in software working
previously. It can be performed on all test levels and include all the test types mentioned earlier.
6.1.7. Tool Support for Testing
The Meaning and Purpose of Tool Support for Testing
Test tools can be used for several different activities that supports daily work for a tester. Example
of activities are:

Execution tools, test data generation tools and result comparison tools are examples of
tools that are used directly for testing.
Test management tools used to manage tests, results, data, requirements, incidents,
defects etc. They could also support reporting and monitoring of test execution.
Reconnaissance tools used for exploration, for example, monitor the file activity for an

application.
Common tools that aids the testing, e.g. a spreadsheet that contains information about
the tests or testing.

Tool support for testing can have one or more of the following purposes depending on the context:

Improve efficiency of test activities by automating repetitive tasks or supporting manual

test activities, e.g. planning, design, reporting and monitoring
Automate activities that require significant resources when carried out manually, e.g.
static testing
Automate activities that can not be executed manually, e.g. large scale performance
testing
Increase reliability of testing, e.g. automating large data comparisons or simulating
behaviour

6.1.8. Test Tool Classification

Tool support for Management of Testing and Tests
Test Management Tools
These tools provide interfaces for executing tests, tracking defects and managing requirements
along with support for quantitative analysis and reporting of the test objects.
They also support tracing the test objects to requirement specifications and might have an
independent version control capability or an interface to an external one.
Requirement Management Tools
Requirement management tools store requirement statements, related attributes (e.g. priority) and
provide unique identifiers to the requirements. They can also give warning if there are
inconsistencies or undefined (missing) requirements.
Requirement management tools enable individual tests to be traceable to requirements, functions
and/or features.
Incident Management Tools
Incident management tools store and manage incident reports, i.e. defects, failures, change requests
or perceived problems and anomalies, and help in managing the life cycle of incidents. Example of
supported functionality are:

Prioritization of incidents

Assigning actions to people

Tracking incident status (e.g. rejected, ready to be tested or deferred to next release)

These tools often provide support for statistical analysis.

Configuration Management Tools

Configuration management tools are not strictly testing tools but are necessary for storage and
version management of testware and related software. These tools enable traceability between
testware, software work products and product variants.
They are particularly useful when developing on more than one configuration of the
hardware/software environment (e.g. for different operating system versions, libraries, compilers or
browsers).

Tool Support for Static Testing

Review Tools
Review tools assist the review processes, checklists and review guidelines. They are used to store
and communicate review comments, reports on defects and efforts.
They may also support online reviews, which is useful if the team is geographically dispersed.
Static Analysis Tool
Static Analysis Tools support developers and testers in finding defects prior to dynamic testing.
The static analysis tools may support:

Enforcement of coding standards

Analysis of structures and dependencies in the code

Help in planning and risk analysis through metrics for the code, e.g. complexity

Modelling Tools
Modelling tools are used to validate models of the software. For example, it may find defects and
inconsistencies in a physical data model (PDM) for a relational database.
Other modelling tools may find defects in a state model or an object model. These tools can often
aid in generating some test cases based on the model.
The major benefit of static analysis tools and modelling tools is the cost effectiveness of finding
defects earlier in the development process.
Tool Support for Test Execution and Logging
Test Execution Tools

Test execution tools enable tests to be executed automatically, or semi-automatically, using stored
inputs and expected outcomes, through the use of a scripting language. This makes it possible to
repeat the test with different data. Normally they also provide a test log for each run.
Test execution tools can also be used to record tests, they may be referred to as capture-playback
tools. These tools usually support scripting language or GUI-based configuration for
parameterization of data and other customization in the tests.

Unit Test Framework Tool

A test harness, or unit testing framework, is used to simulate the environment in which a test object
will run. It could either be a component or a part of a system.
This may be done either because other components of that environment are not yet available and are
therefore replaced by stubs and/or drivers, or simply to provide a predictable and controllable
environment in which any faults can be localized to the object under test.
Test Comparators
Test execution tools can include a comparator. Comparators determine differences between files,
databases or test results.
Test execution tools typically include dynamic comparators, but post-execution comparison may be
done by a separate comparison tool.
A test comparator may use a test oracle, especially if it is automated.
Coverage Measurement Tools
Code coverage tools measure the percentage of specific types of code structures that have been
exercised (e.g. statements, branches or decisions, and module or function calls). These tools show
how thoroughly the measured type of structures has been exercised by a set of tests.
Coverage measurement tools can be either intrusive or non-intrusive depending on the measurement
techniques used, what is measured and the coding language.
Security Testing Tools
Security Testing Tools are used to evaluate the security characteristics of software. This includes
evaluating the ability of the software to protect data confidentiality, integrity, authentication,
authorization, availability and non-repudiation.
Security tools are mostly focused on a particular technology, platform and a specific purpose.
Tool Support for Performance and Monitoring

Dynamic Analysis Tools

Dynamic analysis tools find defects that are evident only when software is executing, such as time
dependencies or memory leaks.
Dynamic analysis tools are typically used in component and component integration testing, and
when testing middleware.
Performance/Load/Stress Testing Tools
Performance testing tools monitor and report how a system behaves under a variety of simulated
usage conditions. It could be the number of concurrent users, ramp-up patterns, frequency or
relative percentage of transactions.
They simulate a load on an application, a database, or a system, e.g. a server. Performance testing
tools are often based on automated repetitive execution of tests, controlled by certain parameters.
The simulation load is achieved by means of creating virtual users carrying out a selected set of
transactions, spread across various test machines commonly known as load generators.
Monitoring Tools
Monitoring tools continuously analyze, verify and report on usage of specific system resources, and
give warnings of possible service problems.

6.1.9. The Psychology of Testing

Independence
The right mindset is important while testing and reviewing software. It differs from that used while
developing software, i.e. the developer tries to construct the correct software while the tester tries to
find flaws in the construction.
With the right mindset developers are able to test their own code but it is normally a testers
responsibility. This will help focus effort and provide an independent view by trained and
professional testing resources. Independent testing can be carried out at any level of testing.
The tester is often more effective at finding defects and failures if he/she have a certain degree of
independence and thus avoiding author bias. We have to keep in mind that independence is not a
replacement for familiarity because developers can efficiently find many defects in their own code.
When talking about independence there is a number of levels from low to high:
Tests designed by the developer who wrote the code, i.e. low level of independence.
Tests designed by another member of the development team, not the developer of the code.
Tests designed by person from another team in the organization, for example an independent test
team or test specialists.

Tests designed by a person(s) from a different organization or company, for example a outsourced
test site.

Objectives
It is important that the objectives of testing is clearly stated because people and projects are driven
by objectives, which could vary between projects. People tend to meet the objectives that apply, or
at least that they think are applicable.
For example, in one project the objective could be finding as many defects as possible but in
another project it could be to get confidence in a business process.
Focusing on only one objective could lead to that other product aspects are missed, e.g. only
focusing on finding defects may result in a system that is not user friendly.
Communication
Identifying failures during testing may be perceived as criticism against the product and against the
author. Therefore, testing is often seen as a destructive activity, even though it is very constructive
in the management of product risks. To be successful looking for failures in a system requires:

Curiosity
Professional pessimism
A critical eye
Attention to details
Good communication with development peers
Experience on which to base error guessing

Bad feelings between the testers and colleagues, for example analysts, designers and developers,
can be avoided if issues are communicated in a constructive way. This is true for testing as well as
reviews.
Testers and test leaders need good interpersonal skills to be able to communicate factual
information in a constructive way about, for example, defects, progress and risks. This information
can help the author of the software or document to improve their skills by learning from their
mistakes.
Communication problems may occur if testers are seen only as messengers of unwanted news about
defects, i.e. we shall not shoot the messenger. There are several ways to improve communication
between testers and project members. Things to think about are for example:

Start collaboration rather than battles

Communicate findings on the product in a neutral and fact-focused way without
criticizing the person who created it
Try to understand how the other person feels and why they react as they do
Confirm that the other person has understood what you have said and vice versa

Mutual understanding and collaboration between testers and developers is a powerful tool to
increase product quality and project efficiency.

6.1.10. Code of Ethics

Working with testing means that you and your colleagues will have access to confidential
information about products and processes, e.g. requirements and test results. This information must
be handled with care and not used in an inappropriate way. A code of ethics is necessary and ISTQB
states the following:
Public - Certified software testers shall act consistently with the public interest.
Client and employer - Certified software testers shall act in a manner that is in the best interests of
their client and employer, consistent with the public interest.
Product - Certified software testers shall ensure that the deliverables they provide (on the products
and systems they test) meet the highest professional standards possible.
Judgement - Certified software testers shall maintain integrity and independence in their
professional judgement.
Management - Certified software test managers and leaders shall subscribe to and promote an
ethical approach to the management of software testing.
Profession - Certified software testers shall advance the integrity and reputation of the profession
consistent with the public interest.
Colleagues - Certified software testers shall be fair to and supportive of their colleagues, and
promote cooperation with software developers.
Self - Certified software testers shall participate in lifelong learning regarding the practice of their
profession and shall promote an ethical approach to the practice of the profession.

6.2. Test Techniques

6.2.1 Categories of Test Design Techniques
Black-box Techniques
Black-box techniques also called specification-based techniques. The test object is represented as a
black-box, i.e. we do not use any information about the internal structure, but we can analyse the
behaviour or output based on a certain input.
Black-box techniques are a way to derive and select test conditions, test cases or test data based on
an analysis of the test basis documentation describing the software or expected behaviour of its
components.
The experience of developers, testers and users could also be used.
Black-box tests include both functional and non-functional testing.
White-box Techniques
White-box techniques (also called structural or structure-based techniques) are based on an analysis
of the structure of the component or system.
Information about how the software is constructed is used to derive the test cases. The information
could, for example, be derived from the code and detailed design.
The extent of coverage of the software can be measured for existing test cases, and further test cases
can be derived systematically to increase coverage.

Experience-based Techniques
The knowledge and experience of people are used to derive the test cases. Other sources of
information are:

Knowledge of testers, developers, users and other stakeholders about the software, its usage
and its environment

Knowledge about likely defects and their distribution

6.2.2 Specification-based or Black-box Techniques

Equivalence Partitioning
Input boxes are divided into groups and consist of input values that are expected to exhibit similar
behaviour. Equivalence partitions or classes can be found for both valid and invalid data, i.e. values
that should be rejected.
Partitions can also be identified for outputs, internal values, time related values and for interface
parameters.
Tests can be designed to cover partitions, and equivalence partitioning as a technique can be used to
achieve input and output coverage. Equivalence partitioning is applicable at all levels of testing.

Boundary Value Analysis (K3)

Boundary values are values at the edge of an equivalence partition, the maximum and minimum
value. Both valid and invalid partitions have boundary values.
Behaviour at the edge of each equivalence partition is more likely to be incorrect, so boundaries are
an area where testing is likely to yield defects. Detail specifications are helpful to in determining the
interesting boundaries.
When using this technique, a test case for each boundary value is used. Boundary value analysis can
be applied at all test levels. It is relatively easy to apply and its defect finding capability is high.
Decision Table Testing
Using decision tables is a good way to capture system requirements that contain logical conditions.
When creating a decision table the specification is analysed, and conditions and actions are
identified. They are stated in such a way that they can either be true or false.
Each column of the table corresponds to a business rule that defines a unique combination of
conditions, which result in the execution of the actions associated with that rule.
Decision tables are especially useful to ensure that important combinations are not overlooked.
The coverage standard commonly used with decision table testing is to have at least one test per

column, which typically involves covering all combinations of triggering conditions.

State Transition Testing
A system may exhibit a different response depending on current conditions or previous history (its
state). In this case, that aspect of the system can be shown as a state transition diagram.
This diagram allows the tester to get a good overview of it's states, transitions between them, inputs
or events that trigger transitions and the actions which may result from those.
The goal with state transition testing is to exercise all the states and transitions in the application.
Tests can be designed to cover every state or a typical sequence of states. It can also be designed to
exercise every transition, specific sequences of transitions or to test invalid transitions.

Use Case Testing

Use cases describe the "process flows" through a system based on its actual likely use. Test cases
derived from use cases are most useful in uncovering defects in the process flows during real-world
use of the system.
Use cases, often referred to as scenarios, are very useful when designing acceptance tests with
customer/user participation. They also help uncover integration defects caused by the interaction
and interference of different components, which individual component testing would not detect.

6.2.3 Structure-based or White-box Techniques

Statement Testing and Coverage
Statement coverage is a type of structural testing that looks at the percentage of all code statements
that have been executed by a test case suite. The intention is to increase statement coverage by
designing
more
test
cases
until
we
reach
defined
goals.
Statement coverage is determined by the number of executable statements covered by executed test
cases divided by the number of all executable statements in the code under test.
Decision Testing and Coverage
Decision coverage is a technique that looks at the percentage of decision outcomes that have been
reached.
Decision coverage has higher percentage coverage than statement coverage i.e. 100 % decision
coverage guarantees 100 % statement coverage, but not vice versa.

6.2.4

Experience-based Techniques

Experienced-based testing is when tests are derived from the tester's skill and intuition as well as
their experience with similar applications and technologies.
These techniques can be useful in identifying special tests not easily captured by formal techniques.
An experienced-based technique may yield varying degrees of effectiveness depending of the
tester's experience.
Error Guessing Technique
A commonly used experienced-based technique is error guessing, i.e. testers anticipate defects
based on experience.
A structured approach is to enumerate a list of possible defects and to design tests attacking these
defects. This systematic approach is called fault attack.
These defect and failure lists can be built based on experience, available defect and failure data, and
from common knowledge about why software fails.
Exploratory Testing
In exploratory testing, test design, test execution, test logging and learning, based on a test charter
containing test objectives and performed within time-boxes.
It is an approach that is most useful where there are few or inadequate specifications and severe
time pressure, or in order to augment or complement other, more formal testing. It can serve as a
check to help ensure that the most serious defects are found.

6.3. Defect Management

6.3.1 Reporting defects
Writing a good defect or bug report goes a long way in identifying and resolving the problems
quickly. Here is a I list of elements that are normally included in a bug report.
In no particular order:
Defect Identifier, ID
The identifier is very important in being able to refer to the defect in the reports. If a defect
reporting tool is used to log defects, the ID is normally a program generated unique number which
increments per defect log.
Summary
The summary is an overall high level description of the defect and the observed failure. This short
summary should be a highlight of the defect as this is what the developers or reviewers first see in
the bug report.
Description
The nature of the defect must be clearly written. If a developer reviewing the defect cannot
understand and cannot follow the details of the defect, then most probably the report will be
bounced back to the tester asking for more explanation and more detail which causes delays in
fixing the issue.
The description should explain exactly the steps to take to reproduce the defect, along with what the
expected results were and what the outcome of the test step was. The report should say at what step
the failure was observed.
Severity
The severity of the defect shows how sever the defect is in terms of damaging to other systems,
businesses, environment and lives of people, depending on the nature of the application system.
Priority
Once the severity is determine, next is to see how to prioritize the resolution. The priority

determines how quickly the defect should be fixed. The priority normally concerns the business
importance such as impact on the project and the likely success of the product in the marketplace.
Date and Time
The date and time that the defect occurred or reported is also essential. This is normally useful when
you want to search for defects that were identified for a particular release of software or from when
the testing phase started.

Version and Build of the Software Under Test

This is very important too. In most cases, there are many versions of software; each version has
many fixes and more functionality and enhancements to the previous versions. Therefore, it is
essential to note which version of the software exhibited the failure that we are reporting. We may
always refer to that version of software to reproduce the failure.
Reported by
Again, this is important, because if we may need to refer to the person who raised the defect, we
have to know who to contact.
Related requirement
Essentially, all features of a software application can be traced to respective requirements. Hence,
when a failure is observed, we can see what requirements have been impacted. This can help in
reducing duplicate defect reports in that if we can identify the source requirement, then if another
defect is logged with the same requirement number, we may not need report it again, if the defects
are of similar nature.
Attachments / Evidence
Any evidence of the failure should be captured and submitted with the defect report. This is a visual
explanation of the description of the defect and helps the reviewer, developer to better understand
the defect.
An example of Defect Report in Jira is shown in the pictures below:

6.3.2 Defect Life cycle

Defect Life Cycle is the stages that the
defect or bug goes through from when it is
first reported until it is fixed and confirmed.
Generally speaking, the defect life cycle is made
of four stages:

up
Stage
1,

defects

are

found

and

reported.
Stage 2, defects are reviewed and delegated.
Stage 3, defects are debugged and removed.
Stage 4, removed defects are confirmed.

Stage 1: Defects are found and reported by the testing team

After the development team have developed the software and is ready for testing, the software is
then released to the testing team. Items released to testing generally include Software Requirement
Specification (SRS), Software Design Specification (SDS), executable application and any third
party libraries that may be required to make the application work.
Once the application is available to testing team, testing of the application starts based on the Test
Procedures. The activities may include Functional Testing, Non-Functional Testing, Performance
Testing, etc.
During testing, testers may find defects either in the application or in the documents e.g. test
procedure.
If a defect is found, either in the application (source code) or in the document, a defect report form
is filled and passed on to the test team lead or test manager for review.
Defect Status is: Open and Not Reviewed. This is the first stage of defect life cycle.

Stage 2: Defects are reviewed and delegated

Defects get reviewed first by test team lead for any obvious mistakes by the testers, such as missing
information on the defect form (see Defect Reporting) and then reported to software development
manager for further review.
A meeting is scheduled which includes members of the development team and testing team to

discuss
the
validity,
severity
and
priority
At this stage, defects are categorised into three different states:

of

the

defects.

Not a defect
Defect will be fixed
Defect wont be fixed in this release
Defect Status is: Reviewed and Assigned. This is the second stage of defect life cycle.

Stage 3: Defects are debugged and removed

After the meeting and agreed actions, the software development manager delegates defects to
different developers in the team.
For defects that are not going to be fixed in the current version of the software, an impact analysis
must be performed to identify any potential failures that may occur as a result of the existing defects
in the system.
Developers will analyse the source code to identify the root cause and ultimately remove the defect.
Defect Status is: Being Debugged. This is the third stage of defect life cycle

Stage 4: Removed defects are confirmed

The Fixed version of the software will be released to the testing team and testers will start to test
the fixed software. At this stage, two types of testing are performed by the testers: confirmation
testing and regression testing.
Based on the outcome of the confirmation testing, the fixed defects are either confirmed as fixed
(defect is removed) or not fixed (defect still exists).
For defects that are fixed, their status is changed to fixed and can be closed.
For defects that are not fixed, i.e. they still exist in the system, they go round the cycle again, i.e. are
reported to the development team, gets reviewed by the development team, faults are removed, a
new version of software is released to testing for confirmation + regression testing and so on.
This cycle is repeated until all the defects (that were decided to be fixed in the current version) are
fixed and verified by testers.
Defect Status is Fixed & Closed. This is the fourth and last stage of defect life cycle.

6.3.3 Severity and Priority

Both Severity and Priority are attributes of a defect and should be provided in the bug report. This

information is used to determine how quickly a bug should be fixed.

Severity of a defect is related to how severe a bug is. Usually the severity is defined in terms of
financial loss, damage to environment, companys reputation and loss of life.
Priority of a defect is related to how quickly a bug should be fixed and deployed to live servers.
When a defect is of high severity, most likely it will also have a high priority. Likewise, a low
severity defect will normally have a low priority as well.
Although it is recommended to provide both Severity and Priority when submitting a defect report,
many companies will use just one, normally priority.
In the bug report, Severity and Priority are normally filled in by the person writing the bug report,
but should be reviewed by the whole team.
High Severity High Priority bug
This is when major path through the application is broken, for example, on an eCommerce website,
every customers get error message on the booking form and cannot place orders, or the product
page throws a Error 500 response.
High Severity Low Priority bug
This happens when the bug causes major problems, but it only happens in very rare conditions or
situations, for example, customers who use very old browsers cannot continue with their purchase
of a product. Because the number of customers with very old browsers is very low, it is not a high
priority to fix the issue.
High Priority Low Severity bug
This could happen when, for example, the logo or name of the company is not displayed on the
website. It is important to fix the issue as soon as possible, although it may not cause a lot of
damage.
Low Priority Low Severity bug
For cases where the bug doesnt cause disaster and only affects very small number of customers,
both Severity and Priority are assigned low, for example, the privacy policy page take a long time to
load. Not many people view the privacy policy page and slow loading doesnt affect the customers
much.
The above are just examples. It is the team who should decide the Severity and Priority for each
bug.

6.4. Test Management

6.4.1. Test Organization and Independence
The effectiveness to finding defects and failures by reviews and testing can be improved by using
independent testers. When talking about independence there is a number of options from low to
high:

No independent testers, i.e. developers test their own code

Independent tester with in the development teams

Independent test team or a group within the organization, reporting to project

management or executive management

Independent testers from the business organization or user community

Independent test specialist for specific test types, e.g. usability testers, security testers or
certification testers

Independent testers outsourced or external to the organization

6.4.2. Tasks of the Test Leader and Tester

Typically the test leader plans, monitors and controls the testing activities, while the tester is
responsible for carrying out test base analysis, design and implementation of tests.
Typical test leader tasks may include:

Coordinate the test strategy and plan with project managers and others
Write or review a test strategy for the project and a test policy for the organization
Planning of the tests considering the context and understanding the test objectives and
risk including selecting test approaches, estimating the time, effort cost of testing,
acquiring resources, defining test levels, cycles and planning incident management
Initiate the specification, preparation, implementation and execution of test, monitor the
test results and check the exit criteria
Adapt planning based on test results and progress, and take any action necessary to
compensate for problems
Set-up adequate configuration management of testware for traceability
Introduce suitable metrics for measuring test progress and evaluating the quality of the
testing and the product
Decide what should be automated, to what degree and how it should be done
Select tools to support testing and organize any training in tool use for testers
Decide about the implementation of the test environment
Write test summary reports based on the information gathered during testing

Typical tasks for a tester may include:

Review and contribute to test plans

Analyse, review and assess user requirements, specifications and models for testability
Create test specifications
Set up the test environment
Prepare and acquire test data
Implement tests on all test levels, execute and log the tests, evaluate the results and
document the deviations from expected results
Use test administration or management tools and test monitoring tools as requiered
Automate tests (may be supported by a developer or a test automation expert)
Measure performance of components and systems
Review tests developed by others

6.4.3. Test Planning and Estimation

Test Planning Activities
Test planning activities may include:

Defining a overall approach for the project, including the definition of test levels and
entry and exit criteria
Determining the scope, risks and objectives of testing
Integrating and coordinating testing activities with software lifecycle activities, e.g.
acquisition, supply, development, operation and maintenance
Assigning resources for the different activities defined
Making decisions about what to test, what roles will perform the test activities, how the
activities should be done, and how the test results will be evaluated
Determine roles and assignments to the project
Scheduling test analysis and design activities, test implementation, execution and
evaluation
Defining the amount, level of detail, structure and templates for the test documentation
Selecting metrics for monitoring and controlling test preparation and execution, defect
resolution and risk issues
Setting the level of detail for test procedures in order to provide enough information to
support reproducible test preparation and execution

Entry Criteria
Entry criteria define when to start testing such as the beginning of the test level or when a set of test
is ready for execution. Typically entry criteria may cover the following:

Test environment availability and readiness

Test tool readiness in the test environment
Availability of testable code and test data

Exit Criteria
It is not possible to find all bugs and defects in a product, because you cannot test everything. This
is one reason why an exit criteria is needed. The exit criteria define when to stop testing such as the
end of the test level or when set of tests has achieved a specific goal.
Typical exit criteria may cover the following:

Thoroughness measures, for example, coverage of code, functionality or risk

Estimates of defect density or reliability measures
Cost
Schedules such as those based on time to market
Residual risks, for example, defects not fixed or lack of test coverage in certain areas

Test Estimation
Because time and resources are limited, planning involves estimating how much test effort will be
needed. Two approaches for estimation of test effort are:

The metrics-based approach - Estimating the testing effort based on metrics of former or
similar projects or based on typical values

The expert-based approach - Estimating the tasks based on estimates made by the owner
of the tasks or by experts

Once the test effort is estimated, resources can be identified and a schedule can be created.
The testing effort may depend on a number of factors that can be organized into three main groups.
Characteristics of the product
This includes the quality of the specification and other information used for test models (i.e. the test
basis), the size of the product, the complexity of the problem domain, the requirements for
reliability and security and the requirements for documentation
Characteristics of the development process
This includes the stability of the organization, tools used, test process, skills of the people involved
and time pressure
The outcome of testing
This includes the number of defects and the amount of rework required. Used on ongoing projects
which requires re-scheduling.

6.4.4. Strategy and Approach

The test approach is the implementation of the test strategy for a specific project. The approach is
defined in the test plans and test design. Typically includes the decisions made based on the
project's goal and risk assessment. It is the starting point for:

Planning the test process

Selecting test design techniques and test types to be applied
Defining entry and exit criteria

The selected approach depends on the context and may consider risks, hazards and safety, available
resources and skills, the technology, the nature of the system, test objectives and regulations. It
should be noted that it is possible to combine different approaches.
Classifying test approaches or strategies is based on the point in time at which the bulk of the test
design work is begun:

Preventative approaches, where tests are designed as early as possible

Reactive approaches, where test design comes after the software or system has been
produced

We can base our testing strategy on a variety of factors - risks, standards and methodologies. They
will be discussed in the following pages.

6.4.5. Test Progress Monitoring and Control

Test Progess Monitoring (K1)
The purpose of test monitoring is to give feedback and visibility about test activities. Information to
be monitored may be collected manually or automatically and may be used to measure exit criteria,
such as test coverage. Metrics may also be used to assess progress against the planned schedule and
budget.
Common test metrics for progress monitoring include:

Percentage of work done in test case preparation (or percentage of planned test cases
prepared)
Test coverage of requirements, risks or code
Percentage of work done in test environment preparation
Test case execution (e.g. number of test cases run/not run, and test cases passed/failed)
Defect information (e.g. defect density, defects found and fixed, failure rate, and re-test
results)
Subjective confidence of testers in the product
Dates of test milestones

Testing costs, including the cost compared to the benefit of finding the next defect or to
run the next test

Test Reporting (K2)

At the end of a test level, a test summary report is created with information about the testing
endeavour. This report includes:

What happened during a period of testing, such as dates when exit criteria were met

Analysed information and metrics to support recommendations and decisions about

future actions, for exmaple, assessment of defects remaining, the economic benefit of
continued testing, outstanding risks, and the level of confidence in tested software

A test summary report should contain metrics collected during and at the end of a test level in order
to assess:

The adequacy of the test objectives for that test level

The adequacy of the test approaches taken

The effectiveness of the testing with respect to its objectives

Test Control
Test control describes any guiding or corrective action taken as a result of information and metrics
gathered and reported. Actions may cover any test activity and may affect any other software life
cycle activity or task.
Examples of test control actions are:

Making decisions based on information from test monitoring

Reprioritizing test when an identified risk occurs, e.g. late delivery of a software build
Changing the test schedule due to availability on unavailability of a test environment
Setting an entry criterion requiring fixes to have been re-tested (conformation tested) by
a developer before accepting them into a build

6.4.6. Risk and Testing

Risk can be defined as the chance of an event, hazard, threat or situation occuring and resulting in
undesirable consequences or a potential problem.
The level of risk will be determined by the likelihood of an adverse event happening and the impact,
i.e. the harm resulting from that event.

Project Risks
Project risks are the those that surround the project's capability to deliver its objectives. These can
be organizational, technical or related to external parties, such as suppliers.
When analyzing, managing and mitigating these risks, the test manager shall follow wellestablished project management priciples. In IEEE Std 829 the test plan outline requires risks and
contingencies to be stated.
Organizational factors

Skill, training and staff shortages

Personel issues
Political issues such as problems with testers communicating their needs and test results,
and failure by the team to follow up on information found in testing and reviews
Improper attitude toward or expectations of testing, e.g. not appreciate the value of
finding defects

Technical issues
Problems in defining the right requirements
The extent to which requirements cannot be met given existing constraints
Test environment not ready on time
Late data conversion, migration planning and development, and testing tools to support
this
Low quality of the design, code, configuration data, test data and tests
Supplier issues
Failure of third party
Contractual issues
Product Risks
Potential failure areas in the software or system are known as product risks, as they are a risk to the
quality of the product. Products risks are a special type of risk to the success of a project.
Testing as a risk-control activity provides feedback about residual risk by measuring the
effectiveness of critical defect removal and contingency plans.
Risks are used to decide where to start testing and where to test more, i.e. test is used to reduce the
risk of an adverse effect occurring or to reduce the impact of the adverse effect.
Example of potential failure areas could be:
Failure-prone software delivered

The potential that the software/hardware could cause harm to an individual or company
Poor software characteristics, .e.g. reliability, usability and performance
Poor data integrity and quality, e.g. migration issues, conversion problems and violation
of data standards
Software that does not perform its intended functions

A risk-based approach to testing provides proactive opportunities to reduce the levels of product
risk, starting in the initial stages of a project. It involves the identification of product risks and their
use in guiding test planning and control, specification, preparation and execution of tests. In a riskbased approach the risks identified may be used to:

Determine what techniques to use

Determine the extent of testing to be carried out
Prioritize testing in an attempt to find the critical defects as early as possible
Determine whether any non-testing activities could be employed to reduce risk, e.g.
providing training to inexperienced testers

Risk-based testing draws on the collective knowledge and insight of the project stakeholders to
determine the risks and the levels of testing required to address those risks.
To ensure that the chance of a product failure is minimized, risk management activities provide a
disciplined approach to:

Assess and re-assess on a regular basis what can go wrong

Determine what risks are important to deal with
Implement actions to deal with those risks

In addition, testing may support the identification of new risks, may help to determine what risks
should be reduced, and may lower uncertainty about risks.

6.5. Testing Tools

6.5.1. Manual and Automation testing
Manual Testing
Manual testing includes testing a software manually, i.e., without using any automated tool or any
script. In this type, the tester takes over the role of an end-user and tests the software to identify any
unexpected behavior or bug. There are different stages for manual testing such as unit testing,
integration testing, system testing, and user acceptance testing.
Testers use test plans, test cases, or test scenarios to test a software to ensure the completeness of
testing. Manual testing also includes exploratory testing, as testers explore the software to identify
errors in it.

Automation Testing
Automation testing, which is also known as Test Automation, is when the tester writes scripts and
uses another software to test the product. This process involves automation of a manual process.
Automation Testing is used to re-run the test scenarios that were performed manually, quickly, and
repeatedly.
Apart from regression testing, automation testing is also used to test the application from load,
performance, and stress point of view. It increases the test coverage, improves accuracy, and saves
time and money in comparison to manual testing.
What is Automate?
It is not possible to automate everything in a software. The areas at which a user can make
transactions such as the login form or registration forms, any area where large number of users can
access the software simultaneously should be automated.
Furthermore, all GUI items, connections with databases, field validations, etc. can be efficiently
tested by automating the manual process.
When to Automate?
Test Automation should be used by considering the following aspects of a software:
Large and critical projects
Projects that require testing the same areas frequently
Requirements not changing frequently

 Accessing the application for load and performance with many virtual users
Stable software with respect to manual testing
Availability of time

How to Automate?
Automation is done by using a supportive computer language like VB scripting and an automated
software application. There are many tools available that can be used to write automation scripts.
Before mentioning the tools, let us identify the process that can be used to automate the testing
process:

Identifying areas within a software for automation

Selection of appropriate tool for test automation
Writing test scripts
Development of test suits
Execution of scripts
Create result reports
Identify any potential bug or performance issues

6.5.2. Automation Testing Tools

Automated software testing is becoming more and more important for many software projects in
order to automatically verify key functionality, test for regressions and help teams run a large
number of tests in a short period of time. Many teams (especially larger projects) still require a
significant amount of manual functional testing in addition to automated testing, either because of
the lack of sufficient resources or skills to automate all tests.
There are various tools that help software teams build and execute automated tests. Many teams are
actively using unit tests as part of their development efforts to verify critical parts of their projects
such as libraries, models and methods. Historically, testing user interfaces of desktop-based
applications via automated tests have been more challenging, and currently available tools for this
are usually commercial and quite expensive.
With the growing number of web-based applications this is changing, however, as verifying and
testing web-based interfaces is easier and there are various tools that help with this, including free
open source projects. Please see below for a list of popular and useful tools, projects, books and
resources to get started with automated software testing.
6.5.2.1 Automated Web Testing Tools
Automatically testing your web application is a good way to ensure that new versions of your
application don't introduce bugs and regressions. Automation of your web application testing also
allows your development team to make changes and refactor code with more confident, as they can
quickly verify the functionality of the application after every change.

However, actually building automated tests for web applications can be challenging because the
user interface of your application might change regularly, because of incompatibilities between
browsers and because you usually need to support various server or client platforms. The following
tools make it easier to build and execute automated tests for your web application.

The most popular free automated test tools are:

Selenium
Selenium is a popular automated web testing tool and helps you automate web browsers across
different platforms. Selenium has the support of some of the largest browser vendors who have
taken steps to make Selenium a native part of their browser.

Watir
Watir is a set of Ruby libraries for automating web browsers and allows you to write tests that are
easy to read and maintain. Watir drives browsers the same way people do (it clicks links, fills in
forms, presses buttons etc.) and also checks results such as whether expected text appears on the
page.

Windmill
Windmill is a web testing tool designed to help testers automate and debug web applications. It
comes with a cross-browser test recorder, JavaScript integration and an interactive shell to automate
web browsers.

Ranorex
Ranorex allows you to automate your web application testing (among other things) and both record
user interactions and play them back to execute your tests. Ranorex is one of the more popular
commercial tools to build and run automated web and GUI tests.

SoapUI
SoapUI is a cross-platform functional testing tool. It has been specifically designed to help
automatically test APIs such as SOAP and REST interfaces to ensure the interoperability of
different applications.

Sahi
Sahi is a tool for automation of web application testing. Sahi is available as a free open source
edition as well as a commercial Pro edition.

Tellurium
Tellurium is a web automation tool that allows you to design and write your automated tests using

plain English without any scripting or programming experience.

The best commercial test tools are:

UFT(QTP)

HPE Unified Functional Testing (UFT) software, formerly known as HP QuickTest Professional
(QTP),provides functional and regression test automation for software applications and
environments. HPE Unified Functional Testing can be used for enterprise quality assurance.
Cost: Commercial , Start at 8000 US Dollar These licenses are normally sold to organizations and
not to individuals.

TestComplete Platform

TestComplete Platform, which powers TestComplete Desktop, TestComplete Web, and

TestComplete Mobile, helps you create accurate and repeatable automated tests across multiple
devices, platforms, and environments easily and quickly.
Reduce initial investment cost on a testing software, but also realize long term saving on test
maintenance and support costs.
Cost: 1200-3000 Euro

Telerik Test Studio

Telerik Test Studio is a Windows-based software testing tool for web and desktop, it support is an
innovative and easy-to-use automated web, WPF and load testing solution. Test Studio tests support
essential technologies like ASP.NET AJAX, Silverlight, PHP and MVC.
Cost: 1000-3500 Dollars
6.5.3. Selenium IDE
6.5.3.1. Introduction to Selenium IDE
The Selenium-IDE (Integrated Development Environment) is the tool you use to develop your
Selenium test cases. Its an easy-to-use Firefox plug-in and is generally the most efficient way to
develop test cases. It also contains a context menu that allows you to first select a UI element from
the browsers currently displayed page and then select from a list of Selenium commands with
parameters pre-defined according to the context of the selected UI element. This is not only a timesaver, but also an excellent way of learning Selenium script syntax.

Features of Selenium IDE

Menu Bar
It is located at the top most portion of the IDE. The most commonly used menus are the File, Edit,
and Options menus.

File menu
It contains options to create, open, save, and close tests.
Tests are saved in HTML format.
The most useful option is "Export" because it allows you to turn your Selenium IDE test
cases into file formats that can run on Selenium Remote Control and WebDriver

"Export Test Case As..." will export only the currently opened test case.

"Export Test Suite As..." will export all the test cases in the currently opened
test suite.

Edit Menu

It contains usual options like Undo, Redo, Cut, Copy, Paste, Delete, and Select All.

The two most important options are the "Insert New Command" and "Insert New
Comment".

The newly inserted command or comment will be placed on top of the currently selected
line.

Commands are colored black.

Comments are colored purple.

Base URL Bar

It has a dropdown menu that remembers all previous values for easy access.
The Selenese command "open" will take you to the URL that you specified in the Base
URL.
In this tutorial series, we will be using http://newtours.demoaut.com as our Base URL. It is
the site for Mercury Tours, a web application maintained by HP for web testing purposes.
We shall be using this application because it contains a complete set of elements that we
need for the succeeding topics.
The Base URL is very useful in accessing relative URLs. Suppose that your Base URL is
set to http://newtours.demoaut.com. When you execute the command "open" with the target
value "signup", Selenium IDE will direct the browser to the sign-up page. See the
illustration below.

Test Case Pane

In Selenium IDE, you can open more than one test case at a time.
The test case pane shows you the list of currently opened test cases.
When you open a test suite, the test case pane will automatically list all the test cases
contained in it.
The test case written in bold font is the currently selected test case
After playback, each test case is color-coded to represent if it passed or failed.
Green color means "Passed."
Red color means "Failed."
At the bottom portion is a summary of the number of test cases that were run and failed.

Editor
You can think of the editor as the place where all the action happens. It is available in two views:
Table and Source.

Table View

Most of the time, you will work on Selenium IDE using the Table View.
This is where you create and modify Selenese commands.
After playback, each step is color-coded.

To create steps, type the name of the command in the "Command" text box.

It displays a dropdown list of commands that match with the entry that you are
currently typing.

Target is any parameter (like username, password) for a command and Value is the input
value (like tom, 123pass) for those Targets.

Log Pane

The Log Pane displays runtime messages during execution. It provides real-time updates as to
what Selenium IDE is doing.
Logs are categorized into four types:
Debug - By default, Debug messages are not displayed in the log panel. They show up only
when you filter them. They provide technical information about what Selenium IDE is doing
behind the scenes. It may display messages such as a specific module has done loading, a
certain function is called, or an external JavaScript file was loaded as an extension.
Info - It says which command Selenium IDE is currently executing.
Warn - These are warning messages that are encountered in special situations.
Error - These are error messages generated when Selenium IDE fails to execute a command,
or if a condition specified by "verify" or "assert" command is not met.

Logs can be filtered by type. For example, if you choose to select the "Error" option from the
dropdown list, the Log Pane will show error messages only.

Reference Pane

The Reference Pane shows a concise description of the currently selected Selenese command
in the Editor. It also shows the description about the locator and value to be used on that
command.

6.5.3.2. Creating your First Selenium IDE script

We will use the Mercury Tours website as our web application under test. It is an online flight
reservation system that contains all the elements we need for this tutorial. Its URL is
http://newtours.demoaut.com/ and this will be our Base URL.
Step 1

Launch Firefox and Selenium IDE.

Type the value for our Base URL: http://newtours.demoaut.com/.
Toggle the Record button on (if it is not yet toggled on by default).

Step 2
In Firefox, navigate to http://newtours.demoaut.com/. Firefox should take you to the page similar to
the one shown below.

Step 3

Right-click on any blank space within the page, like on the Mercury Tours logo on the
upper left corner. This will bring up the Selenium IDE context menu. Note: Do not click on any
hyperlinked objects or images

Select the "Show Available Commands" option.

Then, select "assertTitle exact: Welcome: Mercury Tours". This is a command that
makes sure that the page title is correct.

Step 4

In the "User Name" text box of Mercury Tours, type an invalid username, "invalidUN".
In the "Password" text box, type an invalid password, "invalidPW".

Step 5

Click on the "Sign-In" button. Firefox should take you to this page.

Step 6
Toggle the record button off to stop recording. Your script should now look like the one shown
below.

Step 7
Now that we are done with our test script, we shall save it in a test case. In the File menu, select
"Save Test Case". Alternatively, you can simply press Ctrl+S.

Step 8

Choose your desired location, and then name the test case as "Invalid_login".
Click the "Save" button.

Step 9.
Notice that the file was saved as HTML.

Step 10.
Go back to Selenium IDE and click the Playback button to execute the whole script. Selenium IDE
should be able to replicate everything flawlessly.

6.5.3.3

Selenium Commands Selenese

Selenese commands can have up to a maximum of two parameters: target and value.

Parameters are not required all the time. It depends on how many the command will need.

A command tells Selenium what to do. Selenium commands come in three flavors: Actions,
Accessors, and Assertions.
Actions are commands that generally manipulate the state of the application. They do things
like click this link and select that option. If an Action fails, or has an error, the execution
of the current test is stopped.
Many Actions can be called with the AndWait suffix, e.g. clickAndWait. This suffix tells
Selenium that the action will cause the browser to make a call to the server, and that Selenium

should wait for a new page to load.

Accessors examine the state of the application and store the results in variables, e.g.
storeTitle. They are also used to automatically generate Assertions.
Assertions are like Accessors, but they verify that the state of the application conforms to
what is expected. Examples include make sure the page title is X and verify that this
checkbox is checked.
All Selenium Assertions can be used in 3 modes: assert, verify, and waitFor. For example,
you can assertText, verifyText and waitForText. When an assert fails, the test is aborted.
When a verify fails, the test will continue execution, logging the failure. This allows a single
assert to ensure that the application is on the correct page, followed by a bunch of verify
assertions to test form field values, labels, etc.
waitFor commands wait for some condition to become true (which can be useful for testing Ajax
applications). They will succeed immediately if the condition is already true. However, they will
fail and halt the test if the condition does not become true within the current timeout setting (see the
setTimeout action below).

Common Commands

Command

Number of Parameters

Description

open

0-2

Opens a page using a URL.

click/clickAndWait

Clicks on a specified element.

type/typeKeys

Types a sequence of characters.

verifyTitle/assertTitle

Compares the actual page title

with an expected value.

verifyTextPresent

Checks if a certain text is found

within the page.

verifyElementPresent

Checks the presence of a certain

element.

verifyTable

Compares the contents of a

table with expected values.

waitForPageToLoad

Pauses execution until the page

is loaded completely.

waitForElementPresent

Pauses execution until the

specified element becomes
present.

Verify Presence of an Element

We can use following two commands to verify the presence of an element:

verifyElementPresent - returns TRUE if the specified element was FOUND in the page;
FALSE if otherwise
verifyElementNotPresent - returns TRUE if the specified element was NOT FOUND
anywhere in the page; FALSE if it is present.
The test script below verifies that the UserName text box is present within the Mercury Tours
homepage while the First Name text box is not. The First Name text box is actually an element
present in the Registration page of Mercury Tours, not in the homepage. strong>Verify Presence of
a Certain Text.

Verify Presence of a Certain Text

verifyTextPresent - returns TRUE if the specified text string was FOUND somewhere in
the page; FALSE if otherwise
verifyTextNotPresent - returns TRUE if the specified text string was NOT FOUND
anywhere in the page; FALSE if it was found
Remember that these commands are case-sensitive.

In the scenario above, "Atlanta to Las Vegas" was treated differently from "atlanta to Las Vegas"
because the letter "A" of "Atlanta" was in uppercase on the first one while lowercase on the other.
When the verifyTextPresent command was used on each of them, one passed while the other failed.

Verify Specific Position of an Element

Selenium IDE indicates the position of an element by measuring (in pixels) how far it is from the
left or top edge of the browser window.
verifyElementPositionLeft - verifies if the specified number of pixels match the distance of
the element from the left edge of the page. This will return FALSE if the value specified
does not match the distance from the left edge.
verifyElementPositionTop - verifies if the specified number of pixels match the distance of
the element from the top edge of the page. This will return FALSE if the value specified
does not match the distance from the top edge.

Wait commands

andWait commands
These are commands that will wait for a new page to load before moving onto the next command.
Examples are
clickAndWait
typeAndWait
selectAndWait

waitFor commands
These are commands that wait for a specified condition to become true before proceeding to the
next command (irrespective of loading of a new page). These commands are more appropriate to be
used on AJAX-based dynamic websites that change values and elements without reloading the
whole page. Examples include:
waitForTitle
waitForTextPresent
waitForAlert
Consider the Facebook scenario below.

We can use a combination of "click" and "waitForTextPresent" to verify the presence of the text
"Providing your birthday".

We cannot use clickAndWait because no page was loaded upon clicking on the "Why do I need to
provide my birthday?" link. If we do, the test will fail

Storing Variables and the Echo command

Store
To store variables in Selenium IDE, we use the "store" command. The illustration below stores the
value "tutorial" to a variable named "myVariable".

To access the variable, simply enclose it in a ${ ... } symbol. For example, to enter the value of
"myVariable" onto the "userName" textbox of Mercury Tours, enter ${myVariable} in the Value
field.

StoreElementPresent
This command stores either "true" or "false" depending on the presence of the specified element.
The script below stores the Boolean value "true" to "var1" and "false" to "var2". To verify, we will
use the "echo" command to display the values of var1 and var2. The Base URL for the illustration
below was set to Mercury Tours homepage.

StoreText
This command is used to store the inner text of an element onto a variable. The illustration below
stores the inner text of the tag in Facebook onto a variable named 'textVar'.

Since it is the only element in the page, it is safe to use 'css=h1' as our target. The image below
shows that Selenium IDE was able to save the string "Sign Up" in the 'textVar' variable by printing
its value correctly.

Create a Script Manually with Firebug

Now, we shall recreate the same test case manually, by typing in the commands. This time, we will
need to use Firebug.
Step 1
Open Firefox and Selenium IDE.
Type the base URL (http://newtours.demoaut.com/).
The record button should be OFF.

Step 2
Click on the topmost blank line in the Editor.

Type "open" in the Command text box and press Enter.

Step 3
Navigate Firefox to our base URL and activate Firebug
In the Selenium IDE Editor pane, select the second line (the line below the "open"
command) and create the second command by typing "assertTitle" on the Command box.
Feel free to use the autocomplete feature.

Step 4
In Firebug, expand the <head> tag to display the <title> tag.
Click on the value of the <title> tag (which is "Welcome: Mercury Tours") and paste it onto
the Target field in the Editor.

Step 5
To create the third command, click on the third blank line in the Editor and key-in "type" on
the Command text box.
In Firebug, click on the "Inspect" button.

Click on the User Name text box. Notice that Firebug automatically shows you the HTML code for
that element.

Step 6
Notice that the User Name text box does not have an ID, but it has a NAME attribute. We shall,
therefore, use its NAME as the locator. Copy the NAME value and paste it onto the Target field in
Selenium IDE.

Still in the Target text box, prefix "userName" with "name=", indicating that Selenium IDE should
target an element whose NAME attribute is "userName."

Type "invalidUN" in the Value text box of Selenium IDE. Your test script should now look like the
image below. We are done with the third command. Note: Instead of invalidUN, you may enter any
other text string. But Selenium IDE is case sensitive and you type values/attributes exactly like in
the application.

Step 7
To create the fourth command, key-in "type" on the Command text box.
Again, use Firebug's "Inspect" button to get the locator for the "Password" text box.

Paste the NAME attribute ("password") onto the Target field and prefix it with "name="
Type "invalidPW" in the Value field in Selenium IDE. Your test script should now look like
the image below.

Step 8
For the fifth command, type "clickAndWait" on the Command text box in Selenium IDE.
Use Firebug's "Inspect" button to get the locator for the "Sign In" button.

 Paste the value of the NAME attribute ("login") onto the Target text box and prefix it with
"name=".
Your test script should now look like the image below.

Step 9
Save the test case in the same way as we did in the previous section.

6.5.3.4. Locators in Selenium IDE

Locators tell Selenium IDE which GUI elements ( say Text Box, Buttons, Check Boxes etc) its
needs to operate on. Identification of correct GUI elements is a prerequisite to create an automation
script. But accurate identification of GUI elements is more difficult than it sounds. Sometimes, you
end up working with incorrect GUI elements or no elements at all! Hence, Selenium provides a
number of Locators to precisely locate a GUI element.
The different types of locator are:
ID
Name
Link Text
CSS Selector

Tag and ID

Tag and class

Tag and attribute

Tag, class, and attribute

Inner text
DOM (Document Object Model)

getElementById

getElementsByName

XPath

dom:name
dom: index

There are commands that do not need a locator (such as the "open" command). However, most of
them do need Locators.
The choice of locator depends largely on your Application Under Test. In this tutorial, we will
toggle between facebook, new tours.demoaut on the basis of locators that these applications
support. Likewise in your testing project you will select any of the above listed locators based on
your application support.

Locating by ID

This is the most common way of locating elements since ID's are supposed to be unique for each
element.
Target Format: id=id of the element
For this example, we will use Facebook as our test app because Mercury Tours does not use ID
attributes.
Step 1. Navigate to http://www.facebook.com. Inspect the "Email or Phone" text box using Firebug
and take note of its ID. In this case, the ID is "email".

Step 2. Launch Selenium IDE and enter "id=email" in the Target box. Click the Find button and
notice that the "Email or Phone" text box becomes highlighted with yellow and bordered with
green, meaning, Selenium IDE was able to locate that element correctly.

Locating by Name
Locating elements by name are very similar to locating by ID, except that we use the "name="
prefix instead.
Target Format: name=name of the element
In the following demonstration, we will now use Mercury Tours because all significant elements
have names.
Step 1. Navigate to http://newtours.demoaut.com/ and use Firebug to inspect the "User Name" text
box. Take note of its name attribute.

Here, we see that the element's name is "username".

Step 2. In Selenium IDE, enter "name=username" in the Target box and click the Find button.
Selenium IDE should be able to locate the User Name text box by highlighting it.

Locating by Name using Filters

Filters can be used when multiple elements have the same name. Filters are additional attributes
used to distinguish elements with the same name.
Target Format: name=name_of_the_element filter=value_of_filter
Let's see an example Step 1. Log on to Mercury Tours using "tutorial" as the username and password. It should take you
to the Flight Finder page shown below.

Step 2. Using Firebug, notice that the Round Trip and One Way radio buttons have the same name
"tripType." However, they have different VALUE attributes so we can use each of them as our filter.

Step 3.
We are going to access the One Way radio button first. Click the first line on the Editor.
In the Command box of Selenium IDE, enter the command "click".
In the Target box, enter "name=tripType value=oneway". The "value=oneway" portion is
our filter.

Step 4. Click the Find button and notice that Selenium IDE is able to highlight the One Way radio
button with green - meaning that we are able to access the element successfully using its VALUE
attribute.

Step 5. Press the "X" key in your keyboard to execute this click command. Notice that the One Way
radio button became selected.

You can do the exact same thing with the Round Trip radio button, this time, using "name=tripType
value=roundtrip" as your target.

Locating by Link Text

This type of locator applies only to hyperlink texts. We access the link by prefixing our target with
"link=" and then followed by the hyperlink text.
Target Format: link=link_text
In this example, we shall access the "REGISTER" link found in the Mercury Tours homepage.
Step 1.
First, make sure that you are logged off from Mercury Tours.
Go to Mercury Tours homepage.
Step 2.
Using Firebug, inspect the "REGISTER" link. The link text is found between and tags.
In this case, our link text is "REGISTER". Copy the link text.

Step 3. Copy the link text in Firebug and paste it onto Selenium IDE's Target box. Prefix it with
"link=".

Step 4. Click on the Find button and notice that Selenium IDE was able to highlight the REGISTER
link correctly.

Step 5. To verify further, enter "clickAndWait" in the Command box and execute it. Selenium IDE
should be able to click on that REGISTER link successfully and take you to the Registration page
shown below.

Locating by CSS Selector

CSS Selectors are string patterns used to identify an element based on a combination of HTML tag,
id, class, and attributes. Locating by CSS Selector is more complicated than the previous methods,
but it is the most common locating strategy of advanced Selenium users because it can access even
those elements that have no ID or name.

CSS Selectors have many formats, but we will only focus on the most common ones.

Tag and ID
Tag and class
Tag and attribute
Tag, class, and attribute
Inner text

When using this strategy, we always prefix the Target box with "css=" as will be shown on the
following examples.

Locating by CSS Selector - Tag and ID

Again, we will use Facebook's Email text box in this example. As you can remember, it has an ID of
"email" and we have already accessed it in the "Locating by ID" section. This time, we will use a
CSS Selector with ID in accessing that very same element.
Syntax
css=tag#id

tag = the HTML tag of the element being accessed

# = the hash sign. This should always be present when using a CSS Selector with ID
id = the ID of the element being accessed

Keep in mind that the ID is always preceded by a hash sign (#).

Step 1. Navigate to www.facebook.com. Using Firebug, examine the "Email or Phone" text box.
At this point, take note that the HTML tag is "input" and its ID is "email". So our syntax will be
"css=input#email".

Step 2. Enter "css=input#email" into the Target box of Selenium IDE and click the Find button.
Selenium IDE should be able to highlight that element.

Locating by CSS Selector - Tag and Class

Locating by CSS Selector using an HTML tag and a class name is similar to using a tag and ID, but
in this case, a dot (.) is used instead of a hash sign.

Syntax
css=tag.class

tag = the HTML tag of the element being accessed

. = the dot sign. This should always be present when using a CSS Selector with class
class = the class of the element being accessed
Step 1. Navigate to www.facebook.com and use Firebug to inspect the "Email or Phone" text box.
Notice that its HTML tag is "input" and its class is "inputtext".

Step 2. In Selenium IDE, enter "css=input.inputtext" in the Target box and click Find. Selenium
IDE should be able to recognize the Email or Phone text box.

Take note that when multiple elements have the same HTML tag and name, only the first
element in source code will be recognized. Using Firebug, inspect the Password text box in
Facebook and notice that it has the same name as the Email or Phone text box.

The reason why only the Email or Phone text box was highlighted in the previous illustration is that
it comes first in Facebook's page source.

Locating by CSS Selector - Tag and Attribute

This strategy uses the HTML tag and a specific attribute of the element to be accessed.

Syntax
css=tag[attribute=value]
tag = the HTML tag of the element being accessed
[ and ] = square brackets within which a specific attribute and its corresponding value will
be placed
attribute = the attribute to be used. It is advisable to use an attribute that is unique to the
element such as a name or ID.
value = the corresponding value of the chosen attribute.

Step 1. Navigate to Mercury Tours' Registration page

(http://newtours.demoaut.com/mercuryregister.php) and inspect the "Last Name" text box. Take
note of its HTML tag ("input" in this case) and its name ("lastName").

Step 2. In Selenium IDE, enter "css=input[name=lastName]" in the Target box and click Find.
Selenium IDE should be able to access the Last Name box successfully.

When multiple elements have the same HTML tag and attribute, only the first one will be
recognized. This behavior is similar to locating elements using CSS selectors with the same tag and
class.

Locating by CSS Selector - tag, class, and attribute

Syntax
css=tag.class[attribute=value]
tag = the HTML tag of the element being accessed
. = the dot sign. This should always be present when using a CSS Selector with class
class = the class of the element being accessed
[ and ] = square brackets within which a specific attribute and its corresponding value will
be placed
attribute = the attribute to be used. It is advisable to use an attribute that is unique to the
element such as a name or ID.
value = the corresponding value of the chosen attribute.

Step 1. Navigate to www.facebook.com and use Firebug to inspect the 'Email or Phone' and
'Password' input boxes. Take note of their HTML tag, class, and attributes. For this example, we
will select their 'tabindex' attributes.

Step 2. We will access the 'Email or Phone' text box first, thus, we will use a tabindex value of 1.
Enter "css=input.inputtext[tabindex=1]" in Selenium IDE's Target box and click Find. The 'Email or
Phone' input box should be highlighted.

Step 3. To access the Password input box, simply replace the value of the tabindex attribute. Enter
"css=input.inputtext[tabindex=2]" in the Target box and click on the Find button. Selenium IDE
must be able to identify the Password text box successfully.

Locating by CSS Selector - inner text

As you may have noticed, HTML labels are seldom given id, name, or class attributes. So, how do
we access them? The answer is through the use of their inner texts. Inner texts are the actual
string patterns that the HTML label shows on the page.

Syntax
css=tag:contains("inner text")
tag = the HTML tag of the element being accessed
inner text = the inner text of the element

Step 1. Navigate to Mercury Tours' homepage (http://newtours.demoaut.com/) and use Firebug to

investigate the "Password" label. Take note of its HTML tag (which is "font" in this case) and notice
that it has no class, id, or name attributes.

Step 2. Type css=font:contains("Password:") into Selenium IDE's Target box and click Find.
Selenium IDE should be able to access the Password label as shown on the image below.

Step 3. This time, replace the inner text with "Boston" so that your Target will now become
"css=font:contains("Boston")". Click Find. You should notice that the "Boston to San Francisco"
label becomes highlighted. This shows you that Selenium IDE can access a long label even if you
just indicated the first word of its inner text.

Locating by XPath
XPath is the language used when locating XML (Extensible Markup Language) nodes. Since
HTML can be thought of as an implementation of XML, we can also use XPath in locating HTML
elements.
Advantage: It can access almost any element, even those without class, name, or id attributes.

Disadvantage: It is the most complicated method of identifying elements because of too many
different rules and considerations.
Fortunately, Firebug can automatically generate XPath locators. In the following example, we will
access an image that cannot possibly be accessed through the methods we discussed earlier.

Step 1. Navigate to Mercury Tours Homepage and use Firebug to inspect the orange rectangle to the
right of the yellow "Links" box. Refer to the image below.

Step 2. Right click on the element's HTML code and then select the "Copy XPath" option.

Step 3. In Selenium IDE, type one forward slash "/" in the Target box then paste the XPath that we
copied in the previous step. The entry in your Target box should now begin with two forward
slashes "//".

Step 4. Click on the Find button. Selenium IDE should be able to highlight the orange box as
shown below.

6.5.3.5. Selenium IDE Troubleshooting

Below is a list of image/explanation pairs which describe frequent sources of problems with
Selenium-IDE:

Table view is not available with this format.

This message can be occasionally displayed in the Table tab when Selenium IDE is launched. The
workaround is to close and reopen Selenium IDE.
________________________________________________________________________________
error loading test case: no command found
Youve used File=>Open to try to open a test suite file. Use File=>Open Test Suite instead.
________________________________________________________________________________

This type of error may indicate a timing problem, i.e., the element specified by a locator in your
command wasnt fully loaded when the command was executed. Try putting a pause 5000 before
the command to determine whether the problem is indeed related to timing. If so, investigate using
an appropriate waitFor* or *AndWait command before the failing command.
________________________________________________________________________________

Whenever your attempt to use variable substitution fails as is the case for the open command above,
it indicates that you havent actually created the variable whose value youre trying to access. This
is sometimes due to putting the variable in the Value field when it should be in the Target field or
vice versa. In the example above, the two parameters for the store command have been erroneously
placed in the reverse order of what is required. For any Selenese command, the first required
parameter must go in the Target field, and the second required parameter (if one exists) must go in
the Value field.
________________________________________________________________________________
error loading test case: [Exception... Component returned failure code: 0x80520012
(NS_ERROR_FILE_NOT_FOUND)
[nsIFileInputStream.init]
nresult:
0x80520012
(NS_ERROR_FILE_NOT_FOUND) location: JS frame :: chrome://selenium-ide/content/fileutils.js :: anonymous :: line 48 data: no]
One of the test cases in your test suite cannot be found. Make sure that the test case is indeed
located where the test suite indicates it is located. Also, make sure that your actual test case files
have the .html extension both in their filenames, and in the test suite file where they are referenced.
________________________________________________________________________________

Your extension files contents have not been read by Selenium-IDE. Be sure you have specified the
proper pathname to the extensions file via Options=>Options=>General in the Selenium Core
extensions field. Also, Selenium-IDE must be restarted after any change to either an extensions file
or to the contents of the Selenium Core extensions field.