Course Title: Software Engineering

Segment -5
Course Code: CSE-3505

Topic
Validation and Verification

Verification and Validation is the process of investigating that a software system satisfies
specifications and standards and it fulfills the required purpose. Barry Boehm described
verification and validation as the following:

Validation: Are we building the right product?

Verification: Are we building the product right?

What is Validation?
Definition: The process of evaluating software during or at the end of the development process
to determine whether it satisfies specified requirements.
Validation is the process of evaluating the final product to check whether the software meets the
customer expectations and requirements. It is a dynamic mechanism of validating and testing the
actual product.
Methods of Validation : Dynamic Testing

• Testing

What is Verification?
Definition : The process of evaluating software to determine whether the products of a given
development phase satisfy the conditions imposed at the start of that phase.

Verification is a static practice of verifying documents, design, code and program. It includes all
the activities associated with producing high quality software: inspection, design analysis and
specification analysis. It is a relatively objective process.
Verification will help to determine whether the software is of high quality, but it will not ensure
that the system is useful. Verification is concerned with whether the system is well-engineered
and error-free.
Methods of Verification : Static Testing
• Walkthrough
• Inspection
• Review
Differences between Verification and Validation

Criteria Verification Validation

Definition The process of evaluating work- The process of evaluating software

products (not the actual final during or at the end of the
product) of a development phase to development process to determine
determine whether they meet the whether it satisfies specified
specified requirements for that business requirements, needs or
phase. expectations.

Objective To ensure that the product is being To ensure that the product actually
built according to the requirements meets the user’s needs and that the
and design specifications. In other specifications were correct in the
words, to ensure that work products first place. In other words, to
meet their specified requirements. demonstrate that the product fulfills
its intended use when placed in its
intended environment.

Question Are we building the product right? Are we building the right product?

Evaluation Plans, Requirement Specs, Design The actual product / software.

Items Specs, Code, Test Cases

Activities Reviews
Walkthroughs Testing
Inspections

What is Software Testing?

Software Testing is the process of identifying the correctness and quality of software programs.
The purpose is to check whether the software satisfies the specific requirements, needs, and
expectations of the customer. In other words, testing is executing a system or application in order
to find software bugs, defects or errors. The job of testing is to find out the reasons for
application failures so that they can be corrected according to requirements.
Example:
Car manufacturer tests the car for maximum speed, fuel efficiency and safety from crash.These
test later become the part of advertising strategy for car sales.

Roles And Responsibilities Of A Tester

Many people are not completely aware of the roles and responsibilities of a software tester.
Below, you’ll learn a great deal more about these roles and responsibilities.• Analyzing the SRS,
System Requirement Specifications, and understanding the necessities
• Determining an estimation for the testing
• Preparing or understanding the test plan
• Putting together test cases
• Assembling test data
• Testing and finding defects in the software
• Reporting the defects immediately to make it easier for the developer to fix
• Testing again after the initial problem has been fixed
• Carrying out regression testing
• Offering suggestions for improving SDLC processes
• Delivering support to customers testing the software
• Partaking in the implementation of the software
• Providing support after the implementation is completed

The importance of software testing is immense. This is why more and more companies will
continue to spend more money on software testing. In fact, it could soon become of the greatest
expense that any technology company will incur. Nevertheless, it is also true that the costs will
be well worth it in the long run.

Ways of Software Testing

• Manual Testing: Test Cases executed manually.
• Automation Testing: Testing performed with the help of automation tools.

Testing Objectives

➢ Testing is a process of executing a program with the intent of finding errors.

➢ A good test case is one that has a high probability of finding an as-yet undiscovered error.
➢ A successful test is one that uncovers an as-yet undiscovered error.
➢ Testing cannot show the absence of defects, it can only show that software errors are present.

Testing Process

Testing is a complete process. For testing we need two types of inputs.First is software
configuration. It includes software requirement specification, design specifications and source
code of program.Second is test configuration. It is basically test plan and procedure.

Software configuration is required so that the testers know what is to be expected and tested
whereas test configuration is testing plan that is, the way how the testing will be conducted on
the system. It specifies the test cases and their expected value. It also specifies if any tools for
testing are to be used. Test cases are required to know what specific situations need to be tested.
When tests are evaluated, test results are compared with actual results and if there is some error,
then debugging is done to correct the error. Testing is a way to know about quality and
reliability. Error rate that is the occurrence of errors is evaluated. This data can be used to predict
the occurrence of errors in future.
 Fig . Testing Process

Test Case design

We now know, test cases are integral part of testing. So we need to know more about test cases and
how these test cases are designed. The most desired or obvious expectation from a test case is that it
should be able to find most errors with the least amount of time and effort.

A software product can be tested in two ways. In the first approach only the overall functioning of the
product is tested. Inputs are given and outputs are checked. This approach is called black box testing.
It does not care about the internal functioning of the product.

The other approach is called white box testing. Here the internal functioning of the product is tested.
Each procedure is tested for its accuracy. It is more intensive than black box testing. But for the
overall product both these techniques are crucial. There should be sufficient number of tests in both
categories to test the overall product.

Types of testing

There are two basic types of testing

Black-box testing

Black-box testing (also known as functional testing) treats software under test as a black-box
without knowing its internals. Tests are using software interfaces and trying to ensure that they
work as expected. As long as functionality of interfaces remains unchanged, tests should pass
even if internals are changed. Tester is aware of what the program should do but does not have
the knowledge of how it does it. Black-box testing is most commonly used type of testing in
traditional organizations that have testers as a separate department, especially when they are not
proficient in coding and have difficulties to understand the code. It provides external
perspective of the software under test.
 Some of the advantages of black-box testing are:

1. Efficient for large segments of code.

2. Code access is not required.
3. Separation between user’s and developer’s perspectives.

Some of the disadvantages of black-box testing are:

1. Limited coverage since only a fraction of test scenarios is performed.

2. Inefficient testing due to tester’s luck of knowledge about software internals.
3. Blind coverage since tester has limited knowledge about the application.

White-box testing
White-box testing (also known as clear box testing, glass box testing, transparent box testing,
and structural testing) looks inside the software that is being tested and uses that knowledge as
part of the testing process White-box testing requires internal knowledge of the system and
programming skills. It provides internal perspective of the software under test.

Some of the advantages of white-box testing are:

1. Efficient in finding errors and problems.

2. Required knowledge of internals of the software under test is beneficial for thorough
testing.
3. Allows finding hidden errors.
4. Programmers introspection.
5. Helps optimizing the code.
6. Due to required internal knowledge of the software, maximum coverage is obtained.

Some of the disadvantages of white-box testing are:

1. Might not find unimplemented or missing features.

2. Requires high level knowledge of internals of the software under test.
3. Requires code access.
Basic Path Testing

Basic path testing a white box testing technique .It was proposed by Tom McCabe.These tests
guarantee to execute every statement in the program at least one time during testing. Basis set is
the set of all the execution path of a procedure.

Flow graph Notation

Before basic path procedure is discussed, it is important to know the simple notation used for the
representation of control flow. This notation is known as flow graph. Flow graph depicts control
flow and uses the following constructs.

These individual constructs combine together to produce the flow graph for a particular
procedure

Sequence - Until -

If - While - Case -

Basic terminology associated with the flow graph

 Node: Each flow graph node represents one or more procedural statements. Each node that
contains a condition is called a predicate node.
Edge: Edge is the connection between two nodes. The edges between nodes represent flow of
control. An edge must terminate at a node, even if the node does not represent any useful
procedural statements.
Region: A region in a flow graph is an area bounded by edges and nodes.

Cyclomatic Complexity
Cyclomatic complexity: Independent path is an execution flow from the start point to the end
point. Since a procedure contains control statements, there are various execution paths depending
upon decision taken on the control statement. So Cyclomatic complexity provides the number of
such execution independent paths. Thus it provides a upper bound for number of tests that must
be produced because for each independent path, a test should be conducted to see if it is actually
reaching the end point of the procedure or not.
Cyclomatic Complexity for a flow graph is computed in one of three ways:
1. The numbers of regions of the flow graph correspond to the Cyclomatic complexity.
2. Cyclomatic complexity, V(G), for a flow graph G is defined as
V(G) = E – N + 2
where E is the number of flow graph edges and N is the number of flow graph nodes.
3. Cyclomatic complexity, V(G), for a graph flow G is also defined as
V(G) = P + 1
Where P is the number of predicate nodes contained in the flow graph G.
Example: Consider the following flow graph
Region, R= 6
Number of Nodes = 13
Number of edges = 17
Number of Predicate Nodes = 5

Cyclomatic Complexity, V( C) :

V( C ) = R = 6;
Or

V(C) = Predicate Nodes + 1

=5+1 =6
Or

V( C)= E-N+2
= 17-13+2

Deriving Test Cases

The main objective of basic path testing is to derive the test cases for the procedure under test.
The process of deriving test cases is following

1. From the design or source code, derive a flow graph.

2. Determine the Cyclomatic complexity, V(G) of this flow graph using any of the formula
discussed above.
· Even without a flow graph, V(G) can be determined by counting the number of conditional
statements in the code and adding one to it.
3. Prepare test cases that will force execution of each path in the basis set.
· Each test case is executed and compared to the expected results.
 N1 N2 N3 N4 N5 N6 N7 N8 N9 N10 N11 N12 N13
0 1 N1 1
1 2 N2 1 1
0 1 N3 1
0 1 N4 1
1 2 N5 1 1
1 2 N6 1 1
0 1 N7 1
0 1 N8 1
0 1 N9 1
0 1 N10 1
1 2 N11 1 1
1 2 N12 1 1
+ N13
1
=6
Testing Documentation

There are two aspects to test documentation

The Test Plan- This can start to be developed at the analysis stage and can further
refined during design and implementation

The Test Log- This is a log that records the tests applied and the results of each test. For
a module this may look the table on the following page

Test Log (for classify)

Condition Expected Result Actual Result Remarks

a=3, b=3, c=4 True True Pass
a=10, b=2, c=7 False True Problem bad return from
order
a=-7, b=3, c=6 False False Pass

Validation Testing

After the integration testing we have an assembled package that is free from modules and
interfacing errors. At this stage a final series of software tests, validation testing begin.Validation
succeeds when software functions in a manner that can be expected by the customer.

Major question here is what are expectations of customers. Expectations are defined in the software
requirement specification identified during the analysis of the system. The specification contains a
section titled “Validation Criteria” Information contained in that section forms the basis for a
validation testing.

Software validation is achieved through a series of black-box tests that demonstrate conformity with
requirements. There is a test plan that describes the classes of tests to be conducted, and a test
procedure defines specific test cases that will be used in an attempt to uncover errors in the
conformity with requirements.

After each validation test case has been conducted, one of two possible conditions exists:

The function or performance characteristics conform to specification and are accepted, or

A deviation from specification is uncovered and a deficiency list is created. Deviation or error
discovered at this stage in a project can rarely be corrected prior to scheduled completion. It is often
necessary to negotiate with the customer to establish a method for resolving deficiencies
 Testing with Customers

The developers of software will not always foresee exactly how the customer will use
their software. Therefore testing with the customer is important. When the software is a
custom product for one specific product a series of Acceptance Tests are undertaken
where the customer determine whether his/her requirements are satisfied ranges from a
‘test-drive’ to systematic testing procedure. Where software is to be used with many
customers, the developer must test with a representative sample. Most developers use
alpha and beta testing-

Alpha Test

Alpha testing is a type of acceptance testing; performed to identify all possible issues/bugs
before releasing the product to everyday users or public. Alpha testing is carried out in a lab
environment and usually the testers are internal employees of the organization. To put it as
simple as possible, this kind of testing is called alpha only because it is done early on, near the
end of the development of the software, and before beta testing
-Customer is taken developer’s site and asked to use the software with developer watching and
recording problems any errors

Beta Test
Beta Testing of a product is performed by "real users" of the software application in a "real
environment" and can be considered as a form of external User Acceptance Testing.

Beta version of the software is released to a limited number of end-users of the product to obtain
feedback on the product quality. Beta testing reduces product failure risks and provides increased
quality of the product through customer validation.

It is the final test before shipping a product to the customers. Direct feedback from customers is amajor
advantage of Beta Testing. This testing helps to tests the product in real time environment.

• Number of customers use the software on

their sites

• Developer is not present

• Customer records problems

Alpha Testing Vs Beta testing:

Following are the differences of Alpha and Beta Testing:

Alpha Testing Beta Testing

Alpha testing performed by Testers who are usually Beta testing is performed by Clients or End
internal employees of the organization Userswho are not employees of the
organization

Alpha Testing performed at developer's site Beta testing is performed at client location or
enduser of the product

Reliability and Security Testing are not performed Reliability, Security, Robustness are checked
in-depth Alpha Testing duringBeta Testing

Alpha testing involves both the white box and black Beta Testing typically uses Black Box Testing
box techniques

Alpha testing requires lab environment or testing Beta testing doesn't require any lab environment
environment or testing environment. Software is made
available to the public and is said to be real time
environment

Critical issues or fixes can be addressed by Most of the issues or feedback is collected from
developers immediately in Alpha testing Beta testing will be implemented in future
versionsof the product

Top down and bottom up approach of software testing

Top down Testing: In this approach testing is conducted from main module to sub
module. ifthe sub module is not developed a temporary program called STUB is used
for simulate the sub module.

Advantages:
-Advantageous if major flaws occur toward the top of the program.
-Once the I/O functions are added, representation of test cases is easier.
-Early skeletal Program allows demonstrations and boosts morale.

Disadvantages:
- Stub modules must be produced.
- Stub Modules are often more complicated than they first appear to be.
- Before the I/O functions are added, representation of test cases in stubs can be difficult.
- Test conditions may be impossible, or very difficult, to create.
- Observation of test output is more difficult.
- Allows one to think that design and testing can be overlapped.
- Induces one to defer completion of the testing of certain modules.

Bottom up testing: In this approach testing is conducted from sub module to main
module, if the main module is not developed a temporary program called DRIVERS
is used to simulate the main module.

Advantages:
- Advantageous if major flaws occur toward the bottom of the program.
- Test conditions are easier to create.
- Observation of test results is easier.

Disadvantages:
- Driver Modules must be produced.
The program as an entity does not exist until the last module is added.

The basic differences between top down and bottom up approach of software testing are:

No. Top down approach Bottom up approach

1. Top down testing conducted from Bottom up testing conducted from sub module to
main module to sub module. main module.
2. If sub module is not developed a If main module is not developed a temporary
temporary program called Stub is used program called Driver is used to simulate the
to simulate the sub module. main module.
3. Top down testing good if major flaws Bottom up testing good if major flaws occur
occur toward the top of the program. toward the bottom of the program.
4. In this test condition difficult to create. In this test condition easy to
create.
5. Observation of test output is more Observation of test output is
easier.
difficult.
Figure: Creating stubs and drivers to do integration tests for C