SOFTWARE TESTING

(ITS 64704)
Dynamic Testing – Black Box
(Part I)
Dynamic Testing – Fundamentals

Topics
Idea of the Black Box Test Design Techniques

Equivalence Partitioning

Boundary Value Analysis

State Transition Testing

Decision Table Testing

Further Black Box Test Design Techniques

Dynamic Testing – Fundamentals

Topics
Idea of the Black Box Test Design Techniques

Equivalence Partitioning

Boundary Value Analysis

State Transition Testing

Decision Table Testing

Further Black Box Test Design Techniques

 Dynamic Testing

Dynamic Testing - terms

• Testing that involves the execution of software or component of a system

Test basis
• All documents from which the requirement of a component or a system can be inferred.
• The document on which the test cases are based

Test condition
• An item or event e.g., a function, transaction, feature, quality attribute, or structural element
of a component or system that could be verified by one or more test cases

Test design specification

• A document specifying the test conditions (coverage items) for a test item, the detailed test
approach and identifying the associated high level test cases
 Test case
• Contains the following information:
• for the execution necessary preconditions,
Dynamic Testing - terms
• a set of input values (one input value per parameter of the test object),
• the amount of expected results, as well as
• the expected postconditions.
• Test cases are developed for a particular objective or test condition, such as to exercise
a particular program path or to verify compliance with a specific requirement

Pre condition
• Environmental and state conditions that must be fulfilled before the test object can be
executed with a particular test or test procedure

Post condition
• State conditions (and/or environmental conditions) that must be fulfilled on a test
object after the execution of a test or test procedure

Test case specification

• A document specifying a set of test cases (objective, inputs, test actions, expected
results, and execution preconditions) for a test item
Dynamic Testing Relationship
Dynamic Testing Procedures
 Dynamic Testing Procedures
1) Design of tests to determine test conditions
• Analysis of test basis documents is used to identify what needs to be tested
and to establish the test conditions.
• Test conditions must be traceable to specifications or standards, as it could
determine the impact of changes on specifications or standards (coverage)
on the test (impact analysis).
Dynamic Testing Procedures
2) Dynamic
Specifying test case
Testing Procedures
• Development and detailed description of test cases and test data using the
chosen test design technique.
• Predicted (expected) results include outputs, changes to data and states and
any other consequences to the test.
• If expected results can‘t be defined, then a plausible but erroneous result
could get interpreted as the correct one.
• Expected results should be defined prior to test execution!
Dynamic Testing Procedures
3) Construct the test procedure specifications
Dynamic Testing Procedures
• Bring test cases into an executable sequence, while
considering their pre-and post-conditions.
• Test procedure specifications or test scripts define the
sequence of activities for the execution of a test
• When a test execution tool is used, the test scripts are
automated
Dynamic Testing Procedures
4) Test execution schedule
• Dynamic
Describe theTesting
sequence inProcedures
which the different test
procedure specifications or test scripts (automated)
are to be executed, and also when and by whom will it
be executed.
• Factors like regression tests (repeated test execution),
prioritizing and logical correlations between test cases
or test scripts are to be considered.
• Verification of the
Dynamic Test in Early Test Levels

adherence of the test

object to the defined
standards by executing the
program
• Detecting possible
deviations and defects
• Checking the highest
possible amount of
standards or detecting
defects with minimum
effort
dynamic testing
Objectives of
 Dynamic Test in Early Test Levels

• The execution of dynamic tests requires the execution of the test object.
• To run the test, an executable program must be at hand!
• Separate classes, modules/components and system parts are generally not
executable by themselves
• They often provide operations or services for other software items
• They generally have no “main program“ to coordinate the procedure calls
• They often rely on the operations or services of other software items
• In the early test levels of class/module tests, component tests and integration
tests, the test object must often be embedded in a “test harness“

Test Harness: A test environment comprised of stubs and drivers needed to

execute the test
Test Design Techniques
Black Box vs White Box Testing
Black Box vs White Box Testing
 Black Box Testing

• Synonym: Specification-based Test Design Technique

• No information available about the program code or the internal structure
• The behaviour of the test object is observed externally (PoO - Point of
Observation is outside the test object)
• Control of the test object is only possible by means of the choice of input test
data (PoC - Point of Control is outside the test object)
• Models of the test object, either formal or informal, are used for the
specification of the problem to be solved, of the software or its components.
• Test cases can be derived systematically from these models.
White Box Testing

• Synonym: Structure-based Test Design Technique

• Information about how the software is constructed is used to derive
the test cases, e.g., the code and the algorithm
• The level of code coverage can be measured for existing test cases
• Further test cases can be derived systematically to increase coverage
• During test case execution, the inner procedure of the test object is
analysed (Point of Observation is inside the test object)
• An intervention in the execution of the test object is possible, e.g.,
when for negative tests a defective control sequence cannot be
achieved (Point of Control can be inside the test object)
Experience based Testing

• Uses knowledge and experience of people to derive test cases

• Knowledge of testers, developers, users and others involved with the
software, its usage and its environment
• Knowledge about likely defects and their distribution
Dynamic Testing – Fundamentals

Topics
Idea of the Black Box Test Design Techniques

Equivalence Partitioning

Boundary Value Analysis

State Transition Testing

Decision Table Testing

Further Black Box Test Design Techniques

 Functional testing

Specification based
• Dynamic testing, in which Testing:
the test cases are derivedTerms
using the
functional specifications of the test object and the
completeness of the testing (coverage) is evaluated according
to the functional specifications
• The capability of the software product to provide functions
which meet stated and implied needs when the software is
used under specified conditions [ISO 9126].
• Sub characteristics of functionality after ISO 9126 are: accuracy,
suitability, interoperability, security, conformance

quality
Specification based Test Case and
Test Data Definition
Dynamic Testing – Fundamentals

Topics
Idea of the Black Box Test Design Techniques

Equivalence Partitioning

Boundary Value Analysis

State Transition Testing

Decision Table Testing

Further Black Box Test Design Techniques

 Equivalence Partitioning

• The domain of definition of input and output are divided (partitioned) into
equivalence classes, such that all values within a class represent a similar behavior of
the test object.
• Assumptions:
• If a value chosen from the equivalence class detects a defect, it is assumed that
this defect will also be detected for every other value in the equivalence class.
• If a value chosen from the equivalence class does not detect a defect, it is
assumed that no other value from the equivalence class will detect a defect.
 Equivalence Partitioning: Procedure

• Determine input and output domains from the specification

• Perform equivalence partitioning for each of the following:
• If an area of validity is specified, then: define one valid and two invalid
equivalence classes
• If a minimum and a maximum value is specified, then: define one valid and two
invalid equivalence classes
• If a number of values are specified which would not be treated differently,
then: define an valid equivalence class for those values and, in addition, define
one invalid equivalence class
• If a “true“ and “false“ situation is specified, then: define equivalence for the
“true” and “false” conditions
• Remember, if values within an equivalence class will likely not be treated the same,
then we must split up the equivalence class into smaller ones where the values really
are equivalent.
 Equivalence Partitioning: Example

• If an area of validity is specified, then: define one valid and two invalid equivalence
classes

The specification of the test object defines an input integer with possible
values from 1 to 100

• What would be the equivalence classes?

 Equivalence Partitioning: Example

• If an area of validity is specified, then: define one valid and two invalid equivalence
classes

The specification of the test object defines an input integer with possible
values from 1 to 100

Invalid Valid Invalid

(<1) 1-100 >100
Data: 0, -1, -2, -3, ….. Data: 1, 3, 14,…100 Data: 101, 112, 203,…….
 Equivalence Partitioning: Example

• If a minimum and a maximum value is specified, then: define one valid and two
invalid equivalence classes

Specification: A member of the sports club has to register himself, at least one
type of sport. A member can only be registered for a maximum of three types
of sport.
 Equivalence Partitioning: Example

• If a minimum and a maximum value is specified, then: define one valid and two
invalid equivalence classes

Specification: A member of the sports club has to register himself, at least one
type of sport. A member can only be registered for a maximum of three types
of sport.

Invalid Valid Invalid

<1 1-3 >3
Data: 0, -3 Data: 1,2,3 Data: 4, 25, …
 Equivalence Partitioning: Example

• If a number of values are specified which would not be treated differently, then:
define an valid equivalence class for those values and, in addition, define one invalid
equivalence class

Specification: The sports club offers following sports: Football, Hockey,

Handball, Basketball and Volleyball.
 Equivalence Partitioning: Example

• If a number of values are specified which would not be treated differently, then:
define an valid equivalence class for those values and, in addition, define one invalid
equivalence class

Specification: The sports club offers following sports: Football, Hockey,

Handball, Basketball and Volleyball.

Valid Invalid
Football, Hockey, Handball, Basketball, Anything else than valid value:
Volleyball e.g Badminton, Swimming, etc
 Equivalence Partitioning: Example

• If a “true“ and “false“ situation is specified, then: define equivalence for the “true”
and “false” conditions

Specification: Every member of the sports club received a unique member ID. It
must start with the first letter of the family name of the member.
 Equivalence Partitioning: Example

• If a “true“ and “false“ situation is specified, then: define equivalence for the “true”
and “false” conditions

Specification: Every member of the sports club received a unique member ID. It
must start with the first letter of the family name of the member.

Valid (True) Invalid (False)

First digit is a letter First digit is not a letter
e.g N, P H e.g numbers/ special characters, etc
 Minimizing the number of test case

• Combine test cases of all representations and sort them afterwards

according to “frequency of use” (“Usage profiles”)
• Prioritizing test cases according to this sequence
• Select only test cases which are “usage relevant”
• Prefer test cases which contain boundary values or combinations of
these
• Ensure that each representation of an equivalence class is combined with
one other equivalence class in a test case
• That means, we define paired combinations instead of all possible
combinations
• Minimal criteria: At least one representation of an equivalence class in at
least one test case
• Representations of invalid equivalence classes must not to be combined
with representations of other invalid equivalence classes (to avoid “defect
masking”). one test case - 2/3/4/5 invalid test data
Equivalence partitioning: Defect
masking
 Equivalence partitioning: Exit criteria

• A specific exit criteria can be defined based on the number of equivalence classes
represented in tests:

• Example:
• If 18 equivalence classes have been determined from the statements or
specifications of the input data, and if only 15 of these have been tested in test
cases, an equivalence class coverage of 83 % has been reached.
• Equivalence class coverage = (15 / 18) * 100% = 83.33 %
Equivalence partitioning: Minimal test
case

• All equivalence classes are to covered by at least one test case

• In each test case
• cover several valid equivalence classes, or
• exactly one invalid equivalence class (i.e. each one in a
separate test)
 Benefits Disadvantages
Equivalence partitioning
• Amount of test cases is • Observes conditions for
smaller as compared to separate input and output
exhaustive testing parameters
• Appropriate for programs • Observation of interactions
with many input and output and dependencies of
parameters conditions is very complex

Recommendations :
To choose effective test data for defect finding, combine Equivalence
Partitioning with Boundary Value Analysis
Dynamic Testing – Fundamentals

Topics
Idea of the Black Box Test Design Techniques

Equivalence Partitioning

Boundary Value Analysis

State Transition Testing

Decision Table Testing

Further Black Box Test Design Techniques

 Boundary Value Analysis (BVA)

• Idea: When programming branching- and loop-conditions, there are often

conditions which are still “just” applicable or are “just not” applicable. These
are called “boundary conditions”.
• Such boundary conditions can be error-prone (“off by one”).
• E.g. One time too many through a loop.
• Test data specified to test boundary values, find more defects than test
data which,
• for example, test equivalent partitions.
• Best success in combination with other techniques
• Frequently used in combination with equivalence partitioning
• Testing the boundaries of the equivalence class (highest and lowest value)
• Generally test the boundary value itself and the values directly above or below
Just below minimum value
Boundary
Minimum value
Value Analysis (BVA)
Maximum value
• For each equivalence partitioning classes (range of classes), you must identify
Just above maximum value
value from the following:
Boundary Value Analysis (BVA):
Example
Let's consider the behavior of tickets in the Flight reservation application, while booking a
new flight.

Test condition:
1. Any Number greater than 10 entered
in the reservation column is
considered invalid.
2. Any Number less than 1, then it is
considered invalid.
3. Numbers 1 to 10 are considered valid
4. Any 3 Digit Number say -100 is invalid
Boundary Value Analysis (BVA):
Example
• We cannot test all the possible values because if done, the number of test cases will be
more than 100.
• To address this problem, we use equivalence partitioning hypothesis where we divide the
possible values of tickets into groups or sets as shown below where the system behavior
can be considered the same.

<1 1-10 >10 – 99

=>100
(2 digit)
Boundary Value Analysis (BVA):
Example
• Apply Boundary Values Analysis to test boundaries between equivalence partitions.

Maximum Maximum
value value Just above
Just below Minimum Minimum
maximum
minimum value value value
value
Boundary Value Analysis (BVA):
Example
• Apply Boundary Values Analysis to test boundaries between equivalence partitions.

Test scenario Expected outcome

test data
Boundary value = 0 System SHOULD NOT accept.
Boundary value = 1 System SHOULD accept.
Boundary value = 10 System SHOULD accept.
Boundary value = 11 System SHOULD NOT accept.
Boundary value = 99 System SHOULD NOT accept.
Boundary value = 100 System SHOULD NOT accept.
 Boundary Value Analysis (BVA): Example2

• A text box has a minimum character limit of 4 and maximum character limit of
10
• Write BVA test cases for this example
 Boundary Value Analysis (BVA): Example2

• A text box has a minimum character limit of 4 and maximum character limit of
10
• Write BVA test cases for this example

Invalid Valid Invalid

<4 4-10 >10

• STEP 1: Segregate based on equivalence partitioning classes

 Boundary Value Analysis (BVA): Example2

Invalid Valid Invalid

<4 4-10 >10
3 4 10 11

• STEP 2: Apply Boundary Values Analysis to test boundaries between equivalence

partitions. min and max - apply BVA
Test scenario Expected outcome
Boundary value = 3 System SHOULD NOT accept.
Boundary value = 4 System SHOULD accept.
Boundary value = 10 System SHOULD accept.
Boundary value = 11 System SHOULD NOT accept.
You will have 4 test cases/test data
 Boundary Value Analysis
Benefits Disadvantages

• Defects are more often found at the • Procedures for the choice of test data
boundaries of equivalence classes than are not easily provided
inside these classes • Definition of all relevant boundary
• The Boundary Analysis is, if used values is difficult
correctly, one of the most useful • Creativity required for finding
methods for the test case design successful test data
• Efficient combinations with other • Often not efficiently enough applied, as
techniques, which permit a certain it seems to be too simple
degree of choice regarding test data
 (tbc)
THANK YOU