Chapter 6

Testing Object Oriented Programs

6.1 Testing Object Oriented Programs
The main advantages of Object Oriented development are in the modelling, design and development of systems. Some of the key ideas that make object oriented systems development so successful are the following. A. The language in which object oriented systems are modelled and designed is very close to the language in which objects oriented systems are implemented. In practice this often means a smaller semantic gap between design model and implementation. Systems are organised into objects that encapsulate data and operations. Dividing systems into objects allows developers to: divide the system up into intellectually more manageable components that are easier understand, implement and modify; separate the concerns between the provider of services and the clients of those services by providing a clear and unambiguous interface between client objects and objects that provide services; provide sound semantic structuring principles such as inheritance, aggregation and associations between objects so that systems can be easily built out of smaller components; C. Objects and object hierarchies are veried components that can be reused between projects and even sold on the market. This nal point is a key point to understanding why validating, verifying and testing object oriented components is so important. This is a point that we will explore later in the chapter. The chief question that concerns us is what is the impact of Object Oriented programs on testing and how to assure components.

B.

69

6.2 Object-Oriented Programming Languages

Without looking too deeply at object oriented programming we will briey review the essentials of object oriented programming languages in order to explore the impact on testing later. Object Oriented programming languages support a number of features that are aimed at making the design, maintenance and reuse of code much easier. (1) Object Oriented programming languages support the use of Abstract Data Types for: Encapsulating data and the operations that create, update, or examine that data; Information hiding by hiding the details of the implementation of a data type and providing a clear and unambiguous interface through which data can be accessed and manipulated. (2) Object Oriented programming languages support Inheritance, Aggregation and Associations as structuring mechanism for programmers to create large systems out of smaller, more manageable components. (3) Object Oriented programming languages supports reuse through structuring mechanisms by: helping to facilitate changes to a set of objects or classes, for example, in the case of inheritance changes to a parent class are reected in all of its child classes; allowing programs to use existing components through dynamic binding (polymorphism).

Inheritance, Aggregation and Associations

In object oriented languages each class denes a Type. Whenever a new object is created it is an Instance, or element, of that type. Every object in the class has: The Methods dened by the class; and The Instance Variables (or attributes) dened by the class; There my be multiple instances of an object each with its own instance variables but whatever the case it will have the methods and attributes dened by its class. Also, each of the methods of an object has access to the instance variables of that object and different methods in an object can interact through the instance variables. By this we mean that one method can dene instance variables, or give instance variables their values, while another method relies on those values. For example, the class Shape in Figure 6.1 denes a type where each object of that type contains a private hidden instance variable origin and a publicly visible method Area(). Generalisation, or Inheritance, relationships can exist between classes. The parent is the more general class providing fewer methods or more general methods while the child specialises the parent. A child child class inherits all of the properties of its parent but child class may override operations in the parent and extend the parent by adding more attributes and operations.

70

POINT CLIENT
Client Uses

SHAPE
Data M -origin: 1 M

-point: POINT +setPoint(in X:float,in Y:float): void +getX(): float +getY(): float

+clientFunction(in S:SHAPE): void N

POINT

+Area(): float

Polygon
-points: [] POINT +setPoints(in pointVector:[] POINT): void +getPoints(): [] POINT +Area(): float

CURVE
-points: [] POINTS +setPoints(points:[] POINTS): void +getPoints(): [] POINTS +Area()

RECTANGLE
+setTopLeft(in p:POINT): void +setBottomRight(in p:POINT): void +Area(): float

TRIANGLE
+setFirst(in p:POINT): void +setSecond(in p:POINT) +setThird(in p:POINT)

ELIPSE
-Minor: float -Major: float +Area(): float

CIRCLE
-Radius: float +Area(): float

SQUARE
+setSideLength(in length:float): void

Figure 6.1: Inheritance and polymorphism.

71

In particular, generalisation means that the objects of the child class can be used anywhere that the objects of the parent class can be used but not the converse! In the case of Single Inheritance a class may only inherit from from one, and only one, parent class. In the case of Multiple Inheritance a class may inherit from one or more parent classes. The parent class is called the Superclass and the child is called the Subclass. Figure 6.1 shows an inheritance hierarchy with the class Shape at the base of the hierarchy. Any of the classes that have Shape as the superclass can be used anywhere that Shape can be used. For example, the class Client has a method called clientFunction that takes an object of type Shape as a parameter. However, the actual parameter can be an object of type Shape, and object of type Polygon, an object of type Circle or indeed an object of any type that has Shape as one of its ancestors. This is the essence of Dynamic Binding, or as it is often called Polymorphism in object oriented languages. Associations are another kind of structural relationship that can occur between classes. If two classes have an association between them then you can Navigate between objects of those classes. An associations can between classes A and B can be implemented by: dening the instance variables of a class A to be of type B; passing objects of class A to the methods of class B; or sending messages between A and B.

WHOLE

PART

Figure 6.2: The Part-WHOLE relationship. Aggregation is an association between classes and objects that relates parts to their whole. An aggregation relation as in Figure 6.2 is implemented when objects of the class PART are stored in objects of the class WHOLE. The other relationships are typically between peers but the the aggregation relationship introduces a strict hierarchy between the whole and its parts.

6.3 Testing Object Oriented Programs

In Chapters 3, 4 and 5 we dealt primarily with procedural programs. In procedural languages the basic components for unit testing are functions or procedures (sometime referred to as subroutines). The testing methods that we looked at relied on an analysis of the ) input and output domains regardless of whether they were black box or white box and execution based tests required that we execute the function or procedure to compare actual results with expected results. In object oriented testing where classes are: class = object state + set of operations

72

the internal states of the object become relevant to the testing. As a consequence, the correctness of an object is not based only on the output given by method calls, but also on the internal state of the object. Methods are meaningless when separated from their class and treated in isolation. For example, a method may require an object to be in a specic state before it can be executed, where that state can only be set by another method (or combination of methods) in the class. Classes are thus the natural unit for testing Object Oriented Programs However, once this path has been taken then we will need to explore the effects of object oriented testing in the presence of: A. B. C. D. Encapsulation; Generalisation; Polymorphism; and Other associations.

The Implications for Encapsulation

The rst thing to note is that encapsulation is not usually a source of errors more often its the converse but it may be an obstacle to testing. The problem is that to test a class that encapsulates both data and operations, some of which may be private. In testing we will need to examine the concrete state of the object and test the results of private methods. This often means having to break the encapsulation for testing! The are a number of avenues open to us to break the encapsulation of a class. Language Features Languages like C++ explicitly provide language constructs such the friend construct for limited access to private variables and methods. For example, in C++ a class B can be declared as a friend to a class A in which case it has access to the private methods and instance variables. We can now create testing classes and declare them as friends of the classes under test. Debugging Tools and Low Level Probes A number of tools allow you to step through the execution of a test case (slow and painful) or to execute a program on test data until a specic point, sometimes called a Breakpoint, is reached. You can then dump object state to a le and examine it either manually or with other tools. Instrumenting the Code You can also embed debugging code into the actual class to: 1. dump the state and the results of private method calls to a le; or 2. to act as a testing oracle on the state of an object or the results of private method calls. Instrumenting the code in such a way is extremely useful for debugging the system throughout the life of the system and especially when running regression tests and performing system evolution.

73

Abstract Operation Sequences An Alternative to Breaking Encapsulation

It may not always be possible to break the encapsulation of an class. For example, if all that you have is access to compiled libraries and their interfaces then it may not be possible to step through the execution of class methods in any intelligible, or timely, way. The alternative is to construct meaningful sequences of operations calls based on the public interface for the class. Typically, this involves creating scenarios that exercise sequences of operation calls. The expected results for the sequence of method calls necessarily needs to be a value or object that is visible to us and whose correctness can be determined relatively easily (but this is not always possible in practice). A variation on this idea is to look at a two different of sequences of method calls that should return the same visible result. In this approach the object state is implicitly tested via the access methods. We will return to this point later in state based testing methods.

Testing Inheritance Hierarchies

Inherited features often require re-testing, because every time class inherits from its parent the state and operations of the parent are placed into new context the context of the child. Multiple inheritance complicates this situation by increasing the number of contexts to test. Ideally, an inheritance relationship should correspond to a problem domain specialisation, for example, from Figure 6.1 a Polygon is a special kind of Shape, a Rectangle is a special kind of Polygon and so on. The re-usability of superclass test cases depends on this idea. Unfortunately, many inheritance relationships do not respect this rule and simple inherit from classes when they want to use library functions. Which functions must be tested in a subclass? Figure 6.3 shows a simple parent-child inheritance hierarchy. When testing child, we need to retest range() because of the overloading in class parent { int foo(int X); intrange(); // returns between 1-10 } class child extends parent { intrange(); // returns between 0-20 }

Figure 6.3: A simple parent child relationship for re-testing. the subclass, but do we need to retest foo()? Suppose foo() contains the line X = X/intrange(); In this case foo() depends on child.intrange() and re-testing is necessary, especially because the new intrange can cause a divide by 0, but maybe we do not need to retest completely. Can tests for a parent class be reused for a child class? 74

First we need to observe that parent.range() and child.range() are two different functions with different specications and different implementations. Test cases are derived from the different specications, however, the functions are likely to be similar. Thus, if we minimise the overloading by using principles such as the open/closed principle in our design, then the greater the chance that the inherited methods will not need re-testing in the child context. The new tests that are necessary will be those for child.Intrange() requirements that are not satised by the parent.Intrange() tests. For example, consider the following program fragment in Figure 6.4. The tests for the parent class Parent { public void describeSelf() { if (val < 0) message(Less); else if (val == 0) message (Equal); else message(More); } } class Child { public void describeSelf() { if (val < 0) message(Less); else if (val == 0) message (Zero Equal); else { message(More); if (val == 42) message(Jackpot); } } }

Figure 6.4: What new tests are necessary? and child classes appear in the following table.

Value -1 0 1 42

Parent Response Less Equal More

Child Response Less Zero Equal More Jackpot

Test Changes OK Changed OK Add

75

Building and Testing Inheritance Hierarchies

When dealing with inheritance hierarchies it is important to consider both testing and building at the same time. There are two key reasons for this: (i) the rst is to keep control of the number of test cases and test harnesses that need to be written; and (ii) the second is to localise faults within the inheritance hierarchy as much as possible. A rst approach to inheritance testing involves Flattening the Inheritance Structure. In effect each subclass is tested as if all inherited features were newly dened in the class under test so we make absolutely no assumptions about the inherited parent classes. Tests in parent classes may be re-used after analysis but many tests will be redundant and many tests will need to be newly dened. A second approach to testing and building is Incremental Inheritance-based Testing. Incremental building and testing proceeds as follows: Step 1 First test each base class by: (1) Testing each method using a suitable black box or white box test case selection method; and (2) Testing interactions among methods by using the state based methods described below. Step 2 Then, consider all sub-classes that inherit or use (via aggregation or association) only those classes that have already been tested. A child inherits the parents test suite which is used as a basis for test planning. We only develop new test cases for those entities that are directly or indirectly changed. Incremental inheritance-based testing does save time by reducing the number of test cases that need to be developed but there is an overhead in the analysis of what tests need to be changed. It certainly reduces the number of test cases that need to be selected over a attened hierarchy. Inheritance based testing can also be considered to be a form of regression testing where the aim is to minimise the number of test cases needed to exercise a modied class.

Implications of Polymorphism
Consider the inheritance hierarchy in Figure 6.1. The implementation of Area() that actually gets called will depend on the state of the object and the runtime environment. In procedural programming, procedure calls are statically bound we know exactly what function will be called and when at compile time and further, the implementation of functions do not change (well, not unless there is some particularly perverse programming) at runtime. In the case of object oriented programming each possible binding of a polymorphic class requires a separate set of test cases. The problem for testing is to nd all such bindings after-all the exact binding used in a particular object instance may only be known at run-time. Dynamic binding also complicates integration planning. Many service and library classes may need to be built and tested before they can be integrated to test a client class. 76

There are a number of possible approaches to handling the explosion of possible bindings for a variable. One approach is to try and determine the number of possible bindings through a combination of static and dynamic program analysis. Consider the Client class in Figure 6.1. The problem from a testing point of view is to know which actual Shape object gets called at run-time. If an object or type Square is bound to the variable S in clientFunction then we would expect a different result for the Area() computation than if Circle were bound to S. If we instrumented the code for Shape and all of its descendents to reveal the types of the actual objects that are bound to S then we could use that information to determine the subset of the class hierarchy to test. Beware however, this approach is not foolproof and is biased heavily towards the data used to generate the bindings. Remark 19 On average, however, the number of bindings found in practice is 2.

6.4 State Based Testing

State based testing views classes as state machines. The idea is to model each class as a nite state automaton where related sets of class states form the states of the automaton and methods cause transitions between the automaton states. Test cases are selected to test sequences of transitions in the automaton.

Analysis for State Based Testing

The analysis for state based testing proceeds as follows. Step 1 is to identify the starting state for the class , the exiting state for the class and the set of legal states for the class; Step 2 is to identify which methods are enabled in a state and which methods are not enabled in a state methods that can be called safely in a given state; Step 3 The automaton denes the valid sequences of method calls that cause transitions between legal states, for example, we cant pop from an empty stack; Test cases are devised to set the class into one of the legal states and then to exercise exercise transitions emanating from that state. For example, consider the simple state model of the Stack class whose interface is shown in Figure 6.5. A testing automaton for the Stack class is shown in Figure 6.6. The choice of states requires some further discussion. For the purposes of testing we have chosen the following states. There are actually numerous stack states. The aim for the state transition graph of a class describes the transitions between sets of valid states. The aim of the transition graph is to implicitly test the states of the class by exercising sequences of method calls. The question is How do we group states that are equivalent in some way so that we can reduce the number of sequences of transitions to test? 77

public class Stack { public void Stack(); // Creates an empty stack with no // data values. public void Push(Object o); // Push a data object onto the stack. public void Pop(); // Remove the top element from the // the stack. public Object Top(); // Returns the value on top of the // stack. public boolean IsEmpty(); // Returns true if the stack is empty, that // is contains no values. } Figure 6.5: A Java stack class

State State 1 State 2 State 3

Description An undened stack. A stack that is dened but is empty, that is, a stack with no values in the stack. A stack with at least one value in it. Table 6.1: States for the Java Stack class.

Cannot pop from this state Push

Push

Create

Pop [If stack size == 1]

Pop [if stack size > 1]

Figure 6.6: An Finite State Automaton for the Stack ADT

78

One method is to group stack states that enable the same set of operations into a single testing automaton state. For example, in the stack example above we have collected all non-empty stacks into a single testing automaton state which we have called State 3 above.

Explicitly Testing the State Transition Diagram

Explicitly testing the state transition diagram means adding in testing code to break the encapsulation in a class. Once this is done the characteristics of the class change, for example, (i) any testing code added to the class can be unreliable and introduce faults that interfere with the testing process; (ii) any testing code added to the class may alter the response times of the methods and consequently change the performance of the class; (iii) any testing code added to the class may change the level of security of the classes (or indeed the system because now we have a back-door for testing that can be exploited by anyone who knows about it). The points at which to insert instrumenting code into a class are to observe the values of private state variables, observe the values of private method calls and even observe intermediate states in class methods. This is not as obvious as it sounds because of the problem of inheritance. Consider the class Node for implementing a doubly linked list in the following example. public class Node { private Object data; private Node previous, next; } In order to expose the state of Node we need to get access to the values of the data instance variable. Now consider the inheritance hierarchy in Figure 6.7. In order to reveal the state of the Node class we will need to ensure that there are methods to access the private state elements of all the objects that inherit from Object1 , that is, we would need to extend all elements of the hierarchy with testing code. In this case there is may be a great deal of additional code to write which may well further impact the characteristics of the class. In languages like C and C++ we can use the pre-processor to compile in test code when required, or to omit the testing code if the program is to be released. Once, we have instrumented our classes then test cases are chosen to exercise each of the transitions in the testing automaton.

Implicitly Testing the State Transition Diagram

There will be occasions when you will not have access to the internal structure of a class or its hierarchy. In this case, or in the case where you simply need to test a class without adding
In the case of the type Object in Java this can mean a large number of classes in theory, but in practice it is not often that large.
1

79

Object

Shape

Polygon

Curved

Rectangle

Elipse

Bezier Curve

Figure 6.7: Inheriting from the Object type. code to all of the descendents of some part of the state, as we needed to in Figure 6.7 then implicit testing can be used. The idea is to test each of the transitions in the testing automaton. The methods in the class can be divided into: Constructors Extenders Transformers Observers The constructors for the class, such as the Stack() method in the Stack class. Methods to add data to class state, such as the Push() method in the Stack class. Methods to change the state of the class, such as Pop() method in the Stack class. Methods to observe the class state, such as the IsEmpty() method and the Top() in the Stack class.

Table 6.2: One classication of the methods of a class. For the Stack automaton in Figure 6.6 we could derive a set of test cases as follows. Testing the Stack constructors and observers: A place to begin is to gain assurance that the stack constructors work properly. The sequence of method calls: Stack(); IsEmpty() should return TRUE and Stack90; Top() 80

should return an exception or an error message. If the rst sequence does not return TRUE then there is a fault in either the IsEmpty() method or the Stack() method. If the second sequence does not return an exception or an error message then there is a problem with Stack() or Top(). If we cannot trust the observers, then its time to instrument the code and determine the exact nature of the fault. Testing the Stack extenders and observers: Since all stacks can be built up using just the stack constructor and the Push() operation then we can test Push() next. The rst problem here is that the input domain for Push() is Object2 . Object in Java is the type that is the parent of every other type and only contains a single public type. The specication for Push() does not require us to manipulate objects of type Object and so we can choose any actual type to test out Push(); in our case int is convenient. Given that we can trust IsEmpty() and Top() then the sequence Stack(); Push( 3 ); IsEmpty() should return FALSE and Stack(); Top() should return 3. If they do not then there is a fault in the Push() method and the return value of Top() may indicate the fault. Testing the Stack transformers and other operation: With condence in the constructors, observers and extenders we can set up sequences to test the other operations, for example, the sequence of method calls Stack(); Push( 3 ); Push( 4 ); Pop(); Top() should return 3. Given that we are condent in Stack(), Push() and Top() then the fault lies in Pop(). Remark 20 (i) Each of the test cases above consists of a sequence of method calls rather than just a single call. (ii) It is possible that a test case may consist of an equality between two sequences of method calls, for example, Stack(); Push( 3 ); Pop(); == Stack(); if such a test for equality can be made. (iii) If methods exist for forcing a class into one of the testing states then this can make testing long sequences of method calls easier. The design of object oriented system is a very important part of the testing methodology. Designing object oriented programs for testing is just as important as designing object oriented systems to meet their requirements even though there may be a tension between the two. Fortunately, for state based testing state models are often created as part of a design methodology. For example, UML uses state-charts precisely for the purpose of specifying and understanding the legal sequences of actions on an object.
2

Assuming we dont have access to the state.

81

Problems with State Based Testing

Of course there are some problems with state based testing and we list some of these below. It may take a very lengthy sequence of operations to get an object into some desired state. State based testing may not be useful if the class is designed to accept any possible sequence of method calls. State control may be distributed over an entire application with methods from other classes referencing the state of the class under test. System-wide control makes it difcult to verify a class in isolation and requires that we identify class hierarchies that collaborate to achieve a particular functionality. Here the collaboration and behaviour diagrams are the most useful.

6.5 Black Box and White Box Testing for Object Oriented Programs
We conclude this chapter with a brief look at how our existing testing methods for imperative programs can be applied in Object Oriented programs.

Black-box Testing Conventional black-box methods are useful for object-oriented systems! We dont need the details of the internal states for objects and can rely on the interface of a class and the specications for the class.

White-box Testing White-box techniques can be adapted to method testing, but are not sufcient for testing classes. The reason for this is that methods can call each other within a class and that they interact through the class state. In this case the coverage criteria can become dependent on the internal state. In turn the state of an object may well depend on the preceding sequence of object method calls and their parameter values.

Testing and Building Object Oriented Programs

Recall that in Object oriented testing the units are classes. Unit testing focuses on behaviour of individual classes. Tests are derived from class specications, algorithms or source code when available and testing drivers and testing stubs usually required Integration testing focuses on testing the communication between classes and the and interfaces between classes and their methods. Test cases are derived from class interfaces and 82

detailed (lowest level) of the architecture specication. In practice, integration testing means understanding how objects interact! It also means testing specic contexts such as dynamically bound variables and parameters, and polymorphic operators. There are a number of building strategies that help to systematically integrate and test object oriented programs. Thread-based The idea is to incrementally build up threads that respond to some input or event. A thread consists of all the classes needed to respond to a set of related external inputs or events. Each class is unit tested, and then the set of classes in the thread is exercised. Uses-based The idea is to begin by testing classes that use few or no other classes, then, test classes that use the rst group of classes, then follow this by testing the classes that use the second group, and so on. Regression testing is all about re-testing xed or modied code and ensuring that this is done as quickly as possible. Of note is that changes may have greater impact because of inheritance problems discussed earlier. System testing focuses on the behaviour of the system as a whole. Tests are derived from the requirements specications and the the system is often tested as a Black-Box. Drivers and stubs are usually not needed though automation eases re-testing.

83

Chapter 7

Reviews, Inspections and Walkthroughs

84

Chapter 8

Software Reliability Theory and Practice

85

Chapter 9

Software Safety

86

Chapter 10

Software Security

87

Chapter 11

Software Performance Engineering

88

Chapter 12

Portability

89

Chapter 13

Concluding Remarks

90