CSE 4313 Software Engineering Testing

Report 1 Specification
Due: Wednesday, January 27
Where: In class
Format: Individual
If the class has begun your report is late

1 Main Points
Be sure to read and follow all the guidelines from the links on reports and academic honesty from the
WWW home page for the course. The specification is the union of this document plus the program text
you are given.

1.1 Learning objectives

The purpose of the assignment is the following.
1. To gain experience reading and evaluating bug reports written by other people. This will
give you practice thinking about what a professional bug report should contain, as well
as gaining experience in analyzing bugs.
2. Designing and implementing functional test cases based on boundary analysis,
equivalence classes and decision tables.
3. Using Junit to create tests for Java programs.

1.2 What to submit

Hand in, in class, packages 1 and 2. If a package is too big to staple as one unit, then please staple in
multiple parts. Cover page as part 1 of N, the remaining as part x of N. Include your name(s) on each part.
Package 1
• Cover page (bug report version) – attached to announcement. Fill in your name, Prism username
and your Issue Tracker user name, as well as the bug report numbers in the order they appear in
your report.
• Your bug report document

Package 2
• Cover page (Standard version) – attached to announcement. Fill in your name and Prism username
• Your functional testing report.

Package 3
Submit electronically before the deadline your Sreadhex project, where Sreadhex is the directory
containing the project.
submit 4313 r1 Sreadhex

2 To get started
Section 2.1 describes how to get started with the open office bug reports. Section 2.2 describes how to get
started with the functional testing system.
2.1 Run Open Office
Run Open Office by typing ooffice on Prism to get familiar with it. You can also install Open Office on
your computer by downloading it from http://www.openoffice.org.
There are three different versions of Open Office available on Prism. You can run each one with the
following commands. For this report you are to use version 3.3.1, which is the default download for both
Mac OS and Windows.
• ooffice.2.4.1
• ooffice.3.1.1
• ooffice – also ooffice.3.1.0

2.1.1 Join the QA project of Open Office

Go the web page http://www.openoffice.org/servlets/Join to join the project. Your user
name must begin with your family name and you must use your CSE email account.
2.1.2 Bugs and issues explained
Read about the issues and why bug tracker is used.
http://qa.openoffice.org/issue_handling/project_issues.html

2.1.3 Issue writing guidelines

The following web page explains how issue reports should be reported.
http://qa.openoffice.org/issue_handling/bug_writing_guidelines.html

2.2 Functional testing system

Download the file 4313_ report1.tar.gz to a local directory. When you untar with the command
tar xzf 4313_report1.tar.gz, you will get a directory called Sreadhex that contains the
directory sreadhex that corresponds to the sreadhex Java package in which the system is implemented.
sreadhex contains the following files.
1. AllTests.java– the starting point for the system. You should not have to change this file,
as it invokes a test class for each of the three sets of test cases.
2. BoundaryTest.java – This file has some sample test cases as an illustration of how Junit
testing may be done. You will modify this file to implement a subset of your boundary
test cases.
3. DecisionTest.java – Initially it is the same a BoundaryTest.java. You will modify this file
to implement a subset of your decision table test cases.
4. EquivalenceTest.java – Initially it is the same a BoundaryTest.java. You will modify this
file to implement a subset of your equivalence class test cases.
5. Setable_Integer.java – Do not modify this class. It is needed for the translation of the C
version to Java. It implements the pointer to an integer needed for some of the
parameters of sreadhex.
6. sreadhex.jar – It contains the class that contains the variations of the Java version of the
sreadhex program that you are testing.
7. sreadhex.c – The C language version of the program.
In Eclipse do the following.
1. Select File -> New -> Java Project
2. Fill in the project name – I used Sreadhex
3. Click the button Create separate folders for source and class files
4. Click the button Create project from existing source
 5. Click browse and select the folder Sreadhex that you downloaded (this is the highest
level folder
6. Click Next at the bottom of the page
7. Click the Libraries tab, select Add Library
8. Select Junit, select Next, select Junit 4, select Finish
9. Click Add Jars, expand the lowest directory until you can select sreadhex.jar, click OK
10. Now you are back to the Libraries tab, click Finish
At this point you should have the project listed under the package tab in the left frame. Figure 1 shows
what you should see when you expand Sreadhex by clicking on the triangle to the left of the name.

Figure 1: Sreadhex project expanded in the Package Explorer Frame

Sreadhex should be compiled and executable. Select the project in the package tab, select Run –> Run
as –> Junit Test. You can also be in a listing of one of the test files, when you select Run –> Run as –>
Junit Test, then only that test file will be run.

3.1 Ooffice bug reports

Find 2 bug reports in Issue Tracker about problems with Open Office that have not been resolved. The
Issue Type for these reports should be DEFECT, the Status UNCONFIRMED, NEW, STARTED, and
REOPENED (all four are selected at once), and the Resolution should be -None-, All platforms, All OS,
Version 3.1.1. IMPORTANT: the bug report should require a few steps to replicate; it should not be
trivial, for example, screen format or spelling error.
You can search for reports from the following web page. Ooffice issues appear towards the end of the
Version combo box.
http://qa.openoffice.org/issues/query.cgi
For each bug report:
• Review the report for clarity and tone.
• Attempt to replicate the bug.
• Perform further analysis using various types of follow-up testing.
• Add comments as appropriate – these would be added to the comments box in Issue Tracker.
These comments could mention that you successfully replicated the bug on XXX configuration in
YYY build, or describe a simpler set of replication steps (if you found one). Your comments
 should NEVER appear critical or disrespectful of the original report or of the person who wrote it.
If you edit the report in the database, never change what the reporter has actually written. You are
not changing his work; you are adding comments to it at the end of the report.
The content of your comments can vary depending on the problem. The key thing is that they are
useful to the reader. Following is a list of things that can be addressed.
• Reporting the configuration and build you used.
• Reporting a simpler set of replication steps.
• Reporting the results of a good follow-up test.
• Making a good argument regarding the importance of the issue (pro or con).
• Demonstrating that you credibly tested for reproducibility of a bug that is not reproducible.
Your report includes an evaluation of the issue reports you worked with. For each issue report,
comment on what was done well, what was done poorly, and what was missing that should have been there.
Do not include the online comments in your written report. The marker will find them through Issue
Tracker.
The following items could be addressed in your report.
• Is the summary short (about 50-70 characters) and descriptive?
• Can you understand the bug report? As you read the description, do you understand what the
reporter did? Can you envision what the program did in response? Do you understand what the
failure was?
• Is it obvious where to start (what state to bring the program to) to replicate the bug?
• Is it obvious what files to use (if any)? Is it obvious what you would type?
• Is the replication sequence provided as a numbered set of steps, which tell you exactly what to do
and, when useful, what you will see?
• Does the bug report include unnecessary information, personal opinions or anecdotes that seem
out of place?
• Is the bug report too long? Too short? Does it have a lot of unnecessary steps?
• Is it overly general, e.g. “Insert a file and you will see '”? Is there an example, like “insert a file
like blah.foo or blah2.fee”?
• Can you replicate the bug?
• Did you need additional information or steps?
• Did you get lost or wonder whether you had done a step correctly? Would additional feedback,
e.g. “the program will respond like this...'”, have helped?
• Did you have to guess about what to do next?
• Did you have to change your configuration or environment in any way that wasn't specified in the
bug report?
• Did some steps appear unnecessary? Were they unnecessary?
• Did the description accurately describe the failure?
• Did the summary accurate describe the failure?
• What were the follow-up tests that you run on this bug report?
• Since you will probably NOT have time or the resources to run many follow-up tests, what other
tests would you like to run? What would you hope to learn from these tests?
• Does the description include non-factual information (such as the tester's guesses about the
underlying fault) and if so, does this information seem credible and useful or not?
• Does the description include statements about why this bug would be important to the customer or
to someone else? The bug report need not include such information, but if it does, it should be
credible, accurate, and useful.
For your report, the marker will be looking for details that suggest you had insight into what makes
issue reports good or bad, effective or ineffective. You are not expected to go through every item in the list
that could be addressed (not all these items apply to all bugs). Instead, you should address a few items of
interest. For each bug report, you might address different items.
3.2 Functional testing
3.2.1 The tasks
You are to develop test cases using each of boundary value, equivalence class and decision table methods;
i.e. you develop three sets of test cases based on the three methods. I recommend that you work up from the
least sophisticated method to the most sophisticated method.
Your report should discuss how you developed the test cases and give a rationale for tests you select
and reject. Limit your report to discussion each method independently. Do not include any comparison of
methods, as that will be the first task for Report 2; as a consequence, I suggest you log your impressions as
you proceed with this task.

4 The SREADHEX problem

sreadhex is a function from the GhostScript program, which processes the Postscript page-description-
language – the basis of pdf files. The problem, as described in this specification, comes from the book The
Craft of Software Testing, Brian Marick, Prentice Hall PTR, 1995, pp. 24-29.

4.1 Motivation
sreadhex is a function that takes a string of ASCII characters where each character represents a
hexadecimal digit. For example, the string “A49F” represents the hexadecimal digits 10, 4, 9, and 15.
Each digit can be represented in four bits, thus the string corresponds to the bit sequences, 1010, 0100,
1001 and 1111. As a consequence, a pair of digits can be packed into a byte, while a string of digits can be
packed into a byte-array of half the length of the string.
A49F  byte 1: 10100100 and byte2: 10011111
Question: What should be done if there are an odd number of characters? Since two digits fit per byte,
what would be done with the unused half of the last byte in the result string? Given, “34ABF”, what value
should the ???? below have?
Byte 1: 00110100
Byte 2: 10101011
Byte 3: 1111????
Answer: The solution is not to put the last digit in the result array, but to return it as a separate return
value. Thus the result string always has an even number of digits. In the above case, the result string
would be the first two bytes and the integer 15. The “extra” return value is typically used in the next call to
sreadhex. That is, if 15 is returned by one call, then the next call has 15 as input, where it is used as the
first 4 bits of the next result byte-array.
Question: What should be done if there are characters in the source string that do not represent
hexadecimal digits?
Answer: sreadhex completely ignores them. Thus, all of the following strings will yield the same
result.
“34AB”
“3 4 A B”
“3G4GAMB-“

4.3 Specification
Here is the specification for sreadhex. It is complex. Other styles of specification read more naturally but
are harder to use for devising test cases. To make the specification easier to read, names of variables and
definitions are in bold.
sreadhex(str, rlen, nread, odd_digit, s)
• str is a byte array to fill. It’s the destination.
• rlen is the maximum number of bytes to be filled in str.
 • nread is a pointer to an integer. When the routine finishes, it leaves the number of bytes it used in
that integer.
• odd_digit is a pointer to an integer. The caller uses it to pass in the first digit (if the last call used
an odd number of digits); the routine leaves either –1 or the extra digit in it after return. (The
“odd” in odd_digit refers to its being set when there are an odd number of hexadecimal characters
to be placed in str.
• s is a null-terminated string. It is the source of hexadecimal characters.
• sreadhex returns 0 or 1. 1 means sreadhex ran out of characters before filling str.

Definitions
There are 22 hexadecimal characters. They are listed here, along with their corresponding integer
values. Notice that both uppercase and lowercase characters are used.
0 1 2 3 4 5 6 7 8 9 A B C D E F a b c d e f
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 10 11 12 13 14 15

hex_char[index] – This is the indexth hexadecimal character in s. For example, if s is “Aa-0”, then
hex_char[0] is ‘A’
hex_char[1] is ‘a’
hex_char[2] is ‘0’
Notice that the dash is not counted. Numbering begins with 0.
num_hex_chars is the number of hexadecimal characters in s. If s is “Aa-0”, then
num_hex_chars is 3.
start – This is the location where the value of the first hexadecimal character in s is to be placed in str.
if (*odd_digit == -1) then start is 0
else start is 1
digit[index] – This is the indexth digit (4-bit chunk) of str. Digits 0 and 1 are in byte 0 of str, digits 2
and 3 are in byte 1, and so on. Remember that *odd_digit, if not –1, is placed in digit 0.
Preconditions
1. Assumed: str is a non-null pointer to an array that can hold rlen bytes (hence 2*rlen
digits).
2. Assumed: rlen ≥ 0.
3. Assumed: nread is a non-null pointer to an integer.
4. Assumed: odd_digit is a non-null pointer to an integer.
5. Assumed: *odd_digit is in the range [–1, 15].
6. Assumed: s is a non—null pointer to a null-terminated string.
7. Validated: rlen is not 0. On failure: *nread is 0. The return value is 0.

Postconditions
1. (An even number of digits that don’t fill str: use them all)
if start + num_hex_chars < rlen*2
and start + num_hex_chars is even
then A. the return value is 1.
B. *nread is (start + num_hex_chars)/2.
C. For any hexadecimal character in s at index index,
digit[start + index] = hex_char[index] (For example, if character 0 is ‘9’ and start
th
is 0, 9 is placed as the 0 digit of str.
D. *odd_digit is –1.
2. (An odd number of digits that don’t fill str; use them all)
if start + num_hex_chars < rlen*2
and start + num_hex_chars is odd
then A. the return value is 1.
 B. *nread is (start + num_hex_chars – 1)/2.
C. For any hexadecimal character in s at index index except the final hexadecimal
character, digit[start + index] = hex_char[index].
D. *odd_digit is set to the value of the last hexadecimal character in s.
3. (Enough digits to fill str; ignore any excess)
if start + num_hex_chars ≥ rlen*2
and start + num_hex_chars is odd
then A. the return value is 0.
B. *nread is rlen.
C. For any hexadecimal character in s at index index such that start + index < rlen*2
digit[start + index] = hex_char[index]
D. *odd_digit is unchanged.
4. if *odd_digit is initially ≥ 0
then digit[0] = the initial value of *odd_digit.
The following is the program in C. This version contains a fault that has long since been fixed in later
versions. This version, its name is v1 (see BoundaryTest.java) has been translated into Java (I hope
correctly, thereby containing that bug and no other). v2 contains one seeded bug. v3 contains a different
seeded bug. v4 contains both bugs in v2 and v3.

1 typedef unsigned char byte;

2 typedef unsigned int uint;
3
4 /* Read a hex string from a stream. */
5 /* Answer 1 if we reached end-of-file before filling string, */
6 /* 0 if we filled the string first, or <0 on error. */
7 /* *odd_digit should be -1 initially. */
8 /* if an odd number of hex digits was read, *odd_digit is set */
9 /* the odd digit value, otherwise *odd_digit is set to -1. */
10 static byte decode_hex[257]={0}; /* not initialized yet */
11 #define hex_none 0x10
12 #define hex_eofc 0x20
13 #define sgetc(s) (*((s)++))
14
15 int
16 sreadhex(str, rlen, nread, odd_digit, s)
17 byte *str;
18 uint rlen;
19 uint *nread;
20 int *odd_digit;
21 register byte *s;
22 { byte *ptr=str;
23 byte *limit = ptr+rlen;
24 byte val1 = (byte)*odd_digit;
25 byte val2;
26 register char *decoder = (char*)(decode_hex+1);
27
28 if (decoder[-1]==0) /* not initialized yet */
29 { static char hex_chars[]= "0123456789ABCDEFabcdef";
30 int i;
31 memset(decoder-1, hex_none, 257);
32 for (i=0; i<16+6; i++)
33 decoder[hex_chars[i]] = (i>=16? i-6: i);
34 decoder[0] = hex_eofc;
35 }
36 if (val1<=0xf) goto d2;
37 d1: while ((val1=decoder[sgetc(s)])>0xf)
38 { if (val1==hex_eofc) {*odd_digit=-1; goto ended;}
39 }
40 d2: while ((val2=decoder[sgetc(s)])>0xf)
41 { if (val2==hex_eofc) {*odd_digit=val1; goto ended;}
42 }
43 *ptr++ = (val1<<4) +val2;
44 if (ptr<limit) goto d1;
45 *nread = rlen;
46 return 0;
47 ended: *nread = ptr - str;
48 return 1;
49 }
Program notes

Line 13 – The program gets characters via sgetc. In the original program, sgetc got characters from a
stream datatype; in this simpler version it picks characters from a string.
Lines 10, 26 – The program uses a lookup table, decoder, to convert characters into values. The character
is used as an index in the table. decoder[character] is the corresponding value. The table is initialized the
first time sreadhex is called. In line 26 decoder is set to point to the last 256 characters of the decode_hex
array. The first character (decoder_hex[0] or decoder[-1] indicates whether the table has already been
initialized.
Lines 28-35 – decoder is initialized with memset, which sets every entry to hex_none indicating that the
corresponding character is not hexadecimal. Then the entries for hexadecimal characters are set by
iterating through a string of all such characters, calculating the value, and placing that in decoder. Note
that the characters “abcdef” are given the same values as “ABCDEF”. Character ‘\0’ (which is found at the
end of a C string) has value hex_eofc (value 0x20), which causes sreadhex to return.
Lines 36-42 – The first while loop finds the first digit in a byte; the second one finds the second. The loops
skip non-hexadecimal characters. If the first digit is passed in, it should be in the range [0 .. 15]. –1 means
no digit was passed in. The case on line 24 converts the –1 to a large positive number, so that val1 ≤ 0xf
evaluates to false.

5 Grading scheme
The grade for the report is partitioned into the following parts.
• Part 1: Overall presentation – 10%
• Part 2: Bug reports – 36% – 18% each
• Part 3: Boundary value testing – 18% – half for the test cases and half for the report
• Part 4: Equivalence class testing – 18% – half for the test cases and half for the report
• Part 5: Decision table testing – 18% – half for the test cases and half for the report