Scribd
Upload
58 views

Glimpses From The History of Software Testing: Sagar Sen Software Engineering Dept., Simula Research Laboratory

1. Software testing has evolved from a debugging focus before 1956 to various orientations like demonstration, destruction, evaluation, and prevention over time. Key developments include the formalization of testing processes, emergence of test professionals and conferences, and the growth of test automation. 2. The concept of "bug" originated from Thomas Edison's work repairing issues with telegraphs in the late 1800s, and the first computer bug was a moth found in the Harvard Mark II computer in 1947. 3. Influential testing approaches include model-based testing developed in the late 1970s, exploratory testing from the late 1980s, and test-driven development from the early 2000s. Pairwise and fuzz testing strategies

Uploaded by

Gherca Ovidiu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
58 views

Glimpses From The History of Software Testing: Sagar Sen Software Engineering Dept., Simula Research Laboratory

1. Software testing has evolved from a debugging focus before 1956 to various orientations like demonstration, destruction, evaluation, and prevention over time. Key developments include the formalization of testing processes, emergence of test professionals and conferences, and the growth of test automation. 2. The concept of "bug" originated from Thomas Edison's work repairing issues with telegraphs in the late 1800s, and the first computer bug was a moth found in the Harvard Mark II computer in 1947. 3. Influential testing approaches include model-based testing developed in the late 1970s, exploratory testing from the late 1980s, and test-driven development from the early 2000s. Pairwise and fuzz testing strategies

Uploaded by

Gherca Ovidiu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 33

Glimpses from the History

of Software Testing
Sagar Sen
Software Engineering Dept., Simula Research
Laboratory

1
Wednesday 20 November 13

The Growth of Testing (Bill Hetzel)

Until 1956 - Debugging oriented


19571978 - Demonstration oriented
19791982 - Destruction oriented
19831987 - Evaluation oriented
19882000 - Prevention oriented
2
Wednesday 20 November 13

Bug in a Technical System

Quadraplex telegraph

Thomas Alva Edison

Mangled Messages! The 26-year-old engineer combined diplex and duplex circuits to send two
messages in each direction using changes in current direction and strength. The problem in this
approach was the false break in a messages signal created by the changing polarity of the
electromagnet in the diplex circuit when the current switched direction.
Edison worked around this by building what he later called a bug trap to isolate the unwanted
break so that it wouldnt interfere with the meaning of the Morse-coded signal.
3

Wednesday 20 November 13

Bug in a Technical System

Insomnia squad debugging with 3 hrs sleep


In 1889 the reporter for a British newspaper wrote, Mr. Edisonhad been up the two
previous nights working on fixing a bug in his phonographan expression for solving a
difficulty, and implying that some imaginary insect has secreted itself inside and is causing all
the trouble. And in a 1916 article about Edisons insomnia squad, a reporter noted that
they worked like fiends when they [were] fishing for a bug. That means that they are
searching for some missing quality, quantity, or combination that will add something toward
the perfect whole.
4
Wednesday 20 November 13

Talking Doll (1890) with Phonograph

Can you guess the tune from


the sound?
5
Wednesday 20 November 13

First Computer Bug

In 1947 Grace Hopper and her staff started using the word
bug to describe technical glitches when they found a dead
moth [above] in Relay 70 of Harvard's Mark II computer.

6
Wednesday 20 November 13

Checking a Large Routine (The idea of Assertions)


(1949)

7
Wednesday 20 November 13

The Growth of Testing (Bill Hetzel)

Until 1956 - Debugging oriented


19571978 - Demonstration oriented
19791982 - Destruction oriented
19831987 - Evaluation oriented
19882000 - Prevention oriented
8
Wednesday 20 November 13

Difference between debugging and


testing! (1957)
Charles Baker
reviewed
Dan McCrackens First Book
on Computer Programming

Debugging

Testing
Make sure a program solves the
problem!

Make sure a program runs!

As computer applications increased in number, cost,


and complexity, testing came to assume more
significance because of greater economic risk
9
Wednesday 20 November 13

Gerald Weinberg Assembles First Test Team


(1958)

A test team became suddenly important for the first


human-flight mission
10
Wednesday 20 November 13

Chess Players, Music Lovers, and Mathematicians (1969)


Applicants
Chess player, dude
with IQ 172 (selftaught pianist at 10),
crystallographer from
Oxford, PhD. in
English (Columbia
University), exfashion model,
proto-hippie, and
more chess players...

11
Wednesday 20 November 13

Testing Software Modeled by Finite-State Machines


Tsun S. Chow (1978)
1. Tsun Chow derived test sequences from a
finite-state model of a program (comment printer
for instance)
2. This played a key-role in the origins of modelbased testing of software

12
Wednesday 20 November 13

The Growth of Testing (Bill Hetzel)

Until 1956 - Debugging oriented


19571978 - Demonstration oriented
19791982 - Destruction oriented
19831987 - Evaluation oriented
19882000 - Prevention oriented
13
Wednesday 20 November 13

Glen Myers Definition of Testing (1979)

The process of executing a program with the intent of


finding errors
Demonstration model aimed to show that program had no faults. But Myers wanted to
demonstrate that the program had faults.

14
Wednesday 20 November 13

Software Engineering Economics (1980)

Barry Boehm
stated that the cost
to fix error increases
exponentially the
later you detect it!

15
Wednesday 20 November 13

The Growth of Testing (Bill Hetzel)

Until 1956 - Debugging oriented


19571978 - Demonstration oriented
19791982 - Destruction oriented
19831987 - Evaluation oriented
19882000 - Prevention oriented
16
Wednesday 20 November 13

Federal Information Processing Systems(FIPS)


Methodology

Analysis

Review

Test

Product Evaluation

No single VV&T technique can guarantee correct, error-free software.


However, a carefully chosen set of techniques for a specific project can help to
ensure the development and maintenance of quality software for that project.

17
Wednesday 20 November 13

Growth of Professionalism in Testing


Testing takes off!
Jobs

National
Publications
Meetings

Standards

Test manager
Testing books at
Testing
Lead tester
average rate of 2/year
Conference
Test analyst
since Myers (1979)
(Bill Hetzel)
Test technician
Academic Testing Workshops
Industrial Testing Conf. (1984)

18
Wednesday 20 November 13

ANSI/IEEE
Standard for Test
Documentation
(1983)
Standard for Unit
Testing (1987)

Information Systems Manifesto (1984)

Studies distribution of defects, showing that over half of the defects


have their root cause in poorly defined requirements.

19
Wednesday 20 November 13

The Growth of Testing (Bill Hetzel)

Until 1956 - Debugging oriented


19571978 - Demonstration oriented
19791982 - Destruction oriented
19831987 - Evaluation oriented
19882000 - Prevention oriented
20
Wednesday 20 November 13

Exploratory Testing (1988)


Exploratory testing is a style of testing in which we learn about the
behavior of a system by designing a small test, executing it immediately,
using the information gleamed from the last test to inform the next.

Explore area
with resources
to discover information

Explore editing profiles


with the authentication feature
to discover surprises

Simultaneously learning about the system while designing and executing


tests, using feedback from the last to inform the next.

21
Wednesday 20 November 13

Fuzz/Random Testing (1988)


Automated or semi-automated, that involves providing invalid, unexpected, or random
data to the inputs of a computer program. The program is then monitored for exceptions
such as crashes, or failing built-in code assertions or for finding potential memory leaks.

"The Monkey" was a Macintosh application developed by Steve Capps prior to 1983. It used
journaling hooks to feed random events into Mac programs, and was used to test for bugs in
MacPaint.
Shortcomings...
1. Usually finds very simple faults
2. Poor code coverage
3. Hard to catch boundary value conditions
22
Wednesday 20 November 13

Classification Tree Method


(Grimm, Grotchman 1993)
Any system under test can be described by a set of classifications, holding
both input and output parameters. (Input parameters can also include
environments states, pre-conditions and other, rather uncommon parameters).

23
Wednesday 20 November 13

Pairwise Test Strategy (1996)


Pairwise testing (Cohen)
In their paper The Combinatorial Design Approach to Automatic Test
Generation, authors David Cohen, Siddhartha Dalal, Jesse Parelius and
Gardner Patton introduce pairwise testing. They base their combinatorial
design theory to generate tests on the usage of the Automatic Efficient
Test Generator (AETG), during projects at Bellcore.

Test Every OS with Browser, Every Browser with Resolution, and Every OS with Resolution.
How?
The AETG system uses new combinatorial algorithms to generate test sets
that cover all valid n-way parameter combinations. The size of an AETG test
set grows logarithmically in the number of test parameters.
24
Wednesday 20 November 13

Context-driven Testing (1999)


Cem Kaner, James Bach, Brian Marick and Bret Pettichord start the Context-Driven
school of testing based on the principle that 'The value of any practice depends
on its context'.
Best practices coming from academia didnt work so well when you tried to
apply them in industry; that best practices published in books by people who
worked in the telecom business didnt work so well for people in Silicon Valley;
Bachs Schools of Testing

Routine School

Math School

Quality School
25

Wednesday 20 November 13

Context-driven School

Testing in Recent Times

26
Wednesday 20 November 13

Continuous Integration (Martin Fowler, 2000)


Battle royale between coders (Feature Content) and testers (Feature Working)!

Never comment out failing tests.


Helps avoid one developer's work in progress breaking another developer's copy.
27
Wednesday 20 November 13

Test-Driven Development (2002)


Test-Driven Development (Beck)
In the book Test Driven Development: By Example Kent Beck publishes the software
development technique in which a test is written before the functionality is coded.

28
Wednesday 20 November 13

How to Break Your Software: James Whittaker (2002)

A recipe book of 23 attacks to break software


Example:
Generating System Faults
a) Monitor your system
b) Inject tests of your choice
Faults to look for...
- Out of Memory
- Locked Memory
- Out of disk space
- CRC errors

29
Wednesday 20 November 13

Testing Experience
Testing Experience magazine launched in 2008
The magazine Testing Experience is launched by the German company Daz & Hilterscheid.
A magazine to share experiences and opinions of testers from the industry.

After all humans are the weakest link


in causes for software failure!

30
Wednesday 20 November 13

Types of Testing Today: A Plethora!

31
Wednesday 20 November 13

The Future of Testing?

32
Wednesday 20 November 13

What next?
1. Journey of Software Testing: We journeyed from the debugging era (before 1956)
to test-driven development where testing comes first.
2. Testing tools? The only tools that made an impact in the industry are (a) test
automation tools such as continuos integration (b) Coverage analysis tools (Peter
Russell, Testing Experience in Future of Testing)

3. Glimpses from the future of testing? (a) Design for testability (b) Lean is hot. Lean
is Every. Test Management should be Lean (c) Agile Testing (d) Test data management
(rejuvenation of test data) (e) Automatic test generation with human-in-the-loop (f)
effective testing system of systems.
4. 22nd Century Testing? Software that is completely verified automatically. What will
happen to testers?

33
Wednesday 20 November 13

You might also like