Scribd
Upload
62 views

Exploratory Testing: Mattias Göransson

Exploratory testing involves simultaneous learning, test design, and test execution. The document provides an example of a large project where exploratory testing found 200 defects in 5 days compared to 600 defects found from scripted testing over 90 days. Exploratory testing works best when requirements are not fully known and to go beyond obvious tests, while scripted testing is better for safety critical applications with known requirements. A mix of both approaches is often ideal. Skilled exploratory testers challenge assumptions, alert clients to issues, collaborate, think critically, design experiments, take useful notes, and get the most value by exploiting inconsistencies and learning testing techniques.

Uploaded by

sen2nat5693
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPS, PDF, TXT or read online on Scribd
62 views

Exploratory Testing: Mattias Göransson

Exploratory testing involves simultaneous learning, test design, and test execution. The document provides an example of a large project where exploratory testing found 200 defects in 5 days compared to 600 defects found from scripted testing over 90 days. Exploratory testing works best when requirements are not fully known and to go beyond obvious tests, while scripted testing is better for safety critical applications with known requirements. A mix of both approaches is often ideal. Skilled exploratory testers challenge assumptions, alert clients to issues, collaborate, think critically, design experiments, take useful notes, and get the most value by exploiting inconsistencies and learning testing techniques.

Uploaded by

sen2nat5693
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPS, PDF, TXT or read online on Scribd
You are on page 1/ 18

Exploratory Testing

Mattias Göransson
Acknowledgements

James Bach, Satisfice, Inc.


Exploratory testing – statistics

There is no reliable study showing exactly


how efficient Exploratory Testing is…
Exploratory testing – facts of life

…so I will give you an example from a


large project where I acted as Test
Manager.
Exploratory testing – a real project (1/2)

Requirements in the form of Use Cases


All flows through them gave one positive + three
negative Test Cases.
Walkthroughs with the owners of these Use Cases.
For each Iteration the testers spent app. 90 days on
design, documentation and execution of these Test
Cases.
This resultet in app. 600 Defect reports
Exploratory testing – a real project (2/2)

After we had done all the work with the scripted


testing I was allowed to spend app. 5 days of each
testers time on Exploratory Testing.
This resulted in app. another 200 Defect reports.
Exploratory testing

Exploratory testing is simultaneous


learning, test design, and test execution.
Contrasting approaches

Scripted testing
• Tests are first designed and recorded.
• Then they may be executed at some later time or by a
different tester.
• Done well: 70% of the Testers time is spent on design &
documentation and 30% on Executing Tests

Exploratory testing
• Tests are designed and executed at the same time, and they
often are not recorded.
• Done well: 80-90 % of the Testers time is spent on Executing
Tests and 10-20% is spent on Documentation.
What is exploratory testing good for

Exploratory testing is useful when…


• it is not obvious what the next test should be
• we want to go beyond the obvious tests
• AND when we don’t know all every detail of every
requirement on the product under test.
When is Exploratory testing not to be used

Exploratory testing is not a method for


testing safety critical applications where
you up front know all the requirements.
Severity matrix – A mix is often best

2 4

Frequency

1 3

Impact
The core practice of a skilled ET Tester

Excellent exploratory testers…


• challenge constraints and negotiate their mission.
• alert their clients to project issues that prevent good testing.
• spontaneously coordinate and collaborate, as needed.
• train their minds to be cautious, curious, and critical.
• know how to design questions and experiments.
• know the difference between observation and inference.
• have developed resources and tools to enhance performance.
• adapt their test strategy to fit the situation.
• tolerate substantial ambiguity, indecision, and time pressure.
• take notes and report results in a useful and compelling way.
• have earned the trust placed in them.
Getting the most out of exploratory testing
(1/5)

Exploit inconsistency.
• Let yourself be distracted by anomalies and new ideas.
• Avoid repeating the exact same test twice.

Exploit the human factor.


• Encourage variability among testers.
• Exploit subject matter expertise.
• Use your confusion as a resource.
• Work in pairs or groups, whenever possible.
• Get to know your developers.
• Test in short bursts.
Getting the most out of exploratory testing
(2/5)

Learn the logic of testing


• correlation and causality
• design of experiments
• forward, backward, and lateral thinking
• biases, heuristics, and human error
• risk, benefit, and the meaning of “good enough”

Practice critical reading and interviewing


• Analyzing natural language specifications
• Analyzing and cross-examining a developer’s explanation.
Getting the most out of exploratory testing
(3/5)

Adopt a clear and consistent testing vocabulary


• bug, specification, risk, test strategy, test case, test plan

Learn to model a product rapidly


• flowcharting; data flows; state model
• matrices and outlines
• function/data square
• study the technology

Use a “grid search” strategy to control coverage


• Model the product in some way, then specify broad test areas
in terms of that model (not specific test cases). Keep track of
what areas you have and have not tested in.
Getting the most out of exploratory testing
(4/5)

Learn to take reviewable notes


• Take concise notes so that they don’t interrupt your work.
• Record at least what your strategy was, what you tested, what
problems you found, and what issues you have.

Practice responding to scrutiny


• Why did you test that?
• What was your strategy?
• How do you know your strategy was worthwhile?

Learn to spot obstacles to good testing


• Not enough information, yet.
• The product isn’t testable enough.
• Don’t have enough of the right test data.
Getting the most out of exploratory testing
(5/5)

Develop and use testing heuristics


• Try the heuristic test strategy model.
• Practice critiquing test techniques and rules of thumb.
• Notice the test ideas you use that lead to interesting results.

If you’re in a highly structured environment, consider


using session-based test management to measure
and control exploratory testing.
• Packages ET into chunks that can be tracked and measured.
• Protects the intuitive process; gives bosses what they want.
Exploratory testing

Exploratory testing is simultaneous


learning, test design, and test execution.

You might also like