Introduction To

Agile Testing
Welcome
Philip Juhl
INTRODUCTION & EXPECTATIONS

Overview of different test-practises for Agile teams

Role for a tester and QA manager in an Agile development team

Recommendation and overview of test-practises and test automation

AGENDA

Purpose of Testing

Testing in an agile vs traditional context

Quality assurance

The testers roles

Test-driven development

Input for Test-strategy & Test-automation

WHAT ARE THE VALUES OF TESTING?

Align expectation

Get confirmation

Measure progress

Measure quality

Learn about the system

PART OF THE FEEDBACK LOOP

What do we want?
Adjust or What do we
Continue want?

Build it Describe test

Test Driven
Execute test Describe test
Execute test

Adjust or Continue Build it

DEMING’S CYCLE
MANY FEEDBACK LOOPS
What do we want feedback on How we test it
Code-change integrate with latest source code Check-in, Merge & CI Build
New functionality is correct Functional testing
Existing functionality still works Regression test
We haven’t missed anything Exploratory testing
Integrates correct with other systems System integration testing
Complete workflow is correct End-2-End testing
Implementation is user friendly Usability testing
Implementation is secure Security testing
Works as user expects Acceptance testing
Implementation has the desired effect Validation testing
FEEDBACK LOOPS INSIDE LOOPS

Short delay is better

ALWAYS PUT THE MARSHMALLOW ON TOP
 YOUR FEEDBACK LOOPS AND LEAD TIME?

1. What are the feedback loops in your systems(s)

2. What are the lead time?
AGENDA

Purpose of Testing

Testing in an agile vs traditional context

Quality assurance

The testers roles

Test-driven development

Input for Test-strategy & Test-automation

PLAN-DRIVEN IN A REAL ENVIRONMENT
Feedback-time?

Requirement analysis Test report Acceptance test

System Design System/Integration test

Change- Code
advisory board freeze
Component Component test

Implementation
Requirement Implementation
Phase Phase Test Phase
Time
PROJECT SIZE AND SUCCESS-RATE
59%
56%

31%

18% 19%

9%

Large projects Medium projects Small projects

Agile Waterfall

Source: Standish Group, Chaos Studies 2013-2017

COMPLICATED VS COMPLEX
Complex Complicated

Loosely coupled Tightly coupled

Emergent practice Good practice
th o d
Probe – Sense - Respond Sense – Analyse - Respond
e n me
h o d r i v
em e t
Pla n -d
Agil
Chaos Obvious

De-coupled No degree of freedom

Novel practice Best practice
Act – Sense - Respond Sense – Categorize - Respond

Cynefin framework
AGILE ONION

- Mindset
- Values
- Principles
- Technics
- Tools
MANIFESTO FOR AGILE SOFTWARE DEV.

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right,

we value the items on the left more.
PRINCIPLES FROM THE AGILE MANIFESTO
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to
the shorter timescale.

Business people and developers must work together daily throughout the project.

Continuous attention to technical excellence and good design enhances agility.

The most efficient and effective method of conveying information to and within a development team is
face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
HOW DOES THESE PRINCIPLES AFFECTS TEST
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference
to the shorter timescale.

Business people and developers must work together daily throughout the project.

Continuous attention to technical excellence and good design enhances agility.

The most efficient and effective method of conveying information to and within a development team is
face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
SCRUM
The Scrum Team is responsible for all product-related activities from stakeholder collaboration,
verification, maintenance, operation, experimentation, research and development,

Scrum Teams are cross-functional, meaning the members have all the skills necessary to create value
each Sprint.

Developers are the people in the Scrum Team that are committed to creating any aspect of a usable
Increment each Sprint.
HOW DOES AGILE AND SCRUM AFFECT TEST?
AGENDA

Purpose of Testing

Testing in an agile vs traditional context

Quality assurance

The testers roles

Test-driven development

Input for Test-strategy & Test-automation

WHAT IS THE IDEAL RATIO?
What’s the ratio between
Software-testers and
Software developers for …

1. The successful projects?

2. The worst projects?
TESTER & SOFTWARE-DEVELOPER RATIO

1:2
2:1
1:1

Testers Software-developers
TESTER & SOFTWARE-DEVELOPER RATIO

Testers Software-developers
EXPERIENCE FROM DIFFERENT PROJECTS
Successful projects do not seem to have an ideal
ratio.

Unsuccessful projects seems to have a correlation

with a high number of software testers per
software developer.

Correlation not Causality.

Elisabeth Hendrickson: testobsessed.com
BETTER TESTING, WORSE TESTING
Better testing at one level tends to
result in worse testing in another
given no other changes in the system.

Elisabeth Hendrickson: testobsessed.com

 Can be managed
TEST AFFECTS TEST Unmanaged effect
Unmanaged negative effect
Perceived
Quality

Unknown Software
Known defects Defect Testers
defects Developers
released Prevention Test Effort
released Test Effort

Defects
Delivered to
System test

Defects Found
in System test
 Can be managed
MULTIPLE FOCUS FOR TESTER Unmanaged effect
Unmanaged negative effect
Perceived
Quality

Unknown Software
Known defects Defect Testers
defects Developers
released Prevention Test Effort
released Test Effort

Defects
Delivered to
System test

Defects Found
in System test
A TRUE STORY ABOUT A DEV. TEAM
SHARED RESPONSIBILITY
AGENDA

Purpose of Testing

Testing in an agile vs traditional context

Quality assurance

The testers roles

Test-driven development

Input for Test-strategy & Test-automation

FOCUS AND TECHNIQUES FOR TESTER
Testable acceptance criteria 3 amigos
Defect
Prevention Discuss how to test/verify before implementation

Transparency for what is unit-tested Encourage unit-testing

Software
Developers Share responsibility
Test Effort Pair-testing Include in test-strategy

Testers Include software-developers to build test-data and test-environments

Test Effort
Exploratory testing Test that critize the product
Risk-based testing

Known defects
released
Handling bugs that are released as other backlog items
 TEST QUADRANT
Business

Support the team Criticize the product

Progress/Verification Validation

Technical
AGENDA

Purpose of Testing

Testing in an agile vs traditional context

Quality assurance

The testers roles

Test-driven development

Input for Test-strategy & Test-automation

 TEST-DRIVEN DEVELOPMENT

TDD BDD/SBE ATDD

The correct behaviour
Developer trust the
Focus for the system. Combine Writing Acceptance test
code
Documentation, test & code

Participants Software-developer, Tester, Software-developer &

Tester, Software-developer &
Technical tester Business/Customer
Business/Customer

Natural language,
Language Programming language
Gherkin syntax
Natural language

Xunit, Junit, MSTest, SpecFlow, Gherkin, Microsoft teams,

Tools Jasmine, Karma Cucumber, JBehave Jira, Word/Excel
 BDD / SPECIFICATION BY EXAMPLE
§ Early Clarification through deliberate discovery
§ Common Domain language
§ Foster collaboration across domain-knowledge
§ Support iterative development
§ Documenting functional requirements
§ Possible to automate
SCRUM WATER FALL
Sprint board

Stories ToDo Doing Done

Story 1 T T C A
C

Analyse
Story 2 T C A
C A
Code

Story 3 A T
C Test
 PROCESS WITH BDD
Scope
l
goa
pe from User User User User
sc o
ve story story story story
Deri

3 Amigos
Business goal
ples Illustrate using
m
exa
ine Examples Specifying
Ref
collaboratively

Key examples
Specification with
Examples Validate
Automate frequently
validation Living
Executable Documentation
Specification
From Gojko Adzic - Specification By Example
SPECIFICATION IN GHERKIN-FORMAT
§ Readable for humans
§ Describe behaviour not implementation
§ What we tests and not how we test it
§ Executable for machines
EXAMPLES IN GHERKIN
Feature: Delivery cost

Scenario: VIP customers gets free delivery for orders with 3 or more items
Given the customer is VIP
And the customer has 4 items in the shopping basket
When the customer goes to payment
Then the delivery cost is free

Link
SPECIFICATION EXERCISE
We want to increase the awareness and behaviour of using
Apple Pay for payment transactions. We would like to start
a campaign where the user get a bonus of 50,- DKR if
(s)he makes an Apple Payment with one of his/her’s credit
cards. The campaign is only active for payments that are
completed the first day of the month.
Info:
A user with Apple Pay can assigned more than one credit
card to Apple Pay.
A user can have multiple devices that can be used to pay
with Apple Pay.
SPECIFICATION EXERCISE
1. Assign one to take the Product Owner
role for the Apple Pay story
2. As a team write acceptance criteria for
the functional requirements - You can use
the Gherkin syntax writing scenarios
3. Can your acceptance criteria be used to
split the story?
HOW TO START USING BDD

Establish place Merge Test-scenarios Build structure for

to communicate with requirement- Test automation
Test-scenarios documentation
AGENDA

Purpose of Testing

Testing in an agile vs traditional context

Quality assurance

The testers roles

Test-driven development

Test-strategy & Test-automation

TRANSACTION AND HOLDING COST

RELEASE NEW FUNCTIONALITY

KEEP NEW FUNCTIONALITY

TRANSACTION AND HOLDING COST
Total cost

Optimal batch size

Holding cost
Cost

Automation

Transaction cost

Items (featues) per batch (release)

 FREQUENT RELEASES &
SUSTAINABLE PACE
Iterative development Test activities

Iterations 1 2 3 4

New functionality Acceptance test

Acc.
test
Existing Functions Integration test
Integration test
Unit test
Unit test
 TEST AUTOMATION
Functional Business
End-2-End testing
testing A/B-
testing

Usability
testing
Acceptance
Guide the team testing Exploratory Criticize the product
Progress/Verification testing Validation

Component Performance
testing Security
testing
testing

Unit-level
testing Manual test

Automated test
Technical
CONSIDERATIONS

d at ing
Up
test
n
entatio t est
m p
not imple Setu
t
i rem
en
m a tisk
Test
requ
t r uc ture Auto kation
e d for le t est-s verif
i
N e s ab
t es ting Reu em ent
r
re
th r equi
New lity ple wi
n c t iona Cou
fu
a n uel
M tion
if ic a
ver
Automate
or
manual
 VALUE IN THE SHORT AND LONG TERM
Execution time/cost
value

Easier to maintain than to implement

Time

Fragility
Maintainability
Debug time
 AUTOMATION
Cheap

Blinders Exo Skeleton

Maintenance cost

Expensive

Fast Food Cement

Poor Important
Risk Detection
HOW TO GET STARTED
§ Build test strategy together with the team
§ Make Test-automation a team responsibility
§ Avoid “Fast-food” test- tool
§ Encourage test-automation-exercises

§ Test strategy in a scaled setup

RECOMMENDED BOOKS
§ Agile Testing
§ Specification By Example
AGILE TESTING PRACTICES FOR TEAMS
Exercises with
§ How to split stories an test early
§ How to use Specification By Example