SCRUM PRODUCT OWNER

THE ART OF BUILDING TWICE THE PRODUCT IN HALF THE TIME

!
!
!
!
!
!
!
! States - 3313 (0.53%) in Germany
503,306 (7%) open Scrum jobs in the United
With help from Citrix Online, Google, Yahoo, Microsoft, IBM, Oracle, MySpace, Adobe, GE, Siemens, Disney Animation,
BellSouth, Nortel, Alcatel-Lucent, EMC, GSI Commerce, Ulticom, Palm, St. Jude Medical, DigiChart, RosettaStone,
Healthwise, Sony/Ericsson, Accenture, Trifork, Systematic Software Engineering, Exigen Services, SirsiDynix, Softhouse,
Philips, Barclays Global Investors, Constant Contact, Wellogic, Inova Solutions, Medco, Saxo Bank, Xebia, Insight.com,
SolutionsIQ, Crisp, Johns Hopkins Applied Physics Laboratory, Unitarian Universalist Association, Motley Fool, Planon,
FinnTech, OpenView Venture Partners, Jyske Bank, BEC, Camp Scrum, DotWay AB, Ultimate Software, Scrum Training
Institute, AtTask, Intronis, Version One, OpenView Labs, Central Desktop, Open-E, Zmags, eEye, Reality Digital, DST, Booz
Allen Hamilton, Scrum Alliance, Fortis, DIPS, Program UtVikling, Sulake, TietoEnator, Gilb.com, WebGuide Partner,
Emergn, NSB (Norwegian Railway), Danske Bank, Pegasystems, Wake Forest University, The Economist, iContact, Avaya,
Kanban Marketing, accelare, Tam Tam, Telefonica/O2, iSense/Prowareness, AgileDigm, Highbridge Capital Management,
Wells Fargo Bank, Deutsche Bank, Hansenet/Alice, GlobalConnect, U.S. Department of Defense, Agile Lean Training,
EvolveBeyond, Good Agile, Océ, aragostTRIFORK, Harvard Business School, Schuberg Philis, ABN/AMRO Bank, Acme
Packet, Prognosis, Markem-Imaje International, Sonos, Mevion, Autodesk, First Line Software, SCRUMevents, UPC
Cablecom, NIKO, CWS-BOCO, BottomLine, Lean Enterprise Institute, Liberty Global, Monster, Dartmouth University, Health
Leads, Samsung R&D Center, Monster.com, Grameen Foundation, Diplomat, Silicon Valley Leadership Network, Lockheed
Martin, Domo

© 1993-2013
1993-2012 Jeff Sutherland
Preorder discounted new Scrum book on Amazon
or Barnes and Noble and get a free signed copy
of Power of Scrum!

2012 Scrum Inc. © 2014 Scrum Inc.

2
© 2011-2014 Jeff Sutherland
Introduction – Your Facilitators
Dr. Jeff Sutherland – CEO, Scrum Inc. 
Jeff is the co‐creator of Scrum and a leading expert on how the framework has evolved to 
meet the needs of today’s business. The methodology he developed in 1993 and 
formalized in 1995 with Ken Schwaber has since been adopted by the vast majority of 
soNware development companies around the world. But Jeff realized that the benefits of 
Scrum are not limited to soNware and product development. He has adapted this 
successful strategy for several other industries including: Finance, healthcare, higher 
educaTon and telecom. 
As the CEO of Scrum Inc. and the Senior Advisor and Agile Coach to OpenView Venture 
Partners, Jeff sets the vision for success with Scrum. He conTnues to share best pracTces 
with organizaTons around the globe and has wriXen extensively on Scrum rules and 
methods. With a deep understanding of business process — gleaned from years as CTO/
CEO of eleven different soNware companies — Jeff is able to describe the high level 
organizaTonal benefits of Scrum and what it takes to create hyper‐producTve teams.

Alexander Brown – COO, Scrum Inc. 
Alex Brown is the Chief OperaTng Officer of Scrum Inc. He has deep experience in Agile/
Lean management techniques and business strategy. Alex has experTse in helping 
execuTve leadership and working teams collaborate effecTvely through common goals 
and metrics. He is acTvely involved in adapTng the Scrum methodology beyond its 
tradiTonal home in soNware development into other creaTve team environments, and 
has a parTcular interest in pushing the envelope in financial and progress reporTng in a 
Scrum context. 
Prior to joining Scrum Inc. Alex was a Principal at The Boston ConsulTng Group, where he 
led more than twenty projects to improve compeTTve posiToning and transform fortune 
100 companies to leaner and more agile operaTons. His project experience includes cases 
in the retail/consumer, IT manufacturing, healthcare, finance, and government sectors.

© 1993-2014 Jeff Sutherland

 As a group we need
Introductions
in order to work together effectively

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
Group Introductions

!
• Who’s in the group?
• Pair introductions
• Line up across the room by level of Scrum
experience
• Line up across the room by job function
• Line up across the room by size of organization/
department

© 1993-2014 Jeff Sutherland

Self-Organize Teams

• Based on line exercise, divide up into cross-

functional teams.
!
!
• Then:
• Select a team name
• Select a Product Owner
• Select a Scrum Master
• Create a learning backlog – what do you hope to get
out of the class individually and as a team

© 1993-2014 Jeff Sutherland

Scrum Boston

Do Doing Done

© 1993-2014 Jeff Sutherland

Course Overview
• Day One
• Sprint 1:
• Introductions 6/Agenda 2/History 9/Core Scrum 10
• Snowflake game 9/Roles 4
• How & Why Scrum works 3/Product Owner role 11
• Sprint 2:
• Vision 8/Backlog 10/User Stories 11/Personas 5
• Ready and Done 10

© 1993-2014 Jeff Sutherland

Course Overview
• Day Two
• Sprint 3:
• Daily Scrum 5/Learning Backlog 9/Business Goals 9/
Business Value 3
• Estimation Sizing 9/Sprint Planning 2/Patterns 9
• Sprint 4:
• Release Planning 13/Value Stream 3/Learning
Backlog 24
• Sprint Review 3/Retrospective, Happiness Metric 9
• Course Review and Retrospective 4
• Optional - Agile Contracts/Acceptance Tests/
Technical Debt/Distributed Scrum

© 1993-2014 Jeff Sutherland

 As a PO, I need to know
Where Scrum Comes From
to achieve my goals

© 2011 Scrum Inc.

History and Context of Scrum
• Jeff Sutherland developed the methodology in
1993 at Easel Corporation and formalized it in
1995 with Ken Schwaber.
• Inspired by wanting to solve a really hard
problem: SW projects kept getting later and
later and more and more expensive – knew
there had to be a better way.
• Formalizing the Scrum process was based on
empirical process design.

11

© 1993-2014 Jeff Sutherland

Defined Process
• Traditional waterfall development is a “defined
process.” A plan is defined at the beginning and
precisely followed to the end.
• This assembly line approach requires minimizing
deviations from plan to be successful.
• On average 65% of requirements change during
software development causing waterfall projects
to have an 14% worldwide success rate during
the past decade. (Jim Johnson, Standish Group, 2011)

Defined plan with one input and one output and (hopefully) no deviations 12

© 1993-2014 Jeff Sutherland

Empirical Process
• Controlling a process that has many unexpected
changes requires introducing a feedback loop in order
to inspect and adapt.
• Product is build iteratively and incrementally where
each set of features is fully operational after a short
cycle. Results are inspected and changes are made in
repeated cycles as work progresses.
• Inspecting and adapting require full transparency of
the work process to be successful.
• During the past decade, the worldwide success rate of
software projects developed with empirical processes
and been triple the success rate of defined projects.

13
Empirical plan with a new input after each cycle
© 1993-2014 Jeff Sutherland
 Scrum in Church
• Saving the World One Team at a Time

Rev.  Arline Conan Sutherland 14
© 1993-2014 Jeff Sutherland
MicroEnterprise Lending to the Poor

Giving Back to Nobel Laureate Grameen Bank

Lean
Scrum
Initiative
15

© 1993-2014 Jeff Sutherland

Ikujiro Nonaka: Grandfather of Scrum

Sutherland, Kenji, Nonaka - Tokyo, Jan 2011

The Japanese view Scrum as:

•A way of doing
•A way of being
•A way of life
16

© 1993-2014 Jeff Sutherland

Nonaka’s Project Management Styles
Requirements Analysis Design Implementation Testing

Type A – Isolated cycles of work NASA Waterfall

Fuji-Xerox Scrum

Type B – Overlapping work

Honda Scrum

Type C – All at once

The overlapping of phases does away with traditional notions about division of labor.
Takeuchi and Nonaka (1986)
17

© 1993-2014 Jeff Sutherland

Scrum team characteristics
!
• Transcendence (life)
• Autonomy (liberty)
• Cross-fertilization (pursuit of happiness)

18

© 1993-2014 Jeff Sutherland

 Lean Enterprise Institute

Scrum Lean

Product Production
Creation Techniques

19

© 1993-2014 Jeff Sutherland

 Lean is Based on the PDCA Cycle
9th Hidden Turning Point in History
U.S. News and World Report, 21 Apr 1991 - see also J. Womack, D. Jones, D Roos, “The Machine
that Changed the World: The Story of Lean Production.” Harper Perennial, 1991.

• PLAN: In Scrum, the Product Owner has a product

backlog.
• DO: Team executes the sprint backlog.
• CHECK: The Product Owner inspects the results of
team work.
• ACT: The Scrum Master facilitates a retrospective.
• Continuous improvement is the outcome of the
PDCA cycle. This and respect for people is the
Toyota Way as well as the Scrum Way.

20

© 1993-2014 Jeff Sutherland

 Agile Manifesto

www.agilemanifesto.org!
We are uncovering better ways of developing 

software by doing it and helping others do it. 

Through this work we have come to value:!
!
Individuals and interactions over processes and tools!
Working software over comprehensive documentation!
Customer collaboration over contract negotiation!
Responding to change over following a plan 


That is, while there is value in the items on 


the right, we value the items on the left more.

21

© 1993-2014 Jeff Sutherland

As a PO, I must be able to articulate
the Elements of Core Scrum
clearly so I can work effectively with
the team and stakeholders

22

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
Elements of Core Scrum
!
• Roles
• Meetings (ceremonies/rituals)
• Artifacts (social objects)
• Scrum Values (courage, commitment,
focus, respect, openness)

23

© 1993-2014 Jeff Sutherland

Scrum has Three Roles
1. Product Owner:
• Define and prioritize the features of the Product Backlog
• Decide on release date and content PO

• Responsible for the profitability of the product (ROI)

!
2. ScrumMaster
• Facilitates the Scrum process and Team self-organization
• Removes obstacles SM
• Shields the team from interference
!
3. Team
• Cross-functional (incl. testing)
• Self-organizing/-managing group of individuals, autonomy regarding
how to achieve its commitments
• Typically 5-9 people
T T T

24

© 1993-2014 Jeff Sutherland

Scrum has Four Meetings

• Sprint Planning
• Product Backlog must be READY
• Daily Scrum
• maximum 15-minutes, 3 questions
• Self-organize to improve performance
• Sprint Review
• Decide what is DONE. That determines velocity.
• Retrospective
• Identify the top process improvement and put in the
backlog for the next sprint
• Backlog Refinement is essential before Sprint Planning
• Refine upcoming backlog to ensure it is READY

25

© 1993-2014 Jeff Sutherland

Scrum Makes Work Visible

• Product Backlog
• Sprint Backlog
• Scrum Board
• Burndown Chart
• Show work remaining
• Velocity
!
Scrum is designed to be self-reporting

26

© 1993-2014 Jeff Sutherland

The Sprint

What is it?
• A cycle of work
• A team-determined length of time in which the team
commits to producing a meaningful increment of work
• Timeboxed and usually lasts 1-4 weeks
!
Why do it?
• A fixed anchor
• A tool that allows a team to calculate velocity
• A period of time in which the team can derive lessons
for the future
• A fixed planning horizon
27

© 1993-2014 Jeff Sutherland

The Sprint Length
Too long, too short or just right? How do you
determine the length for what you do?
!
• The Sprint cycle should be long enough to produce a
meaningful increment of work and short enough to
maintain momentum.
• Long enough to realize improvements and short enough
to identify improvements for next Sprint.
• A fixed duration that is never extended.

28

© 1993-2014 Jeff Sutherland

“Velocity” is the Key Metric in Scrum
Start of the Sprint End of the Sprint

Product Sprint Sprint

Backlog Backlog Backlog
The team pulls their
desired number of
stories into the Done!
8 current sprint 8 8
Actual velocity
Estimated Done! = 18 points
5 5 velocity
5
5 = 26 points Done! 5
5
Almost done
3 3 3
Not started
5 5 5
5
User stories that are not
8 Each user story includes completely done at the end of
an estimated number of the sprint do not count toward
5 “points” as a measure of velocity, and are carried into
effort required to the next sprint.
3 complete !
5 Teams can also pull stories
from the top of the product
backlog if they finish the full
sprint backlog early
29
Adapted from materials by Henrik Kniberg
© 1993-2014 Jeff Sutherland
 Release Burndown Chart Makes Team’s
Velocity and its Implications Visible

Points Points
V

Apr
2008
May
2008
June
2008

Q3
2008

Q4
2008

2009

Sprint/Time Sprint/Time

30

© 1993-2014 Jeff Sutherland

Sprint - Iterative and Incremental

31

© 1993-2012 Arline Sutherland

Tracking Velocity Over Time Allows Team to
to Measure Output Improvement
Raw Scrum Inc. Velocity History
Points
(not adjusted for fluctuaTon in team capacity by sprint)

300

225

Team Velocity

150
12x output with 3x FTEs 

75
Holiday 2011

0
1 11 23 33 45 55 69 81 91 101 111 122 132 142 Sprint

Source: Scrum Inc. performance data
32

© 1993-2014 Jeff Sutherland

 The Scrum Framework

Product  Product 
Backlog Scrum PO Owner

Scrum 
Sprint  3 Social Master
Backlog 3 Roles SM
Objects

Make  Scrum Board 

Work  Points  T
Velocity  T Team
Visible
Burndown Chart T
5 Ceremonies

Sprint Planning Daily Scrum Backlog Refinement Sprint Review Sprint Retrospective
Sprint Backlog Replan Ready Backlog Product Increment Kaizen
Velocity
Feedback 33

© 1993-2014 Jeff Sutherland

 Paper 





Snowflakes
34

© 2011 Scrum Inc.

The Exercise

• The objective is to run a profitable business by 
creating and selling paper snowflakes. 
• One team per table

35

© 1993-2014 Jeff Sutherland

 36

© 1993-2014 Jeff Sutherland

We Buy

Snowflakes 
 

1‐5 coins

37

© 1993-2014 Jeff Sutherland

We Sell

2 papers for 1 coin 
1 pair of scissors for 3 coins

38

© 1993-2014 Jeff Sutherland

Tracking at the end of each round

Round: Start 1 2 3 4 5 6 7

Number of 
produced 
Number of  0
sold 
Amount Sold  0
for (outcome)
Total Cash  5
(impact)

39

© 1993-2014 Jeff Sutherland

Debrief – Discuss within your teams

• How did you find out what to produce? 
• Where did you spend most time? 
• Produce, learn, sell… ? 

• What would you do differently if you would do this 
exercise again? 
• What will you change in your way of working at 
your workplace based on reflection of this 
exercise?

40

© 1993-2014 Jeff Sutherland

 As a PO, I need a clear
understanding of
All the Roles in Scrum
in order to work well together

41

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
Scrum Master Role &
Responsibilities
• Facilitator
• Knowledgeable about the Scrum process
• Coach – to enhance team performance
• Removes impediments
• Protects the team from interruptions
• Holds Scrum values and practices

42

© 1993-2014 Jeff Sutherland

The Scrum Master Owns the
HOW
• Scrum is a simple framework that requires
consistent discipline
• Scrum Master owns the process
• Facilitates Daily Stand-Up
• Facilitates Sprint Planning
• Facilitates Retrospective
• Protects the team
• Removes impediments

43

© 1993-2014 Jeff Sutherland

Teams are:

• Cross-functional - most members can do more

than one thing
• Self-organizing - they decide how they will work
• Self-managing - they decide how much work
they can do in a Sprint

44

© 1993-2014 Jeff Sutherland

Management -> Leadership

• Need a business plan that works

• Provide resources the team needs
• Know the velocity of teams
• Remove impediments that slow teams down
• Provide challenging goals for the teams
• Hold Product Owners accountable for value
delivered per point
• Hold Scrum Masters accountable for process
improvement and team happiness

45

© 1993-2014 Jeff Sutherland

Exercise: Project Manager/Leader

• As a team, write down all responsibilities of

a traditional project manager/leader
• Put one responsibility on each sticky note
• 4 minutes

46

© 1993-2014 Jeff Sutherland

Exercise: Project Leader

!
• Arrange the sticky notes in these categories
• Product Owner
• ScrumMaster
• Team
• Waste
• Other
!
• 5 minutes

47

© 1993-2014 Jeff Sutherland

 As a PO, I need a clear
understanding of the
Product Owner Role
in order to perform this role well

48

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
Product Owner Owns The WHAT
• Have a compelling product vision that is executable,
generates lots of cash, and arouses passion in the team,
company, and customers
!
• Build a roadmap for rolling out the vision that everyone
can see and sign up for
!
• Build a Product Backlog of “enabling specifications” that
are “just enough, and just in time.”
!
• Spend half the time with customers, sales, and marketing
in order to be the customer proxy on the team.
!
• Spend the other half working closely with the team
clarifying specifications.

49

© 1993-2014 Jeff Sutherland

The Product Owner Must Balance
Multiple Product Attributes

50

© 1993-2014 Jeff Sutherland

 A Successful Product Owner…
• Delivers:
• The right product set to excite customers

Value
• At the right time
• In the order that maximizes business value Time

!
• Responds dynamically to change faster than
competitors
• Clarifies customer need to development teams so that
Product Owner
uncertainty is removed and developer velocity is
maximized
!
• The Product Owner is ultimately accountable for
winning in the market!

$ 51

© 1993-2014 Jeff Sutherland

 Right Order Maximizes Revenue
• A good Product Owner will get at least 20% more
revenue by delivering the right features in right
order compared to typical companies
! $
! Billions of ways
! to order

!
• 80% of value is in 20% of features. If team
velocity is 5x and you deliver the “right” 20% of
product you are 5x5=25 times faster and will
disrupt waterfall competitors

Risk,
Mark Denne and Jane Cleland- Huang. Software by Numbers: Low-
High-Return Development. Prentice Hall 2003.

52

© 1993-2014 Jeff Sutherland

To Do This, A Product Owner
MUST…
• Be Knowledgeable, Available, Decisive, Accountable
• knowledgeable about the customer and product
• available to the team to clarify goals and desired output
• empowered and able to make clear and rapid decisions
• measured by value delivered per point
• Focus on Business Goals
• know company strategy and market segmentation
• generate value based on business plan, ROI, revenue,
customer satisfaction, service measures
• review team’s output to ensure quality
• Launch a Minimum Viable Product and Measure
Outcomes
• early validation of the market is critical
53

© 1993-2014 Jeff Sutherland

Trust is Essential for a Product
Owner
• Product Owner will break trust if:
• He tells people how to implement the product
• She assigns people tasks
• He changes the Sprint Backlog during a Sprint
• The developers find out the Product Owner doesn’t really
know what the customer wants
• She tries to force team to do what they will not sign up for
• Any compromise of integrity or neglect of the team.
!
• The Product Owner is a special kind of leader and will
be held accountable by the team for leadership
qualities - honesty, integrity, clarity, and ability to
align the whole company behind product creation.
54

© 1993-2014 Jeff Sutherland

 Product Owner is a Big Job
• Initially, one Product Owner may be able
to generate ready backlog for several
P teams.
TTT TTT TTT • As team velocity increases, a Product
Owner team, led by a Chief Product
Owner, will be needed.
• The Product Owner team are domain
experts that describe the user
experience, the screen shots, the
workflow, the data requirements, the look
and feel as needed to meet the Definition
PO
CPO
PO
of Ready.

T T T T T T T T T

55

© 1993-2014 Jeff Sutherland

“Ramping Up” as Product Owner

1. Get one sprint of READY backlog

• Team can get started
!
2. Get two sprints of READY backlog
• Team can go sprint to sprint
!
3. Build out Release Plan
• Company can predict revenue
!
4. Build one year roadmap
• Customers can be briefed on company strategy
56

© 1993-2014 Jeff Sutherland

How Do You Find the Right Product
Owner?
• What happens if the Product Owner is a
business analyst?
• What happens if the Product Owner is the
ScrumMaster? Works on the team?
• What if the Product Owner is a business
leader who does not have enough time
available to produce a good backlog?
• How would you find the right Product
Owner?

57

© 1993-2014 Jeff Sutherland

 As a PO, I need to know 

How Scrum Works because these principles "
should guide me adapting Scrum to my
team’s needs

58

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
 Why Scrum Works
Change and uncertainty are not an inconvenience, but a source of
Embracing
1 opportunity. Organizations that manage uncertainty better possess
Change a strategic competitive advantage
Everyone, from the CEO to entry-level employees, is responsible
2
Self-
for getting work done and empowered to decide how to do it. Any
Organization structure more than the minimum needed to work is WASTE

Business All activities are viewed through the lens of what creates value for
3 the business. Every sprint, we prioritize the work that generates
Value Focus the most business value at that point in time
Every day and in everything we do, we think about how to work
Continuous
4 better and then follow-through to remove the top impediment. Our
Improvement iterative work approach allows us to improve faster
We work best when we focus on one thing until it is DONE, all the
5
Non- way thru to a saleable product. Multi-tasking imposes switching
interruption costs on us, and is less efficient

Customer- The customer is the core motivation for our work. We are eager to
6 discover what they like, and our highest priority is to satisfy the
Centrism customer early and often

2014 Scrum Inc.

Radical Having everyone know where we stand in as close to real-time as
7 possible allows us to make better and more responsive decisions
Transparency
© 1993-2014 Jeff Sutherland
Remove “Waste” to Improve Efficiency
Sources of waste From Taiichi Ohno’s “The Toyota Way”
Partially implemented user stories, bugs or incomplete
In-Process Inventory work that cannot generate business value
Working on low-value features that customers don’t care
Overproduction about rather than saving capacity for high-value work

Unnecessary management processes, redundant quality

Extra Processing checks, relearning others’ work

Muda Transportation Handoffs across roles, teams, divisions and so on

(Wasted Effort)
Switching back and forth between tasks or “multi-tasking,”
Wasted motion delays from interruption of the sprint

Delays, dependencies, capacity imbalances resulting from

Waiting specialized capabilities without cross-training

Fixing bugs or other errors that should have been caught

Correcting Errors earlier or systematically avoided to begin with

Mura Unevenness Varying granularity of work between team members

(Variability Different definitions of “Done” and process variations that

Inconsistency
Waste) make defining “potentially shippable” product difficult

Absurdity Stress due to excessive scope

Muri Stress from an expectation that heroic actions to save the

(Emotional Unreasonableness day are normal

2014 Scrum Inc.

Waste) Overburden Stress due to excessive workload from “overhead” tasks

© 1993-2014 Jeff Sutherland

How Scrum Works

2014 Scrum Inc.

© 1993-2014 Jeff Sutherland
 As a PO, I need to know what a
Great Product Owner
looks like so I can be certain to win.

© 2011 Scrum Inc.

To Execute Well You Need to Understand:

• John Boyd’s OODA cycle

"The Iraqi army collapsed morally and intellectually

under the onslaught of American and Coalition forces.
John Boyd was an architect of that victory as surely
as if he'd commanded a fighter wing or a maneuver
division in the desert." Charles Krulak, Commandant
of the U.S. Marine Corps

63

© 1993-2014 Jeff Sutherland

OODA Loop Drives Product Backlog Creation
“The fundamental, unavoidable and all-pervasive presence of
uncertainty is the starting point.” John Boyd, 1975, Destruction and Creation

“I am on the verge of a fantastic breakthrough in the thinking processes

and how they can be taught to others.” John Boyd, 1972, Thailand

64

© 1993-2014 Jeff Sutherland

Observation
• The market is rapidly changing
• The company is changing
• Understanding of the business and product
line is changing
• The Product Backlog is changing

65

© 1993-2014 Jeff Sutherland

Orientation
• The centerpiece of the OODA loop
• Formed through genetic heritage, cultural
tradition, previous experiences, and unfolding
circumstances.
• The product is formed by the Product Owner’s
personality (think Steve Jobs)

66

© 1993-2014 Jeff Sutherland

Decision

• Boyd’s goal was to reduce decision time

to effectively zero.
• Decidability is a key aspect of the Product
Owner job description.
• How effective are you in making difficult
time-critical decisions?
• How effective are you in motivating a
team to execute them?

67

© 1993-2014 Jeff Sutherland

Act

• Execute an intervention.
• This provides feedback that sets up another
OODA loop.
• Are you testing your hypotheses in the market,
then pivoting?

68

© 1993-2014 Jeff Sutherland

Product Backlog is Fast Changing (OODA)
Sprint Backlog is Fixed (PDCA)
Product Sprint
Backlog Backlog
• Anyone can put anything
on the Product Backlog
8
8 • Product Owner orders the
5 5 backlog
5 5
3
• Stable Sprint Backlog will
3
5 5 increase velocity of team
5 • Small batches increase
8 velocity
5
3
5

69

© 1993-2014 Jeff Sutherland

 As a PO, I need a
Product Vision
to motivate stakeholders and the team

© 2011 Scrum Inc.

Exercise
• Choose a PO, SM, a User, and a Product for your team
• Agile Project Management Tool
• A tool to help manage Scrum teams, projects and processes. Release/
sprint planning, quality visualization, forecasting, team skillset and
process evolution, etc.

• Storefront Designer
• Let companies design their own stores in an interactive, visual
environment. Simulate traffic flows, choose from templates, cost out
components from multiple suppliers, etc.

• Performance Tracker
• Allow people to track and see their progress in any area. Track times,
sports scores, leaderboards, etc.

• Physical Book Store

• Set up a bookstore in a shopping mall with real books for sale.

71

© 1993-2014 Jeff Sutherland

Exercise: List and Rank Problems
“Big opportunities come from overcoming big problems.”
Randy Ottinger discussing Elon Musk

!
• Discuss the top problems your User is having
!
• Ask your User to rank these problems in order of
severity
!
• Ask how they solve these problems today
(alternatives)

72

© 1993-2014 Jeff Sutherland

Vision

• It all starts with a Vision

• Build a car that gets 100 miles per gallon that is a safe,
low-cost, ultra-efficient, road-legal vehicle.
• Product Owner is responsible for the Vision
• PO works with management to craft Vision or
takes a business case
• PO shares Vision with the Team.
• Vision helps to clarify and focus goals

73

© 1993-2014 Jeff Sutherland

 ScrumLab Vision Statement

!
• FOR - experienced Scrum practitioners (Jill) and
beginners (Bob)
• WHO - need clear and precise information about Scrum
• PRODUCT NAME - ScrumLab
• Is a repository of 20 years of Scrum knowledge captured
by the inventor of Scrum
• That clarifies the essentials and explains best practices
• Unlike other more superficial Scrum resources
• Our Product will show you how to create great teams!
74

© 1993-2014 Jeff Sutherland

Exercise: Craft A Vision Statement
• Why and for Whom would your system do
What? How is this product better than the
competition?
• FOR - target audience
• WHO - have some problem - get to root by asking why
• The PRODUCT NAME
• Is a PRODUCT FUNCTION
• That has KEY FEATURES
• Unlike ALTERNATIVES
• Our PRODUCT DIFFERENTIATOR
• Write and be prepared to present your
Vision
75

© 1993-2014 Jeff Sutherland

 As a PO, I need to understand the
Product Backlog so I can build a
better product & deliver value quickly

76

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
The Product Backlog
• An ordered list of everything that might be needed in
the product
• Composed of product “features” that describe the product
• THE single source of requirements for any changes to be
made to the product and focus of team discussions
• The Product Backlog is shared across teams working on
the same product to drive coordination
• There is only one Chief Product Owner who ultimately
owns the product backlog
• For scaling we want one enterprise backlog which may
have multiple products
• Ordered to provide maximum return on $
investment, accounting for: Billions of ways to
• Market demand order Product
Backlog
• Team capability
• Risk
77

© 1993-2014 Jeff Sutherland

PB a Working Vision of the Product

Unlike the Sprint Backlog, it can change at any time

Product Sprint
Backlog
• Any stakeholder can add anything to
Backlog
the Product Backlog
8 8
• Product Owner orders the backlog
5 5
5 • Backlog items that are lower priority
5
3 3 can be defined more roughly
5 5
• The PO and team should spend time
5
each sprint to “refine” the backlog
8
• Build in lessons learned from earlier work
5
3 • Specify feature definitions so they are
5 “ready” when the time comes

78

© 1993-2014 Jeff Sutherland

How to do it

Big
bucks

Zillions of ways to order Product

Backlog

79

© 1993-2012 Jeff Sutherland

Product Backlog Items
• aka PBI’s
• Issues, features, user stories, use cases,
functions, requirements, specifications, fixes
• Are in the customer space: what, not how
• Can start out very rough but evolve into
enabling specifications if needed
• Each potentially shippable increment includes
one or more PBIs

Specification defects (cost 10 hrs/defect)

• Clear enough to test
• Unambiguous to intended readers
• No design options
80

© 1993-2014 Jeff Sutherland

Increase Granularity of Features
• Product Backlog Items, often User Stories, are broken down
into smaller and smaller pieces.
• The Process helps to create a conversation
• The simpler the description the easier to get to done.
• During the Sprint, the Team needs to re-visit items further
down the backlog and make sure they’re ready to move into
the Sprint backlog during the next Sprint Planning meeting.

2012 Scrum Inc.

81
© 1993-2012 Jeff Sutherland
Qualities of a Product Backlog

• Detailed - more so at the top

• Estimable
• Emergent
• Prioritized
!
• PBs can take many forms and may well include
charts, wire frames, models etc

82

© 1993-2014 Jeff Sutherland

Not All Features Are Created Equal!
80% of 
value 
typically 
resides in 
50.0000 20% of 
features
Value to Customer

37.5000 65% of features provide liXle to no value, 
are rarely used and/or aren’t actually 
desired by the customer
25.0000
The rest are OK, 
but not as 
12.5000 important

0.0000

Features

How can you tell ahead of time which

features add value and which don’t?
83

© 1993-2014 Jeff Sutherland

What Do We Build First?
Value

High Value High Value

Low Effort High Effort

Effort

Low Value Low Value

Low Effort High Effort

84

© 1993-2014 Jeff Sutherland

What We Build First Depends on
Competitive Environment

• MoSCoW
– Must have this
– Should have
– Could be nice to have
– Won’t have this – maybe later

85

© 1993-2014 Jeff Sutherland

Earned Value - cost per sprint
• You have a project backlog estimated
at 240 points
• The team has a velocity of 40 points (2 week
iterations)
• The team’s cost is $50,400 per iteration (7 people @
$90/hour x 80hrs)
• Schedule is 12 weeks (6 iterations)
• Budget is $302,400 ($50,400 × 6)
• EVM CONVERSION
• PV = $50,400 per iteration (your planned costs)
• EV = $1260 per Story Point delivered ($302,400 /
240 = $1260)

86

© 1993-2014 Jeff Sutherland

Earned Value Calculation
• ITERATION #1
• The team delivered 42 story points
• EV = $52,920 ($1260 × 42)
• AC = $50,400 (actual cost)
• PV = $50,400
!
• CPI = 1.05 (EV/AC) – you’re under budget!
• SPI = 1.05 (EV/PV) – you’re ahead of schedule!
!
• ITERATION #2
• A develop resigned from the team effective day-1 of the iteration. They’re not
replaced
• The team delivers 38 story points
• EV = $100,800 ($1260 × 38 + $52,920)
• AC = 93,600 ($43,200 + $50,400)
• PV = $100,800
!
• CPI = 1.08 (EV/AC) – your under budget!
• SPI = 1.0 (EV/PV) – you’re exactly on schedule!
87

© 1993-2014 Jeff Sutherland

Kano Analysis

• Noriaki Kano: Quality is subjective

– Kano analysis is a quality measurement tool used to prioritize
customer requirements based on their impact to customer
satisfaction. [John Carter, isixsigma.com]
• We can divide perceived quality into four groups
– Exciters: positive, beyond expectation
– Performers (or Satisfiers): linear qualities – the more the better
– Basic needs: we expect them to be there, if not we are
dissatisfied
– Indifferent: we don’t expect them, and we don’t care. Some
might be annoying.

88

© 1993-2014 Jeff Sutherland

 Establishing Business Value
Completely Satisfied Customers

WOW!
To make a
competitive
Want
push…

Need All
Not INDIFFERENCE Needs
Met Met
Exciters/Delighters

Must

r es
atu
r Fe
a Must Haves
iL ne Dissatisfied Customers R
TM

www.RapidScrum.com
2011 Copyright Rapid Scrum LLC. All rights reserved.
 As a PO, I need clear 

Epics and User Stories to
communicate what needs to be done.

90

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
Definition of an Epic
• An Epic is a Product Backlog Item or User Story
that is too big to be completed in one Sprint
• Simple Epics may be small enough to be
completed in as few as two Sprints
• Huge Epics may take the entire company several
quarters or years to complete
• Simple Epics need to broken down so that the
Team can deliver value in a given Sprint - Done at
Backlog Refinement
• Larger Epics require the Product Owner to work
with Leadership and the Team to create a Road
Map so most valuable features created first

91

© 1993-2014 Jeff Sutherland

Epics as PBIs

• Most User Stories or PBIs as originally written

are Epics
• Usually written by a Product Owner or a
customer with knowledge of the product but not
of the development process.
• Backlog Refinement meeting is where the Team
works with the Product Owner to break the Epic
down appropriately

92

© 1993-2014 Jeff Sutherland

Epics and Business Value

• Epics are components of the enterprise’s vision

• Business value can best be estimated at this
level

93

© 1993-2014 Jeff Sutherland

Exercise: Create Epics and User
Stories
• Create 2 or 3 Epics that outline your product’s
higher level goals - one sticky note per Epic
• Write at least 5 lower level User Stories under
one of the Epics - one sticky note per Story
• Refine the definition of any stories that do not
meet the INVEST criteria, splitting the stories
that now see are Epics, as needed.
!
• Prioritize the stories.

94

© 1993-2014 Jeff Sutherland

Break Epics into Stories

As a frequent flyer I
want to book a trip using
As a frequent miles so that I can save
flyer I want to money

book flights As a frequent flyer I

customized to want to easily book a
trip I take often So that I
my can save time

preferences, As a premium frequent

so I save time flyer I want to request an
upgrade So I can be more
comfortable

95

© 1993-2014 Jeff Sutherland

The Origin of User Stories

• User Stories derive from XP

• Now widely used for all agile processes
• While not required, most Scrum teams use User Stories
!
• Written from the end user’s perspective
• Helps Team visualize product end use
!
• Card
• Template, notes
!
• Conversation
• Face to face communication is part of the specification
!
• Confirmation
• Include internal acceptance tests
96

© 1993-2014 Jeff Sutherland

User Story - Guidelines
Immediately actionable
Product vision
Negotiable
Product
Backlog
Valuable
A Estimable
Sized to fit
B
Testable
Modified from Bill Wake – www.xp123.com

C C
A B C
GUI

Client
Server
DB schema
97

© 1993-2014 Jeff Sutherland

User Story Template

• As a <role> I would like to be able to <action> so

that <business value>

98

© 1993-2014 Jeff Sutherland

What’s Wrong with This Story?

99

© 1993-2014 Jeff Sutherland

User Story Mapping
• Epics at top, stories underneath
• Shows workflow
• Can be large features, company initiatives
• Two dimension view easier to understand than
linear ordering
• Tool for identifying MVP
• Allows the team to see the big picture

100

© 1993-2014 Jeff Sutherland

 As a PO, the Team and I need clear
definitions of READY and DONE so the
Team can complete the Backlog quickly and
effectively

101

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
What does it mean to be READY?

1. Defined clearly enough that all members of the team

understand what must be done
• Assume some ongoing discussion to refine, coordinate and clarify

2. Includes clear statement of resulting business value

that allows the Product Owner to prioritize
3. Includes any required enabling specs, wire frames, etc.
4. Fully meet INVEST criteria for user stories
• Estimated and sized to complete easily within one sprint

5. Free from external dependencies

• I.e. there is nothing beyond the team’s control that must be done
first in order to complete the story

102

© 1993-2014 Jeff Sutherland

Moving toward READY

• The Team regularly estimates items as part of

“Refining” the backlog
– Expect to spend ~10% of team time
– Often once a week at "The Wednesday Afternoon Meeting"
– Update at the Sprint planning meeting
!
• The team needs Product Owner support to clarify
PBIs
!
• Product Owner supports and informs the team, but
may direct them only through the product backlog

103

© 1993-2014 Jeff Sutherland

 User Story Readiness
Progression
• All inputs accepted
New Card • Promotion: Product Owner determines this story
Nursery matches product goals

• Analysts decompose
Increasing Readiness

Elementary • User experience experts research context

• Business alignment needs identified
School • Promotion: Matches release goals

• Card details, acceptance criteria, UI pre-work

(wireframes, visual and content prototypes
Junior High • Legal & compliance issues reviewed
• Promotion: Alignment with key stakeholders on
features, functions, and visuals

• Ready for sprint

High School • Candidates for Release Planning/Sprint Planning
• Minimal refinement expected on core User Experience
104

© 1993-2014 Jeff Sutherland

Refining the Product Backlog
Register new Register new
user user

Edit existing Edit existing

user user
Administrate
users View Invoice in
Find user HTML, PDF, or
Excel format
View Invoice in As a helpdesk
HTML, PDF, or Delete user operator I want to
Excel format see who is logged
in
As a helpdesk View Invoice in
operator I want to HTML, PDF, or
see who is logged Find user
Excel format
in
As a helpdesk Operations
operator I want to
Operations see who is logged manual
manual in
100
Operations simultaneous
100 manual users
simultaneous
users 100
simultaneous Delete user
users 105
Source: Henrik Kniberg
© 1993-2014 Jeff Sutherland
Team Works to Balance the
Product Backlog
Stories too big Stories too small Just right

106

© 1993-2014 Jeff Sutherland

Product Owner Roadmap

| Q1 | Q2 | Q3 | Q4 |

One year roadmap is developed in consultation

with customers and leadership. It is a living
document that is usually refined in quarterly
meetings. A one year roadmap is adequate.
107

© 1993-2014 Jeff Sutherland

What Does it Mean to be DONE?

1. “Definition of Done” (DoD) decided on

beforehand – along with acceptance tests
• DoD can be standard across a group of common
stories, or defined specifically for unique ones

2. Done means the feature has been developed,

tested AND meets all required acceptance tests
3. Ideally, Done means the feature could be
shipped to a customer
4. Product Owner officially “Accepts” Done features
back from Team at the Sprint Review meeting

108

© 1993-2014 Jeff Sutherland

 Some Definitions of Done

Default Definition of Done Default Definition of Done

• Releasable • Acceptance tested
• Release notes written
• Releasable
• No increased technical debt
Default Definition of Done
• Unit/Integration tested
• Ready for acceptance test
• Deployed on demo server = I haven’t messed up
the codebase

What else must be done before shipping the code?

- For example ”customer acceptance test + user documentation”
Why not? Who does it? When? What happens if a problem turns up?
Burn up this work in release burndown!
109
Source: Henrik Kniberg

© 1993-2014 Jeff Sutherland

Teams Succeed by Driving READY Stories to
DONE, while Removing Impediments
 
Sprint   
Daily 
Retrospec-ve Remove Impediments Standup

READY!

DONE!
Velocity

2013 Scrum Inc.

As a PO, I must live the point of view of
Customers and Personas
to ensure that we will delight them

111

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
 A Successful Product Must
Delight All “Customers”

Internal
Users Purchasers Influencers
Stakeholders
• Interact directly with • Make buying or • Interface with the • Influence scope,
the product adoption decisions product or its users priorities, budget
and schedule
• Knowledge of • Have their own wish • Support, install,
current usage lists that may have deploy or benefit • Assist with or may
patterns helps to little to do with the from use of the be dependent on
design better, more users’ needs product product releases
usable products
• Make purchasing • Can also wield • Can constrain or
• Unsatisfied users decision, so if they significant influence evaluate
work around the aren’t happy, you on decision to architecture or
product, nullifying won’t get in the purchase, retain or product
its benefits and door change products development
eventually processes
eliminating it

112

© 1993-2014 Jeff Sutherland

Use Customer “Personas” to Focus

Goal Directed Design by Alan Cooper

• Not a real person, they are archetypes

 Real people are too quirky
 Discovered as byproduct of investigation process

• Precise description of a user and what he wishes to

accomplish
 Design for just one person

• The Elastic User doesn’t exist, be specific.

 Precise not accurate
 Give realistic look at skill levels
 Personas end feature debates

• Cast of characters
 Primary persona is critical
 Probably needs separate user interface
113

© 1993-2014 Jeff Sutherland

An Example Persona

MSDN Magazine > Issues > 2009 > April > The Power of Personas
114

© 1993-2014 Jeff Sutherland

Exercise: Generate Personas
• Brainstorm Personas that would represent
customers, users, and stakeholders of your
product.
• Choose a high-priority Persona.
• Write a brief story that outlines the motivations
and goals of this Persona.
!
Vision Statements, Epics, and User Stories often
contain Personas

115
http://www.agilemodeling.com/artifacts/personas.htm

© 1993-2014 Jeff Sutherland

Exercise: Create a Scrum Quiz

What are the top two priority questions

on your backlog for today’s material?

116
© 2011 Scrum Inc.
As a PO, I need to know my role in the
Daily Scrum
to help the team move faster

© 2011 Scrum Inc.

Teams Succeed by Driving READY Stories to
DONE, while Removing Impediments
 
Sprint   
Daily 
Retrospec-ve Remove Impediments Standup

READY!

DONE!
Velocity

2013 Scrum Inc.

 © 2014 Scrum Inc.
All Blacks, New Zealand http://www.youtube.com/watch?v=0C434QFTjok © 2011-2014 Jeff Sutherland

119

© 1993-2014 Jeff Sutherland

Purpose of Daily Scrum

• Intensify team focus

• Collaboration and clarification
• Crush impediments
• Motivate team spirit

120

© 1993-2014 Jeff Sutherland

What is the Daily Stand-Up?

• 15 minutes, same time every day

• 3 questions:
• What did I do yesterday that helped the Team meet the
Sprint goal?
• What will I today to help the Team meet the Sprint
goal?
• Do I see any impediment that prevents me or the Team
from meeting the Sprint goal?
• Not a status meeting
• If further discussion is needed, takes place after
Stand-Up

121

© 1993-2014 Jeff Sutherland

Exercise
• Daily Scrum
• What did you learn yesterday?
• Update your Scrum Board

• What is top priority for today?

• Prioritize your backlog

• Any process improvements?

• Scrum of Scrums
• All teams report
• Will all teams complete the release successfully?
• (ready for certification at 5pm today)

122

© 1993-2014 Jeff Sutherland

Product Owners Job at Daily Meeting

!
• LISTEN!!!
• Motivate the team
• Clarify business value
• Answer questions about the backlog
• Share as appropriate

123

© 1993-2014 Jeff Sutherland

Care About The Whole Product We Rock!

Drive Stories to Done...

As a Team!

Not just your little task!

© 2014 Scrum Inc.

I’m more
eicient if I just work
on my stuf...

124
© 2011-2014 Jeff Sutherland
Warning Sign #1

To Do: Doing: Done!

Sprint Goal:
Get a ready release!

Buffer Burndown
33 Points

100

90
Daily Clean
Code 80
Kaizen 3pts
70

60

Points
code 50
cleanup. 2
write a failing
pts 40
Deposit DB Design test.. 2 pts.
Integration test 3pts DAO 30
2 pts 3pts
20

10
GUI spec 2
pts 0
code write a failing
Migration ToolTapestry ry cleanup. 2 test.. 2 pts.
spike pts Miigration 8
8 pts pts Days

Implement Integrate w// write a failing

Miigration 8 pts
Backend Login GUI
5 ots
boss
8 pts
test.. 2 pts.

GUI design Clarify Req

5 pts 3 pts
Backend User write a failing Implement
test.. 2 pts.

© 2014 Scrum Inc.

Admin GUI
5 ots

125
© 2011-2014 Jeff Sutherland
 As a PO, I need to
Measure Business Goals
to adapt quickly to market dynamics

© 2011 Scrum Inc.

Keys to Business Success

• Get more customers than you lose.

• Increase the rate at which you gain customers.
• The cost of getting a customer must be less than
the customer pays.
• Your customers must be happy enough to refer
other customers.

127

© 1993-2014 Jeff Sutherland

Business Goals - measurable targets
that support/confirm your vision
• A goal might span multiple releases
• For example, goal/outcome might be: Increase
retention on eBanking Website.
• Goals need specific outcome measures
• You should try to make this more concrete:
Increase retention of small business eBanking
customers through a 20% reduction in monthly
cancellations.

128

© 1993-2014 Jeff Sutherland

Examples of Outcome Measures
• Sticky engine of growth
• The churn rate is defined as the fraction of customers in
any period who fail to remain engaged with the
company’s product.
• If the rate of new customer acquisition exceeds the
churn rate, the product will grow.
• Viral engine of growth
• Paid engine of growth

129

© 1993-2014 Jeff Sutherland

Viral Engine of Growth - Hot Mail
• 1996 - slow growth until adding clickable link
“Get your free e-mail at Hotmail”.
• 6 months to 1 million subscribers, 18 months to
12 million subscribers
• Sold company to Microsoft for $400M

130

© 1993-2014 Jeff Sutherland

Paid engine of growth

• Lifetime value of customer (LTV) is amount

customer pays minus variable expenses
• Cost per acquisition (CPA) is cost of sales and
marketing to acquire a new customer
• If LTV is larger than CPA the company will grow

131

© 1993-2014 Jeff Sutherland

Net Promoter Score

Source: Satmetrics.com

132

© 1993-2014 Jeff Sutherland

Exercise: Outcomes

• What outcome measures would be a good

leading indicator of success for your team’s
product
• Write at least three quantifiable goals that you
can measure to see progress towards your
product’s Vision

133

© 1993-2014 Jeff Sutherland

As a PO, I need to know how to estimate
Business Value and ROI so that I can
order the Product Backlog effectively

134

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
A Focus on Business Value
Fundamentally Changes Value Delivery

135

© 1993-2014 Jeff Sutherland

Business Value
• Determined first at the Epic level (usable
features)
• What Epics will maximize ROI
• Epic BV can be allocated to Features in proportion to
effort (a quick approximation)
• Determined in Release Planning
!
• Consult with your Finance Department
• They probably have an established ROI model
• Learn how to communicate in their language
• They can become a strong supporter
!
• Keep it as simple as possible…and no simpler

136

© 1993-2014 Jeff Sutherland

Value Sources to Consider

Will this feature allow us to:

• Sell more units?
Market Value • Charge a higher price?
• Reduce the cost of providing the product/service?

How will completing this story allow us to:

Risk • Develop or refine hypotheses about the market?
Reduction • Prove technical assumptions?

Will completing this story:

Capability • Enable our team to do something we couldn’t before?
Building • Reduce or eliminate the need for low-value activity?

137

© 1993-2014 Jeff Sutherland

Methods for Determining
Value • Start at the top of a list of stories
• Compare value of stories one at a time
Faster

• Move the lower value story down one place in list

Bubble Sort • Repeat until all stories have been compared

! •
•
Pick a low value item and assign it 3 points
Use estimation cards to independently estimate a story
• Show estimates, discuss highs and lows, estimate again
Planning Poker • When everyone is within three cards, average the estimates

! • Compare cost of feature creation with expected

incremental revenue of feature
Break-even analysis • How many incremental units would we need to sell to
equal the development cost?
! • = [Total expected revenue from new feature]/total
cost to develop feature] – 1
Return On Investment • Expressed as a percent

!
More Detailed

• Over a reasonable planning horizon, what are the

revenues and expenses associated for a feature in each
Cash Flow Analysis month?
• What is the net effect on cash flow over that horizon?
!
• Building on the cash flow analysis, what is the effect of
Net Present Value including the “time value of money” in the calculation? (i.e.
a dollar today is worth more than a dollar tomorrow)

138

© 1993-2014 Jeff Sutherland

Cash Flow Analysis

Smart prioritization of the backlog can dramatically reduce

maximum cash injection and reduce payback time
139

© 1993-2014 Jeff Sutherland

 Net Present Value
140

NPV = ∑ Rt ÷ (1+i)t
t-1

Rt: Expected Revenue at Time t.

t: Number of Periods to be Calculated
i: Discount Rate/Opportunity Cost of Capital per Period
R
TM

www.RapidScrum.com
2011 Copyright Rapid Scrum LLC. All rights reserved.
Problems with NPV
• One of our largest clients abandoned any return
on investment analyses as it led to gaming the
system.
• It is like a black box to decision makers. It is
hard to tell what is really going on.
• Some major companies are moving to simple
rules or guidelines to make decisions at a
strategic level.
• This is a Scrum based approach as simple rules
are used to generate self-organization at every
level from the team to the company in Scrum.

141

© 1993-2014 Jeff Sutherland

 As a PO, I need to understand
Story Sizing and Estimation so that
I can assist the Team in estimating
stories

142

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
Scrum Estimating Strategy
• Don’t estimate time
• Estimate relative size of stories
• Measure velocity per sprint
• Derive release plan
!
• Estimates are done by the people who are going to
do the work
• Not by the people who want the work done
!
• Estimate continuously during the project, not all up
front
!
• Prefer verbal communication over detailed, written
specifications

143

© 1993-2014 Jeff Sutherland

Points vs. Hours

• Rand Corporation received a grant from U.S.

DOD in the 1940’s to determine best way to
estimate tough projects
• Discovered estimation in hours has high error rate and
wide variance
• Found people could put things in relative size piles best
!
• Fibonacci growth pattern easiest for humans
• Seen everywhere in nature
• RAND called it the Delphi technique

144

© 1993-2014 Jeff Sutherland

Fibonacci Series is All Around Us

145

© 1993-2014 Jeff Sutherland

Why Points are Better Than
Hours
Cone of Uncertainty

1. Repeated studies
have shown Gray Line - Hours
estimates in Red Line - Points

points are more

accurate

2. There are a fixed number of hours in a day, so a

velocity based on hours can not increase!
Laurie Williams, Gabe Brown, Adam Meltzer, Nachiappan Nagappan (2012) Scrum + Engineering Practices: Experiences
of Three Microsoft Teams. IEEE Best Industry Paper Award, 2011 International Symposium on Empirical Software
Engineering and Measurement.
146

© 1993-2014 Jeff Sutherland

 Beware of Common Estimating
Pitfalls Same spec plus
Spec extraneous detail

Irrelevant
Information
20 pts! 39 pts!

Spec Same Spec Same Spec

25 pts., never 10 pts., never

Anchoring mind me mind me
20 pts!
30 pts! 15 pts!

147

© 1993-2014 Jeff Sutherland

Estimates Are Estimates
• The actual work will be different than estimated
work.
• However, averages will get you close,
particularly if you use points.
• The team needs to commit to the sprint goal but
velocity will vary, typically about 20% sprint to
sprint.
• For this reason, the work “commitment" was
removed from the Scrum Guide and the word
“forecast” was substituted.

148

© 1993-2014 Jeff Sutherland

 Estimating the Product Backlog

T T T SM PO

Development Team Scrum Master Product Owner

• Estimate backlog • Facilitates • Available to clarify intent of
• Estimates are forecasts, process PBIs to help estimate
not commitments • But does NOT estimate

Two options for estimating process:

Estimate • Lay out all stories
Stories • Agree which one is least effort; call this “3-points”
Individually • Estimate other stories relative to the reference story

Estimate • First group stories into similarly sized piles of related activity
Groups of • Then estimate number of points for each pile
Stories • Fast way to estimate a large number of stories
149

© 1993-2014 Jeff Sutherland

 As a X
I want Y 8

Exercise: Planning Poker so that Z

As a X
I want Y 2
• Estimate stories so that Z

• Pick smallest relevant story and give As a X

I want Y 3
it 3 story points so that Z

• Estimate relative size of other stories As a X

I want Y 5
• Discuss outliers and vote again until so that Z

all numbers are within 3 cards, then As a X

I want Y 1
average. so that Z

! As a X
I want Y 2
• Extra time? so that Z
As a X
• Break down big stories into smaller I want Y 5
so that Z
stories
As a X 13
I want Y
so that Z
150

© 1993-2014 Jeff Sutherland

Estimate User Stories with
“Planning Poker”

151

© 1993-2014 Jeff Sutherland

 As a PO, I need to participate in
Sprint Planning to get the team
off to the right start of a sprint

152

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
The Product Owner Wants a READY Backlog Before
Sprint Planning in Order to Increase the Team’s
Velocity
Daily
Meeting

Sprint
I
R M D
P
E E O
A D N
D I
 E
M

Value Y E Velocity
N

T

S

153

© 1993-2014 Jeff Sutherland

Sprint Planning

• Review velocity and set Sprint capacity

• Adjust for vacations, holidays, etc.
• Forecast how many User Stories will be completed in the
Sprint using Yesterday’s Weather
• The team commits to do the best they can to hit the forecast. There
will be normal variation (about 20%).
• Select top-priority User Stories from Product Backlog
• Select Sprint Goal
• Done collectively among team and PO, 1-2 sentence description
of what the team plans to accomplish.
• Discuss and finalize Acceptance Criteria
• Good Product Backlog Refining = at least 1 hour/week
of each Sprint

154

© 1993-2014 Jeff Sutherland

Roles in Sprint Planning
Product Owner
• Present the backlog
• Answers questions to clarify the backlog
• Identify highest priority PO

!
Scrum Master
• Facilitate the meeting
• Confirm team capacity
SM
• Ensure stories are ready and have a definition of done
!
Team
• Ask questions
• Decide how much backlog to pull into the sprint
• Identify stories each member would like to work on T T T
• Agree on a sprint goal

155

© 1993-2014 Jeff Sutherland

Calculating Velocity of the Team
• Velocity is calculated by counting the number
of points in Stories that are completely Done
at the end of the Sprint.
• Normal variation in velocity from one Sprint to
another is +/- 20%. Therefore to get a more
accurate rate of velocity average the last three
Sprints. This figure can be used to determine
how many points the Team can be expected to
complete.
• Need this information to calculate how many
points to responsibly bring into a Sprint and to
do Release Planning.

156

© 1993-2014 Jeff Sutherland

 As a PO I need an introduction to Key
Scrum Patterns which can dramatically
improve the performance of my Scrum

157

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
State of Agile
Chaos Manifesto 2011, Standish Group International, Inc.
The agile process is the universal remedy for software development
project failure. Software applications developed through the agile process
have three times the success rate of traditional waterfall method and a
much lower percentage of time and cost overruns. The secret is the trial
and error and delivery of the iterative process.

2012 Scrum Inc. © 2014 Scrum Inc.

Source:
Patterns are Important

159

© 1993-2014 Jeff Sutherland

 Key Scrum Patterns

Patterns are not required, but enhance Scrum

Teams that Finish Early Accelerate Faster - How do you become

hyper-productive?

Stable Teams - How do you get started?

Yesterday’s Weather - How do you pull backlog into a sprint?

Swarming - How do you get work done quickly?

Interrupt Pattern - How to deal with interruptions during the sprint?

Daily Clean Code - How to get defect-free product at sprint end?

Scrumming the Scrum - How to ensure you improve continuously?

Happiness metric - How do you ensure teams aren’t overburdened?

www.scrumplop.org
160

© 1993-2014 Jeff Sutherland

Pattern: Stable Teams

Teams as an island of stability in a sea of change
Moving people from crisis to crisis helps us paint over the cracks in capability. So we
never really understand what the organization is capable of delivering. Moving people from
team to team when starting projects, or crisis to crisis during delivery leads to an unstable
environment with added costs of;
• administration of keeping track of what people are working on
• productivity as teams work through the Tuckman cycle every time they acquire a
new member
• exposure to Brook’s Law
The thinking that the requirements of what to build can be fixed and having everything else
changed around the requirements is like taking a geocentric view of the solar system

Creating the stable team is moving to a heliocentric view, we accept that the requirements
change, but we need to fix (positional) something to measure this variation from.

Therefore…

Keep teams stable and avoid shuffling people around between teams. Stable teams
tend to get to know their capacity, which makes it possible for the business have
some predictability. Team members should be dedicated to a single team whenever
possible.

161

© 1993-2014 Jeff Sutherland

Pattern: Yesterday’s Weather

How much work to pull into the Sprint
.... a team is progressing with its historical staff and a stable sprint length, and the time has come
to assess the level of the team's commitment for the upcoming sprint.
!
It's human nature that individuals and teams with self-esteem set increasingly higher
goals for themselves. By trying to achieve these goals, teams learn. Sometimes such strivings
leads to immediate improvement, particularly when the team challenges itself to improve through
use of a new-found technique or technology. Sometimes, however, the new technique doesn't
pan out and the team's performance remains the same or even gets worse.
!
Sometimes, the team raises the bar just to raise the bar, either to test the limits or to
unconsciously publish its bravado. Even these self-challenges can pay off, especially in the short
term. It is more common that these higher levels are unsustainable in the long term.
!
Therefore: In most cases, the number of Estimation Points completed in the last Sprint is
the most reliable predictor of how many Estimation Points will be completed in the next
Sprint.

162

© 1993-2014 Jeff Sutherland

Pattern: Swarming

• How to get work done quickly

163

© 1993-2014 Jeff Sutherland

Exercise: Implementing Product
Backlog

Policy D AV E Dave

LI S Lisa
Never keep a
customer waiting B O B Bob
!
Start early
= Finish early
E R I Eric

M A R Maria

164

© 1993-2014 Jeff Sutherland

 Round 2 – New Development
Process

Policy D AV E Dave

Limit WIP L I SA Lisa

(work in process)
!
Current limit = Bob
1 customer
at a time
Eric

Maria

165

© 1993-2014 Jeff Sutherland

Weinberg Table of Project
Switching Waste

Weinberg, Gerald M. (1992) Quality Software Management: Systems Thinking. Dorset House, p. 284.

166

© 1993-2014 Jeff Sutherland

 Pattern: First Things First

Prioritizing Between Projects

Product A =
A1 A2 A3 A

Product B =
B1 B2 B3 B

=
Product C C1 C2 C3 C

Traditional strategy: ”Everything is important! Do it all at once!”

A B C
A1 B1 C1 A2 B2 C2 A3 B3 C3
January February March April May June July
Agile strategy: ”Prioritize & focus!”

A B C
A1 A2 A3 B1 B2 B3 C1 C2 C3

January February March April May June July

Adapted from Henrik Kniberg
167

© 1993-2014 Jeff Sutherland

Pattern: The Interrupt Pattern

Dealing with the unexpected
Beginning of sprint
Product Sprint
Backlog Backlog Support
Kaizen
8 8 Management
5 5
5 5
Sales
PO
3 Now
3
5
5 5 Buffer
Later
8

5
3
5 Low Priority

On Buffer Overflow ABORT, Replan, Dates Slip

168

© 1993-2014 Jeff Sutherland

Pattern: Scrum Emergency Procedure

Know how to respond when all hell breaks lose

When team is more than 20% behind, execute the Scrum

Emergency Procedure by mid-Sprint:
• Innovate - do something different
• Identify impediments, root cause analysis, remove
impediments, otherwise ...
• Offload Sprint Backlog - get someone else to do it,
otherwise ...
• Reduce scope in collaboration with Product Owner, or if
this is not possible then ...
• Abort the Sprint
• Recommended for new teams that need to learn how to do
better estimates
buyinggoldcoins.net

169

© 1993-2014 Jeff Sutherland

 As a PO, I should understand how to
manage Release Planning in order to
deliver products efficiently, transparently
and on time

2013 Scrum Inc.

© 2011 Scrum Inc.
Planning in Scrum Process

• Not just in the beginning when know the least

about product
• Based on Inspect and Adapt
• Takes place during whole process
• Takes place on every level
• Product/Vision
• Release Plan
• Sprint Plan
• Daily Stand-Up

171

© 1993-2014 Jeff Sutherland

Release Planning Meeting

• not an “official” Scrum Meeting but it is extremely

helpful and part of the PO’s responsibilities in
planning
• Product Owner invites: Team members,
Stakeholders, and others as needed
• Meeting held as PO deems necessary - often once a
quarter

172
© 1993-2014 Jeff Sutherland
 Elements of a Scrum Release Plan

•1 Clear Vision
• Tied to concrete business value
• Aligns stakeholders
!
•2 Vision decomposed into Feature Feature Feature Feature

independent features Story Story Story Story

• Prioritized and estimated Story Story Story Story

Story Story Story Stpry

• Explicit ROI considered
! 400

•3 Burndown chart of progress on Release 

Backlog 
prioritized backlog items (points)

• Measured in Points! Sprint

!
•4 Feature Roadmap timeline
• Best guess – subject to change

2013 Scrum Inc.

173
Release Cycles

• Goal: every sprint results in potentially releasable product increment.

• Product owner decides when to release.

Release 1 Release 2 Release 3 Release 4 Release 5 Release 6

(Alpha) (Beta) (Live) (maintenance) (maintenance) (maintenance)

Sprint # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Source: Henrik Kniberg

174

© 1993-2014 Jeff Sutherland

Product Owner Responsibility

• Get one sprint READY backlog

• Team can get started
• Get two sprints READY backlog
• Team can accelerate sprint to sprint

• Build out Release Plan

• Company can predict revenue

• Build one year roadmap

• Customers can be briefed on company strategy

175

© 1993-2014 Jeff Sutherland

Traditional Release Planning

• Scope, Time and Resources are

locked in a fixed relationship
• Scope – the work that needs to
be done
• Time – how long you have
• Resources – the number of people

Tim !
pe

• In theory, any one of these

Sco

e
dimensions can be changed to
meet release requirements…
!
• …However, in practice
resources are seen as easiest
to change
• Scope viewed as a fixed
Resources constraint
• Time generally seen as fixed

176

© 1993-2014 Jeff Sutherland

 But Adding People to Late Projects
Only Makes them More Late!
Caused by deteriorating team
This is called “Brook’s Law”
communication saturation

30 6 direct  
4 people communica-on 
pathways
23

15
9 direct 
5 people communica-on 
8 pathways

0
2 4 6 10 17 15 direct 
Time 6 people communica-on 
Cost pathways
Source: hJp://www.qsm.com/process_01.html (491 projects)
177

© 1993-2014 Jeff Sutherland

Scrum Allows you to Break the Iron Triangle

Complete More Scope in Less Time with Fewer People

• Small & stable teams are key

!
• Flexing scope actually much
easier than changing resources
• Requires scope defined as

Tim
pe

independent features, and

Sco

prioritized by value
e !
• Increasing velocity allows
Velocity team to get more done in the
same time
• Accomplished by removing
impediments
Resources

As a bonus, product quality and team

2013 Scrum Inc.

satisfaction also improve
 Elements of a Scrum Release Plan

•1 Clear Vision
• Tied to concrete business value
• Aligns stakeholders
!
•2 Vision decomposed into Feature Feature Feature Feature

independent features Story Story Story Story

• Prioritized and estimated Story Story Story Story

Story Story Story Stpry

• Explicit ROI considered
! 400

•3 Burndown chart of progress on Release 

Backlog 
prioritized backlog items (points)

• Measured in Points! Sprint

!
•4 Feature Roadmap timeline
• Best guess – subject to change

2013 Scrum Inc.

 Release Burndown Chart Makes Team’s
Velocity and its Implications Visible

Points Points
V

Apr
2008
May
2008
June
2008

Q3
2008

Q4
2008

2009

Sprint/Time Sprint/Time

180

© 1993-2014 Jeff Sutherland

 Two More Considerations for 

Burndown

Emerging Bugs and Customer

Requirements Feedback
Additional user stories beyond
those known in the backlog that Additional work that cannot be
are “discovered” as the project anticipated in the release plan, but
evolves and require the team to do you know it will come up as
more work
!
  product functionality is released
!
 
Generally happens as a consistent Track as a separate buffer as a
percentage of estimated work, percent of estimated work, and try
which can either be added to the to manage down the percent of
backlog as a “buffer” or subtracted velocity devoted to bugs as a way
from velocity in calculating to speed up the team
burndown

Both factors must be accounted for to

2013 Scrum Inc.

determine accurate burndown
 Feature Roadmap Helps Stakeholders 

Know when to Expect New Functionality
• Facilitates conversations on feature priority
• Aligns stakeholders and heads off distraction
!
• Ground rule: Timeline is only an estimate, and
subject to change
Q1 • Basic platform with ability to create new user 
• Homepage and introduction 
• Ability to view account status

Q2 • Ability to update account information/address 

• Select communication options and preferences 
• “Share with friends” link

Q3 • Ability to rate individual articles 

• Ability to sort by top rated articles 
• Ability to refer friends for a referral bonus

Q4 • New premium content offering 

2013 Scrum Inc.

• Corporate portal for company viewing
Three Common Approaches to

Release Planning
• Deadline-based
• External deadline specified for team, they must
complete as much of a given backlog as possible
before that date
!
• Regular-Departure
• Set cadence of product releases. (e.g. quarterly)
• Ready features are included in the release, non-
ready ones wait for next release
!
• Value-Based
• Team produces incremental potentially-shippable
product each Sprint
• When PO decides enough new value has been
created, features are released to customers

2013 Scrum Inc.

Deadline–Based Release Planning

Medco case study
On July 7 2006, Medco CEO promised Wall Street analysts a completely new
pharmacy fulfillment system to be implemented by July 7, 2007 
• Unfortunately, he didn’t check with the development team first!?!!

Points Medco Stock
1,400 1,450
1,320
1,230

990

700 Plan
Code
Test
3 2 1
90
60

Jul‐06 Dec‐06 Jul‐07 Dec‐07 Time

2013 Scrum Inc.

Promised 
Delivery Date
Regular Departure Release Planning

Microsoft Case Study
• Prior to 2005, Microsoft released a new version of
its Team Foundation Server (TFS) product roughly
every 18 months
• Using Scrum, it now deploys a new version
internally every 3 weeks

Release Release
2005
18mo. 18mo.

Release Release

2008
5wk. 5wk.

Release Internally Every Sprint

2012
3wk. 3wk.

2013 Scrum Inc.

Source: Sam Guckenheimer and Neno Loje. Agile Software Engineering with Visual Studio. Microsoft Press, 2012.
Value-Based Release Planning

Healthcare Startup Case Study
• Venture-backed healthcare startup looking to raise additional
investment
• Needs to demonstrate value creation rapidly to court investors

1 2 3
Divide work into Decompose epics into As each Epic is
Epics, prioritized by actionable user stories completed, release
expected revenue and start working into market

ED Module

Ambulatory Module

Registra-on Module

ePrescribing

Pa-ent Portal

2013 Scrum Inc.

 As a PO, I need to understand
Value Stream Mapping and
Constraint Theory
to keep work fast and smooth

© 2011 Scrum Inc.

Theory of Constraints – Smooth Flow
Goal
P

S T

Problem

Strategy

3.
Inc
rea
se

4.
1. int 2.
ak Fix

Fi
Re e

x
du bo

ne
ce ttle

xt
int n ec
ak k
e
© 1993-2012 Jeff Sutherland
 Game backlog Design-ready games Production-ready games

8 15 12

Concept Lisa
Graphics Sound Integr. &
Sam assigns Dev
pres . resources
design design deploy T
2d 1m 6m 1w 6m 6m
2h 4h 1d 1m 3w 3m 3w
(1m+2m)
Games out of date
⇒ Missed market windows Process
3 m value added time
⇒ Demotivated teams = 12% cycle
⇒ Overhead costs efficiency
25 m cycle time

Estimate

w1 w2 w3 w4 w5 w6 w7 w8 2 m cycle time = 12x faster

Preliminary result

3-4 m cycle time = 6-8x faster

w1 w2 w3 w4 w5 w6 w7 w8

Source: Henrik Kniberg

© 1993-2012 Jeff Sutherland

 Case Study 

Take-away Points w1 w2 w3 w4 w5 w6 w7 w8

Speeding up product development is often a matter of

improving the process rather than adding people.
Value stream mapping is a great tool for spotting bottlenecks
Scrum is a great tool for removing bottlenecks
But beware the trap – suboptimization!!
The pictures make it look easy.... Hey, let’s do Scrum here! Maybe
we can cut dev time in half!
But executing the change is usually hard From 3 months to 1.5 months...

Ty
Concept Lisa Graphics Sound Integr. &
Sam assigns Dev Typ
pres. design design deploy
2d 1m 6m 1w 6m 6m
2h 4h 1d 1m 3w 3m 3w
(1m+2m)

Source: Henrik Kniberg

© 1993-2012 Jeff Sutherland

 As a PO, I need a simple system for
managing Technical Debt so that it does
not build up and ultimately slow the team
down

191

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
 Clean & Simple Code Dog.java v1.1
Dog.java v0 Big & hairy
Dog.java v1.0 import java.sql.Connection;

Quick & dirty !

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class Dog {

public class Dog {

public static void main(String[] args) {
!
private Executor executor = Executors.newFixedThreadPool(18);
private int CACHE_SIZE = 50;

public Dog()
{
System.out.println("WOOF 1!"); try
System.out.println("WOOF 2!"); {
Class.forName("oracle.jdbc.ThinDriver");
} connection = DriverManager.getConnection("jdbc:oracle:thin:@prod", "admin", "beefhead");

} ! statement = connection.prepareStatement("insert into Dog values (?, ?, ?)");

} catch (ClassNotFoundException e) {}

!
new Thread().start();
}

public void woof(Person woofCaller) {

Connection connection = null;
PreparedStatement statement = null;
try {
connection = DriverManager.getConnection("jdbc:oracle:thin:@prod", "admin", "beefhead");
statement = connection.prepareStatement("insert into Dog values (?, ?, ?)");
statement.setLong(1, System.currentTimeMillis());
statement.setString(2, person.getName());
statement.setString(3, person.getPhoneNumber().getNumber());
statement.executeUpdate();
Dog.java v1.2 }
}

Clean & simple }

} Connection a = DriverManager.getConnection("jdbc:oracle:thin:@prod", "admin", "beefhead");
b = a.prepareStatement("select * from Dog where name = '" + name + "'");
c = b.executeQuery();
public class Dog { if (c.next()) {

private final String name; Simple code: String foundName = c.getString("name");

PhoneNumber phoneNumber = new PhoneNumber(c.getString(“woofCount"));
Person person = new Person(foundName, phoneNumber);

! private int woofCount = 0;

1.Passes all tests return person;
} else {

public Dog(String name) {

this.name = name;
2.No duplication !
}
return new Person("", null);

} catch (SQLException e) {
return null;
3.Readable
! }

public void woof() { 4.Minimal

} catch (IllegalArgumentException x) {

!
}
throw x;
}

}
++woofCount;
! public List<Person> getAll() {
connection = DriverManager.getConnection("jdbc:oracle:thin:@prod", "admin", "beefhead");
statement = connection.prepareStatement("insert into Dog values (?, ?, ?)");
statement.setLong(1, System.currentTimeMillis());
} Simple is hard! }
if (statement != null) {
if (c.next()) {
String foundName = c.getString("name");
PhoneNumber phoneNumber = new PhoneNumber(c.getString(“woofCount"));
Person person = new Person(foundName, phoneNumber);
return person;
} else {

Source: Henrik Kniberg 192

© 1993-2014 Jeff Sutherland

 Technical Debt & Release
Planning
Remaining
story points

400
We’ll be
done by
300 sprint 10!

Sorry, we’re late!


We should definitely by
200 Um... we’re done
done by sprint 12!
when we’re done!

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Sprint

Source: Henrik Kniberg 193

© 1993-2014 Jeff Sutherland

 Dealing with Technical Debt

Definition of Done Definition of Done

• .... bla bla .... • .... bla bla ....
Vmax • No increased technical debt • Technical debt decreased

Vactual
Ro
ad
to
Velocity

he
ll

g p ace!
n
nc reasi
Sustainable pace I

Second step
First step (optional)
Slow down Slow down even more
Stop accumulating debt Start repaying debt

Time

Source: Henrik Kniberg 194

© 1993-2014 Jeff Sutherland

As a PO, I lead The Sprint Review to
ensure that we are delivering
incremental value in each Sprint

195

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
Sprint Review

Time Boxed to one hour per week of Sprint length
!
!
• This is when the state of the Product becomes transparent!
• The Team shows the Product Owner and other interested
stakeholders what work was accomplished.
• The Product Owner reviews and accepts the work if it meets
the Definition of Done.
• The main purpose is an in depth conversation between the
Team and the Product Owner to see where the product is in the
process, what they have learned, and what adaptations need to
be made.
• As little time as possible should be spent preparing for the
Sprint Review.

196

© 1993-2014 Jeff Sutherland

Product Owner’s Job at Sprint Review

• Get the right people in the room

• Be clear about what is Done and accepted and
what is not
• Focus on positive feedback - Atta Boy!
• Lead an energetic discussion
• Get, filter, and prioritize feedback
• Adjust Product Backlog as needed

197

© 1993-2014 Jeff Sutherland

 As a PO I need to participate in the
Retrospective to capture learnings that allow
us to become more efficient

198

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
The Retrospective – What is it?
!
• Team meeting that occurs at the end of the Sprint,
following Sprint Review
!
• Questions for the team:
• What worked well last Sprint?
• What could work better next Sprint?
• What process improvement would I like to try?
• As a team, everyone agrees on what change to try

!
• Inspect and Adapt
!
• Continuous improvement

199

© 1993-2014 Jeff Sutherland

Sprint Retrospective

Long term effect

Velocity

1 2 3 4 5 6 7 8 9 10 11 12 13
Sprint

Effective velocity over time Effective velocity over time

(with retrospectives) (without retrospectives)

Source: Henrik Kniberg

200

© 1993-2014 Jeff Sutherland

Scrumming the Scrum
Implement the Pattern
• Team identifies the single most Using the Happiness Metric
important impediment at the Sprint
Retrospective.
! During
retrospective,
• Product Owner adds it as the top team members
priority in the Product Backlog, the lists likes and
dislikes of
kaizen. previous
! sprint.
Team
• Team pulls the kaizen into Sprint Add to top of
discusses
Backlog as a user story with sprint
dislikes and
backlog as
acceptance tests and DoD. the team
how they

! kaizen.
interfere with
performance.
• Constant focus on making the
process better is the way to improve
velocity and ultimately achieve
hyperproductivity. Create a
! story for
resolving
Top
improvement
• Otherwise the team may flatline or with a is agreed on
even have consistent velocity definition of by team.
decline. DONE.

201

© 1993-2014 Jeff Sutherland

 Using the Happiness Metric
The Happiness Metric is included as part of the Sprint
Retrospective meeting…
For each person:
1. On a scale of 1-5, how happy are you with your role in the
company?
2. On the same scale, how happy are you with the company?
3. What specific events or activities increased your happiness?
4. What specific events or activities decreased your happiness?
5. What would increase your happiness moving forward?

For the team:

• What would make the team as a whole
happier in the next sprint?
• Identify the top priority for the team
• Execute the pattern “Scrumming the
Scrum”
202

© 1993-2014 Jeff Sutherland

 Employee Happiness is Important

• People are naturally motivated by intrinsic factors

as well as traditional extrinsic ones
!
   

Extrinsic
Intrinsic
Purpose Money
!
Mastery   Power  
! Autonomy Status
!
• This is not just “warm and fuzzy”…happier people
do better work, and are more effective
• Doctors in a positive mode show three times the intelligence
and creativity and diagnose 19% faster
• Optimistic salespeople outsell pessimistic ones by 56%
• Retail stores with higher employee life satisfaction generate
$21 more in earnings/SF than the other stores (Gallup)

2013 Scrum Inc.

 Happiness Impacts the Bottom Line
Top 5 public companies1 from the Forbes 2011 list of “Top 100 companies to work for,”
significantly outperform the bottom 5 companies2.

Top companies slightly … are significantly … and growing more

larger than bottom ones… more profitable…

1. Top ranked public companies were: SAS (1), Google (4), NetApp (5), Camden Properties Trust (7), REI (9)  
2. Bottom ranked public companies were: W.W. Granger (100), Starbucks (98), Darden Restaurants (97), Morningstar (95),

2013 Scrum Inc.

Aéropostal (94) 
 
Note: Public companies used given availability of financial return information; results reflect a weighted average by revenues
Source: Forbes.com; Dunn and Bradstreet
 Team Happiness is a Leading
Indicator of Performance
First signs
of trouble
Happiness Velocity • A happy Scrum team
typically improves velocity
by 5-10% each sprint
5.0000 30.0000 !
• Sudden and systematic
3.7500 22.5000 declines in team happiness
often precede rapid decline
in velocity by a sprint or
2.5000 15.0000
two
• Often indicates a major
1.2500 7.5000 process challenge for the
team, or an unresolved
impediment
0.0000
1 2 3 4 5 6 7 8 9 101112131415
0.0000 !
• Acting quickly on early
Happiness Velocity signs of unhappiness can
avert problems

205

© 1993-2014 Jeff Sutherland

 Happiness Metric as an Analytic Tool

Happiness Happiness by Happiness vs.

trends team member velocity
Happiness Happiness

5.0 4.0000
3.8 3.0000
2.5 2.0000
1.3 1.0000
0.0 0.0000
1 2 3 4 5 6     0.0 6.5 13.0

Sprint Velocity
Role Company

• Links team happiness to • De-average team results • Track relationship

specific events or work to reveal pockets of between work pace and
features unhappiness happiness
• Good leading indicator of • Identify uneven • Highlights need to slow
future performance workload distribution or down to reach
(when happiness starts role definition issues sustainable pace (or
trending down, velocity speed up to increase
follows) team satisfaction)
206

© 1993-2014 Jeff Sutherland

 As a Group we need a
Course Review and Retrospective
to determine what is DONE

© 2011 Scrum Inc.

Next Steps

• Names and email addresses will

be uploaded to Scrum Alliance
• You will get email with username and
password
• Login and update your profile
• You will get a web page that
identifies you as a Certified Scrum
Product Owner

208

© 1993-2014 Jeff Sutherland

Motivation

209

© 1993-2014 Jeff Sutherland

Stay Connected
• Our Website
• check in for announcements, new content and services,
book releases, and more!
• www.scruminc.com
!
• ScrumLab
• join the conversations on our forums with the scrum
community and your class.
• coming soon: articles, videos, papers on all things scrum
• scrumlab.scruminc.com
!
• Blog
• scrum.jeffsutherland.com
!
• Webinars
• advance your learning with our interactive webinars. visit
the scrumlab store to view upcoming topics.
!
• Twitter, Facebook, and G+
• @jeffsutherland, scrum and scrum inc.
210

© 1993-2014 Jeff Sutherland

 As a PO, I should understand how to
structure Agile Contracts to leverage
Scrum’s natural competitive advantage

211

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
Not All Features Are Created
Equal!
80% of 
value 
typically  The fact that a large share of
resides in  requested features end up being
50.0000 20% of 
unimportant opens up many “win
features
win” contracting opportunities
Value to Customer

37.5000

65% of features provide liJle to no value, 
are rarely used and/or aren’t actually 
25.0000 desired by the customer
The rest are OK, 
but not as 
12.5000 important

0.0000

Features

212

© 1993-2014 Jeff Sutherland

 Agile Contracts: Money for
Nothing
1. Projects are usually prioritized by return on
investment.
2. Ordering your Product Backlog allows you to prioritize
features by return on investment.
3. Since 65% are never or rarely used, during the project
it will become evident that the next low priority feature
costs more than the value it delivers.
4. Stop the project at that point
50.0000
and deploy the valuable

Customer
Value to
37.5000
features. Contractor and
25.0000
customer split remaining
12.5000
budget.
0.0000
5. All projects should deliver Features
early and save money. End project Split this
here remaining
budget
213

© 1993-2014 Jeff Sutherland

 Agile Contracts: Change for Free

1. Create a prioritized backlog of work to be done with

highest business value items first.
2. Implement in short sprints, always less than a month.
3. When higher priority requirements emerge, put them
in the next sprint.
For any
4. Cut an equivalent amount of changes
the lowest priority items out added here

of the project to the work 50.0000

Customer
added. These features are Value to 37.5000 Remove an
equivalent amount of
unlikely to be used anyway. 25.0000 work here
12.5000
5. Change for free allows you
0.0000
to meet your budget and
deliver on time. Features
214

© 1993-2014 Jeff Sutherland

Contracts

• How would you estimate a fixed price contract?

!
• How would you deal with changes?
!
• How would you incent everyone to deliver early?
!
• How would you bill the contract?

215

© 1993-2014 Jeff Sutherland

 As a PO I need to know how to 

Scale Scrum so we can use it to
accomplish larger and more complex
goals

2013 Scrum Inc.

216 Inc.
© 2011 Scrum
 Scrum Is Implemented Across Very
Large Organizations

• 3,000 developers working in • As of Nov 2012, close to

Scrum Teams 1,000 Scrum teams
• As of 2011, all new software • Line Management role
development done using transitioned to facilitating
Scrum removal of impediments
• Wrote book on • Parallel Product Owner
experience using management structure now
Scrum at Scale in place as well

2013 Scrum Inc.

217
Scrum Scales “Fractally” Rather 

than “Hierarchically”

2013 Scrum Inc.

218
As a Result Linear Scalability Can be Maintained with
Size

Scrum Teams
Velocity

Waterfall

Project Size
• J. Sutherland, A. Viktorov, J. Blount, and N. Puntikov, "Distributed Scrum: Agile Project
Management with Outsourced Development Teams," in HICSS'40, Hawaii International
Conference on Software Systems, Big Island, Hawaii, 2007.
• J. Sutherland, C. Jacobson, and K. Johnson, "Scrum and CMMI Level 5: A Magic Potion for Code

2013 Scrum Inc.

Warriors!," in Agile 2007, Washington, D.C., 2007.

219
 Scrum Master and Product Owner
Functions Scale Differently

Scrum Master Product Owner

• Share best practices • Maintain clear and consist
• Collectively solve problems & product vision
remove impediments • Optimize business value
• Respond decisively to changing
market
Product PO team

Scrum of Scrum of Scrums

Component PO team CPO Component PO team

Scrum of Scrums
CPO CPO
Team

PO PO PO PO

2013 Scrum Inc.

T T T T T T T T T T T T T T T T T T

220
Acceptance Tests

© 2011 Scrum Inc.

Modern Agile Acceptance Model

• The move toward Acceptance Test Driven

Development requires a more complete model of
progressing requirements acceptance
• Example model (sharper definition of done)
• Conditions of Satisfaction - broad terms
• Acceptance Criteria - further refined
• Examples - actual scenarios or data
• Executable Examples - Executable Spec

222

© 1993-2014 Jeff Sutherland

Simple Scenarios

• Suppose we are creating a registration function

• It consists of specifying email (as User ID) and
Password

Enter your Email and Password to Register

Email Address:

Password:

Register

223

© 1993-2014 Jeff Sutherland

Acceptance
Condition of Satisfaction
• Conditions of Satisfaction are high level criteria
established with an initial Epic/Feature/User
Story
• In our previous example, the conditions of acceptance
for the password would be:
• Ensure the password is not easy to crack.

• The PO would define the conditions of

acceptance in concert with business stakeholders

Enter your Email and Password to Register

Email Address:
Password:
Register
224

© 1993-2014 Jeff Sutherland

Acceptance
Acceptance Criteria
• Acceptance Criteria specifies the details
that the team can then build against:
• Following the example for Password:
• Must be at least 8 characters and no more than 12
• Must contain only alphanumberics and the period
• Must contain at least one digit
• Must contain at least one character
• Etc. (there may be more criteria)

• The PO works closely with a tester,

developer, and business stakeholder to
define these criteria Enter your Email and Password to Register
Email Address:
Password:
225
Register
© 1993-2014 Jeff Sutherland
Acceptance
Examples Make It Real
• Actual examples are the ultimate way to
communicate requirements:
!
Password Expected Expected Message Comment

!
abc123 Invalid Your password must be at least 8 characters
and no more than 12 charcters long
!
abcdefghi Invalid Your password must contain at least one
character and one number
!
1aaaaaaaa Valid Why valid?

!
ajx972dab Valid
!
• The PO works closely with a tester, developer,
and business stakeholder to define these
criteria

226

© 1993-2014 Jeff Sutherland

Acceptance Test Driven Development (ATDD)
Tools: Fit and Cucumber Cucumber
FIT (Framework for Integrated • New tool for natural language scenarios
Test) and Fitnesse (Wiki front
end)
• Test specified in table format In order to ensure my account is correct
As a Registered User
• Developers generates classes I want to check my recent activity
(“fixtures”) to hook into !
application Scenario: Recent Account Activity
Given I am a registered user “Jsmith”
• Users/testers use Wiki or Excel to And I am logged in with password “xyx123”
specify inputs and outputs And I have had account activity in the last 45 days
And I am on the home page
When I click the “Recent Activity” button
numerator denominator quotient Then I should see the “Account Activity” Page
And I should see a list of my activity over the last
45 days
10 2 5 !
Scenario: No Recent Account Activity
Given I am a registered user “Jsmith”
And I am logged in with password “xyx123”
12.6 3 4.2 And I have had account activity in the last 45 days
And I am on the home page
When I click the “Recent Activity” button
Then I should see the “Select Account Activity
100 4 25 expected
Period” Page
24 returned And I should get a message: “You had no activity
in the last 45 days, please select a time frame to 227
review”
© 1993-2014 Jeff Sutherland
 As a PO or SM, I need know ways to
work with Distributed Teams so that
we can get to output effectively

228

© 2011 Scrum Inc.

© 1993-2014 Jeff Sutherland
Guidance on Distributed Scrum

From Craig Larman and Bas Vodde

• On to our key recommendation: After working for some years

in the domains of large , multisite , and offshore development,
we have distilled our experience and advice down to the
following: Don’t do it.
• There are better ways to build large systems than
with many developers in many places. Rather, build
a small group of great developers and other talents
that can work together in teams, pay them well, and
keep them together in one place with product
management or whoever acts as the voice of the
customer.
• But of course you are still going to do large,
multisite, or offshore development. This is because
your existing system is already structured that way,
or because—in the case of large groups—there is the
mindset that “big systems need lots of people.”

229

© 1993-2014 Jeff Sutherland

Distributed & Outsourcing Styles

Fully Isolated Scrums

Coordinated backlog and Scrum of Scrums

Distributed Daily Meeting

230

© 1993-2014 Jeff Sutherland

Scrum Changes the Basic
Economics of Outsourcing
• What happens if you outsource $2M of
development?
 Industry data show 20% cost savings on average
• Outsourcing from PatientKeeper to Indian
waterfall team:
 Two years of data showed breakeven point occurs when
Indian developer costs 10% of American Scrum
developer
 Actual Indian cost is 30%
• $2M of Scrum development at my company
costs $6M when outsourced to waterfall teams

231

© 1993-2014 Jeff Sutherland

 Distributed Scrum Success Story

SirsiDynix – Digital Library Catalogue

PO PO PO

SyrsiDynix
• Provo, UT SM
• Denver, CO Dev
• Waterloo, ONT Dev
Dev

Test lead
Dev
Starsoft Labs Dev
• St. Petersburg, Dev
RUS

Catalogue Serials Circulation Search Reporting

232

© 1993-2014 Jeff Sutherland

 SirsiDynix: Daily Meeting Scheduled
to Fit Both Team’s Calendar

Daily Meeting
SyrsiDynix
• Provo, UT 7:45
• Denver, CO
• Waterloo, ONT

Starsoft Labs
• St. Petersburg,
RUS
Local 17:45
Team
Meeting

Daily Meeting conducted by video to enhance robustness

of communication
233

© 1993-2014 Jeff Sutherland

SirsiDynix: Robust Metrics
Dashboard Supported Transparency

234

© 1993-2014 Jeff Sutherland

SirsiDynix: Outcome

Waterfall1 Scrum1 SirsiDynix2

Person Months 540 54 827

Lines of Java 58,000 51,000 671,688

Function Points 900 959 12,673

Function Points/
2.0 17.8 15.3
Developer Mo.

1. M. Cohn, User Stories Applied for Agile Development. Addison-Wesley, 2004

2. J. Sutherland, A. Viktorov, J. Blount, and N. Puntikov, "Distributed Scrum: Agile Project Management with Outsourced
Development Teams," in HICSS'40, Hawaii International Conference on Software Systems, Big Island, Hawaii,

235

© 1993-2014 Jeff Sutherland

 As a PO, I need to calculate
Return on Investment (ROI)
to deliver business value quickly
!
!

© 2011 Scrum Inc.

What We Build First Depends on
Timing
• How do we balance long term returns against
short term?
• Net Present Value
– Formula
– Require ROI
– Estimate return on capital

237

© 1993-2014 Jeff Sutherland

 Establishing Business Value

NPV = ∑ Rt ÷ (1+i)t
t-1

Rt: Expected Revenue at Time t.

t: Number of Periods to be Calculated
i: Discount Rate/Opportunity Cost of Capital per Period TM

R
www.RapidScrum.com
2011 Copyright Rapid Scrum LLC. All rights reserved.
Effect of NPV Prioritization on Time to Market

239

© 1993-2014 Jeff Sutherland

Effect of NPV Prioritization on Market Share

240

© 1993-2014 Jeff Sutherland

Problems with NPV
• One of our largest clients abandoned any return
on investment analyses as it led to gaming the
system.
• It is like a black box to decision makers. It is
hard to tell what is really going on.
• Some major companies are moving to simple
rules or guidelines to make decisions at a
strategic level.
• This is a Scrum based approach as simple rules
are used to generate self-organization at every
level from the team to the company in Scrum.

241

© 1993-2014 Jeff Sutherland