8 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action

8 principles for enabling build/measure/learn
lean engineering in action
eBay Classiﬁeds TechCon
June 2013
@billwscott
Sr. Director
User Interface Engineering
@paypal

continuous customer feedback (GOOB)
customer metrics drive everything
think it. build it. ship it. tweak it
fail fast. learn fast.
lots of experimentation... build/measure/learn
a different view of
engineering

paypal vs netﬂix
contrast this with a large enterprise like paypal (circa 2011)

guess what i found (in 2011)
roll your own. disconnected delivery
experience. culture of long shelf life.
inward focus. risk averse.

In 2011, even a simple
content copy change
could take as much as 6
weeks to get live to site

new dna inserted
jan 2012
ﬂeshed out ui layer that could support rapid
experimentation
march 2012
david Marcus becomes president of PayPal
april 2012
formed lean ux team to reinvent checkout
experience

hermes project lean ux/engineering in action
from whiteboard to code from code to usability
learningsstart again

change has started working its way out

lean startup movement
founded on build/measure/learn cycle
get out of the building (GOOB)
invalidate your risky assumptions
fail fast, learn fast
get to the pivot
go for the minimal viable product (MVP)

lean ux
designing products for build/measure/learn
requires 3 rules to be followed at all times
get to & maintain a shared understanding
form deep collaboration across disciplines
keep continuous customer feedback ﬂowing

purpose of lean engineering
build
measure
learn
LEANENGINEERING
EnablingBuild/
Measure/Learn
withLeanStartup
Principles

build
embrace
continuous delivery
make mistakes fast
measure learn
the etsy way. Kellan Elliott-McCrea, CTO etsy
use metrics driven
development
know that you made a
mistake
blameless post
mortem
learn from your
mistakes

LEANENGINEERING
8principlesforenabling
build/measure/learn

1. focus on learning, not delivery

one of our biggest challenges is moving
from a culture of delivery to a
culture of learning

too many teams can create silos within the
exerience
common silos that can affect experience:
• number of scrum teams
• specialization of skills
• device channels
• regional adaptations
CE2
don’t let delivery drive
experience

2. build a culture of rapid experimentation

long shelf life for software
when software is not dynamically updatable
when it takes herculean effort to deliver
result
engineers run the asylum
delivery dates drive the experience
BDUF & waterfall prevail

16 different test cells in the initial PS3 Launch (2010)
focus is on build/measure/learn
four distinct PS3 experiences launched on same day
typical netflix release

ramping vs experimenting
ramping model results in one
experience (with some tweaks along
the way) after a long ramp up time
experimentation model results in
many experiences being tested all
along the way

avoid disconnected
delivery experience
circa 1985
deliver to disk then to user
everything was focused on getting it
perfect for stamping on the disk
no user in the loop. experience
happened somewhere down the
supply chain

you have to engineer
for volatility
change is the norm
experimentation is not a one time event
launching a product is giving birth to the
product. the product’s life just begins.
design for throwaway-ability
majority of the
experience code
written is thrown
away in a year
the ui layer is the
experimentation layer

experiences must learn
Our software is always tearing itself apart
(or should be)
Recognize that different layers change at
different velocities
All buildings are predictions.
All predictions are wrong.
There's no escape from this grim
syllogism, but it can be softened.
Stewart Brand

building experiences
circa 1985
merry band of three. dropped out of
college for semester. it was nirvana.
however...

roll your own “everything”
(close your eyes & imagine)
no internet. no google. no blogs. no email. no
blogs. no stackoverﬂow. no github. no twitter.
much of the software era has been about
building from scratch.
of course open source was gaining momentum.
unix. gnu. linux. perl. mozilla.

work in open source model
internal github revolutionizing
our internal development
rapidly replacing centralized
platform teams
innovation democratized
every developer encouraged
to experiment and generate repos
to share as well as to fork/pull request

give back to open source
we have projects that we will open source
node webcore (similar to yeoman)
we are contributing back to open source
contributions to bootstrap (for accessibility)
contributions to bootstrap (for internationalization)
core committer on dustjs project

using github for continuous *
use github for continuous integration
starting to use github repo model for continuous deployment
marketing pages
product pages
content updates & triggers into i18n, l10n, adaptation
components

btw, agile doesn’t have a brain...
agile has been a big step in the right direction
but is an engineering discipline
doesn’t address the full life cycle
agile has become big business and sometimes collapses under the weight
of “ceremonies” (process)
but agile is a good “engine” for delivery if you know what to roughly build
agile needs a brain...

lean ux: enable a brain for agile
user interface engineering - agile scrum team (production)
lean ux - lean team track (prototyping)
engineering - agile scrum teamsprint 0
usability usability usability usability usability
release release release release
{agile

6. make your product a living spec

create a living spec
enabling the prototype
learning

stack circa 2011/early 2012
simple change could take minutes
to see
follows an “enterprise application”
model. ui gets built into the “app”
java
jsp***
restricted
capabilities*
prototyping
was hard
“ui bits” could
only live here
* assumed client developers were low-skill
* required server side java eng for simple client changes
** java server pages. server-side java templating solution
server side
components**
client
server

we blended prototype & production
we enabled the “ui bits” to be
portable between the prototyping
stack and the production stackjava (rhinoscript)node.js
{dust}
JS template
prototype
stack
production
stack
{dust}
JS template
either stack

java (rhinoscript)
production
stack
{dust}
JS template
one stack: prototype & production
node.js
{dust}
JS template
prototype
stack
the ﬁnal step is we made the
prototype stack and production
stack the same technology
throughout the application stack

7. refactor your way out of debt

technical debt
rarely do you have a clean slate
generally you will have to refactor your
way to a nimble framework

we separated the ui bits
code = JS
(backbone)
templates =
JS
{dust}
style = CSS
(less)
images
re-engineered the user
interface stack so that
the only artifacts are:
• javascript
• css
• images
ditched the server-side
mentality to creating UIs
• no more server-side
only templates
components
managing the ui

code = JS
(backbone)
templates =
JS
{dust}
style = CSS
(less)
images
we used javascript templating
templates get converted
to javascript
<p>Hello {name}</p>
JavaScript compiles to...
javascript
executed
to generate ui

we used natural web artifacts - “web bits”
server-side language independent
server/client agnostic
CDN ready
cacheable
rapid to create
code = JS
(backbone)
templates =
JS
{dust}
style = CSS
(less)
images

ensured we could run on new & legacy
JS templating can be run
in client browser or
server on the production
stack
we can drag & drop the
ui bits from prototyping
stack to the production
stack
java
(rhinoscript)
node.js
{dust}
JS template
prototype
stack
production
stack
{dust}
JS template
client OR
server
either stack

experience debt
don’t just think about our technical debt
consider our “experience debt”
cripples our ability to capture market and
inhibits learning

mobile strategy ≠ just iOS app
native apps make it easier to create a rich
experience
however, they are limited in reach and in
learning capability
app install rates will only be a subset of the
customer base
you need both a native and html5 strategy in
order to maximize learning

html5 is critical to learning strategy
new users will see your html5 experience
the onramp to onboarding is the lowly link
network delivery makes a/b testing
straightforward
netﬂix gambled on html5 for mobile (iOS,
android) and for game consoles, bluray players,
hdtvs, etc.
why? build/measure/learn. network delivery.

summary
rethink engineering. every dimension of your engineering needs to be about
enabling build/measure/learn
technology. but not for tech sake. we are doing it for the experience to
support lean startup principles.
process. enabled lean ux and put a brain on agile.
people. revitalizing our existing talent and started attracting new talent.

designing web interfaces
O’Reilly
picture credits
http://www.flickr.com/photos/decade_null/2053134780/
http://www.flickr.com/photos/therevsteve/3104267109/
http://www.flickr.com/photos/juanpol/16287486/
http://www.flickr.com/photos/giesenbauer/4092794246/
http://www.flickr.com/photos/not_wise/182849352/
http://www.flickr.com/photos/mbiskoping/6075387388/
http://www.flickr.com/photos/37217398@N02/3442676067/
http://www.flickr.com/photos/proimos/3473264448/
http://www.flickr.com/photos/epsos/8463683689/
http://www.flickr.com/photos/stuckincustoms/2380543038/
http://www.flickr.com/photos/matthewpaulson/6176787688/
http://www.flickr.com/photos/90585146@N08/8222922317/
http://www.flickr.com/photos/cote/63914774/
http://www.flickr.com/photos/olvrbrown/4542851399/
http://www.flickr.com/photos/donpezzano/3257999898/
follow me on twitter
@billwscott
Designing Web Interfaces
O’Reilly
Bill Scott & Theresa Neil

8 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action

Recommended

More Related Content

What's hot (19)

Viewers also liked (10)

Similar to 8 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action (20)

More from Bill Scott (14)

Recently uploaded (20)

8 Principles for Enabling Build/Measure/Learn: Lean Engineering in Action