SlideShare a Scribd company logo
Beyond Agile 
Testing to Lean 
Development 
James O. Coplien 
Gertrud & Cope
[Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development
Processes and 
Tools over 
Individuals and 
Interactions
Test 
Testing 
System 
Testing 
Exploratory 
Testing 
Usability Testing
[Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development
Why doesn’t unit 
testing work? 
We can’t test things of business 
value 
Not knowing what to test, we test 
many things that don’t need testing 
Automated crap is still crap
Unit tests can’t test 
things of business value 
Can you demonstrate that Method X of 
Object Y will ever be invoked in practice? 
If a unit test fails, how does it affect ROI? 
What’s amazing is that we continue to test… 
Paths that will never be exercised 
Methods that are never used 
Naive method invocation sequences
Most unit tests are 
unnecessary 
We test scenarios that will never be 
exercised in the real world 
We test methods that will never be 
invoked 
We supply data that a real application 
would never have to process
Automated crap is 
still crap 
Design requires intelligence: 
intelligence cannot be automated 
Autonomation instead of automation 
Testing is about exploratory inference 
Get manual procedures working first 
and then automate the mundane parts
Automated testing is not 
testing
Nokia Test Score 2: 
Testing 
No dedicated QA - 0 
Unit tested - 1 
Feature tested - 5 
Feature tested as soon as completed - 7 
Software passes acceptance testing - 8 
Software is deployed - 10
Weinberg’s Law of 
Decomposition
Weinberg’s Law of 
Decomposition 
Used Interface 
Used Interface 
Mu s t unit-te s t! 
Must unit-test! 
Must unit-test!
Weinberg’s Law of 
Decomposition 
Wasted testing effort 
Use 
Size 
Test 
Size
Part III: You Will Have Bugs! 
Scenarios we 
test in the lab 
Scenarios 
exhibiting 
faults 
Scenarios 
invoked in 
the field
Testing waste 
Removing tests 
leaves field faults 
Scenarios we 
test in the lab 
Scenarios 
exhibiting 
faults 
Scenarios 
invoked in 
the field 
undetected! 
Type 2 
Muda 
Type 1 
Muda 
Type 4 
Muda 
Type 1: Defects 
Type 2: Stuff 
customers 
don’t ask for 
Type 4: 
Type Overprocessing 
1 Muda 
(field)
Why do we unit 
test? 
Because we can 
It doesn’t require teamwork 
We don’t have to talk with the business 
We get to interact with tools instead of 
people 
Our decisions are emotional, not rational
Reduction of Waste 
Type 1: Better testing (e.g., going beyond 
contextual testing to exploratory & experience-based 
testing) (and better yet, better analysis 
and design, where the real payoffs are) 
Type 2: Don’t test dead code 
Type 4: Remove redundant and tautalogical 
tests, and tests that give no information 
Type 1 in the field: Ship your tests!
Don’t take the tests 
out when you ship! 
Remember, you will have bugs in the 
field! 
Deployment is your largest test driver 
Ship your tests as part of the code 
IN case of a failure, it saves you 
from Type 9 muda (unsafe execution)
Be Customer- 
Focused 
Assertions and Design by Contract!
How Adding ASSERTS reduces work 
2500000 
2000000 
1500000 
1000000 
500000 
0 
15 bugs p=0.01 15 bugs p=0.02 
10 bugs p=0.01 10 bugs p=0.02 
5 bugs p=0.01 5 bugs p=0.02 
0 1 2 5 10 20 50 100 
number of ASSERTS 
Work, Tsinglechecks 
Overload 
Magazine, 
October 2014
Defensive 
Programming 
… is largely the incorporation of 
tests in the code 
Long known for high-reliability 
systems 
The value of asserting the obvious 
Design by Contract
Assertions make code 
readable! 
- (void) prepareForSegue: (UIStoryboardSegue*) segue 
sender: (id) sender 
{ 
assert(segue != nil); 
if ([[segue identifier] isEqualToString: 
@"SegueAvatarPickerToPickOneLoginPlayer"]) { 
assert((UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPhone)); 
// The field parentViewController_ (instance variable) is 
// maybe null if we are in iPhone mode and doing a real 
// segue, instead of a popover as we do on the iPad. So 
// get the right object from the segue record. 
KnowsyPlayerPickerCommonViewController *parentViewController = 
[segue destinationViewController]; 
[parentViewController setKnowsyData: knowsyData_]; 
} else { 
// iPad - no special processing - should be no popover 
assert(parentViewController_ == nil); 
} 
}
Part IV: Beyond agile 
testing, Lean testing 
When a test finds a bug, it’s about 
fixing the process — not the bug 
The long-term payoff is higher! 
No bug-tracking systems: we fix 
problems NOW. 
The tester should be the team’s 
conscience
Other Lean-isms 
Deming: Quality is inversely 
proportional to the number of 
testers 
GM kept around its cars for 1 - 6 
weeks for test and repair: Toyota 
drove them off the end of the 
assembly line onto the car carriers.
Conclusion 
Be humble about what tests can accomplish 
It’s not about testing, but value 
Design tests to assess properties of business value 
Go beyond unit testing to defensive 
programming and design-by-contract 
As agile testers, focus on process first and 
product second
[Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development
Ad

More Related Content

What's hot (20)

Automating Strategically or Tactically when Testing
Automating Strategically or Tactically when TestingAutomating Strategically or Tactically when Testing
Automating Strategically or Tactically when Testing
Alan Richardson
 
Exactpro FinTech Webinar - Global Exchanges Test Oracles
Exactpro FinTech Webinar - Global Exchanges Test OraclesExactpro FinTech Webinar - Global Exchanges Test Oracles
Exactpro FinTech Webinar - Global Exchanges Test Oracles
Iosif Itkin
 
5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing
Mary Clemons
 
Dependency Injection in iOS
Dependency Injection in iOSDependency Injection in iOS
Dependency Injection in iOS
Pablo Villar
 
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in FlexassertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
michael.labriola
 
Push Functional Testing Further
Push Functional Testing FurtherPush Functional Testing Further
Push Functional Testing Further
Alan Richardson
 
What is Regression Testing?
What is Regression Testing?What is Regression Testing?
What is Regression Testing?
Alan Richardson
 
Exploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextExploratory Testing in an Agile Context
Exploratory Testing in an Agile Context
Elisabeth Hendrickson
 
Journey to unit testing
Journey to unit testingJourney to unit testing
Journey to unit testing
Scrum Day Bandung
 
Joy of Coding Conference 2019 slides - Alan Richardson
Joy of Coding Conference 2019 slides - Alan RichardsonJoy of Coding Conference 2019 slides - Alan Richardson
Joy of Coding Conference 2019 slides - Alan Richardson
Alan Richardson
 
Automating Pragmatically - Testival 20190604
Automating Pragmatically - Testival 20190604Automating Pragmatically - Testival 20190604
Automating Pragmatically - Testival 20190604
Alan Richardson
 
.Net Debugging Techniques
.Net Debugging Techniques.Net Debugging Techniques
.Net Debugging Techniques
Bala Subra
 
How to Deliver the Right Software (Specification by example)
How to Deliver the Right Software (Specification by example)How to Deliver the Right Software (Specification by example)
How to Deliver the Right Software (Specification by example)
Asier Barrenetxea
 
Evil testers guide to technical testing
Evil testers guide to technical testingEvil testers guide to technical testing
Evil testers guide to technical testing
Alan Richardson
 
Justin Ison
Justin IsonJustin Ison
Justin Ison
CodeFest
 
Test Drive Development
Test Drive DevelopmentTest Drive Development
Test Drive Development
satya sudheer
 
First steps in testing analytics: Does test code quality matter?
First steps in testing analytics: Does test code quality matter?First steps in testing analytics: Does test code quality matter?
First steps in testing analytics: Does test code quality matter?
Andy Zaidman
 
Your Automated Execution Does Not Have to be Flaky
Your Automated Execution Does Not Have to be FlakyYour Automated Execution Does Not Have to be Flaky
Your Automated Execution Does Not Have to be Flaky
Alan Richardson
 
Jeremias Rößler
Jeremias RößlerJeremias Rößler
Jeremias Rößler
CodeFest
 
Effective Software Testing for Modern Software Development
Effective Software Testing for Modern Software DevelopmentEffective Software Testing for Modern Software Development
Effective Software Testing for Modern Software Development
Alan Richardson
 
Automating Strategically or Tactically when Testing
Automating Strategically or Tactically when TestingAutomating Strategically or Tactically when Testing
Automating Strategically or Tactically when Testing
Alan Richardson
 
Exactpro FinTech Webinar - Global Exchanges Test Oracles
Exactpro FinTech Webinar - Global Exchanges Test OraclesExactpro FinTech Webinar - Global Exchanges Test Oracles
Exactpro FinTech Webinar - Global Exchanges Test Oracles
Iosif Itkin
 
5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing5-Ways-to-Revolutionize-Your-Software-Testing
5-Ways-to-Revolutionize-Your-Software-Testing
Mary Clemons
 
Dependency Injection in iOS
Dependency Injection in iOSDependency Injection in iOS
Dependency Injection in iOS
Pablo Villar
 
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in FlexassertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
assertYourself - Breaking the Theories and Assumptions of Unit Testing in Flex
michael.labriola
 
Push Functional Testing Further
Push Functional Testing FurtherPush Functional Testing Further
Push Functional Testing Further
Alan Richardson
 
What is Regression Testing?
What is Regression Testing?What is Regression Testing?
What is Regression Testing?
Alan Richardson
 
Exploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextExploratory Testing in an Agile Context
Exploratory Testing in an Agile Context
Elisabeth Hendrickson
 
Joy of Coding Conference 2019 slides - Alan Richardson
Joy of Coding Conference 2019 slides - Alan RichardsonJoy of Coding Conference 2019 slides - Alan Richardson
Joy of Coding Conference 2019 slides - Alan Richardson
Alan Richardson
 
Automating Pragmatically - Testival 20190604
Automating Pragmatically - Testival 20190604Automating Pragmatically - Testival 20190604
Automating Pragmatically - Testival 20190604
Alan Richardson
 
.Net Debugging Techniques
.Net Debugging Techniques.Net Debugging Techniques
.Net Debugging Techniques
Bala Subra
 
How to Deliver the Right Software (Specification by example)
How to Deliver the Right Software (Specification by example)How to Deliver the Right Software (Specification by example)
How to Deliver the Right Software (Specification by example)
Asier Barrenetxea
 
Evil testers guide to technical testing
Evil testers guide to technical testingEvil testers guide to technical testing
Evil testers guide to technical testing
Alan Richardson
 
Justin Ison
Justin IsonJustin Ison
Justin Ison
CodeFest
 
Test Drive Development
Test Drive DevelopmentTest Drive Development
Test Drive Development
satya sudheer
 
First steps in testing analytics: Does test code quality matter?
First steps in testing analytics: Does test code quality matter?First steps in testing analytics: Does test code quality matter?
First steps in testing analytics: Does test code quality matter?
Andy Zaidman
 
Your Automated Execution Does Not Have to be Flaky
Your Automated Execution Does Not Have to be FlakyYour Automated Execution Does Not Have to be Flaky
Your Automated Execution Does Not Have to be Flaky
Alan Richardson
 
Jeremias Rößler
Jeremias RößlerJeremias Rößler
Jeremias Rößler
CodeFest
 
Effective Software Testing for Modern Software Development
Effective Software Testing for Modern Software DevelopmentEffective Software Testing for Modern Software Development
Effective Software Testing for Modern Software Development
Alan Richardson
 

Similar to [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development (20)

Unit Testing in Android
Unit Testing in AndroidUnit Testing in Android
Unit Testing in Android
Md Shamsul Arafin Mahtab
 
TDD Best Practices
TDD Best PracticesTDD Best Practices
TDD Best Practices
Attila Bertók
 
Unit Testing
Unit TestingUnit Testing
Unit Testing
David Rogers
 
Unit testing (workshop)
Unit testing (workshop)Unit testing (workshop)
Unit testing (workshop)
Foyzul Karim
 
Why Unit Testingl
Why Unit TestinglWhy Unit Testingl
Why Unit Testingl
priya_trivedi
 
Why Unit Testingl
Why Unit TestinglWhy Unit Testingl
Why Unit Testingl
priya_trivedi
 
Why unit testingl
Why unit testinglWhy unit testingl
Why unit testingl
Priya Sharma
 
Test driven development
Test driven developmentTest driven development
Test driven development
namkha87
 
Android testing part i
Android testing part iAndroid testing part i
Android testing part i
Kan-Han (John) Lu
 
I Smell A RAT- Rapid Application Testing
I Smell A RAT- Rapid Application TestingI Smell A RAT- Rapid Application Testing
I Smell A RAT- Rapid Application Testing
Peter Presnell
 
Unit tests & TDD
Unit tests & TDDUnit tests & TDD
Unit tests & TDD
Dror Helper
 
Unit Testing
Unit TestingUnit Testing
Unit Testing
Anuj Arora
 
Practical unit testing tips
Practical unit testing tipsPractical unit testing tips
Practical unit testing tips
Typemock
 
Anatomy of Test Driven Development
Anatomy of Test Driven DevelopmentAnatomy of Test Driven Development
Anatomy of Test Driven Development
Dhaval Shah
 
Dev fest2015androidunittestingbyoyewaleademolasao
Dev fest2015androidunittestingbyoyewaleademolasaoDev fest2015androidunittestingbyoyewaleademolasao
Dev fest2015androidunittestingbyoyewaleademolasao
Oyewale Ademola
 
Unit-testing and E2E testing in JS
Unit-testing and E2E testing in JSUnit-testing and E2E testing in JS
Unit-testing and E2E testing in JS
Michael Haberman
 
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Turkish Testing Board
 
QA Best Practices
QA  Best PracticesQA  Best Practices
QA Best Practices
James York
 
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
TEST Huddle
 
Agile Testing Pasadena JUG Aug2009
Agile Testing Pasadena JUG Aug2009Agile Testing Pasadena JUG Aug2009
Agile Testing Pasadena JUG Aug2009
Grig Gheorghiu
 
Unit testing (workshop)
Unit testing (workshop)Unit testing (workshop)
Unit testing (workshop)
Foyzul Karim
 
Test driven development
Test driven developmentTest driven development
Test driven development
namkha87
 
I Smell A RAT- Rapid Application Testing
I Smell A RAT- Rapid Application TestingI Smell A RAT- Rapid Application Testing
I Smell A RAT- Rapid Application Testing
Peter Presnell
 
Unit tests & TDD
Unit tests & TDDUnit tests & TDD
Unit tests & TDD
Dror Helper
 
Practical unit testing tips
Practical unit testing tipsPractical unit testing tips
Practical unit testing tips
Typemock
 
Anatomy of Test Driven Development
Anatomy of Test Driven DevelopmentAnatomy of Test Driven Development
Anatomy of Test Driven Development
Dhaval Shah
 
Dev fest2015androidunittestingbyoyewaleademolasao
Dev fest2015androidunittestingbyoyewaleademolasaoDev fest2015androidunittestingbyoyewaleademolasao
Dev fest2015androidunittestingbyoyewaleademolasao
Oyewale Ademola
 
Unit-testing and E2E testing in JS
Unit-testing and E2E testing in JSUnit-testing and E2E testing in JS
Unit-testing and E2E testing in JS
Michael Haberman
 
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Testistanbul 2016 - Keynote: "Why Automated Verification Matters" by Kristian...
Turkish Testing Board
 
QA Best Practices
QA  Best PracticesQA  Best Practices
QA Best Practices
James York
 
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
TEST Huddle
 
Agile Testing Pasadena JUG Aug2009
Agile Testing Pasadena JUG Aug2009Agile Testing Pasadena JUG Aug2009
Agile Testing Pasadena JUG Aug2009
Grig Gheorghiu
 
Ad

More from Rakuten Group, Inc. (20)

EPSS (Exploit Prediction Scoring System)モニタリングツールの開発
EPSS (Exploit Prediction Scoring System)モニタリングツールの開発EPSS (Exploit Prediction Scoring System)モニタリングツールの開発
EPSS (Exploit Prediction Scoring System)モニタリングツールの開発
Rakuten Group, Inc.
 
コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話
コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話
コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話
Rakuten Group, Inc.
 
楽天における安全な秘匿情報管理への道のり
楽天における安全な秘匿情報管理への道のり楽天における安全な秘匿情報管理への道のり
楽天における安全な秘匿情報管理への道のり
Rakuten Group, Inc.
 
What Makes Software Green?
What Makes Software Green?What Makes Software Green?
What Makes Software Green?
Rakuten Group, Inc.
 
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...
Rakuten Group, Inc.
 
DataSkillCultureを浸透させる楽天の取り組み
DataSkillCultureを浸透させる楽天の取り組みDataSkillCultureを浸透させる楽天の取り組み
DataSkillCultureを浸透させる楽天の取り組み
Rakuten Group, Inc.
 
大規模なリアルタイム監視の導入と展開
大規模なリアルタイム監視の導入と展開大規模なリアルタイム監視の導入と展開
大規模なリアルタイム監視の導入と展開
Rakuten Group, Inc.
 
楽天における大規模データベースの運用
楽天における大規模データベースの運用楽天における大規模データベースの運用
楽天における大規模データベースの運用
Rakuten Group, Inc.
 
楽天サービスを支えるネットワークインフラストラクチャー
楽天サービスを支えるネットワークインフラストラクチャー楽天サービスを支えるネットワークインフラストラクチャー
楽天サービスを支えるネットワークインフラストラクチャー
Rakuten Group, Inc.
 
楽天の規模とクラウドプラットフォーム統括部の役割
楽天の規模とクラウドプラットフォーム統括部の役割楽天の規模とクラウドプラットフォーム統括部の役割
楽天の規模とクラウドプラットフォーム統括部の役割
Rakuten Group, Inc.
 
Rakuten Services and Infrastructure Team.pdf
Rakuten Services and Infrastructure Team.pdfRakuten Services and Infrastructure Team.pdf
Rakuten Services and Infrastructure Team.pdf
Rakuten Group, Inc.
 
The Data Platform Administration Handling the 100 PB.pdf
The Data Platform Administration Handling the 100 PB.pdfThe Data Platform Administration Handling the 100 PB.pdf
The Data Platform Administration Handling the 100 PB.pdf
Rakuten Group, Inc.
 
Supporting Internal Customers as Technical Account Managers.pdf
Supporting Internal Customers as Technical Account Managers.pdfSupporting Internal Customers as Technical Account Managers.pdf
Supporting Internal Customers as Technical Account Managers.pdf
Rakuten Group, Inc.
 
Making Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdfMaking Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdf
Rakuten Group, Inc.
 
How We Defined Our Own Cloud.pdf
How We Defined Our Own Cloud.pdfHow We Defined Our Own Cloud.pdf
How We Defined Our Own Cloud.pdf
Rakuten Group, Inc.
 
Travel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech infoTravel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech info
Rakuten Group, Inc.
 
Travel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech infoTravel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech info
Rakuten Group, Inc.
 
OWASPTop10_Introduction
OWASPTop10_IntroductionOWASPTop10_Introduction
OWASPTop10_Introduction
Rakuten Group, Inc.
 
Introduction of GORA API Group technology
Introduction of GORA API Group technologyIntroduction of GORA API Group technology
Introduction of GORA API Group technology
Rakuten Group, Inc.
 
100PBを越えるデータプラットフォームの実情
100PBを越えるデータプラットフォームの実情100PBを越えるデータプラットフォームの実情
100PBを越えるデータプラットフォームの実情
Rakuten Group, Inc.
 
EPSS (Exploit Prediction Scoring System)モニタリングツールの開発
EPSS (Exploit Prediction Scoring System)モニタリングツールの開発EPSS (Exploit Prediction Scoring System)モニタリングツールの開発
EPSS (Exploit Prediction Scoring System)モニタリングツールの開発
Rakuten Group, Inc.
 
コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話
コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話
コードレビュー改善のためにJenkinsとIntelliJ IDEAのプラグインを自作してみた話
Rakuten Group, Inc.
 
楽天における安全な秘匿情報管理への道のり
楽天における安全な秘匿情報管理への道のり楽天における安全な秘匿情報管理への道のり
楽天における安全な秘匿情報管理への道のり
Rakuten Group, Inc.
 
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...
Simple and Effective Knowledge-Driven Query Expansion for QA-Based Product At...
Rakuten Group, Inc.
 
DataSkillCultureを浸透させる楽天の取り組み
DataSkillCultureを浸透させる楽天の取り組みDataSkillCultureを浸透させる楽天の取り組み
DataSkillCultureを浸透させる楽天の取り組み
Rakuten Group, Inc.
 
大規模なリアルタイム監視の導入と展開
大規模なリアルタイム監視の導入と展開大規模なリアルタイム監視の導入と展開
大規模なリアルタイム監視の導入と展開
Rakuten Group, Inc.
 
楽天における大規模データベースの運用
楽天における大規模データベースの運用楽天における大規模データベースの運用
楽天における大規模データベースの運用
Rakuten Group, Inc.
 
楽天サービスを支えるネットワークインフラストラクチャー
楽天サービスを支えるネットワークインフラストラクチャー楽天サービスを支えるネットワークインフラストラクチャー
楽天サービスを支えるネットワークインフラストラクチャー
Rakuten Group, Inc.
 
楽天の規模とクラウドプラットフォーム統括部の役割
楽天の規模とクラウドプラットフォーム統括部の役割楽天の規模とクラウドプラットフォーム統括部の役割
楽天の規模とクラウドプラットフォーム統括部の役割
Rakuten Group, Inc.
 
Rakuten Services and Infrastructure Team.pdf
Rakuten Services and Infrastructure Team.pdfRakuten Services and Infrastructure Team.pdf
Rakuten Services and Infrastructure Team.pdf
Rakuten Group, Inc.
 
The Data Platform Administration Handling the 100 PB.pdf
The Data Platform Administration Handling the 100 PB.pdfThe Data Platform Administration Handling the 100 PB.pdf
The Data Platform Administration Handling the 100 PB.pdf
Rakuten Group, Inc.
 
Supporting Internal Customers as Technical Account Managers.pdf
Supporting Internal Customers as Technical Account Managers.pdfSupporting Internal Customers as Technical Account Managers.pdf
Supporting Internal Customers as Technical Account Managers.pdf
Rakuten Group, Inc.
 
Making Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdfMaking Cloud Native CI_CD Services.pdf
Making Cloud Native CI_CD Services.pdf
Rakuten Group, Inc.
 
How We Defined Our Own Cloud.pdf
How We Defined Our Own Cloud.pdfHow We Defined Our Own Cloud.pdf
How We Defined Our Own Cloud.pdf
Rakuten Group, Inc.
 
Travel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech infoTravel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech info
Rakuten Group, Inc.
 
Travel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech infoTravel & Leisure Platform Department's tech info
Travel & Leisure Platform Department's tech info
Rakuten Group, Inc.
 
Introduction of GORA API Group technology
Introduction of GORA API Group technologyIntroduction of GORA API Group technology
Introduction of GORA API Group technology
Rakuten Group, Inc.
 
100PBを越えるデータプラットフォームの実情
100PBを越えるデータプラットフォームの実情100PBを越えるデータプラットフォームの実情
100PBを越えるデータプラットフォームの実情
Rakuten Group, Inc.
 
Ad

Recently uploaded (20)

Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
organizerofv
 
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
2025-05-Q4-2024-Investor-Presentation.pptx
2025-05-Q4-2024-Investor-Presentation.pptx2025-05-Q4-2024-Investor-Presentation.pptx
2025-05-Q4-2024-Investor-Presentation.pptx
Samuele Fogagnolo
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPathCommunity
 
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveDesigning Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
ScyllaDB
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-UmgebungenHCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
panagenda
 
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business Consulting
Trs Labs
 
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
BookNet Canada
 
Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)
Ortus Solutions, Corp
 
Generative Artificial Intelligence (GenAI) in Business
Generative Artificial Intelligence (GenAI) in BusinessGenerative Artificial Intelligence (GenAI) in Business
Generative Artificial Intelligence (GenAI) in Business
Dr. Tathagat Varma
 
Rusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond SparkRusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond Spark
carlyakerly1
 
How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?
Daniel Lehner
 
Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
organizerofv
 
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...
TrustArc
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
2025-05-Q4-2024-Investor-Presentation.pptx
2025-05-Q4-2024-Investor-Presentation.pptx2025-05-Q4-2024-Investor-Presentation.pptx
2025-05-Q4-2024-Investor-Presentation.pptx
Samuele Fogagnolo
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPathCommunity
 
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveDesigning Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
ScyllaDB
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-UmgebungenHCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
panagenda
 
TrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business ConsultingTrsLabs - Fintech Product & Business Consulting
TrsLabs - Fintech Product & Business Consulting
Trs Labs
 
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
BookNet Canada
 
Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)
Ortus Solutions, Corp
 
Generative Artificial Intelligence (GenAI) in Business
Generative Artificial Intelligence (GenAI) in BusinessGenerative Artificial Intelligence (GenAI) in Business
Generative Artificial Intelligence (GenAI) in Business
Dr. Tathagat Varma
 
Rusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond SparkRusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond Spark
carlyakerly1
 
How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?
Daniel Lehner
 

[Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development

  • 1. Beyond Agile Testing to Lean Development James O. Coplien Gertrud & Cope
  • 3. Processes and Tools over Individuals and Interactions
  • 4. Test Testing System Testing Exploratory Testing Usability Testing
  • 6. Why doesn’t unit testing work? We can’t test things of business value Not knowing what to test, we test many things that don’t need testing Automated crap is still crap
  • 7. Unit tests can’t test things of business value Can you demonstrate that Method X of Object Y will ever be invoked in practice? If a unit test fails, how does it affect ROI? What’s amazing is that we continue to test… Paths that will never be exercised Methods that are never used Naive method invocation sequences
  • 8. Most unit tests are unnecessary We test scenarios that will never be exercised in the real world We test methods that will never be invoked We supply data that a real application would never have to process
  • 9. Automated crap is still crap Design requires intelligence: intelligence cannot be automated Autonomation instead of automation Testing is about exploratory inference Get manual procedures working first and then automate the mundane parts
  • 10. Automated testing is not testing
  • 11. Nokia Test Score 2: Testing No dedicated QA - 0 Unit tested - 1 Feature tested - 5 Feature tested as soon as completed - 7 Software passes acceptance testing - 8 Software is deployed - 10
  • 12. Weinberg’s Law of Decomposition
  • 13. Weinberg’s Law of Decomposition Used Interface Used Interface Mu s t unit-te s t! Must unit-test! Must unit-test!
  • 14. Weinberg’s Law of Decomposition Wasted testing effort Use Size Test Size
  • 15. Part III: You Will Have Bugs! Scenarios we test in the lab Scenarios exhibiting faults Scenarios invoked in the field
  • 16. Testing waste Removing tests leaves field faults Scenarios we test in the lab Scenarios exhibiting faults Scenarios invoked in the field undetected! Type 2 Muda Type 1 Muda Type 4 Muda Type 1: Defects Type 2: Stuff customers don’t ask for Type 4: Type Overprocessing 1 Muda (field)
  • 17. Why do we unit test? Because we can It doesn’t require teamwork We don’t have to talk with the business We get to interact with tools instead of people Our decisions are emotional, not rational
  • 18. Reduction of Waste Type 1: Better testing (e.g., going beyond contextual testing to exploratory & experience-based testing) (and better yet, better analysis and design, where the real payoffs are) Type 2: Don’t test dead code Type 4: Remove redundant and tautalogical tests, and tests that give no information Type 1 in the field: Ship your tests!
  • 19. Don’t take the tests out when you ship! Remember, you will have bugs in the field! Deployment is your largest test driver Ship your tests as part of the code IN case of a failure, it saves you from Type 9 muda (unsafe execution)
  • 20. Be Customer- Focused Assertions and Design by Contract!
  • 21. How Adding ASSERTS reduces work 2500000 2000000 1500000 1000000 500000 0 15 bugs p=0.01 15 bugs p=0.02 10 bugs p=0.01 10 bugs p=0.02 5 bugs p=0.01 5 bugs p=0.02 0 1 2 5 10 20 50 100 number of ASSERTS Work, Tsinglechecks Overload Magazine, October 2014
  • 22. Defensive Programming … is largely the incorporation of tests in the code Long known for high-reliability systems The value of asserting the obvious Design by Contract
  • 23. Assertions make code readable! - (void) prepareForSegue: (UIStoryboardSegue*) segue sender: (id) sender { assert(segue != nil); if ([[segue identifier] isEqualToString: @"SegueAvatarPickerToPickOneLoginPlayer"]) { assert((UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPhone)); // The field parentViewController_ (instance variable) is // maybe null if we are in iPhone mode and doing a real // segue, instead of a popover as we do on the iPad. So // get the right object from the segue record. KnowsyPlayerPickerCommonViewController *parentViewController = [segue destinationViewController]; [parentViewController setKnowsyData: knowsyData_]; } else { // iPad - no special processing - should be no popover assert(parentViewController_ == nil); } }
  • 24. Part IV: Beyond agile testing, Lean testing When a test finds a bug, it’s about fixing the process — not the bug The long-term payoff is higher! No bug-tracking systems: we fix problems NOW. The tester should be the team’s conscience
  • 25. Other Lean-isms Deming: Quality is inversely proportional to the number of testers GM kept around its cars for 1 - 6 weeks for test and repair: Toyota drove them off the end of the assembly line onto the car carriers.
  • 26. Conclusion Be humble about what tests can accomplish It’s not about testing, but value Design tests to assess properties of business value Go beyond unit testing to defensive programming and design-by-contract As agile testers, focus on process first and product second

Editor's Notes

  • #2: BEYOND AGILE TESTING TO LEAN DEVELOPMENT - Jim Coplien • Testing is a business function that should be focused on risk  • Lean defines testing as waste — why do it?  • Automated tests should reflect requirements, but testers shouldn't test requirements  • Testers are the team's conscience 50 Minutes
  • #3: Mike Cohn’s testing triangle
  • #11: http://qz.com/196200/toyota-is-becoming-more-efficient-by-replacing-robots-with-humans/ Max Niesen, April 7, 2014 James Bach: “Testing Requires Inference.” “Exploratory inference may sound like a strange idea. It means one idea leading to another in ways you can’t predict in advance” [and, therefore, can’t script].
  • #13: What if the methods we break them down with are private and used just to make the code more readable? I'm sure you could pronounce it faster... And I'm sure that it takes less time to read individual sentences than to read all of Moby Dick. But readability isn't the entire story, and comprehension is a big part of that. Appreciation comes in the whole. You don't appreciate the beauty of the Mona Lisa by studying only her nose: you take in the whole. It is more than the sum of its parts, just as understanding must be more than the sum of its parts. There are some strange loops going in the other direction, and this is where I've picked up a new tool in the past weeks: Weinberg's law of decomposition. Each part is much more than a decomposition of the whole. Because it is so much more, it is harder to understand as an entity than it is to understand when contextualized by its surroundings. So the phrase, "Frankly, dear, I don't give a damn" has some meaning in its own right but 100 times as much meaning when read in the context of "Gone With The Wind." Poetry is about about compression: about using cultural context, and the context of the rest of the work, to amplify meaning. Great software is group poetry writing. If you're not doing that, you're just a machine automating the generation of some object code in an assembly language called Java.
  • #14: What if the methods we break them down with are private and used just to make the code more readable? I'm sure you could pronounce it faster... And I'm sure that it takes less time to read individual sentences than to read all of Moby Dick. But readability isn't the entire story, and comprehension is a big part of that. Appreciation comes in the whole. You don't appreciate the beauty of the Mona Lisa by studying only her nose: you take in the whole. It is more than the sum of its parts, just as understanding must be more than the sum of its parts. There are some strange loops going in the other direction, and this is where I've picked up a new tool in the past weeks: Weinberg's law of decomposition. Each part is much more than a decomposition of the whole. Because it is so much more, it is harder to understand as an entity than it is to understand when contextualized by its surroundings. So the phrase, "Frankly, dear, I don't give a damn" has some meaning in its own right but 100 times as much meaning when read in the context of "Gone With The Wind." Poetry is about about compression: about using cultural context, and the context of the rest of the work, to amplify meaning. Great software is group poetry writing. If you're not doing that, you're just a machine automating the generation of some object code in an assembly language called Java.
  • #15: What if the methods we break them down with are private and used just to make the code more readable? I'm sure you could pronounce it faster... And I'm sure that it takes less time to read individual sentences than to read all of Moby Dick. But readability isn't the entire story, and comprehension is a big part of that. Appreciation comes in the whole. You don't appreciate the beauty of the Mona Lisa by studying only her nose: you take in the whole. It is more than the sum of its parts, just as understanding must be more than the sum of its parts. There are some strange loops going in the other direction, and this is where I've picked up a new tool in the past weeks: Weinberg's law of decomposition. Each part is much more than a decomposition of the whole. Because it is so much more, it is harder to understand as an entity than it is to understand when contextualized by its surroundings. So the phrase, "Frankly, dear, I don't give a damn" has some meaning in its own right but 100 times as much meaning when read in the context of "Gone With The Wind." Poetry is about about compression: about using cultural context, and the context of the rest of the work, to amplify meaning. Great software is group poetry writing. If you're not doing that, you're just a machine automating the generation of some object code in an assembly language called Java.
  • #17: Type 2: Weinberg’s Law of Decomposition Type 4: Keeping tests that never fail
  • #18: Last point: Most tests are irrational: i.e., you can never presume software correctness from the tests. But we do feel better by having checked, so we test anyhow.
  • #20: Be customer-focused
  • #22: Debug Complexity: How Assertions Affect Debugging Time. Sergey Ignatchenko and Dmytro Ivanchykhin, Overload, October 2014
  • #25: Conscience… Curmudgeonly and like Emperor’s New Clothes — not in the sense of standing in the way. That’s business’s job.
  • #26: No unit - subsystem - system - integration - acceptance test leveling.