Build the "right" regression suite using Behavior Driven Testing (BDT)
3 likes861 views
Slides from the workshop conducted in ThoughtWorks, vodQA Gurgaon on - "What is BDT, and how can you use this technique to identify the 'right' regression suite for your product"
More Related Content
What's hot (18)
Viewers also liked (13)
Similar to Build the "right" regression suite using Behavior Driven Testing (BDT) (20)
More from Anand Bagmar (20)
Recently uploaded (20)
Build the "right" regression suite using Behavior Driven Testing (BDT)
- 1. BUILD THE “RIGHT” REGRESSION SUITE USING BEHAVIOR DRIVEN TESTING (BDT) vodQA
- 3. WHAT DO YOU EXPECT FROM THIS SESSION?
- 4. CASE STUDY • Multiple, long running projects • Legacy applications • Integrated
- 5. • Limited / flaky / long-running Automation • Long regression cycle • Huge cost of fixing defects FUNCTIONAL TESTING CHALLENGES
- 6. • Brittle • Little / less value • No visibility into what is tested • Maintenance nightmare • Outdated • Expensive AS A RESULT
- 7. • Trust deficit on the team • In existing Automation • In manual testing • Finger pointing / blame game BIGGEST PROBLEM
- 10. Concept #1 Given -> When -> Then
- 11. Expected Functionality: The customer should be able to withdraw money from his account via an ATM machine
- 12. BDD Example: Given the account has balance of 5000 When the customer requests 1000 Then the account is debited by 1000
- 13. 13
- 14. Workshop Begins – Iteration 1 Case Study
- 16. PHASE 1 REQUIREMENTS • Guest User is able to search for a flight for a single traveller • Select option from Search Results page • Specify contact information for person booking the flight
- 17. A Guest User can … WORKFLOW Search Search Results Contact Information
- 18. SEARCH MODULE
- 19. SEARCH RESULTS
- 20. CONTACT DETAILS
- 21. CONTACT DETAILS
- 22. PHASE 1 REQUIREMENTS • Guest User is able to search for a flight for a single traveller • Select option from Search Results page • Specify contact information for person booking the flight
- 23. A Guest User can … WORKFLOW Search Search Results Contact Information
- 24. LETS GET STARTED … WRITE 2 TEST SCENARIOS
- 26. Concept #2 Test Specification Styles
- 27. • Imperative • Declarative TEST SPECIFICATION STYLES
- 28. A Guest User can … AN EXAMPLE Search Search Results Contact Information
- 29. IMPERATIVE STYLE Given I am a guest user on the home page And I choose “round” trip option And I select “Chicago” from the origin dropdown And I select “San Francisco” from the destination dropdown And I select departure date as “5 December 2013” And I select returning date as “25 December 2013” When I click on Search Then I should see the search results page And I should see at least 1 option for my criteria ...
- 30. IMPERATIVE STYLE … … When I select the “first” option Then I am on the Contact Information page When I enter first name as “Anand” And I enter last name as “Bagmar” … … And I click the “Select and Continue” button Then I should be on the next page
- 31. DECLARATIVE STYLE Given I am a guest user When I search for flight options for a “one-way” trip for “1” “Adult” from “Chicago” to “San Francisco” And I select the “first” flight And I enter “valid” contact details for “traveller1” Then I am able to Save and Continue
- 32. • Ben Mabey - http://benmabey.com/2008/05/19/ imperative-vs-declarative-scenarios-in- user-stories.html • Alister Scott - http://watirmelon.com/2010/12/10/ cucumber-imperative-or-declarative-that- is-the-question/ REFERENCES
- 33. Workshop – Iteration 2 Case Study Continues
- 34. • User registration from Contact information page • Registered user can login from home page (below search criteria) • For logged-in user, when booking ticket, contact information is pre-filled with logged-in user information PHASE 2 REQUIREMENTS
- 37. REGISTERED USER
- 39. • User registration from Contact information page • Registered user can login from home page (below search criteria) • For logged-in user, when booking ticket, contact information is pre-filled with logged-in user information PHASE 2 REQUIREMENTS
- 41. LETS GET STARTED … WRITE 2 TEST SCENARIOS
- 43. WHY IS THIS IMPORTANT?
- 44. Concept #3 The Test Pyramid
- 45. AUTOMATION – A SAFETY NET
- 46. • “Succeeding with Agile” – Mike Cohn • Martin Fowler – Test Pyramid martinfowler.com/bliki/TestPyramid.html THE TEST PYRAMID
- 47. Time Value Cost IDEAL TEST AUTOMATION PYRAMID Manual / Exploratory UI Web Service View JavaScript Integration Unit Technology- facing Tests Business-facing Tests
- 48. REALITY
- 49. UI Web Service View JavaScript Integration Unit Manual / Exploratory ICE CREAM CONE ANTI PATTERN
- 50. Developers Test Pyramid QA Team Test Pyramid Web Service View JavaScript Integration Unit Manual / Exploratory UI Web Service DUAL TEST PYRAMID ANTI-PATTERN
- 51. Concept #4 Behavior Driven Testing (BDT)
- 52. BEHAVIOR DRIVEN TESTING (BDT)
- 53. • How is this functionality going to be used by the end-user? • What is their thought process going to be when using this product? • What “core-business-value” should be delivered to the end-user? THE BIG PICTURE
- 54. Manual / Exploratory UI Web Service View JavaScript Integration Unit Technology- facing Tests Business-facing Tests Identify the “right” type of tests that sit on top of the Test Pyramid
- 55. My Case Study
- 57. HOW TO SOLVE THE PROBLEM?
- 58. • Remove the ambiguity • Same visibility for all SOLUTION
- 59. • Whiteboards • Mind maps • Flow charts • Collaboration!! TOOLS TO ENABLE BDT
- 60. HOW DID WE ACCOMPLISH THIS?
- 62. • Identify business flows based on personas, and how the personas use the business functionality, or, • Identify business flows, and how different user personas can use the business flows NOTE: Business flows are different from UI flows HOW TO IDENTIFY BUSINESS FLOWS ?
- 64. 1st candidate • Automate! • High Value, Low Cost 2nd candidate • Automate! • High Value, High Cost Can Automate? • Manual Regression? • Low Value, Low Cost Do Not Automate • Manual Regression! • Low Value, High Cost High Value High Cost Low Value Low Cost
- 65. AUTOMATE THE TOP LAYER USING • BDD Tools, or, • Directly in any appropriate programming language / tools
- 66. • Identification of Tests • Automate it! WHAT’S NEXT?
- 67. AUTOMATE TESTS THAT ARE VALUABLE, NOT BECAUSE YOU CAN OR LIKE TO! CAUTION!
- 68. Who is doing the action? What is the business functionality? Depicts a user flow / user journey Actions & Verifications happen implicitly Given auctioneer creates an auction And 5 vehicles are added to the auction When auctioneer starts the auction And auctioneer sets a starting bid of 5000 And buyer_1 bids And buyer_2 bids Then auctioneer Sells the vehicle And buyer_2 wins the vehicle EXAMPLE
- 70. 1st candidate • Automate! • High Value, Low Cost 2nd candidate • Automate! • High Value, High Cost Can Automate? • Manual Regression? • Low Value, Low Cost Do Not Automate • Manual Regression! • Low Value, High Cost High Value High Cost Low Value Low Cost
- 71. Project Management tool Mingle Test case repository (manual + automated) Cucumber .feature files with appropriate tagging Feature files / Scenarios 33 / 65 # of Manual Tests 0 Tests running on CI (Jenkins) Since the 1st test was automated Smoke tests execution time 15 minutes Regression tests execution time 45 minutes FINAL STATE … IN MY CASE STUDY
- 72. • Builds and deployments were automated • Tests ran on demand, automatically! • Smoke / Regression / Javascript / Unit • Functional Performance • Concurrent Testing ADDITIONALLY …
- 73. How did this Technique (BDT) become successful?
- 74. CONTEXT IS KING! Domain Distributed Teams Skills Capabilities Product-under test Tech Stack Process Tools
- 75. NOT work in isolation BDT WILL …
- 76. BDT REQUIRES … Lower layers of Test Pyramid to be well automated!
- 77. BDT REQUIRES … Dedicated and Focused testing (manual / exploratory)
- 79. BDT helps identify the “right” type of regression tests!
- 80. Executable Specs BDT Scenarios Business Idea Features Epics Stories Iterations Releases BDT VALUE PROPOSITION
- 81. BDT VALUE PROPOSITION … Incrementally build Business Flows Regress Business Value Test in-sync with Product Functionality Effective Tests
- 82. BDT VALUE PROPOSITION … Remove ambiguity Understand new functionality Living Documentation Onboarding