Ad
More Related Content
What's hot (20)
Similar to Cucumber spec - a tool takes your bdd to the next level (20)
Ad
More from nextbuild (13)
Ad
Recently uploaded (20)
Cucumber spec - a tool takes your bdd to the next level
- 1. CUCUMBERSPEC A TOOL TAKES YOUR BDD TO THE NEXT LEVEL 30-5-2015
- 2. PROJECT TEAM 2 • Adrian Yankov – Graduation Intern • Kyra Hameleers – Test Engineer @ Nspyre • Rachid Kherrazi – MBT competence Leader @ Nspyre – Project Leader of this project
- 3. GOALS • Sharing knowledge on 3 topics – Behaviour-Driven Testing – Model-Based Testing – Combined strengths • Encourage the usage of Model-Based Testing
- 4. CONTENT • Introduction – Behaviour-Driven Testing(BDT) • What is it and why? – Model-Based Testing(MBT) • What is it and why? – Using Gherkin in MBT • CucumberSpec Tool • Testing a drone with this approach 4
- 5. BEHAVIOUR-DRIVEN TESTING • An example 5
- 6. BDT LANGUAGE AND BENEFITS Why? • Main benefit – the team gets together and writes down a common, shared knowledge in a well-known format. • Generates test cases from a high-level formal human language. Tools • There are several tools such as SpecFlow, Behave and Cucumber etc. 6
- 8. SOME NICE FEATURES 8 • Data Parameterization – Supports placeholders. – Contained within < > Example:
- 9. BDT WITH SPECFLOW 9 GUI Application/ database layer Testinterface Physical test cases Logical test cases (Gherk in) Requirements Features Use cases Test Engineer Architect/ designer SUT create
- 10. MODEL-BASED TESTING Manual Automatic Testcase specification (design van logische test cases) Testcase execution (includes reports) Testcase generation Modelbasedtesting Traditionalmanualtesting Traditionalautomatictesting Manual C# Scripting Testing skills + Scripting skills + Modeling skills Modeling Lightweight Modeling languages Generate Scripting 10
- 11. LOOK AND FEELING (SPEC EXPLORER) 11
- 12. MBT WITH SPEC EXPLORER 12 GUI Application/ database layer Testinterface Automatic generation of test scripts Test Model Model includes expected behavior Requirements Features Use cases Gherkin Test Engineer Architect/ designer SUT create create
- 13. BDT IN COMBINATION WITH MBT Our tool CucumberSpec generates a SpecExplorer test model from the separate scenarios. 13 CucumberSpecIsolated scenarios
- 14. USING GHERKIN IN MBT Benefits Visualization of interaction space, generate more and better (exhaustive) testcases by using different smart exploration strategies to suit the tester’s needs. 14 GUI Application/ database layer Testinterface Automatic generation of test scripts Test Model Model includes expected behavior Requirements Features Use cases Gherkin Test Engineer Architect/ designer SUT CucumberSpec check create
- 15. CUCUMBERSPEC SUMMARY Idea of our CucumberSpec • Generates a Test model from a feature file, Connecting the separate scenarios in a model • Intention is to support most features (parameterization, tags….)supported by BDD tools like Behave and SpecFLow • + A lot of “Model Based” Benefits, • Generate more tests cases based on different coverage used in MBT – Testing interaction between the scenarios – More data permutations and combinations – This allows exhaustive testing • No refinement is needed. • Fully automatic process. 16
- 17. AR.DRONE 18 Mobile AR.Drone SW Hardware Router Camera 1 Motors Camera 2 Hoogte Sensor GPS Application UI Controller Communication Business Layer Data Acces Layer Commun ication Layer
- 18. REQUIREMENTS • Functional requirements: 1. Takeoff and Land via Android commands 2. Fly around via Android commands 3. Always be able to use the emergency command • Non-Functional requirements: 1. Usability 2. Reliability 3. Security 19 To keep it simple in this PoC, we focused only on a limited number of functional requirements.
- 19. TEST SCOPE • Functional requirements 1. Takeoff and Land via Android commands 1. Takeoff 2. Land 2. Fly around via Android commands 1. Left/Right 2. Up/Down 3. Forward/Backward 4. Turn Lef/right 3. Always be able to use the emergency command 20
- 20. TESTING THE DRONE 21 Actions: 1. Take off 2. Emergency 3. Land 4. R: Right 5. L: Left 6. F: Forward 7. B: Backward 8. l: Turn left 9. r: Turn right 10. u: Up 11. d: Down
- 21. TEST MODEL States: 1. Take off 2. Flying 3. Landing 4. Landed 5. Emergency 22 Actions: 1. Take off 2. Emergency 3. Land 4. R: Right 5. L: Left 6. F: Forward 7. B: Backward 8. l: Turn left 9. r: Turn right 10. u: Up 11. d: Down Take off Flying Landing Landed Emergency Take off x Reached 30 cm land X Emergency Flying x Right/ Left/ Up/ Down/ Forward/ Backward/ Turn left/ Turn right land x Emergency Landing take off, height<0 take off, height>0 X Reached 0 cm Emergency Landed take off X X x Emergency Emergency x Emergency, height>0 X Emergency, height<0 x Some issues with traditional test design: Traditional state transition test technique does not scale high coverage (>1-switch) is complex, costly and difficult to maintain
- 22. CREATE GHERKIN (SMALL EXAMPLE FOR EXPLANATION PURPOSES) 23 0-switch state transition test cases in a high-level formal human language(Gherkin). Landed Flying Landed x Take off Flying Land Forward/ Backward
- 23. GHERKIN SCENARIOS (SMALL EXAMPLE FOR EXPLANATION PURPOSES) 24 Isolated scenarios
- 24. GENERATED SPEC EXPLORER (SMALL EXAMPLE FOR EXPLANATION PURPOSES) 25 Visualization of interaction space, This allows generation of more and better (exhaustive) testcases by using different smart exploration strategies (supported by the MBT tool) to suit the tester’s needs.
- 25. COMPARISON (SMALL EXAMPLE FOR EXPLANATION PURPOSES) 26 Isolated scenarios model CucumberSpec
- 26. BACK TO TEST SCOPE 27 Take off Flying Landing Landed Emergency Take off x Reached 30 cm land X Emergency Flying x Right/ Left/ Up/ Down/ Forward/ Backward/ Turn left/ Turn right land x Emergency Landing take off, height<0 take off, height>0 X Reached 0 cm Emergency Landed take off X X x Emergency Emergency x Emergency, height>0 X Emergency, height<0 x
- 29. TEST CASES 30
- 30. TEST EXECUTION • Test code can be generated • The Tool can generate test scripts for different execution tools e.g NUnit/JUnit. • No code to test it too 31
- 31. CONCLUSION 33 Metrices Cucumber MBT CucumberSpec Approach using gherkin files and action definition to generate and execute test suite using models to generate and execute test suite using gherkin files to generate models, and generate and execute test suite Technique Behavior Driven Testing Model Based Testing Combining strength of BDD and MBT Modeling Create gherkin files Create test model Generate test model Effort High Medium/High Medium Coverage 0-switch Transition coverage Transition coverage Number of testcases 20 7 7
- 32. NEXT STEP • Next step: Explore automatic test execution via Xamarin/Calabash (benefit cross-platform testing) • We are looking for a pilot project based on real industry case from industry • If you are interested please contact us • or send a mail to [email protected] 34
- 33. SUMMARY • Combining the best of 2 world – Behavior-Driven Testing – Model-Based Testing 35 CucumberSpecIsolated scenarios
- 34. 36