Testing katas - Try before you buy
Download as PPTX, PDF•2 likes•1,096 views
The document discusses a session on testing katas, emphasizing their role in improving testing skills through repeated practical exercises in a collaborative environment. It covers key concepts such as oracles and heuristics and provides a hands-on activity involving a calculator web application to identify inconsistencies. The session encourages knowledge sharing and aligns katas with other educational activities to enhance learning opportunities.
Testing katas - Try before you buy
- 1. Testing Katas: Try Before you Buy Emma Armstrong @EmmaATester www.taooftesting.co.uk Jonathan Watts @whatie
- 2. Today’s Session • What’s this session all about? • What are katas and why use them for testing? • Some background on using oracles for testing • The kata • Introduce the objective • Testing the application • Debrief session • Just ask questions as we go through
- 3. What are katas? • Repeated practical exercises to ingrain behaviours until they become instinctual • Allow you to cover various topics and situations in a safe learning environment • Encourage knowledge sharing and collaboration
- 4. Why use katas for testing? • Deliberate practice • Learning and development – Tacit skills • Consistency • Knowledge sharing • Quality is everyone’s responsibility
- 5. Let’s try it then… • Introduce the topic • Get into groups • Test the application as a group • Report back on your findings • Group discussion
- 6. Heuristics and Oracles A heuristic is a mechanism to try and solve a problem or learn something Testing examples include: • Brute force • Testing at the boundaries • Input methods • Starvation • Working backwards • Common sense / Educated guess • Rule of thumb
- 7. Heuristics and Oracles “An oracle is a heuristic principle or mechanism by which someone recognizes a problem.” Michael Bolton, 2010 Examples: • Requirements • Marketing claims • Laws and standards • Scientific models • Previous versions • Maths • Comparable products • Experts • Your experience When do you use this? • Bug evaluation • Reporting • Test design They are fallible
- 8. The kata - The consistency heuristic “We expect the system to be consistent with systems that are in some way comparable. This includes other products in the same product line; competitive products, services, or systems; or products that are not in the same category but which process the same data; or alternative processes or algorithms.” Michael Bolton Examine the calculator web application from a black box perspective using the consistency heuristic and document your findings. For example, compare the behaviour our calculator to another equivalent application and see where the inconsistencies are. http://bit.ly/1v4Nw6R (http://emmaarmstrong.github.io/CalculatorJavaScript/Calc.html)
- 9. Debrief • What did you find? • How did you go about finding them?
- 10. Where do we go from here? • There is also a development opportunity for those people running the katas • We are aligning them with other educational activities at Red Gate • We are sharing this talk and the katas we have run with anyone who wants them • Come and join in our katas
- 11. Questions? The calculator https://github.com/EmmaArmstrong/CalculatorJavaScript Code katas http://codekata.com/ http://codingkata.net/ http://dev.red-gate.com/category/blog/code-katas/ Test katas http://www.soapui.org/Testing-Katas/what-are-testing-katas.html http://testing-challenges.org/tiki-index.php http://weekendtesting.com/ Test heuristics http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf Thank you for your time
Editor's Notes
- #2: About Emma Armstrong Jonathan Watts Red Gate
- #4: What are katas? Kata is a Japanese word describing exercises that were done repeatedly until they became instinctual. Dave Thomas, the co author of The pragmatic programmer first coined the term in respect of code katas and testing katas are no different. We are talking about exercises done in groups, pairs or even individually that allow you to practice something and then learn from others experiences through discussion. We are not suggesting the same exercise done again but different exercises or the same exercise with different constraints. We believe that the forms of katas can be used to help learn any skill but especially testing, where testing should be happening at all stages in the project and it should be instinctual, testers should know what to do, and adapt their approach based on the context they are in.
- #5: Why do katas? As we know, complete testing is rarely possible especially if you have over about 2 lines of code. This can mean the testing tasks at hand can seem infinite and so katas provide a way of enabling deliberate practice. Maintaining product quality across multiple teams. We work in cross functional teams, which means some of the knowledge transfer of skills in testing that might have occurred due to being immersed in a testing team does not, so to try and develop expertise we need to facilitate this (instructional expertise) in a different way. If you google Agile and quality, the phrase “Quality is everyone’s responsibility” will come up a few times. We believe that to achieve this you need to expose people of other disciplines to the activities that testers perform. The testing katas that we run are open to the whole company. Format – At Red Gate we have deliberately chosen to keep the exercises down to only a 20-30minute exercise and then a 30minute -1hr meeting afterwards and only do 1 kata a month. We have also run 1hr workshops.
- #6: In today’s kata we are not going to give you any specifications we want you to apply the consistency oracle, specifically looking at the consistency with a comparable product heuristic. So we are focussing on using the behaviour of a comparable product to help evaluate the behaviour of the product under test.
- #7: There are several oracles that you will already be using daily even if you are not calling them out as such. As Cem Kaner says, an oracle is a mechanism for determining whether a program has passed or failed a test. The oracle that you use depends on several things, the testing that you are doing, the environment you are in, the application you are testing, the context of the testing for example it could be a specification, it can be laws/statutes, other products, you, etc.. Specs being wrong Consistency with ‘something’ is an oracle we often rely on. Specifications Laws Previous versions Comparable products Marketing claims
- #8: There are several oracles that you will already be using daily even if you are not calling them out as such. As Cem Kaner says, an oracle is a mechanism for determining whether a program has passed or failed a test. The oracle that you use depends on several things, the testing that you are doing, the environment you are in, the application you are testing, the context of the testing for example it could be a specification, it can be laws/statutes, other products, you, etc.. Specs being wrong Consistency with ‘something’ is an oracle we often rely on.
- #11: Don’t do a kata on something you haven’t tried out yourself a few times Try to keep the examples simple Keep a note of examples Need a backlog Emma and Jonathan have gained some teaching experience Improved communication 2 new testers Improved aware of testing activities and methodologies We have learnt new things Continue to do our own deliberate practice? Would you like to use our previous katas? Coding katas talk