![Be happy - do good - leave the world a better place than you found it
Isabel
Evans
References (1)
[1] Lehner A, Dumke R., Software Metrics: Research & practice in software measurement, Available at: http://bit.ly/2kSZrs2, Accessed: 3 Feb 2017).
[2] Tulpule, N, (2016), Developer Experience, FWT!, Conference keynote presentation, Google Test Automation Conference, GTAC 2016 (Isabel attended)
[3] Havelock, P, Testware or Shelfware - the reality, Article on BCS website, available at http://bit.ly/2lBmxmF (Accessed 11 Feb 2017)
[4] Kaner, C, (1998), Avoiding Shelfware: A Managers’ View of Automated GUI Testing, Paper STAR98 conference, Available at http://bit.ly/2kSMyOI (Accessed 11 Feb 2017)
[5] Fewster, M. and Graham, D. (1999) Software test automation: Effective use of test execution tools. Harlow: Addison-Wesley Educational Publishers.
[6] Graham, D. and Fewster, M. (2012) Experiences of test automation: Case studies of software test automation. United States: Addison-Wesley Educational Publishers.
[7] Tassey, G. (2003) The economic impacts of inadequate infrastructure for software testing: Final report. Diane Pub Co. Available at http://bit.ly/2kT52OP
[8] Jones, C. (2015) Wastage: The Impact of Poor Quality on Software Economics, Software Quality Professional Journal, (article) Dec2015, Vol. 18 Issue 1
[9] IEEE, (2014)IEEE 730-2014 IEEE Standard for Software Quality Assurance Processes, 3.2
[10] Nielsen, J and Norman, D, The definition of user experience, Nielsen Norman NN Group Website, available at http://bit.ly/1T72ZR5 {accessed 11 Feb 2017}
[11] Hassenzahl, M., The encyclopedia of human computer interaction, Interaction Design Foundation, chapter 3 “User Experience and Experience Design”. Available at http://bit.ly/2kZKI0W (accessed 11 Feb 2017)
[12] Atoum, I., Bong, C., Kulathuramaiyer, N., (2013) “Towards Resolving Software Quality-in-Use Measurement Challenges”, University of Malaysia Sarawak
[13] Zubrow, D., Software Quality Requirements and Evaluation, the ISO 25000 Series, Carnegie Mellon University, Report of standards working party
[14] Richardson, A and Evans, I, Conversation: Automation and People, the UKSTAR 2017 programme includes a debate on technical skills required by testers (http://bit.ly/2kZNzGY, see track T10) (conference debate)
[15] Whittle, J., Hutchinson, J., Rouncefield, M., Burden, H., and Heldan, R., (2015) A Taxonomy of Tool-Related Issues Affecting the Adoption of Model-Driven Engineering, Software and Systems Modeling. ISSN 1619-1366
[http://eprints.lancs.ac.uk/77975/]
[16] Johnson, B., Song, Y., and Murphy-Hill, E., Bowdidge, R, (2013) Why Don’t Software Developers Use Static Analysis Tools to Find Bugs?, NCSU { https://people.engr.ncsu.edu/ermurph3/papers/icse13b.pdf} (accessed 11 Feb
2017)
[17] CA Consulting, (2016) “Moving IT Service Management to the 21st Century”, available at http://bit.ly/2kggqlX
[18] Chemaly, S, (07/27/2015 03:52 pm ET | Updated Jul 27, 2016) What gender stereotypes and sexism have to do with Algorithms and Robots, Huffington Post, Available at http://huff.to/1I5pek4 (accessed 11 Feb 2017)
[19] Lent, Knowledge, skills and abilities for software QA jobs, available at http://bit.ly/1fJ0iXq (accessed 11 Feb 2017, no date on website)
[20] Own experiences, anecdotal evidence, emails, conference discussions & feedback (UCAAT, PSTQB, EuroSTAR16, CREST workshop, UCL, UK Software testing retreat)](https://image.slidesharecdn.com/qualityinuse45minpresentation16-9slidesharenokiatestdive2017-171031080554/85/Quality-in-use-45-min-presentation-16-9-slideshare-nokia-test-dive-2017-49-320.jpg)
![Be happy - do good - leave the world a better place than you found it
Isabel
Evans
References (2)
[21] Smartbear tools vendor, available at https://smartbear.com/learn/automated-testing/ (accessed 11 Feb 2017)
[22] Johnson, B., Pandita, R., Smith, J., Ford, D., Elder, S., Murphy-Hill, E., Heckman, S., Sadowski, C., (2016) A Cross-Tool Communication Study on Program Analysis Tool Notifications, ACM SIGSOFT International Symposium on
the Foundations of Software Engineering (FSE), [http://bit.ly/2kT4EQt] (accessed 11 Feb 2017)
[23] Moore, G., Crossing the Chasm 2nd Edition (1999) Capstone
[24] Hendrikson, E., Do Testers Have to Write Code? Blog, [Available at http://bit.ly/2lzuMij] (accessed 11 Feb 2017)
[25] Lambert, R, Why testers really should learn to code, Blog, [available at http://bit.ly/2kZNW4k ] (accessed 11 Feb 2017)
[26] Gill, M., Software Testing Skills that you must have: An overview, Blog, [available at http://bit.ly/2kZLmLM] {accessed 11 Feb 2017)
[27] Beckwith, L., Kissinger, C., Burnett, M., Weidenbeck, S., Lawrance, J., Blackwell, A., Cook, C., (2006) Tinkering and Gender in End-user Programmers’ Debugging, [available at http://bit.ly/2hPq0iI], (accessed 11 Feb 2017)
[28] Morrison, P., Pandita, R., Murphy-Hill, E., and McLaughlin, A., (2016), Veteran Developers’ Contributions and Motivations: An Open Source Perspective, [Available at http://bit.ly/2ke1wSe] (accessed 11 Feb 2017)
[29] Fisher, M., Cao, M., Rothermel, G., Brown, D., Cook, C., Burnett, M., Integrating Automated Test Generation into the WYSIWYT Spreadsheet Testing Methodology, [available at
http://cse.unl.edu/~grother/papers/tosem06.pdf], (accessed 11 Feb 2017)
[30] Schroeder, K. (2010), Gender dimensions of Product Design, Expert Paper for UNDAW / UNESCO, [online] [available at http://bit.ly/1MXNTPf] (accessed 11 Feb 2017)
[31] Pauleen, D., Evaristo, R., Davison, R., Ang, S., Alanis, M., and Klein, S. (2005) Cultural Bias in Information Systems Research and Practice: are you coming from the same place I am?, ICIS Panel discussion summary [online
http://bit.ly/2bfI5lY] (Accessed 11 Feb 2017)
[32] Isabel Evans’ Conversations with attendees at the Women in Innovation (WII) meeting. (2016). Quotes: UK Government Infocus/Innovate UK awards Women in Innovation 2016, see http://bit.ly/1P05CBG [accessed 7 Feb
2017]
[33] Nielsen, J. (2017). Ten Usability Heuristics. [online] [Available at: https://www.nngroup.com/articles/ten-usability-heuristics/] [Accessed 4 Feb. 2017].
[34] Firestone, D. (2014). Common System and Software Testing Pitfalls,e-book, Pearson Education Available at Amazon.co.uk
[35] Borg, A; Porter, C; Micallef, M. (2015). Poster: Is Carmen better than George? Testing the Exploratory Tester using HCI Techniques, International Conference on Software Engineering, to appear, 2015, Florence, Italy
[36] Micallef, M., Porter, C. (2016). HCI – The Tester’s New Sidekick?, The Tester Magazine – BCS 03/2016 (pp 12 -15)
[37] Micallef, M., Porter, C., Borg, A. (2016). Do Exploratory Testers Need Formal Training? An Investigation Using HCI Techniques., The 11th Workshop on Testing: Academia-Industry Collaboration, Practice and Research
Techniques, April 11, 2016, Chicago, IL, USA
[38] Micallef, M., Porter, C. (2017). Help! I’m only human! Understanding and supporting the human tester, Keynote at the British Computer Society SIGIST Testing Conference, March 14, 2017, London, UK](https://image.slidesharecdn.com/qualityinuse45minpresentation16-9slidesharenokiatestdive2017-171031080554/85/Quality-in-use-45-min-presentation-16-9-slideshare-nokia-test-dive-2017-50-320.jpg)
![Be happy - do good - leave the world a better place than you found it
Isabel
Evans
References (3)[39] Constructing Grounded Theory, Kathy Charmez
[40] Quality in Use: Meeting User Needs for Quality, Nigel Bevan, Serco Usability Services Journal of System and Software, 1999 (in press)
[41] “The test tool marked all the tests as passed except 1, but in fact none of the tests marked “passed” had actually run” Quote from Fewster and Graham “Experiences of Test Automation”
[42] Quotes from attendees at the WII briefing meeting: “I spend 50% of my time wrestling with the technology instead of solving the problem I am working on” and “it is like working with something designed to be used by a
12-year-old boy in his bedroom in the 1980’s”. “Why would I want to use a tool called Github?”
[43] Gender and other bias in IT tools, for example in tool default behaviour, use of language, voice recognition, gendering of support tools, is increasingly being studied and remarked on e.g. http://huff.to/1I5pek4,
http://bit.ly/1MXNTPf , http://bit.ly/2bfI5lY.
[44] Problems with customer support tools: “Ethnographic research paints a sad picture of the current state of the ITSM market. …vision is to build a solution designed for humans, not processes”
[http://blogs.ca.com/2016/01/27/moving-itservice-management-to-the-21st-century/]
[45] Evidence that tools do not work for IT people “…a lack of consideration for how people work and think … basically it’s still the mindset that the human adapts to the computer, not vice-versa.” (A Taxonomy of Tool-Related
Issues Affecting the Adoption of Model-Driven Engineering by Whittle, Hutchinson, Rouncefield, Burden and Heldal)
[46] Evidence that developers do not find tools easy to use: “…so now I wanna know why raising a string exception is bad. Like what should I be doing instead? Since it thinks it’s a problem. And so none of these really help
me…” (Why Don’t Software Developers Use Static Analysis Tools to Find Bugs? By Johnson, Song, and Murphy-Hill).
ISO 25000 Series of standards (quality in use attributes)
[47] http://alarmingdevelopment.org/
[48] http://blogs.ca.com/2016/01/27/moving-it-service-management-to-the-21st-century/
[49] https://www.youtube.com/watch?v=oQ455i1aCQI
[50] https://www.nngroup.com/articles/ten-usability-heuristics/
[51] http://research.csc.ncsu.edu/dlf/](https://image.slidesharecdn.com/qualityinuse45minpresentation16-9slidesharenokiatestdive2017-171031080554/85/Quality-in-use-45-min-presentation-16-9-slideshare-nokia-test-dive-2017-51-320.jpg)
Quality in use 45 min presentation 16 9 slideshare nokia test dive 2017
- 1. Be happy - do good - leave the world a better place than you found it Isabel Evans Nokia TestDive 2017 Krakow Quality in Use: the beating heart of the user experience Isabel Evans fbcs citp [email protected] www.isabelevans.uk
- 2. Be happy - do good - leave the world a better place than you found it Isabel Evans What do you test with? Head
- 3. Be happy - do good - leave the world a better place than you found it Isabel Evans WHY IS IT IMPORTANT TO CONSIDER THE USER EXPERIENCE? The user has a choice – always… stay go
- 4. Be happy - do good - leave the world a better place than you found it Isabel Evans Making life better example Cyclist’s UX
- 5. Be happy - do good - leave the world a better place than you found it Isabel Evans X: Ignorance A: Uncertainty B: Awakening C: Enlightenment D: Wisdom E: Certainty We know why we do not have UX problems It’s the stupid users! Why are our users having a problem? Do they always have to have a problem? We are identifying and resolving UX problems so our users don’t experience them We do UX defect preventionUX Maturity Based on Bevan
- 6. Be happy - do good - leave the world a better place than you found it Isabel Evans Functionality Performance Security Maintainability Interoperability Portability Compatibility Etc. Usability Flexibility Safety Accessibility Etc. Seductiveness Excitement Flow Trust Usefulness Etc. Internal quality Quality in Use UX-D
- 7. Be happy - do good - leave the world a better place than you found it Isabel Evans UX – the business imperative “In today’s business environment, the user experience and the commercial imperatives have become overwhelmingly important. It is vital that we understand quality in use and the user experience, in order that we focus projects correctly.” NB: It’s not just that we need to provide a good experience – it is also that we need to provide excellent recovery from bad experiences
- 8. Be happy - do good - leave the world a better place than you found it Isabel Evans Our world or our users’ world? “I regret to inform you that you are amongst a small percentage of people that were affected by a problem during the upgrade of our queuing infrastructure.” Written in IQ terms when it needs to be in QiU or UX terms; Meant nothing to the person who received it
- 9. Be happy - do good - leave the world a better place than you found it Isabel Evans The Apple lover’s choice? “It was so beautiful I wanted to marry it … And then it lost my calendar entries and emails”
- 10. Be happy - do good - leave the world a better place than you found it Isabel Evans The silver emailer’s choice? “…library is open again and – to my horror - have … up dated their computers. So everything is just slightly different which the old girl is finding very confusing !!!!!!!!!!!!! Am not sure what the improvement has been but it has left me struggling a wee bit…”
- 11. Be happy - do good - leave the world a better place than you found it Isabel Evans AND THE USER IS NOT JUST THE END-USER… The user has a choice – always… stay go
- 12. Be happy - do good - leave the world a better place than you found it Isabel Evans The consultant doctor’s choice? “This is the clunkiest piece of technology I have ever had to use …. I will have to ask my secretary to print your test results”
- 13. Be happy - do good - leave the world a better place than you found it Isabel Evans The hill farmer’s choice? In the U.K. one farmer a week commits suicide “… farmers … took their lives because the person who used to complete paperwork, … livestock passports or single farm payment applications, had died or moved away.” A farmer, quoted about subsidy forms: “I usually leave it to the last minute with the paper forms because it only takes about three hours to do, but so far I have spent three days doing the online version and I am only halfway through it.” “another eye-watering sum has been paid to technology suppliers and consultants: £154m.”
- 14. Be happy - do good - leave the world a better place than you found it Isabel Evans 1987: The manual workers’ choice? Manual workers: Expect to be paid on Friday at lunchtime in cash Work remotely from headquarters IT Staff: Monthly salary into bank accounts Performance of the payroll batch – did it matter? Does it matter in 2017?
- 15. Be happy - do good - leave the world a better place than you found it Isabel Evans
- 16. Be happy - do good - leave the world a better place than you found it Isabel Evans We are all connected…? “Of the 49.4 million adults living within Great Britain, 1.1% have poor internet access This equates to 530,000 adults with limited internet speed, of which 330,000 say they would be willing to bank online These 530,000 adults have: – Average download speeds of 1.4Mbit/s – 15 times slower than average. – Superfast broadband is available to just 3% of these people, but uptake is low. – Mobile provides an alternative for 470,000 of the 530,000 people (88%). – However, 63,000 people in parts of Wales and Scotland, and small pockets of England, still have no acceptable digital access at all (63,000 adults in total).”
- 17. Be happy - do good - leave the world a better place than you found it Isabel Evans Think about the whole system • Not just the software – But the whole user experience • The user is – not just the person who uses the software – the person who uses or is affected by the service • IT -> employee – -> customer- > family ->…
- 18. Be happy - do good - leave the world a better place than you found it Isabel Evans UX: failure or success? • Where the users have a choice, and meet an unfortunate experience, you lose a customer: • they will go elsewhere. • Where the users have no choice, and meet an unfortunate experience, the affects can be devastating: • frustration, loss of productivity • loss of motivation and morale • rebellion and aggression • even death.
- 19. Be happy - do good - leave the world a better place than you found it Isabel Evans Test with more than our heads… Head Heart Gut Soul
- 20. Be happy - do good - leave the world a better place than you found it Isabel Evans HOW DO WE UNDERSTAND THE USER EXPERIENCE? The user has a choice – always… stay go
- 21. Be happy - do good - leave the world a better place than you found it Isabel Evans Start to think UX Identify WHO uses your product / service Identify WHY they use it Identify WHAT they want to do Identify their EMOTIONS and EMPATHISE! Score the UX, Quality in Use (QiU) and Internal Quality (IQ) attributes Write RICH STORIES Do INTERACTION DESIGN Do a USABILITY REVIEW and OBSERVATIONS
- 22. Be happy - do good - leave the world a better place than you found it Isabel Evans Empathy and thought experiments • You need questions – Who? Why? Where? When? How? – What mood? What emotion? • You don’t need a specification, user stories, or the software • You just need intelligence and empathy
- 23. Be happy - do good - leave the world a better place than you found it Isabel Evans Identify WHO uses your product / service Identify WHY they use it Identify WHAT they want to do Identify their EMOTIONS and EMPATHISE!
- 24. Be happy - do good - leave the world a better place than you found it Isabel Evans The Apple lover “It was so beautiful I wanted to marry it And then it lost my calendar entries and emails” Social and demographic characteristics. Needs, desires, goals: on trend, connected, finish tasks fast Habits (consumer habits, behaviour): Apple, impulse buyer Expertise: tech savvy, but not an IT specialist, relies on being connected Cultural background: interest in the Arts, Media Motivations: keeping connected (work and social) Product Must do: on demand, transparent updates Product Must never: lose data, stall/halt User experience goals: exciting, seductive, flowing, desirable, flexible
- 25. Be happy - do good - leave the world a better place than you found it Isabel Evans The silver emailer “…library is open again and – to my horror - have … up dated their computers. So everything is just slightly different which the old girl is finding very confusing !!!!!!!!!!!!! Am not sure what the improvement has been but it has left me struggling a wee bit…” Social and demographic characteristics. Needs, desires, goals: keep in touch, link to “paper ways” Habits (consumer habits, behaviour): buys little, whole food, ecology Expertise: low IT expertise Cultural background: politics, world news, dance Motivations: keeping connected (family, friends and world news) Product Must do: on demand, real-world-like Product Must never: behave unexpectedly User experience goals: trust, simplicity, consistency
- 26. Be happy - do good - leave the world a better place than you found it Isabel Evans The consultant “This is the clunkiest piece of technology I have ever had to use I will have to ask my secretary to print your test results” Social and demographic characteristics. Needs, desires, goals: treat and cure patients Habits (consumer habits, behaviour): expert, impatient, caring Expertise: medicine… Cultural background: interest in the Arts, Science Motivations: treat patient, work at speed, not be held up Product Must do: fast, accurate, timely, easy to find information Product Must never: hold up appointment, give false information User experience goals: flow, trust, tailored/curated
- 27. Be happy - do good - leave the world a better place than you found it Isabel Evans Identify WHO uses your product / service Identify WHY they use it Identify WHAT they want to do Identify their EMOTIONS and EMPATHISE! Score the UX, Quality in Use (QiU) and Internal Quality (IQ) attributes
- 28. Be happy - do good - leave the world a better place than you found it Isabel Evans Apple lover v. Silver emailer v. Consultant 0 2 4 6 8 10 Effectiveness Efficiency Satisfaction Conformity Extendibility AccessibilityCommercial Health & Safety Environment Excitement Desirability Apple lover Silver emailer Hospital consultant
- 29. Be happy - do good - leave the world a better place than you found it Isabel Evans Small farmer, farm manager, government official 0 2 4 6 8 10 Effectiveness Efficiency Satisfaction Conformity Extendibility AccessibilityCommercial Health & Safety Environment Excitement Desirability farmer farm manager gov. official
- 30. Be happy - do good - leave the world a better place than you found it Isabel Evans Identify WHO uses your product / service Identify WHY they use it Identify WHAT they want to do Identify their EMOTIONS and EMPATHISE! Score the UX, Quality in Use (QiU) and Internal Quality (IQ) attributes Write RICH STORIES
- 31. Be happy - do good - leave the world a better place than you found it Isabel Evans The silver emailer’s story As an intermittent email user I want to keep in touch with my friends and family Without changes to the way I do it Because I have better things to do than learn new technology Transparency Intuitiveness Flow Usability Backward compatibility
- 32. Be happy - do good - leave the world a better place than you found it Isabel Evans The Apple User’s story “As a non-technical person I want to transfer to the latest gadgets and software Without being tied to one source Without losing my calendar entries and emails Because otherwise I am sad and frustrated” Interoperability, compatibility, portability Flow Trust Seductiveness Flexibility
- 33. Be happy - do good - leave the world a better place than you found it Isabel Evans The consultant’s story “As a consultant doctor I want to concentrate on treating my patients and communicating well with them I don’t want to be blocked by the software Because otherwise my patients and I become frustrated and are subjected to unnecessary worry” Flow Trust Speed Usability Flexibility Safety Functional suitability Security BYOD /tablet?
- 34. Be happy - do good - leave the world a better place than you found it Isabel Evans Acceptance criteria • NOT perfection • Minimum acceptable quality • Plus, optionally, a “Desirable” goal Current position World record Minimum acceptance Desirable
- 35. Be happy - do good - leave the world a better place than you found it Isabel Evans The silver emailer: acceptance criteria Given that not all end-users want to negotiate tech When the system is updated Then all the user’s favourite task paths are clearly signposted Transparency Intuitiveness Flow Usability Backward compatibility
- 36. Be happy - do good - leave the world a better place than you found it Isabel Evans The Apple User: acceptance criteria Given there are multiple apps and platforms When we upgrade our platform Then we ensure compatibility with other platforms remains Interoperability, compatibility, portability Flow Trust Seductiveness Flexibility
- 37. Be happy - do good - leave the world a better place than you found it Isabel Evans The consultant: acceptance criteria Given that the patient’s records are online When I request a patient’s record Then - I can do so by name search - The latest results are displayed first - The result display is accurate, secure and written in medical terms - I can access the latest results in fewer than 4 clicks Flow Trust Speed Usability Flexibility Safety BYOD /tablet? Functional suitability Security
- 38. Be happy - do good - leave the world a better place than you found it Isabel Evans Identify WHO uses your product / service Identify WHY they use it Identify WHAT they want to do Identify their EMOTIONS and EMPATHISE! Score the UX, Quality in Use (QiU) and Internal Quality (IQ) attributes Write RICH STORIES Do INTERACTION DESIGN Do a USABILITY REVIEW and OBSERVATIONS
- 39. Be happy - do good - leave the world a better place than you found it Isabel Evans WHEN DO WE TEST THE USER EXPERIENCE? The user has a choice – always… stay go
- 40. Be happy - do good - leave the world a better place than you found it Isabel Evans You know the answer to this question
- 41. Be happy - do good - leave the world a better place than you found it Isabel Evans Start as early as possible Continue to test after live Test ideas Test personas Test context Test stories Test design Test build Test rollout Test use Test follow up
- 42. Be happy - do good - leave the world a better place than you found it Isabel Evans The tester’s choice stay go What impact can a tester have on the user experience?
- 43. Be happy - do good - leave the world a better place than you found it Isabel Evans The Tester’s choice? • Narrow scope • Do what you asked to do • Work alone • Deliver functionality • Think of “them and us” • Use your head • Broad scope • Ask questions • Work with the Dev’s, BA’s & UXD’s • Deliver services • Think of “we” • Use your head, heart, guts and soul
- 44. Be happy - do good - leave the world a better place than you found it Isabel Evans Isabel’s choice? It took an hour and 15 minutes to get Apple iTunes to sell & download to me “Should I stay or should I go?” It took 3 minutes on Amazon, including listening to the song sample.
- 45. Be happy - do good - leave the world a better place than you found it Isabel Evans Three key take-aways • Your users have a choice –And so do you • What they require is not just functionality –And that’s only part of what you provide • Big picture (emotions via thought experiments) –Versus the detail (this tab order is wrong) stay go Internal quality Quality in Use UX-D
- 46. Be happy - do good - leave the world a better place than you found it Isabel Evans What will you do? “As the gap between customer expectations and existing CX grows, there will be more opportunities to improve CX and expand your business. But only some companies will be able to take advantage of this growing CX thirst; others will see an exodus of increasingly disappointed customers. Choose your path." https://experiencematters.wordpress.com/2016/03/28/examining- massive-decline-in-customer-experience-ratings
- 47. Be happy - do good - leave the world a better place than you found it Isabel Evans Your choice… Head Heart Gut Soul
- 48. Be happy - do good - leave the world a better place than you found it Isabel Evans Nokia TestDive 2017 Krakow Thank you for listening! Isabel Evans fbcs citp [email protected] www.isabelevans.uk
- 49. Be happy - do good - leave the world a better place than you found it Isabel Evans References (1) [1] Lehner A, Dumke R., Software Metrics: Research & practice in software measurement, Available at: http://bit.ly/2kSZrs2, Accessed: 3 Feb 2017). [2] Tulpule, N, (2016), Developer Experience, FWT!, Conference keynote presentation, Google Test Automation Conference, GTAC 2016 (Isabel attended) [3] Havelock, P, Testware or Shelfware - the reality, Article on BCS website, available at http://bit.ly/2lBmxmF (Accessed 11 Feb 2017) [4] Kaner, C, (1998), Avoiding Shelfware: A Managers’ View of Automated GUI Testing, Paper STAR98 conference, Available at http://bit.ly/2kSMyOI (Accessed 11 Feb 2017) [5] Fewster, M. and Graham, D. (1999) Software test automation: Effective use of test execution tools. Harlow: Addison-Wesley Educational Publishers. [6] Graham, D. and Fewster, M. (2012) Experiences of test automation: Case studies of software test automation. United States: Addison-Wesley Educational Publishers. [7] Tassey, G. (2003) The economic impacts of inadequate infrastructure for software testing: Final report. Diane Pub Co. Available at http://bit.ly/2kT52OP [8] Jones, C. (2015) Wastage: The Impact of Poor Quality on Software Economics, Software Quality Professional Journal, (article) Dec2015, Vol. 18 Issue 1 [9] IEEE, (2014)IEEE 730-2014 IEEE Standard for Software Quality Assurance Processes, 3.2 [10] Nielsen, J and Norman, D, The definition of user experience, Nielsen Norman NN Group Website, available at http://bit.ly/1T72ZR5 {accessed 11 Feb 2017} [11] Hassenzahl, M., The encyclopedia of human computer interaction, Interaction Design Foundation, chapter 3 “User Experience and Experience Design”. Available at http://bit.ly/2kZKI0W (accessed 11 Feb 2017) [12] Atoum, I., Bong, C., Kulathuramaiyer, N., (2013) “Towards Resolving Software Quality-in-Use Measurement Challenges”, University of Malaysia Sarawak [13] Zubrow, D., Software Quality Requirements and Evaluation, the ISO 25000 Series, Carnegie Mellon University, Report of standards working party [14] Richardson, A and Evans, I, Conversation: Automation and People, the UKSTAR 2017 programme includes a debate on technical skills required by testers (http://bit.ly/2kZNzGY, see track T10) (conference debate) [15] Whittle, J., Hutchinson, J., Rouncefield, M., Burden, H., and Heldan, R., (2015) A Taxonomy of Tool-Related Issues Affecting the Adoption of Model-Driven Engineering, Software and Systems Modeling. ISSN 1619-1366 [http://eprints.lancs.ac.uk/77975/] [16] Johnson, B., Song, Y., and Murphy-Hill, E., Bowdidge, R, (2013) Why Don’t Software Developers Use Static Analysis Tools to Find Bugs?, NCSU { https://people.engr.ncsu.edu/ermurph3/papers/icse13b.pdf} (accessed 11 Feb 2017) [17] CA Consulting, (2016) “Moving IT Service Management to the 21st Century”, available at http://bit.ly/2kggqlX [18] Chemaly, S, (07/27/2015 03:52 pm ET | Updated Jul 27, 2016) What gender stereotypes and sexism have to do with Algorithms and Robots, Huffington Post, Available at http://huff.to/1I5pek4 (accessed 11 Feb 2017) [19] Lent, Knowledge, skills and abilities for software QA jobs, available at http://bit.ly/1fJ0iXq (accessed 11 Feb 2017, no date on website) [20] Own experiences, anecdotal evidence, emails, conference discussions & feedback (UCAAT, PSTQB, EuroSTAR16, CREST workshop, UCL, UK Software testing retreat)
- 50. Be happy - do good - leave the world a better place than you found it Isabel Evans References (2) [21] Smartbear tools vendor, available at https://smartbear.com/learn/automated-testing/ (accessed 11 Feb 2017) [22] Johnson, B., Pandita, R., Smith, J., Ford, D., Elder, S., Murphy-Hill, E., Heckman, S., Sadowski, C., (2016) A Cross-Tool Communication Study on Program Analysis Tool Notifications, ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE), [http://bit.ly/2kT4EQt] (accessed 11 Feb 2017) [23] Moore, G., Crossing the Chasm 2nd Edition (1999) Capstone [24] Hendrikson, E., Do Testers Have to Write Code? Blog, [Available at http://bit.ly/2lzuMij] (accessed 11 Feb 2017) [25] Lambert, R, Why testers really should learn to code, Blog, [available at http://bit.ly/2kZNW4k ] (accessed 11 Feb 2017) [26] Gill, M., Software Testing Skills that you must have: An overview, Blog, [available at http://bit.ly/2kZLmLM] {accessed 11 Feb 2017) [27] Beckwith, L., Kissinger, C., Burnett, M., Weidenbeck, S., Lawrance, J., Blackwell, A., Cook, C., (2006) Tinkering and Gender in End-user Programmers’ Debugging, [available at http://bit.ly/2hPq0iI], (accessed 11 Feb 2017) [28] Morrison, P., Pandita, R., Murphy-Hill, E., and McLaughlin, A., (2016), Veteran Developers’ Contributions and Motivations: An Open Source Perspective, [Available at http://bit.ly/2ke1wSe] (accessed 11 Feb 2017) [29] Fisher, M., Cao, M., Rothermel, G., Brown, D., Cook, C., Burnett, M., Integrating Automated Test Generation into the WYSIWYT Spreadsheet Testing Methodology, [available at http://cse.unl.edu/~grother/papers/tosem06.pdf], (accessed 11 Feb 2017) [30] Schroeder, K. (2010), Gender dimensions of Product Design, Expert Paper for UNDAW / UNESCO, [online] [available at http://bit.ly/1MXNTPf] (accessed 11 Feb 2017) [31] Pauleen, D., Evaristo, R., Davison, R., Ang, S., Alanis, M., and Klein, S. (2005) Cultural Bias in Information Systems Research and Practice: are you coming from the same place I am?, ICIS Panel discussion summary [online http://bit.ly/2bfI5lY] (Accessed 11 Feb 2017) [32] Isabel Evans’ Conversations with attendees at the Women in Innovation (WII) meeting. (2016). Quotes: UK Government Infocus/Innovate UK awards Women in Innovation 2016, see http://bit.ly/1P05CBG [accessed 7 Feb 2017] [33] Nielsen, J. (2017). Ten Usability Heuristics. [online] [Available at: https://www.nngroup.com/articles/ten-usability-heuristics/] [Accessed 4 Feb. 2017]. [34] Firestone, D. (2014). Common System and Software Testing Pitfalls,e-book, Pearson Education Available at Amazon.co.uk [35] Borg, A; Porter, C; Micallef, M. (2015). Poster: Is Carmen better than George? Testing the Exploratory Tester using HCI Techniques, International Conference on Software Engineering, to appear, 2015, Florence, Italy [36] Micallef, M., Porter, C. (2016). HCI – The Tester’s New Sidekick?, The Tester Magazine – BCS 03/2016 (pp 12 -15) [37] Micallef, M., Porter, C., Borg, A. (2016). Do Exploratory Testers Need Formal Training? An Investigation Using HCI Techniques., The 11th Workshop on Testing: Academia-Industry Collaboration, Practice and Research Techniques, April 11, 2016, Chicago, IL, USA [38] Micallef, M., Porter, C. (2017). Help! I’m only human! Understanding and supporting the human tester, Keynote at the British Computer Society SIGIST Testing Conference, March 14, 2017, London, UK
- 51. Be happy - do good - leave the world a better place than you found it Isabel Evans References (3)[39] Constructing Grounded Theory, Kathy Charmez [40] Quality in Use: Meeting User Needs for Quality, Nigel Bevan, Serco Usability Services Journal of System and Software, 1999 (in press) [41] “The test tool marked all the tests as passed except 1, but in fact none of the tests marked “passed” had actually run” Quote from Fewster and Graham “Experiences of Test Automation” [42] Quotes from attendees at the WII briefing meeting: “I spend 50% of my time wrestling with the technology instead of solving the problem I am working on” and “it is like working with something designed to be used by a 12-year-old boy in his bedroom in the 1980’s”. “Why would I want to use a tool called Github?” [43] Gender and other bias in IT tools, for example in tool default behaviour, use of language, voice recognition, gendering of support tools, is increasingly being studied and remarked on e.g. http://huff.to/1I5pek4, http://bit.ly/1MXNTPf , http://bit.ly/2bfI5lY. [44] Problems with customer support tools: “Ethnographic research paints a sad picture of the current state of the ITSM market. …vision is to build a solution designed for humans, not processes” [http://blogs.ca.com/2016/01/27/moving-itservice-management-to-the-21st-century/] [45] Evidence that tools do not work for IT people “…a lack of consideration for how people work and think … basically it’s still the mindset that the human adapts to the computer, not vice-versa.” (A Taxonomy of Tool-Related Issues Affecting the Adoption of Model-Driven Engineering by Whittle, Hutchinson, Rouncefield, Burden and Heldal) [46] Evidence that developers do not find tools easy to use: “…so now I wanna know why raising a string exception is bad. Like what should I be doing instead? Since it thinks it’s a problem. And so none of these really help me…” (Why Don’t Software Developers Use Static Analysis Tools to Find Bugs? By Johnson, Song, and Murphy-Hill). ISO 25000 Series of standards (quality in use attributes) [47] http://alarmingdevelopment.org/ [48] http://blogs.ca.com/2016/01/27/moving-it-service-management-to-the-21st-century/ [49] https://www.youtube.com/watch?v=oQ455i1aCQI [50] https://www.nngroup.com/articles/ten-usability-heuristics/ [51] http://research.csc.ncsu.edu/dlf/