Adopting Agile Testing
1 like2,441 views
The document outlines a four-step approach for transforming a traditional QA team into an agile QA team, emphasizing the need for organizations to adopt agile methodologies to enhance software development and delivery speed. Key benefits include improved quality, better team alignment, and faster market delivery, necessitating changes in team dynamics as QA becomes integrated within development processes. Transitioning requires a mindset shift, new skills in automation and exploratory testing, and a thorough adoption of agile practices to ensure effective collaboration and testing throughout the development lifecycle.
Adopting Agile Testing
- 1. Agile Testing Transforming a Traditional QA team to Agile: A 4 Step Approach White Paper
- 2. ADOPTING AGILE TESTING Executive Summary A growing need for quicker and adaptive solutions to tech problems is pushing firms to adopt the agile methodology. Today more and more companies are addressing different technology issues by adopting this iterative approach to software development and releasing high quality software, faster and more efficiently. Organizations see agile software development as a faster way to create products, thereby reducing the Go To Market time. The benefits of adopting the agile software methodology include improved quality, flexibility to midcourse correction, improved business satisfaction, better alignment between the teams and improved time to market. However, adopting the agile development methods means that traditional software development organizations must change, in particular the functioning of QA teams. This white paper discusses the approach to transition a traditional QA team into an agile QA team, when an organization decides to adopt the agile methodology 2 Page
- 3. ADOPTING AGILE TESTING What is Agile QA? The objective of any testing is to uncover defects (deviations from the defined business requirements) in software and to ensure that it works before it is shipped to customers. This objective stays the same for traditional as well as agile QA; and the traditional testing techniques are very relevant to testing in agile as well. In fact, testers on agile teams exploit the benefits of traditional QA within an iterative development process, but the structure, communication, test execution frequency and escalation methods are different in agile testing. In the traditional QA model, testing is performed by a separate group with its own line of reporting and organization. In this model, while the developers design the code, the QA team develops the test plan. For both teams, the requirements specifications from the requirements engineering group form the baseline. The QA team receives the product, often after months of design and coding for an agreed set of requirements by stakeholders. Then the QA team tests this code and reports defects for the development team to fix. When developers fix the defects filed by QA, they release the newer version of the software to QA; this cycle repeats until the product is ready to be shipped. Although, this practice has been followed for many years across different organizations, it has its own limitations. • Developers test their code less and rely more on testers, leaking defects to testers who find defects late in development lifecycle, impacting time to market • Significant delays before developers receive feedback from testers, again impacting time to market • Frequent changes to test plan and test cases, as business requirements change. • If the changed requirements are not communicated to all stake holders, then there is a mismatch in expec- tations in the final product across all stake holders • QA is usually the last activity before the published release date, and any delay in either requirements gathering or development, leads to a shorter time frame for testing. Extreme pressure from management to complete testing within the available time will compromise quality of the final product. • Business owners are involved in UAT after testing is formally completed. Observations from business users come very late in the project lifecycle before deployment, and development may have moved to the next version of the release. On the contrary, in the agile methodology, QA is normally entrenched in the agile project team, working alongside the developers and business users at every stage of the process. This early involvement and transparency ensures that the QA team is engaged much earlier in the software development life cycle, and gets the requirements directly from business users. This leads to early identification of dependencies, technical or testing challenges, project roadblocks and these are discussed with stake holders and built into estimation and project planning. Some of the characteristics of agile testing include: • Agile testers are part of a cross functional team and talk directly to developers and have their say in all phases of the software development lifecycle. • The test team is colocated with the dev team and their participation in daily scrum meetings ensures information from all stakeholders is uninterrupted. • Continuous feedback loop to development team • Frequent iterative releases mean, more regression, hence the agile test team frequently makes use of test automation as a way to cut down testing time. 3 Page
- 4. ADOPTING AGILE TESTING • Agile testing drives development by refining acceptance criteria and questioning stories during iteration planning • Test cases always exist, are expanded / modified and executed continuously • Usually the unit, integration and acceptance tests are automated. These characteristics give agile testing certain advantages over traditional testing, such as • Ongoing feedback to developers, allowing testers to ask the right questions and fine tune testing • As the final acceptance criteria are established prior to the work beginning, testing is always embedded in agile process and cutting corners such as skipping testing cannot happen. In fact testing measures the project progress in this methodology • The agile process emphasizes testing at lower levels such as unit and integration tests. Regression risks are managed in lower levels of testing, improving time to market. Agile Testing Transformation in Organizations When an organization decides to move from traditional development to agile development, transitioning the traditional QA team to an agile QA team is a critical task for the management. This task is not difficult if planned properly. This transformation not only requires changing the way the testing team works, but also requires changes throughout the entire organization, from management to business organizations to programmers and support teams. This section describes the different aspects to be considered while transitioning a traditional QA team to an Agile QA team. Management must emphasize the following critical aspects though training, counseling and coaching the traditional QA team before starting the transition. 1. Change in mindset Team Work: In the traditional model, testers start testing after the coding is complete. Testers and developers often communicate through the defect tracking system and they belong to different groups within the organization. However, an agile tester is tightly integrated with the development process, and participates in planning, estimating, scheduling and story definition. An agile tester does not merely check the quality of the software; instead he/she moves from a mindset of “how can I break the system” to a mindset of “how can I deliver quality software successfully”. Collaborate and communicate: Agile teams do not produce hundreds of pages of requirements documents that form the basis for test cases in traditional QA. Agile testers must proactively work with customers and developers to get the requirements. In fact, agile testers need to serve as a bridge between customers and developers. When requirements are unclear; customers, programmers, and agile testers discuss together to clarify requirements. The agile methodology values constant feedback and communication in order to avoid surprises in the later stages of development. This means that unlike the traditional QA team which usually keeps test design, test document, test progress and QA specific metrics within QA boundaries, and publish them once in a while, agile testers provide visibility into testing activities during daily standup meetings, so issues can be addressed early in the sprint cycles. 4 Page
- 5. ADOPTING AGILE TESTING Unlearn: When an organization moves from the traditional approach to agile, many testers from the traditional QA team move to agile QA with the traditional QA mindset. Learning the agile way of testing is pretty straight forward, as agile is a light weight process. However unlearning past behaviors and practices is the difficult part. For example, in traditional QA, an organization may be comfortable with manual testing, and may not invest in test automation. However, with the time boxed nature of agile, investing in test automation will pay dividends, since manual testing will increase cost and time. In the beginning, when test automation is in progress, it makes sense to keep the traditional manual testing. Once test automation is completed, manual system testing is both expensive and inefficient. Practices and beliefs such as manual testing, that have made positive contribution in the traditional QA, now need to be unlearned if the team is to reach full potential. 2. Develop new Skills Automation: The time boxed nature of the agile practice presents some challenges to the QA team. Traditional QA considers testing as a mass effort at the middle or near the end of the project. However in agile, the dev team adds new features in each sprint and testing has to ensure that the features previously built, continue to work by incorporating testing in each sprint. Also agile encourages change and must keep changes under control, preventing bugs from leaking into production. To achieve this, an agile QA team must shorten the feedback cycle to the development team, on features implemented. This is where the importance of test automation comes into picture. Teams that are transiting from traditional QA to agile QA, must invest more in test automation. Without automation, the test team will be bogged down with running an ever increasing set of test cases manually, under a tight schedule. This is not scalable and sustainable. With automation at all levels of development i.e. unit, integration and system levels and continuous execution of automated test cases, the agile test team achieves speed, quality and efficiency Change the Way Traditional QA Measures Quality: Many organizations transitioning from traditional QA to agile QA, continue using the same metrics they used for traditional QA. Because the two processes are fundamentally different, organizations and project teams adopting the agile practice must come up with a different set of metrics for quality. For example, the metric “test pass and fail ratio”, in traditional QA, does not have any meaning in the agile methodology, since the moment tests start failing, the dev team stops and fixes the regression right away. Teams transitioning to agile QA, can come up with metrics related to parameters such as velocity, cycle time, code coverage, burn down and defects raised by customers; depending on business needs of stakeholders, customers and projects. Programming Skills: Any tester who can write a test script in a simple word processor should be able to work on an agile team. However, a tester with very good programming and automation skills will be able to make the testing activity more productive and efficient by continuously automating features in each sprint. Exploratory Testing: Test automation is an integral part of agile testing to address the challenges of the time boxed nature of the agile practice. However, test automation poses its own challenges to development work, as automation cannot always address issues related to usability, reliability, performance, compatibility and other quality criteria. Using exploratory testing, some of the gaps left by test automation may be covered. The other reason why agile testers need exploratory testing is that in an agile project, the requirements may not be concrete, and changes are part of any agile project. As it is difficult to add test cases for unexpected changes and review them, exploratory testing is an essential skill to uncover the additional considerations. 5 Page
- 6. ADOPTING AGILE TESTING Exploratory testing helps agile test teams to cope with the ever changing context of the project. Management must invest in training the QA team on exploratory testing skills. Once the testers adopt exploratory testing, they will understand which risks are important to the team, where they should focus, how to improve test coverage and make appropriate daily decisions to deal with unexpected changes in the project. Working with Lean Process and Documentation: A typical agile practice moves QA activities upstream in the project lifecycle, enabling QA to prevent defects rather than finding them at the end of project lifecycle as in traditional QA. Agile believes in less documentation and agile testers may not get the required documents at the beginning of the project lifecycle or for that matter, throughout project lifecycle. For this reason, agile testers must develop skills to document test case/test script or test requirements. Testers can share these test requirements with customers, so the gaps in the original requirements can be filled immediately. Test Driven Development (TDD) or Behavior Driven Development (BDD) can be adopted for this purpose. A traditional QA team, transitioning to agile QA, must learn how techniques such as TDD and BDD work and how to write these test cases. 3. Adapt new tools A traditional QA team transitioning to an agile QA, must be deeply involved with advance practices such as Test Driven Development, Behavior Driven Development, Test Automation and Continuous Build and Integration, All these have significant impact on the day to day activities of the QA team. These new practices also influence the selection of tools for testing activities. In addition, as the Agile QA team is tightly integrated with development and project activities, they cannot distinguish between QA, Project Management and Development tools, instead should have knowledge on all the tools used by different personnel on the agile team as a whole. Some of the ALM tools that are used in the agile methodology that testers must be familiar with, are shown in the table below. ALM Phase Tools Exepctations from QA Project Planning Project Management tool Testers must be familiar Project Status Sprint and Release status, Resource allocation and status, Burn Down charts Testers must be hands on Story Capture Theme, epic and story capture, acceptance criteria Story Capture Testers must be hands on Documentation management Simple tools like wiki or any other tools Testers must be hands on Software Development Continuous integration, unit testing tools, build tools, source control, checklist, code coverage Software Development Testers must be hands on Manual test case creation, TDD & BDD creation, automated test case management, Defect Management Integral part of Agile QA team Defect creation and tracking, Defect integration with sprints, stories and requirements. Integral part of Agile QA team Test Management Defect Management 6 Page
- 7. ADOPTING AGILE TESTING As can be seen from the table, an Agile QA team uses most of the tools in different stages of the ALM. Organizations must provide, tools, support and training to agile QA teams, to help them adapt to this new landscape of tools. may get frustrated to test multiple builds with changed requirements in the same iteration. It is the responsibility of management to make these QA teams understand the importance of iterative system testing and motivate them to adopt the agile testing practice. 4. Position your System Testing Phase in Agile QA • In many organizations, system testing slots are booked for different projects to optimize the hardware resources shared by many projects. If agile test teams decide to execute system testing whenever a feature is changed within any given iteration, it is very difficult to block the hardware in advance. Organizations adopting agile practices must come up with plans to address such situations, including any external dependencies. In traditional QA, System Testing is positioned towards the end of the development lifecycle, starting after development is completed, but before the product is released to customers. This is well controlled by entry criteria to start system testing. In contrast, the agile process demands that the build from every sprint cycle, be ready for deployment. When organizations transition to agile QA, they must come up a proper strategy, addressing the following issues: • Typically, traditional QA segregates system testing specialists to a single group, performing system testing for different development teams. Breaking down the virtual walls between different teams and tightly coupling the system testers to agile teams may be a challenge. Organizations must design a strategy where system and performance testing are an integral part of the agile development process. • Testers may not get fully completed features in each sprint, making system testing a challenge. Organizations must work with different stakeholders and set expectations that system testing cycles will be executed every iteration (say, every third or fourth sprint) rather than at the end of every sprint. • An agile process is adaptive, designed to accommodate and encourage inevitable changes. This may lead to frequent changes in the system, and the testers Conclusion Organizations adopt the agile software development methodology as a faster way to create and release products. The benefits of adopting the agile software methodology include improved quality, flexibility to midcourse correction, improved business satisfaction, better alignment between the teams and improved time to market. However, adopting agile development methods means that traditional software development organizations must change the way QA teams work. Traditional QA teams must change their mindset, break down the virtual wall with development teams, learn newer testing techniques such as exploratory testing, testing based on TDD and BDD. Organizations must see automated testing as an integral part of agile testing and not just rely on manual testing. The testers in the agile methodology must learn newer tools and skills. They must ask the right questions, while working collaboratively with other groups, in overcoming some of the challenges of testing in the agile methodology. 7 Page
- 8. ADOPTING AGILE TESTING About the Author Harsha B N works as a Test Architect in the Mobility division of Idexcel. He has twelve years of experience in development and testing mobile applications. Prior to joining Idexcel Harsha worked with Nokia for eight years in various capacities as Program Manager, Chief Test Engineer, Project Manager working on OTA infrastructure development, Mobile Payments services, S60 SDK. About Idexcel Idexcel is an innovative provider of IT Products & Services focused on emerging technologies. We help world leading companies build efficiencies and stronger businesses. With more than 15 years into existence Idexcel’s main focus is client satisfaction and technology innovation. Our industry expertise and a global, collaborative workforce forms the backbone of our services. We offer high degree of skills in Enterprise Applications, Cloud Services, Data-warehousing, Big Data, Analytic, QA & Testing Services, IT consulting and Staffing. Idexcel product line includes: NDS, ERP, and Cync - A revolutionary credit monitoring application for the manufacturing and financial management. For more information log on to www.idexcel.com. Global Head quarters 459 Herndon Parkway Suite 11 Herndon, VA 20170 Tel: 703-230-2600 Fax: 703-467-0218 Email: [email protected] India Operations “Crystal Plaza” 9, 10 ,11 Bhuvanappa Layout, Hosur Road Bengaluru – 560 029 Karnataka Tel: +91-80-2550 8830 Email: [email protected] © Copyright, Idexcel. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Idexcel. The information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.