White Paper

Testing in an Agile
Environment
Last Updated: 15th January, 2009
Introduction
The principal indicators of a growing business are a The testing agenda during the development phase is very
widening customer base and rising return on investment different to the traditional approaches that support the
(ROI). Various parameters steer these indicators in a Waterfall or V-Model development methodologies because
positive direction, chiefly cost, time and quality. Balancing Agile uses a Test Driven Development (TDD) model.
these parameters is of paramount significance for business However, many of the requirements of the extended testing
growth. Companies need to deliver, cost effectively, the right model will still apply.
product, in the right way, to meet customer requirements
with assured quality. They must do this first time every
Project Initiation Phase
time to ensure that they can bring new products and
services to market ahead of their competition. Companies For an Agile project to succeed, it requires effective
also need to extensively consider market dynamics planning and management. As part of the project team
including technological developments, changing customer the Test Manager is responsible for establishing the quality
preferences, evolving global standards and regulation. processes, identifying test resources and delivering the test
These market developments add further complexity and strategy. The test strategy will include details of the Agile
force companies to enhance the frequency of their product development process being used. It will also include details
updates and accelerate delivery schedules. of test phases not directly related to the Agile development
for example, Operational Acceptance or Performance
In software product development, an Agile methodology
Testing.
is one strategy adopted while addressing the challenges
of market dynamics. As customer satisfaction and ROI are
linked to product quality and cost of quality, QA and testing Development Phase
services have a critical bearing on the success of any Agile
True Agile development uses a “Test Driven Development”
project.
(TDD) approach. The testers, developers, business analysts
This white paper explores QA and testing activities in an and project stakeholders all contribute to kick-off meetings
Agile environment and the critical characteristics of the where the “user stories” are selected for the next sprint.
testing team that ensure the success of development Sprints are usually of fixed duration and selecting the
projects in an Agile environment. As there is broad-based right number of stories and estimating the time to deliver
acceptance of agile practices, an introduction to these them is the key to success. The estimation process can
practices and agile development methodologies is not often be inaccurate at the beginning of an Agile project
provided in this paper. but improves as the team’s experience grows within the
specific implementation they are working on.

Once the user stories are selected for the sprint they are
Testing Agenda in Agile Projects
used as the basis for a set of tests. The testers create test
Agile methodologies are designed to break the software scenarios which are presented to the business analysts
down into manageable parts that can be delivered earlier and project stakeholders for their approval and signoff.
to the customer. The aim of any Agile project is to deliver These test scenarios are then broken down to test cases
a basic working product as quickly as possible and then that offer adequate test coverage for the given functionality.
to go through a process of continual improvement. An The developers then write code that will pass the tests.
Agile project is characterized by having a large number of In this approach the development and testing take place
short delivery cycles (sprints) and priority is given to the continuously throughout the sprint – there is no separate
feedback-loops from one cycle to the next. The feedback- testing phase. In the TDD approach the features are tested
loops drive continuous improvement and allow the issues throughout the sprint and the results presented to the
that inevitably occur (including the way requirements have stakeholders for immediate feedback. The test scenarios
been defined or the quality of the solution provided) to be defined are not limited to functional testing but can include
dealt with much earlier in the development life cycle. To other types of testing including performance and integration
achieve this, companies need to re-think their approach to testing when the product is mature enough.
delivery and have their previously independent contributors While the development is underway the user stories for the
(Business Analysts, Developers, Testers, End Users etc.) next sprint are written. These include the stories specified
work together in teams.

AppLabs.com
App_WhitePaper_Testing_In_Agile_Environment_1v00 Page  © 2009 AppLabs
in the delivery plan but will also include additional stories Resource Management
required to cover any issues that have been identified as
part of the feedback process from previous sprints. The Agile approach required a mixture of test skills
– usually held across one team. Test resource will be
Sprints in an Agile project can extend to multiple levels in required to define the scenarios and test cases, conduct
a complex system. A sprint might not lead to a product manual testing alongside the developers, write automated
release if it does not add enough functionality to the regression tests and execute the automated regression
product being developed. The stakeholders take a decision packs. As the project progresses, specialist skills will be
on when the application should be moved to the release required to cover further test areas that might include
phase depending on the market need or the level of key integration and performance testing. The availability of a
functionality being added to the system. While multiple pool of professional test resources offers the scope for
iterations may be required to release a product, there may effective resource management across the agile project
also be cases where releases are more regular owing to the life cycle. There should be an appropriate mix of domain
additional value delivered at each iteration level. Whichever specialists who plan and gather requirements in addition
release approach is adopted, the testing team’s goal is to test engineers who will have multiple skill sets and own
to have a release available with minimal defects and low the test execution.
implementation risk at the end of the Sprint.

As functionality grows with each iteration, regression testing Communication

must be performed to ensure that existing functionality has
The benefits of independent testing will not be realized
not been impacted by the introduction of new functionality
unless good communication exists between the testers,
in each iteration cycle. Defect fixes should also be followed
developers, business analysts and project stakeholders.
by extensive regression testing. The scale of the regression
The iterative model of development and the frequency
testing grows with each sprint and to ensure that this
of releases mandate that all teams have a common
remains a manageable task the test team should use test
understanding of the user requirements. The testing teams
automation for the regression suite and focus their manual
should be skilled in the use of change management tools.
testing effort towards locating new defects during the build
The communication model used by the testing team itself
phase.
must enable both regular and ad-hoc project updates from
various parties engaged in the project. The testing team
Release Phase should adopt the most efficient and effective methods
of conveying information to the developers, project
The aim of any Agile project is to deliver a basic working
stakeholders, domain specialists using a combination
product as quickly as possible and then to go through a
of face-to-face conversation, meetings and workshops,
process of continual improvement. This means that there
phone calls, email and WebEx meetings.
is no single release phase for a product. The product will
move into production when enough of the functionality Processes
is working and the quality of the product is acceptable
to the stakeholders. Prior to release, a final acceptance Another key success factor for Agile Development is the
test is performed before transitioning the application into implementation of quality governance processes such
production. The testing activities listed above are not as configuration management. Typically, an organization
exhaustive but broadly cover the areas which the testing that has no processes in place will be in chaos. As formal
team contributes to the Agile approach. processes are implemented the organization will be able
to follow a repeatable and consistent delivery method.
Organizations considering the use of Agile should ensure
Getting Equipped for Agile Testing that configuration management, change management,
project management and release management are in place.
Agile projects present their own challenges to the testing Testing teams which bring with them best practices and
team. Unclear project scope, multiple iterations, minimal are accredited with globally recognized certifications (e.g.
documentation, early and frequent testing needs and active TMMi, CMMi, ISO etc.) will be able to help organizations
stakeholder involvement all demand special and diverse accelerate testing and enable lower cost of quality to be
skills from the testing team. Some of the essential skills are achieved.
illustrated here:

AppLabs.com
App_WhitePaper_Testing_In_Agile_Environment_1v00 Page  © 2009 AppLabs
Requirements Elicitation Conclusion
Agile projects do not necessarily have comprehensive Companies which adopt Agile projects should note the
documentation. Product requirements in Agile projects importance of engaging test teams at project initiation. This
are often captured in the form of user stories. The testers will ensure an accelerated delivery of working software. If
should be experienced and smart in analyzing and defining Agile projects are to achieve customer satisfaction and ROI
the testing requirements in the absence of any product then time, cost and quality must be controlled and balanced.
documentation. The ability to work side-by-side with the To ensure accelerated delivery of working software which
developers, analysts and customers is an important part conforms to the desired quality, the testing team should be
in quickly understanding the test requirements. The testers involved from the beginning of every iterative development
should also be equipped with alternative ways of deriving cycle and not just after the first couple of sprints. The
test requirements. A few such approaches include: testing team must develop the necessary mindset for an
Agile project. Their own agility and flexibility is essentially
 Getting access to all the material that can provide input
the key to their success in the project.
for the intended product behavior. This can include
input provided to the developer to build the application,
feature list, brief write-up on application capabilities
and partially written use cases etc.

 Knowledge transfer from product owners and subject

matter experts (SME). Direct communication will be the
best practice here as time spent with the SME helps
to elicit more product knowledge than is expressed in
written documents.

 Gaining insights into the capabilities needed from

the product under test by exploration of comparable
products, reading user manuals and user documentation,
exploring help text via user interface and taking an
application tour through the GUI.

 The testing team performing exploratory testing at

various stages of the life cycle. The testing team
should be well trained to scope the work, ask the
right questions, and deliver a valuable output within a
few hours. Exploratory test engineers should be able
to analyze a product, think critically to evaluate risks
and craft test cases that systematically explore the
product.

 The frequency of testing is greater in an Agile project

than in a traditional development model. The testing
team should be thought leaders in test automation.
The testing team should, in conjunction with the
development team, develop an automation framework
that fits the unique requirements of the Agile project.
In addition, the testing team should be efficient in
identifying the candidates for automation and build a
suite of automated tests around the application which
enable rapid iterations.

Specialist testers from the test team should have these and
many more skills which are attained through experience
as testing professionals. Testing teams delivering services
in an Agile environment need to develop and demonstrate
these skills to contribute to the success of Agile projects.

AppLabs.com
App_WhitePaper_Testing_In_Agile_Environment_1v00 Page  © 2009 AppLabs