IT Knowledge Topic

Agile Software Development

May 2020

For Internal Use Only

Disclaimer
This document is for reference information only. The use of this material is optional. It does not modify the
audit methodology or guidance set out in the relevant manual.
If there is a mandatory/specified approach (or document) for your local member firm, please use that. For
example, this document should not be used by US member firm staff. If you are unsure of your member
firm’s policy for use of this document it is recommended you contact your relevant Risk or Methodology
team, including DPP resources.

This document may not cover all risks or considerations related to the specified topic. These materials are
provided for consideration and should be assessed for use, if appropriate, on an engagement-specific basis.
Wherever possible, audit work is documented directly in eAudIT/KPMG Clara workflow.
This document should not be put on file.

Overview
This document provides a high level overview of Agile Software Development, and is split into three sections:
1) What is Agile Software Development?
2) General considerations when obtaining an understanding of Agile Software Development controls; and
3) Example test considerations
Appendix A: Agile methodology concepts
Appendix B: Glossary

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
What is Agile Software Development?
“Agile” is a development concept that favours frequent Inherent risks of using Agile methodology:
iterative releases over traditionally long-phased —— Lack of documentation: Inconsistent application of
releases. Agile software development covers a group principles driven by individual experience and/or
of software development methodologies based on knowledge.
the concept of iterative and incremental development
through defined ‘sprints,’ which are typically two-week —— Continuous changes in design: Design
cycles of focused development. Through the use of requirements may change over the course of
“Agile methodology”, requirements and solutions product development without revisiting security or
evolve through collaboration among self-organizing, control requirements.
cross-functional teams. Agile processes also include —— Scaling requires careful management: Large, cross-
change, release, and configuration management functional teams and complex solutions can cause
practices. additional work, not less.
The Agile software development process can be —— Dependencies on ‘soft’ controls: ‘Soft’ controls (i.e.
broken down into 6 phases: team skills, knowledge, and communication) may
1. Concept: Scope out and prioritize projects lead to compliance challenges.
2. Inception: Diagram requirements for the initial —— High levels of autonomy across teams and business
sprint units: Inconsistent approaches to meeting control
objectives increase the risk of objectives not being
3. Construction/Iteration: Deliver working product met.
4. Release: Test and deploy the iteration into Below is an example high-level process flow of an
production Agile development lifecycle. The above-mentioned
5. Production: Operate and ongoing support for the risks correspond to more specific process risk points,
software release which are embedded throughout the process flow
below. These process risks points are described in the
6. Retirement: Remove system from production
following section.

Agile Software
Development Life Cycle

Release
Concept Construction/ Production Retirement
Inception
Iteration Test and
Select and Operate Remove
Initiate the Deploy
prioritize Initiate the and support system from
project release into
projects project release production
production

Figure 1. Agile Software Development Lifecycle.

The controls embedded within the Agile development processes and controls in place to make sure business
lifecycle that address the above process risks points requirements are met, that the software functions
can be either manual or automated, and are designed as intended and coding defects are identified and
to make sure business requirements are coded and resolved. Agile quality assurance strategies may be
implemented quickly, but are done so securely, implemented and facilitated in any number of ways.
completely and accurately. In addition, there could Testing approaches could include embedding testing
be a number of quality assurance and system testing resources with the development team as a means
© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
What is Agile Software Development? (Continued)
PD PD
1a 1c
1. Define Requirements
PD
PD 2d
2g

Release 2. Initial Estimations

PD
3h
3. High-Level Planning
PD PD
1b 3i
10. System Testing
4. Begin Iteration N

PD PD
3h 2e, 2f
9. Is Additional
Development Needed? 5. Write Story & Scenario
PD
3j
PD
PD PD
6. Implement Functionlaity & 3i
2e, 2f 1b
8. Quality Assurance Acceptance Tests
PD PD PD
3i 1b 2e, 2f

7. Deploy
PD
2g

Figure 2. A general Agile software development process flow diagram, and the relevant process risk points.

to conduct continuous and near real-time testing, or For more information on Agile methodology
performing static and dynamic code analysis. These concepts, refer to Appendix A. For a Glossary of
tests could include tests of code quality, system Agile methodology terms, refer to Appendix B.
integrity, security and functionality. Some of these
controls are contemplated upon in the next section.

General considerations
Considering the risks identified in the previous section, engagement teams may consider the following example
questions in gaining an understanding the entity’s Agile software development process, the relevant process risks
points and the associated controls:

Agile risk points Questions to understand the relevant control activities

1a Inconsistent implementation —— Is there a formal, duly approved and documented Agile development
of approach policy in place within the entity?
Agile software development —— What Agile software development approach(es) is/are being utilized
approach is not defined, and/ by the entity? Are the agreed approaches being used consistently
or documented, or is not across development teams/projects?
consistently understood by the
—— If there are more than one Agile approach, how does this vary per
development and operations
business/technology unit (i.e. which business/technology unit uses
teams, resulting in inconsistent
which approach)? Also, what is the business reason for the use of
implementation across project
different types of approaches?
teams and/or iterations.
—— How are businesses and/or development teams trained on the
agreed Agile approach(es)?
—— How is compliance by the project teams to the Agile policy
measured by the entity?

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
General Considerations (Continued)

Agile risk points Questions to understand the relevant control activities

1b Unauthorized changes —— What are the required approvals at each stage of the project? Some
example approvals may include:
Unauthorized or untested
changes are pushed to –– Project requirements
production resulting in
–– Iteration planning
production outages, service
impairment, or data integrity –– Acceptance testing
concerns/data loss.
–– Quality assurance
–– System testing
–– Iteration deployment
—— How and where are these approvals documented?
—— Where applicable, how are developer peer review processes
documented? Is this incorporated into standard operating
procedures?
1c Unauthorized emergency —— What triggers an emergency change, and how do the project and
changes development teams differentiate between a normal change vs.
an emergency change in the Agile methodology employed by the
Unclear emergency change
entity?
management policies may
result in unauthorized or —— How does the entity mitigate the risk of inappropriate changes being
untested changes promoted made via the emergency change process?
to production, resulting in
—— How is the process different for emergency changes vs normal
production outages, service
changes? Has management defined an emergency change/response
impairment, or data integrity
process with key decision points and service level agreements
concerns and/or data loss.
around timing and minimum required documentation/artifacts?
2d Unauthorized access to —— How are individuals and teams assigned to projects?
development tools and the
—— How are roles and responsibilities clearly articulated and
production environment
communicated to project team members? Is there a document
Inappropriate and/or that formally defines each critical role and the corresponding
unauthorized users have responsibilities?
access to pre-production and
—— What are the key utilities/tools used throughout the entity’s Agile
production environments,
software development process? How is logical access over the Agile
software development tools,
software development tools managed?
project management tools,
release management tools, —— What controls are in place to ensure an adequate segregation of
and secure code repositories. duties (SOD) during development, testing, release and production
Because of multiple small support processes?
teams working on Agile
–– Are there any roles/functions that are considered “potentially
projects, it may lead to overlap
incompatible functions”, where SoD is enforced by policy?
of responsibilities in the
absence of clearly defined –– Are there user groups defined for each type of project team role
roles and responsibilities. that enforces these SoDs?
–– If there are no user groups defined, how are SoDs enforced in the
process?

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
General Considerations (Continued)

Agile risk points Questions to understand the relevant control activities

2e Inappropriate developer —— Other than system development and building, what other type of
access profiles actions are developers allowed within the Agile workflow?
Authorized developer activity –– Are these other types of actions granted to developers via a
is not pre-approved, or defined user group, or are these part of the default access all
defined, approved, and/or developers have within the process?
reviewed periodically resulting
–– For these other types of actions, how is access granted to the
in unauthorized or untested
developers? How is the appropriateness of these types of access
changes being pushed to
checked and how often?
production.
—— Are there “pre-approved” types of actions developers are allowed
within the Agile workflow? How are these pre-approved types of
actions monitored by management, to ensure that the results of
these actions are in accordance with the purpose for which they
were requested?
2f Lack of or inadequate audit —— Are audit trails/logs turned on, over the course of the period under
trails audit, within the relevant Agile tools/utilities?
Logging/audit trails do not —— Where are the log repositories, and the code that generates the log
have an adequate level of reports? Who has access to them?
detail to enable the reviewer to
—— Are these audit logs subject to periodic review by management?
understand the production data
How are issues noted in the audit log raised, tracked and resolved?
edits/other changes performed
by the developer. Audit logs —— Has there been an issue raised during the period under audit that
are not retained long enough to appeared fraudulent, and if yes, how was this resolved?
allow for adequate operational
support and/or forensics
purposes.
Developers have access to
edit the logs or have access
to change code related to the
control functions (e.g. script
which generates the Log
Reports or governs the review
and post-approval workflow).
2g Lack of or inadequate —— How is the software code versioning tool used in the Agile
program version control workflow? Are software code repositories also in use?
The continuous and speed —— How is logical access over the software code versioning utility and
of changes in design may software code repository managed?
overwhelm the version
—— Is there a way for users to circumvent the version control process
tracking and code integration
(by releasing changes directly to the code base or manipulating the
process, which may result
version control systems via privileged accounts)?
in inappropriate codes being
worked on, and deployed into –– Who has access to these privileged accounts and what controls
production. are in place to ensure that only authorized individuals have access
to these privileged accounts?
—— How are versioning conventions enforced for consistency across the
environment?

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
General Considerations (Continued)

Agile risk points Questions to understand the relevant control activities

3h Lack of or inadequate —— What controls are in-place to ensure that business requirements are
iteration planning and understood by the development team and other technical resources?
feedback
—— How are team goals, objectives and priorities defined and
Poor release and iteration planning communicated for the overall project, and for each iteration?
could result in unclear project
objectives and the inability to —— How are improvement points and/or defects tracked and resolved?
improve product value if iteration
—— How are stakeholder feedback from the previous iterations carried
accomplishments are not
measured and feedback is not
over into the new iteration?
incorporated into future iterations.
3i Inadequate testing strategy —— How are testing personnel/teams assigned to each project or
Lack of a comprehensive iteration?
testing strategy could lead to —— What quality assurance and systems testing mechanisms (i.e.
errors in the product, leading Refactoring, Non-solo development, Static and Dynamic Code
to customer dissatisfaction, Analysis, Reviews and Inspections, Lightweight Milestone Review)
inefficiencies and increased are in place? Are the quality assurance mechanisms adequate based
cost. on the type(s) of coding changes?
—— Are automated controls or software scanning capabilities embedded
in the process to assist with identifying coding, security, functional,
and/or systems integration risk?
—— What or who determines whether a testing plan is comprehensive
enough to adequately test all the changes/functionality within an
iteration?
—— Are there specific types of testing required to be performed for each
type of change/iteration? If yes, where is this defined and how is this
communicated to the project team?
—— What project metrics are being measured and communicated to the
team to determine the effectiveness and efficiency of the project?
–– How are these compiled?
–– How often are they communicated?
–– How are exceptions tracked through to resolution?
3j Ineffective automated —— Does the entity utilize any DevOps/continuous delivery tool to
continuous delivery process automate certain stages of the Agile development process?
With the automation of code —— What stages of the development process is automated (i.e. testing,
reviews and code migration, code review, deployment process)?
there is an increased risk of –– What is the logic of the automated processes? What are the
inadequate and/or inappropriate attributes of the automated process?
coverage of testing, and there
–– How are the configurations managed/changes? Who has access
is a risk of unauthorized/
to change them?
untested changes being
migrated into production. —— When was the last time these automated processes were baselined,
to determine that they were performing these automated processes
Note that this risk is only correctly? How often are these baselined?
applicable when the client
employs an automated —— Are there reports that summarize the results of these automated
continuous delivery platform. processes?
–– Have the reports been baselined for completeness and accuracy?
–– Describe the review process that management performs over
these reports. How are exceptions resolved?

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
Example controls and test considerations

Controls and test procedures will vary based on the facts and circumstances of the particular audit engagement.
Be mindful that each audited entity’s processes, risk points and internal controls will vary depending on the
IT environment and development methodologies used. The table below provides example risks auditors
may consider to identify the controls in place, and relevant considerations in designing their procedures. The
information in this section is provided as reference information only.

Specific risks to Example control(s) Considerations when determining the nature, timing
Agile and extent of audit procedures
(PD 1) Newly developed IT systems or major enhancements to existing IT systems are not authorized.
a. Inconsistent —— Approved Agile
—— Are there policies in place defining a singular approach
implementation development approaches
to be followed for agile development and continuous
of approach are documented within
changes in design?
company policy/standards.
— — Is training regarding the Agile development policy
Agile software
development approach — — Key stakeholders within the and approach delivered, in accordance with current,
is not defined, and/ business and information approved Agile policy and methodology? How is this
or documented, or technology communities evidenced by the entity?
is not consistently are required to attend —— For the Sprint Retrospective Charts, are retrospections
understood by periodic Agile software carried out at the end of each sprint to understand that
the development development training. project team members are aware of the approach?
and operations —— On a project by project How is this documented in the process?
teams, resulting basis, the approach,
in inconsistent including process steps and
implementation across decision points, are clearly
project teams and/or defined.
iterations.
b. Unauthorized Approved developer peer —— Are peer review guidelines documented? Do the peer
changes review processes are review guidelines include procedures covering the
documented and incorporated following:
Unauthorized or
into standard operating –– Documentation and guidance for the expectations of
untested changes are
procedures and are firmly
pushed to production peer reviewers
embedded in the development
resulting in production –– Whether the code change adhere to the change
lifecycle.
outages, service
requirement
impairment, or data
integrity concerns/data –– Whether the accompanying test results include a
loss. review of the adequacy of tests coverage
—— Did the team present sufficient information on the
intent of the change, as well as the actual code
changes being checked-in, to allow a peer reviewer to
perform an effective review? Is the engagement team
able to re-perform them? Are there tools used by the
peer reviewer process to enable reviewers to verify
whether the code change adheres with the change
requirement, e.g. tools used for line-by-line comparison
of code changes from current version in development
vs. production version of the code?
—— Where the change request originated from the
business, did the reviewer consider their requirements,
or were the business users involved in the testing?
How is their involvement documented alongside the
change?
—— For a selection of code changes, was a peer review
completed prior to implementation and story
completion?

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
Example controls and test considerations (Continued)

Specific risks to Example control(s) Considerations when determining the nature, timing
Agile and extent of audit procedures
c. Unauthorized A formally documented and —— What are the conditions that can trigger the emergency
emergency approved emergency change change process (e.g. incident record) and are the
changes management process exists as mechanisms that link the change to its trigger is clearly
a means to expedite changes defined and documented within the emergency change
Unclear emergency
into production. E-changes policy?
change management
are categorized in the change —— Are the logging mechanisms in place capable of
policies may result
management system and detecting an emergency change?
in unauthorized or
change protocols are enforced
untested changes —— Do the code repository and version control systems
through management
promoted to provide reporting to alert the management teams of an
monitoring and reporting.
production, resulting emergency change being introduced to production, and
in production outages, can developers circumvent these controls?
service impairment, or —— If peer review and testing controls are circumvented
data integrity concerns in order to deploy the change, is a post-release review
and/or data loss. undertaken to validate the change was implemented
in line with the requirements and in response to the
incident, and that the review takes place timely and
whether identified discrepancies are resolved?
—— If a password release or direct production edit is
required, were these activities reviewed and approved,
and did the credentials used only support the
emergency change that required them?
(PD 2) Newly developed IT systems or major enhancements to existing IT systems are introduced into the
production environment prior to their approval.
d. Unauthorized Logical access for pre- —— Is there segregation of roles and responsibilities for key
access to production and production roles/teams such as Product Owner, Scrum Master,
development environments, software Feature team, Component team, Solution Infrastructure
tools and the development and release Manager, Release Manager? Are these formally
production management systems, defined in Project Scrum Boards, RACI charts, policies,
environment software and tools is restricted processes, org charts etc.? Are user groups defined in
to authorized individuals and accordance with SoD considerations as per the above
Inappropriate and/
does not present a conflict of groups, and only authorized individuals have access to:
or unauthorized
interest. –– Pre-production (development, testing regions) and
users have access
to pre-production production environments
Roles and responsibilities are
and production –– Software development, project management and
clearly defined prior to the start
environments,
of each project for key project testing tools (e.g. JIRA, Veracode, Fortify, Visual
software development
stakeholders, including project Studio, automated testing scripts, etc.)
tools, project
sponsor/product owner.
management tools, –– Release management software and secure code
release management An adequate segregation repositories?
tools, and secure of duties is maintained with
code repositories. respect to role assignments.
Because of multiple
small teams working
on Agile projects, it
may lead to overlap
of responsibilities in
the absence of clearly
defined roles and
responsibilities.

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
Example controls and test considerations (Continued)

Specific risks to Example control(s) Considerations when determining the nature, timing
Agile and extent of audit procedures
e. Inappropriate Rules, workflow access points —— Are there system controls that require management
developer access and process steps are formally authorization before developers can make certain types
profiles defined and configured for of production data edits?
developer pre-authorization —— Are adequate approvals established? Is stakeholder
Authorized developer
checks. input solicited and whether the approval was gained
activity is not
pre-approved, or from all impacted parties? Can inappropriate activities
defined, approved, be performed under the preauthorized criteria? Do
and/or reviewed logging mechanisms capture developer activity in
periodically resulting sufficient detail?
in unauthorized or —— Does management review the log, escalates and
untested changes follows up on non-routine events as necessary and
being pushed to within the required timeframes?
production. —— Consider what supporting GITCs may exist over the
tools or systems used for implementing and managing
pre-authorization checks, i.e. user access, change
management, computer operations, etc. Identify
relevant GITCs that support the consistent operation
of the application controls within these tools, and test
them accordingly.
f. Lack of or —— Audit logs are configured —— Are tools/mechanisms used in development logged and
inadequate audit to capture security, reviewed by management for appropriateness?
trails system and operational —— Do logs contain the level of detail which captures
activity in accordance with the specific activity performed by the developer, to
Logging/audit trails do
internal standards and understand the activity performed?
not have an adequate
requirements.
level of detail to —— Do the approvals/reviews completed timely (based
enable the reviewer —— Where systems are unable on existing policy), and are unauthorized activities
to understand the to comply with minimum monitored and investigated through resolution?
production data logging requirements,
exceptions are documented —— Are the reviewers/approvers appropriately aligned with
edits/other changes the developers’ organization who are knowledgeable of
performed by the and communicated to
standards owners for the activities being performed?
developer. Audit logs
are not retained long approval. —— Are the log reports complete and accurate? For relevant
enough to allow for —— Management monitoring of guidance on testing C&A of information produced
adequate operational logged activity is completed by the entity, refer to the Global Audit – Information
support and/or in accordance with defined Produced by the Entity: A Guide to identifying and
forensics purposes. monitoring protocols. evaluating IPE including addendum
—— Audit logs are retained in —— Is there an audit log retention policy/standard, and does
Developers have it comply with internal standards.
access to edit the accordance with company
logs or have access policy/standard. Log —— Is developer access reviewed periodically for
to change code retention standards are appropriateness? Are these user access re-certifications
related to the control reviewed and approved completed timely, and any access marked for deletion
functions (e.g. script annually. Developer access was removed timely?
which generates to production tools and log/ —— Do access to the log reports or the code generating the
the Log Reports or audit files is managed and log reports is restricted to authorized individuals and
governs the review approved in accordance cannot be changed by developers?
and post-approval with a least privileged
—— Consider what supporting GITCs may exist over the
workflow). approach and re-certified
tools or systems used for implementing and managing
on an annual basis.
pre-authorization checks, i.e. user access, change
management, computer operations, etc. Identify
relevant GITCs that support the consistent operation
of the application controls within these tools, and test
them accordingly.

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
Example controls and test considerations (Continued)

Specific risks to Example control(s) Considerations when determining the nature, timing
Agile and extent of audit procedures
g. Lack of or Source code versioning —— Do these tools function as code version control,
inadequate software/repositories are continuous integration etc. Are tools also in place to
program version configured to track and maintain a single source code repository?
control managed code changes, —— Do users have the ability to circumvent the version
including maintaining an audit control process, either by releasing changes directly
The continuous and
trail of who made changes, to the code base or through manipulating the version
speed of changes
summary of changes, etc. control systems with the use of privileged accounts?
in design may
overwhelm the —— Is there a review processes in place to enforce and
version tracking and review version tracking and consistency across the
code integration environment (e.g. handling of replication conflicts in a
process, which may globally distributed code base)?
result in inappropriate —— Consider what supporting GITCs may exist over the
codes being worked tools or systems used for version control, i.e. user
on, and deployed into access, change management, computer operations,
production. etc. Identify relevant GITCs that support the consistent
operation of the application controls within these tools,
and test them accordingly.
(PD 3) Newly developed IT systems do not function as intended.
(PD 4) Major enhancements to existing IT systems do not function as intended.
(PD 5) Incomplete and/or inaccurate data is migrated to the production environment of newly developed IT
systems.
h. Lack of or —— Clearly defined project/ —— Is there a defined high-level vision and market/business
inadequate iteration objectives and objectives that have been communicated to the team?
iteration product owner priorities —— Is there an agreement about what constitutes the
planning and are documented and release product for each iteration (i.e. “Definition of
feedback communicated to the Done”)?
project team.
Poor release and —— Before each new sprint, is business owners’ feedback
iteration planning —— A continuous improvement from the previous iteration, along with the updated
could result in unclear process exists where market situation and deadlines, are articulated and
project objectives accomplishments are communicated to the development team?
and the inability to tracked and improvement
—— Is there a process for mapping dependencies, in the
improve product opportunities are evaluated
form of input from other teams and subject matter
value if iteration for future iterations.
professionals, at every stage of the iteration?
accomplishments
—— Is there a process for tracking how impediments from
are not measured
previous sprints have been addressed? Does this
and feedback is not
“product backlog” allow stack-ranking to reflect the
incorporated into
priorities of the product owner?
future iterations.

i. Inadequate —— Traceability matrices are —— Do the development teams follow the “whole
testing strategy developed and documented team strategy” where people with testing skills are
linking all requirements to embedded into the development team?
Lack of a
respective design, testing —— Is testing performed by an independent team, and
comprehensive
scenarios, acceptance whether the team has the necessary skills to perform
testing strategy
criteria and results. comprehensive testing?
could lead to errors
in the product, —— If the development team uses the process/tools to
leading to customer make their working build, are these also available to the
dissatisfaction, independent test team on a regular basis?
inefficiencies and
increased cost.

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
Example controls and test considerations (Continued)

Specific risks to Example control(s) Considerations when determining the nature, timing
Agile and extent of audit procedures
—— Independent testing —— Is there a mechanism for the independent test team to
is performed by report defects back to the development team, and that
knowledgeable personnel, these defects are treated as a type of “requirement”
and the test results by the development team in that the defects are
are communicated to prioritized, estimated, and put on the work item stack?
the project team and —— Do the tests performed provide sufficient coverage
documented. for the code change? This may include tests of code
—— The PBI (Product Backlog quality, system integrity, security, and intended
Items) is prepared and functionality.
reviewed on a periodic —— Does the agile quality strategy addresses the
basis by key stakeholders. organization’s needs? Some of the common agile
quality techniques are: Refactoring, Non-solo
development, Static and Dynamic Code Analysis,
Reviews and Inspections, Lightweight Milestone
Review. Are project metrics captured and made
available to the team? Some of the metrics associated
with agile projects are: Customer Satisfaction Survey,
Scrum Team Satisfaction Survey, Business Value Burn
up (= total # of user stories + each user story’s unit
value/sprint duration), Team Velocity (= # of units of
effort completed/sprint duration), Retrospective process
improvement, Technical debt = product backlog total #
of units (story points)/sprint duration.
j. Ineffective Automated routines are —— Do projects have automated unit and regression testing
automated developed, tested and secured procedures?
continuous in accordance with company —— Are thresholds configured to systemically identify
delivery process standards. test failures? Does management regularly review the
With the automation thresholds periodically for reasonableness?
of code reviews and —— Where automated tests are configured, does
code migration, there management regularly reviews the automated tests for
is an increased risk reasonableness and adequacy of test coverage for each
of inadequate and/or type of automated test?
inappropriate coverage —— Do developers have the ability to select which tests to
of testing, and there is run at check-in?
a risk of unauthorized/
—— Do code changes that fail automated test requirements
untested changes
and thresholds are automatically configured to fail the
being migrated into
check-in process (i.e. changes that fail the automated
production.
tests/fail to meet the thresholds are not eligible for
migration into production)?
Note that this risk is
only applicable when —— Consider what supporting GITCs may exist over the
the client employs an tools or systems. Identify relevant GITCs that support
automated continuous the consistent operation of the automated code
delivery platform. reviews, deployment process, etc. within these tools,
and test them accordingly.

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
Appendix A: Agile methodology concepts

Twelve Agile Principles

The Agile Manifesto was written in February 2001 by a group of 17 developers including the creators of Scrum,
Extreme Programming (XP), Dynamic Systems Development Method (DSDM) and Crystal, a representation of
feature-driven development, and several other thought leaders in the software industry. The Agile Manifesto
established a common set of overarching values and principles for all of the individual Agile methodologies at
the time. The Agile Manifesto lists a set of values that if believed in and implemented would result in a shift from
traditional development methodologies to being more flexible and realistic in the IT world.
The Agile Alliance, an industry recognized non-profit organization committed to supporting the Agile community,
has published 12 principles in implementing and executing with agility. These 12 principles are:
1. Customer satisfaction by early and continuous delivery of valuable software
2. Welcome changing requirements, even in late development
3. Deliver working software frequently (weeks rather than months)
4. Close, daily cooperation between business people and developers
5. Projects are built around motivated individuals, who should be trusted
6. Face-to-face conversation is the best form of communication (co-location)
7. Working software is the primary measure of progress
8. Sustainable development, able to maintain a constant pace
9. Continuous attention to technical excellence and good design
10. Simplicity—the art of maximizing the amount of work not done—is essential
11. Best architectures, requirements, and designs emerge from self-organizing teams
12. Regularly, the team reflects on how to become more effective, and adjusts accordingly
Agile vs. DevOps
Agile is an iterative approach to development, which focuses on collaboration, customer feedback, and small,
rapid releases. DevOps is considered a practice of bringing development and operations teams together. DevOps,
a compound term of “Development” and “Operations”, is commonly considered the natural evolution of the Agile
movement, as organizations looked for the ability to release their software faster and more frequently. DevOps is
a practice that aims at unifying software development (Dev) and software operation (Ops). The main characteristic
of the DevOps movement is to strongly advocate automation and monitoring at all steps of software construction,
from integration, testing, releasing to deployment and infrastructure management. DevOps aims at shorter
development cycles, increased deployment frequency, and more dependable releases, in close alignment with
business objectives.
DevOps vs. Continuous Delivery
DevOps is the machine that builds the service, while continuous delivery is the conveyer belt that rolls the
services off the production line—one big unified service development cycle. Continuous delivery automates the
entire software release process. The approach ensures that every change to the system can be released and any
version can be released at the push of a button1. Every revision that is committed triggers an automated flow that
builds, tests, and then stages the update2.

1  DevOps and Continuous Delivery: Not the Same, https://devops.com/devops-and-continuous-delivery-not-same/

2  Continuous Delivery vs. Continuous Deployment, https://aws.amazon.com/devops/continuous-delivery/

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
Appendix B: Glossary

Note to engagement teams: Organizations may use different terms to refer to certain concepts and/or
elements within the Agile Software Development process. This section lists some of the common terms used
by organizations, and should not be used as an exhaustive or definitive listing of terms or definitions. When
gaining an understanding of the Agile methodology as implemented in these organizations, double-check the
definitions with the relevant process owners.

Agile Software Development Approaches

DEVOPS: DevOps is a software development methodology that combines software development (Dev) with
information technology operations (Ops) participating together in the entire service lifecycle, from design through
the development process to production support.
DYNAMIC SOFTWARE DEVELOPMENT METHOD (DSDM)3: is an Agile method that focuses on the full project
lifecycle. It was created in 1994, after project managers using RAD (Rapid Application Development) sought
more governance and discipline to this new iterative way of working. Its success is due to the philosophy “that
any project must be aligned to clearly defined strategic goals and focus upon early delivery of real benefits to
the business.” Supporting this philosophy are eight principles, which allows teams to maintain focus and achieve
project goals.
EXTREME PROGRAMMING (XP): XP revolves around small development cycles to increase developer focus
and productivity.
FEATURE DRIVEN DEVELOPMENT (FDD)4: is a client-centric, architecture-centric, and pragmatic software
process. The term "client" in FDD is used to represent what Agile Modelling (AM) refers to as project stakeholders
or customers. A feature is a small, client-valued function expressed in the form <action><result><object>.
KANBAN: Unlike other Agile frameworks, Kanban development does not require fixed iterations. Work moves
through the development process as a continuous flow of activity.
LEAN SOFTWARE DEVELOPMENT: Lean development can be summarized by seven principles, which are very
close in concept to the following lean manufacturing principles - Eliminate waste, Amplify learning, Decide as late
as possible, Deliver as fast as possible, Empower the team, Build integrity in and See the whole
SCRUM: Scrum is a lightweight management framework with broad applicability for managing and controlling
iterative and incremental projects of all types.
Agile Development Team Members
BUSINESS ANALYST5: The Business Analyst is both active in supporting the project-level roles and fully
integrated with the Solution Development Team. The Business Analyst facilitates the relationship between the
business and technical roles, ensuring accurate and appropriate decisions are made on the Evolving Solution on a
day-to-day basis. The Business Analyst ensures that the business needs are properly modelled and analysed and
are correctly reflected in the guidance the team needs to generate the solution.
BUSINESS STAKEHOLDERS: Responsible for performing continuous user testing and providing feedback to the
project team on a rapid basis.
DOMAIN EXPERT/TECHNICAL ADVISOR5: supports the team by providing specific, and often specialist,
technical input to the project, often from the perspective of those responsible for operational change
management, operational support, ongoing maintenance of the solution, etc.
PRODUCT OWNER: Responsible for bridging the gap between the customer, business stakeholders, and the
development team managing the clarification of user requirements and application functionality daily.

3  Definition from https://www.agilebusiness.org/page/whatisdsdm

4  Definition from http://agilemodeling.com/essays/fdd.htm
5  Definition from https://www.agilebusiness.org/page/ProjectFramework_07_RolesResponsibilities

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A
RELEASE MANAGER: Coordinates and schedules QA and production release process, manages the release
cycle, automation tools, and monitors release activities.
SCRUM MASTER: Drives the development team, clearing organizational roadblocks, and keeps the Agile process
consistent. Serves as both a facilitator and project manager.
SOLUTION INFRASTRUCTURE MANAGER: Coordinates implementation of development, test, QA, and
production environments.
THE COMPONENT TEAM: Teams that work on a particular component or layer, and their responsibilities remain
within the boundaries of the component or the layer they are tasked to develop.
THE FEATURE TEAM: Team that is made up of 5-9 people with cross functional skills who do the work and is
self-led. This team is responsible for all tasks of the iteration sprint including plan, build, test, and feedback on
user stories.
Quality Assurance and Systems Testing Mechanisms
REFACTORING6: The process of clarifying and simplifying the design of existing code, without changing its
behavior. Agile teams are maintaining and extending their code a lot from iteration to iteration, and without
continuous refactoring, this is hard to do. 
NON-SOLO DEVELOPMENT7: In Extreme Programming (XP), pair programming is a practice that suggests two
developers should work together sharing one keyboard as they code. This is also a type of code review/design in
real-time as one person watches when the other codes. 
STATIC AND DYNAMIC CODE ANALYSIS8: Under Static Testing, code is not executed. Rather it manually checks
the code, requirement documents, and design documents to find errors. The main objective of this testing is to
improve the quality of software products by finding errors in the early stages of the development cycle. Under
Dynamic Testing, a code is executed. It checks for functional behavior of software system, memory/CPU usage
and overall performance of the system. The main objective of this testing is to confirm that the software product
works in conformance with the business requirements.
LIGHTWEIGHT MILESTONE REVIEW9: Milestones mark specific progress points on the development timeline,
and they can be invaluable in measuring and monitoring the evolution and risk of a program. There can be three
types of milestones: Program Increment (PI), fixed-date, and learning milestones.
Agile Project Metrics
CUSTOMER SATISFACTION SURVEY10: There are several well-known metrics used to measure customer
satisfaction. One is the Net Promoter Score (NPS), which measures if users would recommend the software to
others, do nothing, or recommend against it. Using a consistent customer satisfaction metric and measuring it for
every release indicates whether the scrum team is meeting its end goal—to provide value to customers.
SCRUM TEAM SATISFACTION SURVEY10: Surveying the scrum team periodically to see how satisfied they are
with their work can provide warning signals about culture issues, team conflicts or process issues.
BUSINESS VALUE BURN UP: equivalent to ‘total # of user stories + each user story’s unit value/sprint duration’
TEAM VELOCITY: equivalent to ‘# of units of effort completed/sprint duration’
TECHNICAL DEBT: equivalent to ‘product backlog total # of units (story points)/sprint duration’
RETROSPECTIVE PROCESS IMPROVEMENT/SPRINT RETROSPECTIVE10: A summary meeting at the end
of the sprint to share what went well, what went less well, and ideas for improvement. Provides qualitative
information that can help assess the health of the scrum team and the process.

6  Definition from https://resources.collab.net/agile-101/code-refactoring

7  Definition from https://innoroo.com/blog/2018/08/08/non-solo-development-glossary/
8  Definition from https://www.guru99.com/static-dynamic-testing.html
9  Definition from https://www.scaledagileframework.com/milestones/
10  Definition sourced from https://www.sealights.io/software-development-metrics/11-scrum-metrics-and-their-value-to-scrum-teams/

© 2020 KPMG International Cooperative (“KPMG International”), a Swiss entity. Member firms of the KPMG network of independent firms are affiliated with KPMG International. KPMG
International provides no client services. No member firm has any authority to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International
have any such authority to obligate or bind any member firm. All rights reserved. NDP041742-1A