Test Maturity Assessment

and Improvement Using

TPI® and Quality Blueprint
Performance driven. Quality assured.
 2
Benchmark, Blueprint, and Execute to
Build a World-Class Test Organization
The testing world has Many organizations lack sufficient Capgemini and Sogeti have developed
insight into the effectiveness of their a highly effective approach to test
changed; focus has current testing processes however, process assessment and improvement,
based on the combination of
moved from testing single leaving them unable to efficiently
achieve the system quality required our world-leading Test Process
systems to ensuring by the business. Improvement (TPI®) methodology
and our Quality Blueprint framework
multiple business This lack of maturity in a test (Figure 1). This service can help
processes are robust and organization is often most evident in: you address your key test process
challenges, realize a higher level
fault-free for users. ƒƒ Frequent conflicts between the of test maturity, and achieve more
business and IT; streamlined and efficient processes.
ƒƒ High operating costs, including
The steps we take are:
resources, set-up, and test-
center maintenance; ƒƒ Measure and score your ƒ
existing maturity—using our
ƒƒ Poor product quality and inadequate
TPI® foundation;
intelligence on why quality level
has dropped; ƒƒ Benchmark the test maturity ƒ
of your organization against the
ƒƒ Slower time-to-market and high
industry standard;
employee churn.
ƒƒ Define a roadmap for success—
Many organizations recognize
actionable plans that focus effort
that improving the test process is
and value on business priorities;
essential for ensuring the quality of
the system being tested and business ƒƒ Leverage industry best-practice
process itself—but remain unsure Accelerators to reduce cost and
of the steps required to make this improve productivity.
improvement successful.
Figure 1: End-to-end assessment
Capgemini & Sogeti TMap® & TPI® frameworks
& Accelerators
Foundations

and improvement
World Quality Report forum

Innovation Labs & Accelerators – Quality Blueprint

TPI® framework Quality Blueprint

Leverage Innovation Labs for
‘Accelerated Implementation’

Define roadmap per area

Benchmark against your peers

Assess maturity

Identify assessment areas

Assess & suggest Benchmark, plan & implement

Test Maturity Assessment and Improvement using TPI® and Quality Blueprint 3
Business-Driven Optimization
of Test Processes
Four Stages to Test Process Improvement

1. Measure and Score your TPI® carries out a detailed analysis As an integral part of the broader
Existing Maturity of your current test process, software development cycle, testing
categorizing it into 16 Key is influenced by the other processes.
The first step is based on TPI®, our
Areas, ranging from Stakeholder TPI®’s Enablers provide a better
world-leading model for providing
commitment to Test environment. understanding of the explicit
an objective step-by-step guide
The assessment identifies the key correlation between testing and
to business-driven test process
challenges in your test organization the adjacent software development
improvement. Applying this model
and analyses the readiness of your lifecycle, helping you identify the
provides an in-depth insight into the
organization to meet current and quick wins that have broader impact.
maturity of your organization’s test
processes as well as recommendations emerging test challenges against
Each test process assessment is
for gradual and controllable three Maturity Levels (Controlled,
customized depending on the needs of
improvement steps. Built on the Efficient, and Optimizing), and
the organization—taking into account
knowledge and practical experience associated Checkpoints. The
industry, business goals, and software
of our testing experts and our clients output—the Maturity Matrix—is
development methods (such as
who use it, the TPI® model can be a comprehensive analysis of your
Agile). TPI® can be used in many test
structurally aligned with your business organization’s testing ability,
situations such as integration testing,
drivers, rather than just with IT or enabling you to clearly visualize
outsourced managed testing, and
testing goals—meeting your specific what areas need to be improved,
even for starting up a test organization
requirements as cost-effectively and as illustrated in Figure 2.
from scratch.
efficiently as possible.

Improvement Suggestions
Maturity Levels
Figure 2: The essential
Initial Controlled Efficient Optimizing
elements of the Business-
Driven TPI model, shown in
1. Stakeholder commitment A B B C F H H K M M
the Maturity Matrix 2. Degree of involvement A B C E H H J L L
3. Test strategy A A B E F F H K L
4. Test organization A D D E I I J J K L L
5. Communication B C C D F F J M M
6. Reporting A C C F G G K K
Enablers

7. Test process management A A B B G H J K M

Key Areas

8. Estimating and planning B B C C G H I I K L L

9. Metrics C C D G H H I K K
10. Defect management A A B D F F H J K L L
11. Testware management B B D E I I J L L L
12. Methodology practice C D E F H J J M M
13. Tester professionalism D D E E G G I I K K M
14. Test case design A A E F I I J K K M
15. Test tools E E E F G G I L M M
16. Test environment C D D E G H J J L M M

Checkpoints grouped into Clusters

4
2. Benchmark the Test Maturity of 3. Define Your Quality Blueprint These are both integrated into our
Your Organization Roadmap for Success Quality Blueprint roadmap that is
The next stage is to enter the results It can be challenging to define usually developed quarter by quarter
of your assessment into our Quality the steps required to improve and across a range of the most important
Blueprint automated tool, which control test processes but, with a parameters for your organization—
contains comparable data from starting point and destinations clearly such as estimation, coverage, and
hundreds of international Capgemini identified, we can then work with you team structure.
and Sogeti test assessments. It to plan an effective action plan.
This business-driven approach,
uniquely enables a comparative
Our experience has shown there is illustrated in Figure 4, prioritizes
benchmark against the industry
no standard formula to achieve the quick-wins based on value versus
standard—the test maturity of other
perfect roadmap; it must be built effort to ensure that the highest yield
firms in your sector, including your
collaboratively in close alliance with improvements are initiated first, and
competitors. This market intelligence
your test organization with its unique provides an opportunity to motivate
enables you to understand how
business priorities, knowledge, and your team and build ownership of
others are addressing similar
key stakeholders. the transformation.
challenges, helps you make better-
informed and competitive-based The ‘Improvement Suggestions’
decisions, and serves as a starting within TPI®, based on practical
point for your test transformation,
experience, provide realistic ‘how
see Figure 3.
to’ recommendations and the TPI
The data in our tool is continuously Checkpoints, grouped into
updated based on new assessments Clusters, clarify the necessary
and industry reports including stepwise improvements for
the findings from our World incremental transformation.
Quality Report1. Carried out with
the support of HP, this provides
market intelligence on the sector- Figure 3: An example quality benchmark carried out for a client
specific quality trends around testing Global Benchmark
technologies, processes, organizations, Client assessment data
Degree of Involvement
outsourcing, and economics. Stakeholder Commitment 4.0 Test Process Management
Test Process Innovations Estimating & Planning
Using this information, we work with 3.0
you to define ‘success’ and establish Test Organization Test Coverage
the short- and long-term destination 2.0
Communication Test Case Design
points of your transformation, based
on your own business-specific 1.0

objectives, competitive positioning, Tester Professionalism Defect Management

or sector-specific standards. 0.0

Knowledge Management Testware Management

Test Environment Reporting

Metrics Testing Types

1 World Quality Report, an annual in-depth study of Methodology Practice Test Automation
enterprise application quality assurance testing and the
emerging trends in Application Lifecycle Management,
Test Tools Test Strategy
produced by Capgemini and Sogeti, with HP. Test Parties Co-ordination

Test Maturity Assessment and Improvement Using TPI® and Quality Blueprint 5
4. Leverage and Implement Industry Both industry-specific and generic Initially changes in test maturity
Best-Practice Accelerators examples include: should be monitored regularly
Our Quality Blueprint framework through parameter improvements,
ƒƒ Mainframe Automation Tool; but as test maturity reaches the
is more than just a methodology.
Using automated tools, frameworks, ƒƒ Defect Prediction Modeling Tool; highest levels, benefits will become
accelerators, and solutions more visible and easier to measure,
ƒƒ Model-based Testing Framework; through reduced production defects,
developed in our Innovation Labs
or with specialist partners, you will ƒƒ TMap® Accelerator—a plug-in for increased test efficiency, and lower
gain the benefit of our extensive key partner software delivery tools. costs of quality assurance.
testing experience.
Our Quality Blueprint framework also Our experience indicates that you
These Accelerators have been enables you to measure the impact of can reap initial benefits from this
developed by us over the years, across your test maturity after transformation quality process transformation in just
multiple domains, technologies, and across all the maturity parameters— four weeks through actionable plans
geographies. They use real-world data such as cost and productivity. prioritized based on effort and value,
from hundreds of test assessments but most projects take three months
performed to give additional rigor to or longer.
your improvement roadmap.

Figure 4. Mapping the step-by-step transformation for a Financial Services client

Sample Roadmap
Roadmap Q4 2009 Q1 2010 Q2 2010 Q3 2010 Q4 2010
Team Structure
& Resource ∆ Define team ∆ Augment SOA ∆ Build multi- ∆ Define competency framework
Competency structure process skills site team

Test
Management & ∆ Migrate ∆ Coverage for ∆ Coverage for ∆ Coverage for ∆ Coverage for
Coverage manual test cases VisionPlus Triad finance IVR

Test Process
∆ Testing ∆ End-to-end STLC & central ∆ Refine process for UAT,
Methodology
Glossary repository conversions, upgrades

Test Estimation ∆ Bottom-up estimation ∆ Estimation for enterprise apps,

for VisionPlus finance and Triad

Types & Levels

∆ Augment ∆ Independent ∆ Triad testing ∆ Finance UAT
of Testing
team skills system testing

6
A Clear Path Forward: Assess.
the Benefits of TPI® Understand.
and Quality Blueprint Improve.
This business-driven Additional services we can provide: Our Test Process Improvement
and Quality Blueprint service will
approach to improving ƒƒ Quick Scan—a condensed give you a detailed assessment
assessment offering rapid insight of your current testing process
your testing process into your test process quality maturity and a clearly defined
provides several and prioritizing short-term road to improvement.
actions for improvement;
significant benefits: With a clear understanding of
ƒƒ Training—customized courses
where you are now and how
for embedding improvements in
you want to proceed, we can
ƒƒ Clear and improved insight your business;
help you implement the steps
into your testing maturity—in
ƒƒ Consultancy—expert and that will achieve greater
comparison with your industry
objective advice on the best consistency in quality, greater
peers, as well as understanding
ways to prioritize and implement control of test execution, and
the strategic value of testing
improvement activities, as part of improved cost savings.
within the overall quality
a wider, ongoing strategy.
assurance of your systems’ This process of improvement is not
development and configuration; a one-off solution but an iterative,
ƒƒ A clear roadmap to improvement repeatable approach, enabling you
—an inclusive program that to really optimize your testing
optimizes, structures, and refines performance. Establishing the
your approach to testing; sound foundations for short- and
long-term improvements, using
ƒƒ Test processes aligned with your TPI® and Quality Blueprint, will
key business drivers—80% of ensure that your applications
users report that TPI® provides and services operate to their full
very good ROI, strengthening potential at all times.
quality and agility in line with
business goals; A Client Example
ƒƒ Increased productivity and A leading credit bureau faced
reduced time-to-market— numerous challenges relating
automation and repeatable to budget allocations, resource
processes reduce the time it takes skills, and process inconsistencies
to test software and applications; in their QA department. Using
our TPI® and Quality Blueprint
ƒƒ Access to a wealth of industry framework, the client was able to
best-practices and experience— improve their test maturity and
Innovation Labs, knowledge- beat industry benchmarks within
sharing, and the support of an two years. Their centralized and
expert team will ensure that you decentralized test organization
have the tools and skills now uses well-defined reporting
to accelerate the implementation mechanisms and is supported by a
of your testing strategy; vibrant, positive test team culture.
ƒƒ Minimized risk—using proven
frameworks to provide regulatory
compliance, particularly relevant
in financial services, telecoms,
pharmaceuticals, and government.

Test Maturity Assessment and Improvement Using TPI® and Quality Blueprint 7
About Capgemini and Sogeti
The Capgemini Group is one of the world’s foremost providers of consulting,
technology and outsourcing services, enabling its clients to transform and
perform through the use of technologies. Present in over 30 countries, the
Capgemini Group reported 2009 global revenues of EUR 8.4 billion and employs
over 90,000 people worldwide. Sogeti, its wholly-owned subsidiary, is a leading
provider of local professional services, bringing together more than 20,000
professionals in 15 countries and is present in over 200 locations in Europe,

2010-04_TMP UK. © Shutterstock.com / Afaizal (cover) / Rafa Irusta (page2)

the US and India.

Together, Capgemini and Sogeti have developed innovative, business-driven

quality assurance (QA) and testing services, combining best-in-breed testing
methodologies (TMap® and TPI®) and the global delivery model, Rightshore®,
to help organizations achieve their testing and QA goals. Capgemini and Sogeti
have one of the largest dedicated testing practices in the world, with over 6,400
test professionals and a further 11,000 application specialists, notably through
a common center of excellence with testing specialists developed in India.

Contact Europe
For more information about how Stefan Gerstner
Capgemini and Sogeti’s Testing E: [email protected]
Services can help organizations T: +31 (0) 6 6493 2665
achieve their testing and QA goals, Julien Bensaid
please contact your local Capgemini E: [email protected]
or Sogeti testing representative or our T: +31 (0) 30 689 19 85
Global Testing Services Sales Team:
North America
Charlie Li
E: [email protected]
T: + 1 (0) 678 644 7594

www.capgemini.com/testing
www.sogeti.com/testing

Rightshore® is a registered trademark belonging to Capgemini.

TMap®, TMap® NEXT, TPI® and TPI® NEXT are registered trademarks of Sogeti.
Copyright © 2010 Capgemini and Sogeti. All rights reserved. No part of this document may
be modified, deleted or expanded by any process or means without prior written permission
from Capgemini.