TMMi Implementation Guideline
10 likes7,849 views
This document discusses reaching Level 3 process maturity in the TMMi (Test Maturity Model integration) framework. It provides an overview of the five levels of testing process maturity in TMMi, from Level 1: Initial to Level 5: Optimization. It then focuses on the specific practices required at Level 2, including establishing a test policy and strategy, performing test planning, monitoring and control, designing and executing tests, and developing test environments. For each process area at Level 2, it lists the specific goals and practices and provides brief guidelines for implementing them.
TMMi Implementation Guideline
- 1. Path to reach Level 3 test process maturity in TMMi framework M C Nagaraj [email protected] in.linkedin.com/in/mcnagaraj
- 2. Copyright STAG Software Pvt. Ltd., 2011 Agenda TMMi – Level 1 to 5 Requirement interpretation Linking current process to TMMi Specific practices All TMMi related information are taken from TMMi framework document V 3.1 created and Copyrights owned by TMMi foundation. www.tmmifoundation.org
- 3. Copyright STAG Software Pvt. Ltd., 2011 Test Measurement Software Quality Evaluation Advanced Peer Reviews Defect Prevention Total Process Optimization Quality Control Test Policy and Strategy, Test Planning, Test Monitoring and Control, Test Design and Execution, Test Environment Test Organization, Test Training Program, Test Lifecycle and Integration, Non-functional Testing, Peer Reviews Level 1 : Initial TMMi Framework Five levels of Testing Process Maturity Level 2 : Managed Level 3 : Defined Level 4 : Measured Level 5 : Optimization
- 4. Copyright STAG Software Pvt. Ltd., 2011 Test Measurement Software Quality Evaluation Advanced Peer Reviews Defect Prevention Total Process Optimization Quality Control Test Policy and Strategy, Test Planning, Test Monitoring and Control, Test Design and Execution, Test Environment Test Organization, Test Training Program, Test Lifecycle and Integration, Non-functional Testing, Peer Reviews Level 1 : Initial TMMi Framework Five levels of Testing Process Maturity Level 2 : Managed Level 3 : Defined Level 4 : Measured Level 5 : Optimization
- 5. Copyright STAG Software Pvt. Ltd., 2011 Level-1 • Testing is a chaotic process • Not distinguished from debugging • Documented set of specification not available • Tests are developed in ad-hoc way after coding is completed • The objective of testing is to show that software works
- 6. Copyright STAG Software Pvt. Ltd., 2011 Test Measurement Software Quality Evaluation Advanced Peer Reviews Defect Prevention Total Process Optimization Quality Control Test Policy and Strategy, Test Planning, Test Monitoring and Control, Test Design and Execution, Test Environment Test Organization, Test Training Program, Test Lifecycle and Integration, Non-functional Testing, Peer Reviews Level 1 : Initial TMMi Level 2 Five levels of Testing Process Maturity Level 2 : Managed Level 3 : Defined Level 4 : Measured Level 5 : Optimization
- 7. Process areas at TMMi Level 2 2.1 Test Policy and strategy 2.2 Test Planning 2.3 Test Monitoring and Control 2.4 Test Design and Execution 2.5 Test Environment Copyright STAG Software Pvt. Ltd., 2011
- 8. 2.1 Test Policy and Strategy Specific Goals Specific Practices Guidelines to implement SG 1 - Establish a Test Policy SP 1.1 Define test goals SP 1.2 Define test policy SP 1.3 Distribute the test policy to stakeholders -Look at organization business goal and role of testing in achieving the same - Policy should address test cycle activities are planned, done by trained engineers, process improve with management support/focus SG 2 - Establish a Test Strategy SP 2.1 Perform generic product risk assessment SP 2.2 Define test strategy SP 2.3 Distribute the test strategy to stakeholders -Use organization standard way of identifying risks in a project -Define quality levels and objectives SG 3 - Establish Test Performance Indicators SP 3.1 Define test performance indicators SP 3.2 Deploy test performance indicators -Define how you measure success of test process -Set some target for key metrics and measure what you achieved Copyright STAG Software Pvt. Ltd., 2011
- 9. Process areas at TMMi Level 2 2.1 Test Policy and strategy 2.2 Test Planning 2.3 Test Monitoring and Control 2.4 Test Design and Execution 2.5 Test Environment Copyright STAG Software Pvt. Ltd., 2011
- 10. 2.2 Test Planning Specific Goals Specific Practices Guidelines to implement SG 1- Perform Product Risk Assessment SP 1.1 Define product risk categories SP 1.2 Identify product risks SP 1.3 Analyze product risks -Look at from end users perspective value & prioritize requirements -Can you predict what can upset end users? SG 2 - Establish a Test Approach SP 2.1 Identify items and features to be tested SP 2.2 Define the test approach SP 2.3 Define entry criteria SP 2.4 Define exit criteria SP 2.5 Define suspension and resumption criteria - Extract list of features -Extract list of user types and business use cases they use -Extract potential defect types -Quality levels and types of defects to be uncovered at each levels -Entry, exit, suspension and resumption criteria for all levels SG 3 - Establish Test Estimates SP 3.1 Establish a top-level work breakdown structure SP 3.2 Define test life cycle SP 3.3 Determine estimates for test effort and cost -Have organization standard test life cycle activity list -Customize this to project specific -Estimate effort, review update and baseline -Track number of cycles of re- estimation done Copyright STAG Software Pvt. Ltd., 2011
- 11. 2.2 Test Planning Specific Goals Specific Practices Guidelines to implement SG 4 – Develop a Test Plan SP 4.1 Establish the test schedule SP 4.2 Plan for test staffing SP 4.3 Plan for stakeholders involvement SP 4.4 Identify test project risks SP 4.5 Establish the test plan -Milestone definition -Detail schedule -Risk assessment and prioritization -Plan document with important sections in place SG 5 – Obtain Commitment to the Test Plan SP 5.1 Review test plan SP 5.2 Reconcile work and resource levels SP 5.3 Obtain test plan commitments -Review plan with all concerned -Update feedback and ensure confidence is high to meet the plan -Team commitments for all dependency will help Copyright STAG Software Pvt. Ltd., 2011
- 12. Process areas at TMMi Level 2 2.1 Test Policy and strategy 2.2 Test Planning 2.3 Test Monitoring and Control 2.4 Test Design and Execution 2.5 Test Environment Copyright STAG Software Pvt. Ltd., 2011
- 13. 2.3 Test Monitoring and Control Specific Goals Specific Practices Guidelines to implement SG 1 – Monitor Test Progress against Plan SP 1.1 Monitor test planning parameters SP 1.2 Monitor test environment resources provided and used SP 1.3 Monitor test commitments SP 1.4 Monitor test project risks SP 1.5 Monitor stakeholder involvement SP 1.6 Conduct test progress reviews SP 1.7 Conduct test progress milestone reviews -Monitor effort/cost, schedule variance against plan -Ensure test environment is same as stated in expectation in document -Milestone review, weekly review etc. -Stakeholders update on key milestone and seek feedback -Risk tracking SG 2 – Monitor Product Quality against Plan and Expectations SP 2.1 Check against entry criteria SP 2.2 Monitor defects SP 2.3 Monitor product risks SP 2.4 Monitor exit criteria SP 2.5 Monitor suspension and resumption criteria SP 2.6 Conduct product quality reviews SP 2.7 Conduct product quality milestone reviews -Quality levels exit criteria defined and actual - Monitor suspension and resumptions cycles - Test summary report at key quality levels achievement Copyright STAG Software Pvt. Ltd., 2011
- 14. 2.3 Test Monitoring and Control Specific Goals Specific Practices Guideline to implement SG 3 – Manage Corrective Action to Closure SP 3.1 Analyze issues SP 3.2 Take corrective action SP 3.3 Manage corrective action -Track project issues formally -Track action items and ensure closure Copyright STAG Software Pvt. Ltd., 2011
- 15. Process areas at TMMi Level 2 2.1 Test Policy and strategy 2.2 Test Planning 2.3 Test Monitoring and Control 2.4 Test Design and Execution 2.5 Test Environment Copyright STAG Software Pvt. Ltd., 2011
- 16. 2.4 Test Design and Execution Specific Goals Specific Practices Guidelines to implement SG 1 – Perform Test Analysis and Design using Test Design Techniques SP 1.1 Identify and prioritize test conditions SP 1.2 Identify and prioritize test cases SP 1.3 Identify necessary specific test data SP 1.4 Maintain horizontal traceability with requirements -Have a clear test case architecture to be followed across organization -Organize test cases at quality level -Maintain requirement traceability to test scenarios/test cases, fault type, types of test applicable SG 2 – Perform Test Implementation SP 2.1 Develop and prioritize test procedures SP 2.2 Create specific test data SP 2.3 Specify intake test procedure SP 2.4 Develop test execution schedule -Test cycle schedule -Build health check criteria -Create re-usable test data - Quality level entry/exit criteria Copyright STAG Software Pvt. Ltd., 2011
- 17. 2.4 Test Design and Execution Specific Goals Specific Practices Guidelines to implement SG 3 – Perform Test Execution SP 3.1 Perform intake test SP 3.2 Execute test cases SP 3.3 Report test incidents SP 3.4 Write test log -Verify entry criteria before starting any levels of testing - Execute test as per plan and record outcome -Capture defects to track and close SG 4 – Manage test incident to closure SP 4.1 Decide deposition of test incidents in configuration control board SP 4.2 Perform appropriate action to close the test incident SP 4.3 Track the status of test incidents -Be part of change control board to resolve all issues reported during testing - Ensure defects fixed are verified and closed in each release - Defect tracking report in various forms so that helps to take right decision Copyright STAG Software Pvt. Ltd., 2011
- 18. Process areas at TMMi Level 2 2.1 Test Policy and strategy 2.2 Test Planning 2.3 Test Monitoring and Control 2.4 Test Design and Execution 2.5 Test Environment Copyright STAG Software Pvt. Ltd., 2011
- 19. 2.5 Test Environment Specific Goals Specific Practices Guidelines to implement SG 1 – Develop Test Environment Requirements SP 1.1 Elicit test environments needs SP 1.2 Develop the test environment requirements SP 1.3 Analyze the test environment requirements -Capture required environment clearly at one place - Ensure environment available and required are identified SG 2 – Perform Test Environment Implementation SP 2.1 Implement the test environment SP 2.2 Create generic test data SP 2.3 Specify test environment intake test procedure SP 2.4 Perform test environment intake test -Set up test environment as needed - Create generic test data and archive for re-use - Verify whether needs of environment required for testing is met formally Copyright STAG Software Pvt. Ltd., 2011
- 20. 2.5 Test Environment Specific Goals Specific Practices Guidelines to implement SG 3 – Manage and Control Test Environments SP 3.1 Perform systems management SP 3.2 Perform test data management SP 3.3 Co-ordinate the availability and usage of the test environments SP 3.4 Report and manage test environment incidents -Plan and track system requirements -Plan development of test data and ensure it is version controlled - Ensure required co- ordination process in place when environment are shared - If environment issues found capture and track till it is closed Copyright STAG Software Pvt. Ltd., 2011
- 21. Copyright STAG Software Pvt. Ltd., 2011 What you can see in L2 organization? • Organization test goals and policy published • There is a clear separation between debugging and testing phase • It is a planned activity in project plan • Plan starts after coding is complete (only execution based testing visible) • Basic testing techniques in place • Testing is multi-leveled • Test environment managed as required
- 22. Copyright STAG Software Pvt. Ltd., 2011 Test Measurement Software Quality Evaluation Advanced Peer Reviews Defect Prevention Total Process Optimization Quality Control Test Policy and Strategy, Test Planning, Test Monitoring and Control, Test Design and Execution, Test Environment Test Organization, Test Training Program, Test Lifecycle and Integration, Non-functional Testing, Peer Reviews Level 1 : Initial TMMi Framework Five levels of Testing Process Maturity Level 2 : Managed Level 3 : Defined Level 4 : Measured Level 5 : Optimization
- 23. Process areas at TMMi Level 3 3.1 Test Organization 3.2 Test Training Program 3.3 Test Lifecycle and Integration 3.4 Non-functional testing 3.5 Peer Reviews Copyright STAG Software Pvt. Ltd., 2011
- 24. 3.1 Test Organization Specific Goals Specific Practices Guidelines to implement SG 1 – Establish a Test Organization SP 1.1 Define the test organization SP 1.2 Obtain commitment for the test organization SP 1.3 Implement the test organization -Clear definition of test team organization - Management commitment to support the test organization - Create team to fill all roles SG 2 – Establish Test Functions for Test Specialists SP 2.1 Identify test functions SP 2.2 Develop job descriptions SP 2.3 Assign staff members to test functions -Define key test lifecycle activities -Map role and responsibilities to key test life cycle activities SG 3 – Establish Test Career Paths SP 3.1 Establish test career path SP 3.2 Develop personal test career development plans -Ensure there is clear definition of career path for test engineers - Ensure individual career plans prepared and tracked by managers Copyright STAG Software Pvt. Ltd., 2011
- 25. 3.1 Test Organization Specific Goals Specific Practices Guidelines to implement SG 4 – Determine, Plan and Implement Test Process Improvement SP 4.1 Assess the organization’s test process SP 4.2 Identify the organization test process improvements SP 4.3 Plan test process improvements SP 4.4 Implement test process improvements -Assess organization test process formally - Identify improvement areas - Plan test process improvements by having a team - Ensure team work as per plan and achieve improvements planned SG 5 – Deploy the Organization Test Process and Incorporate Lessons Learned SP 5.1 Deploy standard test process and test process assets SP 5.2 Monitor implementation SP 5.3 Incorporate lessons learnt into the organizational test process -Define standard process - Capture test process assets - Monitor adherence to process by planned audits - Analyze completed projects and recommend any improvement to process needed Copyright STAG Software Pvt. Ltd., 2011
- 26. 3.2 Test Training Program Specific Goals Specific Practices Guidelines to implement SG 1 – Establish an Organizational Test Training Capability SP 1.1 Establish the strategic test training needs SP 1.2 Align the organizational and project test training needs SP 1.3 Establish an organizational test training plan SP 1.4 Establish test training capability -Define role based competency and standard training needs - Identify project specific competency required and plan training - Have clear training plan for team and project specific training on need basis -Ensure trainers internal/external identified and ready for all topics SG 2 – Provide Necessary Test Training SP 2.1 Deliver test training SP 2.2 Establish test training records SP 2.3 Assess test training effectiveness -Conduct training as per plan - Maintain test training records - Verify the effectiveness of training conducted on job Copyright STAG Software Pvt. Ltd., 2011
- 27. 3.3 Test Lifecycle and Integration Specific Goals Specific Practices Guidelines to implement SG 1 – Establish Organizational Test Process and Assets SP 1.1 Establish standard test process SP 1.2 Establish test lifecycle model descriptions addressing all test levels SP 1.3 Establish tailoring criteria and guidelines SP 1.4 Establish the organization’s test process database SP 1.5 Establish the organization's test process asset library SP 1.6 Establish work environment standards -Have organization standard test process approved - Define standard test lifecycle activities -Establish tailoring guidelines and criteria -Capture test process followed at project level as data - Capture process assets including tailoring done - Ensure work environment is defined and adhered to as per plan SG 2 – Integrate Test Lifecycle Models with Development Models SP 2.1 Establish integrated test lifecycle models SP 2.2 Review integrated lifecycle models SP 2.3 Obtain commitments on the role of testing within the integrated lifecycle models -Ensure multiple test cycles are planned with focus on all levels of testing - Review with all concerned on depends across each team and agree on entry/exit criteria Copyright STAG Software Pvt. Ltd., 2011
- 28. 3.3 Test Lifecycle and Integration Specific Goals Specific Practices Guidelines to implement SG 3 – Establish a Master Test Plan SP 3.1 Perform product risk assessment SP 3.2 Establish the test approach SP 3.3 Establish test estimates SP 3.4 Define the organization for testing SP 3.5 Develop the master test plan SP 3.6 Obtain commitment to the master test plan -Prioritize customer expectation - Define test strategy - Estimate effort required - Define team required -Have master test plan with milestone - Have detail plan for all milestone - Have commitment from all concerned in team on schedule Copyright STAG Software Pvt. Ltd., 2011
- 29. 3.4 Non-functional Testing Specific Goals Specific Practices Guidelines to implement SG 1 – Perform a Non- Functional Product Risk Assessment SP 1.1 Identify non-functional product risks SP 1.2 Analyze non-functional product risks -Identify non-functional product requirements - Analyze what can go wrong here SG 2 – Establish a Non- Functional Test Approach SP 2.1 Identify features to be tested SP 2.2 Define the non- functional test approach SP 2.3 Define non-functional exit criteria -Identify scope for NF testing - Define test approach for NF testing - Define exit criteria for all planned NF tests Copyright STAG Software Pvt. Ltd., 2011
- 30. 3.4 Non-functional Testing Specific Goals Specific Practices Guidelines to implement SG 3 – Perform Non-Functional Test Analysis and Design SP 3.1 Identify and prioritize non- functional test conditions SP 3.2 Identify and prioritize non- functional test cases SP 3.3 Identify necessary specific test data SP 3.4 Maintain horizontal traceability with non-functional requirements -Identify use cases where NF requirements may not be fully met - Have scenarios and tests cases to replicate the identified use case - Ensure NF requirement to test assets traceability is established SG 4 – Perform Non-Functional Test Implementation SP 4.1 Develop and prioritize non- functional test requirements SP 4.2 Create specific test data -Scope and plan for NF testing -Create required scripts to produce the targeted use case scenario SG 5 – Perform Non-Functional Test Execution SP 5.1 Execute non-functional test cases SP 5.2 Report non-functional test incidents SP 5.3 Write test log -Execute planned NF tests -Report NF test issues and track till closure -Capture test outcome Copyright STAG Software Pvt. Ltd., 2011
- 31. 3.5 Peer Reviews Specific Goals Specific Practices Guidelines to implement SG 1 – Establish a Peer Review Approach SP 1.1 Identify work products to be reviewed SP 1.2 Define Peer Review Criteria -Agree upon work products which will go through peer review - Expectation defined for peer review SG 2 – Perform Peer Reviews SP 2.1 Conduct peer reviews SP 2.2 Testers review test basis documents SP 2.3 Analyze peer review data -Peer review happens as per plan - Test engineers are part work product reviews which impacts their deliverables -Analyze peer review outcome and improve organization checklist Copyright STAG Software Pvt. Ltd., 2011
- 32. Copyright STAG Software Pvt. Ltd., 2011 What you can see in L3 organizations? • There is an established test organization • Testing is integrated to SDLC • Test plan is developed, tracked and controlled (Integrated with project plan) • Key work products go through peer reviews • Organization review checklist for all documents evolve • Test engineers drive test process improvement • Users/clients attend milestone meeting • User/clients support in developing usability test plans
- 33. Copyright STAG Software Pvt. Ltd., 2011 Thank You Email ID: [email protected]