Lecture 17 – Systems Management

Processes

1
 Overview

• This lecture is a set of odds and ends

• They are things that you need to know about
• They are things that other courses might spend a
lot of time on
• They tend to be more process based than
technical
• My rubric for topics here has been to give you a
top level introduction to any topics that we haven’t
previously discussed that would be considered as
mandatory knowledge for a systems engineer
– I don’t want anyone to turn to you and say “I thought
you said that you took systems engineering?”

2
 Overview
• The two halves of systems engineering
• Systems Engineering Management Plan
(SEMP)
• Capability Maturity Model Integration
(CMMI)
• Change/Configuration Control
• Key Performance Parameters (KPP) and
Technical Performance Metrics (TPMs)
• Master Verification Plan

3
 Systems Engineering Management Plan
(SEMP)
• My experience:
– If you don’t have one, people will insist that you have one
– If you do have one, and senior management doesn’t want to
follow it, they probably won’t
– At best, it provides you a mechanism for you to think through
how you would like to work
– At worst, it is another piece of administrative paperwork
• Some people think it is absolutely critical
• Shuttle Return to Flight was hung up by Stafford Covey until we
wrote one
• The ink wasn’t even dry when people started running all over it
• My advice: minimize the parts that don’t add value and
maximize the engineering value and use it to get your
own thinking straight

4
 Systems Engineering Management Plan (SEMP)
– The basics

• Foreword
Contents
1. SCOPE
2. APPLICABLE DOCUMENTS
3. TECHNICAL PROGRAM PLANNING AND
CONTROL
4. SYSTEMS ENGINEERING PROCESS
5. ENGINEERING SPECIALTY INTEGRATION
6. NOTES

http://sparc.airtime.co.uk/users/wysywig/semp.htm
5
 SEMP - Technical Program
Planning and Control
• Technical program planning and control identifies
organizational responsibilities and authority for system
engineering management, including control of
subcontracted engineering:
– levels of control established for performance and design
requirements and control of the method used;
– technical program assurance methods (verification-ITAD);
– plans and schedules for design and technical program reviews;
– control of documentation;
– design approval and certification;
• Bottom line – describe how you are going to control the
engineering requirement and development process

6
 SEMP – Systems Engineering
Process
• This section contains a detailed description of
the process to be used, including the specific
tailoring of the process to the requirements of
the system and project:
– the procedures to be used in implementing the
process;
– in-house documentation;
– the trade study methodology;
– the types of mathematical and/or simulation models to
be used for system and cost effectiveness
evaluations;
– and the generation of specifications
• Bottom line : describe the tools and
methodologies you are going to use
7
 SEMP – Engineering Specialty
Integration
• The integration and coordination of the program efforts for
engineering specialty areas (structures, software, avionics, GN&C,
mechanisms, electrical power, hydraulics, etc…), to achieve a best
mix of the technical/performance values incorporated in the
contract, shall be described with the detailed specialty program
(project) plans being summarized and/or referenced, as appropriate.

• The specialty efforts and parameters will be integrated into the

system engineering process at each iteration. The process will be
described and considerations taken defined.
• You may have to reduce the performance or capabilities of one part
of the system in order to maximize the total performance of the
system
– For example, you might have to reduce the cargo carrying capacity of
the system in order to reduce weight enough to meet range
– You might have to reduce passenger volume in order to make room for
fuel or cargo
• Bottom line: describe how you are going to integrate different
technical disciplines
8
Change or Configuration Control

9
 Change and Configuration Management
• If you are going to have an orderly design and development process with
many designers working simultaneously, change or configuration
management (control) is critical
• Uncoordinated change can create havoc in a development
• Uncontrolled change can have large impacts to a project’s ability to maintain
cost, schedule or other technical performance goals such as weight
• Once a baseline has been established at a milestone (such as a design
review) then it is critical that ALL CHANGES to that baseline are reviewed
for technical, cost and schedule impact and communicated to all affected
parties
– Changes need to be approved by the responsible official – typically the chief
engineer and the project/program manager
• This review and approval typically takes place at a Configuration Control
Board (CCB) where representatives of all of the key organizations are
present
– The change is documented on a Change Request (CR) form with attached
presentations and engineering products such as drawings
– A presentation is normally made by the person/organization proposing the
change
• Once approved, paper is signed to authorize a change to drawings, ICDs or
other engineering products, to change allocation of moneys or to change
schedules
• CCB’s may issue actions to gather additional data prior to making a decision
or to initiate subsequent changes 10
 A Guide to Successfully Running/Presenting to CCBs

• When presenting to a CCB

– Make a clear and complete statement of the problem you are trying to
solve
– Make a clear and complete statement of your proposed solution and it’s
entire impact
– Try to understand the potential impacts and discuss the change with
people and organizations that are impacted by the change ahead of
time
– Keep to the topic on hand
– Just the facts

• When running a CCB

– Hold them regularly
– Listen carefully, ask lots of questions
– Know what your budget and schedule flexibilities are before you commit
– Gather reasonable data and then MAKE A DECISION – don’t keep
sending people back for more data or more options

11
Systems Engineering Capability Maturity Model
Integration (CMMI)

12
 CMM
• The Capability Maturity Model (CMM) is a way to develop and
refine an organization's processes. The first CMM was for the
purpose of developing and refining software development
processes. A maturity model is a structured collection of elements
that describe characteristics of effective processes. A maturity
model provides:
• a place to start
• the benefit of a community’s prior experiences
• a common language and a shared vision
• a framework for prioritizing actions
• a way to define what improvement means for your organization.
• A maturity model can be used as a benchmark for assessing
different organizations for equivalent comparison. The model
describes the maturity of the company based upon the project the
company is handling and the related clients.
• The software CMM was generalized into a systems engineering
CMM and then rewritten as the CMMI
• This standard has been developed and maintained by Carnegie
Mellon University
Courtesy of Wikipedia 13
 Levels of CMMI
• Level 1 – Initial - can do systems engineering,
depends on quality of people, hit or miss
• Level 2 – Repeatable – can repeat the process
(e.g. write requirements, do a PDR, CDR, etc…)
• Level 3 – Defined – procedures are well
documented
• Level 4 – Quantitatively Managed – metrics are
maintained to evaluate how well you are doing
• Level 5 – Optimized – you use metrics to identify
problems and apply resources to maximize the
positive outcome

14
 Value of CMMI
• Important for self-evaluation – how are you doing ?
– I initiated a CMMI audit of Shuttle Systems Engineering and
Integration after I took over after the accident
– It identified the key weaknesses
– With limited resources at our disposal, I focused our efforts on
fixing the key weaknesses found by the audit rather than trying
to fix everything
• You can use it to help you continuously improve
• Unfortunately, People use it as a gimmick
– “we’re level 4….”
– You can almost hear them saying “nyaah,nyaah,nyaah,nyaah”
• To go to the source on CMMI, reference
http://www.sei.cmu.edu/cmmi/

15
Key Performance Parameters (KPP) and
Technical Performance Metrics (TPMs)

16
 KPPs and TPMs
• Key Performance Parameters (KPPs)
– Are the values that a system must attain to meet its most
important requirements
– Example of key performance parameters would be
• Range
• Cargo/Passengers
– The DOD often goes beyond establishing a requirements
specification by actually prioritizing the requirements. The
prioritization is accomplished by defining what is called Key
Performance Parameters (KPP), which are requirements that
are so critical to the program, that the failure to meet any KPP
would be grounds for program termination.

17
 KPPs and TPMs
• Technical Performance Metrics
– Are the values expressing how well a system is moving towards
meeting its KPPs
– , it needs to measure something of importance to the program--a
KPP that is essential to proper system operation in order to meet
a mission requirement. Some programs may track a few of these
TPMs or maybe a few dozen.
– Contractors may have many more TPMs in order to track
derived requirements and to ensure proper technical progress
toward major system requirements.
– A typical KPP may be system or subsystem weight. A weight
TPM may have an objective (defined as the goal or required
value at the end of the technical effort) or both an objective and
a threshold (defined as the limiting acceptable value that if not
met can jeopardize the project).
– A TPM can also have tolerance bands that show the allowed
variation, which is based on the projected estimation

18
Master Verification Plan

19
 Master Verification Plan (MVP)
• Identifies the set of facilities and major test
environments to verify requirements
• May show relationship between test and
analysis by a network diagram (see next few
examples)
– Very important for integrated verification where
multiple independent projects need to cooperate to
accomplish verification
• Identifies each major system requirement and its
verification method (Inspection, Test, Analysis,
Demonstration)
20
21
22
 Risk Management

In the end, it always comes to the point where it is

time to shoot the engineers and go fly the airplane
- Boeing President

In Internet speak – T2SE – Time to Shoot the

Engineers

23
 Risk Management
• Use a 5x5 matrix to rate – Very Similar To Hazard Analysis
• Risk mitigation plans established for yellow and red risks
• Allocate resources (money, schedule) to risks –
– RSS resources to avoid having to hold too much in reserve for
covering risks – treat each risk as an independent random
variable
– Multiply risk resources by probability of risk coming true
• Top X list – weekly or monthly review the top risks and their
mitigation plans
• Reducible vs irreducible risk
– Reducible risks are those where you can work harder to reduce
or eliminate them
– Irreducible risks – those were no matter how hard you work, you
can’t reduce them
• Good example – combined environments – where you
cannot expose an entire flight vehicle structure to the
combined thermal, inertial, pressure load simultaneously
before you actually fly
24
 Advice on Risk
• Irreducible risk – be bold
• Reducible risk – be conservative
– Control the things you can

• Problem – people say – irreducible risk is greater than

reducible so why work so hard on reducible
– Stupid to lose crew/mission for something you could’ve fixed
– Irreducible may be less defined – may not be as large as you or
assume
• In our history – people generally die from things we
could’ve anticipated and fixed

25
26
27
28
US Army
Pre-mission
Risk Evaluation
For Helicopter Missions

29
Any Questions ?

30