[P] ROOK 1986 - Controlling software projects
[P] ROOK 1986 - Controlling software projects
by Paul Rook
In recent years the software industry has seen the increasing tured design and structured analysis.
imposition of structure and discipline on technical development Structured programming provides
activities in an attempt to improve the efficiency of software rules for choosing the building blocks for
programs. Structured design helps the
development and the reliability of the software produced. The clear designerto distinguish between good and
emphasis in the modern approach to software engineering is to focus bad designs. Structured analysis assists in
attention on the overall development process and the co-ordination of the production of a specification that is
all aspects of software development. This paper examines the correct and consistent and can be deter-
principles of managing and successfully controlling software mined to be complete. The introduction
in turn of each of these three techniques
development from a software engineering basis. has thrown up problems which have been
introduced through lack of discipline in
1 Introduction development and ought to be managed the earlier stages of the development
and controlled using very similar tech- process.
The management of a large software niques to those used in hardware engin-
eering development. The genuine In addition to this stage by stage attack
development is a complex and intrin-
dissimilarities listed above are the very on the problems of development, it is
sically difficult task: a large software sys-
factors which make an engineering clear that all activities will have problems
tem is itself very complex and its
approach much more critical for software unless the goals of each activity are clearly
production may involve hundreds of man-
development. stated and set within the context of the
years of skilled effort with correspond-
structure for the whole project.
ingly large budgets. Contributing to the difficulties of soft-
ware management is the much publicised The clear emphasis in the modern
Nearly every software development
view of the programmer as the unbridled approach to software engineering is to
project is faced with numerous diffi-
genius, whose creative process will be focus attention on the overall develop-
culties. When a project is successful it is
stifled by any of the recommended pro- ment process. This is the aim of struc-
not because there were no problems but
ject management controls, design stan- tured software development, which
because the problems were overcome.
dards and programming standards. breaks down the project into a series of
Many of the problems are technical but
Forced to contend with this view is the distinct phases, each with well defined
often the critical ones are managerial.
software manager, often a recently pro- goals, the achievement of which can be
Software development depends on docu-
moted analyst or programmer who has verified, ensuring a sound foundation for
mentation and communication — it is
worked on projects managed as a collec- the succeeding phase. It also breaks down
only structured if a structure is imposed
tion of creative artists doing their indepen- the work to be performed into a series of
and controlled. Everything that is done
dent thing. Management's job in these discrete manageable packages, and cre-
right in software development is done
projects was to try, somehow, to steer this ates the basis for the appropriate
early — there is very little opportunity for
collection of individualists in a common organisational structure. This allows over-
catching up when things are discovered
direction so that their products would all planning of 'how' the software is going
to be going wrong later in the
accomplish the project goals, be able to to be developed as well as considering
development.
interface with each other, be finished 'what' is going to be developed as the
There is much discussion about com-
within the project cost and schedule con- product.
parisons between managing software
development and managing hardware straints, and, with a little luck, come rea-
development. There are genuine dif- sonably close to accomplishing what the 3 Computer-based tools
ferences between hardware and software, customer had in mind for the software.
as follows: Such a software manager has been well An equally important development has
grounded in how a project should not be been the introduction of computer-based
• Software has no physical appearance. managed, but has had little exposure or tools to assist with specific tasks. The
• Few software quality metrics exist. training in the use of effective software earliest tools were concerned with the
• Software has much higher complexity management techniques. production of code. These have been fol-
than hardware. lowed by tools which assist, for example,
• It is deceptively easy to introduce specification, design, estimating, plan-
changes into software. 2 Structuring software ning, documentation and configuration
• Effects of software change propagate development management. In fact tools are now avail-
explosively. able to support most of the software
• Software includes data as well as To tackle these problems, the software development activities.
logic. industry has seen the increasing imposi- The right tools assist in increasing pro-
• Software development makes very tion of discipline on technical develop- ductivity and visibility of work achieved,
little use of pre-existing components. ment activities in an attempt to improve provide a source of data for future pro-
the reliability of software development. posal preparation, estimation and project
However, in many important ways soft- Thus we have seen, in turn, the tech- planning, and maintain continuity
ware development is like hardware niques of structured programming, struc- between projects. They also provide auto-
Software Engineering Journal January 1986
mated testing and reduce iteration of odology' to describe a systematic set of detected late, inadequate control, techni-
work and thus aid improved quality. Tools procedures followed from the original cal skill, support and visibility).
are especially useful in detecting errors conception of a system through the speci-
early, when they are less expensive to cor- fication, design, implementation, oper- If the project is to be successful, then
rect, thus leading to a more successful ation and evolution of the software in that potential risks must be identified, and
software project and product. system. A methodology not only includes eliminated or controlled. A control system
Although tools alone will not ensure technical methods to assist in the critical for a project is based on the usual prin-
success, the selection and installation of tasks of problem solving, documentation, ciple of establishing suitable feedback
the right set of tools is seen to be neces- analysis, design, coding, testing and con- loop(s) to ensure that the controlled sys-
sary for an effective software development figuration management, but also includes tem is oriented to its objective. The objec-
project. management procedures to control the tive of a software development project is
development process and the deploy- to produce the correct product on time
4 The management of complexity ment of the technical methods. and to budget.
The management and technical Fig. 1 illustrates, in a simplified form, a
Thus the key to the management of the aspects of the methodology support and project control system for software
complex task of large software develop- gain strength from each other: the techni- development. Technical development is
ment is twofold: cal methods provide the basis needed for what is controlled and project manage-
effective managerial control, while the ment is the controller. Estimating is a pre-
• reducing complexity by imposing a management procedures provide the requisite for control, and a number of
structure on to the process organisation and resources which enable feedback loops are set up which operate
• using computer-based tools to make the technical development to proceed via status and progress reports to com-
the remaining complexity more tractable. effectively. Tools support the method- pare actual progress with the plans based
ology and provide the information needed on the estimates.
5 Software development by project management. The feedback loops operate directly
methodology from the technical development and also
6 Project control from the quality and configuration man-
In planning how to develop the software, it agement systems. Fig. 1 illustrates a con-
is the responsibility of project manage- The software development process is tinuous process, as indicated by the inner
ment to ensure that a coherent system of inherently subject to risks which are mani- product loop of feedback of intermediate
methods and tools is chosen, integrated fested as financial failures (time scale development products to the activities of
and supported. overrun, budget overrun) and technical technical development. The quality and
However, differences in organisation failures (failure to meet requirements, configuration management systems
structures, applications and existing over/under-engineered). The sources of operate continuously in the development
approaches make it impractical to pre- risk can be placed in three main process, not only on the products finally
scribe a single scheme that can be univer- categories: delivered to the customer.
sally followed. Methods, tools, manage- While the inner loop represents the
ment practices or any other element of • perturbations (requirement changes, work on the product, the outer feedback
the total development environment can- detection of problems, errors and loops represent the basis for control. Con-
not be chosen without considering each failures) trol consists of obtaining information to
element in its relationship to the other • personnel (wrong people available, make decisions and ensuring timely
parts of the development system. too many/too few people available) detection and correction of errors, thus
Software engineering has introduced • project environment (undefined controlling time scale and budget and
the term 'software development meth- methodology, unknown quality, errors minimising technical risks.
INTERMEDIATE
DEVELOPMENT PRODUCTS
CUSTOMER
REQUIREMENTS
1
HIGHER-LEVEL i
MANAGEMENT
DIRECTIVES
^(STANDARDS A N D l -
^ j PROCEDURES ~"
REPORTS TO HIGHER-LEVEL
MANAGEMENT AND
CUSTOMER i r
.CHANGE.
REQUESTS
r i r
i "1
CHANGE
CONTROL
DECISION (AND
—*• WORK
WORK
ASSIGN MENTS TECHNICAL
HIGHER-LEVEL MAKING REPLANNING) DEFINITION DEVELOPMENT
MANAGEMENT DIR8:CT»VES AND CONSTRAINTS
j i 1
\ \
1f
V ORGANISATION
ESTIMATING
I
\ STANDARDS AND
PROCEDURES
PROBLEM RECORDING OF
DENTIFICATION PROJECT DATA
i
improve visibility.
In practice, on a large-scale project, the
precise breakpoints between the project
phases are not easy to define clearly and
depend to some extent on project man-
agement decision. Because completely
rigid phase control is impractical, status
and risk analysis at milestones is par-
ticularly important. This can only be
obtained from a system of technical
Fig. 3 The stages in software development confidence
reviews.
However, once this reality is recog-
nised, it does not lead to the conclusion plan, with milestones, resources, respon- 9 Software development work
that the life cycle model is impractical. sibilities, schedules and major activities.
Having escaped the simplistic interpreta- Defined standards and procedures. In the same way that the progress of a
tion, the life cycle model does represent a • Requirement specification phase: A software development project may be
realistic recognition of what is actually complete, validated specification of the partitioned into a number of discrete
involved in the technical work of software required functions, interfaces and phases, so may the technical work
development. performance for the software product. A involved be divided into a number of
Phases do indeed have to be imposed detailed project plan. clearly identified 'activities'.
by project management; they will not hap- • Structural design phase: A complete, A close parallel has been deliberately
pen of their own accord. The definitions verified specification of the overall hard- adopted between the names assigned to
and concepts in the life cycle model rep- ware-software architecture, control struc- the project phases and the technical
resent the best current understanding of ture and data structure for the software activities, respectively. This means that, in
software development methodology — product, along with such other necessary most cases, the name of a development
gained from experience in applying soft- components as draft user's manuals and phase, structural design, for example,
ware engineering to development test plans. indicates the principal activity taking
projects. • Detailed design phase: A complete, place within that phase. This is not to say
verified specification of the control struc- that structural design is the only activity
These definitions are the worked-out
ture, data structure, interface relations, taking place during the structural design
basis for real control of software develop-
sizing, key algorithms and assumptions phase.
ment, but that control has to be explicitly
planned, based on an implemented meth- for each program component. Conversely, not only must significant
odology actually used by the develop- • Code and unit test phase: A complete, initial structural design work be per-
ment staff, and actually applied. It does verified set of program components. formed prior to the structural design
not happen naturally, as is apparent from • Integration and test phase: A properly phase, but also there must be a con-
the response from some projects that the functioning software product. tinuing activity to deal with updates to the
life cycle model does not correspond to • Software acceptance test phase: An design during subsequent phases.
reality. Project management has to make accepted software product handed over Similarly, although coding a module
its version of the life cycle model realistic. to the customer. does not properly commence before the
• Maintenance phase: A fully function- completion of the detailed design of that
8.3 Software development life ing update of the software product. This module, some programming activities
cycle phases goal is repeated for each update, which must be performed during the earlier
follows the complete development phases, such as planning coding
Listed below are the baseline outputs of sequence each time. methods and facilities, the acquisition
each phase of the software development • Project termination phase: A com- and installation of tools and, in some
life cycle: pleted project history document compar- cases, exploratory investigations into
ing estimates and plans with actual algorithms and operations.
• Project initiation phase: A validated development schedule and costs as a Errors detected in the integration phase
system architecture, founded on a design contribution to the accumulated database will require code and unit test activity even
study with basic hardware-software of experience. though the phase of that name has been
allocations, and an approved concept of • Product phase-out: A clean transition completed.
operation including basic human- of the functions performed by the product In general, all activities continue across
machine allocations. A top-level project to its successors (if any). all phases of the project, although the
Software Engineering Journal January 1986 11
emphasis shifts from activity to activity as importantly, on a project of even a reason- W&T is checking the correctness of the
the project proceeds from phase to able size, each activity is necessarily a full- products of each phase (baselines) and is
phase. It follows that, in a large software time job, or more. performed by the software development
development, each activity should be It is hard for the project manager to staff. The activity should, as far as pos-
staffed by a distinct group of people, delegate the project management tasks sible, be carried out by staff within the
whose numbers might expand and con- to allow time for technical work. It is project organisation, but not by the orig-
tract as the emphasis of the project impossible for the technical controller to inators of the work. For this reason it is the
changes but whose existence is identifi- delegate technical control duties without only development activity which may be
able from project start to project end. This compromising the conceptual integrity of the responsibility of a series of different
is illustrated in the example shown in the product. It is sometimes possible to teams as the project proceeds through
Fig. 4. run a project with the technical control the life cycle phases.
exercised by the senior manager in
9.1 Technical control charge of the project and almost all pro-
9.4 Quality assurance
ject management tasks delegated to a
Since the technical activities are per- second-in-command. It is much more
Quality asssurance (QA) is checking
formed by members of a number of usual for the project manager to be in
the correctness of the procedures being
teams, it is vital to ensure the overall tech- command, with the technical controller
followed, i.e. whether the development
nical correctness of the product, which having the technical authority. In this case
staff are following the intended pro-
can be distinguished from such concerns it is important that the technical controller
cedures (in all their work, not just the
as schedule, budget, organisation, staff- does have enough authority for decisions
W&T activities). This is carried out by QA
ing etc. which are solely the responsibility without being in management line above staff either from a separate QA depart-
of project management. This concern all the project teams. ment or from staff assigned to QA work
with technical matters is referred to as within the project. The checking of pro-
'technical control'. 9.2 Quality system cedures is backed by audits (spot checks)
Technical control is regarded as part of of the quality (and conformance to stan-
technical development (see Fig. 1) and is In the context of product development dards) of the products to find out if the
defined as the continuous process of the word 'quality' is defined as the degree procedures are effective. Generally the
making certain that what is being pro- of conformance of the product to its QA staff provide an independent voice on
duced is technically correct, coherentand stated requirements, i.e. 'fitness for pur- all quality issues, especially on the setting
consistent. pose'. This definition is applied to the up of standards and procedures at the
It includes planning ahead for all the intermediate products of the develop- beginning of the project (i.e. 'how' the pro-
necessary modelling and testing. Its role ment as well as to the final product. The ject will develop the 'what' defined in the
is strategic in that it makes certain that the development process is fundamental to requirement specification). The respon-
overall technical integrity of the product is the ability of the project to produce prod- sibilities of the QA staff are:
not lost in the tactics of the individual ucts of acceptable quality. Quality is built
technical activities. It requires an overall into the product by the activities of the • advising on standards and
technical authority but does not neces- software development staff as a continu- procedures
sarily imply managerial authority over the ous process of building the product to the • monitoring the procedures actually
development staff. When sub-contractors specified quality. employed on the project
are involved in the project, the activity of • auditing and certifying the quality of
Quality is everybody's responsibility —
technical control becomes even more products achieved.
it cannot be added by any testing or con-
important in co-ordinating the technical
trol on the products of phases. Such test-
aspects of all the work between the sub-
ing and quality control activities do,
contractors and the integrity of the sub-
however, provide early warning of prob-
9.5 Configuration management
contract products. system
lems; changes can be made at much
Primary examples of technical control lower cost than in the later stages of
are the maintenance of the integrity of the The successful realisation of a software
development, provided, as always, that
design in the presence of changes follow- product requires the strictest control over
proper change control procedures are
ing the completion of the structural the defining, describing and supporting
followed.
design phase, and test planning. Test documentation and the software code
The quality system comprises two dis-
planning is a strategic activity from the constituting the product. It is inevitable
tinct activities: verification, validation and
very start of the project which defines and that the definition will be subject to contin-
test (W&T), and quality assurance (QA).
co-ordinates all the test methods, tools uous pressure for change over the life
and techniques to be used throughout the cycle of the product, to correct errors,
9.3 Verification, validation and
life cycle. It also identifies critical compo- introduce improvements and respond to
test
nents that need the most testing, what test the evolving requirements of the mar-
data is required, and when it is to be ketplace. Configuration management
The terms are defined as follows:
prepared. provides the disciplines required to pre-
vent the chaos of uncontrolled change.
While it can be seen to be difficult to • Verification: To establish the corre-
A comprehensive approach to config-
separate the two activities of project man- spondence between a software product
uration management requires:
agement and technical control and it is (documentation or code) and its specifi-
reasonable on very small projects for the cation — 'Are we building the product
project manager to undertake both right?' • clear identification of software items
activities, on large projects such a com- • Validation: To establish the fitness of and documents, and their successive ver-
bination of roles is very rarely workable. a software product for its operational mis- sions and editions
Firstly, it is rare to find people who com- sion — 'Are we building the right • definition of the configuration of soft-
bine both the strong management talent product?' ware products, and their related config-
and strong technical talent necessary for • Testing: The actual running of code to uration items
large projects. Secondly, and more produce test results. • physical control over the master files
12 Software Engineering Journal January 1986
of software code and documentation entirely of documentation or of docu- interface, cost/schedule performance
• control of the introduction of changes mentation and code. Furthermore, during management, management reviews and
to these files by a change control board the design phases, documents are the audits, and includes acquisition of man-
and a set of change procedures sole means by which the successive agement tools.
• maintenance of a system of config- stages of the design process are • Technical control: Responsibility for
uration records, reflecting the definition of recorded, and against which each phase the technical correctness and quality of
products in the field. is validated. So much of the output of a the complete product. Responsibility for
software project is in the form of docu- maintaining the integrity of the whole
The output of each development phase mentation that it is impossible to separate design during the detailed design, pro-
should be verified and validated against the scheduling of the documentation gramming and testing phases. Specifica-
the relevant preceding baselines. Config- constituting the baselines from that of the tion, review and update of integration test
uration management disciplines ensure project as a whole. Therefore careful and acceptance test plans and pro-
that all necessary corrections are intro- attention to the planning, structure, con- cedures. Acquisition of requirements and
duced before this output, in turn, is base- tent, preparation, presentation and con- design verification and validation tools.
lined and that only up-to-date definitions trol of documentation is vital. Acquisition and support of test drivers,
of baselines are used by subsequent Documentation produced by the soft- test tools and test data.
phases. The configuration management ware development process may: • Requirement specification: Deter-
system should be able to react to the time mination, specification, review and
scales needed by different phases of the • define the software product in terms update of software functional, perform-
project. of requirement and design specifications ance, interface and verification require-
Once a baseline has been formally • describe the product to the customer ments, including acquisition of
established its contents may only be or to current or future members of the requirements analysis and specification
changed by the operation of the formal development team tools. Development of requirement speci-
change control process. This has the fol- • support the product in the field in the fication level defining and describing
lowing advantages: form of the user's manual, operator's documentation. A continuing responsi-
manual and maintenance manual. bility for communication between cus-
• No changes are made thereafter with- tomer requirements and the technical
out the agreement of all interested parties. 9.7 Software development model development.
• The higher procedural threshold for • Structural design: Determination,
change tends to stabilise the product. Having discussed all the activities of a specification, review and update of hard-
• There is always available a definitive software development project, the full list ware-software architecture, software
version of the product, or of any of the of ten activities covering all the manage- design and database design, including
controlled intermediate products ment and technical work can be defined acquisition of design tools. Development
(baselines). as follows: of structural design level defining
documentation.
9.6 Documentation • Project management: Project level • Detailed design: Detailed design of
management functions. Includes project individual computer program compo-
The output of each phase of the whole level planning and control, contract and nents. Development of detail design level
software development project consists sub-contract management, customer defining documentation. When a signifi-
Control
Specification
Structural
DMlgn
k L
01mm
Design
Unit Tast
Verification
-
and Test —
Manuals
Management
A •••:•
* -
Total ___-- s
" < ' A/*
project estimating, project management, project management, project management, project management, project management, project management,
planning, scheduling, project planning, status monitoring, status monitoring, status monitoring, status monitoring, status monitoring,
procedures, contracts, liaison etc. contracts, liaison etc. contracts, liaison etc. contracts, liaison etc. contracts, liaison etc. contracts, liaison etc.
organisation etc.
technical strategy, system models and risk design quality, models design integrity, design integrity, design integrity, design integrity,
technical plans, analysis, acceptance and risk analysis, draft detailed test plans, detailed test plans, support test tools, support test tools,
technical standards test plan, acquire V and test plans, acquire test acquire test tools install test tools monitor testing monitor acceptance
V tools for reqmnts and tools
design, top-level test
plan
analyse requirements, analyse existing update requirements update requirements update requirements update requirements update requirements
determine user needs system, determine user
needs, integrate
document and iterate
requirements
design planning develop basic develop structural update design update design update design update design
architecture, models, design, models,
prototypes prototypes
identify programming prototypes of models, algorithms detailed design, update detailed design update detailed design update detailed desig
methods and resources algorithms, team investigation, team component
planning planning documentation
identify programming identify programming acquire programming integration planning code and unit test integrate software, update code
methods and resources tools, team planning tools and utilities, team update code
planning
V and V requirements V and V specification V and V structural V and V detailed V and V top portions of perform product test, V perform acceptance
design design, code, V and V design and V design changes test, V and V design
V and V design changes changes
changes
define user's manual outline portions of draft user's, operator's draft maintenance full draft users and final users, operators acceptance of manua
user's manual manuals, outline manual operator's manuals and maintenance
maintenance manual manuals