Inf4290-Re Qa
Inf4290-Re Qa
Mehrdad Sabetzadeh
April 7, 2010
What is a Requirement?
➜ No universally agreed definition
➥ But intuitively: A requirement is something that
someone needs in order to solve a problem or achieve
an objective
➜ IEEE Definition
"A condition or capability that must be met or
possessed by a system or system component to
satisfy a contract, standard, specification, or
other formally imposed document. The set of all
requirements forms the basis for subsequent
development of the system or system component".
[IEEE Std 830-1998]
2
Types of Requirements
➜ Functional requirements:
➥ What the system does: the interactions between the system and
its environment
➥ Should be as independent as possible from implementation
➜ Non-functional requirements:
➥ Observable aspects of the system that are not directly related
to functional behavior
➣ e.g. performance, reliability
➜ Safety/security requirements ("shall not" properties)
➥ A kind of nonfunctional requirement
➥ Behavior the system must never exhibit
➣ e.g. it must be impossible to apply reverse thrust in mid-flight
➜ Constraints (Pseudo requirements)
➥ Imposed by the client or environment in which the system
operates
➥ Often concern the technology to be used (language, operating
system, middleware etc.)
3
What do Requirements Engineers do?
Focus of
today’s
lecture
4
Domain Understanding and Elicitation
➜ Domain Understanding
➥ Identify the problem / opportunity
➥ Understand the organizational context
➥ Identify stakeholders
➥ Specify the scope of the software system
➜ Requirements Elicitation
➥ Elaborate stakeholders’ goals
➥ Identify alternative options for satisfying these goals
➥ Identify organizational / technical constraints and
assumptions
➥ Define typical scenarios illustrating desired interactions
between the system and its environment
➥ Articulate the requirements the system should meet in
order to conform to all of the above
5
Evaluation and Negotiation
➜ Evaluation Tasks
➥ Identify conflicting concerns and resolve them
➣ Conflicts often arise from multiple viewpoints and
different expectations.
➥ Identify and assess risks associated with the system
➣ Budget, schedule, etc.
➥ Compare the alternative options identified during
elicitation and select best options
➥ Prioritize the requirements
➣ give weight to requirements that are essential to the
system
➣ drop lower-priority requirements that would together
exceed budgets and deadlines
➜ All the above involve some kind of negotiation to
arrive at a consensus
➜ Output is a preliminary requirements document
6
Specification and Documentation
➜ Goal
➥ Detailing, structuring, and documenting
the agreed characteristics of the system
as they emerge from the evaluation and negotiation
activity
➜ Contents
➥ A detailed and precise description of the following:
➣ Objectives
➣ Concept definitions
➣ Relevant domain properties
➣ Responsibilities
➣ System requirements
➣ Software requirements
➣ Environment assumptions
7
Requirements Quality Assurance (QA)
➜ Goal:
➥ Quality assurance is aimed at
checking that the items specified
in a Requirements Document (RD)
meet the desired qualities attributes
➣ Completeness, consistency, adequacy
➜ Why is Requirements QA important?
➥ Because cost of requirements failures is very high
“Finding and fixing a software problem after delivery
is often 100 times more expensive than finding and
fixing it during the requirements and design
phase.” [Boehm and Basili]
➜ Major Issues
11
Common Requirements Defects
➜ Minor Issues
12
Techniques for Requirements QA
➜ Prototyping
➥ Partial implementation for validation with customers
➜ Simulation
➥ Simulates the environment of a system and checks
the appropriateness of specified behaviors
➜ Formal checks
➥ Completeness and sanity checks
➥ Model checking and theorem proving
13
Prototyping
➜ Definitions
➥ “A software prototype is a partial implementation constructed primarily to enable
customers, users, or developers to learn more about a problem or its
solution.” [Davis 1990]
➥ “Prototyping is the process of building a working model of the system” [Agresti
1986]
➜ Approaches to prototyping
➥ Presentation Prototypes
➣ explain, demonstrate and inform – then throw away
➣ e.g. used for proof of concept; explaining design features; etc.
➥ Exploratory Prototypes
➣ used to determine problems, elicit needs, clarify goals, compare design
options
➣ informal, unstructured and thrown away
➥ Breadboards or Experimental Prototypes
➣ explore technical feasibility; test suitability of a technology
➣ Typically no user/customer involvement
➥ Evolutionary (e.g. “operational prototypes”, “pilot systems”):
➣ development seen as continuous process of adapting the system
➣ prototype is an early deliverable, to be continually improved.
14
Inspections, Reviews, Walkthroughs
➜ Note: these terms are not widely agreed
➥ Formality
➣ informal: from meetings over coffee, to team get-togethers
➣ formal: scheduled meetings, prepared participants, defined agenda, specific
format, documented output
➥ Management reviews
➣ E.g. preliminary design review, critical design review, …
➣ Used to provide confidence that the design is sound
➣ Attended by management and sponsors (customers)
➣ Usually a “dog-and-pony show”
➥ Walkthroughs
➣ developer technique (usually informal)
➣ used by development teams to improve quality of product
➣ focus is on finding defects
➥ (Fagan) Inspections
➣ a process management tool (always formal)
➣ used to improve quality of the development process
➣ collect defect data to analyze the quality of the process
➣ written output is important
➣ major role in training junior staff and transferring expertise
15
Benefits of Formal Inspections
➜ Formal inspection works well for programming:
➥ For applications programming:
➣ more effective than testing
➣ most reviewed programs run correctly first time
➠ compare: 10-50 attempts for test/debug approach
➥ Data from large projects
➣ error reduction by a factor of 5; (10 in some reported cases)
➣ improvement in productivity: 14% to 25%
➣ percentage of errors found by inspection: 58% to 82%
➣ cost reduction of 50%-80% for V&V (even including cost of inspection)
➥ Effects on staff competence:
➣ increased morale, reduced turnover
➣ better estimation and scheduling (more knowledge about defect
profiles)
➣ better management recognition of staff ability
➜ These benefits have been shown to apply to requirements inspections too
16
Inspection Constraints
➜ Size
➥ enough people so that all the relevant expertise is available
➥ min: 3 (4 if author is present)
➥ max: 7 (less if leader is inexperienced)
➜ Duration
➥ never more than 2 hours; concentration will flag if longer
➜ Output
➥ all reviewers must agree on the result: accept; re-work; re-inspect;
➥ all findings should be documented
➣ summary report (for management); detailed list of issues
➜ Scope
➥ focus on small part of a design, not the whole thing
➜ Timing
➥ Examines a product once its author has finished it
➣ not too soon: product not ready - find problems the author is already aware of
➣ not too late: product in use - errors are now very costly to fix
➜ Purpose
➥ Remember the biggest gains come from fixing the process
➥ collect data to help you not to make the same errors next time
17
Inspection Guidelines
➜ Prior to the review
➥ schedule Formal Reviews into the project planning
➥ train all reviewers
➥ ensure all attendees prepare in advance
➜ During the review
➥ review the product, not its author
➥ keep comments constructive, professional and task-focused
➥ stick to the agenda
➥ leader must prevent drift
➥ limit debate and rebuttal
➥ record issues for later discussion/resolution
➥ identify problems but don’t try to solve them
➥ take written notes
➜ After the review
➥ review the review process!
18
Inspection Guidelines
➜ Possibilities
➥ specialists in reviewing (e.g. QA people)
➥ people from the same team as the author
➥ people invited for specialist expertise
➥ people with an interest in the product
➥ visitors who have something to contribute
➥ people from other parts of the organization
➜ Exclude
➥ anyone responsible for reviewing the author
➣ i.e. line manager, appraiser, etc.
➥ anyone with known personality clashes with other reviewers
➥ anyone who is not qualified to contribute
➥ all management
➥ anyone whose presence creates a conflict of interest
19
Structuring the Inspection
➜ Can structure the inspection in different ways
➥ Free mode
➣ Rely on expertise of the reviewers
➥ Checklists
➣ uses a checklist of questions/issues
➣ checklists tailored to the kind of document
➥ Active reviews (perspective-based reading)
➣ each reviewer is given a specific process to follow for defect
search and reads for a specific purpose
➣ effectively different reviewers take different perspectives
➜ The differences may matter
➥ Some studies indicate that:
➣ active reviews find more faults than free mode or checklist
methods
➣ no effective difference between free mode and checklist
methods
➣ the inspection meeting might be superfluous!
20
Inspection Checklists
➜ Example
21
Inspection Checklists
➜ Example (continued)
22
Simulation
➜ Definition
➥ Mimicking possible behaviors of the environment and executing a model
of the software system to respond to these events
➥ Often accompanied by animation
➣ a visual representation that shows how the system evolves as the
model is being executed
➜ Prerequisite
➥ An (abstract) executable model of the system needs to be built
➜ Types of visualization
➥ Textual: input events are entered as textual commands; model
reactions are displayed as execution traces.
➥ Diagrammatic: input events are entered by event selection among those
applicable in the current state; model reactions are displayed as
tokens progressing along the model diagrams
➥ Domain-specific visualization: input events are entered through domain
-specific control devices displayed on the screen; model reactions are
displayed as new values on domain-specific control panels
23
Formal Checks
➜ A wide range of mathematically defined checks
that can be automated by tools
➥ The specification must be formal.
➜ Example 1
➥ Checking disjointness of input cases
➣ For every two cases C1, C2 we must have C1 ∧ C2 = false
24
Formal Checks (continued)
➜ Example 2
➥ Checking completeness of input cases
➣ For C1, C2, … Cn we must have: C1 ∨ C2 ∨ · · · ∨ Cn = true
25
References
➜ Lecture contents adapted from:
➥ Requirements Engineering From System Goals to UML
Models to Software Specifications by Axel van
Lamsweerde
➣ Chapter 5 Requirements Quality Assurance
➥ Course slides from the Requirements Engineering course
at the University of Toronto by Steve Easterbrook
➣ http://www.cs.toronto.edu/~sme/CSC2106S/
➜ More Resources
➥ [IEEE Std 830-1998] IEEE Recommended Practice for Software
Requirements Specifications -Description
➥ [Boehm&Basili] Software Defect Reduction Top 10 List
➥ [Blum] Software Engineering: A Holistic View
26