0% found this document useful (0 votes)
48 views26 pages

Inf4290-Re Qa

The document discusses requirements quality assurance. It defines a requirement and lists types of requirements like functional, non-functional, safety/security. It describes what requirements engineers do like domain understanding, elicitation, evaluation, negotiation, specification and documentation. It discusses common requirement defects and techniques for quality assurance like prototyping, inspections/reviews, and formal checks. Inspections are particularly effective for finding defects and improving productivity.

Uploaded by

markyray
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
48 views26 pages

Inf4290-Re Qa

The document discusses requirements quality assurance. It defines a requirement and lists types of requirements like functional, non-functional, safety/security. It describes what requirements engineers do like domain understanding, elicitation, evaluation, negotiation, specification and documentation. It discusses common requirement defects and techniques for quality assurance like prototyping, inspections/reviews, and formal checks. Inspections are particularly effective for finding defects and improving productivity.

Uploaded by

markyray
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Requirements Quality Assurance

Mehrdad Sabetzadeh

April 7, 2010

What is a Requirement?
➜  No universally agreed definition
➥ But intuitively: A requirement is something that
someone needs in order to solve a problem or achieve
an objective

➜  IEEE Definition
"A condition or capability that must be met or
possessed by a system or system component to
satisfy a contract, standard, specification, or
other formally imposed document. The set of all
requirements forms the basis for subsequent
development of the system or system component".
[IEEE Std 830-1998]

2

Types of Requirements
➜  Functional requirements:
➥  What the system does: the interactions between the system and
its environment
➥  Should be as independent as possible from implementation
➜  Non-functional requirements:
➥  Observable aspects of the system that are not directly related
to functional behavior
➣  e.g. performance, reliability
➜  Safety/security requirements ("shall not" properties)
➥  A kind of nonfunctional requirement
➥  Behavior the system must never exhibit
➣  e.g. it must be impossible to apply reverse thrust in mid-flight
➜  Constraints (Pseudo requirements)
➥  Imposed by the client or environment in which the system
operates
➥  Often concern the technology to be used (language, operating
system, middleware etc.)

3

What do Requirements Engineers do?

Focus of
today’s
lecture

4

Domain Understanding and Elicitation

➜  Domain Understanding
➥ Identify the problem / opportunity
➥ Understand the organizational context
➥ Identify stakeholders
➥ Specify the scope of the software system
➜  Requirements Elicitation
➥ Elaborate stakeholders’ goals
➥ Identify alternative options for satisfying these goals
➥ Identify organizational / technical constraints and
assumptions
➥ Define typical scenarios illustrating desired interactions
between the system and its environment
➥ Articulate the requirements the system should meet in
order to conform to all of the above
5

Evaluation and Negotiation
➜  Evaluation Tasks
➥  Identify conflicting concerns and resolve them
➣  Conflicts often arise from multiple viewpoints and
different expectations.
➥  Identify and assess risks associated with the system
➣  Budget, schedule, etc.
➥  Compare the alternative options identified during
elicitation and select best options
➥  Prioritize the requirements
➣  give weight to requirements that are essential to the
system
➣  drop lower-priority requirements that would together
exceed budgets and deadlines
➜  All the above involve some kind of negotiation to
arrive at a consensus
➜  Output is a preliminary requirements document
6

Specification and Documentation
➜  Goal
➥ Detailing, structuring, and documenting
the agreed characteristics of the system
as they emerge from the evaluation and negotiation
activity
➜  Contents
➥ A detailed and precise description of the following:
➣ Objectives
➣ Concept definitions
➣ Relevant domain properties
➣ Responsibilities
➣ System requirements
➣ Software requirements
➣ Environment assumptions
7

Requirements Quality Assurance (QA)
➜  Goal:
➥ Quality assurance is aimed at
checking that the items specified
in a Requirements Document (RD)
meet the desired qualities attributes
➣ Completeness, consistency, adequacy
➜  Why is Requirements QA important?
➥ Because cost of requirements failures is very high
“Finding and fixing a software problem after delivery
is often 100 times more expensive than finding and
fixing it during the requirements and design
phase.” [Boehm and Basili]

➜  Remainder of lecture is about Requirements QA


8

Requirements Quality Attributes
➜  Completeness
➥  The requirements must be sufficient to ensure that the
system will satisfy all its objectives
➜  Consistency
➥  The requirements must be satisfiable when taken together,
i.e. they must be compatible
➜  Unambiguity
➥  The requirements must be specified in a way that
precludes different interpretations.
➜  Measurability
➥  The requirements must be formulated at a level of
precision that enables analysts to evaluate alternative
options, and to test or verify whether an implementation
satisfies them.
➜  Relevance
➥  The requirements must each contribute to the satisfaction
of one or several objectives underpinning the system
9

Requirements Quality Attributes (continued)
➜  Feasibility
➥  The requirements must be realizable in view of the budget,
schedule, and technology constraints
➜  Comprehensibility
➥  The formulation of requirements must be comprehensible by
the people who need to use them.
➜  Good structuring
➥  The requirements document should be organized in a way
that highlights the structural links among its elements
➜  Modifiability
➥  It should be possible to revise, adapt, extend, or contract
the requirements document through modifications that are
as local as possible
➜  Traceability
➥  The context in which an item of the requirements document
was created, modified, or used should be easy to
retrieve. Such context should include the rationale for
creation, modification, or use.
10

Common Requirement Defects

➜  Major Issues

11

Common Requirements Defects
➜  Minor Issues

12

Techniques for Requirements QA
➜  Prototyping
➥ Partial implementation for validation with customers

➜  Inspections, reviews, walkthroughs


➥ Independent inspectors search for defects and
recommend appropriate actions on agreed defects

➜  Simulation
➥ Simulates the environment of a system and checks
the appropriateness of specified behaviors

➜  Formal checks
➥ Completeness and sanity checks
➥ Model checking and theorem proving
13

Prototyping
➜  Definitions
➥  “A software prototype is a partial implementation constructed primarily to enable
customers, users, or developers to learn more about a problem or its
solution.” [Davis 1990]
➥  “Prototyping is the process of building a working model of the system” [Agresti
1986]
➜  Approaches to prototyping
➥  Presentation Prototypes
➣  explain, demonstrate and inform – then throw away
➣  e.g. used for proof of concept; explaining design features; etc.
➥  Exploratory Prototypes
➣  used to determine problems, elicit needs, clarify goals, compare design
options
➣  informal, unstructured and thrown away
➥  Breadboards or Experimental Prototypes
➣  explore technical feasibility; test suitability of a technology
➣  Typically no user/customer involvement
➥  Evolutionary (e.g. “operational prototypes”, “pilot systems”):
➣  development seen as continuous process of adapting the system
➣  prototype is an early deliverable, to be continually improved.
14

Inspections, Reviews, Walkthroughs
➜  Note: these terms are not widely agreed
➥  Formality
➣  informal: from meetings over coffee, to team get-togethers
➣  formal: scheduled meetings, prepared participants, defined agenda, specific
format, documented output
➥  Management reviews
➣  E.g. preliminary design review, critical design review, …
➣  Used to provide confidence that the design is sound
➣  Attended by management and sponsors (customers)
➣  Usually a “dog-and-pony show”
➥  Walkthroughs
➣  developer technique (usually informal)
➣  used by development teams to improve quality of product
➣  focus is on finding defects
➥  (Fagan) Inspections
➣  a process management tool (always formal)
➣  used to improve quality of the development process
➣  collect defect data to analyze the quality of the process
➣  written output is important
➣  major role in training junior staff and transferring expertise
15

Benefits of Formal Inspections
➜  Formal inspection works well for programming:
➥  For applications programming:
➣  more effective than testing
➣  most reviewed programs run correctly first time
➠  compare: 10-50 attempts for test/debug approach
➥  Data from large projects
➣  error reduction by a factor of 5; (10 in some reported cases)
➣  improvement in productivity: 14% to 25%
➣  percentage of errors found by inspection: 58% to 82%
➣  cost reduction of 50%-80% for V&V (even including cost of inspection)
➥  Effects on staff competence:
➣  increased morale, reduced turnover
➣  better estimation and scheduling (more knowledge about defect
profiles)
➣  better management recognition of staff ability
➜  These benefits have been shown to apply to requirements inspections too

16

Inspection Constraints
➜  Size
➥  enough people so that all the relevant expertise is available
➥  min: 3 (4 if author is present)
➥  max: 7 (less if leader is inexperienced)
➜  Duration
➥  never more than 2 hours; concentration will flag if longer
➜  Output
➥  all reviewers must agree on the result: accept; re-work; re-inspect;
➥  all findings should be documented
➣  summary report (for management); detailed list of issues
➜  Scope
➥  focus on small part of a design, not the whole thing

➜  Timing
➥  Examines a product once its author has finished it
➣  not too soon: product not ready - find problems the author is already aware of
➣  not too late: product in use - errors are now very costly to fix
➜  Purpose
➥  Remember the biggest gains come from fixing the process
➥  collect data to help you not to make the same errors next time

17

Inspection Guidelines
➜  Prior to the review
➥  schedule Formal Reviews into the project planning
➥  train all reviewers
➥  ensure all attendees prepare in advance
➜  During the review
➥  review the product, not its author
➥  keep comments constructive, professional and task-focused
➥  stick to the agenda
➥  leader must prevent drift
➥  limit debate and rebuttal
➥  record issues for later discussion/resolution
➥  identify problems but don’t try to solve them
➥  take written notes
➜  After the review
➥  review the review process!

18

Inspection Guidelines
➜  Possibilities
➥  specialists in reviewing (e.g. QA people)
➥  people from the same team as the author
➥  people invited for specialist expertise
➥  people with an interest in the product
➥  visitors who have something to contribute
➥  people from other parts of the organization
➜  Exclude
➥  anyone responsible for reviewing the author
➣  i.e. line manager, appraiser, etc.
➥  anyone with known personality clashes with other reviewers
➥  anyone who is not qualified to contribute
➥  all management
➥  anyone whose presence creates a conflict of interest

19

Structuring the Inspection
➜  Can structure the inspection in different ways
➥  Free mode
➣  Rely on expertise of the reviewers
➥  Checklists
➣  uses a checklist of questions/issues
➣  checklists tailored to the kind of document
➥  Active reviews (perspective-based reading)
➣  each reviewer is given a specific process to follow for defect
search and reads for a specific purpose
➣  effectively different reviewers take different perspectives
➜  The differences may matter
➥  Some studies indicate that:
➣  active reviews find more faults than free mode or checklist
methods
➣  no effective difference between free mode and checklist
methods
➣  the inspection meeting might be superfluous!

20

Inspection Checklists
➜  Example

21

Inspection Checklists
➜  Example (continued)

22

Simulation
➜  Definition
➥  Mimicking possible behaviors of the environment and executing a model
of the software system to respond to these events
➥  Often accompanied by animation
➣  a visual representation that shows how the system evolves as the
model is being executed
➜  Prerequisite
➥  An (abstract) executable model of the system needs to be built
➜  Types of visualization
➥  Textual: input events are entered as textual commands; model
reactions are displayed as execution traces.
➥  Diagrammatic: input events are entered by event selection among those
applicable in the current state; model reactions are displayed as
tokens progressing along the model diagrams
➥  Domain-specific visualization: input events are entered through domain
-specific control devices displayed on the screen; model reactions are
displayed as new values on domain-specific control panels
23

Formal Checks
➜  A wide range of mathematically defined checks
that can be automated by tools
➥ The specification must be formal.

➜  Example 1
➥ Checking disjointness of input cases
➣  For every two cases C1, C2 we must have C1 ∧ C2 = false

24

Formal Checks (continued)
➜  Example 2
➥ Checking completeness of input cases
➣  For C1, C2, … Cn we must have: C1 ∨ C2 ∨ · · · ∨ Cn = true

➜  A wide variety of techniques use model checking


and theorem proving
➥ Model checking will be covered in a future lecture

25

References
➜  Lecture contents adapted from:
➥ Requirements Engineering From System Goals to UML
Models to Software Specifications by Axel van
Lamsweerde
➣  Chapter 5 Requirements Quality Assurance
➥ Course slides from the Requirements Engineering course
at the University of Toronto by Steve Easterbrook
➣  http://www.cs.toronto.edu/~sme/CSC2106S/

➜  More Resources
➥  [IEEE Std 830-1998] IEEE Recommended Practice for Software
Requirements Specifications -Description
➥  [Boehm&Basili] Software Defect Reduction Top 10 List
➥  [Blum] Software Engineering: A Holistic View

26

You might also like