An Approach to Carry Out Consistency Analysis on

Requirements
Validating and Tracking Requirements through a Configuration Structure

Padmalata Nistala Priyanka Kumari

Business Systems and Cybernetics Center Business Systems and Cybernetics Center
Tata Consultancy Services Tata Consultancy Services
Hyderabad, India Hyderabad, India
[email protected] [email protected]

AbstractRequirements management and traceability have system and thus unable to trace whether system [3] is being
always been one of grand challenges in software development developed according to the requirement or not.
area. Studies reveal that 30- 40% of software defects can be J. Dick [4] suggested solution for the stated problem by
traced to gaps or errors in requirements Although several models maintaining requirements linking from one level to another
and techniques have been defined to optimize the requirements
process, ensuring alignment and consistency of elicited
which helps in impact analysis and derivation of a need.
requirements continues to be a challenge. All software Management and traceability of the requirements should help
engineering standards and methodologies recognize the in achieving the product to meets the stated requirements. The
importance of maintaining relationships among the software concept of traceability [5] is not only limited to tracing the
elements for traceability. We have leveraged the structured requirement but also on the alignment of customers goals and
relationships among the requirement elements to come up with requirements to system/software specification which will help
an approach to systematically carry out consistency analysis of in achieving the stated objective. Tomoyuki [7] has proposed
requirements for software systems. traceability concept to maintain and demonstrate the
The framework has multiple models: a multi layered relationships between requirements and software artifacts such
requirement model, a configuration structure to link and track
the requirement items, a consistency analysis method to identify
as design models, source code, and test cases.
the inconsistencies in the requirements and a consistency index Standards like SWEBOK [7], ISO/IEC [1], PDTR 30103[8]
computation to indicate the level of consistency in overall have defined the importance of relationships between different
requirements of the software system. This approach helps to configuration items but they have not defined any
validate the requirements from both completeness and layout/framework on achieving traceability or alignment
correctness perspectives and also check their consistency in through structured relationship between configuration items.
forward and backward directions. The paper outlines the Few book on requirements engineering [3] focuses on how
framework, describes the encompassing models and the high level requirements getting transformed into low-level
implementation details from pilot of the framework to an
requirements depending upon the relationships between layers
industry case study along with results.
Index TermsRequirement alignment, requirement
of information. The established Demings principle of process
configuration structure, requirement consistency analysis, step and product quality [9] also supports correlation between
requirement inconsistency matrix, requirement consistency index process step and product quality. These established concepts
requires to creating a relational structure to check the quality at
I. INTRODUCTION each process step. Though these literatures emphasize on some
kind of relationship but they do not exactly focus defining the
According to ISO/IEC 15288 systems definition [1], A structure and related necessary granularity while forming the
system is a combination of interacting elements organized to structure. They lack a robust mechanism for creating a
achieve one or more stated purposes. As the software system structured relationship between the artifacts.
transforms from a set of needs to a final product through In other disciplines such as manufacturing and systems
various forms, at each stage there is a high possibility of loss of engineering we can find models, well established principles
quality due to defects in transformation of configuration items and practices for formulating and tracing the product parts and
from previous phase. Thus, an understanding of system model composition. The modern architecture strongly believes in
[1] is important not only to understand what is being strong relationship between form or structure [10, 11] of the
developed but also to understand how it is being developed. product to the purpose or function. Similarly, the concept of
Though a lot of traceability tools and research is trying to Product Breakdown Structure (PBS) [12, 13, 14] in
overcome the problem in software industry, managing the manufacturing industry is the breakdown of a product into its
requirements to perform traceability continues to be a primary required components. This helps in providing a clear picture of
issue. Kannenberg and Saiedian in their article [2] reflects the the product and helps in handling the complex parts with easy
problem of requirement source traceability without which it is
difficult to determine the incorporation of requirement into

320 RE 2013, Rio de Janeiro, Brasil

978-1-4673-5765-4/13/$31.00
c 2013 IEEE
Industry Track
visuals prints of the components. The relationships and a multi layered requirement model to ensure the
traceability of product parts is visually transparent. alignment of requirements right from the business
Various traceability tools presently used in software goals to the software specifications. It comprises of
industry [3,15] maintain document level linkages focusing on four key players: business layer, process layer,
completeness. Few other tools [15, 16] support some kind of requirements layer and specification layer.
relation but do not have the adequate granularity and holistic a configuration structure to link and track the
coverage of the relations. requirement items for each layer. This provides a
The approach of the proposed framework is to perform visual representation of the requirements flow across
consistency analysis to correctly and completely track the flow various layers.
of requirements with various layers. Here we propose to a consistency analysis method to identify the
leverage the relationships among the requirements inconsistencies in the requirements structure. A
configuration items and propose a model for requirements configuration inconsistency matrix is populated with
alignment and traceability. The research problems taken are missing linkages of requirements in both forward and
To define guidelines for identification of requirement backward directions.
elements or configuration items for software systems a consistency index computation to indicate the level of
To define a framework for requirements tracking through a consistency in overall requirements of the software
requirement configuration structure using the relationships system.
among requirement configuration items
A. Layers of the Model
To conduct consistency analysis on the overall system in
terms of completeness and correctness and identify the In order to realize the value from the software systems, the
gaps at each phase context and business perspective has to be integrated to the
To apply this framework for an industry case of software software requirements [17]. Also, to address the leakage issues/
system and illustrate the applicability of the framework gaps at requirement phase, we initiated the requirement
and the no of inconsistencies in requirements found configuration with the business layer. The key configuration
through this model. layers in the framework are FOUR.
In the subsequent sections, we describe the proposed 1) Business Layer: Here the broad business purpose of the
framework - the requirement configuration structure, its software system as outlined by the customer is captured in the
application for requirements traceability of a software form of business goals and objectives. This layer brings the big
application in a vertical business unit, the result achieved, picture of the business - the purpose of the software system and
followed by the conclusion. is typically stated by the customer in the RFP (request for
proposal).
II. FRAMEWORK 2) Process Layer: The key business processes to realize the
Here in this framework, the complete value chain of goals and objectives are also identified here. This is a crucial
software requirements process is considered in the form of key layer and it is required to identify the end to end chain of
configuration layers. The complete view of framework with its business processes to realize the business goals and objectives.
encompassing layers, configuration items and their structural 3) Requirements Layer: This layer identifies the key
relationships is depicted in Fig 1. business requirements of the system and is typically the starting
The key components of the framework are: point for many software systems.

Figure 1. Requirement Configuration Structure Framework

321
 4) Requirement Specification Layer: This layer describes ISO/IEC/IEEE 24765 defines consistency as the degree
the analyzed requirements in the form of specifications. The of uniformity, standardization, and freedom from
requirement specification could be in any notation depending contradiction among the documents or parts of a system or
on the life cycle process e.g. SRS for structured model; Use component. In our framework, we extend this definition
cases for OO model and User stories for agile model. and propose a requirement to be consistent when both
Each layer will have a set of defined configuration items. As completeness and correctness properties across its path in
a guideline, we have listed the key requirement configuration the structure are true and maintained. The requirement is
items widely followed in software development. inconsistent if one or more gaps found in RCS either due
Business Layer: Business Goals and/or Objectives. to completeness or correctness issues.
Process Layer: Business Process (Id + Name) and Sub The inconsistencies found in analysis are populated into a
Process. configuration inconsistency matrix (CIM). The template of
Requirement Layer: Business Requirements CIM is given in Table-1. X and Y denote the configuration
(Requirements group + Requirements Id + Requirements items among which there is an inconsistency in relationship;
Description). the direction is forward/backward; inconsistency type is
Specification Layer: Use Case (Id + Name). Missing for a completeness gap or Erroneous for a
Depending on the context of the client, the life cycle model correctness gap; analysis describes the gap and count is
and its architecture, the individual configuration items can be summation of the inconsistent configuration items.
tailored within each layer.
TABLE 1. Configuration Inconsistency Matrix
B. Configuration Structure
Process Process for which the related inconsistency exist
This composition of requirement configuration items for
each of these layers along with their relationships helps in X is a configuration item in the requirement
Config Item X
configuration structure
creating the requirement configuration structure (RCS). It
provides guidance in identifying and linking configuration/ Y is a configuration item in the requirement
Config Item Y
configuration structure
components item at the business, process and requirement
layers. Fig-1 depicts the layout of the model Requirement Direction Relationship: Forward/ Backward
Configuration Structure. Type of relationship inconsistency:
The configuration items at one layer are linked to Inconsistency Type
Missing/ Erroneous
corresponding items at another layer to create requirement
Analysis Analysis description
structure of the software product/application under
development. The framework suggests compartmentalizing Count No of inconsistencies in the relationship
requirements to the processes it serves and similarly to all other
elements of the configuration item.
CIM provides a tool to consolidate all the inconsistencies in
C. Consistency Analysis Method the structure defined above and analyse the data. The numbers
The consistency or track of each relationship from the of inconsistencies as found from the CIM are plotted against
above created structure is validated on two dimensions. The the business process as inconsistency graphs. This
relationship consistency analysis involves: information readily gives indication on the completeness and
Completeness analysis Completeness property is the correctness of requirements realization and gaps in
degree to which the set of functions covers all the specified transformation at each layer. These graphs visually depict the
tasks and user objectives [18]. Completeness analysis of inconsistency in quality for the requirements.
RCS involves checking whether for each configuration D. Requirement Consistency Index
item in a layer, are there corresponding configuration
Requirement Consistency Index (RCI) quantifies the
item(s) in forward and backward directions and overall
percentage consistency of the requirements. In the consistency
whether the coverage of the configuration items across the
analysis, a requirement is considered consistent when both
structure is complete. This analysis helps to establish the
completeness and correctness across its path in the structure are
end to end track of requirements in RCS.
maintained and inconsistent if one or more gaps found in
Correctness analysis Correctness property is the degree
analysis. As an initial measurement, RCI considers each
to which a system or component is free from faults in its
consistency issue with equally weightage resulting in an
specification, design, and implementation [19]. average value for the metric.
Correctness analysis of RCS involves checking for faults
Requirement Consistency Index = Percentage consistency
or errors in transformation between any two configuration
of the requirements.
items in the structure. It validates for each relationship of
RCI =A/(B+C)
RCS, do the next configuration item correctly achieves the
where,
objectives of the previous configuration item or not. This
A =Number of Consistent Requirements.
analysis is to validate the correctness of requirement items
This is computed as Xi where i denote a requirement
across layers in RCS.
varying from 1 to n; Xi is consistency value of the

322
 Figure 2. Requirement configuration structure for the project

requirements. 1) Business Layer: The key configuration items in this

B = Total number of elicited requirements layer are business objectives. The inputs for this layer have
C = Number of missing requirements identified primarily been taken from the delivery proposal report, process
diagrams/flow charts. Customer has identified three high level
III. CASE STUDY
business objectives: customer should be automatically
This framework has been piloted for a critical business registered though automatic registration and enhancing the
system for a large insurance client of TCS. The framework was security framework for the existing registration service and to
implemented to carry out the due diligence exercise for enhance the interface of registration service with other
requirements, find out the inconsistencies and validate the connected application.
alignment and traceability of elicited requirements.
2) Process Layer: The above objectives were transformed
The actual implementation of this requirement consistency
into seven end to end business processes (P1 to P7). Automatic
analysis was carried by two researchers who were also
registration while customer is buying a product; registering the
involved in designing the framework with the knowledge help
of business analyst. The implementation period was two employee of the organization where the insurance client have
months which resulted in the formation of requirement enrolled their insurance products; registering customer through
configuration structure and the consistency index which were internal staffs; enhancing registration process with the security
also validated by the SMEs. framework and password reset functionality and providing
The pilot application was an eRegistration System which management information are few of the identified business
provides different methods of automatically registering external processes. Sub-process: Sub-processes are identified by the
customers and maintaining and accessing their accounts. The decomposition of the main processes.
existing registration process is enhanced with implementation 3) Requirements Layer: Requirements layer has been
of security framework for few processes. The new application developed with the business system analyst group. The above
also integrates the Management Information (MI) report business processes (P1-P7) were linked to application
creation. The following section details the framework requirements taken from requirements catalogue which is a
deployment for the case. client provided artifact. The requirement is defined with
specific requirement group and ID.
A. Layers of the Framework
4) Requirement Specification Layer: In this layer, use case
The first step is identifying the layers and configuration documents were provided by customer. Configuration items
items. The framework implementation for the registration (Use Case) for this layer were properly linked to the
system is illustrated in Fig 2 and described below for each layer.
corresponding process and requirements.

323
 Table 2: Configuration Inconsistency Matrix for the project

Process C.Item X C.Item Y Direction Type Analysis Count

P2:Bulk Process2-1: Bulk Registration Requirements Forward Missing 1) Requirements missing regarding security 1
Registration for product customer admin tool main menu for Bulk
Registration as part of tool.
2) Requirements missing regarding bulk
registration file upload.
P2:Bulk Process2-1: Bulk Registration Requirements Forward Missing 1) Requirements not adequately defined 1
Registration for product customer regarding generation of activation code.
P2:Bulk Requirements:9 Use Case Forward Missing 1) Use Case missing for confirmation and 1
Registration generation of activation code, bulk
register file upload, output generation.
P3:Consumer Requirements Use Case: Self Backward Missing 1) Password creation and confirnation 1
Self Elected Registration requirements missing for self registration
Registration
P5:Login and Process5-1: Consumer Login/ Requirements Forward Erroneous 1) Requirements errorneous in nature 1
manage account Logout requirements states that customer has to
answer two security question whereas
customer has to asnwer three security
questions.

B. Configuration Structure
The above figure-2, illustrates the requirement
configuration structure (RCS) created for eRegistration
System linking all the configuration items, taking an
illustration of bulk registration process to fulfill the objective
- to provide automatic registration facility to the customer.
The process is divided into two sub-processes bulk
registration for product customer and bulk registration for
workplace customer All the requirements and use cases are
correctly mapped to these sub-processes as shown in figure-2.
The first row in figure shows the missing use case when going
forward from requirements to specification layer. Thus, it
identifies the completeness issue with use case.
Similarly, the sub-process (P2-1) has a requirement on
Figure 3. Requirement Inconsistency Graph
generation of action code but the listed requirements are
insufficient and do not adequately satisfy the objectives of
The identified seven processes have severe gaps in realizing
activation code generation process. This is a correctness
the objectives of the eRegistration software system.
related problem with requirements.
Requirements for key processes like bulk registration and auto
C. Consistency Analysis Method registration were completely missing.
Based on the RCS structure created, each relationship is Similarly many use cases were missing and are not
analyzed for consistency in terms of completeness and specified.
correctness as described in the framework. The inconsistencies Total no. of requirements for the case is 50 and the No of
found in RCS are logged into configuration inconsistency consistent requirements 24. Applying the formula, the
matrix (CIM). The table-2 shows a view of the configuration requirement consistency index was calculated as 48%.
inconsistency matrix (CIM). The analysis is carried out through
TABLE 3: Requirement Inconsistency Index for the project
tracking the flow and validating the information. The
inconsistencies listed in CIM have been validated with the Requirement Consistency Index (RCI) 48%
SMEs.
D. Requirement Consistency Index
The low value of index readily gives an indication on the
Figure-3 depicts the summary of the requirement quality of the overall requirements and the leakage issues
inconsistencies as analyzed from configuration inconsistency within requirements. It validates the statistics that the
matrix (CIM). It also segregates the data based on correctness requirements are poorly managed in software industry. The
and completeness related problems. It can be seen from the gaps in requirements are hardest to find but could be unearthed
graph that the number of inconsistencies originated from in this framework through establishment of linkages among the
requirements are very high. requirement elements.

324
 The deployment of this framework helped in identifying [3] E. Hull, K. Jackson, J. Dick, Requirements Engineering,2nd
many configuration items which were missing and required for ed., Springer, 2009, ch-9
realizing the business needs into software system. [4] J. Dick, Rich Traceability, Proc. 1st International Workshop
on Requirements Traceability, Edinburgh, Sept 2002
IV. CONCLUSION [5] J. Cleland-Huang, O. Gotel, A. Zisman, Software and Systems
Traceability 1st ed., Springer, 2012
The case study results show that significant requirement [6] T. Arao, E. Goto, T. Nagata, Business Process Oriented
inconsistencies can be found through the requirement Requirements Engineering Process Proc. 13th IEEE
configuration structure framework. Requirement consistency International Conference on Requirements Engineering (RE05),
IEEE computer society, Dec. 2005,
analysis brings out not only the missing configuration items
[7] IEEE Standard Committee, Guide to the Software Engineering
from completeness analysis but also erroneous configuration Body of Knowledge (SWEBOK), ver 3,IEEE Computer
items in the RCS from the correctness analysis. Both forward Society, 2004
and backward traceability for requirements are covered. The [8] ISO/IEC PDTR 30103: Software and systems engineering
consistency track of complete requirements value chain from Lifecycle processes Framework for product quality
achievement
business objectives, business processes and requirements until
[9] William W Scherkenbach, Demings Road to continual
the use cases can be maintained, validated and analyzed improvement, SPC Press Inc., Knoxville, Tennessee (615 584
through this framework. Computation of Requirement 5005) 1991, p26-38
consistency index for overall requirements provides a [10] Louis H. Sullivan,The Tall Office Building Artistically
quantifiable measure on the consistency of requirements and Considered", Lippincott's Magazine, ver 57, March 1896
acts as a requirement quality indicator. This method will [11] S. Dionidis, "A Tale of Four Kernels", Proc. 30th International
benefit the projects to have a structured approach towards Conference on Software Engineering (ICSE 08). Leipzig,
Germany: Association for Computing Machinery, May 2008,
requirements management through the configuration structure pp. 381390
and can go a long way in containing the requirement defects [12] D. Svensson and J. Malmqvist, Strategies for Product Structure
and improving phase containment. Management at Manufacturing Firms. Journal of Computing
Future work involves considering the weightage of and Information Science in Engineering, 2002, pp-50-58
requirements into metrics computation, further variations in [13] Product Breakdown Structure :
http://productbreakdownstructure.com/
requirement scenarios such as change scenarios, reverse
engineering scenarios etc. There are plans to customize the [14] Product Breakdown Structure
http://www.professionalpractice.asme.org/ProductMgmt/System
internal tooling platform include computation of these metrics s/Product_Breakdown_Structure.cfm
and implement for multiple projects. [15] A. Castor, R. Pinto, C. Silva, and J. Castro,Towards
Requirement Traceability in TROPOS , Centro de Informtica,
ACKNOWLEDMENT Universidade Federal de Pernambuco, Brazil , 2004
The authors would like to acknowledge the management of [16] P. Zielczynski, Requirements Management Using IBM
Rational RequisitePro, IBM press , 2007
Tata Consultancy Services for providing an opportunity to
[17] Padmalata Nistala, Supriya Kummamuru & MGPL Narayana
design and validate the framework. We would like to thank Mr An Approach to Understand and Elicit Requirements Using
MGPL Narayana, Vice President, TCS for his encouragement Systemic Models: Ensuring a Connect From Problem Context to
to write this paper, Mrs Lakshmi Suchetha for showing the Requirements CSER 2013,Conference on Systems Engineering
2013, Georgia Institute of Technology, Atlanta, Mar 2013
interest for the implementation of the framework.
[18] ISO/IEC 25010:2011 Systems and software engineering--
REFERENCES Systems and software Quality Requirements and Evaluation
(SQuaRE)--System and software quality models,
[1] Systems Engineering Guide for ISO/IEC (System Life Cycle [19] ISO/IEC/IEEE 24765:2010 Systems and software engineering--
Processes) 15288 , July 2002 . Vocabulary
[2] A. Kannenberg, H. Saiedian, Why Software Requirements
Traceability Remains a Challenge, 14 Crosstalk - Journal of
Defense Software Engineering, July/August 2009,
www.stsc.hill.af.mil

325