Improving the quality of conceptual modeling using cognitive mapping techniques
Improving the quality of conceptual modeling using cognitive mapping techniques
www.elsevier.com/locate/datak
Abstract
Conceptual modeling involves the understanding and communication between system analysts and end-
users. Many factors may affect the quality of conceptual modeling processes as well as the models per se.
Human cognition plays a pivotal role in understanding these factors and cognitive mapping techniques are
effective tools to elicit and represent human cognition. In this paper, we look at the use of cognitive map-
ping techniques to improve the quality of conceptual modeling. We review frameworks on quality in con-
ceptual modeling and examine the role of human cognition in conceptual modeling. The paper also
discusses how human cognition is related to quality in conceptual modeling, the various cognitive mapping
techniques, and how these cognitive mapping techniques can be used in conceptual modeling. Through a
case study, the paper describes ways of incorporating cognitive mapping techniques to a popular systems
development methodology—Soft Systems Methodology—to improve the quality of conceptual modeling.
2005 Elsevier B.V. All rights reserved.
Keywords: Conceptual modeling; Quality; Cognitive mapping; Requirements engineering; Systems development
methodology
*
Corresponding author. Tel.: +1 402 472 3078/2204; fax: +1 402 472 5855.
E-mail addresses: [email protected] (K. Siau), [email protected] (X. Tan).
0169-023X/$ - see front matter 2005 Elsevier B.V. All rights reserved.
doi:10.1016/j.datak.2004.12.006
344 K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365
1. Introduction
Developing information systems (IS) has always been a challenging task. Many factors influ-
ence the success of IS development. Among them, being able to accurately and completely capture
information requirements during systems development is commonly regarded as one of the most
important factors. This point has been shared by both IS researchers (e.g., [8,17,27,63]) and prac-
titioners. As a result, requirements engineering, which is a branch of software engineering con-
cerned with identifying real-world goals, capturing services required, and recognizing
constraints, is perceived as an area of growing importance [45]. Conceptual modeling is the cor-
nerstone of requirements engineering. Improving the quality of conceptual modeling processes as
well as the models per se has received substantial research attention. Some researchers have taken
a systematic approach to develop frameworks for understanding and assessing the quality of con-
ceptual modeling (e.g., [18,32,33,35]).
Requirements engineering is recognized as a social process that is characterized by ongoing
sense making among participants, which include managers, end-users, and system analysts [43].
Moody et al. [40] claimed that software development is more like a craft than an engineering dis-
cipline during the processes of requirements engineering. In this respect, human cognition plays a
pivotal role in understanding human and organizational issues in requirements engineering, and
in identifying ways to improve the quality of conceptual modeling.
Cognitive mapping techniques have been widely used in strategic management and political sci-
ence to depict and explore the cognitive structures of members of organizations [28]. Some
researchers, such as Avison and Fitzgerald [4] and Montazemi and Conrath [39], have hinted
at the usefulness of cognitive mapping in systems development. However, they have only provided
brief coverage on the use of cognitive mapping. In this paper, we have provided a comprehensive
review of three widely used cognitive mapping techniques. We also discuss how they can be em-
ployed to improve the quality of conceptual modeling, and how they supplement popular systems
development methodologies in conceptual modeling.
The paper is organized as follows. Section 2 reviews the literature on requirements engineering
and conceptual modeling, and discusses the frameworks for quality in conceptual modeling. Sec-
tion 3 introduces how human cognition is related to the quality in conceptual modeling. Section 4
reviews and discusses how three popular cognitive mapping can improve the quality in conceptual
modeling. Section 5 describes how these cognitive mapping techniques can supplement a popular
systems development methodology—Soft System Methodology—to improve quality in concep-
tual modeling. A conference organizing case was used for the discussion. Section 6 summarizes
the research and concludes the paper.
2. Literature review
The IS field has witnessed numerous examples of software development failures. These costly
and conspicuous failures cast a serious challenge for IS researchers. Requirements engineering,
which is an early stage in the software development life cycle, plays an important role in successful
information systems development.
K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365 345
Regardless of the systems development methodology used, being able to understand usersÕ
information requirements is vital in any IS project and is a key factor in any successful IS imple-
mentation. Requirements engineering is a set of activities used by systems analysts to assess the
functionality required in a proposed system [8,27,45]. According to Pohl [45], it contains three
dimensions—specification, representation, and agreement. The degree of requirements under-
standing at a given time is measured by the specification dimension (ranging from opaque to com-
plete). The representation dimension (ranging from informal to formal) deals with the different
representations that are used for expressing knowledge about the system. The agreement dimen-
sion (ranging from personal to common) copes with the degree of agreement reached on a spec-
ification. To reflect this three-dimensional view, Bubenko et al. [9] defined requirements
engineering as:
The systematic process of developing requirements through an iterative co-operative process
of analyzing the problem, documenting the resulting observations in a variety of representa-
tion formats and checking the accuracy of the understanding gained (p. 154).
Four tasks to be performed in requirements engineering have been identified [46]. As shown in
Fig. 1, these four tasks are iterative in nature:
• Requirements specification: To understand the organizational situation that the system under
consideration aims to improve. It also describes the needs and constraints of the system under
development.
• Requirements negotiation: To establish an agreement based on the requirements of the system
among the various stakeholders involved in the process.
• Requirements representation: To develop a mapping of real-world needs into a requirements
model.
• Requirements validation: To ensure that the derived specification corresponds to the original
stakeholder needs and conforms to the internal and/or external constraints set by the enterprise
and its environment.
To find ways to improve the quality in conceptual modeling, the question ‘‘What is quality in
conceptual modeling?’’ must be answered first. Some researchers have taken a systematic ap-
proach to define quality in relation to conceptual models. Lindland et al. [35] proposed a frame-
work to provide an in-depth understanding of quality in conceptual modeling. This framework
extended the prior frameworks that defined the desired properties of a requirements specification
(e.g., [18]). Fig. 2 depicts the main points of the framework, which applies three linguistic
concepts—semantic quality, syntactic quality, and pragmatic quality.
The main quality types are indicated by solid lines between four aspects of modeling: language,
domain, conceptual model, and audience interpretation. According to the framework, semantic
quality is the correspondence between the conceptual model and the domain; syntactic quality
is the correspondence between the conceptual model and the modeling language; pragmatic qual-
ity is the correspondence between the conceptual model and the audienceÕs interpretation of it.
Lindland et al. [35] focused on clarifying the goals and means (model properties and modeling
activities) of these three types of quality. Offering a more in-depth treatment of quality, this frame-
work has served as the foundation of the International Workshop on Conceptual Modeling
Quality (IWCMQ Õ02, Õ03, and Õ04), held in conjunction with the Requirements Engineering
conference.
As Krogstie and Sølvberg [34] correctly pointed out, Lindland et al.Õs [35] framework of the
quality in conceptual modeling, to a great extent, echoes the three-dimensional view of require-
ments engineering proposed by Pohl [45]. In their deeper structures, the specification dimension
corresponds to the semantic quality because both reflect the relationship between the domain
and conceptual models. The representation dimension corresponds to the syntactic quality be-
cause both deal with using specific languages to express identified specification in the form of con-
ceptual models. The only discrepancy lies in the agreement dimension and the pragmatic quality.
Focusing on individualÕs understanding of conceptual models, Lindland et al.Õs [35] pragmatic
quality does not sufficiently handle the social aspects associated with PohlÕs [45] agreement dimen-
sion, i.e., the degree of agreement reached on a specification (conceptual model).
Krogstie [32,34] further pointed out that even though the primary objective of semantic quality
is to obtain a valid and complete correspondence between the conceptual model and the domain,
this correspondence cannot be built or checked directly. We must go through the participantÕs
knowledge of the domain in order to build a model. This model must also be checked based
on the audienceÕs interpretation.
Based on the identified deficiencies of Lindland et al.Õs [35] framework, Krogstie and Sølvberg
[34] suggested an extended quality framework, shown in Fig. 3.
Compared with Lindland et al.Õs [35] framework, Krogstie and SølvbergÕs [34] framework
added the element of participant knowledge on the modeling domain as an additional aspect of
modeling. In addition, Krogstie and Sølvberg [34] split the audience interpretation into two sepa-
rate aspects—social actor interpretation (i.e., the set of statements that the social audience
perceives that the externalized model contains) and technical actor interpretation (i.e., the models
as interpreted by technical actors in the audience).
Besides the three types of quality (semantic quality, syntactic quality, and pragmatic quality)
covered in Lindland et al.Õs [35] framework, Krogstie and Sølvberg [34] incorporated four other
types of qualities in their framework.
• The perceived semantic quality, i.e., the correspondence between participant knowledge and
social actor interpretation, can serve as the surrogate for semantic quality. That is, the compar-
ison of the two imperfect approximations—participant knowledge and social audience interpre-
tation—can be directly observed and assessed, partially reflecting the level of true semantic
quality.
• Physical quality involves two basic quality means. Externalization refers to the level of external-
ization of the knowledge of some social actors in the forms of conceptual models. Internaliza-
tion refers to the level of other social actorsÕ knowledge obtained by making sense of the
externalized model.
• Social quality refers to the degree of the agreement on participantsÕ interpretation.
• Empirical quality relates to error frequencies when a model is read or written by different users,
and coding and ergonomics of computer–human interaction (CHI).
The extended framework by Krogstie and Sølvberg [34] is explicitly based on a constructivistic
world-view [6], recognizing that conceptual models are created as part of a dialogue between the
participants involved in conceptual modeling. The knowledge of each individual participant on
the modeling domain changes as modeling takes place. On the other hand, Lindland et al.Õs
[35] framework is implicitly based on realistic view, acknowledging that the domain consisting
of all possible correct and relevant statements for solving the problem is ‘‘right there.’’ As we sub-
scribe to the constructivistic world-view in our research in requirements engineering, we will use
Krogstie and SølvbergÕs [34] framework (Fig. 3) to discuss the role of human cognition in concep-
tual modeling.
The chaotic, nonlinear, and continuous nature of requirements engineering, especially concep-
tual modeling, warrants extensive investigations on the ‘‘people’’ end. Requirements engineering
is characterized by ongoing sense making among stakeholders, including the sponsor of the sys-
tem (managers), end-users (employees), and system analysts [16,17,32]. A cognitive approach that
focuses on sense making processes is helpful for investigating why participants in requirements
engineering understand requirements as they do [20] and why their understanding of requirements
may change and shift [23].
Cognition refers to the belief systems that individuals use to perceive, construct, and make sense
of their world, and to make decisions about what actions to take [23,67,69]. Understanding cog-
nition is becoming increasingly important for a number of management study areas, where cha-
otic and complex organizational contexts are under scrutiny [64]. In IS field, the interest in
studying how people think is growing along with the awareness of human factors when developing
K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365 349
information systems. In the following, we review the role of human cognition in requirements
engineering and conceptual modeling.
Early IS researchers have tapped into the cognition of individuals to understand the problems
in requirements engineering. For instance, Davis [19] listed four sources of difficulties in informa-
tion requirements determination: (i) the constraints on humans as information processors and
problem solvers, (ii) the variety and complexity of information requirements, (iii) the complex pat-
terns of interaction among users and analysts in defining requirements, and (iv) the unwillingness
of users to provide requirements. Valusek and Fryback [63] labeled these difficulties as communi-
cation obstacles ‘‘within’’ individual users, ‘‘among’’ users, and ‘‘between’’ users and analysts.
They further identified the underpinnings of these communication obstacles based on cognitive
and social psychology.
With the advancement of cognitive psychology, IS researchers have been able to identify many
cognitive difficulties of humans, as information processors and problem solvers in requirements
engineering. First, the information processing capacity of humans is limited [37]. The working
memory of humans can hold only a limited amount of information for processing. In addition,
some common cognitive mechanisms may lead to biases in information requirements perception.
Some heuristic-driven biases include [62]:
Other constraints on humans as information processors and problem solvers include satisficing
[56], faulty reasoning [44], and automaticity [57].
These constraints on individualÕs cognition should not be overlooked. In addition, with the pro-
cess of cognition, the knowledge of an individual participant on the modeling domain changes as
modeling takes place. This is the issue that Lindland et al.Õs [35] framework failed to cover, while
Krogstie and SølvbergÕs [34] framework did. The difficulties in individual cognition and the evolv-
ing participantÕs knowledge are related to only part of the human issues found in conceptual mod-
eling. When we discuss conceptual modeling for an organizationÕs information systems, we must
also look at the social aspects inherent in cognition.
Social cognitive research shares the fundamental tenet that people act on the basis of their inter-
pretations of the world, which is cognitively structured through experience and interaction [68].
Furthermore, socio-cognitive approaches in requirements engineering posit that ‘‘requirements
for IT applications do not exist a priori but are socially constructed through interaction among
350 K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365
(conceptual modeling) participants’’ (16, p. 331). In developing technological frames for making
sense of information technology in organizations, Orlikowski and Gash [43] systematically artic-
ulated the role of frames of reference in systems development and use. The frames of reference
held by organizational members are implicit guidelines that serve to organize and shape their
interpretation of events and organizational phenomena, and give meaning to these events and
phenomena [67]. The frames of reference, therefore, become vital to our understanding of the
socio-cognitive processes in conceptual modeling.
First, information requirements elicited in an organization will often have high variability
among users, especially between the sponsor of the system (managers) and end-users (employees),
because they use different frames of reference to perform tasks. Furthermore, the elicited require-
ments may not be stable due to the ongoing evolution of an individual personÕs frames of refer-
ences. Thus, there is variation in requirements both between and within users.
Second, the difference in the frames of reference causes problems between users and analysts.
That is, the different backgrounds of the two types of people are the principal reason of commu-
nication difficulties between users and analysts. This problem can also be viewed using the Johari
Window model of human interaction [36]. The Johari Window highlights the basic communica-
tion issues in user-analyst communication. Some researchers [7,11] labeled this problem as the
lack of common ‘‘language’’ between users and analysts.
Finally, in the context of emerging businesses, such as e-business or mobile-business, the stan-
dardized processes of businesses have not been established. As a result, the frames of reference for
users may not be formed adequately. This means that the information requirements are not
simply waiting to be discovered, or ‘‘mined’’ from the heads of the users [29].
Furthermore, researchers [22,65] have posited the existence of group-level frames of reference,
i.e., shared knowledge and belief systems that formulate expectations and perceptions of the
group.
To summarize, the frames of reference, especially the technological frames [43], are of interest
to researchers in investigating the socio-cognitive aspects in conceptual modeling. Assessing the
frames of reference for various stakeholders (including users and analysts) or different groups
may offer an insight to the socio-technical problems.
With the above understanding of quality in conceptual modeling (Section 2.2) and the role of
human cognition in conceptual modeling (Sections 3.1 and 3.2), we turn to discuss how human
cognition is specifically related to quality in conceptual modeling.
Syntactic quality can be assessed objectively by comparing the models and the language rules.
However, when either the models or the languages are very complex, the limitation of human cog-
nition capacity makes it a daunting task to manually generate conceptual models with high
syntactic quality, or to check conceptual models in terms of syntactic quality.
Semantic quality can be established indirectly using perceived semantic quality as the surrogate.
Perceived semantic quality, i.e., the correspondence between a participantÕs knowledge and an indi-
vidualÕs interpretation, has much to do with human cognition. This is because the social process,
i.e., the dialog between participants, is rooted in individualÕs cognition. In other words, participant
knowledge is determined by his/her cognitive structure on the problem, while an individualÕs inter-
K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365 351
pretation is directly related to the personÕs cognitive ability in interpreting conceptual models. In
addition, the link between the domain and the participantÕs knowledge is the place where human
cognition biases might play an important role. The biases, such as representativeness, the availabil-
ity heuristic, anchoring and adjustment, satisficing, faulty reasoning, and automaticity, make a
participantÕs knowledge only an imperfect approximation of the domain.
Pragmatic quality relates conceptual models to either a social actorÕs or technical actorÕs inter-
pretation. In essence, pragmatic quality affects how to interpret a model with the lowest possible
cognitive effort. For example, visualization is a way to improve pragmatic quality because people
can easily grasp its meanings.
Physical quality, i.e., externalization and internalization, is greatly influenced by the fit between
the modeling language and participants. Due to human cognition preferences, different people
prefer different modeling languages [45]. For example, end-users may like natural languages or
arbitrary graphics, while systems analysts and developers may prefer formal modeling languages.
With appropriate languages, cognition loads in externalizing knowledge into conceptual models
will be lower for participants.
Empirical quality deals with error frequencies, coding, and ergonomics of computer–human
interaction. Computer-aided modeling tools may incorporate mechanisms, such as automatic lay-
out and organization of multiple models, to decrease error frequencies in creating, reading, and
coding conceptual models. This is because those mechanisms are essentially lowering cognition
loads for users of modeling tools.
Finally, social quality is related to a personÕs socio-cognition. The agreement on participant
knowledge is dependent on an individualÕs frame of reference on the problem. The agreement
on individual interpretation, especially between system users and system analysts, is also deter-
mined by their respective frames of reference.
Table 1 depicts how human cognition is related to quality in conceptual modeling.
There are techniques designed to make sense out of complex, uncertain, confusing, and messy
situations. For example, rich picture is one technique used in Soft Systems Methodology (SSM)
[15]. Cognitive mapping, as a technique to elicit an individualÕs belief systems regarding a problem
domain, has great potential in overcoming some cognitive problems and facilitating the under-
standing among stakeholders. Published works (e.g., [4,26,39,47]) have hinted at the usefulness
of cognitive mapping in requirements engineering. These studies, however, have only provided
brief coverage on cognitive mapping. In our study, we review the utility of various cognitive map-
ping techniques in the context of conceptual modeling. We also discuss how these techniques can
supplement a popular systems development methodology, and thus enhance the quality of
conceptual modeling.
Table 1
Cognition and quality in conceptual modeling
Quality type Description How human cognition is related?
Semantic quality The correspondence between the Use individual interpretation and participant
conceptual model and the domain knowledge to establish imperfect approximations
of semantic quality. See perceived semantic quality
Perceived The correspondence between participant Participant knowledge is determined by the
semantic quality knowledge and individual interpretation individualÕs cognitive structure regarding the
problem; individual interpretation depends on
the cognition ability in interpretation
Syntactic quality The correspondence between the conceptual Limitation in human cognition capacity
model and the language demands for simple models and languages
Pragmatic quality The correspondence between the conceptual Good pragmatic quality means interpreting
model and the audienceÕs interpretation of it a single meaning with the lowest possible
cognitive effort
Physical quality The correspondence between participant Good physical quality means expressing a
knowledge and the externalized single meaning with the lowest possible
conceptual model cognitive effort
Empirical quality It is reflected by the error frequency Heavy cognition loads may cause frequent
when a model is read or written errors when conceptual models are being
written or read
Social quality The agreement on participant knowledge Difference in the frames of reference is the
and individual interpretation root reason for errors in social quality
provide a frame of reference for what is known and believed, and exhibit the reasoning behind
purposeful actions [24].
In contrast, cognitive mapping describes a set of techniques used to identify subjective beliefs
and to portray these beliefs externally [24]. The general approach is to extract subjective state-
ments from individuals, in particular within the realm of problem domains, about meaningful
concepts and relations among these concepts, and to then describe these concepts and relations
in some kind of visuospatial layout [58]. The outcome of a cognitive mapping technique is usually
referred to as a cognitive map, which may cause confusion in relation to the conceptual term used
in psychology. Eden [22] clarified the nature of cognitive maps as an artifact rather than the con-
ceptual device developed by Tolman [60]. To avoid confusion, we use the term ‘‘cognitive maps’’
to exclusively refer to the outcomes of cognitive mapping techniques in the following parts of this
paper.
Various cognitive mapping techniques (e.g., [5,13,21]) have been widely used in the study of soci-
ology, political science, organizational behavior, and strategic management. Carley [12] pointed
out that cognitive mapping combines elements of both content analysis and procedural mapping.
When coding texts into cognitive maps, the researcher will determine what concepts are present
(content analysis) and extract the relationships between those concepts (procedural mapping).
Some cognitive mapping techniques have been used in IS research, in particular the require-
ments engineering research. For example, Montazemi and Conrath [39] suggested the use of cau-
K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365 353
ping describe cause–effect relationships, while semantic mapping and concept mapping describe
salient concepts and/or their spatial structures. Table 2 summarizes the main characteristics of
the three cognitive mapping techniques.
The produced maps from these cognitive mapping techniques are only a reconstruction of sub-
jective beliefs that people use to interpret the problem domain and take actions accordingly. They
are shaped by the specific method used to elicit the map [58]. Eden [22] pointed out the reasonable
use of these cognitive maps as:
Table 2
Characteristics of three cognitive mapping techniques
Causal mapping Semantic mapping Concept mapping
Basic elements Concepts expressed in A main idea and branches Concepts and
single-polar or bi-polar phrase, of sub-ideas labeled links
causal relationships among concepts
Theoretical foundations Personal constructs theory [31] Mind map [10] Knowledge
assimilation [41]
Structure Complex network Tree-like structure Complex network
Focus Cause–effect structure of concepts Organization of sub-ideas Semantics-rich links
around the main idea among various concepts
356 K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365
(1) They may represent subjective data more meaningfully than other models and so have
utility for researchers interested in subjective knowledge, and (2) they may act as a tool to
facilitate decision-making, problem-solving, and negotiation within the context of organiza-
tional intervention (p. 262).
Based on a survey of action-oriented research in cognitive mapping, Fiol and Huff [24] summa-
rized a number of important direct functions and the associated indirect impacts of cognitive map-
ping techniques.
(i) Cognitive maps are able to focus attention and trigger memory.
(ii) Cognitive maps can help highlight priorities and key factors.
(iii) Cognitive maps may supply missing information.
(iv) Cognitive maps can reveal gaps in information or reasoning that need more direct attention.
These diagnostic functions are especially useful for conceptual modeling. In particular, by
focusing attention and trigger memory, cognitive mapping can alleviate some cognitive biases,
such as availability heuristics, and automaticity. The highlighted priorities and key factors help
reduce the burden of anchoring and adjustment. The gaps revealed through cognitive mapping
can be used to identify faulty reasoning and wrong representation. This point has been briefly ad-
dressed in Byrd et al. (11, p. 130) when they discussed the use of cognitive mapping in require-
ments analysis and knowledge acquisition. By helping to alleviate cognitive biases, cognitive
mapping techniques can improve the correspondence between the domain and participant knowl-
edge in conceptual mapping.
More recently, Topi and Ramesh [61] suggested that the conceptual model used as a commu-
nication tool between analysts and users should not be as formal and restrictive as that used be-
tween analysts and developers. This point of view was shared by Siau and Tan [52] when they
proposed the use of cognitive mapping techniques, in particular, semantic mapping and causal
mapping, as a communication tool between analysts and end-users. Through a case study, Siau
and Tan [52] discussed the applicability of these cognitive mapping techniques in overcoming
the communication obstacles ‘‘within’’ individual users, ‘‘between’’ users and analysts, and
‘‘among’’ users [63]. As cognitive mapping techniques may represent subjective knowledge more
meaningfully, they can improve the pragmatic quality and physical quality in conceptual model-
ing when they (as an informal language) are used to generate conceptual models. This is because
relatively less cognitive effort is needed to draw or interpret a cognitive map. Furthermore, the
diagnostic quality of the cognitive mapping process can be used to reveal the gaps between par-
ticipant knowledge and individual interpretation. Therefore, cognitive mapping is also useful to
improve the perceived semantic quality in conceptual modeling.
Cognitive mapping can also improve the social quality in conceptual mapping. As a way to eli-
cit an individualÕs cognitive structure regarding the problem, cognitive mapping facilitates the
understanding of different frames of reference among system users, managers, and system ana-
lysts. Better understanding will promote the agreement on participant knowledge and individual
interpretation.
Finally, graph-layout models, in the form of cognitive maps, can improve the readability of
conceptual models. In addition, we can follow the guidelines for graph aesthetics [59] in creating
K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365 357
Table 3
How cognitive mapping can improve quality in conceptual modeling
Quality type or influential factor How cognitive mapping is used How quality is improved
Correspondence between Cognitive mapping is used as a Reveal the errors in participant
modeling domain and diagnostic and communication tool knowledge with relation to the domain
participant knowledge Alleviate human cognition limitations
Social quality Cognitive mapping is used as a Help understand the different frames of
diagnostic and communication tool reference regarding the problem,
thus improve the possibility of
reaching agreement
Pragmatic and physical quality Cognitive mapping is used as an Relatively smaller cognitive effort is
informal language needed to create or interpret a
cognitive map
Perceived semantic quality Cognitive mapping is used as a Reveal the gaps between participant
diagnostic and communication tool knowledge and individual interpretation
Empirical quality Graph-layout models following graph Improve the readability of conceptual
aesthetics guidelines models and reduce error frequencies
cognitive maps to reduce the error frequencies when maps are read by users. As such, empirical
quality can be improved by cognitive mapping techniques.
Table 3 summarizes how cognitive mapping can be used to improve quality in conceptual
modeling.
Parallel with the emerging socio-technical approach to the analysis of IS in organizational con-
texts, many systems development methodologies recognize the discernible impact of human and
organizational issues on systems development. Established systems development methodologies,
such as organization-oriented Soft Systems Methodology (SSM) [15], are especially useful in man-
aging systems development projects in ill-structured situations. SSM emphasizes on building the
right system, which is consistent with the fundamental principle of requirements engineering. Cog-
nitive mapping techniques, with aforementioned functions, can supplement these methodologies
as tools to improve the quality of conceptual models.
In this section, we use a case to demonstrate the advantage of using cognitive mapping in
SSM. The case is based on a standard case used in the first conference by IFIP (International
Federation for Information Processing) WG (Working Group) 8.1. In an effort to compare
various systems development methodologies, the conference invited researchers of methodologies
to apply their methodologies to this standard case mentioned in [42]. The case is about organiz-
ing an IFIP conference (see Appendix A). It must be noted that our case study is not empirical in
nature. Therefore, just like those proceedings in [42], a variety of assumptions were made in our
case study, of which the objective is to descriptively demonstrate the usefulness of the cognitive
mapping techniques. Also, our focus is on improving the quality of conceptual models. As such,
we only discuss the activities associated with requirements determination and conceptual
modeling.
358 K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365
SSM is an acronym for Soft Systems Methodology, which was developed by Checkland [15]. In
essence, SSM is a framework to guide the problem solver to investigate complex situations. The
fuzzy, ill-structured situations are common in information systems development in organizations.
Instead of attempting to solve a pre-defined problem, SSM fosters learning and appreciation of
the problem situation between stakeholders.
SSM Mode 1, described in [14,15], is the version most commonly referred to and the most
useful in an information systems context [4]. The stages involved in SSM include:
• Initial work involves interviews and meetings to gain an understanding of the problem situa-
tion, which is represented by the use of ‘‘rich pictures’’.
• Systems thinking uses concepts of hierarchy, communication, control, and emergent properties
to identify ‘‘relevant systems’’ which may provide useful insights. These relevant systems are
logically defined by constructing ‘‘root definitions’’, which are then used to generate ‘‘concep-
tual models’’ of the selected systems.
• Different conceptual models representing different viewpoints are then used as the basis for dis-
cussion among stakeholders, who through an ‘‘appreciative process’’ can reach consensus
about the feasibility and desirable change, and then action.
SSM can be fitted in the information systems development process by using it as a ‘‘front-end’’
before proceeding to the physical aspects of systems development [3]. In other words, SSM can be
applied as a requirements engineering methodology because it is concerned about ‘‘building the
right system.’’ SSM provides all actors, including system analysts, users of the system, and owner
of the system, opportunities to understand the problem situation.
5.2. How cognitive mapping can supplement SSM in solving the case?
In SSM, rich pictures are used by analysts to express the problem situation. A rich picture is a
pictorial caricature of an organization and helps to explain what the organization is about [3]. The
problem solver (analysts) constructs a rich picture by observing or interviewing various stakehold-
ers. The three most important components of a rich picture are structure, process, and concerns
[38], which are used to identify two main aspects of the human activity system—primary tasks and
concerns. Fig. 7 is a rich picture constructed for the case. Key tasks can be identified by the links
among actors, i.e., the flow of information or documents. Main concerns are depicted as the think
bubbles.
It is obvious that the rich picture is intended to be a broad, high-grained view of the problem
situation. Cognitive mapping techniques can be used in two ways. The first is to use cognitive
mapping as a communication tool when analysts conduct interviews with various stakeholders.
The second way is to decompose a rich picture into cognitive maps of greater detail.
To have a correct understanding of the problem situation, the analyst must be interested in such
factors as interfaces, boundaries, subsystems, control of resources, organizational structure, roles
of personnel, organizational goals, employee needs, and concerns. A high number of information
systems development failures are due to the ignorance of these issues. Before constructing the rich
K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365 359
picture, i.e., the broad understanding of the problem situation, the analyst often interview various
stakeholders (users of the system) to understand their specific issues. Here, cognitive mapping can
be used, before the construction of the rich picture, as an effective communication tool between
the analysts and the users. Fig. 8 shows a causal map of a program committee chair regarding
the problem situation (The case description does not include this information. The causal map,
for demonstration purpose, is created based on an assumptive interview with a professor who
has served on many conferencesÕ program committees).
A rich picture may be very complex in structure, especially when the problem situation is very
complex or ill-structured. In some situations, it is not possible to represent the organization with
one rich picture. In addition, the complexity of the organization, which the users are familiar with
and understand well, may not be apprehended by an outsider looking at the rich picture. In either
case, further detail could be shown on separate cognitive maps. The following concept map (Fig.
9) describes the semantic relationships among key entities regarding the paper submission and the
review process.
360 K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365
5.3. Summary
investigation. After the rich picture is created, we can also use cognitive mapping to decompose it
into greater detail.
In this paper, we examine the use of cognitive mapping as a communication and analytical tool
in conceptual modeling. Many frameworks have been proposed to provide better and in-depth
understanding of quality in conceptual modeling. Lindland et al.Õs [35] framework has gained
extensive recognition. Krogstie and Sølvberg [34] extended the framework by highlighting some
human-related factors that affect a modelÕs overall quality. We review human cognition at both
individual level and socio-cognitive level in order to identify the underpinnings of these human-
related factors. We find that cognitive mapping may effectively overcome some human cognitive
limitations and biases. In addition, cognitive mapping can facilitate understanding among various
stakeholders. In view of its utility in addressing these human and organizational issues, we use a
conference organizing case to discuss how cognitive mapping can supplement a popular socio-
technical systems development methodology.
Being widely used in the management studies to investigate managerial and organizational
cognition, cognitive mapping has great potential to be applied in conceptual modeling process
to improve the quality of conceptual models.
The Organizing Committee begins by preparing a list of people to invite to the conference. The
committee issues priority invitations to National Representatives and to members of related
Working Groups. The organizing committee also ensures that each of the contributing authors
receives an invitation. Individuals who receive an invitation and who intend to come to the con-
ference must indicate their intent to participate by sending an acceptance of invitation to the
Organizing Committee. The final list of participants is then generated by the Committee, who
makes an effort to avoid sending duplicate invitations to any individual.
References
[1] F. Ackerman, C. Eden, SODA—Journey making and mapping in practice, in: J. Rosenhead, J. Mingers (Eds.),
Rational Analysis for a Problematic World Revisited—Problem Structuring Methods for Complexity, Uncertainty
and Conflict, John Wiley & Sons, Chichester, 2001, pp. 43–60.
[2] D. Ausubel, Educational Psychology: A Cognitive View, Holt, Rinehart, and Winston, New York, 1968.
[3] D.E. Avison, G. Fitzgerald, Information Systems Development: Methodologies, Techniques and Tools, second
ed., McGraw-Hill, Glasgow, 1995.
[4] D.E. Avison, G. Fitzgerald, Information Systems Development: Methodologies, Techniques and Tools, third ed.,
McGraw-Hill, New York, 2003.
[5] R. Axelrod, Structure of Decision, Princeton University Press, Princeton, NJ, 1976.
[6] P. Berger, T. Luckmann, The Social Construction of Reality: a Treatise in the Sociology of Knowledge,
Doubleday, Garden City, NY, 1966.
[7] R.P. Bostrom, Successful application of communication techniques to improve the systems development process,
Information & Management 16 (5) (1989) 279–295.
[8] G.J. Browne, V. Ramesh, Improving information requirements determination: a cognitive perspective, Information
& Management 39 (8) (2002) 625–645.
[9] J. Bubenko, C. Rolland, P, Loucopoulos, V. DeAntonellis, facilitating ‘‘fuzzy to formal’’ requirements modelling,
in: Proceedings of IEEE International Conference on Requirements Engineering, Colorado, 1994, pp. 154–158.
[10] T. Buzan, The Mind Map Book, BBC Books, London, 1993.
[11] T.A. Byrd, K.L. Cossick, R.W. Zmud, A synthesis of research on requirements analysis and knowledge acquisition
techniques, MIS Quarterly 16 (1) (1992) 117–138.
[12] K.M. Carley, Extracting team mental models through textual analysis, Journal of Organizational Behavior 18
(1997) 533–558.
[13] K.M. Carley, M. Palmquist, Extracting, representing and analyzing mental models, Social Forces 70 (1992) 601–636.
[14] P. Checkland, Systems Theory, Systems Practice, John Wiley & Sons, Chichester, 1981.
[15] P. Checkland, Soft systems methodology: an overview, Journal of applied systems 15 (1988) 27–30.
[16] E.J. Davidson, Technology frames and framing: a socio-cognitive investigation of requirements determination,
MIS Quarterly 26 (4) (2002) 329–358.
[17] A.M. Davis, P. Hsia, Giving voice to requirements engineering, IEEE Software 11 (2) (1994) 12–16.
[18] A.M. Davis et al., Identifying and measuring quality in a software requirements specification, in: Proceedings of
First International Software Metrics Symposium, Baltimore, 1993, pp. 141–152.
[19] G.B. Davis, Strategies for information requirements determination, IBM Systems Journal 21 (1) (1982) 4–30.
[20] D. Dougherty, Interpretive barriers to successful product innovation in large firms, Organization Science 3 (2)
(1992) 179–202.
[21] C. Eden, Cognitive mapping, European Journal of Operational Research 36 (1988) 1–13.
[22] C. Eden, On the nature of cognitive maps, Journal of Management Studies 29 (3) (1992) 261–265.
[23] O. El Sawy, T. Pauchant, Triggers, templates, and twitches in the tracking of emerging strategic issues, Strategic
Management Journal 9 (5) (1988) 445–473.
[24] C.M. Fiol, A.S. Huff, Maps for managers: where are we? where do we go from here?, Journal of Management
Studies 29 (3) (1992) 267–285.
K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365 363
[25] U. Frank, Conceptual modeling as the core of the information systems discipline—perspectives and epistemo-
logical challenges, in: Proceedings of Fifth AmericaÕs Conference on Information Systems AMCIS 99, Milwaukee,
WI, 1999, pp. 695–697.
[26] B. Henderson-Sellers, A. Simons, H. Younessi, The OPEN Toolbox of Techniques, Addison-Wesley, Harlow,
1998.
[27] K. Holtzblatt, H.R. Beyer, Requirements gathering: the human factor, Communications of the ACM 38 (5) (1995)
30–32.
[28] A.S. Huff, Mapping Strategic Thought, Wiley, New York, 1990.
[29] H.J. Jeffrey, A.O. Putnam, Relationship definition and management: tools for requirements analysis, Journal of
Systems and Software 24 (1994) 277–294.
[30] D.H. Jonassen, K. Beissner, M.A. Yacci, Structural Knowledge: Techniques for Conveying, Assessing, and
Acquiring Structural Knowledge, Lawrence Erlbaum, Hillsdale, NJ, 1993.
[31] G.A. Kelly, The Psychology of Personal Constructs, Norton, New York, 1955.
[32] J. Krogstie, Integrating the understanding of quality in requirements specification and conceptual modeling, ACM
SIGSOFT Software Engineering Notes 23 (1) (1998) 86–91.
[33] J. Krogstie, O.I. Lindland, G. Sindre, Toward a deeper understanding of quality in requirements engineering, in:
Proceedings of IFIP 8.1 Working Conference on Information Systems Concepts (ISCO3): Towards a
Consolidation of Views, 1995, pp. 82–95.
[34] J. Krogstie, A. Sølvberg, Information Systems Engineering—Conceptual Modeling in a Quality Perspective,
Kompendiumforlaget, Trondheim, Norway, 2003.
[35] O.I. Lindland, G. Sindre, A. Sølvberg, Understanding quality in conceptual modeling, IEEE Software 11 (2)
(1994) 42–49.
[36] J. Luft, Of Human Interaction, National Press, Palo Alto, CA, 1969.
[37] G.A. Miller, The magical number seven, plus or minus two: some limits on our capacity for processing
information, Psychological Review 63 (1956) 81–97.
[38] A. Monk, S. Howard, The rich picture: a tool for reasoning about work context, Interactions 5 (2) (1998)
21–30.
[39] A.R. Montazemi, D.W. Conrath, The use of cognitive mapping for information requirements analysis, MIS
Quarterly 10 (1) (1986) 45–56.
[40] D.L. Moody, G. Sindre, T. Brasethvik, A. Sølvberg, Evaluating the quality of information models: empirical
testing of a conceptual model quality framework, in: Proceedings of 25th International Conference on Software
Engineering, Portland, OR, 2003, pp. 295–305.
[41] J.D. Novak, How do we learn our lesson? taking students through the process, The Science Teacher 60 (3) (1993)
50–55.
[42] T.W. Olle, H.G. Sol, A.A. Verrigin-Stuart, Information Systems Design Methodologies: A Comparative Review,
North Holland, Amsterdam, 1982.
[43] W.J. Orlikowski, D.C. Gash, Technological frames: making sense of information technology in organizations,
ACM Transactions on Information Systems 12 (2) (1994) 174–207.
[44] D.N. Perkins, R. Allen, J. Hafner, Difficulties in everyday reasoning, in: W. Maxwell (Ed.), Thinking: The
Expanding Frontier, Franklin Institute Press, Philadelphia, PA, 1983.
[45] K. Pohl, The three dimensions of requirements engineering: a framework and its applications, Information systems
19 (3) (1994) 243–258.
[46] K. Pohl, Process-centered Requirements Engineering, John Wiley & Sons, New York, 1996.
[47] S. Sheetz, D. Tegarden, Distributed capture of system requirements as individual and group cognitive maps, in:
Proceedings of Fourth Americas Conference on Information Systems, Baltimore, MD, 1998, pp. 983–984.
[48] S. Sheetz, D. Tegarden, K.A. Kozar, I. Zigurs, A group support systems approach to cognitive mapping, Journal
of Management Information Systems 11 (1) (1994) 31–57.
[49] K. Siau, Information modeling and method engineering: a psychological perspective, Journal of Database
Management 10 (4) (1999) 44–50.
[50] K. Siau, L. Lee, Are use case and class diagrams complementary in requirements analysis?—an experimental study
on use case and class diagrams in UML, Requirements Engineering 9 (4) (2004) 229–237.
364 K. Siau, X. Tan / Data & Knowledge Engineering 55 (2005) 343–365
[51] K. Siau and P. Loo, Difficulties in learning UML: a concept mapping analysis, in: Proceedings of the Seventh
CAiSE/IFIP8.1 International Workshop on Evaluation of Modeling Methods in Systems Analysis and Design,
Toronto, Canada, 2002, pp. 102–108.
[52] K. Siau, X. Tan, Information systems requirements determination and analysis: a mental modeling approach, in:
Proceedings of the Ninth Americas Conference on Information Systems, Tampa, FL, 2003, pp. 1370–1379.
[53] K. Siau, Y. Tian, The complexity of unified modeling language—a GOMS analysis, in: Proceedings of the
Fourteenth International Conference on Information Systems, New Orleans, LA, 2001, pp. 443–448.
[54] K. Siau, Y. Wand, I. Benbasat, When parents need not have children—cognitive biases in information modeling,
Lecture Notes in Computer Science—Advanced Information Systems Engineering 1080 (1996) 402–420.
[55] K. Siau, Y. Wand, I. Benbasat, The relative importance of structural constraints and surface semantics in
information modeling, Information Systems 22 (2–3) (1997) 155–170.
[56] H.A. Simon, A behavioral models of rational choice, Quarterly Journal of Economics 69 (1955) 99–118.
[57] H.A. Simon, Information processing models of cognition, Annual Review of Psychology 30 (1979) 363–396.
[58] J. Swan, Using cognitive mapping in management research: decisions about technical innovation, British Journal
of Management 8 (2) (1997) 183–198.
[59] R. Tamassia, G.D. Battista, C. Batini, Automatic graph drawing and readability of diagrams, IEEE Transactions
on Systems, Man, and Cybernetics 18 (1) (1988) 61–79.
[60] E. Tolman, Cognitive maps in rats and men, Psychological Review 55 (1948) 189–208.
[61] H. Topi, V. Ramesh, Human factors research on data modeling: a review of prior research, an extended framework
and future research directions, Journal of Database Management 13 (2) (2002) 3–19.
[62] A. Tversky, D. Kahneman, Judgment under uncertainty: heuristics and biases, in: D. Kahneman, P. Slovic, A.
Tversky (Eds.), Judgment Under Uncertainty: Heuristics and Biases, Cambridge University Press, New York,
1982, pp. 3–22.
[63] J.R. Valusek, D.G. Fryback, Information requirements determination: obstacles within, among, and between
participants, in: R. Galliers (Ed.), Information Analysis: Selected Readings, Addison-Wesley, Reading, MA, 1987,
pp. 139–151.
[64] J.P. Walsh, Managerial and organizational cognition: notes from a trip down memory lane, Organization Science 6
(3) (1995) 280–321.
[65] J.P. Walsh, L. Fahey, The role of negotiated belief structures in strategy making, Journal of Management 31 (1986)
873–896.
[66] R. Weber, Ontological Foundations of Information Systems, Coopers & Lybrand, Melbourne, 1997.
[67] K.E. Weick, Cognitive processes in organizations, Research in Organizational Behavior 1 (1) (1979) 41–74.
[68] K.E. Weick, The Social Psychology of Organizing, Addison-Wesley, Reading, MA, 1979.
[69] K.E. Weick, Sensemaking in Organizations, Sage, Thousand Oaks, CA, 1995.
Xin Tan is currently a Ph.D. student in the Management Department at the University of
Nebraska, Lincoln. He obtained his B.E. from Shanghai Jiao Tong University, and M.B.A. from
Miami University. His current research interests include end-userÕs acceptance of advanced
information technology, systems analysis and design, conceptual and data modeling methods.