MODULE – V

Cognitive models - Socio-Organizational issues and stake holder requirements –Goal and task hierarchies
Design Focus: GOMS saves money, Linguistic models, Cognitive architectures, Ubiquitous computing
and augmented realities, Information and data visualization –Communication and collaboration models-
Hypertext.
-------------------------------------------------------------------------------------------------------------------------------------------
COGNITIVE MODELS
A Cognitive model is the designer’s intended mental model for the user of the system: a set of ideas about
how it is organized and operates.

Cognitive modeling is an area of computer science that deals with simulating human problem-solving and
mental processing in a computerized model. Such a model can be used to simulate or predict human
behavior or performance on tasks similar to the ones modeled and improve human-computer interaction.
 hierarchical representation of the user’s task and goal structure
 linguistic and grammatical models
 physical and device-level models.
The first category deals directly with the issue of formulation of goals and tasks.

The second deals with the grammar of the articulation translation and how it is understood by the user.

The third category again deals with articulation, but at the human motor level instead of at a higher level of
human understanding.

Many of these nominally cognitive models have a rather computational flavor.

This reflects the way that computational analogies are often used in cognitive Goal and task hierarchies
psychology. The similarity between the language describing the user and that describing the computer has
some advantages and some dangers. On the positive side it makes communication and analysis of the
combined human–computer system easier.

SOCIO-ORGANIZATIONAL ISSUES AND STAKEHOLDER REQUIREMENTS

 There are several organizational issues that affect the acceptance of technology by users and that
must therefore be considered in system design:
o systems may not take into account conflict and power relationships
o those who benefit may not do the work
o not everyone may use systems.
 In addition to generic issues, designers must identify specific stakeholder requirements within their
organizational context.
 Socio-technical models capture both human and technical requirements.
 Soft systems methodology takes a broader view of human and organizational issues.
 Participatory design includes the user directly in the design process.
 Ethnographic methods study users in context, attempting to take an unbiased perspective.

• Organizational issues affect acceptance

– conflict & power, who benefits, encouraging use
• Stakeholders
– identify their requirements in organizational context
• Socio-technical models
– human and technical requirements
• Soft systems methodology
– broader view of human and organizational issues
• Participatory design
– includes the user directly in the design process
• Ethnographic methods
– study users in context, unbiased perspective
Organizational issues
Organizational factors can make or break a system Studying the work group is not sufficient
– any system is used within a wider context
– and the crucial people need not be direct users
Before installing a new system must understand:
– who benefits
– who puts in effort
– the balance of power in the organization
… and how it will be affected
Even when a system is successful
… it may be difficult to measure that success
Conflict and power
CSCW = computer supported cooperative work
– people and groups have conflicting goals
– systems assuming cooperation will fail!
e.g. computerize stock control
stockman looses control of information
⇒ subverts the system
identify stakeholders – not just the users
Organizational structures
• Groupware affects organizational structures
– communication structures reflect line management
– email – cross-organizational communication
Disenfranchises lower management
⇒ disaffected staff and ‘sabotage’
Technology can be used to change management
style and power structures
– but need to know that is what we are doing
– and more often an accident !
who are the stakeholders?
• system will have many stakeholders with potentially conflicting interests
• stakeholder is anyone effected by success or failure of system
– primary - actually use system
– secondary - receive output or provide input
– tertiary - no direct involvement but effected by success or failure
– facilitating - involved in development or deployment of system
•designers need to meet as many stakeholder needs as possible
– usually in conflict so have to prioritise
– often priority decreases as move down
categories e.g. primary most important
– not always e.g. life support machine
socio-technical modelling
• response to technological determinism (view that social change is determined by technology)
– In contrast socio-technical systems view focuses on interrelationship of human and machine elements
– Human issues should not be overwhelmed by technical considerations
• concerned with technical, social, organizational and human aspects of design
• describes impact of specific technology on organization
• information gathering: interviews, observation, focus groups, document analysis
• several approaches e.g. – CUSTOM, OSTA
OSTA (Open System Task Analysis)
• Eight stage model - focus on task
– primary task identified in terms of users’ goals
– task inputs to system identified
– external environment into which the system will be introduced is described, including physical,
economic and political aspects
– transformation processes within the system are described
in terms of actions performed on or with objects
– social system is analyzed, considering existing internal and external work-groups and relationships
– technical system is described in terms of configuration and integration with other systems
– performance satisfaction criteria are established, indicating social and technical requirements of system
– new technical system is specified
Participatory design
In participatory design: workers enter into design context
In ethnography (as used for design): designer enters into work context
Both make workers feel valued in design
… encourage workers to ‘own’ the products
•User is an active member of the design team.
• Characteristics
– context and work oriented rather than system oriented
– collaborative
– iterative
• Methods
– brain-storming
– storyboarding
– workshops
– pencil and paper exercises
GOAL AND TASK HIERARCHIES
Many models make use of a model of mental processing in which the user achieves goals by solving
subgoals in a divide-and-conquer fashion.
We will consider two models,
 GOMS
 CCT
 Imagine we want to produce a report on sales of introductory HCI textbooks.
 To achieve this goal we divide it into several subgoals, say gathering the data together, producing
the tables and histograms, and writing the descriptive material.
 Concentrating on the data gathering, we decide to split this into further subgoals find the names of
all introductory and then search the book sales database for these books.
 Similarly, each of the other subgoals is divided up into further subgoals, until some level of detail
is found at which we decide to stop.
  We end up with a hierarchy of goals and subgoals. The example can be laid out to expose this
structure:
Example:
produce report
gather data
. find book names
. . do keywords search of names database
<<further subgoals>>
. . sift through names and abstracts by hand
<<further subgoals>>
. search sales database
<<further subgoals>>
layout tables and histograms
<<further subgoals>>
write description
<<further subgoals>>
A goal hierarchy may show how the perfect user would achieve a goal, but what can it say about difficulties
the user may have along the way? In general, pre-diction of error behavior is poor amongst these
hierarchical modeling techniques, though some(cognitive complexity theory (CCT), for example) can
represent error behavior.

GOMS (saves money)

A GOMS description consists of these four elements:
1. G -Goal
2. O-operators
3. M- Methods
4. S- Selection
Goals :
 These are the user’s goals, describing what the user wants to achieve.
 Further, in GOMS the goals are taken to represent a ‘memory point’ for the user, from which he can
evaluate what should be done and to which he may return should any errors occur.
Operators:
 These are the lowest level of analysis. They are the basic actions that the user must perform in order
to use the system.
 They may affect the system (for example, press the ‘X’ key) or only the user’s mental state (for
example, read the dialog box).
 There is still a degree of flexibility about the granularity of operators; we may take the command
level ‘issue the SELECT command’ or more primitive: ‘move mouse to menu bar, press center
mouse button .
Methods :
 As we have already noted, there are typically several ways in which a goal
can be split into subgoals.
 Selected window can be closed to an icon either by selecting the ‘CLOSE’ option from a pop-up
menu, or by hitting the ‘L7’ function key.
 In GOMS these two goal decompositions are referred to as methods, so we have the CLOSE-
METHOD and the L7-METHOD:
EXAMPLE:
GOAL: ICONIZE-WINDOW
. [select GOAL: USE-CLOSE-METHOD
. . MOVE-MOUSE-TO-WINDOW-HEADER
 . . POP-UP-MENU
. . CLICK-OVER-CLOSE-OPTION
GOAL: USE-L7-METHOD
. . PRESS-L7-KEY]
The dots are used to indicate the hierarchical level of goals.
Selection:
 GOMS does not leave this as a random choice, but attempts to predict which methods will be used.
 This typically depends both on the particular user and on the state of the system and details about
the goals.
 where the mouse needs to be used in the game until the very moment the key is pressed. GOMS
captures this in a selection rule for Sam:
 User Sam:
Rule 1: Use the CLOSE-METHOD unless another rule applies.
Rule 2: If the application is ‘blocks’ use the L7-METHOD.
 Problems with goal hierarchies
• a post hoc technique
• expert versus novice
• How cognitive are they?
LINGUISTIC MODELS
The user’s interaction with a computer is often viewed in terms of a language, so it is not surprising
that several modeling formalisms have developed centered around this concept.
 Understanding the user's behaviour and cognitive difficulty based on analysis of language between
user and system.
 Similar in emphasis to dialogue models
1. Backus–Naur Form (BNF)
2. Task–Action Grammar (TAG)
1.Backus-Naur Form (BNF):
 Representative of the linguistic approach is Reisner’s use of Backus–Naur Form (BNF) rules to describe
the dialog grammar. This views the dialog at a purely syntactic level, ignoring the semantics of the
language.
 BNF has been used widely to specify the syntax of computer programming languages, and many system
dialogs can be described easily using BNF rules.
 For example, imagine a graphics system that has a line-drawing function.
 To select the function the user must select the ‘line’ menu option. The line-drawing function allows the
user to draw a polyline, that is a sequence of line arcs between points. The user selects the points by
clicking the mouse button in the drawing area. The user double clicks to indicate the last point of the
polyline
 Terminals – lowest level of user behaviour
o e.g. CLICK-MOUSE, MOVE-MOUSE
 Nonterminal – ordering of terminals – higher level of abstraction
o e.g., select-menu, position-mouse

Example of BNF
 Basic syntax:
– nonterminal: = expression
 An expression
 The names in the description are of two types: – contains terminals and non-terminals
non-terminals, shown in lower case, and terminals, shown in upper case.
Terminals represent the lowest level of user behavior, such as pressing a key, clicking a mouse
button or moving the mouse
 Non-terminals are higher-level abstractions. The non-terminals are defined in terms of other non-terminals
and terminals by a definition of the form
  combined in sequence (+) or as alternatives (|)

draw-line ::= select-line + choose-points + last-point

select-line ::= position-mouse + CLICK-MOUSE
choose-points ::= choose-one | choose-one + choose-points
choose-one ::= position-mouse + CLICK-MOUSE
last-point ::= position-mouse + DOUBLE-CLICK-MOUSE
position-mouse ::= empty | MOVE-MOUSE + position-mouse

Measurements with BNF:

• Number of rules (not so good)
• Number of + and | operators
• Complications
– same syntax for different semantics
– no reflection of user's perception
– minimal consistency checking

2.Task–Action Grammar (TAG)

 Measures based upon BNF have been criticized as not ‘cognitive’ enough. They ignore the advantages
of consistency both in the language’s structure and in its use of command names and letters.

 Task–action grammar (TAG) attempts to deal with some of these problems by including elements such
as parametrized grammar rules to emphasize consistency and encoding the user’s world knowledge
(for example, up is the opposite of down).
To illustrate consistency, we consider the three UNIX commands: cp (for copying files), mv (for
moving files) and ln (for linking files). Each of these has two possible forms.

 They either have two arguments, a source and destination filename, or have any number of source
filenames followed by a destination directory:
 Task–Action Grammar (TAG) can
• Making consistency more explicit
• Encoding user's world knowledge
• Parameterised grammar rules
• Nonterminal are modified to include additional semantic features
Consistency in TAG
• In BNF, three UNIX commands would be described as:
copy ::= cp + filename + filename | cp + filenames + directory
move ::= mv + filename + filename | mv + filenames + directory
link ::= ln + filename + filename | ln + filenames + directory
• No BNF measure could distinguish between this and a less consistent grammar in which
link ::= ln + filename + filename | ln + directory + filenames
• consistency of argument order made explicit using a parameter, or semantic feature for file operations
• Feature Possible values
Op = copy; move; link
• Rules
file-op[Op] ::= command[Op] + filename + filename
| command[Op] + filenames + directory
command[Op = copy] ::= cp
command[Op = move] ::= mv
command[Op = link] ::= ln
 Other uses of TAG
• User’s existing knowledge
• Congruence between features and commands
• These are modelled as derived rules
UBIQUITOUS COMPUTING AND AUGMENTED REALITIES
• ubiquitous computing
– filling the real world with computers
• virtual and augmented reality
– making the real world in a computer!
Challenging HCI Assumptions
• What do we imagine when we think of a computer?
“The most profound technologies are those that disappear.”
• 1990’s: this was not our imagined
Ubiquitous Computing
• Any computing technology that permits human interaction away from a single workstation
• Implications for
– Technology defining the interactive experience
– Applications or uses
– Underlying theories of interaction
Scales of devices
• Weiser proposed
– Inch
– Foot
– Yard
• Implications for device size as well as relationship to people
Device scales
• Inch
– PDAs
– PARCTAB
– Voice Recorders
– smart phones
• Individuals own many of them and they can all communicate with each other and environment.
• Foot
– notebooks
– tablets
– digital paper
• Individual owns several but not assumed to be always with them.
• Yard
– electronic whiteboards
– plasma displays
– smart bulletin boards
• Buildings or institutions own them and lots of people share them.
Defining the Interaction Experience
• Implicit input
– Sensor-based input
– Extends traditional explicit input (e.g., keyboard and mouse)
– Towards “awareness”
– Use of recognition technologies
– Introduces ambiguity because recognizers are not perfect
Multi-scale and distributed output
• Screens of many sizes
– (very) small
 – (very) large

• Distributed in space, but coordinated

The output experiences
• More than eye-grabbing raster displays
– Ambient: use features of the physical environment to signal information
– Peripheral: designed to be in the background
• Examples:
– The Dangling String
– The Water Lamp (shown)

Merging Physical and Digital Worlds

• How can we remove the barrier?
– Actions on physical objects have meaning electronically, and vice versa

– Output from electronic world superimposed on physical world

Application Themes
• Context-aware computing – Sensed phenomena facilitate easier interaction
• Automated capture and access – Live experiences stored for future access
• Toward continuous interaction – Everyday activities have no clear begin-end conditions
New Opportunities for Theory
• Knowledge in the world – Ubicomp places more emphasis on the physical world
• Activity theory – Goals and actions fluidly adjust to physical state of world
• Situated action and distributed cognition – Emphasizes improvisational/opportunistic behavior versus
planned actions
• Ethnography – Deep descriptive understanding of activities in context
Evaluation Challenges
• How can we adapt other HCI techniques to apply to ubicomp settings?
– Ubicomp activities not so task-centric
– Technologies are so new, it is often hard to get long-term authentic summative evaluation
– Metric of success could be very different (playfulness, non-distraction versus efficiency)
ambient wood
• real wood! … filled with electronics
• light and moisture meters
– recorded with GPRS location
– drawn on map later
• ‘periscope’
– shows invisible things
– uses RFID
• triggered sound
City - shared experience
• visitors to Mackintosh Interpretation Centre
– some on web, some use VR, some really there
• interacting
– talk via microphones
– ‘see’ each other virtually
• different places
• different modalities
• shared experience computer!

virtual and augmented reality

VR - technology & experience web, desktop and simulators
AR – mixing virtual and real
virtual reality technology
• headsets allow user to “see” the virtual world
• gesture recognition achieved with DataGlove (lycra glove with optical sensors that measure hand and
finger positions)
• eye gaze allows users to indicate direction with eyes alone
• whole body position sensed, walking etc.
VR headsets
• small TV screen for each eye
• slightly different angles
• 3D effect
immersion
• VR – computer simulation of the real world
• mainly visual, but sound, haptic, gesture too –experience life-like situations
• too dangerous, too expensive – see unseen things:
• too small, too large, hidden, invisible – e.g. manipulating molecules
• the experience – aim is immersion, engagement, interaction
on the desktop
• headset VR – expensive, uncomfortable
• desktop VR – use ordinary monitor and PC
• cheap and convenient
• in games …
• and on the web – VRML – virtual reality markup language
VRML … VR on the web
#VRML V1.0 ascii
Separator {
Separator { # for sphere
Material {
emmissiveColor 0 0 1 # blue
}
Sphere { radius 1 }
}
Transform { translation 4 2 0 }
Separator { # for cone
Texture2 {
filename "big_alan.jpg"
}
Cone {
radius 1 # N.B. width=2*radius
height 3
}}}
command and control
• scenes projected on walls
• realistic environment
• hydraulic rams!
• real controls
• other people
• for:
– flight simulators
– ships
– military
augmented reality (AR)
• images projected over the real world
– aircraft head-up display
– semi-transparent goggles
– projecting onto a desktop
• types of information
– unrelated – e.g. reading email with wearable
– related – e.g. virtual objects interacting with world
• issues
– registration – aligning virtual and real
– eye gaze direction
applications of AR
maintenance
– overlay instructions
– display schematics
examples
– photocopier engineers
• registration critical arrows point to parts
– aircraft wiring looms
• registration perhaps too hard, use schematic
applications of VR
• simulation – games, military, training
• VR holidays – rainforest, safari, surf, ski and moon walk
… all from your own armchair
• medical – surgery
• scans and x-rays used to build model then ‘practice’ operation
• force feedback best – phobia treatment
• virtual lifts, spiders, etc.

between two worlds

• ubiquitous computing – computers fill the real world
• virtual reality and visualisation – real world represented in the computer
• augmented reality, ambient displays … – physical and digital intermingled
… maturity
– VR and visualisation – commonplace
– AR, ubiquity … coming fast!
INFORMATION AND DATA VISUALISATION
VR, 3D and 2D displays
scientific and complex data
interactivity central
scientific and technical data
• number of virtual dimensions that are ‘real’
• three-dimensional space
– visualize invisible fields or values
– e.g. virtual wind tunnel
• two-dimensional space
– can project data value up from plane
– e.g. geographic data
– N.B. viewing angle hard for static visualization
• no ‘real’ dimensions
 – 2D/3D histograms, scatter plots, pie charts, etc.
virtual wind tunnel
• fluid dynamics to simulate air flow
• virtual bubbles used to show movements
• ‘better’ than real wind tunnel …
– no disruption of air flow
– cheaper and faster
structured information
• scientific data – just numbers
• information systems … lots of kinds of data
• hierarchies
– file trees, organization charts
• networks
– program flow charts, hypertext structure
• free text …
– documents, web pages
visualizing hierarchy
• 2D organization chart
– familiar representation
– what happens when it gets wide?

wide hierarchies … use 3D?

networks in 2D
• network or ‘graph’:
– nodes – e.g. web pages
– links – may be directed or not – e.g. links
• planar – can drawn without crossing
• non-planar – any 2D layout has crossings

time and interactivity

• visualising in time
– time dimension mapped to space
– changing values: sales graphs, distance-time
– events: Gantt chart, timelines, historical charts
e.g. Lifelines – visualising medical and court records
• using time
– data dimension mapped to time
– time to itself: fast/slow replay of events
– space to time: Visible Human Project
• interactivity
– change under user control
e.g. influence explorer
COMMUNICATION AND COLLABORATION MODELS
All computer systems, single-user or multi-user, interact with the work-groups and organizations in which
they are used.
• We need to understand normal human–human communication:
– face-to-face communication involves eyes, face and body
– conversation can be analyzed to establish its detailed structure.
• This can then be applied to text-based conversation, which has:
– reduced feedback for confirmation
– less context to disambiguate utterances
– slower pace of interaction but is more easily reviewed.
• Group working is more complex than that of a single person:
– it is influenced by the physical environment
– experiments are more difficult to control and record
– field studies must take into account the social situation.
1.FACE-TO-FACE COMMUNICATION
 It involves speech, hearing, body language and eye-gaze.
 A person has to be familiar with existing norms, to learn a new norm.
 Another factor is the personal space, this varies based on the context, environment, diversity and
culture.
 The above factor comes into pitcher, when there is a video conference between two individuals from
different background.
 The factor of eye gaze is important during a video conference as the cameras are usually mounted
away from the monitor and it is important to have eye contact during a conversation.
 Back channels help giving the listener some clues or more information about the conversation.
 The role of interruptions like 'um's and 'ah's are very important as they can be used by participants in
a conversation to claim the turn.
2.CONVERSATION
 Transcripts can be used as a heavily annotated conversation structure, but still lacks the
back channel information.
 Another structure is of turn-taking, this can be interpreted as Adjacency pairs, e.g.: A-x, B-x, A-y, B-y
 Context varies according to the conversation.
 The focus of the context can also varies, this means that it is difficult to keep track of
context using adjacency pairs.
 Break-downs during conversations is often a case and can be noticed by analyzing the transcripts.
 Reaching a common ground or grounding is very essential to understand the shared context.
 Speech act theory is based on the statements and its propositional meaning.
 A state diagram of the above can be constructed considering these acts as illocutionary points in the
diagram. This is called Conversation for Action.
3.TEXT BASED COMMUNICATION
 Text-based communication is familiar to most people, in that they will have written and received
letters. However, the style of letter writing and that of face-to face communication are very
different. The text-based communication in groupware systems is acting as a speech substitute,
and, thus, there are some problems adapting between the two media.
There are four types of textual communication in current groupware:
i. discrete e.g. email
ii. linear e.g. single transcript
iii. non-linear e.g. linked through hypertext fashion
iv. spatial e.g. messages arranged in 2D surface
 Difference between this and face-to-face communication is that it has lack of back channels and
states
 Turn-taking is the fundamental structure used here.Back channels and affective state
4.GROUP WORKING
 The roles and relationship between the group individuals are different and may change during the
conversation.
 Physical layout is important to consider here to maintain the factors in face-to-face communication.

 Distributed cognition is a theory of human cognition that describes how information processing is
dispersed across people and their workplace, their technologies, and their social organization and how
information processing evolves over time.
HYPERTEXT
Hypertext is a cross referencing tool which connects the links to other text using hyperlinks. Hypertext
is non-linear and multi sequential and it is different from our normal text.
By the help of hypertext one organized way is achieved to present information. This makes the user to
move from one part of the information to another part of the information which is in same page or any
other page.
It makes the documentation simple by providing a way of easy accessible to the end user

Example
Let's look at what a node is now. Each data component in a network is known as a node, and these nodes
connect to other nodes via a link or pointer. This idea is utilized on websites where the hypertext written
over a webpage includes a hyperlink to either the other text alone or to the other webpage.
Advantages
1. Hypertext allows for fast and easy access to information with minimal browsing.
2. New information outside of the person's competence, such as statistics or professional images of an
event, can be attached.
3. It promotes free expression by allowing everyone to view and express their thoughts on an already
discussed topic on the internet.
Disadvantages
1. Hypertext may present false information.
2. Frequently, there is no extra credibility for the original author. Only a portion from prior works is
immediately linked, with no background information. Users may only see the credibility if they go
beyond the hypertext.
3. Information may be untrustworthy