IMAT 3712

Human Computer
Interaction
Thinking and Learning
Acknowledgements
 Some slides have origins in
 Lecture PowerPoint by Shira Elqayam
Thinking and Learning -
Overview
 Mental Actions
 Learning: Acquiring mental actions
 Learning: Skilled Behaviour
 Planning vs Situated Action
 Expert-Novice Differences
 Dual Process Theory
Knowing that and knowing how
 Declarative Knowledge: Knowing that
 Procedural Knowledge: Knowing how

 Declarative Learning:
 More and richer mental representations
 What caused the First World War?
 Procedural Learning:
 More and more refined skills
 How do you knit?
Knowing that and knowing how
 Declarative Knowledge: Knowing that
 Procedural Knowledge: Knowing how

 Separate brain processes

 Losing conscious memories in amnesia doesn’t

affect skills
Mental actions
Minds act purposefully
 Changes in mental state - recognising,

remembering, imagining
 Speech and other communication
 Body movements
Mental actions depend on
Mental State
 Perceptions
 Attention is selective: spotlights focused-on

objects, properties, relationships

 Unattended inputs not completely excluded but

can fail to reach conscious awareness

 Working Memory
 Content of conscious awareness
 Includes goals and desires
 Long Term Memory
 Dependent on activation level
 Includes goals and desires
Mental actions
 Mental actions triggered by
 Trigger conditions enough
 Including goals activation
 Prior activation level to execute

 Mental actions can produce

 New information structures in conscious

awareness
 Speech
 Actions
Learning associations
 We learn associations between
 Information structures that turn up together or in

sequence
 Situations and actions

 We’re sensitive to
 Correlations
 When thoughts and actions are useful or

successful
Learning patterns
 Humans and other animals are very good at
learning
 Patterns
 Correlations
 When we care about what they are telling us
 What things are
 Whether actions succeed or fail

 We can learn to respond correctly to similarities

and differences that are too complicated and
subtle for us to be able to understand what they
are
Learning procedures
Faced with a non-routine problem, learner uses
general purpose rules for thinking about what to do
 Mental representation of problem
 Information from environment

With practice, learner learns how to act in situations

of this type...
 PROCEDURALIZATION
 COMPOSITION

Learner uses rules specific to the task

 TUNING

Learner uses rules tuned to speed up performance

Learning procedures
 Proceduralization: Generating new rules that
 go from input (perceptions and contents of

working memory) to output (thoughts and

actions produced by general-purpose reasoning
with declarative knowledge)
 Increasingly, don’t involve recall of knowledge

that was originally used in reasoning to reach

output

 Composition: Sequences of rules are combined

into one.
Learning procedures
 Tuning:
 Generalization: Rules with different conditions

and same output combined to one with more

general condition.
 Discrimination: Rule applications with ‘same’

condition differentiated to create more, more

precise rules.
 Strengthening: Effective rules become stronger

so more likely to fire, ineffective rules become

weaker.
When does learning happen?
 Successful results lead to
 proceduralization
 composition
 tuning
 generalization

 Discrimination happens on appearance of

failure, so people learn to avoid actions leading to
appearance of failure.
Action schemas
 Action schemata is better Greek

 Schema theory extended to actions

 We learn combination of actions for common

situations
 Sequence of steps
 Hierarchical organization of steps within steps
 Open slots for objects (etc) in particular

categories, filled in by the objects that turn up in

context
 Controversial question in Psychology: Does

structure of action sequence play causal role in

guiding action, or does it emerge from learned
Implications for interface
design
 If possible, ensure that incorrect actions produce
immediate error messages. Otherwise, make the
earlier wrong step visible or easy to find.
 Use cues to highlight differences between
situations where different actions are expected, or
where the same actions will produce different
results.
 Anticipate and support learning of action
sequences for frequent operations.
do it:
Means-Ends Analysis and
Planning
 You need to think!
 Conscious reasoning with consciously

remembered knowledge
 Requires attention (consumes limited mental

resources)
do it:
Means-Ends Analysis and
Planning
 Symbolic problem solving typically involves
 Goals to achieve
 Reasoning about how to achieve goals (Means

Ends Analysis)
 Reasoning about how to break the problem up

into smaller problems (Problem Decomposition

and Setting Subgoals)
do it:
Means-Ends Analysis and
Planning
 Planning is identifying a sequence of goals and
problems forming a path to your ultimate goal
Situated actions: no plans
 Most behaviour is
 tightly linked to particular current circumstances
 much less dependent on planning

 Contingent action is usual (walk to Student

Services office)
 you have a goal and do something that moves

you forward
 complex sequences of actions don’t need plans,

just correct situated actions (drive to Dover car

ferry)
 Opportunistic fulfilment of dormant goals
(take mug downstairs)
Situated actions: no plans
 Most behaviour is
 tightly linked to particular current circumstances
 much less dependent on planning

 Planning is unusual (drive to Bangor; invade

France)
 need to reason about your next goal and how to

achieve it (ocean navigation)

 plan down to the level where sequences of

situated actions will get you there

Situated actions: feedback
Most human actions are dependent on feedback over
a very short timescale - behaviour is adjusted
according to response from environment
 Physical movement - movements made with

expectation of what environment will look like and

feel like
 Communication - human conversation is tightly

controlled by (largely non-verbal) exchange about

 what is understood
 emotional import of message
 whose turn to talk it is
Open world problem solving
Your clothesline has broken….

 Think of an action, do it if rational

(your estimate of benefit exceeds your estimate of
cost)
 No sense of a problem space
 no awareness of range of available actions
 no awareness of possible states you could be in
Becoming an Expert
Unfamiliar Task (Novice Behaviour):
 We reason about what to do using declarative

knowledge and general problem solving methods,

working backwards from objectives (intelligent goal
directed problem solving)

Very Familiar Task (Expert Behaviour):

 We just do it (recognise situation and take correct

actions in correct sequence, without planning)

Highly Practised Skill (Automatic Behaviour):

 No reflective thinking, perhaps little conscious

awareness
Novices versus Experts
Novices
 Knowledge mostly declarative - facts and

assertions
 Problem solving by conscious reasoning
 backward chaining from goals to actions
 heavy use of memory
 slow
 Problems and examples understood and classified

according to obvious ‘surface’ features

 Sparse and superficial connections between

elements of knowledge
 Learning aided by using concrete examples
Novices versus Experts
Experts
 Much task-specific procedural problem solving

knowledge
 Routine problem solving by applying learned

procedures
 forward chaining from situations to actions
 fast
 Problems and examples understood and classified

according to appropriate abstractions

 Rich and sophisticated connections between

elements of knowledge - elaborate schemata and

mental models
 Better at learning from abstract descriptions
Conscious and unconscious
processing
What do you think about when…
 Assembling a model kit? Repairing the plumbing?
 Riding a bicycle? Driving a car?
 Playing a musical instrument? Typing?
 Walking?

Can you do two things at once ?

YES, provided neither of them involve reasoning or
reflection
 The better an action is learnt, the less attention

(memory activation, active control of working

memory) its conditions need to trigger it
Dual Process Theory
 Two decision-making processes
 Sometimes compete and conflict

 System One
 Rapid holistic judgement

 System Two
 Slow, effortful step-by-step reasoning
 Can intervene in response from System One and

modify or reject it

[Daniel Kahneman, Thinking, Fast and Slow, 2011]

Dual Process Theory in practice
Dual Process Theory:
A classic trap question
 A racket and a ball cost $1.10 in total. The racket
costs $1 more than the ball. How much does the
ball cost?

 Five cents
 You can puzzle out the right answer, if you try
 Easy to get wrong, if you don’t realise you need to
be careful
Dual Process Theory:
Using System Two
 Which is better? Depends what for...

 System Two (slow effortful reasoning) needed when

 There is an answer than needs to be puzzled out
 Especially when obvious surface features of

problem are misleading

 Use of System Two

 Correlates with intelligence, but
 Needs recognition that care is needed
 Needs willingness to put effort in
Dual Process Theory:
Using System One
 Which is better? Depends what for...

 System One (rapid holistic judgement) needed

when
 Instant action is needed
 Best to judge by weight of evidence in a

situation too complicated to figure out

completely
 Answer depends on subtle perceptual regularity

you’ve learned but not understood

Implications for Design
 Intelligent people can be trapped into stupid
actions or reasoning mistakes...

 System One used when System Two is needed

 Superficial resemblance to different situation or

problem that is
 Simpler
 More frequent
 Suggested by recent experiences
Implications for Design
 DON’T put users in situations where System One
response is misleading
 Consistency!
 Pay attention to frequencies of tasks and input

conditions
 Show differences and not-most-frequent

situations clearly

 Phishing….
Reading
 Chapter 4 of Rogers, Sharp, Preece, Interaction
Design, 6th ed. (Otherwise the chapter on cognition
in earlier editions.)
 CTEC 3906
Interaction Design
Appendix:
Phishing
Phishing
 User receives an urgent email with a link
 The link leads them to a spoofed website
 The user is asked for sensitive personal information

 The problem has received a lot of attention

 Major scare factor for average user
System One and System Two
1. Rapid holistic judgement
2. Slow, effortful thinking
 Both have uses!
 System 2 thinking needed when (a) surface

appearance might be deceptive (b) you can

reason to a conclusion
 System 2 thinking needs recognition that it’s

needed
Phishing with
System One and System Two
 Phishing depends on
 Users’ perceptual reaction to phishing bait
 System One says it’s trustworthy
 Aim is to create perceptual impression of

trustworthiness
 Either

System Two isn’t actively engaged

 Or

System Two is fooled – it’s a good fake

 Spotting phishing needs (successful) System 2

thinking
Why does phishing work?
1. Lack of knowledge
 Users do not understand how computer systems

work
 Users do not understand security
 Users do not understand security indicators
 For example:
 ✓What does SSL do?
 ✓What's browser "chrome?"
Why does phishing work?
2. Visual Deception
 Visually deceptive text (homographs) from
 î vs. i vs. l Dhamija,
 http://www.bankofthevvest/ Tygar &
 images masking text Hearst
(2006)
 images mimicking windows
 pop-up windows
 deceptive look and feel
Why does phishing work?
3. Bounded Attention
 Too focused on task to pick up signs of

phishing…
 Lack of attention to security indicators
 Lack of attention to the absence of security

indicators
Phishing “solution”:
Teaching people what to look
for
 Too impersonal
 Spelling mistakes
 Substandard graphic design
 BUT bad guys can copy real thing accurately

 URLs shown by browser

 Don’t match URLs shown on page
 Have spelling mistakes
 Variants on name of real thing
 BUT what about dyslexics?
Inadequate mental models of
phishing attacks
 Many users do not understand man-in-the-middle
or keyboard sniffer attacks:
 Websites that have their data are assumed

legitimate
 Websites that "work" and don't have errors are

assumed legitimate
Security toolbars and warnings
 Don’t reliably prevent phishing attacks
 Experimental evidence that…
 Users will rationalize and ignore warnings
 Users can’t distinguish between errors and

attacks
 Are DMU’s out-of-date security certifications
evidence of something phishy going on? (I really
hope not…)
SSL Warning
Inventive rationalizations (from
Wu, Miller & Garfinkel, 2006)
12 subjects (60%) used rationalizations to justify the
indicators of the attacks that they experienced. Nine
subjects explained away odd URLs with comments
like:
 www.ssl-yahoo.com is a subdirectory of Yahoo!, like
mail.yahoo.com.
 sign.travelocity.com.zaga-zaga.us must be an
outsourcing site for travelocity.com.
 Sometimes the company [Target] has to register a
different name [www.mytargets.com] from its brand.
 What if target.com has already been taken by another
company?
 Sometimes I go to a website and the site directs me to
another address which is different from the one that I
have typed.
Rationalizations come from
mental models
 What’s happening when users talk themselves into
dangerous practices?
1. Motivation
 Fear of risks
 Cost-benefit analysis

2. Understanding
 Awareness of risks
 Wrong/dangerous mental model of situation
Phishing solution:
Better spam filters
Phishing solution:
Authenticate the email sender
Phishing solution:
Website authentication
Phishing solution:
Website authentication
Phishing solution:
Website blacklists
Making phishing solutions work
 Technological solutions to security problems
needed
BUT
 Depend on users’ mental models of situation

SO
 Design challenge is to
 Keep situation simple enough to see and

understand
 Enable users to develop correct/useful mental

model of situation, to make wise judgements of

what to do