Vietnam National University of HCMC

International University
School of Computer Science and Engineering

UI/UX Design & Evaluation

★ Human Perception★

Dr Vi Chi Thanh - [email protected]

https://vichithanh.github.io
 Course Overview

1. Introduction to HCI 6. Human abilities and

2. Needfinding theoretical models

3. Defining needs and tasks 7. Visual design and design

patterns
4. Prototyping
8. Heuristic evaluation
5. Design guidelines, principles,
and heuristics 9. Usability testing
10.Advanced interactions

THURSDAY 18 APRIL 2024 2

 COGNITIVE PSYCHOLOGY

• Perception (Input mechanism of Human)

• Sensory memory
• Sensory Processing (Visual perception, Auditory perception, Haptic
perception)
• Movement (Output mechanism of Human)
• Cognition (Processing mechanism of Human)
• Cognitive memory (Working memory, Long-term memory)
• Cognitive Processing (Decision making, Problem solving, Reasoning)

THURSDAY 18 APRIL 2024 3

 HUMAN INFORMATION PROCESSING MODEL
Encoding Central Processing Reaction

THURSDAY 18 APRIL 2024 Wickens' Model of Human Information Processing 4

 HUMAN INFORMATION PROCESSING MODEL
Encoding Central Processing Reaction

THURSDAY 18 APRIL 2024 Wickens' Model of Human Information Processing 5

 HOW DO WE PERCEIVE REALITY?

We understand the world through our

senses:
• Sight, Hearing, Touch, Taste, Smell
(and others..)
Two basic processes:
• Sensation – Gathering information
• Perception – Interpreting information

THURSDAY 18 APRIL 2024 6

THURSDAY 18 APRIL 2024 7
 RELATIVE IMPORTANCE OF EACH SENSE

• Percentage of neurons in brain

devoted to each sense
• Sight – 30%
• Touch – 8%
• Hearing – 2%
• Smell - < 1%

THURSDAY 18 APRIL 2024 8

 OTHER LESSOR KNOWN SENSES…

• Proprioception = sense of body position

• what is your body doing right now
• Equilibrium = balance Acceleration
• Nociception = sense of pain
• Temperature Thirst
• Micturition
• Amount of CO2 and Na in blood

THURSDAY 18 APRIL 2024 9

 SIGHT

THURSDAY 18 APRIL 2024 10

 VISION
• The source of more than 80% of information
• 75% of the human visual operations are related to Fovea (area of
highest resolution; about 2 degrees of arc)

THURSDAY 18 APRIL 2024 11

 THE HUMAN EYE

• Light passes through cornea and lens onto retina

• Photoreceptors in retina convert light into electrochemical signals

THURSDAY 18 APRIL 2024 12

 PHOTORECEPTORS – RODS AND CONES

Retina photoreceptors come in two types, Rods and Cones

• Rods – 125 million, periphery of retina, no color detection, night vision
• Cones – 4-6 million, center of retina, color vision, day vision

THURSDAY 18 APRIL 2024 13

HUMAN HORIZONTAL AND VERTICAL FOV

• Humans can see ~135 vertical (60 above, 75 below)

• See up to ~ 210 horizontal FOV, ~ 115 stereo overlap
• Color/stereo in center, Black & White/mono in periphery

THURSDAY 18 APRIL 2024 14

 COLORS HAVE DIFFERENT CULTURAL
ASSOCIATIONS
Hong Kong Chinese (N=784) Americans

Concept Color % Color %

Safe Green 62.2 Green 61.4
Cold White 71.5 Blue 96.1
Caution Yellow 44.1 Yellow 81.1
Go Green 44.7 Green 99.2
On Green 22.3 Red 50.4
Hot Red 31.1 Red 94.5
Danger Red 64.7 Red 89.8
Off Black 53.5 Blue 31.5
Stop Red 48.5 Red 100
Courtney (1986), Bergum and Bergum (1981) Choice of 8 different colors
THURSDAY 18 APRIL 2024 15
 GESTALT LAWS

THURSDAY 18 APRIL 2024 16

 VISUAL ILLUSIONS: MULLER-LYER ILLUSION

THURSDAY 18 APRIL 2024 17

 BRIGHTNESS PERCEPTION
brightness constancy

THURSDAY 18 APRIL 2024 18

 MOTION PERCEPTION

• The human visual system can process 10 to 12 separate images per second,
perceiving them individually.
• Because of the persistence of vision phenomenon and the phi phenomenon, a
sequence of fast changing images is perceived as a smooth motion
• Frame rate should be at least 24 f/s
• Phi phenomenon: the optical illusion of perceiving continuous motion between
separate objects viewed rapidly in succession
• Persistence of vision is a phenomenon of the eye by which an afterimage is
thought to persist for approximately one twenty-fifth of a second on the retina

THURSDAY 18 APRIL 2024 19

 HEARING

THURSDAY 18 APRIL 2024 20

 ANATOMY OF THE EAR

THURSDAY 18 APRIL 2024 21

 AUDITORY PROCESSING

• Perceptual characteristics of sound

• Pitch (20 Hz ~ 20,000 Hz)
• Loudness
• Timbre: depends primarily upon the spectrum of the stimulus, but it also depends upon the
waveform, the sound pressure, the frequency location of the spectrum, and the temporal
characteristics of the stimulus
• Duration
• Sequential processing
• Duration of sensory storage is relatively shorter than visual
• Cocktail party effect: very sensitive to the changes of familiar sound, e.g., one
can hear voice in a very noisy environment

THURSDAY 18 APRIL 2024 22

 AUDITORY THRESHOLDS

• Humans hear frequencies from 20 – 22,000 Hz

• Most everyday sounds from 80 – 90 dB

THURSDAY 18 APRIL 2024 23

 ABSOLUTE THRESHOLD OF HEARING

• The smallest sound that a average human can hear

THURSDAY 18 APRIL 2024 24

 SENSITIVITY OF HEARING

• http://hyperphysics.phy-astr.gsu.edu/hbase/Sound/earsens.html
• https://www.youtube.com/watch?v=YSYAC7sjNCY

THURSDAY 18 APRIL 2024 25

 SOUND LOCALIZATION

• Humans have two ears

• localize sound in space
• Sound can be localized using 3
coordinates
• Azimuth, elevation, distance

THURSDAY 18 APRIL 2024 26

 SOUND LOCALIZATION

https://www.youtube.com/watch?v=FIU1bNSlbxk
THURSDAY 18 APRIL 2024 27
 ACCURACY OF SOUND LOCALIZATION

• People can locate sound

• Most accurately in front of them
• 2-3° error in front of head
• Least accurately to sides and behind head
• Up to 20° error to side of head
• Largest errors occur above/below elevations and behind head

• Front/back confusion is an issue

• Up to 10% of sounds presented in the front are perceived coming from
behind and vice versa (more in headphones)

THURSDAY 18 APRIL 2024 28

 TOUCH

THURSDAY 18 APRIL 2024 29

 TOUCH

• Mechanical/Temp/Pain stimuli transduced into Action Potentials (AP)

• Transducing structures are specialized nerves:
• Mechanoreceptors: Detect pressure, vibrations & texture
• Thermoreceptors: Detect hot/cold
• Nocireceptors: Detect pain
• Proprioreceptors: Detect spatial awareness
• This triggers an AP which then travels to various locations in the brain
via the somatosensory nerves

THURSDAY 18 APRIL 2024 30

 SENSORY SYSTEMS
• Sensory receptor organs detect energy or substance
Type of Sensory System Modality Adequate stimuli
Mechanical Touch Contact with or deformation of body surface
Joint Position and Movement
Muscle Tension
Hearing Sound vibrations in air or water
Vestibular Head movement and orientation
Thermal Cold Decrement of skin temperature
Warmth Increment of skin temperature
Photic Seeing Visible radiant energy
Chemical Smell Odorous substances dissolved in air or water in the nasal cavity
Taste Substance in contact with the tongue or other taste receptor
Common Chemical Changes in CO2, pH, osmotic pressure(삼투압)
Vomeronasal Pheromones in air of water
Electrical Electroreception Differences in density of electrical current
THURSDAY 18 APRIL 2024 31
 Two Haptic Informations
Kinesthetic
Position
Orientation
Force

Tactile
Contact location
Pressure
Slip and shear
Vibration
THURSDAY 18 APRIL 2024
Temperature 32
 CUTANEOUS SYSTEM
• Skin – heaviest organ in the body
• Epidermis outer layer, dead skin
cells
• Dermis inner layer, with four
kinds of mechanoreceptors

THURSDAY 18 APRIL 2024 33

 MECHANORECEPTORS
• Cells that respond to pressure, stretching, and vibration
• Slow Acting (SA), Rapidly Acting (RA)
• Type I at surface – light discriminate touch
• Type II deep in dermis – heavy and continuous touch

34
PROPERTIES OF SKIN RECEPTORS RELATED TO
TOUCH
Receptors

Merkel’s Disks Ruffini’s Ending Meissner’s Corpuscle Pacinian Corpuscle

Function Edges, Indentation Skin stretch Velocity, Edges Vibration

Receptive Field
small, large, small, large,
sharp vague sharp vague
borders borders borders borders

Stimulus
Response

Slow Adaptation I Slow Adaptation II Rapid Adaptation I Rapid Adaptation II

THURSDAY 18 APRIL 2024 35

 Olfaction
Smell

THURSDAY 18 APRIL 2024 36

 HOW THE NOSE WORKS

• https://www.youtube.com/watch?v=zaHR2MAxywg

THURSDAY 18 APRIL 2024 37

 OLFACTORY SYSTEM

Human olfactory system. 1: Olfactory bulb; 2: Mitral cells; 3: Bone

4: Nasal epithelium; 5: Glomerulus; 6: Olfactory receptor neurons
THURSDAY 18 APRIL 2024 38
 SMELL
Smells are sensed by olfactory sensory neurons in the olfactory
epithelium
• 10 cm2 with hundreds of different types of olfactory receptors
• Human’s can detect at least 10,000 different odors
• Some researchers say trillions of odors
Sense of smell closely related to taste
• Both use chemo-receptors
• Olfaction + taste contribute to flavour
The olfactory system is the only sense that bypasses the thalamus and
connects directly to the forebrain

THURSDAY 18 APRIL 2024 39

 Smells in interactive design

Scentee Machina
THURSDAY 18 APRIL 2024 41
 Smells in interactive design

oPhone DUO
THURSDAY 18 APRIL 2024 42
 Smells in interactive design

Aroma Shooter
THURSDAY 18 APRIL 2024 43
 OWidgets

THURSDAY 18 APRIL 2024 44

Gustation
Taste
 TASTE

• https://www.youtube.com/watch?v=FSHGucgnvLU

THURSDAY 18 APRIL 2024 46

 BASIC TASTES

• Salty
• Sweet
• Sour
• Bitter
• Umami
 Salty

• Sodium chloride (Na+ Cl-)

• Needed for electrolyte balance
• Na-channels
 Sweet - Sugar

• Needed for energy

• Many types of sugar:
• Glucose
• Fructose
• Galactose
• Sucrose (fructose + glucose)
• Maltose
• Lactose
• Receptors: T1R2 + T1R3
 Sour

• Acids (H+)
• E.g. Citric acid - palatability of oranges.
• Acid-sensing channel - the PKD2L1
channel, gated by pH (H+ ion
concentration)
 Bitter

• Indicate foods that

may be poisonous
• T2R receptors
activating the G-
protein
• Supertasters: has
heightened sense of
the taste of bitterness
 Umami

• Taste of essential amino

acids (e.g. glutamate)
needed for protein
synthesis.
• Meaning: savoury taste
• First identified by
Kikunae Ikeda in 1909.
• Receptors: T1R1 + T1R3
+ mGluR4
Basic tastes
 Other possible tastes

• Starch (recently confirmed)

• Calcium
• Tannin
• Metals
• Fat
Papillae and taste buds
A taste receptor cell
 Taste vs. flavour

• Flavour is the combination of taste, smell, and trigeminal sensations

(sensation in the face).
• “It tastes great” == “It has a great flavour”
 Tastes in interactive design

• Stimulate taste
Tastes in interactive design

Ranasinghe, N., Cheok, A., Nakatsu, R., & Do, E. Y. L. (2013, October). Simulating the
sensation of taste for immersive experiences. In Proceedings of the 2013 ACM international
workshop on Immersive media experiences (pp. 29-34). ACM.
Tastes in interactive design

Ranasinghe, N., Cheok, A., Nakatsu, R., & Do, E. Y. L. (2013, October). Simulating the
sensation of taste for immersive experiences. In Proceedings of the 2013 ACM international
workshop on Immersive media experiences (pp. 29-34). ACM.
Tastes in interactive design

Taste+: The Sensation of Taste in the

Future of Immersive Media
Tastes in interactive design

Taste+: The Sensation of Taste in the

Future of Immersive Media
Tastes in interactive design

Taste+: The Sensation of Taste in the

Future of Immersive Media
Tastes in interactive design

LOLLio: exploring taste as playful modality

Taste & Acoustic Levitation

• Hold and move an object in mid-air using sound

• A simple setup:
• Emitter of high frequency waves

• Reflector to reflex the waves back

• Create standing waves with nodes and antinodes

Xie et al. Appl. Phys. Lett. 89, 214102 World's First Levitating Food: Heinz Caprese Salad and Acoustic
(2006) The Levitron. https://goo.gl/ibNyhs Levitation. https://goo.gl/7AMCrC
 Preliminary findings
Recognition • Sweet (86.36%), Umami (70.41%), Bitter (45.56%)
rates
Intensity • Levitation > Pipette

Pleasantness • Levitated bitter is more pleasant

• Sweet is slightly satisfying

Satisfaction • Bitter and Umami are slightly unsatisfying
LeviSense

• Smell
• Taste
• Visual
• Touch
• Audio

THURSDAY 18 APRIL 2024 72

 LeviSense

THURSDAY 18 APRIL 2024 73

THURSDAY 18 APRIL 2024 74
THURSDAY 18 APRIL 2024 75
THURSDAY 18 APRIL 2024 76
 Research Question: using/ stimulate taste to
have an effect?
• Tastes can trigger user experiences:
• Temporal
• Affective
• Embodiment

Obrist et al. Temporal, affective, and embodied characteristics of taste

experiences: a framework for design. CHI '14. ACM, New York, NY, USA,
2853-2862.
 COGNITION

Encoding Central Processing Reaction

THURSDAY 18 APRIL 2024 78

 COGNITION

• Merely serves to connect inputs from the perceptual system to the

right outputs of the motor system
• A group of mental processes that includes attention, memory,
producing and understanding language, learning, reasoning,
problem solving, and decision making
• Usually refers to an information processing view of an individual's
psychological functions

THURSDAY 18 APRIL 2024 79

 HUMAN MEMORY SYSTEM

THURSDAY 18 APRIL 2024 80

 WORKING MEMORY

• To hold the intermediate products of thinking and the presentations

produced by the perceptual system
• rapid access ~ 70ms
• rapid decay ~ 200ms
• limited capacity - 7± 2 chunks

THURSDAY 18 APRIL 2024 81

 LONG-TERM MEMORY

• To store knowledge for the future

• A network of related chunks, accessed associatively from the
contents of the working memory
• slow access ~ 1/10 second
• slow decay, if any
• huge or unlimited capacity

THURSDAY 18 APRIL 2024 82

 TYPES OF MEMORY

THURSDAY 18 APRIL 2024 83

 LONG-TERM MEMORY (CONT.)

Semantic memory structure

• provides access to information
• represents relationships between bits of information supports
inference
Model: semantic network
• inheritance – child nodes inherit properties of parent nodes
relationships between bits of information explicit
• supports inference through inheritance

THURSDAY 18 APRIL 2024 84

 LTM - SEMANTIC NETWORK

THURSDAY 18 APRIL 2024 85

 ENCODE, STORE, AND RECALL

• Memory has the ability to encode, store and retrieve information

• Encode: conversion of stimuli into a construct that can be stored
within the brain (long-term memory)
• Store: the retention of information in the brain for a prolonged
period of time until it is accessed through recall
• Recall: the retrieval of events or information from the past

THURSDAY 18 APRIL 2024 86

 ENCODING AND STORING

Encoding in different memory types

• Sensory and working memory: raw signal
• Long-term memory: only selective info. from working memory goes to
here after encoding of the raw signal
Encoding types
• Visual encoding: visuo-spatial sketchpad
• Elaborative encoding: relating new information to knowledge that is
already in memory
• Semantic encoding: mapping to a meaning

THURSDAY 18 APRIL 2024 87

 ENCODING STRATEGIES

• Rote Rehearsal, e.g., verbal repetition of someone’s name

• Chunking, i.e., categorizing
• Mnemonic devices: linking new to already known, e.g., mental
picture, reword (verbal), method of loci (locational), acronym
• Self-referencing
• Spacing, e.g., five hours study in a row < one hour for five days

THURSDAY 18 APRIL 2024 88

 RETRIEVAL

• Recognition: association of an event or physical object with one

previously experienced or encountered
• Recall: remembering a fact, event or object that is not currently
physically present
• Free recall: without any hint, only contextual overlap
• Cued recall: recall with given hint, contextual + conceptual overlap
• Recognition, cued recall is easier than free recall. Thus, try to convert
free recall to recognition in interaction

THURSDAY 18 APRIL 2024 89

 License
■ These slides are distributed under a Creative Commons license “Attribution-NonCommercial-ShareAlike
4.0 International (CC BY-NC-SA4.0)”

■ Youare free to:

o Share — copy and redistribute the material in any medium or format
o Adapt — remix, transform, and build upon the material
o The licensor cannot revoke these freedoms as long as you follow the license terms.
■ Under the following terms:
o Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes
were made. You may do so in any reasonable manner, but not in any way that suggests the licensor
endorses you or your use.
o NonCommercial — You may not use the material for commercial purposes.
o ShareAlike — If you remix, transform, or build upon the material, you must distribute your
contributions under the same license as the original.
o No additional restrictions — You may not apply legal terms or technological measures that legally
restrict others from doing anything the license permits.
■ https://creativecommons.org/licenses/by-nc-sa/4.0/

THURSDAY 18 APRIL 2024 90

 Vietnam National University of HCMC
International University
School of Computer Science and Engineering

THANK YOU

Dr Vi Chi Thanh - [email protected]

https://vichithanh.github.io