Chapter 2

How We See
 Different Interpretation
• Different interpretation of what is seen.
– Vision is about one-tenth physical and nine-tenths mental.
– Sensory input received by the eyes is transformed by the
brain into meaningful images.
– The interpretation depends on preconditioning, intelligence,
and the physical and emotional state of the viewer.
• Even from the same vantage point, twelve artists will create
twelve different images because of their different experiences,
attitudes, interests and eyesight.
– Each of us potentially sees something differently, and then has a
different interpretation of what is seen.
• We see selectively.
– Since, we cannot focus our attention on everything around
us, we select certain things to look at.
• Deciding what to see is determined by knowledge, belief, and
want.
– The way we think about visual phenomena is affected by our
knowledge which is the basis of belief and attitude.
 Selective Attention
• Selective Attention
– The process of focusing on a particular object
in the environment.
• Attention as a limited resource: Selective attention
allows people to tune out unimportant details and
focus on what matters.
– Key variables that affect selective attention include how
interested we are in the stimulus, and how many distracters
we experience.
• cf.) Inattentional Blindness
– When focusing hard on one thing, people fail to notice
unexpected things entering their visual field.
 Selective Exposure
• Selective Exposure
– The motivated selection of messages matching one’s beliefs.
• The phenomenon whereby people choose to focus on information
in their environment that is congruent with, and confirms their
current attitudes in order to avoid or reduce cognitive dissonance.
– Selective exposure theory is often used in media and communication
research.
» “Audiences were not passive targets of political and commercial
propaganda from mass media, but that mass media reinforce
previously held convictions." (Joseph T. Klapper, The Effects Of Mass
Communication, 1960)
• Selective exposure comprises three sub-processes:
1. Selective exposure - People avoid communication that is opposite to
their existing attitude.
2. Selective perception - When people are confronted with unsympathetic
material, either they do not perceive it, or they make it fit for their
existing opinion.
3. Selective retention - People simply forget attitude-incongruent
information.
• Selective exposure is also known as Confirmation Bias.
– Confirmation bias: To search for, interpret, favor, and recall information in
a way that confirms or strengthens one's prior personal beliefs or values.
» Avoid cognitive dissonance, and pursue cognitive equilibrium
through “biased search for information”, “biased interpretation”, and
“biased memory”.
Making sense of the visual world
• People make sense of visual phenomena
in a number of ways.
1. Resemblance (as in photographs)
2. Cause and effect or logic (as in smoke
implying fire)
3. Convention (as in objects that have
symbolic value)
4. Signification (as in a smile signifying
pleasure)
The drawing looks like a
person, and so we can say
that drawings (as well as
photograph, paintings,
sculptures and the like) are
communicate by resemblance.
We know from our
experiences with fire that
“where there is smoke, there
is fire.” Thus, we have good
reason to believe that the
smoke is caused by fire and
the building is on fire.
We have learned that the
cross is associated with
Christianity. There is no way
for a person to “naturally”
know the meaning of a
cross; there is no logical
connection. The connection
is historical, not logical.
There is no natural
connection between a
word and the object it
stands for. Thus, the word
“tree” and the object it
stands for are not
logically related. The
relationship between a
word and the object it
stands for is arbitrary or
conventional.
 Semiosis
• Semiosis (Greek “sēmeiô”: meaning “to mark”) is
a process that interprets signs as referring to
their objects.
– Semiosis is any form of activity, conduct, or process
that involves signs, including the production of
meaning.
• According to Charles Sanders Peirce, semiosis (or
"action of a sign") is an irreducibly triadic
process, comprising a relation between (1) a sign,
(2) its object, and (3) its actual or potential
interpretant (interprétant. the effect of a sign on
someone who reads or comprehends it).
• Pierce particularly focused on the way that the
interpretant is produced.
 Semiology vs. Semiotics
• Semiology
– The term for the study of signs merely within
the linguistic/textual context. Semiology is
Saussurean (Ferdinand de Saussure) tradition,
and is a subset of semiotics. A linguistic
approach.
• Semiotics
– The general term for the study of signs
associated with broad areas such as linguistics,
anthropology, film theory, and even philosophy.
Therefore, signs could be words, sounds,
images, behavioral gestures, etc. Semiotics is
Peircian (Charles Sanders Peirce) tradition. A
social approach.
 What Are Signs?
• A sign, from the semiotic perspective, is
anything that stands for something else.
– A great deal of communication is done not
directly, but rather indirectly by using various
signs.
• For example signs such as wearing a trench coat and a
slouch hat , carrying a small revolver with a silencer,
and driving a fast sports car suggest a secret agent.
– A sign has a referent.
• A referent is a person or thing to which a sign refers.
– In the sentence Mary saw me, the referent of the word Mary
is the particular person called Mary who is being spoken of,
while the referent of the word me is the person uttering the
sentence.
 Signifier and Signified
• A sign (signe) is the combination of signifier
(signifiant) and signified (signifié).
– Signifier is the form of a sign.
• It is defined as a sound or object that calls to mind a
concept or signified.
• The form might be a sound, a word, a photograph, a facial
expression, a painting, etc.
– Signified is the concept or object that is
represented by the signifier.
• While the concept might not have physical reality, the
object might be an actual thing that is depicted.
• The relationship that exists between the signifier
and signified is arbitrary or conventional.
– The relationship is not natural but must be learned.
 Three Kinds of Signs: Icons,
Indexes, and Symbols
• Peirce identified three kinds of signs: iconic, indexical,
and symbolic.
• The difference is in how the meaning happens to be
attached to (or associated with) the pattern.
 Icons
• Originally, an icon (from Greek eikōn, "image") is a religious work of art.
– The most common subjects include Christ, Mary, saints and/or angels.
• An Icon has a physical resemblance to the signified.
– Icons bear a physical resemblance to what’s being represented.
– With icons there’s a real connection between the signifier and the
signified.
– A photograph is a good example as it certainly resembles whatever it
depicts.
– Icons are easy to interpret like the signs in airports.
• Regardless of the language, icons should be able to understand.
• An icon is based on the conceived similarity or analogy between the
form of a sign and its meaning, as opposed to arbitrariness.
• Four Types of Iconic Representation
– Resemblance icons are direct likenesses of the objects they represent.
(camera icon for a smartphone camera app)
– Exemplar icons depict a common example of the class of objects they
represent. (trowel or rake to represent gardening)
– Symbolic icons convey a concept at a higher level of abstraction than the
object depicted. (cracked wineglass to indicate something is fragile)
– Arbitrary icons have no relationship to an object or concept and their
association must be learned. (computer on/off power button)
 Indexes
• Index (from Latin indico, “to point”) is a sign pointing to (or
indexing) some object.
• An index shows evidence of what’s being represented.
• An indexical sign is logically connected to what it
represents.
– We have to learn about this connection, and do so from
everyday life.
• An index describes the logical connection between signifier
and signified.
– Examples:
• Smoke is an index of fire.
• Dark clouds are an index of rain.
• A footprint is an index of a foot.
• In each case the presence of the former implies the latter exists.
• While an index is a sign that shows the evidence of the
concept or object being represented, an index doesn’t
resemble the object or concept being represented.
 Symbols
• Symbol (from the Greek symbolon, an amalgam of syn-
"together" + bole "to throw") is something used for or regarded
as representing something else.
• Symbolic signs are purely artificial.
– The signifier is linked to the signified only by an arbitrary
human-imposed convention.
– There is no logical connection between the meaning and the
symbol itself.
– It is something we have to learn.
• Symbols are at the opposite end from icons.
– The letters of an alphabet are a good example of symbols.
– The shape of each letter and the sound it represents bear no
physical connection to each other.
• An icon or index can become a symbol over time through
repetition.
– Example:
• The floppy disk is still used to represent saving a digital file, even
though no one uses floppy disks anymore (plenty of people have
probably never even seen one).
 Icons, Indexes, and Symbols
• The boundaries between icon, index, and
symbol are often vague.
– An image can be an icon, index, or a symbol.
• For example, the picture (icon) of a rose can be a
sign of summer (index) and can mean the War of
the Roses (symbol).
– The meaning may shift from one to another
as the communication act progresses.
 Icons, Indexes, and Symbols
• The relation among icons, indexes, and
symbols.

Icon Index Symbol

Signified by Resemblance Causal connection Convention

Examples Photograph Smoke/fire Cross or flag

Process Can recognize Can figure out Must learn

 Signs can mean different things.
“Semiotics is concerned with everything that can be taken as a sign.
A sign is everything that can be taken as significantly substituting for
something else. This something else does not necessarily have to
exist, or to actually be somewhere at the moment in which a sign
stands for it. Thus, semiotics is in principle the discipline studying
everything which can be used in order to lie. If something cannot be
used to tell a lie, conversely it cannot be used to tell the truth; it
cannot in fact be used ‘to tell’ at all.” Umberto Eco (1976), A Theory
of Semiotics.
• Signs always have a double valence.
– According to Eco, they can be used either to tell
the truth or to lie.
• The absence of a sign, when some kind of a sign
is expected, is also a kind of sign.
– This is known as “aggressive passivity.”
– In certain situations, doing nothing or not
responding can be seen as a form of aggressive
behavior.
 Codes
• Since the relationship between signs and meaning is arbitrary,
people have to find ways of making sense of signs.
– People make sense of signs via codes.
• Code is a device to carry information in a verbal and/or
nonverbal form.
• Two processes of coding
– Encoding is the process to convert information from a source
into symbols for communication or storage. Sender-Encoder.
– Decoding is the reverse process to convert code symbols back
into a form that the recipient understands. Receiver-Decoder.
– The Encoding/decoding model of communication was proposed
by cultural theorist Stuart Hall ('Encoding and Decoding in the
Television Discourse.‘ 1973)
• Offered a theoretical approach of how media messages are produced,
disseminated, and interpreted.
• Claimed that TV and other media audiences are presented with
messages that are decoded, or interpreted in different ways depending
on an individual's cultural background, economic standing, and
personal experiences.
• Asserted that audience members can play an active role in decoding
messages.
 Codes
• Codes can be looked at as ways of making
sense of signs.
– Codes are systems of conventions that we are
taught or pick up from our culture.
• Convention: A set of agreed, or generally accepted
standards, norms, or criteria in a society.
– What we know as culture in anthropological
terms can be seen as a collection of codes.
• Codes are created and systematized.
– Like the driving codes, the codes are
collections of rules that tell us what to do
when we see certain signs.
 Decoding Differently
• A code is a set of rules about how people should
behave or about how something must be done.
• Culture is a collection of codes, and decoding is
different from culture to culture.
– When one moves to new culture, culture shock happens.
– Culture shock is the result of finding ourselves in a society
where the codes are different from what is used to.
• Culture shock is the feeling of disorientation experienced by
someone who is suddenly subjected to an unfamiliar culture,
way of life, or set of attitudes.
• Even in the same culture, people often misinterpret
signs.
– Because of the differences in education, region, class, and
so on, people often interpret signs in widely varying ways.
– Aberrant decoding is a problem for people who try to
convey something to people, but find those people
interpreting it in unanticipated ways.
 Aberrant Decoding
• Communication requires that the messages must be encoded
into a set of signs by the sender, then these signs must be
transmitted and decoded by the receiver to understand the
contained messages.
– In order for the communication to succeed, the code system
must be shared by both the sender and the receiver.
• Aberrant decoding (or aberrant reading)
– A concept (proposed by Umberto Eco in his article published in
1965) used in fields such as communication and media studies,
semiotics, and journalism about how messages can be
interpreted differently from what was intended by the sender.
• The right interpretation is called the preferred decoding or preferred
reading.
– According to Eco, aberrant decodings were rare in pre-industrial
societies, when most communication occurred between people
who shared the same culture.
• However in contemporary media, aberrant decodings have become the
norm.
• Eco speculated that because of the aberrant decoding, the power of
media over individuals might be much less influential than is thought.
 Eco and Hall
• Umberto Eco (January 1932 – February 2016)
• Italian novelist, literary critic, philosopher and
semiotician.
• Best known for his 1980 historical mystery novel Il
nome della rosa (The Name of the Rose), an
intellectual mystery combining semiotics in fiction,
biblical analysis, medieval studies and literary
theory.

• Stuart Hall (February 1932 – February 2014)

• Jamaican-born cultural theorist, political activist
and sociologist
• Lived and worked in the United Kingdom from
1951.
• Was one of the founding figures of the school of
thought that is now known as British Cultural
Studies or The Birmingham School of Cultural
Studies.
 Culture Code
• Culture code defines a particular set of
stereotypes in our minds.
– Culture code is a sort of cultural unconscious.
• Clotaire Rapaille contended that every country
has its own distinctive behavioral codes and that
these codes are imprinted on children by the
age of seven. (The Culture Code: An Ingenious
Way to Understand Why People Around the
World Live and Buy as They Do, 2007)
– Every imprint influences us on an unconscious level.
 Condensation and Displacement
• The mind processes signs and symbols and
other visual phenomena through
condensation and displacement.
– Condensation: The process by which we combine
elements of various signs together to form a new
composite sign or symbol. (One dream object
stands for several associations and ideas.)
Surrealistic styles are examples of condensation.
– Displacement: Transferring meaning from one sign
or symbol to another. Instead of directing the
emotion or desire toward the intended person or
object it is transferred onto an unrelated object.
The monument for George Washington maybe
interpreted as the symbol of phallic.
 The Image
• Image: A collection of signs and symbols
(including word).
• Images generally are visual, often mediated
(by media), and connected to information,
values, beliefs, attitudes, and ideas people
have.
– Since an image is a collection of signs, and
each of signs has meaning, we have to learn to
interpret signs and symbols.
• This is because there are many different levels of
meaning and interactions between meanings.
– All the components in an image are signs, and
they are designed to convey information and
generate certain attitudes in the minds of
viewers.
In the images
on the left, the
bubbles in the
beer, the frost
on the stein,
the foam, and
the smile are
signs meant to
generate certain
attitudes in the
minds of
viewers.
 Why is it hard to understand images?
• Images are intentionally designed to be seen and read, and
they have a specific function and impact.
• The complex interaction of the five factors (artist, audience, art
work, society, and medium) makes it difficult to understand
images.
– Artists have their own intentions.
– Viewers have different backgrounds.
– Art work might comprise a number of signs.
– Society (or culture) changes.
– Medium affects the images.
• “Because of all these complications, communicating anything
clearly and unambiguously is difficult. These factors also make
our communications powerful and even put them beyond the
control of those who create signs and symbols. It might be
suggested that communication often takes place between the
unconsciousness of the creators – the artists – and the
unconscious of the receivers – the audience – so that the
situation becomes even more mixed up. Nobody in such a
situation completely understands or fully appreciates what is
being communicated and what impact it is having.” (pp. 68-69)
The Arnolfini Portrait (or The
Arnolfini Wedding, The
Arnolfini Marriage, Portrait of
Giovanni Arnolfini and his Wife,
The Betrothal of the Arnolfini,
or other titles)
Jan van Eyck, 1434

Jan van Eyck (1390~1441) was

a Flemish/Netherlandish
painter active in Bruges.
One of the greatest artists of
the Northern Renaissance, van
Eyck was an early master of oil
painting. He was famed for his
ability to produce detailed
paintings.
 Symbols are connotative.
• According to Jung, a symbol:
– Possesses specific connotations in addition to all its
conventional and obvious meaning.
– Implies something vague, unknown, or hidden from us.
– Implies something more than its obvious and immediate
meaning.
– Has a wider or “unconscious” aspect that is never precisely
defined or fully explained.
• For Jung, symbols are attached to unconscious
elements in our psyches and are often subliminally
experienced.
– Subliminal messages are considered to activate specific
regions of the brain despite participants being unaware.
• A subliminal message is a signal or message designed to pass
below the normal limits of the human mind's perception.
• These messages are unrecognizable by the conscious mind, but in
certain situations can affect the subconscious mind and
importantly, the unconscious mind can negatively or positively
influence subsequent later thoughts, behaviors, actions, attitudes,
belief systems and value systems.
 Decoding Visual Images
Concept Example Method

Icon Photograph Resemblance

Index Smoke from window Cause and effect

Symbol Crucifix Convention

Signifier and signified Bowler hat = Englishness Convention

Condensation Face/automobile Unification

Displacement Rifle = Phallus Substitution

Metaphor Beautiful woman = Rose Analogy

Metonymy Huge mansion = Wealth Association

 Conclusion: Seeing is a mental process (function).
• Mental processes (or mental functions) mean such
functions or processes as perception, introspection,
memory, creativity, imagination, idea, belief, reasoning,
volition, and emotion – in other words, all the things
that we can do with our mind.
• “Seeing is not automatic, and we don’t see without
doing some thinking to make sense of what we see. We
see selectively, focusing our attention on sights that
interest us and paying little attention to ones that don’t.
This is necessary, because if we paid the same amount
of attention to every visual stimulus, we’d never get very
much done. One thing artist do for us is to show that
there are a number of different ways to perceive reality,
and for painters, to paint an image that interests them.
They use their creativity to show these different “takes”
or versions on an image, as the paintings of Jason
Berger demonstrate. Berger painted twenty different
versions of a friend’s sunroom, using a variety of styles.”