Week 2 E.D. Multimodality & Verbal Communication
Week 2 E.D. Multimodality & Verbal Communication
Your professional success depends on having advanced people (a.k.a. “soft”) skills
because most jobs require you to talk to people. Key among these is skill in speaking to
and conversing with others in person. Retail sales, for instance, requires the ability to
listen carefully to what a customer says they want and “read” their nonverbals to
determine what exactly to say and how to say it in order to close the deal with a purchase.
Aside from a handful of jobs with minimal human interaction (Gillett, 2016), the vast
majority require advanced soft skills to deal effectively with customers or clients,
coworkers, managers, and other stakeholders. Though we’re not born with them,
everyone has the capacity to learn, develop, practise, and apply verbal and nonverbal
skills to benefit those audiences, their company as a whole, and themselves.
You’ve certainly participated in countless conversations throughout your life, and the
process of how to conduct a conversation may seem so obvious that it needs no
explanation. Still, you can tell that some are better than others at conversation and some
argue that technology is preventing many from developing these skills, so it’s worth
breaking down how an effective communicator approaches the art of conversation. A
skilled professional knows when to speak, when to go silent and listen, as well as when
stop speaking before the audience stops listening.
Before diving into these topics, however, let’s review what face-to-face meetings are all
about according to the Table 2.3 channel selection guide above.
How your voice quality, volume, and pitch affect your listener’s understanding of the
message’s the content is instrumental especially for persuasive messages. Delivering a
message with a happy and enthusiastic tone will have a much different impact than
serious or sad tones. In most business situations, it is appropriate to speak with some
level of formality, yet avoid sounding stilted or arrogant. Your voice volume should be
normal but ensure your listeners can hear you. If your audience includes English learners,
speaking louder and shouting don’t help them understand you any better compared with
accessible word choices delivered in a normal tone.
Use simple words and short, active-voice sentences of 10-to-20 words, as well as avoid
idioms (figures of speech) that don’t translate literally. Pitch refers to the frequency of your
voice, which you can raise or lower for effect. A pleasant, natural voice will have some
variation in pitch—raised for lighthearted quips and lower for serious statements—to
communicate nuances of meaning and keep the listener engaged. A speaker with the flat
pitch of a robotic-sounding monotone voice tends to bore their listeners because they
sound bored themselves.
Initiation
As the first stage of conversation, initiation requires you to be open to interact and
perhaps use small talk to prime yourselves for the intended topic. You may communicate
openness with nonverbal signals such as approaching someone, stopping four feet away,
facing them, making eye contact, and smiling. When a degree of unfamiliarity comes
between the two speakers, small talk helps “break the ice” to clear a path toward the topic
at hand. Asking how they’re doing, a casual reference to the weather (“Nice day, eh?”) or
a brief back-and-forth about the weekend requires someone to begin the exchange. For
the very shy, this may trigger some anxiety whereas extraverts delight in this stage, and
“ambiverts” (e.g., natural introverts who have learned to play the extravert game) do it in
recognition of its necessity (Bradberry, 2016). If status and hierarchical relationships are a
factor, cultural norms may determine who speaks when. Usually, however, initiation just
requires a willingness to engage in conversation and a purpose—something to talk about
after clearing the small-talk hurdle.
A general reference to a topic may also approach a topic indirectly, allowing the recipient
to either pick up on the topic and to engage in the discussion or to redirect the
conversation away from a topic they aren’t ready to talk about. For instance, a manager
needing to talk to an employee about being late for work too often might start off by
saying, “That was some nightmare traffic on the highway this morning, eh?” Depending
on the employee’s response, the manager could then say what they do to get to work on
time, suggesting that the employee should do the same. To reinforce the point, the
manager could finally explain that continuing to arrive late will affect the employee’s job
security. The savvy communicator would be able to infer from the initial question about
traffic where the manager’s going with this line of questioning and may even begin to offer
up a convincing excuse for why they’re late and suggest a compromise such as a plan to
stay later or make up the lost hours by working from home on the weekend.
Business
You get down to business when you reinforce the goal orientation of the conversation. In
workplace communication, we often have a specific goal or series of points to address,
but we can’t lose sight of the relationship messages within the discussion of content. You
may signal to your conversation partner that you have three points you need to cover,
much like outlining an agenda at a meeting. This may sound formal at first, but in listening
to casual conversations, you’ll often find a natural but unacknowledged list of subtopics
leading to a central point where the conversational partners arrive. By clearly articulating
the main points, however, you outline the conversation’s parameters to keep it efficiently
on track rather than prone to digress from the main point.
Feedback
Similar to the preview stage, this feedback allows speakers to clarify, restate, or discuss
the talking points to arrive at mutual understanding. In some cultures the points and their
feedback may recycle several times, which may sound repetitious in the West where
speakers typically prefer to get to the point and move quickly to the conclusion once
they’ve achieved understanding. If so, then a simple “Are we good?” might be all that’s
necessary at the feedback stage. Communication across cultures, on the other hand,
may require additional cycles of statement and restatement to ensure understanding, as
well as reinforcement of the speakers’ relationship. Time may be money in some cultures,
but spending time is a sign of respect in cultures with rigid social hierarchies. The
feedback stage offers an opportunity to make sure the information exchange was
successful the first time. Failure to attend to this stage can lead to the need for additional
interactions, reducing efficiency over time.
Closing
Accepting feedback on both sides of the conversation often signals the transition to the
conversation’s conclusion. Closings mirror the initiation stage in that they can be signalled
verbally (e.g., “Okay, thanks! Bye”) or nonverbally, such as stepping back and turning
your feet and body in the direction of where you’re about to go next in preparation to
disengage while still facing and speaking with the other.
Sometimes a speaker introduces new information in the conclusion, which can start the
process all over again. If words like “in conclusion” or “okay, one last thing” are used, a
set of expectations is now at play and the listener expects a conclusion in the very near
future. If the speaker continues to recycle at this point, the listener’s patience will be
stretched and frustration sets in.
If anything in the above paragraphs sounds true to life, the onus rests largely on you to
improve your conversation skills with all the advice that is available (ironically) on the very
devices in question. For instance, we can draw on a very accessible TEDtalk by Celeste
Headlee, a talk-radio host and author of We Need to Talk: How to Have Conversations
That Matter (2017). We’ll adapt her well-viewed speech 10 Ways to Have a Better
Conversation (2016) for our own purposes below and build on them with a few points of
our own.
Be Present: Devote your undivided attention to the person you’re speaking with and don’t
multitask. You won’t have to pretend to pay attention by nodding and making eye contact
if you’re doing that anyway by actually paying attention. The worst offenders are those to
whip out their phone and engage with it rather than the people around them, called
“phubbing” (for “phone snubbing”) (Ducharme, 2018). Though you may feel that you can
get away with phubbing in your college classes by discreetly hiding your cell phone under
your desk, your instructor knows exactly what you’re doing when all your attention is on
your lap. The rudeness of it will likely get you slightly blacklisted—or fully blacklisted if you
do it too much.
Be prepared to learn: A conversation is a dialogue, not a monologue where you simply
unload your opinion on someone and receive nothing in return except for the satisfaction
of dominating them with it. In certain situations, such as a TEDtalk itself, you give up your
right to speak because of the faith that you’ll learn much more by listening to a wise
speaker who needs time to get their points across.
Ask open-ended questions: The more vague your questions are (starting with the 5 Ws
+ H), the more freedom you give your conversation partner to answer on their own terms,
whereas very specific questions limit the possible answers. If you ask “How did that make
you feel?” for instance, you’ll get a more expressive answer than if you limited your
speaker to a yes or no answer with a question like “Did that make you happy?”
Go with the flow: Respond to your conversation partner’s main points rather than with
some digressive story you were reminded of by one of their minor points. When you
respond in that way, it reveals that you haven’t been listening past the part that inspired
the barely relevant thing you feel contributes to the conversation, though it really doesn’t
move the conversation along so much as derail it.
Admit to not knowing: Make your confession of ignorance an opportunity to learn rather
than claim to know something you don’t.
Honour the uniqueness of their experience: When the speaker relates something that
happened to them, resist the urge to make it about you by equating their experience with
yours. If they’re talking about grieving a death in the family, for instance, don’t dishonour
that information share by responding with how you felt when your dog died. It’s not the
same.
Cut yourself off before repeating yourself: If you have only one point to make, “hit it
and quit it” rather than spin your wheels saying the same thing over and over, even if you
change the words.
Stay out of the weeds: Rather than struggle to offer up all the details (the names,
places, dates, etc.) and digress on minutiae, focus on your main points.
Listen: A conversation is a dialogue, not a monologue, and therefore requires that you
actively pay attention to what the speaker says in order to understand it rather than to
merely reply to it (see §1.5 above for more on active listening).
Be brief: People are busy and have things to do, so if your conversation detains them for
longer than they have time for, you will stretch their patience. As Headlee says, “A good
conversation is like a miniskirt: short enough to retain interest, but long enough to cover
the subject.”
Our language system is primarily made up of symbols. A symbol is something that stands
in for or represents something else. Symbols can be communicated verbally (speaking
the word hello), in writing (putting the letters H-E-L-L-O together), or nonverbally (waving
your hand back and forth). In any case, the symbols we use stand in for something else,
like a physical object or an idea; they do not actually correspond to the thing being
referenced in any direct way. For example, there is nothing inherent about calling a cat a
cat. Rather, English speakers have agreed that these symbols (words), whose
components (letters) are used in a particular order each time, stand for both the actual
object, as well as our interpretation of that object. This idea is illustrated by C. K. Ogden
and I. A. Richard’s triangle of meaning. The word “cat” is not the actual cat. Nor does it
have any direct connection to an actual cat. Instead, it is a symbolic representation of our
idea of a cat, as indicated by the line going from the word “cat” to the speaker’s idea of
“cat” to the actual object.
The verbal symbols that we use to communicate have three distinct qualities: they are
arbitrary, ambiguous, and abstract. Notice that the picture of the cat on the left side of the
triangle more closely represents a real cat than the word “cat.” However, we do not use
pictures as language, or verbal communication. Instead, we use words to represent our
ideas. This example demonstrates our agreement that the word “cat” represents or
stands for a real cat AND our idea of a cat. The symbols we use are arbitrary and have no
direct relationship to the objects or ideas they represent. We generally consider
communication successful when we reach agreement on the meanings of the symbols
we use.
Not only are symbols arbitrary, they are ambiguous- that is, they have several possible
meanings. For example, when the word “cat” is uttered alone and without context, most
people might envision a small, house cat. However, “cat” has other possible meanings.
Lion? Nickname for Catherine? Cool person? Heavy construction machinery? Imagine
your friend tells you they have an apple on their desk. Are they referring to a piece of fruit
or their computer? If a friend says that a person they met is cool, do they mean that
person is cold or awesome? The meanings of symbols change over time due to changes
in social norms, values, and advances in technology. We are able to communicate
because there are a finite number of possible meanings for our symbols, a range of
meanings which the members of a given language system agree upon. Without an
agreed-upon system of symbols, we could share relatively little meaning with one
another.
Symbols are Arbitrary, Ambiguous, & Abstract
The verbal symbols we use are also abstract, meaning that, words are not material or
physical. A certain level of abstraction is inherent in the fact that symbols can only
represent objects and ideas. This abstraction allows us to use a phrase like “the public” in
a broad way to mean all the people in the United States rather than having to distinguish
among all the diverse groups that make up the U.S. population. Similarly, in J.K.
Rowling’s Harry Potter book series, wizards and witches call the non-magical population
on earth “muggles” rather than having to define all the separate cultures of muggles.
Abstraction is helpful when you want to communicate complex concepts in a simple way.
However, the more abstract the language, the greater potential there is for confusion.
Saying “cat” is not as abstract as saying “animal,” but also will not necessarily bring an
image of a much more specific white cat named “Calliope” to the mind of your listener, if
that is your intention.
As we just learned, the relationship between the symbols that make up our language and
their referents is arbitrary, which means they have no meaning until we assign it to them.
In order to effectively use a language system, we have to learn, over time, which symbols
go with which referents, since we can’t just tell by looking at the symbol. Like me, you
probably learned what the word apple meant by looking at the letters A-P-P-L-E and a
picture of an apple and having a teacher or caregiver help you sound out the letters until
you said the whole word. Over time, we associated that combination of letters with an
apple. Similarly, in a Spanish-speaking community, students assigned meaning to a
different symbol for this fruity referent. The symbol is M-A-N-Z-A-N-A. Same referent,
different symbol.
We attach meanings to words; meanings are not inherent in words themselves. As you’ve
been reading, words (symbols) are arbitrary and attain meaning only when people give
them meaning. While we can always look to a dictionary to find a standardized definition
of a word, or its denotative meaning, meanings do not always follow standard, agreed-
upon definitions when used in various contexts. For example, think of the word “sick”. The
denotative definition of the word is ill or unwell. However, connotative meanings, the
meanings we assign based on our experiences and beliefs, are quite varied. For
example, take the word ‘hippie’, the denotative meaning or dictionary definition is “a
usually young person who rejects the mores of established society (as by dressing
unconventionally or favoring communal living) and advocates a nonviolent ethic”
(Webster, 2019). However, what comes to mind when you think of the word hippie? Long
hair? Tye-dye shirts? Drugs? This is the connotative meaning, the things you associate
with that particular world. Connotative meanings can be positive, negative, and/or neutral
and what WE associate with word may change over time and vary based on individual
experiences. For example, a person who liked road trips as child may have a positive
association with that phrase, while a person who disliked them may have a negative
association.
Multimodal Communication
Have you ever wondered how to analyse communication in different ways to understand
its meaning? This is where the term multimodality comes in handy. As a key concept in
linguistics, multimodality is often considered when analysing discourse (written/spoken
text or images) as a way to understand not only the written meaning of discourse but also
other elements that contribute to the meaning.
Multimodality meaning
Multimodality refers to the use of more than one mode of communication in a text to create
meaning. Although communication has always been multimodal, multimodality in
discourse analysis is a relatively recent approach; linguists began to consider a multimodal
approach around the 1960s.
One person who took an interest in multimodality was Gunther Kress. Alongside
linguist Theo van Leeuwen, he contributed to the study of multimodality and was well
known for writing books on the topic that were instrumental to the understanding and
development of multimodal analysis. Because of this, his work set the foundation for the
study of multimodality and influenced the works of future linguists. Perhaps his work may
influence you too!
Modes
Mediums
'[...] a socially and culturally shaped resource for making meaning. Image, writing, layout,
speech, moving images are examples of different modes'.
People communicate in different ways, so it is important to be aware of the different
modes used in communication to fully understand the meanings that are being conveyed.
These can be broken down into five categories:
1. Linguistic
2. Visual
3. Aural
4. Gestural
5. Spatial
It is important to note that a text does not need to contain all of the above modes to be
considered multimodal; it can contain two or more.
Linguistic mode
The linguistic mode focuses on the meaning of written or spoken language in
communication. For example, this includes: choice of words, vocabulary, grammar,
structure etc.
Visual mode
The visual mode focuses on the meaning of what can be seen by a viewer. This includes:
images, symbols, videos, signs, etc. It also includes aspects of visual design, such as
colour, layout, font type and size, etc.
Aural mode
The aural mode focuses on the meaning of what can be heard by a listener. This includes:
sound effects, music, voice. This can be realised through tone, pitch, speed,
volume, rhythm etc.
Gestural mode
The gestural mode focuses on the meaning of communication through movement. This
includes: facial expressions, gestures, body language, interactions between people. These
are all examples of non-verbal communication, as meaning is conveyed without the use of
speech.
Spatial mode
The spatial mode focuses on the meaning of communication through physical layout. This
includes: position, spacing, the distance between elements in a text, proximity between
people/objects, etc.
Modes are influenced by the type of medium they are carried by. The medium of a text can
determine the purpose of a text and its target audience. Different types of mediums
include, but are not limited to:
Books
Newspapers
Radio
Television/Film
Billboards
Theatre
Websites
There are so many different mediums… can you think of any more?
It is also important to note that there can be more than one medium for a text. For
example, an online newspaper combines the form of a newspaper with the layout/features
of a website for easy, global access. Also, online newspapers have the advantage of being
updated with news in real-time, so news can be reported quickly.
You should then identify the different modes that make up the text and what each mode
communicates. The modes convey a message that can be interpreted by looking at each
mode individually and also together; as communication doesn’t only depend on a single
mode.
For example:
The medium of this text is a sign; particularly a road sign. This tells us that it is
aimed at drivers, and has the purpose of giving instruction to them.
The linguistic mode - the words ’slow down’ let us know of the action to be taken by
the driver. They deliver a clear message in an imperative manner (ordering the
driver to do something).
The visual mode - the font of the text is large, bold and evenly spaced out, making it
easy to read for passing drivers. This stands in contrast to the orange background
of the sign, which is bright and will grab the attention of the driver. Orange could
also be associated with a warning, to signify caution.
The spatial mode - in terms of where the sign is in relation to other objects, it is
situated right by the road, so will be visible to drivers and will be able to caution
them to avoid speeding or accidents on the road.
Visual
Auditory
Kinaesthetic
Reading/Writing
The visual way of learning refers to people who retain information through seeing things.
For example, they may prefer to use graphs, pictures, charts flashcards, etc. to be able to
visualise the information and remember it.
The auditory way of learning refers to people who retain information through hearing.
Those types of learners may prefer to participate in listening exercises in which they hear
information and repeat it.
The kinaesthetic way of learning is suitable for people who retain information through
physical activity. For example, they may prefer to show someone how to do something
instead of simply explaining it through writing.
The reading/writing way or learning refers to people who retain information through
reading texts and writing things down. Learners with this style of learning may prefer to
answer questions from a textbook or eBook.
There are many types of AAC, including aided and unaided systems. Unaided systems,
which are sometimes referred to as no tech AAC, include the use of facial expressions
and manual signs/gestures to communicate. Aided systems, which include low
tech, mid tech, and high tech AAC, include the use of communication boards and
speech-generating devices. (Augmentative and alternative communication (AAC).)
Waving
Shaking hand
Moving/kicking legs
Pen and paper: An individual can write or draw their messages. A portable
whiteboard or Boogie Board (LCD tablet) is another option.
Core words board: A few basic symbols an individual can point to in order to
communicate their message. Core words boards are typically laminated pieces of
paper.
Pictures/symbols: A few symbols (similar to those on a core words board) on
different cards or laminated squares an individual can hand to their
communication partner to communicate their message.
Single button: something like a Big Mack, which can speak a single, pre-
programmed message (e.g. "I want a drink")
Single overlay displays: something like a GoTalk, with a limited number of pre-
programmed messages.
High Tech (Aided) AAC Examples
A tablet with a software program for communication. This program may have
multiple pages and folders (e.g. pressing “School” brings you to a page with
school vocabulary, pressing “Food” brings you to a page with food options).
A computer that has software programs that allow you to type, control a
mouse, send emails, and browse the Internet.