FLE 337

(Aspects of Bilingualism and Multilingualism)

Session 2:
‘Speech Perception & Comprehension’

Bilal Kırkıcı
 Chapter 2: Aims

1. describing how bilinguals process “monolingual speech” - just one language

when there are no elements of the other language(s) in it
2. describing how bilinguals process “bilingual speech” - speech that contains
code-switches and/or borrowings (remember last session?)
3. overviewing the methodology used to study oral language perception and
comprehension in bilinguals

Question: If you want to test speech production, you study the speech
produced. What do you do if you want to study speech
perception/comprehension?

Answer: You run experiments to have an indirect look at the bilingual mind.
 The Processing of Spoken Language

Spoken language processing

1. is something we constantly do, all the time (even now, as I speak)

2. is highly complex
3. is very rapid
4. involves a number of mental components and processes (see next slide)
 The Processing of Spoken Language

context, world
knowledge, etc.
(“top-down info.”)

speech wave
(“bottom-up info.”)

Grosjean & Li, p. 30

Outcome: mental representation

(enriched meaning of the spoken input)
From perception to processing
 From perception to processing

“the class is over”

Semantic
Processing
Lexical Pragmatic
processing Processing

Phonetic
Processing Syntactic
Processing Morphological
Processing
 From perception to processing

“the class is over”

Phonetic Lexical Morphological Syntactic Semantic Pragmatic

Processing processing Processing Processing Processing Processing
 From perception to processing

“the class is over”

Phonetic Lexical Morphological Syntactic Semantic Pragmatic

Processing processing Processing Processing Processing Processing

As soon as speech wave is heard -> identification of phonetic and prosodic

units
• perception of phonemes (ðə klæs ɪz ˈoʊvər or ðə klɑːs ɪz ˈəʊvə)
• perception of intonation (falling intonation)
• perception of speech rate (normal rate)
• perception of emphasis (stress on over)
 From perception to processing

“the class is over”

Phonetic Lexical Morphological Syntactic Semantic Pragmatic

Processing processing Processing Processing Processing Processing

Together with perception of first sounds -> lexical processing

• recognition of the words “the”, “class”, “is” and “over”
• activation of words in mental lexicon, together with other words that
have similar onsets (e.g., closet, clothes)
• access to information in lexicon (meaning(s), PoS, syntactic and semantic
constraints, etc.)
 From perception to processing

“the class is over”

Phonetic Lexical Morphological Syntactic Semantic Pragmatic

Processing processing Processing Processing Processing Processing

Morphological / Syntactic / Morpho-syntactic processing

• is -> 3rd Sing, present tense
• “the window” -> NP
• “is over” - > VP
 From perception to processing

“the class is over”

Phonetic Lexical Morphological Syntactic Semantic Pragmatic

Processing processing Processing Processing Processing Processing

Semantic processing
• computation of literal meaning (“no more class”)
 From perception to processing

“the class is over”

Phonetic Lexical Morphological Syntactic Semantic Pragmatic

Processing processing Processing Processing Processing Processing

Pragmatic processing…to reach a conclusion about the meaning

• consideration of context of utterance
• world knowledge
• rules of communication
• may result in modification of literal meaning (ex: you enter classroom,
someone is sitting all alone in class and you say ”the class is over” ->
meaning?)
 From perception to processing

“the class is over”

Phonetic Lexical Morphological Syntactic Semantic Pragmatic

Processing processing Processing Processing Processing Processing

Caution:

1. Oversimplified representation of processing stages; in fact: highly complex

and details largely unknown (very active research: cognitive science, brain
sciences, psycholinguistics)
2. Processing is largely “parallel”: all processes occur almost simultaneously
and on-line (as the utterance is being produced)
 Monolingual vs. Bilingual Processing

General Observations: The processing components outlined above are valid for
monolinguals and bilinguals

However:

1. Bilinguals: process 2+ languages (and not just 1) -> multiplication of

processing components by the number of languages?
2. Bilinguals: almost never equally fluent in all their languages -> different
linguistic knowledge of their languages -> impact on speech perception and
processing
Ex: La may be better processed
Lb less so (poss. reason: missing linguistic elements like vocab items)
 Monolingual vs. Bilingual Processing

General Observations: The processing components outlined above are valid for
monolinguals and bilinguals

However:

3. In bilinguals: the processing mechanisms of each language may function

differently (because of phonological, morphological or syntactic
differences)
Ex: Turkish-Chinese bilingual
Turkish: requires complex morphological analyses, Chinese: not
Ex: Turkish-English bilingual
Turkish: scrambling (free word order), English: little scrambling
4. Bilinguals: may need to process monolingual or bilingual utterances
(teeming with code-switches and borrowings)
5. Even when in the monolingual mode (the speech signal is monolingual), the
processing mechanisms of bilinguals may be influenced by those of the
other language
 Processing Monolingual Speech
Turkish:
English: active
deactivated • Assume: La -> English, Lb -> Turkish
• listener: bilingual
• speech input: monolingual English (La
active, Lb deactivated -> as when I speak
English in class)
• Note: the monolingual speech input
transmitted to both (or all) language
systems available

monolingual
speech: English (La)
Processing Monolingual Speech

Question: Is Lb (Turkish) really deactivated

during monolingual (La, English) speech
processing?

Is processing in bilinguals…
• …selective? -> “when one language is
heard, only that language is active and is
processed”
• …nonselective? -> “the bilingual’s
different languages intervene while
processing takes place”
Processing Monolingual Speech

Processing in bilinguals appears to be

…nonselective -> “the bilingual’s different

languages intervene while processing takes
place”…even in the monolingual mode!
 Spivey & Marian (1999) – a classic

• Eye-tracking study -> allows you to track where the

participant is looking during speech perception /
comprehension
• Russian-English bilinguals
• Task: “look at board containing various objects and move
target object”
Spivey & Marian (1999) – a classic
Spivey & Marian (1999) – a classic

“stamp” in Russian (marku) and marker

(English) share initial phonetic characteristics:
marker -> competitor object (interlingual
distractor)

But: marker and ruler do not: ruler -> control

object
 Spivey & Marian (1999) – a classic
• Target object: stamp (Russian: marku, lower right square)
• Competitor object: marker (upper left corner)
• or: Control object: ruler (upper left corner)
• Instruction (in Russian): “Poloji marku nije krestika” (Put
the stamp below the cross) -> monolingual mode
• Observation: the participants' gaze briefly shifted to the
interlingual distractor object (marker), whose name in the
irrelevant language (English) was phonetically similar to the
target object (in Russian)
Spivey & Marian (1999) – a classic

Spivey & Marian (1999)

 Spivey & Marian (1999) – a classic
Observation: the word onset of the target object (e.g.,
“marku”) activated
1. Russian words in the Russian lexicon and
2. English words in the English lexicon that began in a similar
way (“marker” is very similar to “marku”)

Spivey & Marian (1999)

“Bottom-up information (the speech input) can activate

both the language being spoken and the bilingual’s other
language” -> processing is non-selective
“Bottom-up information (the speech input) can activate
both the language being spoken and the bilingual’s other
language” -> processing is non-selective

But can the activation of the non-used language also be

reduced or neutralized (selective processing)?
 Spivey & Marian (2003)
But can the activation of the nonused language also be
reduced or neutralized (selective processing)?

• Spivey & Marian observation: in the 1999 study, we put the participants in a bilingual
processing mode -> activated both languages -> encouraged nonselective processing
• Factors listed:
• participants knew they were taking part in an experiment on bilingualism,
• they were tested by bilingual experimenters who were fluent in both languages, and
• the two languages were tested in adjacent experimental sessions
• Different in 2003: monolingual mode
• different experimenters for the Russian and the English sessions who posed as
monolingual speakers,
• during testing only the language of the session was used,
• participants only took part in one or the other session
• Results: Participants looked at interlingual English competitor objects in 8% of the trials; 5%
for the control object (no difference). Compare: 32% vs. 7% in 1999 study.
But can the activation of the nonused language also be
reduced or neutralized (selective processing)?

YES
 Is bilingual processing selective or non-selective?

• Answer: “it depends”

• bottom-up, phonetic, information that is heard is processed by the language(s) that

contain(s) elements of that input -> can lead to nonselective processing
• Ex: when words are used that have similar word beginnings in the other language
• Ex: with homophones (hayli vs. highly), homographs (moral vs. moral), and cognates
(burger vs. burger)

• if the stronger language is being processed, the weaker language will not be
activated as much (or at all), and hence will not “interfere” with processing.
 More permanent influences of a language

• We just saw: the language not being processed can be activated from time to time
& can intervene in the processing of monolingual speech
• Sometimes: influences of one language on the other -> more permanent
• Usually: dominant language influences the processing of the nondominant (weaker)
language

Ex:
• sound categories of a stronger, dominant language can influence sound categories
of non-dominant language
• if dominant language has one category and the weaker language has two
categories, these two categories may be assimilated to just one category in the
weaker language
• /æ/, as in “sat,” and /ɛ/ , as in “set” -> assimilated by Dutch-English bilinguals to one
Dutch category /ɛ/ -> Dutch speakers of English not able to differentiate between
the pronunciation of “sat” and “set.”
 Nice study: Pallier et al. (2001)

• examined how Spanish-Catalan bilinguals perceive Catalan words

• Spanish: has only 5 vowels
• Catalan: has 8 vowels
• /ɛ/ does not exist in Spanish but in Catalan
• In Catalan: /netə/ and /nɛtə/ -> minimal pairs (granddaughter vs. clean)
• In Spanish: /netə/ and /nɛtə/ -> homophones

• Participants: two groups of highly skilled Spanish-Catalan bilinguals

 Group 1: Spanish-dominant bilinguals, raised in a monolingual Spanish
environment before kindergarten
 Group 2: Catalan-dominant bilinguals, raised in a monolingual Catalan
environment before kindergarten

• Experimental Paradigm: repetition priming (auditory)

 Repetition Priming

Repetition priming
• it is easier to recognize a face, word, item etc. if you have recently seen/heard that
same face, word or item.
• reactions to repeated stimuli are faster and more accurate than to non-repeated
stimuli (repetition priming)

Example:
• Participants perform a «lexical
decision» (Word? Non-Word?)
• As «dog» is repeated, your
participants react faster to «dog»
(compared to a non-repeated word) -
> repetition priming effect
 Repetition Priming
Hypothetical Situations

Expectation:
dog1
Faster reaction time for dog2 because of
repeated exposure
purt

speeth

glass

dog2
dog
purt
speeth
glass
dog
 Repetition Priming
Hypothetical Situations

Expectation:

No faster reaction time for bean /biːn/

bin because bin /bɪn/ and bean /biːn/ are
minimal pairs. /ɪ/ and /i/ are two
separate phonemes in English.
purth

table

speet

bean
 Repetition Priming
Hypothetical Situations

Expectation:

Faster reaction time for börek because

bö:rek vowel length is not a contrasting feature
in Turkish. «börek» and «bö:rek» are
perceived as the same word.
çarat

masa

posra

börek
 Pallier et al. (2001)
• /ɛ/ does not exist in Spanish but in Catalan
• In Catalan: /netə/ and /nɛtə/ -> minimal pairs Question:
(granddaughter vs. clean)
• In Spanish: /netə/ and /nɛtə/ -> homophones Will there be a repetition priming effect
for Catalan-Spanish bilinguals in an
auditory lexical decision task?

If yes -> perceived as homophones (like

/netə/1
in Spanish)
If no -> perceived as minimal pairs (like
capa in Catalan)

pesar

gata

2
/nɛtə/

1granddaughter
2clean
 Pallier et al. (2001)
Results:

Catalan-dominant bilinguals: no repetition effect ->

they processed these words as distinct (minimal pairs)
-> showed no facilitation for the “repetition”

Spanish-dominant bilinguals: behaved as if the words

were homophones -> they showed a repetition effect
that was of the same amplitude as a real repetition
 Pallier et al. (2001)

Conclusion:

Despite being very fluent bilinguals, Spanish-dominant

bilinguals had not created separate perceptual categories for
certain Catalan sounds due to the fact that their first language
(Spanish) did not have them (influence of dominant language).

Good example of how a language can have a permanent

/netə/1 influence on the processing of another language.

capa

pesar

gata

2
1granddaughter /nɛtə/
2clean
 Guillelmon & Grosjean (2001)

• examined processing of gender marking by English-French bilinguals

• gender marking: present in many languages
• French: 2 genders
• English: no gender marking

Ex:
• laFEM petiteFEM voitureFEM -> article and adjective agree with feminine Noun
(congruent gender marking)
• leMAS petitMAS problèmeMAS -> article and adjective agree with masculine Noun
(congruent gender marking)
• leMAS petitMAS voitureFEM -> article and adjective DO NOT agree with feminine Noun
(incongruent gender marking)

• congruent: “in agreement or in harmony” (vs. incongruent)

• Congruent gender marking -> faster recognition, processing of Noun

• Incongruent gender marking -> slower recognition, processing of Noun
 Guillelmon & Grosjean (2001)

• Question in Guillelmon & Grosjean (2001): “are early vs. late bilingual listeners
sensitive to gender marking?”
• Early bilinguals: bilingual as early as 5;4 years on average
• Late bilinguals: mean (average) age of onset of learning -> 24;8 years
 Guillelmon & Grosjean (2001)

• Task: “auditory naming”

• Sentences: le/la/leur joli(e) + Noun
themasculine / thefeminine /their nice + Noun

Depending on the determiner (le/la/leur), the gender marking was either

1. congruent (leMAS joliMAS bateauMAS – the nice boat)
2. incongruent (*laFEM jolieFEM bateauMAS)
3. not present (leur joli bateau)

• Participant’s task: listen to phrase (e.g., le joli bateau) and repeat the Noun
(bateau)
• Time needed to repeat -> indicator of underlying processing

• Question: Will early and late bilinguals demonstrate congruency (faster when
gender marking congruent) and incongruency (slower when gender marking
incongruent) effects?
 Guillelmon & Grosjean (2001)

Early Bilinguals
• strong congruency (left figure) and incongruency (right figure) effects
• they use gender marking like monolinguals
 Guillelmon & Grosjean (2001)

Late Bilinguals
• insensitive to gender congruency and incongruency
• they couldn’t use the masculine “le” or feminine “la” cues during processing
• the gender processing mechanism was not acquired
• permanent influence of their native language (English) on their French
 Guillelmon & Grosjean (2001)

In sum, bilinguals do indeed use gender marking during spoken word recognition, but
only if they started acquiring, and using on a regular basis, the gender-agreement
language at an early age. Late bilinguals do not seem to be able to use gender marking
during the recognition process.

Guillelmon & Grosjean (2001)

Processing Bilingual Speech

• listener: bilingual
• speech input: bilingual
• La: base language, most active
• Lb: also active but less so (only used
occasionally)
• code-switches, borrowings

“Yani o da böyle coincidence gibi bir şey. Onun

için böyle birkaç ay çok rahatsız oldum okulda.
There was almost no communication.”
 The Base Language Effect

• base language effect: “the potential effect of the base language on the processing of guest
linguistic material”
• in bilingual discourse: base language -> more strongly activated & makes up majority of
utterances (as in example below)
• Therefore: base-language linguistic units (phonemes, syllables, lexical items) are favoured
over guest-language linguistic units
• Major question: Does the base language have an impact on (e.g., slow down) the processing
of guest-language linguistic units?

“Yani o da böyle coincidence gibi bir şey. Onun için

böyle birkaç ay çok rahatsız oldum okulda. There was
almost no communication.”
 The Base Language Effect
Soares & Grosjean (1984)

• English-Portuguese bilinguals
• Task:
 “listen to sentences and detect words (or non-words) starting with a pre-specified
phoneme;
 Once you detect the word(s), make a lexical decision (press the assigned button)”
• 3 types of sentences:
1. English
2. Portuguese
3. Portuguese with English code-switches
 The Base Language Effect
Soares & Grosjean (1984)

Findings:
• In the English condition -> bilinguals as fast as monolingual English control participants
• English sentences -> read as fast as Portuguese sentences (equally fluent)
• RTs to codeswitches slower than to base language words (~150 ms) -> base language effect

Implication:
• “the base language delays the recognition of code-switches”
 The Base Language Effect
Domenighetti & Caldognetto (1999)

Participants:
• French-Italian bilinguals

RQs:
1. Does the base language delay the recognition of code-switches in a neutral context?
2. Are the words following code-switches also delayed?
a. If yes -> bilingual listener will fall behind the speaker -> is that possible??
b. Maybe: delay is just momentary, listener quickly catches up

Task 1a:
• Participants listen to a short sentence in French
• Sentence ends with a short list of words
• “Repeat the word in the second position of the list” (in example below: “grenouille”)

« J’ai entendu les mots aéroport, grenouille, sapin, collier »

I heard the words airport, frog, fir tree, collar

Task 1b (code-switched version):

• second word in list replaced by an Italian word
« J’ai entendu les mots aéroport, cena, sapin, collier »
Same task in CS version: “Repeat the word in the second position of the list”
 The Base Language Effect
Domenighetti & Caldognetto (1999)

Finding -> Tasks 1a & 1b:

• code-switched words took longer (mean difference: 50ms) to repeat -> base language effect

Tasks 2a & 2b:

• same sentences
• This time: repeat third word

In 2a: J’ai entendu les mots aéroport, grenouille, sapin, collier

In 2b: J’ai entendu les mots aéroport, cena, sapin, collier (sapin -> follows CS)

Finding:
• repeating «sapin» in 2a and 2b (right after CS) -> similar RTs
• Thus: base language effect does not spill over to next word
• listener has already caught-up with speaker in spite of CS
 The Base Language Effect
Domenighetti & Caldognetto (1999)

Overall:
• there is an obvious base-language effect (Tasks 1a & 1b)
• But: base-language effect only short-lived
• base language effect -> not carried over to later words
 The Role of Guest-Word Properties

What are the factors that play a role in how guest words (code-switches and borrowings) are
recognized in bilingual speech?

1. Factors pertaining to the listener (e.g., the fluency of the listener in the guest language)
2. The levels of activation of La and Lb (the base language effect)
3. Factors that involve various codeswitching constraints
4. Factors that concern the properties of the guest word being heard
Grosjean (2008)
 The Role of Guest-Word Properties

What are the factors that play a role in how guest words (code-switches and borrowings) are
recognized in bilingual speech?

1. Factors pertaining to the listener (e.g., the fluency of the listener in the guest language)
2. The levels of activation of La and Lb (the base language effect)
3. Factors that involve various codeswitching constraints
4. Factors that concern the properties of the guest word being heard
Grosjean (2008)

Grosjean (1988)

French-English bilinguals
 Grosjean (1988)

Q1: Are guest words which retain a phonetic cue as to

which lexicon they belong (by being pronounced clearly in
the guest language) easier to process than words which
are integrated phonetically into the base language
(French)?
The potential role of
In other words: are code-switched words easier to process
phonetics
than borrowings?

Remember:
• codeswitches -> pronounced in the guest language;
have phonetic cues as to the lexicon they are a part of
• borrowings -> integrated into the base language; have
lost most their phonetic cues
 Grosjean (1988)
Q2: Are guest words (from English) that are marked The potential role of
phonotactically (phonotactics: the permissible phonotactics
combination of phonemes and syllables in a language)
as belonging to the guest language (English) lexicon
recognized easier than words not marked in this way?

Ex: word-initial consonant clusters “sn” (snap) and

“sl” (slave) -> more frequent in English than in French
-> perceived sooner?

Q3: Are guest words (from English) that have near The potential role of
homophones in the base language (French) interlanguage
recognized with more difficulty than other guest neighbour proximity
language words?

Ex: pick (Engish) & pique (French) -> homophonic

 Grosjean (1988)
Methodology: “Gating Task”

• French-English bilinguals
• French neutral context “Il faudrait qu’on . . .” (We should…), followed by English guest words,
followed by a French final phrase
• Ex: Il faudrait qu’on slash tous les prix (We should slash all the prices)
• Gating task: a spoken word is presented in segments of increasing duration
• 1st gate: Il faudrait qu’on
• 2nd gate: Il faudrait qu’on + the first 40 ms of the guest word
• 3rd gate: after another 40ms
• 4th gate: another 40 ms…incremental increases
• After each presentation (after each gate), task:
 write down the word you think is being presented after “Il faudrait qu’on”
 indicate how confident you are about your guess;
 indicate whether you think the word is French or English
 Grosjean (1988)

Results:

• words that were marked phonotactically as belonging to only English, the

guest language (e.g., “slash,” “blot”), were identified earlier than words not
marked in this way (the phonotactic effect)
• words in the guest language (English) lexicon that had close homophones in
the base language (French) were processed with more difficulty than other
guest language words (the base-language homophone effect)
 Schulpen et al. (2003)

Aim:
• Processing of homophones (e.g., bull– boel)

Participants:
• Dutch-English bilinguals

Experiment 1: Gating Task

• participants -> asked after each gate to guess the word etc (as in Grosjean,
1988)
• Findings as expected: homophones more difficult to process

Experiment 2: Cross-Modal Priming Task

• “cross-modal” -> participants first hear a word, then see a letter string -> lexical
decision (real word in English? unreal word?)

LEAF
“word”
 Schulpen et al. (2003)

Experiment 2: Cross-Modal Priming Task

• Conditions
 homophones
 monolingual control words

Dutch “lief“ -> “lovable”

• Findings
1. Interlingual homophones -> overall more difficult to process than
monolingual control words (no surprise)
2. participants: slower reaction to English homophones when they are
preceded by Dutch spoken primes -> effect of sublexical cues
• sublexical = prelexical cues -> phonological regularities that govern
speech (and language) structure, even if no lexical access is
achieved yet
 BIMOLA

• BIMOLA -> influential computational model of bilingual

lexical access (Lewy & Grosjean)
• Figure: simplified representation
• 3 levels
1. features (phonological; shared across languages)
2. phonemes (shared and independent)
3. words (shared and independent)
• Features -> activate phonemes (bottom-up)
• Phonemes -> activate words (bi-directional)

• top-down activation (for language mode) through

global language activation, higher linguistic
information

• Models
 are very important
 need to be testable (should make clear
predictions)