Machine Translation Usage in a Children's

Workshop
Mondheera Pituxcoosuvarn(Ampere)1, Toru Ishida1, Naomi
Yamashita2, Toshiyuki Takasaki3, Yumiko Mori3
1
Kyoto University, Kyoto, Japan
2
NTT Communication Science Labs, Kyoto, Japan
3
NPO Pangaea, Kyoto, Japan
Introduction (1/3)
• Pangea, a non-profit organization (NPO), organizes an event called
Kyoto Intercultural Summer School of Youth (KISSY) once a year
• To encourage children to develop social bonds across boundaries and
motivate them to communicate with children from different countries
with different languages.
• By working on a shared project
using KISSY tool, which is a
machine translation(MT) tool

2
3
 Introduction (2/3)
• Hida1 studied the KISSY workshops of 2014 and 2015
Problems were present in the children’s
communication and collaboration
– Some of the messages were incomprehensible
because of low MT accuracy
• However, previous work did not discover how the
children overcame the problems caused by MT errors
Hida, S.: Supporting Multi-Language Communication in Children’s Workshop. Master’s thesis. Kyoto University, Kyoto, Japan.
1
4
(2016)
Introduction (3/3)
Objective
• To understand how users collaborated using the MT embedded in the
KISSY tool
Research Question
• ‘How did the children solve their communication problems encountered
when using MT’
Contribution
• We report an ethnographic study of a real-world intercultural children
workshop
5
[1] Shigenobu, T., Evaluation and usability of back translation for intercultural communication. In: Proceedings of Usability and Internationalization Global and Local User Interfaces, pp. 259-265, Springer, Berlin, Heidelberg (2007)
[2] Avramidis, E. et al.: Involving Language Professionals in the Evaluation of Machine Translation. In: Proceedings of the Language Resources and Evaluation Conference 2012, pp. 1127-1130 (2012)
[3] Morita, D. et al.: Collaborative translation by monolinguals with machine translators. In: Proceedings of the 14th international conference on Intelligent user interfaces, pp. 361-366. ACM Press, New York (2009)
[4] Yamashita, N. et al.: Effects of machine translation on collaborative work. In: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, pp. 515-524. ACM Press, New York (2006)
[5] Conference on Usability and Internationalization, pp. 91-97. Springer, Berlin, Heidelberg (2017)
[6] Nakaguchi, T. et al.: Combining Human Inputters and Language Services to provide Multi-language support system for International Symposiums. In: Proceedings of the Third International Workshop on Worldwide Language Service Infrastructure and
Second Workshop on Open Infrastructures and Analysis Frameworks for Human Language Technologies, pp. 28-35 (2016)
[7] Kita, K., et al.: Case study on analyzing multi-language knowledge communication. In: The International Conference on Culture and Computing (ICCC2012) was organized with a Symposium on Digital Media and Digital Heritage to show the latest research
and development results in the state of the art on cultural computing technologies and traditional culture, pp. 35-42. (2012)
[8] Imoto, K., et al.: A multi modal supporting tool for multi lingual communication by inducing partner's reply. In: Proceedings of the 11th international conference on Intelligent user interfaces, pp.330-332. ACM Press, New York (2006)

Related Work (1/2) Multilingual Communication Support

Category Existing Work
MT quality improvement Back-translation [1]
Involving human in MT
-Human interpreter into rating and post-editing [2]
-Monolingual to edit messages for both ends [3]
Support tools Multilingual Chat
AnnoChat [4], Langrid Chat [5], and Online Multilingual
Discussion Tool (OMDT) [6]
YMC system [7] (Specific purpose)
Other
Display possible answers based on the question’s intention type
[8]

6
 Related Work (2/2)
Difficulties of using MT
• Asymmetry in translation causes
difficulties in creating common
ground and trouble in accomplishing
Tangram figures to be arranged [4]
a tangram arranging task [4]

[4] Yamashita, N. et al.: Effects of machine translation on collaborative work. In: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, pp. 515-524. ACM
Press, New York (2006) 7
KISSY2017
Participant
• 8-14 years old
• From Japan,
Korea, Cambodia, etc.
Task
• Create a short clay animation
Duration
• 4 days
8
KISSY
tool
(1/3)

A screen shot of KISSY tool team chatroom

• Using the LanguageGrid, services provided by Google Translate

9
KISSY
tool
(2/3)

A screen shot of discussion screen (Shared the same database with team chatroom)
10
KISSY
tool
(3/3)

A screen shot of ideaboard

11
Method
Team Red
• Conducted an ethnographic study
by observing the participants and
staff at KISSY
– 2 out of 5 teams x1 x4 x2 x1

Team Green

• Video were record from afar

• Face-to-face and online interview
x1 x3 x3 x1
after the event
12
RESULT

13
 Coping with Mistranslation
Face-to-Face - Gesture
Communication
- Shared language - Picture
Alternative - Screen sharing - Language learning book
Methods - Drawing (by pointing the phrase)
(Interpreter
MT translated N/A)
message is not
understandable Talk directly to
the team leader
Interpreter Call
to seek for
Available Interpreter for
information
help
Read message in (Face-to-Face

e d
alternative language

olv
via interpreter)

s
that the interpreter

ot
If n
understand (Face-to-
Face between same
language speaker)
14
Understand Culturally Dependent Context
Machine translated as

What does it ‘餡 Red bean

Why? How?
子’
look like? paste ?
(Anko)

Non - Japanese

Japanese

The problem was solved by using image

browser to search for images of ‘Anko’ 15
Substituting MT (1)
• Tool Limitation
In the workshop, TL2 had just finished creating a new question on ideaboard but one Korean
child(K1) was using a webpage outside the KISSY tool. Since K1 was not on the chatroom
page, she did not know what was going on and which page should be viewed at that moment.
TL2 told an English-Korean interpreter(I3) to tell K1 “Go to the next question”

Interpreter (I3) , on the left-hand side, helped team leader (TL2), the man on the 16
right, to talk to (K1), the girl who is using a PC on the left, about what to do next.
Substituting MT (2)
• Using common words or signs
when there was a common word among all languages or there was a simple word that could be
understood by everyone, face-to-face communication was often used
o “Okay” or an object that everybody understood such as “Soba”, “Sushi”
o Pointing index finger down at the keyboard meant “Vote!”
• Involving Physical Objects in Communication
Team Red had a short meeting before working separately. In the meeting, before team leader
(TL1) explained the work plan of that day using a physical board with written papers on it, he
called a Korean-Japanese interpreter(I1) and a Khmer-English interpreter(I2) to help him.
Then he explained the work, mostly in Japanese and sometimes in English, while pointing on
the board from time to time. I1 translated what TL1 said from Japanese to Korean for Korean
Children in parallel. TL1 spoke English later but not for all messages in Japanese.

17
Problems in KISSY 2017
• Low Language Resource User Support and Problem
Detection
– Low quality MT
– Difficulty to detect problem when a user needs helps
• Human interpreter task overload for minority
languages
18
DESIGN IMPLICATION

19
Design implication
Supporting communication with low MT quality
• Image browser in multilingual chatroom
• Interpreter calling function with prioritization
• Showing translated result in user’s known foreign language in parallel
For more convenient communication
• MT for 1:1 to reduce human workload
• Graphic Signs and Keywords for Changing Method of Communication

20
Future Work
• We are trying to predict when misunderstanding due to background difference will
occur
• From KISSY ’Anko’ problem, people from different cultural background has different
image in their minds
• We might be able to support the intercultural communication with a bottom up
approach, from image search result
• For example, B)
A)

With Google Translate

21
 Google Images

22
 Google Images

23
 団子

Dumpling

Note* If the input

is ‘Dumplings’, the
translation will be
’Gyoza’

24
Image comparison to identify cultural difference
Select a keyword from workshop sticky notes
Assumption
• If the images differences
from 2 languages is high,
Get machine translated result
the speaker of 2 languages
have different mental image
Search for 30-50 images of both languages about one particular word
Use machine to compare search results from each
language to compare cultural or background
depended concept

If the difference is higher than a threshold, warn

the user about misunderstanding 25
Conclusion
• We reported a an ethnographic study based on how
children deal with problems caused by language
difference, cultural difference when MT is available
• Even though MT works correctly, there are still
problems caused by background difference
• We proposed design implications for the future
support system
26
Thank You