Chapter 7

Translation quality

Sonia Colina
 7.0 Introduction
• Section 7.1: specific reasons that make quality evaluation
essential to translation, in professional and educational
contexts
• Section 7.2: basic concepts of evaluation that need to be
considered before embarking on a specific evaluation task
• Section 7.3: Existing approaches to evaluation (experience-
and research-based)
• Section 7.4: Difficulties involved in evaluating translation
quality
• Section 7.5: An approach to evaluation that explicitly attempts
to address these difficulties.
• Section 7.6: An example of a tool developed with the proposed
approach (TQA)
• Section 7.7: Adaptation to other contexts
 7.1 The importance of quality
evaluation for translation
• Quality evaluation: a central component of
translation
• Two most frequent objects of evaluation: student
translations and professional translations.
ü student translations = those produced for
educational and learning purposes, within an
educational context
ü professional translations = those produced
outside the educational context for publication or
for public consumption.
 7.1 The importance of quality
evaluation for translation
• The evaluation can be driven by a wide range of
purposes à the importance of translation
quality evaluation
ü Student translation products: generally
evaluated to assess translator competence
within an educational context à formative or
summative.
ü Professional translations: assess translator
competence à translators and language
industry professionals to make well-informed
and objective employment decisions.
 7.1 The importance of quality
evaluation for translation

• Formative evaluation: provide feedback during the

learning process.
• Summative evaluation: used for final exams or
proficiency exams (to demonstrate students’ levels
of proficiency)
 7.1 The importance of quality
evaluation for translation
• Quality evaluation = a crucial element in an
educational context
(a) it guides the student along the process of
learning/acquisition of translation skills and
competence;
(b) it helps teachers determine whether educational
goals have been achieved.
 7.1 The importance of quality
evaluation for translation
• Evaluation methods: componential and rubric-
like methods (see section 7.6) à assess student
competencies by examining a product à a
learning tool

Examples:
A rubric of translation
evaluation
Keystroke logging and
screen recording
 7.1 The importance of quality
evaluation for translation
• In professional translation environment: crucial
for self-monitoring, for incorporating feedback
from colleagues and as a common, objective
framework for translation criticism and
discussion among professionals.
• Rubric methods: very powerful agents of
education and professional development à to
articulate aspects of the translation process and
research findings not seen by practitioners
7.1 The importance of quality
evaluation for translation
7.2 Basic concepts of evaluation
• Object of evaluation: what is going to be
evaluated, e.g., the translation product, the
translation process, translation competence, etc.
• Purpose of evaluation: the reason for which
the evaluation is being carried out
• Criterion-referenced evaluation: examine
translation quality with respect to a previously
established criterion.
• Norm-referenced evaluation: assess quality by
comparing with a norm
Validity and reliability
Norm vs. criterion-based
 7.3 Major approaches to evaluation
• Assessment tools and methods: two major
groups: experience- based or research-based
assessment
 7.3 Major approaches to evaluation
• Experience-based approaches
üOften consists of point systems or
marking scales to measure quality for a
specific professional organization or
industry.
üMany certification exams developed to
assess translator competencies for work
in the profession.
7.3 Major approaches to evaluation
 7.3 Major approaches to evaluation

• Theoretical and research-based approaches

ü Theoretical and research-based approaches
based on theoretical or empirical frameworks
and/or explicit models on the nature of
translation.
ü Focusing on the user of a translation or the
text.
ü Emphasizing an important aspect of quality,
and all enjoy some degree of success and
applicability.
7.3 Major approaches to evaluation

• Reader-response approaches to
translation
• Textual approaches to evaluation
à Limitations:
üFocusing on partial aspects of quality
üBeing often difficult to apply in professional
or teaching situations.
7.4 Difficulties involved in evaluating
translation quality
• Various reasons for the lack of standardized
methods of evaluation:
ü Multiple perspectives on translation or
evaluation purposes
ü The existence of multiple, unstated purposes
of evaluation, and to multiple purposes of the
target text itself.
ü A multiplicity of approaches additionally
arises from models that focus on partial
aspects of translation
ü Practical concerns about implementation
 7.5 A user-defined, comprehensive
view of quality: Description
• The Translation Quality Assessment (TQA) tool,
outlined in Colina (2008), (2009).
• The TQA tool is an evaluation rubric.
ü Its goal: to improve communication between health-
care providers and patients in the United States; to
provide language services for the patients
ü Problem arising: the quality of existing translations
along with the lack of guidance on and common
standards for evaluating the quality of materials
à an approach to deal with the existing gaps
à the framework used for the design of the TQA.
 7.5 A user-defined, comprehensive
view of quality: Description
• An evaluation approach is defined by its
nature
v Componential
v Descriptive (&
user-defined)
v Theoretically
explicit
v Testable
v Easy to apply
 7.5 A user-defined, comprehensive
view of quality: Description
• Componential
ü Evaluating major aspects of translation quality
in separate components of evaluation
ü Offering a comprehensive view of evaluation
integrating different aspects, allowing to define
the areas, add and omit components relevant.
E.g.. The TQA tool in section 7.6 uses four
components of assessment: Target Language;
Functional and Textual Adequacy; Non-Specialized
Content; and Specialized Content and Terminology.
 7.5 A user-defined, comprehensive
view of quality: Description
• Descriptive
ü Using descriptive statements to classify texts
into one of four assessment categories.
ü Defining the degree of compliance with specific
criteria.
ü Evaluating texts by matching assessment to a
descriptive statement for each evaluation
category.
à The goal: choose the statement that best
describes the text in each evaluation area
à A rubric-style evaluation system.
 7.5 A user-defined, comprehensive
view of quality: Description
• Theoretically explicit foundations
ü Theoretical assumptions must be clearly stated.
ü The translation is not measured against some
abstract notion of equivalence with the ST but all
the quality components in the translation brief.
 7.5 A user-defined, comprehensive
view of quality: Description
• Testable
ü Employing instruments (tool, rubric, etc.) to
various types of empirical testing;
ü Allowing for the testing of the approach
 7.5 A user-defined, comprehensive
view of quality: Description
• Easy to apply
ü Potential users may have some limited success
using many approaches and methods of
evaluation
ü Designers should have the user in mind when
creating evaluation tools.
 7.6 A user-defined, comprehensive
view of quality: Exemplification
The functional-
componential
approach
& the TQA tool
(Colina [2008],
[2009])
 7.6 A user-defined, comprehensive
view of quality: Exemplification
• Components of the TQA tool
ü Assessing the overall quality of a translation by
evaluating various components
ü Choosing the statement that best describes the
text under evaluation in these areas.
ü Tool components: Target Language; Functional
and Textual Adequacy; Non-Specialized Content
(Meaning); and Specialized Content and
Terminology
 7.6 A user-defined, comprehensive view
of quality: Exemplification
[1] Target Language
• The degree of adequacy of the language in the TT as an example of
TL in use.
• Some relevant questions
o Is this text full of grammatical errors, obviously written by
someone who doesn’t have a good command of the language?
o Is this text full of spelling errors, as if it had been written by
someone who does not know how to spell?
o Is the text written in something that looks like the target language,
but makes no sense and is almost incomprehensible?
o Is the text generally comprehensible but not without a certain
degree of effort from the reader to get past some strange/clumsy
constructions?
o Is the text generally acceptable, but with some structures that do
not sound right?
à under the TL component of the translation without reference to the ST
 Evaluation of Target Language
Category Description Check
number one box

1.a extremely difficult to read,

incomprehensible; serious language
proficiency issues…
1.b hard to comprehend, unnecessary
transfer of elements/ structure…
1.c generally readable, but problems and
awkward expressions
1.d similarly to texts originally written in the
TL, respond to the trans. brief…
7.6 A user-defined, comprehensive
view of quality: Exemplification

“The simple and

high item of a
quality which is
not done too
much is
produced.”

Figure 7.6 , 1.a

 Figure 7.7, 1.b
We can make more sense of it.
“Cannot display the image from a camera because
another application that displays the image,”
à no subject
à No preposition of after because
 Figure 7.8,
1.c

“We must trust to divine providence”

à “We must trust in divine providence”.
First, choose different foods from each of
the food groups. Different types of food
give our bodies the various vitamins and
minerals we need. Next, eat balanced
meals and snacks. What is a balanced
meal? A balanced meal has protein, starch
and non-starchy vegetables. If you need to
lose weight, choose the lower number of
servings from the Food Pyramid.
Figure 7.9, 1.d
 7.6 A user-defined, comprehensive
view of quality: Exemplification
[2] Functional and Textual Adequacy
• Being aspects related to conventions to identify text
types and genres.
• Helping readers of a target language know the type of a
text (a letter, an advertisement, an instruction manual,
etc.)
• Properly identifying textual type and genre: crucial for the
translation to accomplish intended purpose.
• Some questions to evaluate this category:
o Does the text sound like a brochure in the target
language? (textual feature)
o Is the language formal enough for the genre?
(formality)
o Would it be a good representative of the brochure
type in the target language? (function)
 Evaluation of Functional and Textual Adequacy

Category Description Check

number one box

2.a Disregard for the goals, purpose, function

and audience of the text….
2.b Giving some consideration to the intended
purpose and audience, but missing some
important aspect(s) …
2.c Approximating to the goals, purpose
(function) & audience needs, but not as
efficient as…
2.d accurately accomplishes the goals, purpose
(i.e., function type: informative,….
Figure 7.13 Example of descriptor (2.a).
Figure 7.15
Example of
descriptor (2.b).
 7.6 A user-defined, comprehensive
view of quality: Exemplification
[3] Non-Specialized Content (Meaning)
• Evaluating the translated text’s conveying the
ST’s meaning.
• Clarifying some translation techniques’ effect is
not measured with respect to some abstract,
idealized notion of equivalence.

“He came late”

Anh ấy đến muộn.
Đến muộn
 Non-Specialized Content (Meaning) component

Category Description Check

number one box

3.a The translation reflects or contains

important unwarranted deviations from
the original…
3.b There have been some changes in meaning,
omissions and/or additions that cannot be
justified…
3.c Minor alterations in meaning, additions or
omissions.
3.d The translation accurately reflects the
content contained in the original, insofar as
it is required …
 7.6 A user-defined, comprehensive
view of quality: Exemplification

[4] Specialized Content and Terminology

• Applying to texts with specialized terminology and
content.
• A deficient text: inadequate terminology, meaning
and coherence due to misunderstanding of
specialized content in the TT.

A physics text
“heat” = “energy” not = “ temperature”
 Specialized Content and Terminology
Component
Category Description Check
number one box

4.a Reveals unawareness/ignorance of special

terminology and/or insufficient knowledge
of specialized content.
4.b Serious/frequent mistakes involving
terminology and/or specialized content.
4.c A few terminological errors, but the
specialized content is not seriously
affected.
4.d Accurate and appropriate rendition of the
terminology. It reflects a good command of
terms and content specific to the subject.
 7.6 A user-defined, comprehensive
view of quality: Exemplification
• The procedural and mechanical steps involved
in the evaluation
• Step 1. Part I of the cover page
• Step 2. Evaluation process
• Step 3. Assignment of numerical values/scores
(optional)
• Step 4. Summary recommendation
Step 1. Part I of the cover page

Formalize a request and

orient raters to the text
à the guiding principle
for the evaluation
The translation brief:
basic information about
the source and target
text, their respective
purpose and use, and the
communicative context.
Step 2. Evaluation process
• Figure 7.5 Target Language
component.
• Figure 7.12 Functional and
Textual Adequacy component.
à Avoid source-text form
influence on rating decisions
about the TL.
• Figure 7.18 Non-Specialized
Content
• Figure 7.12 Specialized
Content
Step 3. Assignment of numerical values/scores
(optional)
Step 4. Summary recommendation

Figure 7.20 TQA tool cover page – Part II.

 7.7 A user-defined, comprehensive
view of quality: Adaptation
• The TQA tool can be
adapted for use in
various situations.
• The TQA tool can be
adapted to reflect a
range of priorities,
e.g. by modifying
numerical scores for
components and
descriptors
Figure 7.21 Customizing the scoring worksheet.
 7.7 A user-defined, comprehensive
view of quality: Adaptation
• Using a functional-componential tool for the
evaluation of student translation products

Figure 7.22
Purpose
and object
of
evaluation
in an
educational
context.
List of competences that
are relevant to each
component of a TQA
example.
 Summary
• Evaluation: essential in both professional and
educational translation practice, crucially
connected to translation theories and
conceptualizations
• Multiple purposes and conceptualizations of
translation à impossible to produce a general
framework of evaluation
• Translation evaluation approaches: explicit
theoretical or empirical foundations, being
testable, flexible, customer-defined and easy-to-
apply.
• A suggested one in this course: the TQA tool by
Colina (2008), (2009).
 Suggested TQA form for the final
examination
• TRANSLATION
QUALITY
ASSESSMENT FORM
(Adapted from Colina
(2009))
• Practice