TRANSLATION QUALITY

ASSESSMENT
 An effective 3-step
Translation Quality Assessment method
 Defining translation quality –
What is a high-quality translation?

A high-quality translation is one that captures the

meaning of the original text and is appropriately
worded, free from error and internally consistent
This definition identifies the 4 parameters that determine translation quality:
• Meaning
• Wording / expression
• Errors
• Consistency

And the standards needed in each for a translation to be high-quality:

• Equivalence of meaning
• Appropriate wording / expression
• No mistakes
• Fully consistent
So we can assess a translation’s quality by seeing if it meets these four
criteria.
Which we can do by asking these four questions:
• Does it capture the meaning of the original?
• Does it read like it should?
• Are there any grammatical or spelling mistakes?
• Has it handled everything consistently?
The 3-step translation assessment model that
follows is a systematic process for answering
these 4 questions. It’s the process we use in our
second translator review in our quality-assured
translations, and a crucial component to our
broader quality assurance measures.
 Highly effective 3-Step Model for Assessing
Translation Quality
Step 1. Assess the translation for accuracy of meaning
Step 1. Assess the translation for accuracy of meaning
What to look for
The focus here is solely on meaning – with no other distractions.

The task is to assess whether all meaning in the source text has been correctly carried over
into the translation, with nothing extra added and nothing omitted.

How to do it
Assessing a translation for accuracy with the translation and source text side-by-side
Assessing a translation for accuracy: best practice is to work from side-by-side printouts,
systematically uncovering and assessing short sections of text.
You need to systematically compare the translation and original, phrase-by-phrase or
sentence-by-sentence.
• You should work from hard copies – because things are much more easily
missed if it’s done on screen.
• We recommend placing the two texts side-by-side, covered over, and
uncovering matching sections only as you’re working on them on. This
keeps your focus solely on the text you’re comparing and minimises
potential distractions.
Important:
It’s essential to work with short chunks of text – no more than 7 to 10 words
at a time. Anything longer makes it hard to retain all the information in
short-term memory, and therefore much more likely you’ll miss a difference
in meaning.

Each chunk of text should ideally be a discrete and complete unit of

meaning. That’s because complete ideas are easier to retain in memory than
incomplete ones. In practice this means we often work at phrase rather than
sentence level.
• Assessing translation accuracy is demanding work!
• It requires intense concentration and is mentally taxing.
• That’s why this phase has a singular focus on meaning – it’s much
more effective in picking up any inaccuracy than in a model where
you’re trying to concentrate on multiple things at once.

What this step achieves

• You’ll either confirm all meaning has been captured in the translation,
or identify places where it hasn’t and so needs changing.
• Because you’re focusing so intensely at phrase and word level, you
should also automatically pick up any typos and spelling or grammar
mistakes.
• And you should notice any inconsistency in vocabulary – the same
word or term translated in different ways.
• requires intense concentration and is mentally taxing.

• That’s why this phase has a singular focus on meaning – it’s much more effective in picking up any inaccuracy than in a model where you’re trying to concentrate on
This step answers question 1 – does the translation have the same
meaning as the original?

And partially answers question 3 – are there any grammatical or

spelling mistakes?

And partially answers question 4 – has everything been handled

consistently?
Step 2: Assess the translation’s quality of
expression
What to look for
• There are three aspects to a translation’s quality of wording –
naturalness, ambiguity and register/style.
• In this step we want to assess whether the translation:
• uses natural sounding language, flows nicely, is easy to understand,
and has no awkward sounding parts;
• has any text that’s ambiguous and so open to misinterpretation;
• has similar tone and formality/informality to the original and an
appropriate style for the intended purpose or function of the text.
How to do it
• Simply read through the translation carefully, looking out for any
wording that seems unnatural, is hard to follow, or could be
misinterpreted.
• Anything that makes you pause should be revisited.
• At the same time, you need to take in the tone and style of the text
and be happy this is appropriate.
• If you’re struggling to handle all this at the same time, split the step
into two phases. Assess the naturalness of the translation wording in
the first phase, and tone and style in the second pass through.
What this step achieves
• You’ll confirm the translation is appropriately worded, or identify
parts needing amending.
• You may also automatically pick up any remaining inconsistency of
vocab or style during this read through. With experience you develop
a sort of sixth sense, an inbuilt radar, for anything that isn’t
consistent.
So:
• This step answers question 2 – does the translation read like it
should?
And may partially answer question 4 – has everything been handled
consistently?
Step 3: Technical/formal check
• Here we run an eye over formatting and layout to ensure they are
appropriate and consistent throughout.
• Optionally, you may also check there are
no spelling or grammar errors, generally using the spell check
function available in most programs.
• We also recommend you double check proper name
spellings, dates and numbers as these are a common source of error.
• This step answers question 3 – are there any grammatical or
spelling mistakes?
And partially answers question 4 – has everything been handled
consistently?
How the 3 steps in our model have assessed the 4 translation
evaluation criteria:
The two great translation assessment
challenges: meaning and wording
• Of the 4 translation quality criteria, identifying errors and
inconsistencies is straightforward.

• But assessing if the translation is accurate and appropriately worded

is not so cut and dried.

• Here are our thoughts on the main issues.

The role of meaning in assessing translation
quality
• Meaning is a notoriously slippery concept.
• As soon as we ask what something really means, things can get
murky and unexpectedly complex. Academics have spent
decades trying to nail it all down.
• And yet, we all know what things mean, don’t we? We couldn’t
communicate or understand each other if we didn’t.
• It’s this practical, intuitive understanding of meaning that anyone
assessing translation quality needs to call on.
Equivalence of meaning – what is it?
• A high-quality translation will express all the meaning contained in
the source text. And only that meaning.
• In other words, it won’t omit, alter or add any meaning. This is what
we typically mean when we say a translation is accurate.
• Simple right? Fortunately, mostly yes.
• Firstly, it’s generally easy to identify when some meaning has
been missed in a translation (omissions). And so too if it adds
something that’s not in the original (additions).
• That just leaves altered meaning – when the translation says
something different to the original. This is often referred to as
a mistranslation or translation mistake.
Most mistranslations will also be obvious and easy to identify.

So linguists will generally agree when meanings match or don’t match.

But not always.

How different is different? – it’s not all black and white

There are grey areas, and what one reviewer might assess as a difference, another
may not.

Take these two sentences:

Place the device on the skin and check the readout on the screen

Attach the device to the skin and check the readout on the screen
• Someone focusing on the precise or literal meaning of the verbs
“place” and “attach” might identify a difference in meaning.
• But someone else might conclude the sentence is all about the device
working and giving a readout, so this distinction isn’t important.
• Anyone involved in evaluating translation quality will have many cases
like this. Times when they need to decide if something is sufficiently
different to no longer reflect the meaning of the original.
• There’ll be times where they’re in two minds.
• One approach to resolving these uncertainties is to focus on the key,
underlying or essential meaning or message. As opposed to the
superficial or literal. So you’d ask “Does the translation capture
the essential meaning/message of the original?”
• Another approach is to assess how relevant or significant a potential
difference in meaning is – is it a difference that matters?
• Both approaches accept that some non-equivalence of meaning is
acceptable, providing it’s not significant.
But is this OK?
Some people aren’t comfortable with this. They feel a translation should
carry over all meaning, both essential and superficial, wherever possible.

And they don’t see it as the translator’s (or reviewer’s) role to decide what
matters and doesn’t matter, or what’s important and what isn’t, in the
original text.
But this can create a problem.
• If we insist on all nuances of meaning being precisely and directly
conveyed, we’re forcing the translator into a pretty literal translation.
• And literal translations frequently don’t read well. They tend to use
vocabulary and wording that isn’t natural in that language. So they can
seem clunky and a bit awkward – which as we’ll see below won’t meet our
definition of a high-quality translation.
So, there’s a dilemma:
If we want every nuance of meaning to be conveyed, we’ll tick the
“Equivalence of Meaning” box but may well fail at the “Quality of
Expression” hurdle.
So what to do?
Our solution for the grey areas
This is how we see it:
1. Languages handle things differently, express ideas differently, view
the world differently. Just like the cultures they reflect. As a result,
meanings and vocabulary don’t always correspond precisely across
languages.
2. A translation needs to respect the (target) language’s way of doing
and saying things if it’s to be well-worded and read naturally.
3. This need for naturalness of wording means subtle variations in
meaning will sometimes occur.

4. Some leeway is therefore needed when comparing shades and

nuances of meaning. Expecting 100% exact (literal) equivalence isn’t
always realistic and may well compromise quality of expression.
People won’t always agree
• Judging translation accuracy and meaning equivalence is inevitably
subjective. It’s often a value judgement.
• So people will sometimes see things differently. They might disagree
on both the extent of any differences and their importance.
• A reviewer might identify a difference in meaning that the translator
doesn’t see. This is best seen as different viewpoints or opinions,
rather than black and white, right and wrong.
• They’ll likely both have sound reasons for seeing things as they do.
Putting all this into perspective
• This may be sounding rather complicated and ill-defined, but here’s
the good news:
• mostly it’ll be clear cut where a translation has captured the meaning
of the original text and where it differs;
• any differences of opinion on this should be few and far between –
just the odd phrase or clause here and there, and
• these will typically involve more subtle shades of meaning rather than
the essential or main message of that section.
Evaluating a translation’s quality of expression

• Only native speakers can really assess quality of expression. It

requires their native speaker “feel” for the language.
• Generally native speakers will agree on what is and isn’t good
wording.
• But again, not always. Phrasing that is well worded to one linguist,
may seem a little clunky to another.
• This shouldn’t be a surprise. As we go through life we all develop
connotations and emotional attachments around certain words and
expressions – both positive and negative.
• This leads to preferred ways of saying things, and other words and
phrasing we don’t like as much. Because everyone’s life experiences
are different, so too are our preferences.

• In practice this means one linguist may occasionally consider some

phrasing unnatural that another thinks is perfectly fine.

• Again it’s not a matter of black and white, right and wrong, but rather
reflections of individual wording preferences. Occasional differences
in viewpoint on wording are natural and to be expected.
This is something editors and reviewers need to bear in mind.
• Do they change something because they prefer different wording, or do
they accept the translator’s wording?

• The problem with making those sorts of stylistic changes is that other
people may not see them as improvements! We generally recommend
allowing the translator’s style and vocab preferences. In other words, only
changing something if there’s a strong reason for doing so.
• Do they change something because they prefer different wording, or do
they accept the translator’s wording?
• The problem with making those sorts of stylistic changes is that other
people may not see them as improvements! We generally recommend
allowing the translator’s style and vocab preferences. In other words, only
changing something if there’s a strong reason for doing so.
About Ambiguity
A sentence is ambiguous if it can be interpreted in different ways. An oft-quoted
example is:

“I saw a man on a hill with a telescope”.

There are six reasonable interpretations here. They depend on who is on the hill
(the writer or the man) and whether the telescope is being used by either or is
installed on the hill.

Another example is President Bush’s response when asked about finding weapons
of mass destruction in Iraq:

“But for those who say we haven’t found the banned manufacturing devices or
banned weapons, they’re wrong, we found them”.

What does “them” refer to – the devices, the weapons or both?

• Some potential interpretations will be unrealistic in context – like “saw a
man” above meaning cutting a man with a saw.
• So how does this relate to translations?
• Generally, we don’t want a translation to be ambiguous and so open to
misinterpretation. Especially not where the original wasn’t ambiguous and
had a single clear meaning.
• However, there are exceptions. Sometimes an ambiguity will be
intentional, in which case the translation needs to reflect that. Perhaps as
in President Bush’s response?
• And in certain genres, writers may encourage reader interpretation – in
poetry for example.
• But for most business texts requiring translation, clients want clarity of
meaning. Translations should reflect that and not introduce ambiguities.
Three key tips for clients to achieve better
translation quality assessment outcomes
1. Use experienced personnel for both your translation and any review
process.
An experienced translator should automatically capture all meaning. And an
experienced reviewer should make sound judgements on any grey areas and
be less likely to make or request changes that others may not see as
necessary.
2. Avoid getting embroiled in discussions over grey area issues.
These seldom reach a definitive conclusion – they’re a matter of opinion
after all. And they can take up a lot of time – for you, the translator and
reviewer.
3. Be crystal clear on who has final say on wording.
If the reviewer, we suggest they simply make any changes they think
necessary. If the translator, the reviewer should flag any queries for the
translator to consider.
 Conclusion
• Our method is a remarkably effective way of assessing translation
quality across most business needs.
• It works so well because it:
– covers all the necessary bases
– breaks the task down into manageable steps
• The key is that each step focuses on a different criterion – so doesn’t
overload the brain by trying to check too many things at once. That
simply doesn’t work and is the perfect recipe for missing things.
• It’s something of a tragedy that sound translation assessment
methodologies aren’t widely known or used.
• They’re seldom taught, either in translation degree courses or in the
workplace.