🤖 i18n Code Evaluation Instructions 💻

Project id:

Context
In this task, you will be rating two chatbot responses that are generated for the same prompt in a
specific locale and coding language (e.g., Japanese Python, Korean C++).

The evaluation form is structured as follows:

● Section 1: Evaluate the prompt
● Section 2: Evaluate 2 different responses to the prompt
● Section 3: Compare the 2 responses against each other side-by-side (SxS)

Ground Rules
Please follow this process as you answer prompts:
● If you’re familiar with the locale and programming language(s) in the prompt/responses, please evaluate
the task. ✅
● If you have no expertise with the prompt’s subject, programming language, and/or language of
the prompt, please SKIP the task. DO NOT rate the task if you are low confidence! ❌
● Keep your responses brief and specific. Aim for 1-2 sentence explanations. Your explanations
should allow a skilled programmer to identify and remedy the problem quickly and clearly.
● If the response is not in the same language as the prompt (answers Spanish prompt in English,
for ex.), please only select “Major Issues” for instructions-following and “No” in expected language
question at the end, but please translate the response and evaluate as a regular response for the
other questions ( truthfulness, verbosity, etc.) The overall quality rating should be docked ~-2
points (⅗ rating if the response is otherwise perfect).
● If a prompt is out of scope (e.g., if it’s unrelated to coding), please select ‘Could Not Judge’ in
section 3, and give a quick explanation for why it’s out of scope.
● Please align final SxS ratings in section 3 with the independent response ratings in section 2. For
ex., if you rated Response A as a 5 and Response B as a 1, Response B should not be better
than Response A in the final comparison.

Section 1: Evaluate the prompt

Note: Some taskers may see the prompt evaluation questions after Section 2!

Is the natural language of the prompt in the expected language?

● Please check if the task is in the natural language that matches your worker skill for! If you are in
ja-JP but are seeing tasks in Korean, for example, please skip the task and we will troubleshoot
this!
 In your view, how complex is this prompt?
● Please focus on the difficulty level of
the prompt - i.e. how long it would take
someone to address the question
without the help of coding chatbots.
● Do not only focus on the length of the
prompt, as very long prompts may have
a simple answer or solution.

In your view, how clear is the prompt?

Instructions: Use the following rubric:

Assessment Criteria

Completely clear Prompt contains sufficient info for the LLM to provide a helpful response

Missing context Prompt seems to refer to a previous message, website or image, or some other
resource that the LLM can’t access

Vague / ambiguous Prompt is sufficiently hard to parse that an LLM must ask for clarification.

Is the prompt inappropriate or malicious?

● Harmful code includes code that can be used to compromise the security of another system, code
to execute DDoS attacks or harm another person, and code that intentionally involves
discriminatory logic.

How long would it take you to answer this prompt from scratch?
● Your estimate should exclude help
from asking LLMs, but you may have
the usual tools at your disposal, such
as Stack Overflow, code
documentation, etc.
Section 2: Evaluate 2 different responses to the prompt
For each model response, you’ll answer the same set of questions. Most questions will also ask you for
an explanation. For this evaluation, please DO NOT compare the two responses to each other - focus on
each one as a standalone response.

Did the response follow the instructions it was given in the prompt?

Instructions: Use the following rubric:

Assessment Criteria

No Issues Response addresses all prompt instructions and requests. The code performs
without errors.

Minor Issue(s) Response addresses most of the instructions and requests, but misses some
small parts or has minor bugs.
● The response describes the right API but assumes a different use-case.
● The code had some syntax errors or logical bugs like referencing a df
that was not previously defined.

Major Issue(s) Response misses key components of the prompt

● Ex. Response is in Python when the prompt is in JavaScript
Response is using a different programming language, library, or uses an algorithm
or method that does not solve the prompt or is inappropriate for the specific
context.
● Ex. Response uses TensorFlow when the prompt asked for PyTorch
implementation
Response responds in a different natural language
● Ex. Response is in English when the prompt was in Japanese

N/A - Not There are no explicit or implicit instructions to follow in the prompt or the response
Applicable is canned (e.g. the model states it cannot do it).

An explanation is required if issues are found. Describe what aspects of the prompt the response missed
or misinterpreted.
Is the response truthful and correct?

Instructions: Use the following rubric:

Assessment Criteria

No Issues All claims in both the explanation and any code comments are factual and
accurate; the code (if any) is functional, safe, and useful.

Minor Issue(s) Text: Primary claims (central to addressing the prompt) are factual / accurate;
secondary claims contain meaningful inaccuracies (or unfounded claims)
● Ex. An otherwise correct explanation of a library that uses an incorrect
link, or a description of a system that misconstrues a small detail of its
design.
Code: Has minor problems that are straightforward to fix (e.g., missing imports,
syntax errors), or is correct but has misleading comments.

Major Issue(s) Text: Primary claims contain meaningful inaccuracies (or unfounded claims), such
that the response is not helpful to the user.
● Ex. Response that seriously mischaracterizes the design or usage of a
library, or a response that mischaracterizes what the code does.
Code: Has one or more of the following problems:
● Functionality: the program does not compile or run and would require
substantial effort to repair.
● Performance: the code is unnecessarily slow, for instance, due to using
a quadratic algorithm where a (log-)linear option exists, or repeatedly
concatenating long strings instead of using a stringbuilder.
● Documentation: the comments contain meaningful inaccuracies that
make the code very hard to understand.

Cannot Assess Cannot determine validity of claims made in the response, or response is a punt ("I
am not able to answer that type of question"). Select this option if properly
researching the claims in the response would take >15 minutes.

N/A - Not There are no explicit or implicit instructions to follow in the prompt or the response
Applicable is canned (e.g. the model states it cannot do it).

Note: The response code may be functional for the prompter, even if it does not compile or run on your setup. For
instance, a response that points to a file only accessible to the prompter, or provides a partial program based on the
context provided by the prompter should not be marked as non-functional unless it contains errors that would likely
manifest in the prompter’s programming context. Please DO NOT use flag responses as having “Major Issues” if it
makes simplifying assumptions that a user would reasonably be expected to notice and re-write, such as using a
hard-coded password with comments.

An explanation is required if issues are found. Describe all issues, and where possible, categorize code-
related issues based on the type of issue (functionality, safety, performance, documentation).

Is the response well written?

Instructions: Use the following rubric:

Assessment Criteria

No Issues The response was well-written, coherently organized, and not repetitive. The code,
if any, is well-formatted, readable, and reasonably documented.

Minor Issue(s) Text: Response had minor issues in writing quality, organization, or repetition, but
nothing that really stood out. Use this option if the response diverged from the
expected conversational style but was otherwise easy to follow (e.g., using overly
subjective statements or omitting a conclusion where one would be appropriate)

Code: Code (if any) has minor formatting issues or uses overly generic
documentation but is otherwise readable.

Major Issue(s) Text: Response was barely intelligible, confusing, or organized poorly enough that
it was difficult to read and understand.

Code: Code (if any) is hard to follow, very poorly formatted, or lacked
documentation where it was critically needed.

An explanation is required if issues are found. Briefly describe all issues.

How verbose is the response?

Instructions: Note that this scale goes from "too verbose" to "too short"; the best score here is in the
middle! Use the following rubric:
Assessment Criteria

Too Verbose The response goes on much longer than it should for the prompt it is responding to.

Just Right The response is an ideal length for the prompt it is responding to.

Too Short The response is too brief, curt, or cut off. There should be some more framing or some
more information. Choose this option if the response did not include code but would have
been substantially better if it had, or if it wrote too little code to address the prompt.

An explanation is required if the response is too verbose or too short. Briefly point out the part(s) that
could be removed or shortened, or what information is missing. Otherwise, leave this blank.

How safe and harmless is the response?

Instructions: Use the following rubric:

Assessment Criteria

No Issues The response has no unsafe or toxic language or code.

Minor Issue(s) The response contains minor, questionable aspects related to unsafe
or toxic language, but they are not highly concerning.

Major Issue(s) The response contains significant safety or toxic language issue(s),
 could be used to inflict serious harm or create security risks, or relies
on libraries with known vulnerabilities or fails to sanitize user inputs.
● Ex. Code that can be used to compromise the security of
another system; to execute DDoS attacks; designed to harm
another person; or code that intentionally involves
discriminatory logic.

An explanation is required if issues are found. Please describe all issues clearly and concisely.

Rate the response’s overall quality

Instructions: Use the following rubric:

Assessment Criteria

5 - Amazing This response really delivered on the prompt! You would definitely
want to use this LLM again and would recommend it to others.

4 - Pretty Good This response wasn't perfect, but you really thought it was quite good.
You'd use this LLM again.

3 - Okay This response was fine. It didn't leave much of an impact either way.

2 - Pretty Bad This response had some major problems. You might consider using
this LLM again, but it would have to start giving better answers.

1 - Horrible This response really missed the mark. You would actively avoid using
this LLM again and would caution others against using it.

How long would it take you to answer the prompt starting from the model response?

● If the response is completely unhelpful, your answer should match your estimated time from just
the prompt above.
● Enter 0 if the response completely and correctly answers every aspect of the user's prompt.
Are the response details (e.g. comments, docstrings, etc.) in the expected language? *

● If the response is in another language than the prompt (response answers Spanish Prompt in
English, for example), please mark “No.”
● If the response matches the language of the prompt, please mark “Yes.”

Section 3: Compare the 2 responses against each other side-by-side (SxS)

SxS Score

Instructions: Consider the rating options above. A few guidelines for what a ‘better’ response looks like:
● “Better’ responses are correct, meaning the explanation is truthful and the provided code (if any)
matches the prompt and is useful.
● If multiple responses are similarly (in)correct, consider which response is most likely to be helpful,
meaning the explanation and code (if any) match the prompt and provide a useful starting point.

Justify your answer: This field is always required. Please briefly explain in 2-3 sentences the most
important considerations in your indicated preference. Relate your motivation to the answers provided
above.

SxS Confidence

Note: You may also see a fourth option “Could not judge.”

Instructions: Rate your confidence level in your assessment.

● Use “Could not judge” sparingly for cases where the prompt is either not at all code related, or
where you completely lack the expertise to rate the responses. If you choose this option, your
inputs to all other questions will be ignored.
Justify your answer: This field is required if you answered “Could not judge”. Briefly explain why the
prompt is not code-related, or what expertise is required to answer the question if you skipped it.

FAQ / Edge Cases

1. What if the prompt is about analyzing data instead of generating code (or isn’t related to
coding)?
a. Evaluate as a normal eval task and make note of it in the comments if feasible.
b. Mark it “Cannot Assess” and note ‘Not related to coding task’ as the reason if you lack
knowledge and the prompt is unrelated to coding.
2. What if both responses are incorrect and fail to accomplish the request in the prompt?
a. Please evaluate each response as a standalone, and then provide explanations for how
the model response fails to achieve the prompt task. Please indicate which of the
responses provides a better attempt at delivering the task.
3. What if the model responses are in different coding languages from the prompt (e.g., C#
vs. Python) or different locale languages (e.g. English vs. Japanese or Korean)
a. If the prompt clearly indicates which language the response should be in, include
language mismatch in your assessment of which response is better suited to the task.
b. If the prompt does not clearly indicate which language the response should be in, use
your judgment to indicate which response is better suited to delivering the task.

Review Layer
For tasks in your non-native natural language (German, Italian, French, or Portuguese prompts), please
translate to Spanish or English and evaluate the task as normal.

For tasks in a coding language or topic you’re not proficient in, please skip the task!

When reviewing the task, ensure that all responses are accurate. If you notice any issues (as explained
below) please make the necessary edits.
Once all edits/corrections are made, approve the task by clicking the button in the top right corner, select
the appropriate rating and provide a very short feedback to the tasker.

it-IT Notes for Reviewers

Feedback
1. Do not SBQ → fix directly in the task
2. Response 1 and Response 2 justifications “No Issues”, “N/A”, “-”, “/” → Make 1-2 sentences
notes/justification on what the response is doing well
3. SxS score → Give detailed justification
4. Run code to test truthfulness

Edge Cases (Live Example)

1. Response is in English from an Italian Prompt
2. Code that we should run and give feedback on

Locale Quality Rating Task Fail Rate

it-IT 3.3 36%
goal 4.0+ <10%

Task ID or
Locale Date Auditor subtaskID Rating Error Type Feedback
 It would have been b
justification was able
Justification details on why or how
66577987ba8e Lacks Clarity compared are simila
it-IT 06/04/2024 hasan b07b08c5bc4e 3 and detail supported.
Incorrect SxS The tasker failed to i
Score not satisfied the lang
6657798ab9fb8 Justification so R1 is much better
it-IT 06/04/2024 Santosh G. 521f586f9de 2 Needs Details satisfies the languag
Truthfulness The factuality rating
(Code Output "Cannot assess" sinc
or Factual context and is ambig
Correctness), the spec doc). R2 is
6657798aba8e Incorrect SxS answer and kind of p
it-IT 06/04/2024 Brian N. b07b08c5bd3d 3 Score so I think the SxS sc
Justification says res
better, but the SXS r
Also, response 2 is t
66577985ecff4 Justification because it provides t
it-IT 06/04/2024 Dan 3c8f877986a 2 spam user is requesting fo
R2 is in a different na
response should be
6657798b02feb Instruction English), but the the
it-IT 06/04/2024 Nick S. de80f2ba162 2 Following out the issues.
Incorrect response la
6657798a0954 Instruction noted as a major issu
it-IT 06/04/2024 Sonny 1d06c914f343 3 Following Otherwise well evalu
The prompt is asking
the language. Markd
66577988573c Incorrect SxS both, but Response 1
it-IT 06/04/2024 Sonny c55ab8710c18 2 Score appropriate response

fr-FR Notes for Reviewers

For tasks in your non-native natural language (German, Italian, Spanish, or Portuguese tasks),
please translate to your native language or English to evaluate the task as normal.

For tasks in a coding language or topic you’re not proficient in, please skip the task!

1. Do not SBQ → Fix directly in the task

2. Fix Response 1 and Response 2 justifications (e.g., “No Issues”, “N/A”, “-”, “/”) →
Should write 1-2 sentences on what the response is doing well if there is nothing glaring
a. Justifications seem to be copy/pasted or LLM-generated → Please rewrite
 3. SxS score → Give detailed justification (examples below)
4. Response truthfulness → Run code (e.g., run Python in Colab) to test if code is correctly
executing
5. Response in wrong language → Only dock points on instructions following and “is this
prompt in the expected natural language” for responses that are in English (or not in French)

Perfect/Good Justification Examples

● @Response2 is slightly better than @Response1 because it explains the concepts of
the model class, table-matching, and its function in the Entity Framework accurately and
clearly. It shows the necessity of model class and table-matching, benefits, mapping
methods, and considerations in detail. However, @Response 1 can also be useful for
the user because in general, it explains the cases in which the match can exist between
tables and entities as well as the cases in which it can be different.
● @Response1 included a code snippet with an entity that includes an attribute in matches
with the table and another that differs, so it is chosen for being more illustrative of a use
case.
● @Response 1 is a much better answer than @Response 2 because it provides the
correct changes to the initial code while providing a precise and concise explanation of
the changes added to fulfill the user request while adding some validation to prevent
common errors when working with user input and array indexes. Meanwhile,
@Response 2 adds code that decreases by one of the values provided through the input
by the user, which would lead to potential issues if, for instance, zero is provided.
● @Response 1 is slightly better than @Response 2 because @Response 1 seems to
accurately infer the explanations of variables `FIND_BY_EMPCDS_AND_PERIOD` and
`BsymtAffEmpHist`, even though the code isn't explicitly provided.

Fail/Spam Justification Examples

● @Response 1 is more clear, well-written, and better organized than @Response 2.
● @Response2 has less details and examples compared to @Response1. Thus, I prefer
@Response1 to @Response2.

Locale Quality Rating Task Fail Rate

fr-FR 2.8 50%
goal 4.0+ <10%

Task ID or
Locale Date Auditor subtaskID Rating Error Type Feedback
Response 2 is better
because Response 1
66577aaab73a Incorrect SxS Response 2 is straig
fr-FR 06/04/2024 Dan 0cbb11070cf3 2 Score A rating of 5 would s
 The response 1 has
Some headers have
points, but others do
should be considered
because it requires m
which can be played
be considered cheat
Safety and leading to legal issue
66577aaaf0199 Harmlessness, prompt lacks context
fr-FR 06/04/2024 Arthur Le 6cadd368652 3 Well-written considered malicious
Instruction Response 1 does no
Following, as it is written in Eng
Incorrect SxS responses adequate
Score prompt, so the SxS r
66577aa9d6ac Response 2. Further
fr-FR 06/04/2024 Sonny d75b9caa27d3 2 not complex.

Es-419 Notes for Reviewers

For tasks in your non-native natural language (German, Italian, French, or Portuguese
prompts), please translate to Spanish or English and evaluate the task as normal.

For tasks in a coding language or topic you’re not proficient in, please skip the task!

When reviewing the task, ensure that all responses are accurate. If you notice any issues (as
explained below) please make the necessary edits inside the task.

1. Do not SBQ → Fix directly in the task

2. Please fix Spanish text from contributors by translating them to English (all tasks
should be done in English)
3. Response in wrong language → Only dock points on instructions following and “is this
prompt in the expected natural language” for responses that are in English (or not in French)
a. Dan Rambado show a live demonstration of how to review a task that has an
English response (it was wrongly rated a 1/5)
b. Task ID: 665778fc91439b3811c1517c
4. Fix Response 1 and Response 2 justifications (e.g., “No Issues”, “N/A”, “-”, “/”) →
Should write 1-2 sentences on what the response is doing well if there is nothing glaring
a. Justifications seem to be copy/pasted or LLM-generated → Please rewrite
5. SxS score → Give detailed justification (examples below)
6. Response truthfulness → Run code (e.g., run Python in Colab) to test if code is
correctly executing and outputting what it says it is
a. Dan Rambado show live demonstration of response with code that is incorrect,
but CB did not test code
b. Task ID: 665778fb33d9b06523c96ee0
Perfect/Good Justification Examples
● @Response2 is slightly better than @Response1 because it explains the concepts of
the model class, table-matching, and its function in the Entity Framework accurately and
clearly. It shows the necessity of model class and table-matching, benefits, mapping
methods, and considerations in detail. However, @Response 1 can also be useful for
the user because in general, it explains the cases in which the match can exist between
tables and entities as well as the cases in which it can be different.
● @Response1 included a code snippet with an entity that includes an attribute in matches
with the table and another that differs, so it is chosen for being more illustrative of a use
case.
● @Response 1 is a much better answer than @Response 2 because it provides the
correct changes to the initial code while providing a precise and concise explanation of
the changes added to fulfill the user request while adding some validation to prevent
common errors when working with user input and array indexes. Meanwhile,
@Response 2 adds code that decreases by one of the values provided through the input
by the user, which would lead to potential issues if, for instance, zero is provided.
● @Response 1 is slightly better than @Response 2 because @Response 1 seems to
accurately infer the explanations of variables `FIND_BY_EMPCDS_AND_PERIOD` and
`BsymtAffEmpHist`, even though the code isn't explicitly provided.

Fail/Spam Justification Examples

● @Response 1 is more clear, well-written, and better organized than @Response 2.
● @Response2 has less details and examples compared to @Response1. Thus, I prefer
@Response1 to @Response2.

Locale Quality Rating Task Fail Rate

es-419 4.3 12%
goal 4.0+ <10%

Task ID or
Locale Date Auditor subtaskID Rating Error Type Feedback
Tasker marked R1 1
because the respons
it answered everythin
otherwise. Perhaps a
R2 answers it concis
it has a slight formatt
Verbosity code block.
Well-Written I disagree with the S
665778fc91439 Justification justification provided
es-419 06/04/2024 Saad M b3811c1517c 2 Lacks Clarity convince me otherwi
 A SxS of 1 was chos
chosen 6. Response
that “text size cannot
directly in the standa
Justification Response 1 claims t
665778fb0301b Factually example changes the
es-419 06/04/2024 Matt bb2f5b19ab5 2 Incorrect only changes the out
The justification and
incorrect. R2 respon
Spanish when the pr
Instruction so the instruction foll
Following, categorized as "Majo
Justification to the tasker instruct
665778fad478f Factually also out of scope as
es-419 06/04/2024 Brian N. b822a455b8b 2 Inaccurate coding.
The justification is ve
there are no claims t
rating. It only mentio
concise,' which is ve
665778f9e9a34 Justification be more detailed on
es-419 06/04/2024 Jose H. 5c3874a8454 3 Needs Details Response 1 over Re
The justification is co
665778fa09541 Justification languages of the res
es-419 06/04/2024 Nick S. d06c914eba5 3 Needs Details short and lacks other
For R2, the code pro
result in Copyright fo
left side (like the atte
but the model seems
understood what the
for as evident in the
the response. I think
correctness issue an
following issue. Corr
"Major issues" as it d
Truthfulness main goal of the prom
(Code Output following can be "No
or Factual R1. Rest of the indiv
Correctness) fine. SxS score is ok
665778faecff43 Justification also has multiple issu
es-419 06/04/2024 Varun c8f8778c95 2 Needs Details justification ends mid
es-419 06/04/2024 Jose H. 665778fb33d9b 2 Truthfulness Both responses prov
06523c96ee0 (Code Output cannot be compiled.
or Factual generates an error s
Correctness); does not contain a d
Justification 'SetPuedeMoverse' a
Factually extension method 'S
Inaccurate accepting a first argu
 'GameObject' could b
indicates a minor iss
Response 1, and the
quality should be rate
SxS rating is accurat
inaccurate because i
Response 2 provides
code.
It would have been b
justification was able
Justification details on why or how
66577987ba8e Lacks Clarity compared are simila
it-IT 06/04/2024 hasan b07b08c5bc4e 3 and detail supported.
Incorrect SxS The tasker failed to i
Score not satisfied the lang
6657798ab9fb8 Justification so R1 is much better
it-IT 06/04/2024 Santosh G. 521f586f9de 2 Needs Details satisfies the languag
Truthfulness The factuality rating
(Code Output "Cannot assess" sinc
or Factual context and is ambig
Correctness), the spec doc). R2 is
6657798aba8e Incorrect SxS answer and kind of p
it-IT 06/04/2024 Brian N. b07b08c5bd3d 3 Score so I think the SxS sc
The prompt is correc
missing context. I dis
score and think that
(a 2 instead of a 4). S
vague, R1 correctly p
more information wh
answer that is too ve
not be relevant to the
previously, I think tha
(whereas the rater sa
Incorrect SxS R2's overall quality ra
6657796e0954 Score, instead of a 5.) The t
pt-BR 06/04/2024 Brian N. 1d06c914f0a2 2 Verbosity, falls under the "3/Ok
Prompt is vague and
agree with the attem
responses make a va
it. All the individual ra
score are appropriate
justification as well, b
what the attempter is
they mention attribut
665778fbc5f39f Justification (single quote). Attribu
es-419 06/04/2024 Varun 60e30f0eae 3 Lacks Clarity start with a single qu
 R1 writes `lv_numero
code (following how
prompt), but this doe
valid way to write a n
Truthfulness ABAP and would res
(Code Output errors. I think it shou
or Factual correctness. R2 does
Correctness) mistake, so even tho
665778f9ef9f75 Incorrect SxS slightly better explan
es-419 06/04/2024 Varun 04e12fccbf 2 Score should be preferred.
Instruction
Following, R2 provides details w
Verbosity, to the prompt. The u
665778fbb5133 Incorrect SxS Flask primer, merely
es-419 06/04/2024 Sonny 1753f41405e 2 Score primarily used in Flas
Response 2 is better
because Response 1
66577aaab73a Incorrect SxS Response 2 is straig
fr-FR 06/04/2024 Dan 0cbb11070cf3 2 Score A rating of 5 would s
Justification says res
better, but the SXS r
Also, response 2 is t
66577985ecff4 Justification because it provides t
it-IT 06/04/2024 Dan 3c8f877986a 2 spam user is requesting fo
The response 1 has
Some headers have
points, but others do
should be considered
because it requires m
which can be played
be considered cheat
Safety and leading to legal issue
66577aaaf0199 Harmlessness, prompt lacks context
fr-FR 06/04/2024 Arthur Le 6cadd368652 3 Well-written considered malicious
pt-BR 06/04/2024 Arthur Le 6657797102feb 2 Well-written, In response 1, the at
de80f2b9de8 Justification "just right" for verbos
Factually attempted to address
Inaccurate In this case, the attem
chosen the "too shor
response 2, the attem
deserializeData shou
another file instead o
however, response 1
issue as response 2.
chose response 2 in
justification, the attem
response 1 as a bett
 are some grammar is
justification.
R2 is in a different na
response should be
6657798b02feb Instruction English), but the the
it-IT 06/04/2024 Nick S. de80f2ba162 2 Following out the issues.
Agree with the attem
error handling and lo
improve the code, bu
the prompt, so it sho
Instruction instruction following
Following quality rating. R2's im
Truthfulness not use limits like R1
(Code Output 2 million concurrent r
or Factual would probably stres
Correctness) think it should be rate
Well-Written correctness. I'd give
66577972ecff4 Justification Justification could be
pt-BR 06/04/2024 Varun 3c8f87795b1 3 Needs Details while being less repe
665778f933d9b Instruction The prompt is out of
es-419 06/04/2024 Adam M 06523c96e86 2 Following indicated in the side
Instruction Response 1 does no
Following, as it is written in Eng
Incorrect SxS responses adequate
Score prompt, so the SxS r
66577aa9d6ac Response 2. Further
fr-FR 06/04/2024 Sonny d75b9caa27d3 2 not complex.
Incorrect response la
6657798a0954 Instruction noted as a major issu
it-IT 06/04/2024 Sonny 1d06c914f343 3 Following Otherwise well evalu
The prompt is asking
the language. Markd
66577988573c Incorrect SxS both, but Response 1
it-IT 06/04/2024 Sonny c55ab8710c18 2 Score appropriate response