Critical Journal Review

“Diagnostic Assessment of Academic Reading: Peeping into Students’

Annotated Texts”

SUBJECT : ASSESSMENT FOR ELT

NAME : LIZA MUHARINA BAYAZID (8236111006)
ANGGINA MAYLA YASI BR KARO (8236111005)
CLASS : LTBI A 2023
LECTURER : Dr. RAHMAD HUSEIN, M.Ed.

ENGLISH APPLIED LINGUISTICS STUDY PROGRAM

POSTGRADUATE PROGRAM

UNIVERSITAS NEGERI MEDAN

2023
 Identity

First Journal:
Title Diagnostic Assessment of Academic Reading: Peeping into Students’ Annotated Texts
Journal Journal of languages
Volume and Page 7(84), 1-19
Year 2022
Author(s) Bassey E. Antia and Karin Vogt
Reviewer Anggina Mayla yasi Br Karo
Date 21 March 2024
ISSN -

Second Journal
Title Classroom-based Diagnostic Assessment Practices of EFL Instructors
Journal Iranian Journal of Language Teaching Research
Volume and Page 10(2), 77-94
Year July, 2022
Author(s) Hossein Farhady, Merve Selcuk
Reviewer Liza Muharina Bayazid
Date 21 March 2024
ISSN -
The First Journal
Abstract
Text annotations are literacy practices that are not uncommon in the reading experience of university students.
Annotations may be multilingual, monolingual, or multimodal. Despite their enormous diagnostic potentials,
annotations have not been widely investigated for what they can reveal about the cognitive processes that are
involved in academic reading. In other words, there has been limited exploration of the insights that signs
(verbal and non-verbal) inscribed by students on texts offer for understanding and intervening in their academic
reading practices. The aim of this exploratory study is to examine the diagnostic assessment potentials of
student-annotated texts. On the basis of text annotations obtained from teacher trainee students (n = 7) enrolled
at a German university, we seek to understand what different students attend to while reading, what their
problem-solving strategies are, what languages and other semiotic systems they deploy, what their level of
engagement with text is, and, critically, how the foregoing provide a basis for intervening to validate, reinforce,
correct, or teach certain reading skills and practices. Theoretically, the study is undergirded by the notion of text
movability. Data suggestive of how students journey through text are argued to have implications for
understanding and teaching how they manage attention, use dictionaries, own text meaning, and appraise text.
Introduction
Text annotation is writing, drawing, or other inscriptions made on the surface of a page, and serves a
range of purposes. It can record one’s reaction to the page contents (in the case of a reader); respond to both the
original contents and previous readers’ reactions (in the case of successive readers of the same physical text);
and indicate suggested ways of engaging with the contents of the page (in the case of an author, a publisher, or
an instructor).
Annotations may be present in text as so-called ‘null-content’ (e.g., highlights, under- lines, circling,
arrow lines to link related ideas) or with a ‘body’ (word gloss, simplification of a construction, summary,
exemplification, guiding questions, critical comment, all of which can be in the same language as the original
text, in a different language, or even in a blend of languages) (Antia and Mafofo 2021). Punctuation marks,
asterisks, strikethroughs, doodles, drawings, and voice recordings are all ways in which texts may be annotated.
However they present, annotations have been associated with a range of functions, includ- ing: improving the
comprehension of the source materials; interpreting and commenting upon the text; facilitating reflective
thinking; recording affective responses to text; calling attention to topics and important passages for future use.
This article draws on text annotations obtained from teacher trainee students (n = 7) enrolled at a German
university to examine what different students attend to while reading and their problem-solving strategies, as
well as the role of language and other semiotic systems in the process, their level of engagement with the text,
and how these insights can be leveraged for assessing and providing feedback on content reading. The study is
framed theoretically by the notion of text movability.
Method
This article used two sets of data were employed, namely, texts annotated by students and text-talks
through which students’ interpretations of their annotations and other relevant information were elicited.
Participants in the study were home language speakers of German. Across the skills of understanding, reading,
writing, and speaking, they considered themselves advanced users of German (C in the Common European
Framework of Reference for Languages).
The 25 texts received ranged from one page of annotated text to complete journal articles of, on average,
thirteen pages (not all of which were annotated). For this exploratory study, annotated texts in English and
German from seven students were selected. The cho- sen samples illustrated very clearly a number of issues
related to assessing and intervening in students’ reading. The texts were on a range of topics: intercultural
communication, physical education, research methodology, the colonial and neoliberal politics of global
English, and scholarship as a profession.
Result
 With respect to how diagnostic assessment may be applied to the service of developing or reinforcing
critical movability in student readers for teachable moments in the annotations associated, the need in
conceivably most fields for student readers to be able to:
 “critique, evaluate and synthesize information from various sources” (Bharuthram 2012, p. 205);
 “understand and evaluate the evidence that is used to support claims made by writers of texts” (NBT
Team/CETAP 2018, p. 4);
 explore or read the connections between language, knowledge, and power, as well as to interrogate the
“origins and implications of assumptions as well as other possibilities for signification’
The instructor can draw on the above lines of scholarship in framing or deconstructing, providing
reinforcing feedback that validates the relevant annotations as helpful for engaging with text and, ultimately, for
developing students as critical readers and for extending the frontiers of knowledge.
There are several ways of creating higher education teaching environments in which annotated texts can
be used for a diagnostic assessment of students’ reading in content areas;
1. Students are introduced to basic notions of annotation, reading levels, and text movability as well as from
an animated video.
2. The very latest, two weeks ahead of a lecture on topic x, the instructor posts (excerpts of) a key reading
in the course folder on the institutional learning management system.
3. The instructor would have skimmed through the ‘annotated annotations’ ahead of the class on topic x to
identify material of particular interest for the intended class objectives.
4. The instructor’s feedback.
Discussion
The notion of text movability describes, simultaneously, how students read or relate to texts and how they
“express their understanding”. It is typically operationalised as text-talks in which students respond to questions
about their reading and from which a determination is made of how they have been reading.
In highlighting the obvious strengths and areas of concern in students’ reading, it can guide and inform
both teaching and learning by drawing attention to (i) what may be attended to in text, and (ii) through what
strategies, in order to (iii) achieve particular ends. The following discussion is framed as capacity- building in
the area of reading for both content instructor and student, the former to be able to incorporate elements of
reading into their teaching on the basis of diagnostic assessment, and the latter, to enhance their metacognition
and to foster peer assessment. Building on the foregoing, a possible teaching environment for implementing a
diagnostic assessment of reading on the basis of annotations is sketched.
The treatment of each such issue entails commentary on all or several of the problem-solving strategies
used, as well as the role of language and other semiotic systems in the process, and the level of students’
engagement with text (as determined by movability types):
1. Overview of Annotation Practices
The participating students all claimed to annotate their academic texts. The thematic analysis of the text-
talks revealed that there were specific contexts that determined the frequency of the practice. A
participant qualified by claiming they would typically endeavour to annotate an English text in English,
ostensibly to make their passage from annotation to written tasks in English a lot easier.
2. Annotating for Structure
It emerged from the data that one issue students attend to while reading is the structure of the text. That
initial skimming also alerted the student that the contrast ended somewhere in the middle of the second
page, an awareness that, in turn, prompted a change in the colour of the pen (now black) used for
underlining.
3. Annotating for Word Meaning
The student uses a question mark to indicate a hypothesis concerning ‘commodified’ as meaning. The
use of inference as a means for determining word meanings is also evident in the annotation.
4. Annotating Graphically to Unpack Dense Text
The data on annotations and text-talks also point to dense or complex information as an aspect of text
that students may attend to, and how they might go about it. It was seen that, besides underlines,
highlights, text, punctuation marks, and so on, students may sometimes draw diagrams to annotate.
Indeed, a need to visualise information that is particularly dense in text is one reason why a student’s
annotation may take the form of a graphical depiction.
5. Annotating for Intertextuality
Making intertextual connections is another issue that students attend to while reading, and this is
reflected in their annotations. The student’s action of consciously building connections to previous
‘texts’ or knowledge exemplifies the form of text engagement referred to as associative movability.
6. Annotating as Rejoinder
Besides acknowledging previous texts or voices, an annotation in a text being read can also polemicise
with these previous texts or voices, interrogating and challenging them. The third form of engagement
with text, critical movability, is all about questioning, refuting, pushing back, or reading against the
grain.
Conclusion
Text annotation can be leveraged for an evidence-based assessment of, and instruction in, content specific
reading. The instructors can use annotation data and scholarship on reading to build their own capacity in
offering diagnostic assessment of students’ reading, especially at a time when the need for reading instruction is
increasingly recognised in higher education and there is concern that there is limited capacity for this task
outside of educational support or academic literacy centres. Through a process of peer diagnostic feedback on
annotated texts, students themselves might be able to enhance the quality of their engagement with text. These
diagnostic assessment potentials of annotations have been illustrated in a number of areas: unpacking the
structure or organisation of text; determining relevant word meanings; text visualisation; intertextual
connections; rejoinders to text content.
Strengths
One notable benefit of this text is in its comprehensive elucidation and inclusion of illustrative instances
pertaining to the experimental treatments implemented. Additionally, this article has an accompanying image
depicting the pupils' work, thereby enhancing the reader's comprehension of the content being elucidated within
the article. Furthermore, the paper exhibits a well-organized and tidily written structure.

Weaknesses
One potential limitation of this material pertains to the utilization of unfamiliar language. The
comprehension of the terminology employed in the essay is contingent upon the reader's own abilities. In order
to comprehend the contents of this article, it is important for readers to possess a comprehensive comprehension
of diagnostic assessment as well as annotated text.

Implementation in Indonesia

The implementation of diagnostic assessments of academic reading in Indonesia can vary depending on
factors such as educational policies, resources, and practices at different levels of the education system.
Generally, diagnostic assessments aim to evaluate students' reading abilities, identify their strengths and
weaknesses, and inform instructional planning and support.
In Indonesia, diagnostic assessments of academic reading may be integrated into various educational
settings, including schools, universities, and language learning centers. These assessments could involve
standardized tests, teacher-created assessments, or digital platforms designed to measure reading
comprehension, vocabulary knowledge, critical thinking skills, and other relevant areas.
Educators in Indonesia may use diagnostic assessment results to tailor instruction, provide targeted
interventions for students who need additional support, and track progress over time. Additionally, government
initiatives or educational programs may influence the adoption and implementation of diagnostic assessments to
improve literacy outcomes nationwide.

Suggestion
Based on the research that has been done, the following suggestions can be formulated by researchers:
1. The choice of language needs to be reconsidered so that readers understand more about the content of the
article.
2. This research was conducted in Germany, I suggest research in Asia, especially Southeast Asia.

The Second Journal

Abstract
Diagnostic assessment (DA) in language classrooms is reported to have a substantial role in providing
immediate and quality feedback for remedial work that would improve learning (Alderson, Brunfaut & Harding;
2014; Hattie & Timperley, 2007; Huhta, 2008; Jang & Wagner, 2013). However, applying DA in class remains a
challenge for many practitioners for at least two reasons: First, there is no clear conceptualization of DA in the
field of second language acquisition (Alderson, 2005; Alderson et al., 2015; Knoch & Macqueen, 2017). Despite
advancements in teacher education (TE) programs, they mostly provide only a handful of courses to equip
teacher candidates with necessary classroom assessment knowledge (Popham, 2011). To address the issue, this
study attempted to explore DA practices and procedures of experienced university EFL instructors with the hope
of developing a sound and empirically supported framework for classroom-based DA in the EFL context. The
data came from 17 instructors with an MA in TEFL and related fields with more than 5 years of teaching
experience. They received scenarios presenting certain problems, asked to explain what procedures and
processes they would employ to diagnose the problem, and what they would do as remedial instruction. Based
on the findings, we propose a data-driven framework for classroom-based DA. The paper also provides a
discussion of the need to incorporate theoretical and practical dimensions of DA in pre-service and in-service TE
programs.
Introduction
The introduction provides a comprehensive overview of the role of diagnostic assessment (DA) in
educational settings, particularly focusing on its significance in second or foreign language (SFL) teaching
contexts. It begins by discussing the responsibilities of teachers in enhancing and measuring learning through
various teaching and assessment methods, highlighting the importance of blending teaching with assessment.
The introduction acknowledges the limitations of traditional summative assessments in providing detailed
information about individual student progress, thus emphasizing the need for formative assessments, particularly
diagnostic assessments, to inform instructional approaches and interventions.
Key concepts such as the purpose of diagnostic assessment, its foundations, and its implementation are
thoroughly explored. The introduction delves into the literature on DA, noting its relative neglect in SFL
teaching contexts despite its importance. It addresses the challenges in conducting diagnostic assessments,
including the requirement for skilled diagnosticians and the shift towards focusing on process rather than
product.
Moreover, the introduction discusses the procedures involved in diagnostic assessment, highlighting
stages such as listening/observation, initial assessment, hypothesis testing, and decision-making. It also
references relevant empirical studies that demonstrate the positive impact of diagnostic assessment on various
language skills, including reading, speaking, and writing.
Overall, the introduction provides a comprehensive overview of the rationale, significance, and challenges
associated with diagnostic assessment in SFL teaching contexts, setting the stage for the subsequent discussion
and research focus of the study. It effectively synthesizes existing literature to justify the need for further
investigation into how experienced EFL instructors conduct diagnostic assessments in practice.
Method
The methodology employed in this study is well-structured and aligns with the research aim of exploring
how experienced EFL instructors engage with the four stages of diagnostic assessment (DA) proposed by
Harding et al. (2015).
1. Participants:
 The participants are appropriately selected based on convenience sampling, which is common in
qualitative research.
 Details regarding participants' background characteristics, including their professional experience and
qualifications, are provided, enhancing the credibility of the study.
 Consent forms were obtained from all participants, ensuring ethical considerations were met.
2. Instrument:
 The use of scenario-based questions is suitable for eliciting detailed responses from participants,
providing insights into their decision-making processes during diagnostic assessment.
 Justification for using scenarios over other data collection methods such as observations or
questionnaires is provided, demonstrating careful consideration of the research design.
 The scenarios are well-designed, focusing on specific reading deficiencies and allowing participants to
engage with each stage of the DA process.
3. Procedure:
 Clear instructions were given to participants, ensuring they understood the purpose of the study and what
was expected of them.
 Providing a two-week period for participants to respond to the scenarios is appropriate, allowing them
sufficient time to provide thoughtful and detailed responses.

4. Data Analysis:
  The data analysis process is systematic and rigorous, involving independent coding by researchers.
 Drawing on the theoretical framework proposed by Alderson et al. (2014) for coding categories enhances
the validity of the analysis.
 The identification of three main themes for analysis ("Means of diagnosis," "Diagnostic procedures," and
"Treatment") demonstrates a clear focus on key aspects of the diagnostic assessment process.
 The process of reaching consensus on codes through discussion among researchers adds to the reliability
of the analysis.

Result
The result presented in the study highlight several insightful observations regarding the implementation
procedures of expert EFL instructors in the diagnostic assessment (DA) process for reading skills.
1. Consistent Application of DA Principles. The study indicates that almost all participants consistently
applied the five principles of DA identified by Alderson et al. (2014). This result suggests a strong
adherence to established principles in guiding the diagnostic assessment process among experienced EFL
instructors.
2. Utilization of Multiple Sources of Information. Teachers were observed to utilize multiple sources of
information to develop a comprehensive understanding of the problem during the stages of data
collection for initial assessment and hypothesis checking.
3. Creativity in Problem-Solving. The data reveal that teachers offered innovative tasks, procedures, and
solutions to the problems presented in the scenarios. This creativity in problem-solving underscores
teachers' adaptability and resourcefulness in addressing student difficulties.
4. Iterative Nature of the DA Cycle. An additional finding indicates that if a teacher's plan does not yield
the desired results at the end of the DA cycle, they regress to earlier stages of the cycle to readdress the
problem. This iterative approach reflects teachers' commitment to ongoing assessment and intervention
refinement.

Discussion
The discussion provides a comprehensive analysis of the findings from the current research on the
implementation of diagnostic assessment (DA) procedures by experienced EFL teachers for reading skills.
1. Adherence to DA Procedures and Stages. The findings indicate that experienced EFL teachers follow the
procedures and stages outlined by Harding et al. (2015) for DA of reading skills. This suggests a
consistent and systematic approach to diagnostic assessment among the participants.
2. Illustration of DA Principles.
  The discussion exemplifies several principles of the DA process based on the participants'
responses. For instance, Principle 1 highlights the use of standardized tests for diagnostic
purposes, with teachers utilizing test results for problem-solving and tailored instruction.
 Principle 2 emphasizes the design of targeted diagnostic tools and activities for individual
students, acknowledging the complexity of designing tools for reading comprehension.
 Principle 3 underscores the importance of incorporating diverse stakeholder views, including
learners' self-assessments, in the DA process, which aligns with the findings showing teachers'
use of self-assessment forms and interviews to involve learners.
 Principle 4 confirms that all participants followed the four diagnostic stages, demonstrating a
comprehensive approach to the assessment process.
 Principle 5 indicates that participants offered multiple suggestions for future treatment, aligning
with the goal of providing feedback and guidance for learning enhancement.
3. Integration of Research Literature. The discussion effectively integrates findings from existing literature
on DA principles, such as the specificity of diagnostic feedback, the design of purpose-built diagnostic
tools, and the incorporation of learners' self-assessments. References to relevant studies support the
discussion points, enhancing the credibility of the analysis.
4. Insights into Teacher Practices. The discussion provides valuable insights into the practices of
experienced EFL teachers, illustrating their ability to creatively address reading comprehension
difficulties and involve learners in the diagnostic process. The analysis highlights the challenges
associated with designing diagnostic tools for reading comprehension and the importance of considering
diverse stakeholder perspectives.

Conclusion
In conclusion, this study contributes to the emerging field of diagnostic assessment (DA) in English as a
Foreign Language (EFL) contexts by examining the behavioral patterns of experienced teachers during the DA
process. By utilizing a scenario-based instrument, the research reveals that teachers demonstrate the capability to
identify and implement essential behavioral patterns across the stages of DA. This finding suggests that despite
the complexities inherent in language assessment, teachers possess the skills necessary for effective classroom
assessment and diagnosis of learner difficulties.
However, the study also highlights several challenges and areas for improvement. Firstly, there is a
discrepancy between the importance of diagnostic assessment and the limited amount of class time dedicated to
it. Additionally, concerns exist regarding the insufficient emphasis on assessment in initial teacher education
programs, indicating a need for enhanced training and professional development in DA practices. Furthermore,
the absence of a universally agreed-upon framework for DA poses a challenge to its widespread implementation
in classrooms.

Strengths
1. The introduction provides a thorough overview of the role and significance of diagnostic assessment
(DA) in educational settings, particularly focusing on its relevance in second or foreign language (SFL)
teaching contexts.
2. The methodology section is well-structured and aligns with the research aim of exploring how
experienced EFL instructors engage with the stages of diagnostic assessment.
3. The results section presents insightful observations regarding the implementation procedures of expert
EFL instructors in the DA process for reading skills. The findings are well-supported with quotes and
examples from participants' responses, enhancing the credibility of the analysis.
4. The discussion provides a comprehensive analysis of the findings, illustrating adherence to DA
procedures and principles among experienced EFL teachers. It effectively integrates existing literature on
DA principles and provides valuable insights into teacher practices.

Weaknesses
1. The study relies on a relatively small sample size of experienced EFL instructors from a specific context,
which may limit the generalizability of the findings to broader populations.
2. Convenience sampling may introduce selection bias, as participants who volunteer to participate may not
be representative of the entire population of EFL instructors.
3. While the study provides valuable insights into teacher practices and perceptions regarding diagnostic
assessment, it lacks empirical evidence to support the functionality of the observed behavioral patterns.
Future research could benefit from incorporating observational data or longitudinal studies to validate the
findings.
4. The discussion section briefly mentions challenges associated with diagnostic assessment, such as the
discrepancy between its importance and the limited class time dedicated to it. However, a more in-depth
discussion of the study's limitations, including potential biases or methodological constraints, would
enhance the transparency and credibility of the research.
Implementation in Indonesia
Implementing the findings and recommendations from this study in Indonesia would require a concerted
effort involving various stakeholders, including educational institutions, teacher training programs,
policymakers, and teachers themselves.
1. Teacher Training Programs. Indonesian teacher training programs should incorporate modules or
 workshops focused on diagnostic assessment (DA) practices. These modules can educate pre-service and
in-service teachers on how to effectively implement DA processes, develop diagnostic tools, and
interpret assessment results.
2. Awareness Raising. Teacher education programs and professional development initiatives should raise
awareness about the benefits of DA for improving learning outcomes. By highlighting the impact of DA
on student progress, teachers can be encouraged to adopt more diagnostic approaches in their classrooms.
3. Development of Diagnostic Tools. Indonesian educators and researchers can collaborate to develop a
variety of diagnostic tools tailored to the needs of EFL learners in the Indonesian context.
4. Pedagogical Implications. The findings of this study can inform the design of instructional materials and
teaching methodologies that prioritize diagnostic assessment. Teachers can use the rich toolkit of
diagnostic tools suggested in the study to identify and address learner weaknesses more effectively.
5. Policy Support. Government agencies and educational authorities in Indonesia should recognize the
importance of DA in improving educational outcomes and consider integrating DA practices into
national curriculum frameworks and teacher certification requirements. Policy support can provide the
necessary infrastructure and resources to facilitate the widespread implementation of DA in Indonesian
schools.
Suggestion

Based on the research that has been done, the following suggestions can be formulated by researchers:
Teacher training programs should prioritize the integration of diagnostic assessment techniques, providing
educators with the necessary skills and knowledge to effectively implement DA in their classrooms. This could
involve incorporating DA principles and practices into existing curriculum frameworks and offering specialized
workshops or courses on assessment literacy. Ongoing professional development opportunities should be made
available to practicing teachers, allowing them to continuously refine their diagnostic assessment skills and stay
abreast of emerging best practices in the field.
Further research is needed to develop and validate standardized frameworks for diagnostic assessment in
language teaching contexts. Collaborative research efforts involving educators, researchers, and assessment
experts can contribute to the creation of robust assessment frameworks that align with the unique needs of
language learners.