Measuring academic language

proficiency

Towards a new scale

Stuart Shaw
Schools Conference, Cambridge, Breakout Session 4
15th September, 2019
 ‘Playground’ vs. ‘Classroom’ language?

Mónica is an English language learner (ELL) student who, after only a few months of
interaction at school with friends and teachers, can basically communicate in English.
Mónica can tell her friends about her weekend, she can answer questions about her
family, and she can talk about her likes and dislikes.

In short, Mónica has a basic level of social language.

When she comes across maths or science terms in the classroom, however, Mónica is
not always comfortable because she is not familiar with such academic vocabulary.

What is the difference between ‘social’ and ‘academic’ language?

 Towards an academic scale of academic language proficiency
 Increasing use of Common European Framework of Reference for Languages CEFR in
international school contexts

 Blurring of 1st, 2nd or foreign language distinctions

 Potential need for a supplementary scale of academic language proficiency

 Many educators tend to focus on the macro scales of the CEFR which only touch on
academic contexts as they have to encapsulate other contexts - social or foreign
language

 Whilst CEFR provides a wealth of specialised linguistic scales, aspects of academic

language are found across various scales, which makes it hard to locate and apply
them to Content & Language Integrated Learning (CLIL) contexts
 Towards an academic scale of academic language proficiency

 Assumed that academic language proficiency features in higher proficiency end of the
CEFR scale, where there are high expectations for foreign language learners that would
not always be met by learners for whom English is L1

 An academic language proficiency scale is proposed that would draw together aspects
of academic language ability found in other scales and, if needed, could add new skills
not currently covered by the CEFR

 Reinforce link between CLIL and language learning in subjects using language of
schooling as a medium of instruction
 CEFR shortcomings?
CEFR is designed with European adult foreign language learners in
mind but was intended to be adaptable to individual contexts
However, whilst CEFR's focus is on foreign language learning there
are two foreign language contexts which are not best
accommodated:

 young learners, because there is no explicit treatment

of cognitive stage
 CLIL because language for learning is not clearly
distinguished from language for social use

Jones, 2014, p.17

Jones, N. (2014). Multilingual Frameworks: The Construction And Use Of Multilingual
Proficiency Frameworks. Studies in Language Testing 40. Cambridge: CUP.
 CEFR and CLIL: using language to learn
 Two foreign language contexts are related:
 CLIL - includes a cognitive dimension not explicitly considered in CEFR – entails
young learners learning content subjects through medium of a foreign language in
a wide variety of L2 learning contexts

 To take these two factors into account it is necessary to expand the familiar
proficiency dimension by an additional two dimensions: age and academic content
area

 3-dimensional matrix where each cell distinguishes a learner at a

 at a specific proficiency level
 at a specific age
 studying a specific subject
 Academic scale of academic language proficiency:
complex and multidimensional
 A descriptor scale for academic language proficiency is a complex and multidimensional
notion, to the extent that a functional description of academic language use inevitably
introduces a range of factors:
cognitive stage
general language proficiency (given that language of schooling may not be learner's L1)
processes and skills involved in mastering specific curricular objectives of each subject
area
processes and skills involved in learning in general

 Neither can it be assumed that these processes and skills are the same across countries or
cultures, given possibly different educational traditions and modes of discourse
 Research project
 Research questions
 What are some of the generic features of an academic language proficiency scale that
could be used in the student learning process in a variety of CLIL contexts?
 How and to what effect can such a scale be used in assessments in CLIL classrooms?

 Context: IGCSE History - general education qualification for 14 - 16 year olds

 Programmes of study/assessments delivered in English
 learners for whom English is an L2 or even an L3

 Scale would reinforce link between CLIL and language learning in subjects using language
of schooling as a medium of instruction

Chapter 2: Achieving in Content Through Language: towards a CEFR descriptor scale for academic language proficiency. Stuart Shaw. In Assessment for
learning in CLIL classrooms: Conceptualisations and practical applications (Springer, in press)
 A social constructivist approach to language learning
 Language - fundamental medium through which learning happens - learning is a social act > better
learner > Community of Practice (Lave & Wenger 1991)….. Interaction is at the heart of learning

 Cognitive constructivism (Piaget, 1977)

 Learners construct their own meaning
 Social-cultural constructivism (Vygotsky, 1981; 1986)
 stress social and collaborative nature of learning in development of cognition

 Mediation, a key feature of interactions - the learner (& teacher) functions as a social agent
creating bridges, conveying meaning either within same language or across languages

 Mediation involves use of culturally-derived psychological tools, such as utterances in natural

language, in transforming the relations between psychological inputs and outputs

 Mediation in cognition considered important for cognitive development

 “language mediates children’s knowledge of reality” (Carpendale & Lewis, 2004, p. 89)
 role of teacher being one of a mediator for student’s cognitive development
 CEFR 2018 – The Companion Volume with New Descriptors
(including mediation)

 Organised differently The user/learner acts as

a social agent who creates bridges
 Enriching descriptors and helps to construct or convey
(especially A1, C1, C2) meaning, sometimes within the same
 New Pre-A1 descriptors language, sometimes from one
language to another.
 New sets of descriptors

 Underlines, expands and

develops 2001 volume
 does not replace it!
 https://rm.coe.int/cefr-companion-volume-with-new-
descriptors-2018/1680787989
 CEFR Companion Volume and the concept of mediation

CEFR rests on a conception of language as action and demonstrates remarkable

coherence with a social-constructivist approach to learning

 CEFR places “the co-construction of meaning (through interaction) at the centre of the
learning and teaching process. This has clear implications for the classroom. At times, this
interaction will be between teacher and learner(s), but at times, it will take a collaborative
nature between learners themselves” (Council of Europe 2018, p. 27)

 Relevance to CLIL because “mediation is increasingly seen as a part of all learning, but
especially of all language learning” (p. 34)

 CEFR concept of mediation: “In both the receptive and productive modes, the written
and/or oral activities of mediation make communication possible between persons who
are unable, for whatever reason to communicate with each other directly” (p.32)
 CEFR ‘Mediation’ and the relevance of CLIL: a proposed lesson plan

 CLIL lesson plan focusses on mediation activities

 Use of academic language descriptors based on CEFR mediation scales provides practical
means of implementing content-based language learning
 scaffold of pedagogic and linguistic support allows learners to access curriculum content

 Primary role of assessment is to support learning

 Key issue central to successful CLIL practice is the achievement of intended content &
language learning outcomes (outlined in lesson plan)
 Specific, Measurable, Achievable, Realistic, and Time-bound (SMART) (Doran 1981) provide a
positive reference point for assessment for learning
 Context: IGCSE History
 Cambridge International General Certificate of Secondary Education (IGCSE)

 Cambridge IGCSE assessment takes place at the end of the course and can include written, oral,
coursework and practical assessment
 broadens opportunities for students to demonstrate their learning, particularly when their L1 is
not English

 Cambridge IGCSE History explores some of the major international issues of the 19th & 20th
centuries, as well as covering the history of particular contexts in more depth
 emphasis is on both historical knowledge and on the skills required for historical research
 encourages learners to raise questions and to develop and deploy historical skills, knowledge
and understanding in order to provide historical explanations

 Two of the syllabus aims encourage development of arguments and communication skills
 Linking IGCSE History English proficiency levels to the CEFR
 Shaw and Imam (2013) - aspects of the CEFR relevant to academic language proficiency in IGCSE
History
 syllabuses, question papers, mark schemes (inputs) and candidate performances were analysed
(outputs)
 history necessarily requires academic language (Cognitive Academic Language Proficiency, CALP) and
key academic language skills were identified

 language of instructions and questions falls mainly within B2 level (CEFR) in terms of structural
and lexical load
 CEFR level B2 could represent a critical CALP level for this age group (Imam, 2010)

 lexical input of accompanying authentic stimulus source material is much higher - students
would need to be at least CEFR C1 level to be able to process text fully
 CEFR C1 level in certain scales influences higher grades for a subject such as History
 IGCSE History is not an English comprehension test and candidates do not need to understand all the text to
perform well
Imam, H. C. & Shaw, S. D. (2013). Assessment of International Students Through the Medium of English: Ensuring Validity and Fairness in Content-Based
Examinations, Language Assessment Quarterly, 10:4, 452-475
 CEFR shortcomings?
Beacco (2007) provides a prototype for a descriptive framework for communicative/linguistic
competences involved in the teaching and learning of History

Comprehension vs. interpretation/critical response

It is plain that the specifications of the CEFR relate more to reading as comprehension than as
interpretation or critical response. For languages of instruction, the comprehension strategies
need to be re-interpreted as a function of the knowledge in the discipline (in this case, critical
comprehension)
Beacco, 2010, p.10
Beacco, J.C. (2010). Items for a description of linguistic competence in the language of schooling necessary for learning/teaching
history (end of obligatory education): An approach with reference points. Language and School Subjects. Linguistic Dimensions of
Knowledge Building in School Curricula. No.1.
 A History teaching/learning framework for
communicative/linguistic

 Beacco (2007) lists and describes the educational values targeted by history teaching:
 social situations of communication involving history
 expected historical knowledge
 existing in-school communication situations for transmission of history

 Beacco (2007) argues that the linguistic-cognitive resources needed for subject competences
could be based on CEFR

 Approach is to formulate sets of language competence descriptors that integrate CEFR can-do
statements e.g.
 ‘place the occurrence under discussion in a broader context (chronological, cultural)’
 ‘distinguish objectified discourse from judgement’ (2010, p. 10)
 CEFR Levels
 Designed as a guideline to characterise achievements of learners of FLs across Europe

 “Provides basis for mutual recognition of language qualifications and enables awarding bodies
 to define and articulate language proficiency levels and interpret language qualifications”
(Council of Europe,
 2001, p.1)

 CEFR is coherent with a social-constructivist approach to learning

CEFR level
What students can do with Type of language user
language at 6 levels of C2: mastery
proficiency within 3 broad types Proficient user C1: effective
of language user operational proficiency
B2: vantage
Independent user
The 6 levels and associated B1: threshold
scales are intended to inform A2: waystage
Basic user
development of language A1: breakthrough
 Research into Cambridge IGCSE History, involving reading (sources) and writing, led to
the following beginnings of a scale (Shaw, Imam & Hughes, 2015)

CEFR history level Quality Descriptor CEFR scales

Pragmatic
CEFR: C2 ‘Evaluate CEFR
Can create coherent and cohesive discourse making full and appropriate use of a variety of
Coherent and cohesive.
& create’ organisational patterns and a wide range of connectors and other cohesive devices
history: bonus
Text Processing
marks
Reconstructs arguments Can summarise information from different sources, reconstructing arguments and accounts in a
from different sources. coherent presentation of the overall result

Clear, complex, logical. Overall Written Production

Can write clear, smoothly flowing, complex texts in an appropriate and effective style and a logical
Smooth substitution for structure which helps the reader to find significant points
specialist words.
Reading for Information & Argument
No descriptor available
IGCSE history mark Strategic
scheme:
Can substitute an equivalent term for a word he/she can't recall so smoothly that it is scarcely
Bonus marks: evaluation of noticeable
sources Socio-linguistic
Appreciates fully the socio-linguistic and sociocultural implications of language used by native
speakers and can react accordingly
1st column (CEFR history level) shows CEFR and history mark scheme levels
2nd column (Quality) relates to a construct of history instantiated in the mark scheme Researcher
rd
3 column (Descriptor) depicts history descriptors that appear to relate to a CEFR description
4th column (CEFR scales) attempts to align the information in column 3 to the most relevant CEFR scale
 Research into Cambridge IGCSE History, involving reading (sources) and writing, led to
the following beginnings of a scale (Shaw, Imam & Hughes, 2015)

CEFR history level Quality Descriptor CEFR scales

Pragmatic
CEFR: C2 ‘Evaluate CEFR
Can create coherent and cohesive discourse making full and appropriate use of a variety of
Coherent and cohesive.
& create’ organisational patterns and a wide range of connectors and other cohesive devices
history: bonus
Text Processing
marks
Reconstructs arguments Can summarise information from different sources, reconstructing arguments and accounts in a
from different sources. coherent presentation of the overall result

Clear, complex, logical. Overall Written Production

Can write clear, smoothly flowing, complex texts in an appropriate and effective style and a logical
Smooth substitution for structure which helps the reader to find significant points
specialist words.
Reading for Information & Argument
No descriptor available
IGCSE history mark Strategic
scheme:
Can substitute an equivalent term for a word he/she can't recall so smoothly that it is scarcely
Bonus marks: evaluation of noticeable
sources Socio-linguistic
Appreciates fully the socio-linguistic and sociocultural implications of language used by native
speakers and can react accordingly
‘Evaluate’ relates to the Coherent and cohesive descriptor and aligns with the ‘Pragmatic’ CEFR scale which addresses the ways in which context
contributes to meaning
Researcher
The next descriptor - Reconstructs arguments from different sources - aligns with the ‘Text Processing’ CEFR scale
Undertaken for each CEFR level
 Outcomes from tables
 Key CEFR level for IGCSE History could be B2 - which moves language
 beyond descriptive realm (B1) into analytic realm

 A crucial jump in the history mark scheme is from explicit understanding to implicit understanding
of texts.
 Understanding implied opinions appear in the CEFR from C1

 Clearly:
 a student at B1 (or lower), using simple, descriptive language, would not have language to be able to
access, analyse, evaluate source material

 another student may have sophistication of language at C2 but may not have sufficient cognitive ability
or historical knowledge or examination technique to evaluate history source material (or all) and gain
marks at the highest level of the mark scheme

 conversely, a student with less sophisticated language at C1 or B2 still may be able to grasp content and
effectively communicate their evaluation to examiners
 IGCSE History grade descriptor and CEFR level
Grade Descriptor CEFR Scale and Level
General Linguistic Range
Grade A: Recall, select and deploy
Upper B2: can express him/herself clearly and without much sign of having to restrict what he/she wants to say. Has
relevant historical knowledge sufficient range of language to be able to...develop arguments...using some complex sentence forms.
C1: ...broad range of language to express him/herself clearly, without having to restrict what he/she wants to say.
accurately to support a coherent and C2: can exploit a comprehensive and reliable mastery of a very wide range of language to formulate thoughts
precisely, give emphasis, differentiate and eliminate ambiguity. No signs of having to restrict what he/she wants to
logical argument say.

Writing Report and Essays

Lower B2: develops an argument, giving reasons in support of or against a particular point of view … can synthesise
information and arguments from a number of sources.
Upper B2: develops an argument systematically with appropriate highlighting of significant point and relevant
supporting detail. Can evaluate different ideas or solutions to a problem.
C1: clear, well-structured expositions of complex subjects, underlining the relevant salient issues. Can expand and
support points of view at some length with subsidiary points, reasons and relevant examples.
C2: smoothly flowing, complex … essays which present a case, or give critical appreciation of proposal or literary
works … appropriate and effective logical structure which helps the reader to find significant points.

Grade A relates to Recall, select and deploy relevant historical knowledge accurately to support a coherent and logical
argument
History Grade A descriptor aligns with upper CEFR B2 for General Linguistic Range descriptor as well as elements of CEFR C1
& C2
Also aligns to elements of upper & lower B2 and instances of C1 and C2 for Writing Reports and Essays descriptor
 IGCSE History grade descriptor and CEFR level

Grade Descriptor CEFR Scale and Level

Grade C: Recall, select and deploy General Linguistic range
relevant historical knowledge in Mid-B2: expresses viewpoints and develops arguments … using some complex sentence forms to
support of a logical argument do so.
Writing Report and Essays
Lower B2: develops an argument … giving reasons in support of or against a particular point of view
… can synthesise information and arguments from a number of sources

History Grade C descriptor relates to middle of CEFR B2 on the General Linguistic Range descriptor to the lower CEFR B2
on the Writing Reports and Essays descriptor

Non-linear correspondence between history and CEFR scales

not a simple task to compare descriptors between scales
 Using academic language proficiency descriptors in the CLIL
classroom
 Illustration of how an academic language scale may be employed in the CLIL classroom is in the
application and use of Learning Outcomes

 Both content subject and language used as medium of instruction are similarly involved in
defining learning outcomes

 Achievement of intended content and language outcomes - key point central to successful CLIL
practice (Mehisto & Ting, 2017, p.214)

 A lesson plan should include what is to be achieved by learners (outcomes, content, language)

 Clear intended content and language learning outcomes afford opportunities for students to:
 establish their own learning targets and create openings for teachers to plan their lessons
 facilitate course development and create learning resources
 provide a mechanism for assessing student learning
 Clarity learning outcomes enhanced through academic CEFR descriptors
Using academic language proficiency descriptors in the CLIL classroom
 Pedagogic exploitation of academic, communicative can-do statements has potential to inform
planning and delivery of lessons, negotiation of syllabus content with learners and build an
effective learning environment

 The clarity of content and academic learning outcomes can be enhanced with references to
academic CEFR descriptors

 Academic can-do descriptors, if clear and specific, not only guide students more effectively in
their learning but also provide measurable outputs for teachers

 Students would need to be presented with exemplars of the types of language-use in order to
achieve outcomes
 based, in part, on authentic student responses
 Using academic language proficiency descriptors in
the CLIL classroom
 Effective lesson planning entailing use of academic language proficiency descriptors enables:

 teacher to set clear targets for content-area learning

 explicit teaching of the language needed to participate in content-area learning
 acknowledgement of the needs of CLIL learners
 learner participation in classroom activity based on an understanding of their language
development
 the use of cognitively challenging tasks that require learners to engage with cognitive
academic language
 provision of models of authentic language in use and opportunities to practise it
 History lesson plan using academic language proficiency descriptors
Learning Outcome(s):

● Learners understand the issues underpinning opposition to Soviet control

● Learners are aware of why and how the USSR reacted the way they did

Cognitive Academic Language Proficiency Descriptors:

● Can understand in detail a wide range of lengthy, complex authentic historic texts.
Focus Points
● Can summarise in writing and speech long and complex historical source texts, respecting the style and register of
the original, interpreting the content appropriately through the meanings of content-compatible language.
Why was there
● Can use high-level phrases, idiomatic and colloquial language in response to historic stimulus material opposition to Soviet
● Can use appropriate content-obligatory terminology which could include phrases relating to specific historic control in Hungary in
periods/events, topics and concepts in the curriculum (mainly nouns and proper nouns). 1956 and
● Can facilitate understanding of a complex historical issue by highlighting and categorising the main points, Czechoslovakia in
presenting them in a logically connected pattern and reinforcing the message by repeating the key aspects in 1968?
different ways.
● Can recognise a complex historical source text in order to focus on the points of most historic relevance to target How did the USSR
audience. react to this
opposition?
 History lesson plan using academic language
proficiency descriptors
Final Task Type of Instructional Processes Micro-Tasks Assessment Example use of cognitive academic language descriptors based on the CEFR ‘Mediating a
Input Differentiation text’ descriptors for Relaying specific information in speech and writing and Processing text
(focus on one
(Scaffold
aspect of in speech and writing:
ing)
language)
● Can summarise in writing and speech long and complex historical source texts,
Present a L1 and L2 CEFR Level C1: Reading for Individual work Vocabulary: Students providing sticky notes
respecting the style and register of the original, interpreting the content appropriately
version of information and Argument according to topic with reasons to a whole class
Textbooks,
historical facts diagram for discussion. through the meanings of content-compatible language:
authentic History Mark Scheme Levels 4
Group work ● Understanding content-compatible language from co-text
documents and 5 A small group exercise involving
Lexico-grammar: ▪ “From very early in 1968, other Communist leaders in Eastern Europe were
ICT and asking the groups to
structures that
produce a short script for a radio alarmed by developments in Czechoslovakia. It was clear to them.0 that the
Oral and written present an
C1: Mediation – Conveying news bulletin to be broadcast to growing freedom could be highly infectious.”
production interaction of time
clearly and fluently in well- the West immediately after the ● Identifying non-essential language to know in order to understand the text
and causes and the
structured language the Soviet response. Where possible
expression of ▪ “Indeed, it was not long before demonstrating Polish students shouted, ‘We
significant ideas in long, details could be based on
temporal markers. want a Polish Dubcek!’ The first sustained pressure put on the Czechoslovak
complex historical texts. authentic material from the time.
leadership came at a meeting with five member states of the Warsaw Pact in
Following presentations the
different approaches could be March 1968.”
discussed. ● Identifying language that needs to be translated
▪ “The meeting in early August between the Czechoslovak leaders and the Soviet
and East European leaders produced a compromise document. At the very time
Learner mediation helps to develop historical concepts and ideas by talking ideas through when this agreement was being reached, the Soviet leadership were sent a
and articulating the thoughts -facilitating understanding and communication
letter they had been asking for to justify an invasion.”
Use of CEFR cognitive mediation scales is relevant for CLIL context where small group, ● Identifying essential to know yet difficult to translate language
collaborative tasks constitute focus of lesson Researcher
▪ “It was a request from the hard-line members of the Czechoslovak leadership
Tasks afford class participants opportunities to share disparate input whilst allowing calling for intervention. The final decision to launch an invasion was taken
learners to exchange information work collaboratively to accomplish a common objective between 15 and 17 August.”
 Reflections
 Mere identification of new bodies of descriptive material - more or less similar to approach taken
by CEFR, does not get fully to the heart of the issue

 which is

 by describing how language operates as the object and medium of learning in the classroom

 to find ways of directing its use more effectively

 This is what the Council of Europe Platform has attempted to address - albeit with varying degrees
of success
 Reflections
 Multidimensional nature of subject is clear

 Would take a drastic degree of abstraction to entirely reduce it to a single dimension describing
something called ‘academic language’

 By accepting this, interesting challenge becomes

 to identify minimal set of constructs and parameters that would address the complexity of the
task

 If successful what would emerge would be a more complex, composite picture of an individual’s
language profile in relation to dealing with academic subject matter
 Reflections

 Work described here is in its infancy and needs significant further development

 Attempts at developing other proficiency scales besides the CEFR, such as

 the US WIDA’s English Language Development Standards
 the FörMig key-stage descriptors for German as a second language
 are already advancing

 Council of Europe Platform and European Centre for Modern Languages are engaged in a related
development that is much broader in scope, possibly involving plurilingual and intercultural
competences
 Further reading

 Shaw, S. D., & Imam, H. C. (2012). The Cambridge International Examinations bilingual research
agenda. Research Matters, 14, 42–45

 Imam, H. C. & Shaw, S. D. (2013). Assessment of International Students Through the Medium of
English: Ensuring Validity and Fairness in Content-Based Examinations, Language Assessment
Quarterly, 10:4, 452-475

 Shaw, S. D., Imam, H. C. & Hughes, S. K. (2015). Language Rich: insights from multilingual schools.
Cambridge: Cambridge University Press

 Chapter 2: Achieving in Content Through Language: towards a CEFR descriptor scale for academic
language proficiency. Stuart Shaw
in Assessment for learning in CLIL classrooms: Conceptualisations and practical applications
(Springer, in press)
 Any questions?
[email protected]