Adult Literacy and Numeracy:

An Overview of the Evidence

Annotated Bibliography
Anne Alkema and Jacqualine Rean
 Adult Literacy and Numeracy: An Overview of the Evidence

CONTENTS

Key findings 3

Introduction 4
-- Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
-- Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Embedding literacy and numeracy . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
-- Summary comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Workplace learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
-- Summary comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Impact of policy investment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
-- Summary comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Assessment Tool. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
-- Summary comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Priority groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
-- Summary comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Using ICT to enhance literacy and numeracy skills . . . . . . . . . . . 35
-- Summary comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Other literacies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
-- Summary comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Return on investment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
-- Summary comment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Concluding comments 41

References 43

Appendix: LN Diagram 48

Page / 2
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

KEY FINDINGS

This annotated bibliography draws on adult literacy, language and

numeracy (LLN) research literature from New Zealand, the UK, Ireland,
Canada, Australia and the USA. It looks at policy and practice in these
countries, the impact of these and implications for work in New Zealand.
The key findings from the literature are:
• Embedding (the practice of combining / integrating literacy and
numeracy into vocational and workplace training) improves the
likelihood of retention and success when:
-- vocational and literacy tutors work together;
-- literacy and numeracy content is deliberately connected to
vocational or real life contexts; and
-- there is a whole of organisation approach.
• LLN programmes in workplaces are successful in attracting learners
who would not otherwise participate in learning programmes. These
learners improve their attitudes to learning, their LLN and workplace
practices, and their employability skills. Programmes work best when:
-- employers understand what literacy and numeracy is and are
interested and supportive; and
-- LLN skills are taught in relation to the LLN demands of the job,
which are made obvious to learners.
• While there is limited evidence about the connection between
proficiency (increased skills) and participation, there is a strong
connection between participation and changed literacy and
numeracy practices that lead to increased knowledge, skills and
motivation. Therefore, there is a need to develop measures of
practice that can be used alongside the measures of proficiency to
fully ascertain the outcomes for learners;
• The hours of learning needed to achieve LLN skills’ gain remain
contentious because of the ways in which the researchers
describe gain:
-- in the USA, 100 hours of teaching is associated with “measurable
gain” described as improving a level / gaining a GED1;
-- studies in the UK and New Zealand found that “statistically
significant” gain was made in 35–39 hours; and
-- short courses are appropriate for those who require a ‘brush-up’
on skills.
• A qualified teaching workforce is essential, and the more qualified
this workforce, the greater progress made by learners. This applies
for the teaching of literacy, numeracy and ESOL;
• There are positive signs that ICT engages learners and contributes
1. The General Education Development Tests (GED) -- a set
to LLN skill gain, but more research is needed in this area; and of five tests that measure writing skills, social studies,
science, interpretation of literature and the arts, and
• Policy interventions need to be consistent over time and need math -- is the most common alternative way to earn a
high school credential. To pass the test, an adult must
guidance from the central government along with acknowledgement achieve a minimum total score and a minimum score
that educational change takes time, resource and support. on each of the five subject tests.

Page / 3
 Adult Literacy and Numeracy: An Overview of the Evidence

INTRODUCTION

Strategic work to improve adult literacy, language and numeracy (LLN)

skills has been underway in New Zealand since 2001 with the release
of ‘More than Words: The New Zealand Adult Literacy Strategy2 by the
Ministry of Education. The Ministry followed this with a programme
of work ‘Learning for Living’3 that gave formal shape to LLN provision
in New Zealand, with a particular focus on professionalising the
workforce and developing the Learning Progressions4. However, in
2001 the recommendations on how to improve literacy for adult Māori
in both te reo Māori and English, by the Māori Adult Literacy Working
Party has not progressed5.
The findings from the Adult Literacy and Lifeskills (ALL) Survey 2006
showed that around 43 percent of the New Zealand adult population
have less than optimal literacy skills, and 51 percent have less than
optimal numeracy skills. While there was a reduction in the proportion
of adults with very low literacy skills since the 1996 International
Adult Literacy Survey (IALS), New Zealand continues to have a large
proportion of the population who are not well equipped to participate
fully in society. This does not mean that these people cannot read
but rather that the literacy demands in today’s workplaces and wider
society require people to have higher skill sets.
In 2007 the LLN work was transferred to the Tertiary Education
Commission (TEC) who, with the Ministry, set the subsequent
operational policy direction. This has been articulated in a number of
documents, including:
• LITERACY, LANGUAGE AND NUMERACY ACTION PLAN 2008 – 2012:
Raising the literacy, language and numeracy skills of the workforce6;
• The New Zealand Skills Strategy7;
• Getting Results in Literacy and Numeracy: Contributing to
the vision that all New Zealanders enjoy a level of literacy and
numeracy that enables them to participate fully in all aspects of life
2010 – 2013 ; and the
• Adult Literacy and Numeracy Strategy 20128.

Over the years, the TEC has taken a three-pronged approach to

2. http://www.minedu.govt.nz/~/media/MinEdu/ improving adult LLN skills. This approach has included building
Files/EducationSectors/TertiaryEducation/ an instructional infrastructure (Learning Progressions, Literacy and
AdultLiteracyStrategy.pdf
3. http://www.minedu.govt.nz/NZEducation/ Numeracy for Adults Assessment Tool, Pathways Awarua), providing
EducationPolicies/TertiaryEducation/ learning opportunities, and introducing professional development for
PublicationsAndResources/Newsletters/
LearningForLiving.aspx the workforce. The activity informed by this policy approach has been
4. The Learning Progressions provide a framework a comprehensive educational change project.
that shows what adult learners know and can do at
successive points as they develop their expertise in
literacy and numeracy.
This annotated bibliography highlights the findings from research into
5. www.beehive.govt.nz/Documents/ adult LLN skills since 2006. The literature has been taken from New
Files/030908TeKawaiOraReport.doc
6. http://www.tec.govt.nz/Documents/Reports%20and%20
Zealand and countries that face similar LLN issues to New Zealand.
other%20documents/literacy-language-numeracy- These include the UK, Ireland, Canada, Australia and the USA. This
action-plan-17-september.pdf
7. http://skillsstrategy.govt.nz/
body of literature has thus been drawn on to inform New Zealand’s
8. http://www.tec.govt.nz/Tertiary-Sector/Tertiary- approach to date, and will continue to shape our thinking on these
Education-Strategy/Literacy-and-Numeracy-
Implementation-Strategy/
issues for the future. Particular attention has been paid to embedded

Page / 4
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

approaches to delivering literacy and numeracy programmes.

The information can be used as a ‘checkpoint’ for New Zealand’s work
in literacy and numeracy, including policy development, research, and
practice. The bibliography is set out in sections by LLN topics with
summaries of key points of each of the research pieces.

Methodology
The literature considered in this bibliography includes books,
evaluations, research, and journal articles written since 2006 in
New Zealand, the UK, Ireland, Canada, Australia and the USA.
It excludes research reports on the Adult Literacy and Lifeskills (ALL)
Survey (2006)9 as these provide an overview of literacy and numeracy
skills in New Zealand rather than research evidence about the impact
of policy interventions.
Research for this overview began with keyword searches on Google
Scholar under the following terms: adult learning, literacy, numeracy,
embedding, integrating, workplace learning, and assessment. All
searches included the keywords and the Boolean term ‘and’ followed
by the specific countries to be covered in the review. Once sourced,
each piece of literature was keyword searched for information, and
finally, the snowball method was used, whereby the reference lists
of publications were checked for additional sources. The Ministry of
Education library was also used for journal searches.
A possible limitation of this overview is that no check was made
to assess the reliability of the sources. However, this limitation was
mitigated by the number of studies included in the overview, which
should ensure that the evidence gathered and conclusions drawn are
sufficiently reliable.

Definitions
The definitions of key terms in this bibliography are in the table below.

Term Definition
Embedded literacy Combining the development of literacy and numeracy
and numeracy with vocational and other skills. The literacy and
numeracy skills developed provide learners with the
competence, confidence and motivation necessary for
them to succeed in the vocational programme, or at work
and in life, (National Centre of Literacy and Numeracy for
Adults)10. Embedding happens at the learner, programme
and organisational level. A diagram showing the
embedding process used by tutors is in the Appendix,
or see [email protected]

Literacy [In the adult context] literacy is the written and oral
language people use in everyday life and work. A 9. These can be accessed on http://www.educationcounts.
person’s literacy refers to the extent of their oral and govt.nz/ [In the adult context] numeracy is the bridge
written language skills and knowledge and their ability between mathematics and real life. A person’s numeracy
refers to their knowledge and understanding of
to apply these to meet the varied demands of their mathematical concepts and their ability to use their
personal study and work lives, (Tertiary Education mathematical knowledge
Commission, 2009, p. 58). 10. http://www.literacyandnumeracyforadults.com/
resources/356158

Page / 5
 Adult Literacy and Numeracy: An Overview of the Evidence

Term Definition
Numeracy Numeracy is the bridge between mathematics and real
life. A person’s numeracy refers to their knowledge and
understanding of mathematical concepts and their
ability to use their mathematical knowledge to meet
the varied demands of their personal, study and work
lives, (Tertiary Education Commission, 2009, p. 59).

Language The term Language was initially included in ‘LLN’ as a

reference to English for Speakers of Other Languages
(ESOL). This ‘L’ has to some extent been dropped, with
any references to language proficiency included in the
general term ‘literacy’.

e-learning This term refers to learning that is facilitated by

electronic technologies. It includes the usage of
computers and handheld data storage and transmittal
devices including mobile phones. It is often used to
facilitate distance learning, (Fletcher, Nicholas and
Davis, 2010).

Digital literacy Digital literacy comprises the technical ability to use,

at basic level, a computer and the internet; understand
and critically evaluate digital media; and create content
and communications (Standing Council on Tertiary
Education, Skills and Employment, 2012).

Financial literacy Financial literacy is the ability to make informed

judgements and effective decisions on the use
and management of money. It covers everything
from having financial knowledge to having the
understanding, confidence and motivation to make
financial judgements and decisions (Commission for
Financial Literacy and Retirement Income, 2012).

Health literacy Health literacy means the capacity to “obtain, process

and understand basic health information and the
services needed to make the appropriate health
decisions,” (Ministry of Health, 2010, p.iii).

Page / 6
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

Embedding literacy and numeracy

At the centre of the work to strengthen literacy and numeracy in
adults and increase the number of learning opportunities has been the
development of the embedded approach. This means integrating literacy
and numeracy into programmes in an open and transparent way.
Guidance for embedding in New Zealand is given through the
“Embedding does
Theoretical Framework11. This framework provides the research-based not mean smuggling
understandings that can be used for making decisions about how to
include and integrate literacy and numeracy into vocational training. LLN learning into the
This section describes the findings from research and evaluation in curriculum by stealth.
relation to the practice of embedding LLN into vocational courses
and programmes in tertiary education organisations and workplaces
Where it is viewed as
through industry training. something to be hidden,
Casey, H., Cara, O., Eldred, J., Grief, S., Hodge, R., Ivanic, R., Jupp, T., there is a danger that
Lopez, D., and McNeil, B. (2006) “You wouldn’t expect a maths teacher
to teach plastering…”: Embedding literacy, language and numeracy its importance will be
in post-16 vocational programmes – the impact on learning and overlooked by both
achievement. London: National Research and Development Centre.12
teachers and learners,
Keywords: adult learning, embedding, literacy, numeracy, further
education and that vital learning
The research took place with around 200 learners in 15 Further
opportunities are lost.”
Education Colleges and one large training provider in five regions in the
UK. It looked at students whose primary aim was to achieve a vocational (Carpentieri, 2007, p.51).
qualification. The key factors related to embedding that contributed to
students’ retention, achievement and attitudes included:
• teamwork between literacy, ESOL, numeracy teachers, and
vocational teachers (note, in this work this sometimes meant
literacy/numeracy specialists working in classrooms alongside
vocational tutors) – where a single teacher was asked to take dual
responsibility for teaching vocational skills and LLN, learners were
less likely to succeed;
• staff understandings, values and beliefs – positive attitudes towards
the learners and the subject matter makes a difference;
• aspects of teaching and learning that connect literacy, ESOL and
numeracy to vocational content; and
• policies and organisational features at institutional level, such as
support from senior management, means that resources can be
directed into embedded programmes.

This was one of the few reports sourced for this bibliography that
reported on outcomes for learners. Here the researchers found that:
• where LLN skills were embedded in courses, there were more
positive outcomes than in courses in which these skills were treated 11. http://www.literacyandnumeracyforadults.com/
resources/356028
separately; and
12. http://nrdc.org.uk/publications_details.asp?ID=73#

Page / 7
 Adult Literacy and Numeracy: An Overview of the Evidence

• there was a positive impact on retention, achievement and success

rates, particularly at level 2, and success rates were higher in
embedded than in non-embedded courses.

Higher retention rates were attributed to two factors:

Learning in workplaces • embedding literacy, ESOL and numeracy skills in vocational
programmes reduced the stigma associated with these areas; and
attracts those who
• teaching LLN within programmes enabled learners to cope more
would not otherwise effectively with the content of the course.
participate in learning,
This can be considered one of the seminal works on embedding
particularly men and literacy and numeracy. The findings of this research were used to
ESOL learners. inform much of the TEC’s work to date on LLN. The findings of this
research are also cited in other publications in this bibliography that
attest to the impact of embedding.
Carpentieri, D (Ed). (2007). Five years on research, development and
changing practice. London: National Research and Development
Centre.13

Keywords: adult learning, literacy, numeracy, workplaces, teaching,

embedding

This report synthesises findings from a wide range of research in the

Skills for Life (SfL) programme in England. The highlights from the
report, as relevant to the NZ context, include:
• learning in workplaces attracts those who would not otherwise
participate in learning, particularly men and ESOL learners. Of those
on programmes, many thought that they were able to do their jobs
better, and 80 percent changed their attitude towards education
and training
• workplace learning for apprentices suggested that there needs to
be a mix of embedded and discrete LLN provision;
• there is a divide between ESOL and literacy teaching, but the
research shows that teachers need to be confident in both areas;
• embedding numeracy into programmes better engages learners in
this area as they are likely to have different motivations for literacy
and ESOL than they have for numeracy;
• the importance of a qualified teaching workforce;
• the success of embedding in helping learners achieve literacy and
numeracy qualifications and vocational qualifications;
• the need for a whole of organisation approach;
• the importance of making literacy and numeracy obvious in
programmes;
• embedding responds to learners’ needs and places them at the
13. http://nrdc.org.uk/publications_details.asp?ID=126#
centre of all learning; and

Page / 8
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

• encouraging learners to practise their skills outside of learning time

led to increases in knowledge, skills and motivation.

Coben, D., Brown, M., et al. (2007). Effective teaching and learning:
numeracy. London: National Research and Development Centre.14

Keywords: numeracy, teaching, hours, further education Teachers’ practice

was quite ‘traditional’
This study took place over two years and involved 412 learners and
34 teachers in 47 classes. Two-thirds of the classes were in further with little use made of
education colleges. Forty percent of the learners were aged between
16–19. In relation to the impact of the courses the study found that:
practical resources
• learners improved their numeracy scores by an average nine percent or ICT.
although there was a range of average gains between classes;
• learners attitudes were more positive at the end of courses;
• average attendance by learners was 39 hours and many had made
Where tutors work
significant learning gain (although others needed more hours); as a team, learners
• there was no correlation between the number of hours of tuition are more likely to
and the gain in scores15. The researchers thought that this might
have been accounted for as a result of the differences in the stay in training and
programmes; complete vocational
• teachers’ practice was quite ‘traditional’ with little use made of
practical resources or ICT;
qualifications.
• teachers developed positive relationships with students; and
• teachers were, on the whole, qualified to teach mathematics
or numeracy.

The findings in this study provide evidence that gain can be made in
fewer hours of teaching. However, it also indicates that more needs
to be known about what contributes to improved learning.
Thomas, G. & Ward, J. (2009). Numeracy for adults. Latest finding
from teaching and learning research. Wellington: Ministry of
Education.16

Keywords: adult learning, embedding, numeracy

This report, based on a literature review, covers a number of aspects

related to teaching numeracy to adults. The report includes a
discussion on the successful features of embedding numeracy.
These features include:
• where tutors work as a team, learners are more likely to stay in
training and complete vocational qualifications; 14. http://nrdc.org.uk/publications_details.asp?ID=86#

• successful approaches to embedding numeracy clearly link LLN 15. Note, this was also found in a New Zealand study by the
Department of Labour in relation to increases in reading
and vocational components of the course; score in workplace programmes, but is contrary to
evidence in other studies.
• effective assessment in programmes makes use of learning
16. http://www.educationcounts.govt.nz/publications/
progressions to provide direction for teaching programmes and to tertiary_education/51931/4

Page / 9
 Adult Literacy and Numeracy: An Overview of the Evidence

monitor progress toward learning goals; and

• embedded numeracy provision is facilitated by appropriate
organisational policies, management structures, resourcing,
and working conditions.

Embedded numeracy Leach, L., Zepke, N., Haworth, P., Isaacs, P. & Nepia, W. (2009).
Organisational factors that affect delivery of adult literacy, language
provision is facilitated and numeracy provision. Wellington: Ministry of Education.17
by appropriate
Keywords: literacy, numeracy, embedding, tertiary education
organisational policies, organisations, ESOL
management structures, This review of international literature looks at embedded LLN practices
resourcing, and working and the organisational factors that impact on this provision. It also
looks at literature on the teaching of ESOL. In relation to embedding,
conditions. the review found:
• there is no ‘single, perfect’ model of embedding;
• a whole of organisation approach is needed;
• LLN needs to be built into courses;
• close collaboration is needed between LLN specialists and
vocational tutors; and
• professional development for staff is necessary.

In relation to ESOL provision, the review found:

• a need for ESOL and bi-literacy provision;
• significant differences between ESOL and literacy provision,
which requires either separate provision or teachers who are
trained in both;
• ESOL learners have diverse levels of literacy knowledge;
• ongoing need for professional development; and
• embedding could make LLN learning invisible to learners, but it is
still beneficial in terms of being relevant and meaningful.

Leach, L., Zepke, N., Haworth, P. & Isaacs, P. (2010). One size does
not fit all: how five tertiary education organisations embed literacy,
language and numeracy. Wellington: Ministry of Education.18

Keywords: embedding, literacy, numeracy, tertiary education

This research was undertaken in five organisations: one wānanga,

17. https://akoaotearoa.ac.nz/download/ng/file/group-189/ one Institute of Technology and Polytechnic (ITP), two Private
organisational-factors-that-affect-delivery-of-adult-
literacy-language-and-numeracy-provision---full- Training Establishments (PTEs), and one Industry Training
literature-review.pdf Organisation (ITO). The literature review summarised by Leach
18. http://www.educationcounts.govt.nz/publications/ et al. above was used to compare the findings against. The key
tertiary_education/one-size-does-not-fit-all-how-five-
tertiary-education-organisations-embed-literacy,-
findings were:
language-and-numeracy-summary-report/key-findings

Page / 10
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

• the need for an organisational champion to lead, motivate and

energise;
• there were different philosophical approaches to literacy – i.e.,
functional literacy as part of a wider approach to critical and cultural
literacy; human capital; social capital;
• the need for a whole of organisation approach – but there are The need for trained
different ways of doing this; staff and the
• the need for planning and policy documents to support the work; organisation to have
• the need for a strong focus on learners, including a constructivist
approach (learners involved in planning); personalised rather than
a commitment to
individualised; learning in groups; ongoing professional
• literacy learning needs to be in an authentic context – for some development of staff.
learners this will be vocational, for others, this will be family or
community-based; and
• the need for trained staff and the organisation to have a
commitment to ongoing professional development of staff.

The primary observation of the research was that while the factors
listed above were common to the organisations, the way in which
they were implemented differed.
The two pieces of research by Leach et al. are useful in that they were
conducted in the New Zealand context. The findings support the TEC’s
recently completed work on the embedded practices referred
to below.
Industry Training Federation. (2010). ITO literacy and numeracy
project: Summary of good practices. Wellington: Author.

Keywords: embedding, literacy, numeracy, industry training

This summary was written as part of a wider piece of action research

on embedding literacy and numeracy into ITOs that took place
in 2008 and 2009. The publication is written for ITOs and provides
information in relation to the culture, planning, processes and
capability required by ITOs in the areas of governance, management,
leadership, standard setting, and managing arrangements for trainees.
The end of the document provides a reflective tool that can be used
by ITOs to inform their embedding work. It is a user friendly document
that provides practical advice for ITOs on embedding.
Another piece of work that is useful for ITOs is the summary of
case studies that can be found on http://www.itf.org.nz/skills-and-
productivity/literacy-and-numeracy/ito-good-practice-project/case-
studies-and-good-stories/
Office for Standards in Education. (2011). Tackling the challenge
of low numeracy skills in young people and adults: Report summary. 19. http://moodle.unitec.ac.nz/pluginfile.php/115773/
mod_resource/content/0/Summary_-_Tackling_the_
Author.19 challenge_of_low_numeracy_skills_in_young_people_
and_adults.pdf

Page / 11
 Adult Literacy and Numeracy: An Overview of the Evidence

Keywords: numeracy, vocational training, embedding,

This report provides evidence from Ofsted inspectors’ evaluations of

59 providers of numeracy programmes. The evidence shows that:
• the most effective provision occurred where it was delivered within
Youth and adults are not vocational programmes rather than as a separate option;

aware that numeracy • greater success was found in programmes where a numeracy
specialist supported the vocational teacher to plan and deliver
is a precondition for sessions;
achieving qualifications • where numeracy was integrated into vocational programmes,
learners made good progress in developing the technical numeracy
and jobs and improving skills required for their vocational qualification and related
personal lives, so more employment;

needs to be done to • learners in successful programmes said that they saw how
numeracy related to their jobs and everyday lives and were
raise this awareness. motivated to put the effort into something they had previously
avoided; and
• youth and adults are not aware that numeracy is a precondition for
achieving qualifications and jobs and improving personal lives, so
more needs to be done to raise this awareness.

Ryan, R., McDonald, H., Sutton, A. & Doyle, S. (2012). Formative

evaluation of ITO embedded literacy and numeracy projects.
Wellington: Department of Labour.20

Keywords: embedding, literacy, numeracy, industry training

This formative evaluation looked at the implementation of pilot

projects to embed literacy and numeracy in Industry Training
Organisations (ITOs). The evaluation took place over two years
from 2009–2011 and included analysis of ITO reports to the TEC,
key informant interviews with 23 ITOs, and three case studies. The
evaluation shows that ITOs undertook the following activities in
relation to embedding:
• strategy development;
• awareness raising with employers;
• profiling the LLN demands of jobs linked to qualifications;
• building internal capability;
• assessment and diagnosis of learners needs (most ITOs developed
their own assessment tool for this);
• embedding in off-job training, including the provision of
professional development for tutors;
• embedding in on-job training, which proved challenging and
sometimes slow; and
20. http://dol.govt.nz/publications/research/formative-
evaluation-ito-literacy/Formative-Evaluation-ITO- • improving resources, including their design, and the inclusion of
Embedded-Literacy-and-Numeracy-Projects.pdf

Page / 12
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

specific literacy and numeracy activities.

Where embedding was found to be the most successful, the key

factors were:
• improving participants’ knowledge of the principles of adult
education and learning; A better alignment
• rapid identification of trainees likely to experience literacy and of the literacy and
numeracy difficulties; and
numeracy demands of
• a better alignment of the literacy and numeracy demands of
training with those of the job. (p. xiii). training with those of
The overall finding was that while ITOs had completed projects
the job. (p. xiii)
funded by the TEC, they still had considerable work to do before they
would be able to say that literacy and numeracy was fully embedded
into training. The issue with the pilots was that at their inception,
the ITOs were encouraged to be experimental, and there was little
guidance on how to implement embedding as a practice. Formal
guidance from the TEC on embedding for ITOs did not come until two
years into the project, when the embedded guidelines for ITOs were
produced.
This evaluation provides a useful insight into the challenges of
implementing policy without sufficient guidance and direction from
policy funders. However, it does show the benefits of embedding in
terms of improving the overall approach to training for ITO trainees.
Hazlewood, K. and Alkema, A. (2013). Literacy and numeracy
professional development: Research findings. Wellington: Tertiary
Education Commission.21

Keywords: literacy, numeracy, embedding, professional development,

tertiary education

The purpose of this research was to gain an understanding of the

extent to which Tertiary Education Institutes (TEIs) were embedding
LLN at the organisational, programme and learner level. It also
looked at how professional development delivered by the National
Centre of Literacy and Numeracy for Adults (the National Centre) was
supporting these TEIs to improve their LLN practice. Evidence was
gathered through interviews with 20 education providers. The key
findings of the research were:
• there was a continuum of practice related to embedding, with the
ITPs showing more mature practice than the PTEs in that they were
embedding across their organisations, had qualified staff, and were
capable of showing leadership in the field;
• those with more mature practices were more likely to have sourced
PD from the National Centre and valued the learning that the
National Centre had provided;
21. http://www.tec.govt.nz/Documents/Reports%20
• all organisations found aspects of embedding challenging, and%20other%20documents/Literacy-and-Numeracy-
Development-Findings-2013.pdf

Page / 13
 Adult Literacy and Numeracy: An Overview of the Evidence

including developing a whole of organisational approach, high staff

turn-over, and embedding literacy and numeracy in programmes
for distance learners;
• in relation to professional development, those interviewed wanted
professional development to be practical rather than theoretical
All organisations found and academic, and while they would prefer face-to-face interaction,
realised because of cost constraints that a blended approach would
aspects of embedding work for them; and
challenging, including • given the continuum of practice, it was recommended that the
developing a whole National Centre provide more targeted professional development,
especially for PTEs who have done little more than start to use the
of organisational Assessment Tool with learners, as it is a funding requirement.
approach, high These research findings were used to inform the development of the
staff turn-over, and TEC’s Embedded Literacy and Numeracy (ELN) Practices22 table that
describes embedded practices at three levels. This will be a useful tool
embedding literacy for organisations to use as they reflect on and judge their progress on
and numeracy in embedding.

programmes for Summary comment

distance learners. The literature is clear that embedding works in that the embedded
approach is more likely than other approaches to engage and retain
learners. In the New Zealand context it has been challenging for
organisations to develop whole-of-organisation approaches, upskill
staff, rethink ways of teaching, and adapt teaching materials. As the
findings from the most recent work cited above (in ITOs and TEIs)
show, there is still a way to go before the embedded approach is fully
integrated into all organisations.
Overall there is limited evidence to date about embedding as a
whole-of-organisation approach in tertiary education organisations.
It would be worthwhile for the TEC to repeat the research interviews,
conducted by Hazlewood and Alkema (2013), at the end of 2014 to
ascertain the extent to which there has been further organisational
change, particularly in the PTEs.

22. http://www.tec.govt.nz/Resource-Centre/Software-
tools/Literacy-and-Numeracy-for-Adults-Assessment-
Tool/

Page / 14
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

Workplace learning
Marr, B. & Hagston, J. (2007). Thinking beyond numbers: Learning
numeracy for the future workplace. Adelaide: National Centre for
Vocational Education Research.23

Keywords: adult learning, workplace learning, numeracy Workers often use a

This Australian study looks at the place of numeracy teaching in the range of numeracy skills
workplace, and the type of numeracy that is required by workers. in their work and do not
Evidence was gathered through a literature review, interviews
with industry representatives, and case studies in three work sites. recognise them as such.
The research found that:
• workers often use a range of numeracy skills in their work and do
not recognise them as such; Workers wanted
• numeracy was still associated in employers’ minds with ‘primary training that was
school arithmetic’;
informal and
• in automated workplaces, workers need the skills to make
judgements about accuracy, such as, having ‘a feel for numbers’ that delivered by peers and
allows for approximation and estimation;
supervisors rather than
• numeracy is important for occupational health and safety;
anything that reminded
• knowing how to store and retrieve data is becoming increasingly
important; them of school.
• workers wanted training that was informal and delivered by peers
and supervisors rather than anything that reminded them of school;
• industry representatives thought that on- and off-job training was
important as it allowed time for practice and reflection;
• numeracy training needs to be framed positively, rather than as a deficit;
• numeracy skills need to be made explicit in training; and
• input into training programmes is required from both numeracy
and industry experts.
23. http://www.ncver.edu.au/publications/1805.html
This research reinforces the UK work on embedding, and is important 24. http://www.dol.govt.nz/publications/research/
for thinking about how to frame conversations with employers, as it upskilling-evaluation/upskilling-evaluation.pdf.
Note, there are also four articles, Benseman, J. (2010).
outlines the role of numeracy in the modern workplace. Transferring literacy skills in the workplace. Reflect,
13(Summer), 9–11; Benseman, J. (2010). Planning and
Department of Labour (2010). Upskilling partnership programme: teaching effective workplace literacy, language and
numeracy programmes: what does research tell us?
Evaluation report. Wellington: Author. Journal of Adult Learning Aotearoa New Zealand 38(2),
19–28; Benseman, J. (2012). Literacy and workplace
Keywords: literacy, numeracy, workplace learning change: evaluation findings from 18 workplace literacy
programs. Journal of Research and Practice in Adult
Literacy, Secondary, and Basic Education, September 2012;
The evidence for this evaluation of workplace literacy programmes, Benseman, J. (2013). Recruiting and retaining learners
in workplace literacy programmes in New Zealand.
conducted in 2007–2009, came from 18 programmes in 15 different Australian Journal of Adult Learning, 53(1), 7–25. They
have not been included in the bibliography as they
workplaces across New Zealand. The evidence in this report has also contain the same material.
generated a number of articles24 that have not been included in this
25. Gray, A. & Sutton, A. (2007, August). The workplace
bibliography. Nor has the report by Gray and Sutton25 as the evidence literacy Fund: A report prepared for the Department
gathered for this report predates the Upskilling work. The key findings of Labour. http://www.dol.govt.nz/PDFs/workplace-
literacy-fund-august-2007.pdf;

Page / 15
 Adult Literacy and Numeracy: An Overview of the Evidence

from the evaluation were that:

• programmes attract those who do not usually attend learning
programmes, including those with low LLN skills, low qualifications,
men, Māori and Pasifika;
• learners attended, on average, 35 hours of learning and there were
Programmes attract small to medium statistically significant gains in their reading and
those who do not writing scores. However, as there was no correlation between the
hours they attended tuition and gain, the writers of the report
usually attend learning concluded that there was no conclusive evidence that scores had
programmes, including improved;
• most participants thought that their literacy skills had improved,
those with low LLN skills, but there was no connection between self-reported improvement
low qualifications, men, and actual scores;

Māori and Pasifika. • workplace practices improved, but there was no connection
between improvements in literacy scores and workplace practices;
• supervisors reported that around 60 percent of the learners had
improved in workplace practices such as teamwork, attitude,
initiative, ability to work unsupervised, willingness to try new tasks,
and completion of paper work;
• the contribution of LLN to improving productivity happens in small
ways; and
• 15 of the 18 programmes continued work in some form after the
initial programmes had been completed.
This report provides some evidence that programmes of up to 40
hours can make a difference for some workers. It also highlights
that practices can change without learners necessarily making a
proficiency gain in reading and writing scores.
Sligo, F., Tilley, E., Murray, N., Watson, B., Comrie, M. & Vaccarino, F.
(2010). Modern apprentices literacy learning: A formative evaluation.
Wellington: Department of Labour.26

Keywords: literacy, workplace learning, apprentices

The evidence for this evaluation came from a literature review, data
on almost 200 apprentices who were receiving additional literacy and
numeracy support to help them complete apprenticeships, and case
studies in 14 workplaces. The evaluation concluded that there were
better outcomes for the apprentices when:
• there was a three-way collaboration between the literacy tutor, the
modern apprenticeship coordinator and the employer. Together
they provided a network of support for the apprentice;
• there was one-to-one tutoring in a non-classroom approach; and
• the literacy tuition was relevant to the context of the vocational
learning.
26. http://www.dol.govt.nz/publications/research/modern-
apprentices/modern-apprentices.pdf

Page / 16
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

In addition to these factors was the need for the workplace to have
a culture of learning including being positive about literacy learning.
There needs to be supportive employers who understand the role
of literacy in the workplace and the potential it has to contribute to
productivity, and who provide time for apprentices to do some of their
bookwork on the job.
The support of
Heathrose Research. (2011). An evaluation of the learning
representatives programme. Wellington: Ako Aotearoa.27 managers and
Keywords: literacy, numeracy, workplaces, union learning
employers was highly
valued by the learners.
This report discusses the findings from the evaluation of the
TEC-funded Learning Representatives programme delivered by
the New Zealand Council of Trade Unions (NZCTU). Evidence was
gathered from documentation and case studies in seven workplaces.
The findings show that:
• the support of managers and employers was highly valued by
the learners;
• there were changes in workers’ attitudes to learning, particularly
those who had previously had a negative attitude and were
encouraged back into learning;
• there was reported evidence LLN gain and evidence of faster
completion of qualifications (although a number of factors may
have contributed to this);
• it took time for the Learning Reps to get up to speed with their
roles; and
• there was an expectation that employers would come on board
and support the work, but this was not always the case in spite of
the NZCTU putting considerable resource into this.

Information in this report could be useful for informing future work

around peer support or mentoring in the workplace that supports
low skilled workers to enrol and maintain their participation in training
programmes.
Australian Industry Group. (2012). When words fail: National
workforce literacy project final project report. Author.28

Keywords: literacy, numeracy, workplace programmes, return on investment

This report sets out the findings of the National Workplace Literacy
Project that started in 2009. It includes evidence gathered through
consultation with employers, workplace trials and evaluation of these
trial sites. The main findings include: 27. http://learningreps.org.nz/files/Evaluation%20of%20
the%20Learning%20Representatives%20Programme.pdf
• 75 percent of employers reported that their businesses were
affected by low levels of LLN; 28. http://www.aigroup.com.au/portal/binary/
com.epicentric.contentmanagement.servlet.
• employers thought that addressing the issue of low LLN skills ContentDeliveryServlet/LIVE_CONTENT/Publications/
Reports/2012/10870_national_workforce_literacy_
needs to be shared between government, individuals, education project_final_report_web.pdf

Page / 17
 Adult Literacy and Numeracy: An Overview of the Evidence

authorities and employers;

• employers do not have the capacity to address the issue (but a
survey undertaken as part of the study indicated that it is possible
to shift employers’ perception about this);
• there needs to be a diversity of training approaches;
Return on investment
• return on investment provided a focus for employers and enabled
provided a focus them to give consideration to developing indicators related to
for employers and productivity, quality, compliance, safety, and human resources;
• employers were interested in return on investment measures, but
enabled them to there was no return on investment tool available to measure such a
give consideration to return;

developing indicators • measuring LLN proficiency gain on its own is not enough –
consideration also needs to be given to other measures of success
related to productivity, such as improvements in confidence, teamwork and employee
quality, compliance, initiative and problem solving;
• the involvement of managers was critical, along with the need to
safety, and human work with and train supervisors and leading hands;
resources. • the need for an increased awareness of the issues and the LLN
demands in workplace documentation and communication; and
• an increased commitment from employers to budget for further
training and apply for further funding assistance.

As a result of this report, the Australian Council of Educational

Research (ACER) is developing Return on Investment instruments to
document the productivity benefits arising from employer provision
of LLN training for employees, and the costs involved in such training.
The instruments will be available mid 201429.
Evans, K., Waite, E. & Kersh, N. (2012). The challenge of establishing
sustainable workplace ‘Skills for Life’ provision in the UK:
organizational ‘strategies’ and individual ‘tactics’. Journal of
Education and Work. DOI:10.1080/13639080.2012.742180

Keywords: literacy, workplace programmes, sustainability

This paper takes evidence from two substantial bodies of work

that looked at the impact of the Skills for Life (SfL) programmes in
workplaces in the England.
• Project One: Adult Basic Skills and Workplace Learning project: a
five-year study (2003–2008) of the impact of workplace ‘Skills for
Life’ provision on organisations and employees (564 learners in 53
organisations from a variety of sectors); and
• Project Two: LLAKES Strand 3 (2009–2011): key factors that facilitate
and inhibit sustainable workplace SfL provision, spatial dimensions
of SfL workplace provision, learner biographies/transitions.
29. http://www.acer.edu.au/vawe/vawe-news/return-on-
investment-in-language-literacy-and-numeracy-training

Page / 18
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

The paper discusses the idea that learning at work is not a priority for
employers who are more like to be concerned with the market and
competition, followed by work organisation and job design. In relation
to the SfL programmes the paper reports that:
• learners engaged in workplace literacy programmes for a range
of reasons in addition to improving the skills for their current job The ‘bottom-up
including: making up for previous lost opportunities in education;
future career aims; helping their children and self-improvement; approach’ used by peers
• learners achieved an increase in confidence; increased their means that workers are
willingness to engage in further learning; and were more able to
help their children with their homework;
more likely to be honest
• it is difficult to sustain provision where there is lack of management
about their needs.
continuity and lack of organisational support at top management
and line management level;
• it has been difficult to sustain programmes during the economic
downturn;
• the organisations have taken a broad approach to learning rather
than promoting it as addressing LLN deficiencies, or just addressing
workplace needs; and
• sustainable programmes require flexibility on the part of the
employer and the provider.

Yasukawa, K., Brown, T. & Black. S. (2012). Workplace literacy and

numeracy learning: An opportunity for trade union renewal in
Australia? International Journal of Training Research. 10(2): 94–104.

Keywords: Workplace literacy and numeracy; social practices; trade union

renewal; vocational education and training

This article used evidence from literature and two small case studies
to inform the authors’ views on workplace literacy and numeracy
programmes. The authors argue for a role for unions to support
learning in the workplaces for the following reasons:
• programmes are usually employer-driven with a narrow focus
on developing functional skills for the workplace (human capital
approach) rather than social capital for learners;
• the ‘bottom-up approach’ used by peers means that workers are
more likely to be honest about their needs; and
• promoting education and training can make unions more relevant
in the workplace
• workers and unions have the opportunity to shape workplace
learning to meet their needs along with those of the employers.

Page / 19
 Adult Literacy and Numeracy: An Overview of the Evidence

Summary comment
While workplace programmes attract learners who might not
otherwise engage in LLN learning, there is limited evidence about
proficiency gain (possibly due to short duration of these programmes).
Rather, there is reported evidence of changes in LLN practices and
workplace practices.

Page / 20
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

Impact of policy investment

Vorhaus, J., Litster, J., Frearson, M. & Johnson, S. (2011). Review of
research and evaluation on improving adult literacy and numeracy
skills. London: Department of Business, Innovation and Skills.

Keywords: adult literacy, numeracy, outcomes However, workers who

This literature review looked at 10 years of research related to use their skills at work
adult literacy and numeracy in relation to the following topics: the and in everyday life
economic, personal and social returns to learning; the quality and
effectiveness of provision; the number of learning hours needed for continue to improve
skills gain; learner persistence; the retention and loss of skills over time;
and the literacy and numeracy skills that are needed.
their skills and are
This was one of the most substantial pieces of research found in the more likely to continue
review. There was strong evidence of: with further education
• benefits of embedding LLN in vocational programmes, including a and training.
higher likelihood of retention and completion;
• the positive impact on learners of working with qualified teachers –
the more qualified the teacher the greater the learner progress;
• positive personal and social returns on improving LLN skills; and
• the need for multiple ways of engaging in learning – in class, self-
study, distance learning, ICT supported learning.

There was growing evidence that improving LLN skills has a positive
effect on earning and employment. There was limited evidence about
the ability of short programmes in workplaces and their ability to
increase literacy skills in the short term. However, workers who use
their skills at work and in everyday life continue to improve their skills
and are more likely to continue with further education and training.
The review also found promising evidence in relation to:
• blended learning – combining face-to-face and technology-based,
formal and self-study methods;
• the significance of techno-mathematical literacies – a combination
of ICT, literacy and numeracy skills; and
• the time required to make significant learning progress – often in
excess of 100 hours30. (p.11).
• There was little evidence found on:
• the impact of LLN on productivity; and 30. The report states that, “Better gains for learners seem
to be associated with courses that allow for levels of
• the cost effectiveness of LLN programmes. participation in excess of 100 hours; learners require
more time to make educationally significant progress
than they generally spend in provision. For those who
Reder, S. & Bynner, J. (Eds). 2009. Tracking adult literacy and only need to ‘brush up’ existing skills, short courses are
often adequate for learner gains; in contrast, learners
numeracy skills: Findings from longitudinal research. New York: working at a higher level may find it more difficult
Routledge. to achieve a qualification within the learning hours
allocated to a single-year course. There is some limited
evidence to show that learners who engage in self-study
between classes make better progress,” (p. 11).

Page / 21
 Adult Literacy and Numeracy: An Overview of the Evidence

Keywords: literacy, numeracy, ESL, workplace, longitudinal studies,

programme outcomes

This book includes chapters by researchers in the UK and the US on

the findings of longitudinal research into the outcomes for adults
in adult LLN programmes. Given the methodological rigour of the
LLN competencies studies these findings can be considered robust. The key points of
continue to develop interest include:

after people leave From Bynner and Parsons

• LLN competencies continue to develop after people leave school;
school.
• There are short-term proficiency gains in LLN and changes in LLN
practices; in the short-term there is a correlation between levels of
proficiency and levels of engagement with practices;
• Over time, participation in LLN programmes does not affect the
development of proficiency31, but it does affect engagement in
practices; and
• Measures of practices rather than proficiency may be a better way
to assess programme impact.

From Alamprese
• Adults with low-level skills make progress in decoding before
increasing their skills in other reading components, so the use of
general literacy tools may not be adequate for measuring the gains
of these learners;
• Pre-test scores are the biggest predictor of learner gain;
• Pre-programme assessment is essential for placing learners in the
right programmes; and
• Pre- and post-test measures are important for reporting learner gain
accurately.
• From Condelli, Wrigley, Yoon (on ESOL learners)
• Comprehension levels changed little, but oral language scores
improved considerably
• Younger learners made more progress than older learners as did
those with higher qualifications, although this advantage faded
over time;
• Learning in real world contexts had a positive effect on growth of
reading skills;
• Students who attended class more regularly made greater gain; and
• The use of learners’ native language to give directions or clarify
concepts improved the rate at which comprehension grew.

From Comings
31. Note, proficiency change is related to a person’s age. • From a number of studies, the conclusion is that 100 hours of
Proficiency reduces with age.

Page / 22
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

instruction or more over a year is the point at which the majority

of adult learners are likely to show measurable progress [note,
‘measurable progress’ relates to improving one grade level, or in the
US pass the General Education Development (GED)].

From Vorhaus, Howard, Brooks, Bathmaker, Appleby

Literacy and numeracy
• In SfL programmes, there were small gains in numeracy and reading
for both literacy and ESOL learners, and writing for ESOL learners,
alone rarely deliver the
but not literacy learners skills needed for the
• Provision worked equally well for all groups of learners. workplace, but without
From Metcalf and Meadows literacy and numeracy
• The study looked at the impact of attending SfL programmes rather skills people are unable
than the connection between the LLN skills gained and wider social
and economic outcomes. Findings were that: to do the kind of
• Learners felt more confident mainstream education
• It took three years for employment benefits to emerge and training courses
• There was no increase in earnings, although some learners had which do enhance
increased the scope of work they were able to undertake
employability.
• Learners were more likely than the comparison group to think that
their LLN skills had improved (p. 240).
• Those with learning disabilities or dyslexia made slower progress
• Learners were more committed than the comparison group
to further education and training (the authors note that this is
an important finding as it “provides evidence to support the
hypothesis that the main value of improving literacy and numeracy
skills for adults is that it opens the way to further learning
opportunities, some of which might help to develop skills which are
directly relevant to work” (p.239)
• The authors’ final point is that: “Literacy and numeracy alone
rarely deliver the skills needed for the workplace, but without
literacy and numeracy skills people are unable to do the kind of
mainstream education and training courses which do enhance
employability” (p. 240).

From Evans, Waite, Admasachew (Workplace)

• Workers attend programmes for personal and work reasons, most of
them had very little post-school learning;
• Learning in the workplace is accessible, familiar and convenient for
workers;
• 40 percent of workers reported changes in the way they did their jobs;
• Workers’ confidence inside and outside of work improved; and
• Workers had an improved attitude towards learning.

Page / 23
 Adult Literacy and Numeracy: An Overview of the Evidence

From Rose
• 40 percent of students needed over 80 hours of instruction before
they could pass their GED.

This is a very useful reference on the impact of literacy and numeracy

Using practice measures programmes and collectively provides evidence for a rationale for LLN
programmes in tertiary and workplace settings. It also highlights the
would “broaden complexity of longitudinal studies in terms of tracking participants
and enrich” policy and developing reliable tools that are able measure change over time.

makers’ and funders’ Reder, S. (2009). Scaling Up and Moving In: Connecting social
practices views to policies and programs in adult education.
perspectives and adult Literacy and numeracy studies. Vol 16/17. Issue 2/1. Pp35–50.
literacy and numeracy Keywords: literacy, numeracy, proficiency, practices
development, and have Evidence for this article is taken from a longitudinal study of 940
the potential to improve adults who did not complete school qualifications in Portland
Oregon. Participants were interviewed six times over the eight year
programmes. period 1998–2006. This is seminal study in that it gathers data over a
reasonable time period and the data has been subjected to rigorous
analysis. The key points of interest in this article relate to the rationale
for measuring literacy practices as well as proficiency and include:
• the need for funders to collect proficiency data to justify the
investment;
• adults’ proficiency growth varies over their lifespan depending
on personal and economic circumstances, for example, gaining
employment and increased earnings are associated with increased
proficiency;
• there was not a clear relationships between proficiency and
participation in adult basic skills programmes;
• there was a strong relationship between programme participation
and higher levels of engagement in practices, from which Reder
concludes that programmes influence practices;
• more frequent reading and writing over time leads to greater
literacy proficiency, but the same was not seen for numeracy;
• the more proficient learners are in numeracy to begin with, the
more likely they are to engage in numeracy practices;
• using practice measures would “broaden and enrich” policy
makers’ and funders’ perspectives and adult literacy and numeracy
development, and have the potential to improve programmes;
• programmes that use authentic materials have higher levels of
engagement in literacy practices and this is maintained by learners
after they leave programmes – there is thus a causal chain as
programmes lead to engagement in practices, which subsequently
leads to increased proficiency; and
• the need to develop practice-based measures of programmes

Page / 24
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

that are not “narrow and reductionist” and are measures that are
important for adults, policy makers and funders.

There is useful information in this study for informing the

development of a literacy practices tool. The information might also
be useful for informing discussions around expectations of literacy
gain and proficiency, along with connections that need to be made
Short workplace
between programmes, proficiency, and practices. courses do not lead
Dorgan, J. (2009) Adult Literacy Policy: A Review for the National Adult to significant gains in
Literacy Agency. Dublin: National Adult Literacy Agency.32
literacy, although they
Keywords: policy, literacy, numeracy, gain, embedding
do improve workers’
This report summarises and discusses research on adult literacy confidence and
policies and initiatives in Australia, Canada, Finland, New Zealand,
Scotland, Sweden, the United Kingdom (England), and the USA. Its social skills. The most
purpose was to inform LLN policy in Ireland. The key points made in
the report include:
successful courses are
• countries that have not maintained policy momentum have
those that have tuition
stagnated, for example, Australia and Canada, although the latter of 100 hours or more.
has undertaken short term initiatives;
• short workplace courses do not lead to significant gains in literacy,
although they do improve workers’ confidence and social skills. The
most successful courses are those that have tuition of 100 hours or
more;
• there is limited evidence in relation to short-term employment
or earning gains, but LLN learning does lead to improved
employability skills, improved health, increased social capital and
greater civic engagement;
• there is value in embedding (note, the evidence for this comes from
research already cited in this bibliography); and
• change takes time, and there is need for ongoing research and
evaluation to inform policy.

National Research and Development Centre. (2011). A Literature

Review of International Adult Literacy Policies. Dublin: National Adult
Literacy Agency.33

Keywords: literacy, numeracy, embedding, workplace learning,

international policies

This research was undertaken by the NRDC for NALA in order to

find out what could be useful to inform Irish literacy and numeracy
32. http://www.nala.ie/publications/adult-literacy-policy-
policy34. It reviewed the policies of Australia, Canada, Finland, New review-national-adult-literacy-agency
Zealand, Scotland, Sweden, United Kingdom, and the USA. It is a 33. http://www.nala.ie/sites/default/files/publications/A%20
substantive piece and the full report is a ‘go-to’ piece for information Literature%20Review%20of%20International%20
Adult%20Literacy%20Policies%20110311_1.pdf
about LLN policies. It includes material similar to Dorgan’s work cited
34. Note, this is very similar to the Dorgan piece cited above.
above.

Page / 25
 Adult Literacy and Numeracy: An Overview of the Evidence

The key points made in the research include:

• countries that have not maintained policy and programme
momentum have stagnated, for example, Australia35 and Canada.
Canada is of particular interest as it has a strong research
programme, which has not been followed by strong policy and a
Family literacy work programme;
programmes are • courses of short duration in workplaces and other settings lead to
increased confidence and social engagement, but there was little
important for the quantifiable evidence of literacy gain (note, this research also cites
development of parents’ the USA research, which finds that 100 hours of teaching are the
most successful in producing gain);
human and social • there was limited evidence either for or against LLN programmes
capital and for engaging producing meaningful employment of increases in earnings;

parents in their children’s • there was clear evidence that programmes did lead to “improved
employability skills, improved health, increased social capital and
learning. greater civic engagement” (p. 5);
• countries have taken different approaches, those that take the
human capital approach with an emphasis on economic gains and
those that take a social capital with an emphasis on social, health
and citizenship benefits;
• the success of the embedded approach in the UK;
• the work from the US showed it takes time to implement policies
see outcomes and refine them;
• the need to prioritise quality over quantity of provision; and
• family literacy programmes are important for the development of
parents’ human and social capital and for engaging parents in their
children’s learning.

While these are the key findings the researchers also commented
on the challenges of their work because of the type, availability and
quality of the research and evaluation available for the review. These
challenges included:
• lack of detailed information of descriptions of implementation
procedures (note, while these may exist they were not able to
locate them through search procedures);
• research that was along theoretical lines rather than focusing on
what happens during implementation;
35. Note, since this research was undertaken Australia
has developed a National Foundation Skills Strategy • where there were descriptions of implementation these tended to
for Adults, (2012), http://www.scotese.natese.gov.au, be in evaluation reports of varying quality;
which has the aspirational target that by 2022 two
thirds of working age Australians will have literacy and
numeracy skills at Level 3 or above as measured through
• there needs to be more quality research on blended and distance
the Program for International Assessment of Adult learning, and what and how LLN programmes and their various
Competencies (PIAAC) Survey. This survey follows the
2006 ALL survey and is being administered for the first components work or do not work;
time in Australia in 2011–12. The results from the survey
will form the baseline for measuring the success of the • the lack of evidence does not suggest evidence of failure, rather the
strategy by 2022.
lack of research; and

Page / 26
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

• the short-term nature of research – it takes several years to fully

realise the impact of LLN programmes on individuals earning and
employment rates. While short-term impact is worth measuring, so
too does the impact of programmes need to be looked at over the
working lives of participants.

New Zealand’s approach is reviewed favourably in the report as

New Zealand’s
being “one of the most active in the world” (p. 35) in relation to the approach is reviewed
development of literacy and numeracy policies (note, this point was
made when the LLN work was part of the wide Skills Strategy). It is favourably in the
useful for affirming the work that has been undertaken in the New report as being “one
Zealand context, and because New Zealand comes across positively
in the international context. However, it also highlights that further of the most active
work needs to be done in order to understand more about LLN
policy and practice.
in the world” (p. 35)
Alkema, A. & Heathrose Research. (2011). Intensive literacy and
in relation to the
numeracy fund stocktake report. Unpublished report. Tertiary development of literacy
Education Commission.
and numeracy policies.
Keywords: literacy, numeracy, hours of learning, outcomes

This research investigated the extent to which the new policy settings
for the Intensive Literacy and Numeracy (ILN) Fund were working in
terms of reaching learners with high LLN needs and the outcomes
for them of 100 hours of learning. Evidence was gathered through 44
providers’ reports to the TEC from 2010, interviews with 27 providers
and 10 community stakeholders, and analysis of Assessment Tool data.
The key findings in the report include:
• ESOL learners, Māori learners, Pasifika learners, those seeking to
pass pre-employment tests and recent school leavers are the most
common learners targeted for recruitment into ILN programmes;
• retaining learners is challenging as there are barriers to their
continued participation and retention;
• providers reported that on entry to the ILN programme the learners
have low reading, writing and numeracy skills (Starting Points to
Steps 1–3 of the Learning Progressions) and make progress of
some kind, either within a Step or by one Step of the Learning
Progressions, although providers noted that it was difficult for those
at the very low level, including ESOL learners to make progress in
100 hours; and
• where the Assessment Tool was used36 it showed average entry
level for numeracy at Steps 3/4 and for literacy, Step 2. Progress
assessments for the small number who had them showed there
had been gain for many learners.

Seventeen of the 26 providers thought that 100 hours was not

enough time for all of their learners to make sufficient or sustainable
progress. Eight providers were comfortable with the 100 hours for 36. The stock-take used data from 2010 when the
Assessment Tool was not compulsory.

Page / 27
 Adult Literacy and Numeracy: An Overview of the Evidence

some or all of their learners although these providers did not have
ESOL learners as their primary target group.
EU High Level Group of Experts on Literacy. (2012). Final report.37

Keywords: literacy, numeracy, adult education

Perhaps the most
This report covers the findings from the policies of a number of
important point to European countries. It concludes that in order to improve LLN for
adults the following needs to happen:
note is the time it takes
• courses need to be of high quality and based on individual learning
for proficiency gain needs. Learners prefer small classes and when motivated are more
and that shorter-term likely to persist;

programmes are more • learners need to be taught by committed and well-qualified

teachers;
likely to impact on • learning needs to be in real-life contexts and embedding has
practices, which in turn, improved qualification rates in vocational training

over time, improves • courses must be of the right intensity and duration (note, this report
also cites the 100 hours required for progress from the US work.);
proficiency. • longitudinal studies are required in order to assess the full impact of
programmes;
• literacy needs to be promoted in workplaces to get to ‘hard-to-
reach’ learners;
• learners can be attracted to study through the workplace and free
programmes; and
• ICT programmes have the potential to attract older workers who
have low LLN skills and this has the double benefit of improving
digital literacy and LLN.

Summary comment
Collectively this research provides evidence that there are benefits to
individuals who undertake LLN learning. Perhaps the most important
point to note is the time it takes for proficiency gain and that shorter-
term programmes are more likely to impact on practices, which in
turn, over time, improves proficiency.

Assessment Tool
Introduced to the sector in 2010, the Literacy and Numeracy for Adult
Assessment Tool (the Tool) has been a funding requirement for level
1–3 courses since 2011. Given the recentness of the introduction of
the Assessment Tool, there is little research evidence about its use and
impact. The three works cited here provide some early evidence about
its use and the proficiency gain it is showing.
37. http://ec.europa.eu/education/literacy/what-eu/high- Haggland, J. & Earle, D. (2013). Engagement is key. Wellington:
level-group/documents/literacy-report.pdf
Ministry of Education.38
38. http://www.educationcounts.govt.nz/publications/
numeracy/engagement-is-key

Page / 28
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

Keywords: adult learning, literacy, numeracy, assessment

This was a small qualitative study that included a survey of 35 tertiary

education providers who deliver programmes funded through Youth
Guarantee. The survey was followed up with interviews with nine of
the providers. The key findings from the research include:
The need for the Tool to
• the need for the Tool to be seen as part of the wider organisational
literacy and numeracy programme; be seen as part of the
• that it is about more than just setting learners up to ‘take the test’; wider organisational
• educators need to be able to understand assessment for learning literacy and numeracy
and be able to use information for diagnostic purposes;
programme.
• learners need to be engaged in the learning process and
understand why the assessment matters;
• organisations need to have the resources to support educators to
use the Tool;
That it is about more
• it was a struggle to engage the learners in the process, especially a than just setting
progress assessment; and learners up to ‘take
• learners did not find the items particularly engaging in that they did
not relate to their real life experiences.
the test.

The findings from this report support anecdotal evidence that the TEC
has been hearing since the implementation of the Tool. The evidence Organisations need to
here has supported the development of a bank of items designed
especially for youth.
have the resources to
Lane, C. (2013). Assessing skills of adult learners in 2011. Wellington: support educators
Ministry of Education.39 to use the Tool.
Key words: literacy, numeracy, assessment

This research reviews the data gathered from the Tool in 2011. It
analyses data from 77,000 learners who undertook over 200,000
individual assessments. The key findings from the research include:
• the majority of assessments are in reading and numeracy;
• there were learners assessed at each qualification level, but the
majority of them were in levels 1–4;
• learners in Youth Guarantee programmes were the most assessed
group;
-- the rate of assessment of learners in Student Achievement
Component (SAC)40 and Youth Guarantee funded programmes
was higher in ITPs (36 percent) than in PTEs (20 percent) and 39. http://www.educationcounts.govt.nz/publications/
numeracy/assessing-skills-of-adult-learners-2011
wānanga (15 percent);
40. The SAC is the Government’s contribution to the direct
-- learners were assessed lower in reading than in numeracy41, “with costs of teaching, learning, and other costs driven by
approximately half of learners assessed for reading scoring in the learner numbers.

top three steps (Learning Progressions Steps 4, 5 and 6), while for 41. Note, this is caused by the differences in the Steps in the
Learning Progressions rather than in the learners’ skills in
general numeracy approximately half of learners scored in the reading and numeracy.

Page / 29
 Adult Literacy and Numeracy: An Overview of the Evidence

top two steps (Steps 5 and 6)”;

• while the learners were not representative of the population, their
patterns of achievement are the same as the adult population in
the ALL Survey 2006;
• more than half of those who scored at Step 1 in their first
More than half of those assessment showed statistically significant gain; and
who scored at Step 1 in • those who scored highest on their first assessment showed the
their first assessment lowest rates of statistically significant gain.

showed statistically The information in this report provides some useful insights into
significant gain. Assessment Tool use in the first year of implementation. It can sit
alongside the data that the TEC continues to gather from the Tool and
will provide a useful comparator over time.
Eagle-Ashmore, P. (2012). Accountability and control in adult
Information from this vocational (foundation) education: Is the adult literacy and
research is useful for numeracy assessment tool a reliable indicator of literacy and
numeracy progress? Journal of Adult Learning Aotearoa New
framing discussions Zealand. Vol. 40 Issue 1, 92–99.
around proficiency gain Keywords: literacy, numeracy, assessment
and for showing that
This article captures the anecdotal evidence that has been discussed
more needs to be known in New Zealand since the introduction of the Tool. The research
about what to expect in methodology is unclear and it appears to be more of an opinion
piece than research based. It describes issues associated with the
relation to proficiency implementation of the Assessment Tool. These issues include:

gain over time as • the negative impact on “vulnerable” learners of having to take an
“examination” in their first week of learning;
measured against the • the length of time it takes for LLN gain (as opposed to the short
Learning Progressions. time frames of programmes where it might not be possible to
measure gain);
• assessment items about topics that might not have been covered
in learning programmes;
• the negative approach by learners to progress assessments;
• the tension between the Tool being used for both formative and
summative assessments; and
• the concern that if it is eventually linked to funding, then teachers
will narrow what is taught and teach to the test.

Summary comment
Information from this research is useful for framing discussions around
proficiency gain and for showing that more needs to be known about
what to expect in relation to proficiency gain over time as measured
against the Learning Progressions. This is emerging work and will
be watched with interest by the sector. It would be very useful to

Page / 30
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

have Assessment Tool evidence available when the Programme for

the International Assessment of Adult Competencies (PIAAC) survey
results come out in 2016 so that any gains in proficiency as measured
by the Assessment Tool can be compared to national survey findings.

Priority groups
Youth who were
Whatman, J., Schagen, S., Vaughan, K., Lander, J., Twist, J., Brooking,
K., Robertson, S. & Spiller, L. (2010). Engaging young people/ apparently
young adults in literacy, language and numeracy skill development.
Wellington: Department of Labour.42
unmotivated at school
can be motivated in
Keywords: literacy, numeracy, youth, motivation, engagement
vocational programmes
This research takes evidence from a literature review, key informant
interviews and case studies as a means to identify LLN programmes
that embed LLN.
that engage youth and improve outcomes for them. A key idea
presented in the report is the difficulty of motivating youth who
are more likely to be motivated: extrinsically, for example, by
employment preparation or learning in work-related contexts; by a
range of approaches to teaching; by the inclusion of mentoring and/
or counselling; and by the ability to access LLN programmes.The key
findings from the report include:
• there was little research evidence about where LLN fits into learning
programmes that seek to engage youth, including those Not in
Education, Employment or Training (NEET);
• it is difficult to measure, in the short term, the impact that LLN gains
have on employment;
• along with LLN, youth reported that they were learning a wide
range of employment skills including motivation, persistence,
decision-making, problem solving, responsibility, self-confidence,
and cooperation with peers. At the same time they were gaining
unit standards;
• youth who were apparently unmotivated at school can be
motivated in vocational programmes that embed LLN;
• NEET youth who were compelled to attend pre-employment
programmes because of the social welfare benefits they were
drawing, appeared motivated and reported enjoying learning and
that they were making progress;
• youth need to be respected and appreciated and develop a sense
of belonging created by positive tutor-learner relationships in a
warm and welcoming organisation
• it is important for youth to have easy access to programmes; and
• retaining youth in programmes continues to be problematic
because of the multiple demands on their lives.

The research developed success indicators of LLN programmes for 42. http://www.dol.govt.nz/publications/research/youth-
youth. These can be accessed on page four of the report. This research literacy-report/youth-literacy-report.pdf

Page / 31
 Adult Literacy and Numeracy: An Overview of the Evidence

is useful for the TEC now and in the future, particularly in relation to
the work underway in Youth Guarantee, future work on NEETs and the
development of the youth items for the Assessment Tool. The table of
success indicators is particularly useful.
The model was then Hutchings, J., Yates, B., Isaacs, P., Whatman, J., and Bright, N. (2012).
trialled with 53 learners Hei Ara Ako ki te Oranga: A model for measuring wellbeing outcomes
from literacy programmes. Wellington: Ako Aotearoa.43
in five Literacy Aotearoa
This research project developed and trialled a Kaupapa Māori
poupou. The approach assessment model that links adult literacy learning and well-being
that underpins the for Māori learners. The conceptual model was developed through an
iterative process of focus groups with learners, tutors and managers.
model assessment This was followed by two wānanga with experts. The model was then
trialled with 53 learners in five Literacy Aotearoa poupou.
model is that “Maori
The approach that underpins the model assessment model is that
adult learners will be “Maori adult learners will be able to see their literacy learning goals
able to see their literacy reflected as well-being outcomes in literacy teaching programmes
and in assessments of their learning” (p. 2). The formative and
learning goals reflected summative assessments conducted using the model sit alongside the
as well-being outcomes literacy and numeracy assessments conducted through the Literacy
and Numeracy for Adults Assessment Tool.
in literacy teaching The trial of the assessment model showed that:
programmes.” and in • the questions and prompts were useful for tutors and learners;
assessments of their • it was beneficial in building a relationship between the tutor and
the learner;
learning. (p. 2)
• it provided learners with the opportunity to reflect on their learning
and progress;
It was beneficial in • the use of whakataukī grounded participants in Māori ways of
building a relationship working; and
• the tutors thought it was a positive process, but it took time to be
between the tutor and properly conducted and they required professional development in
the learner. order to learn how to use it.

Potter, H., Taupo, K., Hutchings, J., Mcdowall, S. & Isaacs, P. (2011).
It provided learners He whānau mātau, he whānau ora: Māori adult literacy and whānau
transformation. Wellington: NZCER.44
with the opportunity to This research aimed to find out about the impact on whānau when
reflect on their learning parents and caregivers participate in programmes that develop their
literacy and numeracy skills in English. It was conducted through
and progress. interviews with four tutors/managers, 23 Māori learners and seven
whānau members. The research found that:
• programmes that engage Māori learners take place in culturally
safe, small group environments with tutors who care about the
43. https://akoaotearoa.ac.nz/download/ng/file/
group-5327/the-kaupapa-maori-wellbeing-assessment- success of learners;
model--hei-ara-ako-ki-te-oranga.pdf
• learners were motivated because they wanted to be able to
44. http://www.nzcer.org.nz/system/files/he-whanau-
matau-he-whanau-ora.pdf contribute positively to their whānau and support their tamariki

Page / 32
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

and mokopuna;
• as a result of the programme learners felt better able to participate
in the education of their tamariki and mokopuna, who in turn found
more enjoyment in learning and schooling;
• learners made a greater contribution to whānau activities; and
Māori learners can
• whānau relationships were strengthened as a result of the learners’
improved communication skills. overcome previous
negative experiences
May, S. (2009). Hangaia te mātāpuna o te mōhio: Learning
foundations for Māori adults. Wellington: Ministry of Education.45 at school when they
Keywords: Māori, adult learners
experience educational
achievement and
This report summarises three research projects46 that explore how
success for Māori adults in the learning foundations of literacy, success.
language and numeracy can be built on the foundations of Māori
culture and identity. While this summary report was based on three
reports on literacy and numeracy learning for Māori learners, it does
not specifically address literacy and numeracy per se, rather it deals
more generally on conditions that create success for Māori learners.
The evidence from the reports shows that:
• Māori learners can overcome previous negative experiences
at school when they experience educational achievement and
success;
• Māori learners often have significant social and economic barriers to
overcome to access learning, alongside wider family commitments;
• Māori are not a homogeneous group and bring diverse experiences
with them;
• there needs to be a holistic approach to teaching and learning
programmes – an approach that acknowledges whānau and is
about more than just the individual;
-- overcoming negative experiences is helped by courses based on
Māori tikanga and pedagogies such as whakawhanaungatanga
and tuakana/teina;
-- Māori values and protocols, such as those listed below, are also
highly valued by students and tutors and seen as central to the
success of programmes:
45. http://www.educationcounts.govt.nz/__data/assets/
◦◦ Te noho ā marae – marae kinship pdf_file/0007/55717/09-07-02-Learning-Foundations-
for-Maori-Adults2.pdf
◦◦ Te hononga ā-iwi – shared iwi links 46. White, H., Oxenham, T., Tahana, M., Williams, K. &
Matthews, K. (2009). Mā te huruhuru, ka rere te manu:
◦◦ Te noho hei whānau – deliberate act of teamwork how can language and literacy be optimised for
Māori adult learning success? Wellington: Ministry of
◦◦ Te noho rūmaki – protocols and customs Education; McMurchy-Pilkington, C. (2009). Te pakeke
hei ākonga: Māori adult learners. Wellington: Ministry
◦◦ Kanohi ki te kanohi – face to face (implies frankness) of Education; Mlcek, S., Timutimu, N., Mika, C., Aranga,
M., Taipeti, N., Rangihau, R., Temara, M., Shepherd, Y. &

◦◦ Te manaakitanga – fostering relationships

McGarvey, H. (2009). Te piko o te māhuri, tērā te tupu
o te rākau : language and literacy in marae-based
programmes. Wellington: Ministry of Education.

Page / 33
 Adult Literacy and Numeracy: An Overview of the Evidence

◦◦ Te tuwheratanga – openness
◦◦ Te whakapono – trust. (p.7)
• deliberate acts of teaching and tracking language acquisition
improve results
Marae-based learning • strong relationships between the students and the tutor that
incorporates quality teaching;
can be particularly • insufficient numbers of teaching staff qualified to teach Māori
beneficial for Māori students and the need for more professional development that
support tutors to teach in a culturally appropriate way; and
learners as marae
• marae-based learning can be particularly beneficial for Māori
“equals identity” and learners as marae “equals identity” and provides “accessible, safe and
provides “accessible, culturally congruent learning context” (p 11).

safe and culturally Tuagalu, C. (2012). Engaging pacific migrant workers in literacy,
congruent learning numeracy & financial literacy training: Reflections on the research
process. Wellington: Ako Aotearoa.47
context. (p 11)
Keywords: workplace learning, literacy, numeracy, Pasifika

The project drew evidence from a literature review and interviews

with learners, their tutors and employers involved with a twenty-hour
workplace literacy, numeracy and financial education programme for
Pacific workers in New Zealand as part of the Recognised Seasonal
Employer Scheme. It can be considered an action research project as it
was conducted by a tutor on the programme48.
The factors that contributed to success for Pasifika included:
• the creation of a Pacific place of learning through artefacts, for
example, tapa cloth, video and audio clips; the use prayer and song
at the beginning and end of lessons; the opportunity for learners to
tell their stories;
• the learners were provided with the opportunity to co-construct
the curriculum and given the opportunity to practise their learning;
• the opportunity to practise was more beneficial for the learners
when it was in a real life setting;
• tutors who made the most difference were those described as
“respectful, caring and understanding” (p.32); and
• the use of Pasifika languages and translation to and from English
supported the learners.

47. https://akoaotearoa.ac.nz/ako-hub/ako-aotearoa-
northern-hub/resources/pages/engaging-pacific-
Summary comment
migrant-workers
There was surprisingly little research available on LLN and the priority
48. A full evaluation of the programme was conducted by groups. More needs be known about the LLN practices that engage
Mathea Roorda and can be found on https://www.aid.
govt.nz/webfm_send/17. It has not been used in this the learners in these groups and the impact of LLN programmes
bibliography as it assesses the overall programme rather
than looking just at the success factors for Pasifika that
on them. However this research is useful for the TEC, particularly in
relate specifically to literacy and numeracy. relation to how to assess wider outcomes that occur for learners and

Page / 34
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

their families through literacy programmes. There is a strong base

of research already available on what leads to success for Māori and
Pasifika learners, and this can to be built on in relation to LLN.

Using ICT to enhance literacy and

numeracy skills Blended learning is
Salomon, M. (2009). Workplace literacy and essential skills: What more likely to meet the
works? and why? (2009). The Centre for Literacy of Quebec.49
needs and schedules of
Keywords: workplace learning, literacy, e-learning, blended learning
workplace learners and
This information is taken from a wider literature review on workplace
literacy programmes that has not been included in this bibliography
increases the likelihood
as the findings are similar to the other research reported here. of recruiting and
However the information on blended and e-learning is worth
including. The key points related to this in the context of workplace
retaining workers.
learning include:
• LLN programmes in the workplace benefit from blended learning,
that is, the use of multiple teaching methods, including e-learning;
• blended learning is more likely to meet the needs and schedules
of workplace learners and increases the likelihood of recruiting and
retaining workers;
• a UK project showed that learners in basic skills programmes
responded positively to e-learning (either in the classroom or by
distance). However older men were resistant to learning this way
and other learners were limited by their degree of familiarity with
computers and the internet;
• the UK study also showed that e-learning was not right for all
learners; and
• a US study noted that not all learners have the discipline to benefit
from distance e-learning.

Davis, N., Fletcher, J., Brooker, B., Everatt, J., Gillon, G., Macke,
J. & Morrow, D. (2010). E-learning for adult literacy, language
and numeracy: a review of the literature. Wellington: Ministry of
Education.50

Keywords: literacy, numeracy, embedding, e-learning

This research investigated how e-learning can be used to reach a

greater number of adult learners and better meet their learning needs.
These findings are from a literature review and are supported by a
case study in one New Zealand polytechnic. While the review found
no direct research evidence showing e-learning improves adults’ LLN
skills, it did find the following:
• distance e-learning is a cost effective way to develop the skills of 49. http://www.nald.ca/library/research/cfl/wles/wles.pdf

those assessed at Level 2 in the ALL survey; 50. http://www.educationcounts.govt.nz/publications/

tertiary_education/76971/1.-introduction

Page / 35
 Adult Literacy and Numeracy: An Overview of the Evidence

• e-learning is relevant and useful for most adults with LLN needs,
provided it is designed to fit their proficiency levels in literacy and
digital technologies;
• tutors require professional development in order to develop or
make changes to programmes that include e-learning and embed
Tutors require LLN into e-learning;
professional • e-learning is more effective when it is part of face-to-face training,
that is blended learning approaches are more effective;
development in order • e-learning may well be a way to retain learners in programmes of
to develop or make learning;
changes to programmes • e-learning is more effective where it has an employment focus
rather than a generic LLN focus;
that include e-learning
• acknowledging cultural context is important for Māori and Pasifika
and embed LLN into learners; and
e-learning. • e-learning can provide a good source of motivation and practice for
ESOL learners.

Thomas, G. & Ward, J. (2010). Numeracy for adults: building skills with
online learning links. Wellington: Ministry of Education. 51
This study included a literature review and case studies of five industry
trainees who used online learning activities that supported their
numeracy learning in the workplace. The trainees were assessed and
then directed to activities that would address their specific skills gaps.
In relation to ICT and the development of numeracy skills the study
found that:
• using ICT made learning more engaging and increased the learners’
motivation and persistence;
• learning through ICT outside the classroom provides the
opportunity for learners to tap into support networks, e.g., family,
friends;
• learning through ICT makes learning more accessible as it can be
undertaken at a time that suits learners; and
• the learners in the study made gains of one to two stages in the
strand of the Learning Progressions that was their focus. This gain
was made over a very short time frame
• learners were able to apply their new numeracy skills in their
personal and working lives.

Maths Technology Limited. (2011). Pathways Awarua: Outcomes

for learners in 2011. Tertiary Education Commission. Unpublished
report.
This evaluation gathered evidence from users of Pathways Awarua
51. http://www.educationcounts.govt.nz/publications/
from July to December 2011 when it was first introduced. The data
tertiary_education/numeracy-for-adults-building-skills- was gathered through surveys of 39 educators and 92 learners,
with-online-learning-links/1.-introduction

Page / 36
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

interviews with 12 learners who had made the greatest gain as

assessed through the Assessment Tool and their educators (eight in
total). Achievement data was captured by the Assessment Tool and
completion of modules was captured through the Pathways Awarua
learning files. Most of the learners were using Pathways Awarua once
to twice a week.
Most of the learners
The evaluation showed that:
were using Pathways
• learners were very engaged with Pathways Awarua with most of
them being repeat users; Awarua once to twice
• the accuracy rate of 93 percent showed that most learners repeat a week.
activities until they have all the answers correct;
• the heavy usage at the lower levels suggested that most learners
are choosing to start at the beginning;
• the learners were enthusiastic about Pathways Awarua and
appreciated being able to retry questions until they got the right
answers and learning at their own pace;
• educators appreciated that learners were able to retry questions
and valued that they could track what the learners had tried and
completed;
• most of educators planned to use Pathways Awarua in future
programmes; and
• the majority of educators would like to have been able to select
more contextualised pathways for their learners.

The evaluation also measured the reading and numeracy gain of

learners. However, as these learners were also involved in learning
programmes their gain cannot be attributed solely to the use of
Pathways Awarua.

Summary comment
As with the priority groups there was little research available about
LLN and e-learning. With the increasing uptake of Pathways Awarua
there is the opportunity to find out more about who is using this
resource, how it is being used and the impact on learners.

Other literacies
While the purpose of this bibliography was literacy and numeracy,
publications that comment on wider literacy policies that have
relevance for New Zealand LLN policy are included here.
Balatti, J., Black, S. & Falk, I. (2009). A new social capital paradigm for
adult literacy: Partnerships, policy and pedagogy. Adelaide: National
Council for Vocational Education Research.52

Keywords: literacy, numeracy, health literacy, financial literacy

This report continues the theory that Balatti, Black et al. have in 52. http://www.ncver.edu.au/publications/2170.html

Page / 37
 Adult Literacy and Numeracy: An Overview of the Evidence

relation to literacy as social capital (these publications are not included

in this bibliography). However, the researchers make some important
points for the New Zealand context in relation to financial literacy and
health literary. These include:
• The increasing number of health literacy and financial literacy
The research concludes programmes that are not linked into wider vocational teaching and
are not being taught by providers who have literacy and numeracy
that Essential Skills expertise. Teachers in these programmes do not see their work as
provide the foundation involving the teaching of literacy and numeracy; and

for financial literacy, but • There needs to be greater connectedness between the providers
of financial literacy and health literacy programmes, education
that financial literacy providers and communities.
itself is not an essential Canadian Foundation for Economic Education (n.d.). Essential
skill because of the skills and financial literacy: exploring the correlations, compatibility,
and success factors. Office of Literacy and Essential Skills Human
financial knowledge Resources and Skills Development Canada.53
that goes with it, which Key words: Essential skills, literacy, numeracy, financial skills
is in addition to basic
This study explores whether financial literacy should be added to the
skills. Essential Skills list in Canada. The research was conducted through a
literature review and interviews with subject matter experts.
The study found limited empirical research on the links between
literacy and financial literacy, but did find links between the Essential
Skills of numeracy, thinking skills, reading text, and document use.
The research concludes that Essential Skills provide the foundation
for financial literacy, but that financial literacy itself is not an essential
skill because of the financial knowledge that goes with it, which is in
addition to basic skills.
New Zealand Vocational Education & Training Research Forum.
(2011). Financial literacy as a context for strengthening literacy and
numeracy for adults: Postgraduate research show case.54

Keywords: literacy, numeracy financial literacy

This presentation of post-graduate research shows that learners can

be motivated to learn about financial literacy, which provides the
context for embedding literacy and numeracy.
Bunker, B. (2010). A Summary of International Reports, Research
and Case Studies of Digital Literacy: Including implications for New
Zealand of adopting a globally-recognised digital literacy standard.
53. http://www.cfee.org/en/pdf/CFEE%20Financial%20 New Zealand Computer Society.55
Literacy%20and%20Essential%20Skills%20-%20Final%20
Report%20July%202012.pdf
This literature review outlines the importance of digital literacy in
54. http://www.itf.org.nz/assets/Events-and-Forums/2011-
Events/NZ-VET-Research-Forum-2011/Session-1- terms of enhancing the personal and working lives of individuals and
Remnant.pdf contributing to workplace productivity. The key findings from the
55. http://www.iitp.org.nz/files/201001%20Digital%20
Literacy%20Research%20Report.pdf report include:

Page / 38
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

• the increasing need for all adults to be able to use ICT given the
number of services, including government services, that need to be
accessed online
• improving the digital literacy skills of the workforce has the
potential to bring about productivity gains
• increasing the digital literacy skills of marginalised groups allows Improving the digital
them to participate more fully in society. literacy skills of the
workforce has the
Summary comment
There is increasing interest in New Zealand in health literacy and
potential to bring about
financial literacy. For example, workbase is heavily involved in teaching productivity gains.
health literacy and developing related resources while Literacy
Aotearoa has financial literacy education programmes. There is also
an increasing demand from workplace literacy providers (in the
employer-led space) to include digital literacy and financial literacy
into their programmes. It may well be time to consider ‘policy’ in
relation to where these literacies sit within the TEC’s LLN work stream
and funding structures.

Return on investment
Department of Labour. (n.d.) The impacts of LLN training
programmes on firm performance: A summary of the findings of the
‘Return on Investment’ case studies. Unpublished report.

Keywords: workplace learning, literacy, cost-benefit

This paper summarises the findings of return on investment case

studies conducted in three New Zealand workplaces between
2007–2009. These workplaces ran on-site literacy programmes funded
through the TEC’s Workplace Literacy Fund. The key findings from the
summary include:
• the lack of robust data held by firms that can be used
• the difficulty of isolating the impact of one training programme
from others that are occurring in companies
• the finding that in one firm the reduction in staff turnover was
large enough to make the programme cost effective, but did not
take into account other factors that may have accounted for this
reduction
• the failure of the studies to find conclusive evidence of cost-benefit
does not mean that the programmes have no impact, rather it
points to the difficulty of research of this nature.

The paper also reports that, in spite of considerable interest and

funding of literacy and numeracy programmes internationally, there
was a shortage of research publications on return on investment.
If research of this kind is to be undertaken, the paper concludes

Page / 39
 Adult Literacy and Numeracy: An Overview of the Evidence

that, “In order to have sufficient statistical power to identify the

effect of the training on a given outcome variable with reasonable
confidence (i.e., to have a reasonable chance of obtaining a statistically
significant result), the training intervention may need to be reasonably
substantial, and the number of sites and/or number of time periods
may need to be relatively large” (p. 8).
Dorgan, J. (2009). A cost benefit analysis of adult literacy training.
Dublin: National Adult Literacy Agency.

Keywords: literacy training, economic returns, cost benefit

This study aims to estimate the return on investment for countries

who invest in literacy and numeracy programmes for adults. The
study includes a review of literature focussed on quantifying the
effect of training on literacy standards, then evaluates the economic
impact of higher standards, and looks at social impacts. The author
concludes that there are income gains to individuals of around 30
percent through improving literacy scores by 100 on the IALS scale,
and as a result there are also returns to government in the form of
taxes. There are also wider social gains that are already reported in
this bibliography. As with the return on investment literature research
cited above, the report also notes the difficulty of undertaking
research of this kind given the paucity of data on the outcomes of
literacy programmes.

Summary comment
Return on investment studies are difficult for the reasons outlined
above. It will be worthwhile keeping track of the instruments that are
currently being developed and trialled by the Australian Council for
Educational Research (ACER). These instruments will allow employers
to record the productivity benefits that arise from LLN training, the
cost associated with the training and this will provide them with the
basis for estimating the return on their investment.

Page / 40
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

CONCLUDING COMMENTS

New Zealand has taken a consistent and focussed approach to

LLN policy, practice and funding over the last 10 or more years,
and the research summarised in this bibliography shows that
NZ’s work in this area stacks up well internationally. Based on the
evidence outlined above, New Zealand’s approach should be
effective to improve personal, economic and social outcomes for
learners. However, as it takes time for outcomes to emerge, these
The final challenge
improvements will only be verified through evidence gathered over in the New Zealand
time in the New Zealand context, and there are many complicating
challenges to this evidence-gathering. context relates to
Currently the impact of specific interventions is measured through embedding. The studies
the Assessment Tool, which gives a measure of changes in proficiency.
But, as the evidence shows, this measure on its own is not enough,
in this bibliography
so consideration needs to be given to what literacy and numeracy show that it works, but
practices could be measured, how these would be measured, and
over what timeframe. only with conditions
Another challenge relates to describing what is meant by in place related to
proficiency gain and what can be expected over what time period.
One hundred hours comes through consistently in the literature as
whole-of-organisation
the teaching time being required to show gain. This finding comes approaches and
from a robust longitudinal study. However, it is 100 hours for a
level gain, which is not how proficiency gain is described in New
programmes that are
Zealand’s Learning Progressions. informed by LLN and
The final challenge in the New Zealand context relates to embedding. vocational content.
The studies in this bibliography show that it works, but only with
conditions in place related to whole-of-organisation approaches and
programmes that are informed by LLN and vocational content.
While the evidence describes what works and how this improves
outcomes for learners, more still needs to be known about:
• embedded LLN practices at classroom and organisational level;
• the role of LLN in engaging and retaining priority learners and the
subsequent outcomes for them;
• the role of ICT and how this might be used to reach learners
within and outside programmes. It could be useful to explore how
Pathways Awarua is being used and the impact of its use; and
• the connections between LLN and wider literacies, such as health,
financial and digital literacy, and the place that these have within
the context of literacy and numeracy work in New Zealand.

Overall, the current LLN work in New Zealand can be viewed as one
stage in an on-going project that still requires considerable change
across education and industry sectors and systems. The LLN work over
the years has been, and still is, a major educational change project.
Fullan (2007) has categorised educational change into three phases:
initiation, implementation and institutionalisation. In these terms, New
Zealand is probably still in the implementation phase, with substantial

Page / 41
 Adult Literacy and Numeracy: An Overview of the Evidence

activity underway and focus now being directed towards collecting

evidence of benefits for learners, and pinpointing specific elements
of LLN implementation that optimise those benefits. There is still a
way to go before the many aspects of LLN activity can be described
as business as usual and it is institutionalised in educational provision
and training in New Zealand.

Page / 42
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

REFERENCES

Alkema, A. & Heathrose Research. (2011). Intensive literacy and

numeracy fund stocktake report. Tertiary Education Commission.
Unpublished report.

Australian Industry Group. (2012). When words fail: National

workforce literacy project final project report. Author.

Balatti, J., Black, S. & Falk, I. (2009). A new social capital paradigm for
adult literacy: Partnerships, policy and pedagogy. National Centre for
Vocational Education and Research.

Canadian Foundation for Economic Education (n.d.). Essential

skills and financial literacy: exploring the correlations, compatibility,
and success factors. Office of Literacy and Essential Skills, Human
Resources and Skills Development Canada.

Carpentieri, D (Ed). 2007. Five years on research, development and

changing practice. London: National Research and Development
Centre.

Casey, H., Cara, O., Eldred, J., Grief, S., Hodge, R, Ivanic, R., Jupp, T.,
Lopez, D. and McNeil, B. (2006) ‘You wouldn’t expect a maths teacher
to teach plastering…’: Embedding literacy, language and numeracy
in post-16 vocational programmes – the impact on learning and
achievement. London: National Research and Development Centre.

Coben, D., Brown, M, et al. (2007). Effective teaching and learning:

numeracy. London: National Research and Development Centre.

Commission for Financial Literacy and Retirement Income. (2012).

Charting a course: A review of financial education in New Zealand.
Wellington: Author.

Davis, N., Fletcher, J., Brooker, B., Everatt, J., Gillon, G., Macke,
J. & Morrow, D. (2010). E-learning for adult literacy, language
and numeracy: a review of the literature. Wellington: Ministry of
Education.

Department of Labour. (n.d.) The impacts of LLN training

programmes on firm performance: A summary of the findings of the
‘Return on Investment’ case studies. Wellington: Unpublished report.

Department of Labour (2010). Upskilling partnership programme:

Evaluation report. Wellington: Department of Labour.

Page / 43
 Adult Literacy and Numeracy: An Overview of the Evidence

Dorgan, J. (2009a) Adult Literacy Policy: A Review for the National

Adult Literacy Agency. Dublin: National Adult Literacy Agency.

Dorgan, J. (2009b). A cost benefit analysis of adult literacy training.

Dublin: National Adult Literacy Agency.

Eagle-Ashmore, P. (2012). Accountability and control in adult

vocational (foundation) education: Is the adult literacy and
numeracy assessment tool a reliable indicator of literacy and
numeracy progress? Journal of Adult Learning Aotearoa New
Zealand. Vol. 40 Issue 1, 92–99.

EU High Level Group of Experts on Literacy. (2012). Final report.

Evans, K., Waite, E. & Kersh, N. (2012). The challenge of establishing

sustainable workplace ‘Skills for Life’ provision in the UK:
organizational ‘strategies’ and individual ‘tactics’. Journal of Education
and Work. DOI:10.1080/13639080.2012.742180

Fletcher, J., Nicholas, K., & Davis, N. (2010). Supporting adults to

address their literacy needs using e-learning. Journal of Open,
Flexible and Distance Learning, 15(1), 17–29.

Fullan, M. (2007). The new meaning of educational change, 4th ed.

New York: Teachers College Press cited in Kite Aotūroa: Improving
Inservice Teacher Educator Learning and Practice. Te Kete Ipurangi.
http://instep.net.nz/

Haggland, J. & Earle, D. (2013). Engagement is key. Wellington:

Ministry of Education

Hazlewood, K. and Alkema, A. (2013). Literacy and numeracy

professional development: Research findings. Wellington: Tertiary
Education Commission.

Heathrose Research. (2011). An evaluation of the learning

representatives programme. Wellington: Ako Aotearoa.

Hutchings, J., Yates, B., Isaacs, P., Whatman, J., & Bright, N. (2012). Hei
Ara Ako ki te Oranga: A model for measuring wellbeing outcomes from
literacy programmes. Wellington: Ako Aotearoa.

Industry Training Federation. (2010). ITO literacy and numeracy

project: Summary of good practices. Wellington: Author.

Page / 44
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

Lane, C. (2013). Assessing skills of adult learners in 2011. Wellington:

Ministry of Education.

Leach, L., Zepke, N., Haworth, P., Isaacs, P. & Nepia, W. (2009).
Organisational factors that affect delivery of adult literacy, language
and numeracy provision. Wellington: Ministry of Education.

Leach, L., Zepke, N., Haworth, P. & Isaacs, P. (2010). One size does
not fit all: how five tertiary education organisations embed literacy,
language and numeracy. Wellington: Ministry of Education.

Māori Adult Literacy Working Party. (2001). Te Kawai Ora: Reading

the world, reading the word, being the world. Wellington: Ministry of
Māori Development.

Marr, B. & Hagston, J. (2007). Thinking beyond numbers: Learning

numeracy for the future workplace. Adelaide: National Centre for
Vocational Education and Research.

Maths Technology Limited. (2011). Pathways Awarua: Outcomes

for learners in 2011. Tertiary Education Commission. Unpublished
report.

May, S. (2009).Hangaia te mātāpuna o te mōhio: Learning

foundations for Māori adults. Wellington: Ministry of Education.

Ministry of Health. (2010). Kōrero Mārama: Health Literacy and Māori

Results from the 2006 Adult Literacy and Life Skills Survey. Wellington:
Author.

National Adult Literacy Agency. (2009). A cost benefit analysis of

adult literacy training. Dublin: Author.

New Zealand Vocational Education & Training Research Forum.

(2011). Financial literacy as a context for strengthening literacy and
numeracy for adults: Postgraduate research show case.

National Research and Development Centre. (2011). A Literature

Review of International Adult Literacy Policies. Dublin: National Adult
Literacy Agency.

Office for Standards in Education. (2011). Tackling the challenge of

low numeracy skills in young people and adults: Report summary.

Page / 45
 Adult Literacy and Numeracy: An Overview of the Evidence

Potter, H., Taupo, K., Hutchings, J., Mcdowall, S. & Isaacs, P. (2011).
He whānau mātau, he whānau ora: Māori adult literacy and whānau
transformation. Wellington: NZCER.

Reder, S. (2009). Scaling Up and Moving In: Connecting social

practices views to policies and programs in adult education.
Literacy and numeracy studies. Vol 16/17, Issue 2/1, 35–50.

Reder, S. & Bynner, J. (Eds). 2009. Tracking adult literacy and

numeracy skills: Findings from longitudinal research. New York:
Routledge.

Ryan, R., McDonald, H., Sutton, A. &Doyle, S. (2012). Formative

evaluation of ITO embedded literacy and numeracy projects.
Wellington: Department of Labour.

Salomon, M. (2009). Workplace literacy and essential skills: What

works? and why? The Centre for Literacy of Quebec.

Sligo, F., Tilley, E., Murray, N., Watson, B., Comrie, M. & Vaccarino, F.
(2010). Modern apprentices literacy learning: A formative evaluation.
Wellington: Department of Labour.

Smith, G. H., and Smith, S. D. (2012), LiteracyNumeracyPro.com,

Adult Literacy Education & Consulting (ALEC) see [email protected]

Standing Council on Tertiary Education, Skills and Employment,

(2012). National foundation skills strategy for adults.

Tertiary Education Commission. (2009). Embedding literacy and

numeracy: Theoretical framework and guidelines. Wellington: Author.

Thomas, G., & Ward, J. (2009). Numeracy for adults. Latest findings
from teaching and learning research. Wellington: Ministry of
Education.

Thomas, G. & Ward, J. (2010). Numeracy for adults: building skills with
online learning links. Wellington: Ministry of Education

Tuagalu, C. (2012). Engaging pacific migrant workers in literacy,

numeracy & financial literacy training: Reflections on the research
process. Wellington: Ako Aotearoa.

Page / 46
 Tertiary Education Commission / Te Amorangi Mātauranga Matua

Vorhaus, J., Litster, J., Frearson, M. & Johnson, S. (2011). Review of

research and evaluation on improving adult literacy and numeracy
skills. London: Department of Business, Innovation and Skills.

Whatman, J., Schagen, S., Vaughan, K., Lander, J., Twist, J., Brooking,
K., Robertson, S. & Spiller, L. (2010). Engaging young people/
young adults in literacy, language and numeracy skill development.
Wellington: Department of Labour.

Yasukawa, K., Brown, T. & Black. S. (2012). Workplace literacy and

numeracy learning: An opportunity for trade union renewal in
Australia? International Journal of Training Research. 10(2): 94–104.

Page / 47
 Collating Analysing

LN focused Learning outcomes

Recording
learning plans Negotiating with learners Including independent learning
Assessment SMART timeframes
Finding
results

Teacher made Contextual- Tools & Specialist

Adapting others isng processes referral

Using
4
National assessment tools
Online adaptive Diagnostic
Paper based
Others

Using the language of the progressions

Training
objectives Knowing the
Analysis of Specific

Texts
Examples & samples LN demands context Number learner Explicit LN learning
activities
Teaching
strategies
Embedded
learning Resources
Tasks
Texts
Measurement Contextualised outcomes
Tasks Teaching plans
Statistics
Calculations My training
Vocabulary Numeracy
Read Deliver
And more Plan LN skills
Write embedded
development
Listen
training
Literacy
Context Speak
specific
Best guess demands
Feedback
Learning Self
progressions Embedding 5 assessment

LN
Working
knowledge
Ako Embedding
3
Formative vs Reuse Pre-test
Tuakana-teina
LN summative diagnostic Post-test
Whakapapa
Whanaungatanga
Kaitiakitanga Mapping
Mana atua
Mana ao turoa Case Study: Approaches Baseline Knowledge
Mana whenua Āhuatanga Maˉori & methods
Mana tangata Involving
Mana reo
Mana whenua Knowing the Knowing 6 Methods
learners

Literacy
what to do Assessing
demands
Tino rangatiratanga
Programme objectieves
Whare tapa whaˉ Numeracy progress Relating to... Learner needs & goals
Learners LN skill levels

2
Learners
Maˉ ori LN

Relevant
ESOL
theory Good ALLS Collaborating
Supervisor
teaching with

Teaching & Definitions

7
Context
learning

1
Culture TEC Others
Pre-colonisation
Oral traditions Evaluating
Arts & crafts effectiveness
Specialist Context Others
Geography
Identifing
organisations changes &
Maˉ ori LN Others improvements
Resources
available
Maˉ ori language
Online Case Study: English language
Development Reporting to
NZ background Policy Teaching
stakeholders strategies
Others
Print
Reflecting
Reasons for
Initiatives critically
low LN National LN infrastructure
Informs Learning
Issues Provider-based planning activities
Technological
Others Future PD
Industry-based
By Maˉ ori for Maˉ ori Learner LN skill development
Management Assessment
Impact of Socioeconomic Educational My organisation of delivery methods
Others
low LN Post-colonisation Others
Post-treaty
Iwi development
Workplace Economic Other

Social Literacy Numeracy Pro.com