Reading Psychology

ISSN: 0270-2711 (Print) 1521-0685 (Online) Journal homepage: https://www.tandfonline.com/loi/urpy20

Exploring the Impact of Engaged Teachers on

Implementation Fidelity and Reading Skill Gains in
a Blended Learning Reading Program

Rachel L. Schechter, Elizabeth R. Kazakoff, Kristine Bundschuh, Jen Elise

Prescott & Paul Macaruso

To cite this article: Rachel L. Schechter, Elizabeth R. Kazakoff, Kristine Bundschuh, Jen Elise
Prescott & Paul Macaruso (2017) Exploring the Impact of Engaged Teachers on Implementation
Fidelity and Reading Skill Gains in a Blended Learning Reading Program, Reading Psychology,
38:6, 553-579, DOI: 10.1080/02702711.2017.1306602

To link to this article: https://doi.org/10.1080/02702711.2017.1306602

© 2017 The Author(s). Published with Published online: 14 Apr 2017.

license by Taylor & Francis© Lexia®
Learning, A Rosetta Stone® Company

Submit your article to this journal Article views: 7149

View related articles View Crossmark data

Citing articles: 5 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=urpy20
Reading Psychology, 38:553–579, 2017
Published with license by Taylor & Francis
ISSN: 0270-2711 print / 1521-0685 online
DOI: 10.1080/02702711.2017.1306602

EXPLORING THE IMPACT OF ENGAGED TEACHERS ON

IMPLEMENTATION FIDELITY AND READING SKILL GAINS
IN A BLENDED LEARNING READING PROGRAM

RACHEL L. SCHECHTER, ELIZABETH R. KAZAKOFF,

KRISTINE BUNDSCHUH, and JEN ELISE PRESCOTT
LexiaÒ Learning, A Rosetta StoneÒ Company, Concord, Massachusetts

PAUL MACARUSO
Department of Psychology, Community College of Rhode Island,
Lincoln, Rhode Island

The number of K–12 classrooms adopting blended learning models is rapidly

increasing and represents a cultural shift in teaching and learning; however,
fidelity of implementation of these new blended learning programs varies widely.
This study aimed to examine the role of teacher engagement in student
motivation and achievement in a blended learning environment. Reading skill
data were analyzed from 19,366 students across 624 schools led by teachers
defined as engaged users of a blended learning reading program (Lexia
Reading Core5 [Core5]). Results showed significant improvements in reading
skills during the analyzed period for the students of the engaged teachers in
comparison to neighboring classrooms (171,850 students in the same 624
schools) of less engaged teachers.

Introduction

As the cost of educational technology decreases and classroom

size increases, millions of students have digital devices integrated
into their classrooms (Horn & Staker, 2011; Johnson & Haria,
2015). No longer is there a strong argument for just supplying

Address correspondence to Elizabeth R. Kazakoff, LexiaÒ Learning, A Rosetta

StoneÒ Company, 300 Baker Ave., Suite 320, Concord, MA 01742. E-mail:
[email protected]
Color versions of one or more figures in the article can be found online at
www.tandfonline.com/urpy
Ó LexiaÒ Learning, A Rosetta StoneÒ Company
This is an open-access article distributed under the terms of the Creative Commons
Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unre-
stricted use, distribution, and reproduction in any medium, provided the original work is
properly cited.

553
554 R. L. Schechter et al.

hardware and software to classrooms; instead, the focus is on

helping teachers understand how to be effective and informed
users of digital tools (Healey, 2015).
Blended learning, the integration of student-directed online
learning with a teacher-led offline component, leverages digital
technologies to provide students with more control over time,
place, path, and/or pace of their own learning (Horn & Staker,
2011). Many studies on blended learning implementation, and
subsequent teacher training, utilize high school or college-level
samples of students and instructors; however, blended learning
as a classroom practice has recently migrated into primary grades
and implementations have been steadily rising (Picciano, Sea-
man, Shea, & Swan, 2012).
Typically, implementations of new curricular and class-
room initiatives, like blended learning, are most successful
with engaged teachers and engaged students. Even before inte-
grating new technologies in the classroom, student learning
outcomes have historically been tied to teacher engagement
(Hardre & Reeve, 2003; Jang, Reeve, Ryan, & Kim, 2009; Reeve,
Jang, Carrell, Jeon, & Barch, 2004; Vansteenkiste, Niemiec, &
Soenens, 2010) as well as both student and teacher motivation
in the classroom (Deci, Vallerand, Pelletier, & Ryan, 1991; Lee
& Reeve, 2012).
In this vein, prior studies on blended learning in secondary
and higher education emphasize four key components to suc-
cess: (1) the importance of teacher training (Gorozidis &
Papaioannou, 2014; Hilliard, 2015); (2) the technical function-
ing of the educational tools (Hilliard, 2015); (3) the effectiveness
of the programs on student outcomes (Hilliard, 2015); (4) and
the importance of understanding blended learning as more than
just placing technology into a classroom—it is about the integra-
tion of student-directed and teacher-directed, online and offline
learning (Vaughan, 2014).
However, little research has been conducted on teacher
engagement and motivation in relation to integrating educa-
tional technology in elementary school classrooms and is a
needed area of study (Schweighofer & Ebner, 2015). Based on
this need, the present study explores the impact of teacher
engagement on student achievement in K-5 classrooms using a
blended learning approach to reading instruction.
 Engaged Teachers and Blended Learning Fidelity 555

Blended Learning

The number of K–12 classrooms adopting blended learning

models, integrating online and offline instruction, is rapidly
growing (Horn & Staker, 2011). Blended learning, when imple-
mented correctly, can be used as a way to support rather than
weaken face-to-face instruction by providing teachers with access
to data immediately (Freeland, 2015; Powell et al., 2015); this
allows teachers to quickly gauge student progress and make
informed pedagogical decisions to differentiate and personalize
instruction (Cavanaugh, Sessums, & Drexler, 2015; Hilliard,
2015).
There are multiple models of blended learning including
flex model (mostly online), enriched virtual model (mainly
online and check-in with teacher), a la carte model (part of
instruction online with the rest face-to-face instruction), and rota-
tional model (small groups of students transitioning between dif-
ferent learning stations in a classroom or computer laboratory
setting) (Horn & Staker, 2011). Blended learning in the early
grades is most frequently achieved through a rotational model
(Staker & Horn, 2012). For example, one station could be the
online component of a blended learning program, while another
station is designated for small-group instruction with a teacher or
paraprofessional, and the remaining stations have students work-
ing independently at their seats or engaged in silent or peer-read-
ing. Research shows that students who use a combination of
online and face-to-face instruction perform better than their
peers who only access content through one mode of instruction
and that teacher involvement is a stronger mediator of success in
a blended model than in online-only programs (Zhao, Lei, Yan,
Lai & Tan, 2005). With the influx of digital devices and educa-
tional software into classrooms, students are becoming more self-
directed learners, and thus teachers’ roles are changing. Teach-
ers now see their role as a mentor, coach, or guide, rather than
predominantly as a lecturer (Powell et al., 2015). Although
implementing a successful blended learning model, as with other
new curricula or pedagogy, should include professional develop-
ment opportunities for teachers and administrators, trainings
may be subject to budget and time limitations and are not always
comprehensive and adequate (Darrow, Friend, & Powell, 2013).
556 R. L. Schechter et al.

It is essential that teachers are provided with sufficient train-

ing opportunities for positive student achievement outcomes
(Darling-Hammond, 2000; Villegas-Reimers, 2003; Yoon, Dun-
can, Lee, Scarloss, & Shapley, 2007). The International Society
for Technology in Education (ISTE) standards outline several
best practices in the context of blended learning, including mak-
ing technology an integral part of teaching and encouraging
administrators to provide trainings so the technology can be
used effectively in the classroom (ISTE, 2008). Goals of the
blended learning program, expected outcomes, and resources
invested should be clearly articulated, addressing the needs of all
stakeholders—teachers, administrators, students, and parents
(Hilliard, 2015; Moskal et al., 2013).

Engagement and Motivation

An expansive body of literature on teacher and student motiva-

tion and the connections between motivation and engagement
provides a lens through which to examine how teacher engage-
ment supports student motivation, engagement, and outcomes
when implementing new technologies. The classroom environ-
ment created by the teacher can, at times, support student moti-
vation and engagement while, at other times, undermine it
(Reeve, 2012).
In order for innovative programs to be successful, teachers
need to be engaged in both new educational practices as well as
the related professional development opportunities. Whether
teachers implement a new curriculum with enthusiasm or return
to their traditional way of instruction depends on their percep-
tion of success (Abrami, Poulsen, & Chambers, 2004). Studies
have found that teacher motivation, particularly intrinsic motiva-
tion (engaging in an activity because of an inherent interest
rather than external factors) (Ryan & Deci, 2000) is an important
predictor of successful implementation of new educational pro-
grams in classrooms (Abrami et al., 2004; Gorozidis & Papaioan-
nou, 2011; Gorozidis & Papaioannou, 2014; Hanfstingl, Andreitz,
M€uller, & Thomas, 2011).
Within a school, teachers typically have similar working con-
ditions, professional development opportunities, educational
 Engaged Teachers and Blended Learning Fidelity 557

resources, and other extrinsic factors, yet motivation and engage-

ment varies. Self-determination theory (SDT) aims to explain
and guide behaviors that increase motivation and engagement
(Reeve, 2012). SDT focuses on the needs of autonomy (an inter-
nal sense of control), competence (feeling of success), and relat-
edness (connection to others) (Deci, Eghrari, Patrick, & Leone,
1994; Hanfstingl et al., 2011; Niemiec & Ryan, 2009; Rigby, Deci,
Patrick, & Ryan, 1992; Ryan & Deci, 2000).
Using SDT to evaluate professional development initia-
tives, researchers found that, although both intrinsic (e.g.,
desire to learn more) and extrinsic (e.g., penalty for not
attending) reasons existed for participating in professional
development courses, only intrinsic motivation positively pre-
dicted teachers’ intentions to participate in trainings and to
implement innovative curriculum in the future. It was con-
cluded that training strategies for teachers must support
intrinsic motivation to lead to successful implementations of
new educational programs (Gorozidis & Papaioannou, 2014).
The motivation mediation model of SDT also accounts for
how aspects of intrinsic motivation in teachers can be used to
predict student performance. According to the model, the
degree to which teachers perceive that they have autonomy
predicts student engagement in the classroom which, in turn,
predicts learning outcomes (Hardre & Reeve, 2003; Jang
et al., 2009; Vansteenkiste et al., 2010). For example, auton-
omy supportive teachers communicate with noncontrolling
language that supports students’ interests and helps students
understand the value of all aspects of their schoolwork (Reeve
et al., 2004).
The SDT motivation mediation model has been validated
across multiple classroom contexts (Assor, Kaplan, Kanat-May-
mon, & Roth, 2005; Black & Deci, 2000; Jang et al., 2009; Van-
steenkiste, Simons, Lens, Soenens, & Matos, 2005; Williams &
Deci, 1996). The use of blended learning for instruction is con-
sistent with SDT as it applies to student motivation and engage-
ment. Blended learning can support autonomy through
student-directed use of the online components of the program
(Powell et al., 2015), and enhances competence by building
students’ sense of mastery as they work on skills at the appro-
priate level of difficulty. The third area of SDT, relatedness, is
558 R. L. Schechter et al.

the factor most reliant on the teacher who develops the class-
room culture as students work towards common goals and cele-
brate success.

Current Study

Teacher engagement and buy-in are key to successful implemen-

tation of new curriculum (Hardre & Reeve, 2003; Jang et al.,
2009; Reeve et al., 2004; Vansteenkiste et al., 2010), however, as
with the addition of any new curriculum to the classroom, the
quality of implementation and level of stakeholder engagement
can vary (Freeland, 2015). To better understand teachers’ actions
in their blended learning classrooms, and the importance of
teacher engagement, the current study explored how teacher
engagement with a blended learning reading program can foster
student engagement and positive learning outcomes in elemen-
tary school.
For this study, teacher engagement was defined in terms of
voluntary participation in a usage contest (described in detail
below). Teachers who chose to participate in the usage contest
were considered more engaged with the blended learning pro-
gram than teachers in the same school who did not participate,
since they engaged with the program enough to be exposed to
the contest participation opportunity and then self-select into
participating. By examining the pre-, during, and postcontest
behaviors of teachers, as well as program usage fidelity and stu-
dent progress in the program, this study aimed to better under-
stand the influence of teacher engagement on implementation
of a blended learning reading program in elementary school.
Teachers were also surveyed on their types of interactions with
the blended learning program to further explore their level of
engagement with the program.

Usage Contest

One of the greatest challenges in implementing a blended learn-

ing program is scheduling online use so that all students are pro-
vided with sufficient time to use the program. The blended
learning program employed in this study varies usage
 Engaged Teachers and Blended Learning Fidelity 559

recommendations (targets) between 20 and 80 minutes per

week, depending on a student’s risk level (i.e., how likely the stu-
dent is to complete grade level materials by the end of the school
year). Students with higher risk levels are assigned higher usage
targets. Minutes of program use is often considered an indicator
of program fidelity; however, if a classroom is allocated
40 minutes of computer time per week, all students may receive
40 minutes of online program use, even though the recommen-
dation is 20, 40, 60, or 80 minutes. As part of a national year-long
effort to increase the number of students meeting usage targets
in the blended learning program (Lexia Reading Core5Ò [Core5])
classes were invited to participate in a “usage contest” in the win-
ter of the 2014–2015 school year.
Participating classes throughout the nation competed to
determine which ones could have students meet their usage
targets with the highest consistency (e.g., all students in the
class meeting their usage recommendation every week). A
pizza party was offered as a prize for classes with the most
points. Pizza parties were awarded to six classes at the conclu-
sion of two, two-week contest rounds. In addition, the contest
provided an opportunity for teachers to interact with one
another and parents by sharing their success on social media
for additional contest points.
Although the contest itself could be viewed as an external
motivator, its inception provided researchers two key data
points: (1) Responses to the announcement of the contest on
the progress monitoring dashboard served as a measure of
engagement, comparing teachers who were interested, goal-ori-
ented, and actively involved in a task (e.g., providing students
with enough minutes in the program) to teachers who were
more apathetic to the contest or the dashboard itself (Reeve et.
al., 2004). (2) The timing of the contest allowed for a baseline
of 3 prior months of program use for comparison to 3 months
of program use which included the usage contest.

Research Questions

This study examined whether teacher engagement with the

blended learning reading program (as defined by participation
560 R. L. Schechter et al.

in a usage contest) leads to greater reading gains for students of

more engaged teachers (participants) compared to students of
less engaged teachers within the same schools (non-participants).
Data from the blended learning program were collected to deter-
mine the following: (1) Does teacher engagement (as measured
by contest participation) lead to differences in (a) program usage
time and (b) student progress in participating classes compared
to students in neighboring nonparticipating classes? (2) In what
ways, if any, does teacher engagement (as measured by contest
participation) relate to increased usage of key implementation
monitoring tools in the blended learning program?

Method

Student usage and progress in the blended learning program

were analyzed for students from contest-participating classes
(participants) compared to students from other classes using the
blended learning program in the same schools but their teachers
chose not to participate in the usage contest (nonparticipants).
Since student risk levels (and corresponding usage recommenda-
tions) are often related to fidelity of program use and progress,
risk level served as a control variable in the analyses. To further
understand how participation in the usage contest related to
teachers’ implementation of the blended learning program, par-
ticipating teachers were given an opportunity to complete a sur-
vey addressing their use of implementation monitoring tools
offered by the program.

Sample

Participants in the study were teachers and students from

schools that used the blended learning program during the
2014–2015 school year. Teachers were invited to participate
in a usage contest by a banner on the implementation moni-
toring dashboard that remained present from two weeks
before the contest up through the final week. The banner
read: “Enter the Usage Contest to win a pizza party: (link to
Usage Contest website).” Teachers could view the banner
 Engaged Teachers and Blended Learning Fidelity 561

whenever they logged into the implementation monitoring

dashboard. An invitation to enter the contest was also posted
on social media accounts (Facebook, Twitter, GoogleC and
LinkedIn) starting in late February. Variations of the invita-
tion were posted, on average, twice a week for the duration
of the contest.
To be included in the study as a participating class, students
had to use the online component of the program for two time
periods during the 2014–2015 school year—before the usage
contest (October-December) and during the contest (January-
March). Classes from schools who had contest-participating class-
rooms, but were not participating in the contest, were included
in analyses as nonparticipating comparison classes. Students in
the study came from 764 schools across the United States. Within
these schools, 1,440 classes participated in the usage contest.
There were 19,366 students in these participating classes (partici-
pants). Students from the same schools who used the blended
learning program during the same time period (October-March)
but did not participate in the usage contest served as a compari-
son group (non-participants). There were 171,850 students in
the comparison group. The distribution of students across kin-
dergarten to fifth grade classes was similar for the participating
and non-participating groups, as shown in Table 1 below, which
outlines the study sample by grade.

Study Tools
BLENDED LEARNING PROGRAM (CORE5)
The blended learning program implemented in this study was
Lexia Reading Core5Ò (Core5). It is an adaptive program for Pre-K
through grade five comprising student-directed online activities
and teacher-implemented offline lessons and enrichment activi-
ties. The program delivers an explicit, systematic, and individual-
ized curriculum that is aligned to the Common Core State
Standards in reading (corestandards.org) by grade level. The 18
levels of material included in the online component provide
instruction in phonological awareness, phonics, structural analy-
sis, automaticity/fluency, vocabulary, and comprehension. The
program provides scaffolding and immediate feedback across
1,243 skill-based units designed to support struggling students’
562 R. L. Schechter et al.

TABLE 1 Study Sample by Grade

Participants Non-Participants
Grade Total Sample N % N %

K 32174 3403 17.6 28771 16.7

1 38886 4047 20.9 34839 20.3
2 38020 3944 20.4 34076 19.8
3 33967 3762 19.4 30205 17.6
4 27762 2481 12.8 25281 14.7
5 20407 1729 8.9 18678 10.9
Total 191216 19366 171850

advancements as well as sustain and enhance the skills of on-level

and above-level students. Teachers are provided with data-driven
action plans based on students’ real-time performance. These
plans recommend teacher-led lessons and paper-and-pencil activ-
ities to differentiate instruction. The program’s implementation
dashboard provides student usage information and progress
reports at the teacher, school, and district administration levels.
An integral part of the blended learning program is
meeting individualized recommended weekly usage targets.
The program provides usage targets which adapt based on
students’ performance in the program during the previous
month. Students are assigned 20–80 minutes of program
usage per week depending on their risk level (i.e., how likely
they are to complete grade level materials by the end of the
school year).

IMPLEMENTATION DASHBOARD
In the survey distributed for this study, self-reported use of the
implementation dashboard (myLexia) served as one indicator of
teacher engagement with the blended learning program. The
dashboard reports contain data on program usage, each
student’s accuracy and progress in completing activities in the
program, and information regarding the student’s strengths and
weaknesses in specific reading skills. Teachers can utilize the
implementation dashboard to monitor and target instruction to
a specific skill or skills offline for students who need extra sup-
port, which ensures that students are productively using the pro-
gram when online.
 Engaged Teachers and Blended Learning Fidelity 563

Monitor student program usage. When teachers log into the

implementation dashboard, they are presented with their whole
class overview report. This page includes a class roster that indi-
cates each student’s usage data including his/her weekly usage
target, time on the program during the current week, and an
indicator if each student reached his/her target the previous
week. Students who need immediate support are flagged with
links to scripted lessons for teachers to provide on the specific
skill area of need.
Assessing students’ individual needs. Each student has an indi-
vidualized report which provides the teacher with up-to-date per-
formance and usage data over the school year. Teachers can use
the included action plan to determine the specific units on which
the student is struggling and to access relevant lessons and activi-
ties for individual or group instruction. Detailed student usage
information for the current week, the last four weeks, and the
entire year are also on the student report.
Through the implementation dashboard, teachers have
access to a resource library with over 340 lessons and 430 pages
of pencil-and-paper independent activities. Lessons help students
with immediate instructional needs so they can progress in the
online skills. The pencil-and-paper activities reinforce ideas from
completed online skills and help build automaticity. By utilizing
these materials, teachers can maximize student productivity
online and help ensure the transfer and extension of skills from
the blended learning program to other reading tasks.
Tracking student progress. These methods of tracking stu-
dent progress give teachers formative assessment information
while students work each day. Formative assessment is a
method that teachers use to gather data on their students’
current progress and inform instruction. The implementa-
tion dashboard that teachers were surveyed about in this
study provides teachers with up-to-date reports for ongoing
formative assessments, as well as the resources to use assess-
ment data to adapt and personalize instruction. A progress
graph depicts a student’s growth each month and provides
an end-of-year benchmark, which corresponds to a student
completing all materials for his or her grade level. Teachers
can also view the percent of completion for each activity in
564 R. L. Schechter et al.

the student’s current program level (i.e., the number of

units the student completed divided by the total number of
units in that activity, usually between 10 and 20 units). For
students working far below grade level, other program-level
markers can be used as progress targets (i.e., number of
units or activities completed).

Study Measures

To evaluate if teachers’ participation in the usage contest had an

effect on student performance in the blended learning program,
student online usage and progress data in the program were ana-
lyzed to compare behavior between participating classes and
non-participating classes prior to and during the usage contest.

TIMING
The time point indicator was coded as a binary variable, where
0 is before the usage contest (October–December), and 1 is dur-
ing the usage contest (January–March).

USAGE AND PROGRESS

Student usage data includes students’ weekly usage recom-
mendations and their actual minutes of use for those corre-
sponding weeks. These data were used to classify students as
‘meeting usage’ or ‘not meeting usage’ each week. The final
meeting usage indicator is defined as the percent of weeks stu-
dents met their usage recommendations during each three
month time period (0–100%). Student progress data consist of
the total number of program units completed online (maximum
1,243) during each time point (before or during the contest).

PARTICIPATION
The participation indicator was coded as a binary variable,
where 0 indicates a student in a non-participating class (non-
participants) and 1 indicates a student from a participating
class (participants).
 Engaged Teachers and Blended Learning Fidelity 565

RISK LEVEL
Students’ usage recommendations are updated monthly
based on the likelihood of reaching end-of-year benchmark in
the blended learning program. Usage recommendations
range from 0 to 80 minutes, with higher risk students having
higher recommendations. Therefore, a risk level between 0–
80 was calculated for each student by taking an average of
students’ six monthly usage recommendations across the two
time periods, three from each time period. For example, if a
student’s usage recommendation was 40 in October, 50 in
November, and 60 in December, risk level for that time
period was 50. The risk level served as a control variable
when comparing performance measures for participating and
non-participating students for each of the two time periods.
Before the contest, the average risk value was similar for non-
participants (46.95) and participants (46.06).

USAGE CONTEST SURVEY

A usage contest survey was created to gather feedback from
participating teachers on how the usage contest, designed to
increase engagement with the blended learning program,
impacted their use of the implementation dashboard. Responses
to open-ended questions from a previously delivered survey to
frequent program users were analyzed using pattern analysis
(Miles & Huberman, 1994) to identify four common behaviors
associated with the implementation dashboard. Questions were
designed as retrospective items, which is recommended for
reflecting on the impact of a teacher education experience
(Pratt, McGuigan & Katzev, 2000). For each of the four common
implementation behaviors, assessing student needs, monitoring
student usage, tracking student progress, and adapting lesson
plans based on student data, teachers were asked (yes or no) to
respond whether or not they engaged in that behavior weekly
before, during, and after the usage contest.
The survey was sent on three occasions in May, each one
week apart, to the 1,440 participating teachers of the usage con-
test. Teachers received a follow-up email if they did not click the
survey link in the prior email(s). Responses were obtained from
154 teachers (See Appendix A for a list of survey questions).
566 R. L. Schechter et al.

Analysis Plan

Analyses sought to explore the impact of teacher engage-

ment—as defined by participation in the usage contest—on
student performance when using a blended learning program.
A linear regression analysis was used to examine if student
usage and progress in the blended learning program differed
for participants compared to non-participants. Time point
(before versus during the usage contest) served as a predictor
variable and risk level as a control variable. The reported
models include only significant predictor and control varia-
bles. Students with valid outcome measures (program usage
and progress) had no missing data on other indicators. As a
result, no additional missing data analyses were conducted. In
addition, to assess the relationship between teacher engage-
ment and implementation behaviors, an analysis of variance
(ANOVA) was used to compare survey results tied to behav-
iors before, during, and after the usage contest.

Results

Meeting Usage

The first analysis examined the relationship between meeting

usage recommendations and contest participation. Average per-
cent of weeks meeting usage before and during the usage contest
is shown comparing participants and non-participants in
Figure 1. Results indicate that both before and during the con-
test, participants met their weekly usage recommendations more
often than non-participants. An analysis of covariance
(ANCOVA) indicated that in the Before Contest time point, par-
ticipants met usage recommendations for a significantly higher
percentage of weeks than non-participants (65.1% vs. 52.6% on
average), even when controlling for student risk level (F(1,
191213) D 2254.137, p < .001). An ANCOVA indicated similar
findings for the During Contest time point, where participants
met usage recommendations for 68.6% of the weeks on average
while non-participant met usage recommendations for only
 Engaged Teachers and Blended Learning Fidelity 567

FIGURE 1 Estimated marginal means for percent of weeks meeting usage

before and during usage contest.

51.8% on average, again controlling for risk level ((F(1, 191213)

D 3724.422, p < .001).
A linear regression analysis, controlling for students’ risk
level, examined the relationship between contest participation
and meeting usage recommendations over time. Meeting
usage recommendations improved more for participants than
non-participants between the two time points examined
(4.6 vs. 1.9 percentage point increase, controlling for student
risk level). Model information can be found in Table 2. The
F statistic for the overall model was 41656.537, with a p-value
<.001, showing that the model is significant. As indicated by
the parameter estimates outlined in Table 3, controlling for
students’ risk levels, there was a significant interaction
between contest participation and time point (F(1, 382,431)
D 64.81, p < .001), meaning that participants showed a
greater improvement in meeting usage recommendations
than non-participants from before to during the contest.

Units Completed

Increased consistency in meeting usage recommendations

translated to completing more units in the online compo-
nent of the blended learning program. Overall, participants
568 R. L. Schechter et al.

TABLE 2 Test of Between Subjects Effects for Meeting Usage

Type III Sum of Mean One-Tailed

Source Squares df square F Significance

Corrected Model 16747.950 4 4186.987 41656.537 .000

Intercept 54511.294 1 54511.294 542335.446 .000
Participation 587.529 1 587.529 5845.354 .000
Timing 36.983 1 36.983 367.950 .000
Risk Level 15984.117 1 15984.117 159026.737 .000
Participation*Timing 6.514 1 6.514 64.810 .000

completed more units in each time period (before and dur-

ing the contest) than non-participants. Before the usage
contest, participants completed 98 units on average com-
pared to 79 units for non-participants. During the usage
contest, participants completed an average of 114 units com-
pared to 85 for participants1. The F statistic for the overall
model was 3260.728, with a p-value <.001, indicating that
the model fit is significant. Model fit information can be
found in Table 4.
Linear regression was used to examine whether there was a
significant difference in the number of units completed by time
period for participants as compared to non-participants (N D
382432). Controlling for students’ risk levels, there was a signifi-
cant interaction between contest participation and time period
(F(1, 382431) D 153.41, p < .001). These results, as shown in
Figure 2, indicate significantly more units were completed dur-
ing the usage contest than before the contest for participants in
comparison to non-participants.

Contest Survey

The contest survey sought to understand how teachers’ participa-

tion in a usage contest, designed to foster classroom-wide com-
munity engagement with the blended learning program,
1
It should be noted that performance in the blended learning reading program
used in this study (Core5) has been shown to be a valid indicator of reading ability. There
are significant correlations between Core5 performance measures and standardized read-
ing assessments such as aimswebÒ , DIBELSÒ , GRADETM , MAPÒ , and STARTM (see Appen-
dix B).
 Engaged Teachers and Blended Learning Fidelity 569

TABLE 3 Parameter Estimates Table for Meeting Usage Targets

Parameter B SE p value

Constant 1.145 .003 .000

Participation .144 .002 .000
Timing .046 .003 .000
Risk Level ¡.010 2.436E-5 .000
Participation*Timing .027 .003 .000

impacted their use of the implementation dashboard. Results

from the survey indicated that for some aspects of implementa-
tion, teachers did not believe their behavior changed before, dur-
ing, or after the usage contest. A majority of teachers stated they
used program supplied offline lessons before (82%), during
(82%), and after (87%) the usage contest. Similarly, nearly all
teachers stated they checked student usage information online
regardless of the timing of the contest (95% before, 94% during,
96% after). ANOVA tests confirmed that the differences between
time points in self-reported offline lesson use and student usage
monitoring were not significant.
Teachers’ self-report of evaluating their students’ gains
and progress online took a dip during the contest. Before the
contest, 89% of teachers reported checking their students’
gains online and 95% reported they checked their students’
gains online after the contest. However, only 37% reported
they checked gains online during the contest (F(2, 480) D
129.766, p < .001). Post hoc comparisons using the Tukey
HSD test confirmed that the proportion of respondents who
reported checking students’ gains online was lower during
the contest compared to both before and after the contest

TABLE 4 Test of Between Subjects Effects for Meeting Usage Targets

Type III Sum of Mean One Tailed

Source Squares Df square F Significance

Corrected Model 67424993.876 4 67424993.876 3260.728 .000

Intercept 677155579.454 1 677155579.454 130991.190 .000
Participation 20173964.230 1 20173964.230 3902.518 .000
Timing 4467987.452 1 4467987.452 864.302 .000
Risk Level 41162546.237 1 41162546.237 7962.618 .000
Participation*Timing 793051.350 1 793051.350 153.410 .000
570 R. L. Schechter et al.

FIGURE 2 Estimated marginal means for change in units gained over time.

(p < .001), while before the contest did not differ from after
the contest.
Despite reporting a dip in using the implementation
dashboard to monitor student progress online, teachers self-
reported an increase in adapting their lesson plans based on
the implementation monitoring dashboard data. When
reflecting on behavior before the contest, only 50% indicated
that they adapted their lesson plans, compared to 58% during
the contest and 65% after the contest. These differences in
self-reported lesson plan adaptations were significant (F(2,
480) D 3.70, p < .03). Post hoc comparisons using the Tukey
HSD test indicated that the proportion of respondents who
reported they adapted their lesson plans before the contest
was significantly lower than the proportion who reported
adapting lessons after the contest (p < .05). Self-reported

TABLE 5 Parameter Estimates for Units Gained

Parameter B SE p-value

Constant 137.900 .579 .000

Participation 28.855 .545 .000
Timing 16.107 .731 .000
Risk Level ¡.493 .006 .000
Participation*Timing 9.547 .771 .000
 Engaged Teachers and Blended Learning Fidelity 571

lesson plan adaptations during the contest did not signifi-

cantly differ from either before or after the contest.

Discussion

This study demonstrated with a large national sample that

teacher engagement can significantly affect student usage
and progress within a blended learning reading program.
Students of engaged teachers (i.e., those who enrolled their
class in the usage contest) significantly increased their fidel-
ity of program use by meeting their weekly usage targets
more often than students of non-participating teachers. Con-
sequently, participants had more time to complete reading
units in the program than non-participants due to their
higher usage fidelity. As this study used participating and
non-participating classes from within the same schools, it is
likely that other aspects of the school environment—curricu-
lum, professional development, administrative support—did
not differ between participants and non-participants. Com-
parison of outcomes for participants compared to non-partic-
ipants is consistent with prior research that indicates
students of engaged teachers perform better than students
of less engaged teachers (Darling-Hammond, 2000; Hardre
& Reeve, 2003; Jang, Reeve, Ryan, & Kim, 2009; Reeve, Jang,
Carrell, Jeon, & Barch, 2004; Vansteenkiste, Niemiec, & Soe-
nens, 2010).
An examination of pre-contest baseline differences between
participants and non-participants indicate that teachers who par-
ticipated in the contest, on average, were more engaged with
monitoring fidelity of use to start than non-participants. These
findings reinforce the theory that engaged teachers supported
the achievement of their students before the contest.
A survey was developed to assess implementation behav-
iors of teachers who opted-in to the contest and learn more
about how teachers engaged with the educator-focused
blended learning tools, such as the implementation dash-
board (myLexia) and offline instructional materials. Interest-
ingly, teachers’ self-reported monitoring of their students’
gains and progress on the implementation dashboard took a
572 R. L. Schechter et al.

dip during the contest, even as their reported behaviors in

monitoring usage and use of offline lessons did not vary. This
dip is possibly due to an increase of teacher engagement with
students as they worked in the classroom as teachers were
more intently focused on their students meeting individual
usage targets. As participating teachers were even more
engaged with their students using the program, they could
see their students earning certificates within the program in
real time and were therefore more aware of their students’
progress without logging into the implementation monitoring
dashboard. In addition, students may have been communicat-
ing their progress in the blended learning program more
vocally as the culture in the classroom increased focused on
program usage during the contest.
Even though the contest participating teachers reported a
dip in using the implementation dashboard to monitor prog-
ress during the usage contest, they self-reported an increase
in adapting lessons based on their students’ data during and
after the contest. This increase may be due to more teachers
learning to locate program data more quickly and becoming
more adept at using the provided resources as a tool for dif-
ferentiating instruction. Additionally, students were complet-
ing units at a higher rate during the contest and feeding
more evaluative data into the dashboards, which may have
made it easier for teachers to adapt lessons on a weekly basis.
Research indicates teacher quality and professional develop-
ment impact student achievement outcomes (Darling-Ham-
mond, 2000; Yoon et al., 2007). In particular, teachers trained
based on self-determination theory tend to be more autonomy-
supportive of their students and display more engagement in
their classrooms (Reeve et al., 2004). Participating teachers likely
enrolled their classrooms to participate in the usage contest due
to self-determined factors to a greater extent than external
rewards (Gorozidis & Papaioannou, 2014) and were able to bet-
ter support their student’s motivation and success in the blended
learning program by fostering relatedness, autonomy, and com-
petence for students in their classrooms. As evidenced by com-
parisons in learning outcomes between participants and non-
participants, this study continued to demonstrate that engaged
teachers have higher achieving students than less engaged
 Engaged Teachers and Blended Learning Fidelity 573

teachers and this trend persists when implementing a blended

learning program.

Implications

This study contributes to the literature by demonstrating with a

large national sample of students that teacher engagement had a
significant impact on student progress within a blended learning
reading program. This study confirms the importance of
engaged teachers on student outcomes and argues for future
explorations of how to best support and develop teacher engage-
ment through professional development and teacher training
programs, especially when integrating new technologies into the
classroom. Professional development programs that provide an
environment where teachers believe they can successfully imple-
ment innovative programs in their classrooms (Abrami et al.,
2004) and foster autonomy-supportive behaviors (Reeve et al.,
2004) are of particular importance when implementing new
classroom programs (e.g., blended learning).

Limitations

Although there was a large set of student data to assess reading

gains for both participants and non-participants, the data were
limited to performance in the blended learning program, Core5.
Performance measures from Core5 are correlated with outside
reading measures (see Appendix B), however, data was not col-
lected on outside measures from students included in data analy-
sis in this study, given the very large number of students and
varied locations of schools across the United States in the sample.
In addition, this study was conducted during one school year
only. Future studies should track long term data to evaluate the
sustainability of the teacher engagement effects when the stu-
dents change grades and classrooms.
Given that participants in this study were self-selected,
there are other limitations. Survey response data are restricted
to teachers who both enrolled in the contest and chose to
complete the survey. Given that less than 15% of participating
teachers responded to the survey, it is possible that findings
574 R. L. Schechter et al.

from the survey do not capture aspects of program implemen-

tation for the full set of engaged teachers.
While this study provided useful insight into teachers’
engagement with a blended learning program and the impact of
engagement on student outcomes, questions still remain of how
to recruit and sustain teachers’ use of the blended learning pro-
gram to support student engagement and subsequent learning
outcomes. Perhaps mandatory participation in quality profes-
sional development for the blended learning programs and/or
general professional development to support self-determined
qualities (i.e., autonomy, competency, relatedness) in the class-
room are solutions that warrant further study.

Conclusion

This study demonstrated that when teachers are engaged in the

implementation of a blended learning reading program as
designed, their students accomplish more in the program than
the students of neighboring teachers who are less engaged.
When teachers who were already implementing the blended
learning program with fidelity participated in a usage contest
they were able to further improve their program implementation
and student outcomes in the program compared to students of
the teachers next door. Teachers responded that they more read-
ily adapted instruction following participation in the usage con-
test, which is an essential element of a blended learning
program.
While prior studies on teacher engagement and self-determi-
nation theory suggest that teachers who are already engaged
teachers were the ones most likely to participate in the usage con-
test, this study adds to the literature on the importance of teacher
engagement for student gains, by both demonstrating greater
reading gains for participants and doing so within a blended
learning environment. As school districts and teachers are adopt-
ing new technologies and blended learning programs into their
classrooms, it is essential to understand how educator’s motiva-
tional attributes impact learning gains for their students.
In sum, this study is informative for the design of blended
learning programs and stresses the importance of consistent
 Engaged Teachers and Blended Learning Fidelity 575

implementation for student gains. The results point to a need

for developing ways to continue to support engaged teachers
while finding ways to engage more reluctant to adopt teachers.
Technology integration alone is not enough—knowledgeable,
trained teachers who are afforded the opportunity to self-
reflect, evaluate, and alter their pedagogy accordingly, are
essential for the proper use of new technological tools and
blended learning programs in their classrooms.

Declaration of Interest

This submission evaluates the effectiveness of a commercial prod-

uct. Four authors of this paper are employed by LexiaÒ Learning,
A Rosetta StoneÒ Company, and one serves as a paid consultant
and is employed by Community College of Rhode Island. None
of the researchers receive commission on sales of the products.
Teachers and school personnel carried out the implementation
of the program.

References

Abrami, P. C., Poulsen, C., & Chambers, B. (2004). Teacher motivation to

implement an educational innovation: Factors differentiating users and
non-users of cooperative learning. Educational Psychology, 24(2), 201–216.
Assor, A., Kaplan, H., Kanat-Maymon, Y., & Roth, G. (2005). Directly controlling
teacher behaviors as predictors of poor motivation and engagement in girls
and boys: The role of anger and anxiety. Learning and Instruction, 15(5), 397–
413.
Black, A. E., & Deci, E. L. (2000). The effects of instructors’ autonomy support
and students’ autonomous motivation on learning organic chemistry: A self-
determination theory perspective. Science Education, 84(6), 740–756.
Cavanaugh, C., Sessums, C., & Drexler, W. (2015). A call to action for research
in digital learning: Learning without limits of time, place, path, pace. . .or
evidence. Journal of Online Learning Research, 1(1), 9–15.
Darling-Hammond, L. (2000). Teacher quality and student achievement. Educa-
tion Policy Analysis Archives, 8(1). doi:10.14507/epaa.v8n1.2000
Darrow, R., Friend, B., & Powell, A. (2013). A roadmap for implementation of
blended learning at the school level: A case study of the iLearnNYC lab schools.
Vienna, VA: International Association for K–12 Online Learning.
Deci, E. L., Eghrari, H., Patrick, B. C., & Leone, D. R. (1994). Facilitating inter-
nalization: The self determination theory perspective. Journal of Personality,
62(1), 119–142.
576 R. L. Schechter et al.

Deci, E. L., Vallerand, R. J., Pelletier, L. G., & Ryan, R. M. (1991). Motivation
and education: The self-determination perspective. Educational Psychologist,
26(3–4), 325–346.
Freeland, J. (2015). Three false dichotomies in blended learning. San Mateo, CA:
Clayton Christensen Institute.
Gorozidis, G., & Papaioannou, A. (2011). Teachers’ self-efficacy, achieve-
ment goals, attitudes and intentions to implement the new Greek physi-
cal education curriculum. European Physical Education Review, 17(2), 231–
253.
Gorozidis, G., & Papaioannou, A. G. (2014). Teachers’ motivation to participate
in training and to implement innovations. Teaching and Teacher Education, 39,
1–11.
Hardre, P. L., & Reeve, J. (2003). A motivational model of rural students’ inten-
tions to persist in, versus drop out of, high school. Journal of Educational Psy-
chology, 95, 347–356.
Hanfstingl, B., Andreitz, I., M€uller, F. H., & Thomas, A. (2011). Are self-regula-
tion and self-control mediators between psychological basic needs and
intrinsic teacher motivation? Journal for Educational Research Online/Journal
u r Bildungsforschung Online, 2(2), 55–71.
f€
Healey, D. (2015). Review of a constructivist approach to the national educa-
tional technology standards for teachers. Language, Learning & Technology,
19(1), 54–58.
Hilliard, A. T. (2015). Global blended learning practices for teaching and learn-
ing, leadership and professional development. Journal of International Educa-
tion Research, 11(3), 179–187.
Horn, M. B., & Staker, H. (2011). The rise of K–12 blended learning. Mountain
View, CA: Innosight Institute.
ISTE. (2008). ISTE standards for teachers. Eugene, OR: International Society for
Technology in Education.
Jang, H., Reeve, J., Ryan, R. M., & Kim, A. (2009). Can self-determination theory
explain what underlies the productive, satisfying learning experiences of col-
lectivistically oriented Korean students? Journal of Educational Psychology, 101
(3), 644.
Johnson, S. N., & Haria, P. (2015). Effects of an IPad-based collaborative
instruction on first graders at-risk for reading delays. International Journal of
Technology and Inclusive Education, 4(2), 645–649.
Lee, W., & Reeve, J. (2012). Teachers’ estimates of their students’ motivation
and engagement: Being in synch with students. Educational Psychology, 32(6),
727–747.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded
sourcebook (2nd ed.). Thousand Oaks, CA: Sage.
Moskal, P., Dziuban, C., & Hartman, J. (2013). Blended learning: A dangerous
idea? Internet and Higher Education, 18, 15–23.
Niemiec, C. P., & Ryan, R. M. (2009). Autonomy, competence, and relatedness
in the classroom: Applying self-determination theory to educational practice.
Theory and Research in Education, 7(2), 133–144.
 Engaged Teachers and Blended Learning Fidelity 577

Picciano, A. G., Seaman, J., Shea, P., & Swan, K. (2012). Examining the extent
and nature of online learning in American K–12 education: The research
initiatives of the Alfred P. Sloan Foundation. Internet and Higher Education, 15
(2), 127–135.
Powell, A., Watson, J., Staley, P., Patrick, S., Horn, M., Fetzer, L., & Verma, S.
(2015). Blended learning: The evolution of online and face-to-face education from
2008–2015. Promising practices in online learning. Vienna, VA: International
Association for K–12 Online Learning.
Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program out-
comes: Using retrospective pretest methodology. American Journal of Evalua-
tion, 21(3), 341–349.
Reeve, J. (2012). A self-determination theory perspective on student engage-
ment. In S. L. Christenson et al. (Eds.), Handbook of research on student engage-
ment (pp. 149–172). New York, NY: Springer.
Reeve, J., Jang, H., Carrell, D., Jeon, S., & Barch, J. (2004). Enhancing students’
engagement by increasing teachers’ autonomy support. Motivation and Emo-
tion, 28(2), 147–169.
Reeve, J., & Lee, W. (2014). Students’ classroom engagement produces longitu-
dinal changes in classroom motivation. Journal of Educational Psychology, 106
(2), 527.
Rigby, C. S., Deci, E. L., Patrick, B. C., & Ryan, R. M. (1992). Beyond the intrin-
sic-extrinsic dichotomy: Self-determination in motivation and learning. Moti-
vation and Emotion, 16(3), 165–185.
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic
definitions and new directions. Contemporary Educational Psychology, 25, 54–67.
Schweighofer, P., & Ebner, M. (2015). Aspects to be considered when imple-
menting technology-enhanced learning approaches: A literature review.
Future Internet, 7(1), 26–49.
Staker, H., & Horn, M. B. (2012). Classifying K–12 blended learning. Mountain
View, CA: Innosight Institute.
Vansteenkiste, M., Niemiec, C. P., & Soenens, B. (2010). The development of
the five mini-theories of self-determination theory: An historical overview,
emerging trends, and future directions. Advances in Motivation and Achieve-
ment, 16, 105–166.
Vansteenkiste, M., Simons, J., Lens, W., Soenens, B., & Matos, L. (2005).
Examining the motivational impact of intrinsic versus extrinsic goal
framing and autonomy supportive versus internally controlling commu-
nication style on early adolescents’ academic achievement. Child Develop-
ment, 76(2), 483–501.
Vaughan, N. (2014). Student engagement and blended learning: Making the
assessment connection. Education Sciences, 4(4), 247–264.
Villegas-Reimers, E. (2003). Teacher professional development: An international
review of the literature. Paris, France: International Institute for Educational
Planning.
Williams, G. C., & Deci, E. L. (1996). Internalization of biopsychosocial values
by medical students: a test of self-determination theory. Journal of Personality
and Social Psychology, 70(4), 767.
578 R. L. Schechter et al.

Yoon, K. S., Duncan, T., Lee, S. W.-Y., Scarloss, B., & Shapley, K. (2007). Review-
ing the evidence on how teacher professional development affects student achievement
(Issues & Answers Report, REL 2007–No. 033). Washington, DC: U.S.
Department of Education, Institute of Education Sciences, National Center
for Education Evaluation and Regional Assistance, Regional Educational
Laboratory Southwest. Retrieved from http://ies.ed.gov/ncee/edlabs
Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, S. (2005). What makes the difference? A
practical analysis of research on the effectiveness of distance education. The
Teachers College Record, 107(8), 1836–1884.

Appendix A
Usage Contest Survey
SOCIAL MEDIA USAGE AND PIZZA CONTEST — FOLLOW-UP
We would like to know more about your experience with the
usage contests this year. All answers will remain anonymous.
Thank you for sharing your stories!

Q1. Did you participate in the contest during the:

Fall
Spring
Both the Fall and Spring
Q2. What motivated you to participate in the contest?
(e.g., the pizza reward, a focused timeframe, competition, other reasons?)
Q3. BEFORE the contest(s), how many of the following actions did you do
WEEKLY?
 used Lexia Lessons or Skill Builders
 checked student’s usage information
 adapted your lesson plans based on myLexia data
 evaluate student gains and progress online
 I didn’t do any of these things on a weekly basis
 Other:
Q4. DURING the contest(s), how many of the following actions did you do
WEEKLY?
 used Lexia Lessons or Skill Builders
 checked student’s usage information
 adapted your lesson plans based on myLexia data
 evaluate student gains and progress online
 I didn’t do any of these things on a weekly basis
 Other:
Q5. SINCE or BETWEEN the contest(s), how many of the following actions
do/did you do WEEKLY?
 used Lexia Lessons or Skill Builders
 checked student’s usage information
 adapted your lesson plans based on myLexia data
 evaluate student gains and progress online
 Engaged Teachers and Blended Learning Fidelity 579

 I didn’t do any of these things on a weekly basis

 Other:
Q6. Anything else you’d like to share with us about the contest, myLexia, or
Lexia in general?
Q7. If you are open to answering more questions or would like to give feed-
back on new myLexia features, please include your email below (optional)

Appendix B

TABLE B1 Correlations between Core5 Benchmark Status and End-of-Year

Scores on Progress Monitoring Tools

Student Grade
N K 1 2 3 4 5

Progress Monitoring Tool

aimsweb 1,796 — .6 .6 .6 .5 .5
DIBELS Next 10,347 .6 .7 .6 .6 .6 .6
MAP 4,577 .4 .5 .5 .5 .6 .5
STAR Early Literacy 779 .2 .5 — — — —
STAR Reading 2,674 — — .5 .5 .6 .6
Notes. Correlations can be categorized into three ranges: High D .7 – .9, Medium D .4 –
.6, and Low D 0 – .3. Sample sizes are collapsed over grades.