Teaching and Teacher Education 60 (2016) 444e451

Contents lists available at ScienceDirect

Teaching and Teacher Education

journal homepage: www.elsevier.com/locate/tate

Fostering effective teaching behavior through the use of data-feedback

H.T.G. Van den Hurk a, *, A.A.M. Houtveen a, W.J.C.M. Van de Grift b
a
Utrecht University of Applied Sciences, Faculty of Education, Department of Literacy, Padualaan 97, 3584 CH Utrecht, The Netherlands
b
Groningen University, Faculty of Behavioural and Social Sciences, Department of Teacher Education, Grote Kruisstraat 2/1, 9712 TS Groningen, The
Netherlands

h i g h l i g h t s

We studied the development of effective teaching behavior.

Connecting theory and professional practice accelerates professional development.

A standardized observational instrument was used to assess teaching behavior.

Teachers were professionalized in a cyclic model of data-driven teaching.

Data-feedback proves to be a lever of change in teachers’ professional development.

a r t i c l e i n f o a b s t r a c t

Article history: In this study data-feedback in a cyclic model of data-driven teaching was used to enhance the teaching
Received 11 March 2016 behavior of students registered in a master course for teachers. Differences between pre- and post-test
Received in revised form measures in a simple one-group pre-test post-tests design proved to be significant with effect sizes
20 June 2016
ranging from d ¼ 0.29 to d ¼ 0.76. Improving teaching behavior in a time span of only six weeks on
Accepted 5 July 2016
Available online 21 July 2016
average is remarkable since earlier studies indicated that it takes over 15 years to master complex
teaching skills with a ‘natural development’ of teaching skills of about 25% of a standard deviation.
© 2016 Elsevier Ltd. All rights reserved.
Keywords:
Data feedback
Professional learning and development
Effective teaching behavior
Classroom observation
Primary education

1. Introduction to retire from education as the population continues to age in the

years to come, as a result of which a large number of novice
At the start of their professional career in primary education all teachers will enter education (Min. OC&W, 2013). These novice
teachers in the Netherlands are by statutory requirement, qualified teachers start working in an educational environment which is
and competent to perform their job (Min. OC&W, 2004). At the becoming increasingly more complex. Due to recent changes in
same time, research has shown that it usually takes teachers a lot of legislation, schools for primary education in the Netherlands have
time to master the more complex teaching skills. Cross-sectional moved towards a more inclusive learning environments in which a
studies in several European countries, for instance, revealed that growing number of children with special educational needs will
teachers roughly need 15e20 years of teaching experience to attend mainstream schools. At the same time the Netherlands, like
develop the most difficult teaching skills like adapting instruction all other European countries, is confronted with growing numbers
and classroom assignments to the relevant differences between of refugees and asylum seekers whose children, at some point, will
pupils or teaching the use of learning strategies (Van de Grift, 2007, enter the educational system. These recent developments bring
2014; Van de Grift, Van der Wal, & Torenbeek, 2011). To date this is about that novice teachers are confronted with a strongly diverging
an important issue in the Netherlands since many teachers are set population of pupils. This makes it extremely important that
teachers accelerate their professional development and master
difficult teaching skills like, for instance, adapting teaching to the
* Corresponding author. diverse needs of their pupils as soon as possible.
E-mail address: [email protected] (H.T.G. Van den Hurk).

http://dx.doi.org/10.1016/j.tate.2016.07.003
0742-051X/© 2016 Elsevier Ltd. All rights reserved.
 H.T.G. Van den Hurk et al. / Teaching and Teacher Education 60 (2016) 444e451 445

The aim of this study is to investigate whether the development Weinstein, 1998). Next to that, positive mutual relations between
of effective teaching behavior can be accelerated in one of the students can endorse cooperative learning and mutual trust and
modules of a master course in education. In this particular part of respect (Ghaith, 2003). Research in the field of the third distin-
the master course, the teacher behavior of the participating guished factor of school climate, the factor of learning and teaching,
teachers is observed with a standardized observational instrument. shows that teachers’ expectations are of major importance for
The development of effective teacher behavior is accelerated student achievement (Dusek & Joseph, 1983; Harris & Rosenthal,
through the use of data-feedback in a cyclic model of data-driven 1985). Furthermore it becomes clear that students’ self-efficacy is
teaching. The assumption behind the application of this model is strongly correlated with learning motivation and student
that it enables students to make informed decisions regarding the achievement (Bandura, 1997). Teachers can enhance students self-
improvement of their teaching behavior. An earlier study on the efficacy by expressing high expectations about student achieve-
application of this model yielded positive results with regard to the ment (Teddlie, Falkowski, Stringfield, Desselle & Gervue, 1984) and
teaching of beginning reading (Van den Hurk, Houtveen & Van de by setting high but attainable learning goals (Latham & Locke,
Grift, 2014). In the current study we investigate whether it is 1991).
possible to accelerate the development of effective teacher
behavior in general. More specifically we are interested in the 2.1.2. Efficient classroom management
possibility to accelerate the more difficult and complex teacher Generally accepted in education is the fact that all students can
behavior. learn. The way in which they learn and the amount of time they
need to do so, differs from student to student. Teachers have the
2. Theoretical and empirical framework important task to organize their lessons efficiently and to limit the
time loss during lesson transitions, on classroom management and
In this section we will briefly discuss the knowledge base of on maintaining order. Minimal time loss ensures optimal oppor-
effective teaching that led to the construction of the observation tunities to learn (Creemers & Reezigt, 1996; Scheerens & Bosker,
instrument that was used in the current study (2.1). Furthermore 1997). Following Carroll (1963) we differentiate between allo-
we will elaborate on the use of data-feedback in the cyclic model of cated time, engaged time and time on task. Although the amount of
data-driven teaching that was used (2.2). time that has to be spend on different academic subjects is formally
specified (Marzano, 2007), huge differences between schools can
2.1. Effective teacher behavior occur in the way this allocated time is used (e.g. Houtveen & Van
der Velde, 2010). The engaged time is the net amount of time
After more than five decades of teacher effectiveness research, used for instructional purposes. Positive correlations have been
scholars generally agree that effective teacher behavior is one of the found between the amount of engaged time and student achieve-
important factors that eventually will lead to students’ learning ment (see further: Marzano, 2003; Scheerens & Bosker, 1997).
gains (for reviews see: Cotton, 1995; Creemers, 1994; Ellis & Research shows that efficient teachers lose 15% less time than their
Worthington, 1994; Hattie, 2009; Levine & Lezotte, 1990, 1995; lesser efficient colleagues. Furthermore they spend 50% more time
Marzano, 2003; Purkey & Smith, 1983; Sammons, Hillman, & on considering topic related interactions (Houtveen & Van de Grift,
Mortimore, 1995; Scheerens, 1992; Scheerens & Bosker, 1997; 2007). Time on task, sometimes indicated as ‘perseverance’, is the
Walberg & Haertel, 1992). In this study we are specifically inter- amount of time that students are actively engaged in learning tasks.
ested in the effective behavior that teachers display in their Teachers are challenged to keep their students’ attention focused
educational practice, that is teacher behavior that can be observed on the learning tasks at hand.
in everyday lessons. Based on the above mentioned research six
categories of effective teacher behavior were distinguished, 2.1.3. Clear instruction
namely: Creating a safe and stimulating learning climate, Efficient A clear instruction ensures that students understand the subject
classroom management, Clear instruction, Activating learning, that is being taught and are able to combine new content with
Adaptation of Teaching and Teaching learning strategies (Van de earlier acquired knowledge (Creemers, 1994; Muijs & Reynolds,
Grift & Lam, 1998; Van de Grift, 2007; Van de Grift, Van den 2010; Scheerens, 1992). Communicating a limited set of clearly
Hurk, & Houtveen, 2016; Van de Grift et al., 2011). In this section defined lesson objectives with the students proves to be of
we will elaborate on the empirical evidence for the effectiveness of importance (Hattie & Clinton, 2008; Smith, Baker, Hattie & Bond,
these categories of teacher behavior. 2008), as well as defining and communicating a limited set of
core concepts that can work as stepping stones or advances orga-
2.1.1. Creating a safe and stimulating learning climate nizers in structuring the lesson topic (Kameenui & Carnine, 1998;
Research shows that at least part of the differences in student Lohman, 1986; Nunes & Bryant, 1996; Pearson & Fielding, 1991;
achievement are accounted for by factors concerning the learning Pearson & Gallagher, 1983; Pressley et al., 1992; Rosenshine &
climate (Cohen, McCabe, Michelli & Pickerall, 2009; Thapa, Cohen, Meister, 1997). During instruction the teacher has to make sure
Guffey, & Higgins-D’Alessandro, 2013). Thapa et al. (2013) distin- that the students understand the subject and execute the lesson
guished three essential factors in school climate, applicable to the assignments as planned (Hattie & Clinton, 2008; Kameenui &
within-class situation: Safety, Relations and Learning and Teaching. Carnine, 1998; Lohman, 1986; Pearson & Fielding, 1991; Pearson
The subject of ‘safety’ concerns the basic social safety in the class- & Gallagher, 1983; Rosenshine & Meister, 1997; Smith et al., 2008).
room. The importance of this subject is illustrated in the results of a
large scale survey on several thousands of schools in the US, which 2.1.4. Activating learning
show that students consistently rate school bullying as a severe Research conducted from the sixties of the last century onwards
problem (Ateah & Cohen, 2009). Clarity on rules and regulations is made clear that the extent of student involvement is positively
conditional but even more important is that teachers maintain correlated with learning outcomes (Anderson, Evertson, & Brophy,
these rules and regulations and are living up to them on a day to 1979; Denham & Lieberman, 1980; Evertson, Anderson, Anderson,
day basis (Evertson & Weinstein, 2013). The second factor deals & Brophy, 1980; Fischer et al., 1980). Teachers can enhance stu-
with the teacher-student relation which is of importance for the dent involvement and activate student learning by intensifying
motivation for learning (Davis, 2003; Osterman, 2000; Perry & their instructions. This can be done by avoiding excessive seatwork
446 H.T.G. Van den Hurk et al. / Teaching and Teacher Education 60 (2016) 444e451

(Hampton & Reiser, 2004; Lang & Kersting, 2007), by posing higher teaching behavior is often referred to as data literacy (Gummer &
order questions that promote comprehension (Craig, Sullins, Mandinach, 2015).
Witherspoon, & Gholson, 2006; Mantione & Smead, 2003), and In an earlier study we found that the process of collecting,
by replacing one-sided, teacher led communication by forms of analyzing and examining data on their instructional behavior
cooperative learning (Meeuwisse, Severiens, & Born, 2010). enabled teachers to make informed decisions about adjustments in
their instructional behavior during following lessons. In this small
2.1.5. Adaptation of teaching scale study 70 first year student teachers were involved. Close
As ‘teaching to the middle’ can’t meet the diverse needs of all the connections were sought between the knowledgebase on interac-
pupils in a classroom (Haager & Klingner, 2005), teachers should tive storybook reading, taught at the teacher training college, and
adapt the instruction and the classroom assignments to the rele- student teachers’ experiences in providing storybook reading ses-
vant differences between their pupils. Adaptation of teaching sions at schools for workplace learning. These connections where
means that teachers have to find ways to tailor their instruction to established by the alternation of theory and practice in a model of
accommodate all learners. Research indicates that placing students data driven teaching. Firstly the theory of interactive storybook
into mixed-ability groups is more likely to lead to better results for reading was dealt with in lessons at the teacher training college.
almost all students (Walmsley & Allington, 1995; Houtveen, Booij, Subsequently student teachers carried out interactive book reading
De Jong, & Van de Grift, 1999; Slavin, 1996), and will definitely sessions and, while doing so, were observed by their mentors who
contribute to higher self-esteem and a better motivation for used standardized observational instruments. A digital interface
learning (Bennett, 1986; Slavin & Braddock, 1993; Houtveen, 2007; provided immediate feedback of the observational results, which
Houtveen & Van de Grift, 2001). Finding ways of extending the made it possible to discuss these results in the following lessons at
amount of instruction time, e.g. by means of pre-teaching and re- the teacher college. In these lessons the student teachers formu-
teaching, is found to be successful when tailoring the instruction lated specific points of improvement for their next interactive
to the students’ learning needs (Kidron & Lindsay, 2014). storybook reading lesson in workplace learning. This second lesson
was processed in exactly the same way as the first one was. The
2.1.6. Teaching learning strategies results of all the steps that were taken in this procedure, were
Students use cognitive and meta-cognitive strategies to solve recorded in a reflection report that all students had to write as a
problems and to memorize and integrate newly acquired knowl- final assignment. The results indicated that the use of data-
edge (Muijs & Reynolds, 2010). The use of these strategies is proven feedback in a cyclic model of data-driven teaching enabled stu-
to be helpful because it ensures a reduction of the cognitive load in dent teachers in initial teacher training to significantly improve the
pupils (Shah & Oppenheimer, 2008). Teachers can help their stu- quality of their reading lessons over a relatively short period of time
dents in acquiring these strategies and in successfully using them (Van den Hurk, Houtveen, Van de Grift, & Cras, 2014).
whenever needed. Several ways of supporting students in the In the present study we again used the procedure of data-
acquisition and use of learning strategies have proven to be suc- feedback in a cyclic model for data-driven teaching that was orig-
cessful. Teachers can explicitly model the targeted behavior, for inally adapted from Marsh (2012, p.53). The model, depicted in
instance by using a think aloud method while deriving meaning Fig. 1, emphasizes the connection between teachers’ professional
from a text. Furthermore teachers can provide support through the practice and the theory that is being discussed during the training
use of scaffolds and provide corrective feedback (Carnine, Dixon, & in the master course. In this model the observational data, gathered
Silbert, 1998; Dixon, Carnine, Lee & Wallin, 1998; Brophy & Good, in the teachers’ professional practice are revalued into useful in-
1986; Hattie & Clinton, 2008; Houtveen & Van de Grift, 2007; formation and eventually into knowledge that can be acted upon in
Rosenshine & Stevens, 1986; Slavin, 1996; Smith et al., 2008). professional practice (see further: Marsh, 2012; Van den Hurk et al.,
The results of earlier studies confirm the assumption that these 2014). The theory concerns the knowledge base of effective
different categories of teacher behavior can be arranged in an teaching that is discussed in Section 2.1. The professional practice of
ascending sequence according to their level of difficulty. Put in the participating teachers is video recorded and observed by peer-
other words, there is a specific sequence in which teachers will students using a standardized observational instrument (see
master these subsequent categories of teacher behavior. The first further: Section 3.4). The connection between theory and profes-
three categories, including the more elementary teaching behavior sional practice is ensured through the use of (data-) feedback. In
which is relatively easy to master, are: Creating a safe and stimu- this process of discussing the available data all students work
lating learning climate (1), Efficient classroom management (2) and closely together in learning teams supervised by lecturers in the
Clear instruction (3). The subsequent three categories comprise master course. The participating teachers are enabled to exchange
relatively more complex and difficult teacher behavior. These cat- ideas and insights in close collaboration with their peers.
egories are: Activating learning (4), Adaptation of Teaching (5) and
Teaching learning strategies (6) (Van de Grift & Lam, 1998; Van de
Grift, 2007; Van de Grift et al., 2011).

2.2. Data-feedback in a cyclic model of data-driven teaching

The results of earlier research showed that the close connection

between theory and professional practice is an important condition
for professional learning and development (Timperley, Wilson,
Barrar, & Fung, 2008; Wayman & Jimerson, 2014). A synthesis of
research on teacher professional learning demonstrated that
teachers who gather data about their own teaching practices and
use these data to improve the effectiveness of their teaching
behavior, will eventually improve their students’ learning and
achievement (Timperley et al., 2008). The ability to make use of
different kinds of data to determine appropriate adjustments in Fig. 1. Model for data-driven decision making (Adapted form Marsh, 2012, p.53, p.53).
 H.T.G. Van den Hurk et al. / Teaching and Teacher Education 60 (2016) 444e451 447

Participation in such a professional community with colleagues is recorded lessons were independently observed by two peer stu-
seen as an integral part of professional learning that eventually has dents. Immediately after the observations the observers discussed
a positive impact on students (Timperley et al., 2008, p. 19). This the observational data in order to reach agreement on the scores.
process of discussing the available data leads to the formulation The agreed upon scores were then entered in a digital version of the
and registration of specific points of improvement of teacher observation instrument. This digital interface ensured the
behavior, to be applied in the following lessons. The tight connec- recording of the results and, at the same time, provided feedback
tion between theory and professional practice ensures that the reports. In this feedback report mean scores on the different scales
participating students are able to practice newly acquired knowl- of the observation instrument were returned to both the observed
edge in their everyday professional practice. Since research in- students and their lecturers. This procedure made it possible that
dicates that teachers are inclined to maintain changes in their all students had access to the observation results while the first
professional practice once they have experienced the positive re- lesson of the module was still in progress.
sults (Fullan, 2007; Guskey & Yoon, 2009; Guskey, 2002), this is an The next three lessons of the module are organized in a standard
important factor in sustaining the newly acquired teacher behavior. pattern. About half of every three hour session is spent on theory
where the lecturers provide the students with theoretical and
3. Method empirical background information on a specific subject matter from
the knowledge base of teacher effectiveness. The second half of
In this section we will start with a short description of the each session is allocated for activities in learning teams of three or
research design we applied (3.1). Subsequently we will describe the four students. These learning team activities typically begin with a
sample of participating teachers (3.2), the treatment (3.3), the 10e15 min discussion about a given proposition from the specific
variables that were used (3.4) and the data-analyses that were subject matter. An example of one of the propositions regarding the
performed (3.5). subject of ‘adaptation of teaching’ is: “Ability grouping doesn’t
improve pupil achievement and is specifically harmful for pupils at
3.1. Design risk. Why do you, or don’t you agree with this proposition? Do you
think that ‘Round Robin Reading’ will improve the reading
The study involved a group of teachers, all registered as students achievement of students at risk? Why do you, or don’t you think
in a Master Course for teachers at the Utrecht University of Applied so?”.
Sciences in the schoolyear 2015e2016. Since the observations were The initial discussion in the learning team is immediately fol-
part of the regular curriculum, all of the participating students were lowed by a closer look at the ‘real-life’ data obtained in the stu-
assigned to the treatment condition and a simple one-group pre- dents’ own professional environment. After having examined the
test-post-test design was used (Shadish, Cook, & Campbell, 2002). observation results of their own lesson, every student selects at
The application of this design asks for some prudence in the least one area of teaching behavior they see fit for improvement.
interpretation of the results. Since we are not able to use a control Some guidelines for the interpretations of the observations are
condition we can’t be sure that possible effects are solely caused by provided in the basic assumption that scale scores exceeding 2.00
the independent variable (i.e. the treatment). can be seen as a sufficiently satisfying result. Scale scores ranging
from 1.00 up to 2.00 on the other hand, can be interpreted as an
3.2. Sample indication that this specific category of teacher behavior is open to
improvement.
This study involved 110 teachers, the majority of which was In their learning teams all students relate the observation results
female (96%). The participating teachers on average had worked and their own teaching behavior displayed in the recorded lessons
9.14 (SD ¼ 7.55) years in primary education. Slightly over 40% of the with the background information that was provided by their lec-
group had up to five years of experience in teaching primary edu- turers. Learning team members question each other and suggest
cation. The major part of the participating teachers was teaching alternatives for the displayed teaching behavior. The discussions in
pupils aged 8 through 12 (57%) while 18% was teaching pupils aged these learning teams are focused on the formulation of points of
6 and 7 and the remaining 25% of the respondents worked with improvement with regard to the students’ instructional behavior in
pupils in the age group of 4 and 5. All of the students were certified their next lessons. Although these learning team activities are
teachers from a University of Applied Sciences (PABO), which is a basically student-led, the lecturers are permanently available if any
form of higher professional education. In the Netherlands the additional support is required.
teacher training programs in higher professional education are Before the fifth lesson of the module, a second lesson is video
more practical orientated than the teacher training programs at a recorded. In this fifth and last lesson of the module these recordings
research oriented university, with a primary objective of ‘the are processed in exactly the same way as the first lesson was. The
transfer of theoretical knowledge and the development of skills students use the feedback on both observations to write their final
that are closely linked to professional practice’ (EP-Nuffic, 2015). assignment that is assessed by their lecturers.
After having finished their four year bachelor’s program (240 ECTS)
the majority of these teachers typically start working in education. 3.4. Training in the use of the observation instrument
At some point in their professional career some of the teachers
enroll in a master’s program (60 ECTS) that either can be attended Prior to the start of this module all of the participating teachers
full-time (1 year) or part-time (2 year). The 110 teachers partici- attended a three hour training session where information was
pating in this study were all enrolled as a student in such a master provided on the theoretical background and the composition of the
course. observation instrument. The mandatory training sessions were led
by trained researchers who were experienced in the use of the
3.3. Treatment observation instrument. During the training all participating
teachers watched and scored a video recorded lesson. The results of
Prior to the start of the first module of the master course all these observation were discussed with the intention to find
students were assigned to make a video- or a digital recording of agreement between observers. Subsequently a second video
one of their lessons. During the first lesson of the module all recorded lesson was processed in exactly the same way. At the end
448 H.T.G. Van den Hurk et al. / Teaching and Teacher Education 60 (2016) 444e451

of the training sessions the mutual consensus between observers sizes from 0.20 up to 0.49 SD can be seen as small effects, effect
was 0.70 and all scores showed an acceptable agreement with the sizes from 0.50 up to 0.79 SD are judged as medium effects and
norm with differences of less than 0.20 of a standard deviation. effect sizes form 0.80 and above can be considered large effects.

3.5. Variables 4. Results

The empirical evidence from almost five decades of teacher In the present study we tried to answer the research question
effectiveness research was used in the construction of the ICALT whether it would be possible to accelerate the development of
(International Comparative Analysis of Learning and Teaching) effective teacher behavior through the use of data-feedback in a
observation instrument that was used in several national and in- cyclic model of data-driven teaching. We were specifically inter-
ternational studies (Van de Grift & Lam, 1998; Van de Grift, 2007; ested in the possibility to accelerate the more difficult and complex
Van de Grift et al., 2011). The 32 items of this standardized obser- teacher behavior. In order to answer our research question, we
vation instrument, all referring to effective teacher behavior, are conducted repeated measures t-tests to evaluate the difference
divided into six categories: Creating a safe and stimulating learning between pre-test and post-test measures on the ICALT observation
climate (4 items), Efficient classroom management (4 items), Clear scales. On average the time span between pre-test and post-test
instruction (7 items), Activating learning (7 items), Adaptation of was 43 days, just over six weeks. The results of the analyses are
Teaching (4 items) and Teaching learning strategies (6 items). The displayed in Table 2. On the pre-test observations the average
items can be scored on a 4-point scale: predominantly weak (0), scores on Creating a safe and stimulating learning climate
more weaknesses than strengths (1), more strengths than weak- (M ¼ 2.45, SD ¼ 0.56), Efficient classroom management (M ¼ 2.22,
nesses (2) and predominantly strong (3). In the hard-copy version SD ¼ 0.57) and Clear instruction (M ¼ 2.03, SD ¼ 0.56) reached
of the observation instrument, every single item is illustrated with values between 2.00 and 3.00, indicating that, although an
examples of ‘good practice’, added to focus the observers’ attention adequate level of performance was already observed, these cate-
and to facilitate the scoring procedure. For instance, one of the gories of teacher behavior were still open for improvement. The
items from the category ‘Efficient classroom management’ is ‘the average scores in the categories of Activating learning (M ¼ 1.74,
teacher uses learning time efficiently’. This item is illustrated by SD ¼ 0.61), Adaptation of teaching (M ¼ 1.43, SD ¼ 0.96) and
three examples of good practice, namely: ‘there is no loss of time at Teaching learning strategies (M ¼ 1.25, SD ¼ 0.74) reached values
the start, during or at the end of the lesson’, ‘there are no dead between 1.00 and 2.00. The scores in this range indicate that an
moments’ and ‘the children are not left waiting’. improvement of teacher behavior in these specific categories is
required to sufficiently meet the needs of all the pupils in the
3.6. Data analyses classroom.
On the post-test observations, we found a significant increase on
In order to be sure that the construct of effective teaching the average scores on all of the ICALT scales, with effect sizes
behavior could reliably be measured by the ICALT observation in- ranging from 0.29 to 0.76. On the post-test observations the average
strument, the internal consistency of the different scales again was scores of the participating teachers on Activating learning, one of
measured using Cronbach’s Alpha (Cronbach, 1951). As shown in the three categories in which an improvement was required, indi-
Table 1, all of the scales proved to be consistent. The results on pre- cated that an adequate level of performance was reached (M ¼ 2.18,
test and post-test measures, gathered with the ICALT observation SD ¼ 0.55). At the same time the standard deviations of the first
instrument, were compared by using repeated measures t-tests for four categories indicate that some of the participating teachers fall
comparing means. In order to interpret the differences that were short in these relatively easy to master categories of teacher
found, effect sizes were calculated. According to Cohen (1988), ef- behavior. The standard deviations of the last two categories of
fect sizes up to 0.19 SD are interpreted as negligible effects, effect teacher behavior show an even larger spread in scores, indicating
that a fair part of the participating students still score well below
mastery level.
Table 1 Although an increase of the mean scores on all of the assessed
Reliability of the ICALT observation scales (n ¼ 110). categories of effective teacher behavior was found on the post-test
Cronbach’s alpha measurements, the trend in the mutual proportions of the scores
Pre-test Post-test on the different categories roughly stayed the same over the two
measurements (Fig. 2).
1 Creating a safe and stimulating learning climate (4) 0.79 0.79
2 Efficient classroom management (4) 0.70 0.69
3 Clear instruction (7) 0.77 0.77 5. Conclusion and discussion
4 Activating learning (7) 0.73 0.77
5 Adaptation of teaching (4) 0.82 0.73 Recent developments in the Netherlands have led to an educa-
6 Teaching learning strategies (6) 0.81 0.84
tional environment which is becoming increasingly more complex.

Table 2
Results on the ICALT observation scales (n ¼ 110).

Pre-test Post-test Difference

M (SD) M (SD) Sign. ES

1 Creating a safe and stimulating learning climate (4) 2.45 (0.56) 2.60 (0.48) 0.01 0.29
2 Efficient classroom management (4) 2.22 (0.57) 2.42 (0.52) 0.01 0.37
3 Clear instruction (7) 2.03 (0.56) 2.35 (0.47) 0.01 0.62
4 Activating learning (7) 1.74 (0.61) 2.18 (0.55) 0.01 0.76
5 Adaptation of teaching (4) 1.43 (0.96) 1.89 (0.84) 0.01 0.51
6 Teaching learning strategies (6) 1.25 (0.74) 1.78 (0.75) 0.01 0.71
 H.T.G. Van den Hurk et al. / Teaching and Teacher Education 60 (2016) 444e451 449

3.00 The results of earlier studies have indicated that the quality of
teacher behavior grows over time as teaching experience increases
2.50 (Shagrir, 2012; Van de Grift et al., 2011). At the same time we know,
from the results of cross sectional studies in several European
2.00 countries, that it roughly takes 15e20 years of experience to
Pretest develop the most difficult teaching skills (Van de Grift, 2007; Van
1.50 PosƩest de Grift et al., 2011). Furthermore a recent study in secondary ed-
Linear (Pretest) ucation proved that the ‘natural development’ of teaching skills is
1.00 Linear (PosƩest) about 25% of a standard deviation (Maulana, Helms-Lorenz, & van
de Grift, 2015). The results of the current study indicate that the
0.50 natural development of teaching skills can be accelerated. This is an
extremely important finding given the fact that novice teachers
0.00 have to cope with an educational environment which is becoming
1 2 3 4 5 6 increasingly more complex. Speeding up the professional devel-
opment of these staff members is essential in maintaining and
Fig. 2. Results on the ICALT observation scales (n ¼ 110).
improving the quality of the educational system. Furthermore, the
results of this study indicate that this accelerated development
At the same time many experienced teachers are set to retire from seems to be caused by the application of the treatment in which
education in the years to come, as a result of which a large number data-feedback was placed in a cyclic model of data driven teaching.
of less experienced and less skillful novice teachers will enter ed- As earlier research has indicated, teachers who gather data about
ucation. This makes it worthwhile to find out whether teachers’ their own teaching practices and use these data to improve the
professional development can be accelerated in order to maintain effectiveness of their teaching behavior, will eventually improve
the high quality educational system that we strive for. Furthermore their students’ learning and achievement (Timperley et al., 2008).
we take into account that data-driven teaching is an important The application of the model for data-feedback ensured the
lever of change in the process of teachers’ professional learning and connection of ‘real-life’ data obtained in the students’ own pro-
development. In this study we answered the research question it fessional environment with evidence from research on teacher
would be possible to accelerate the development of effective effectiveness. The participating students were able to actively apply
teacher behavior through the use of data-feedback in a cyclic model and practice newly acquired knowledge and skills in their everyday
of data-driven teaching. The quality of teacher behavior was professional practice. This is a very important aspect, since earlier
observed with the ICALT observation instrument, containing 32 research has shown that a close connection between theory and
items divided into six scales: Creating a safe and stimulating professional practice is an important condition for professional
learning climate, Efficient classroom management, Clear instruc- learning (Timperley et al., 2008; Wayman & Jimerson, 2014).
tion, Activating learning, Adaptation of Teaching and Teaching Another interesting finding in the current study is that the trend in
learning strategies (Van de Grift & Lam, 1998; Van de Grift, 2007; the mutual proportions of the scores on the different categories of
Van de Grift et al., 2011). effective teacher behavior roughly stayed the same over the two
The results of the current study showed a sufficient internal measurements. This finding also seems to be in line with the results
consistency of the different scales of the ICALT observation instru- of earlier research where was found that the different categories of
ment. The analyses of pre-test and post-test observation results teacher behavior can be arranged in an ascending sequence ac-
proved that the participating students on average succeeded in cording to their level of difficulty. Put in other words, there is a
significantly improving their scores on all of the assessed categories specific sequence in which teachers will master these subsequent
of effective teaching behavior. The effect sizes of the gains on the categories of teacher behavior (Van de Grift et al., 2011).
post-test observations ranged from d ¼ 0.29 to d ¼ 0.76. The results Some limitations of the study should be discussed. Although we
were obtained after an intervention at a master course for teachers are convinced of the benefits of the application of the model of
where a model of data-driven decision making was used with a data-based decision making in our master course, the application of
strong emphasis on the connection between the participating a simple one group pre-test post-test design calls for some
teachers’ professional practice and the theory of effective teacher consideration. Due to the absence of a control condition, we can’t
behavior. The participating teachers succeeded in improving their be sure that the discovered effects are caused by the independent
teaching behavior in a relatively short time span of 43 day on variable in this study. In their final assignments however, the stu-
average which is a time span just slightly over six weeks. These dents reflected on what they saw as probable causes for the ob-
results are in line with the results of our previous work, where we tained results. A large part of the participating students mentioned
found that student teachers in initial teacher training were able to the fact that they were explicitly challenged to formulate alterna-
significantly improve the quality of their interactive storybook tive teaching behavior, as an important lever of change. Another
reading lessons over a relatively short period of time (Van den Hurk important reason was found in the opportunities the students were
et al., 2014). We have to bear in mind that this is the first time that given to employ the alternative teacher behavior in their own
participating teachers are confronted with the processing of data- professional practice. It goes without saying that in future studies,
feedback on their own instructional behavior. In one of the whenever possible, the application of an experimental design
following modules the same procedure is repeated when the stu- should be considered. A second point of concern is the fact that the
dents again are challenged to improve their instructional behavior, recorded lessons are observed and judged by fellow students. As
this time in literacy lessons. It is fair to state that by the end of the stated in 3.4 a lot of time and effort was spent in a training of the
master course they have the skills to execute the routines and they use of the ICALT observation instrument. The acceptable interrater
have gained some valuable experiences in doing so. Completing the reliability that was reached in these training sessions does not
procedure of data-feedback in a cyclic model of data-driven assure that all the other results of the observed lessons are
teaching two times in this master course however, does not completely without bias.
ensure that teachers will make use of the procedure in the context However important this average acceleration of teacher devel-
of their own professional practice. opment may be, it is important to keep in mind that there is a large
450 H.T.G. Van den Hurk et al. / Teaching and Teacher Education 60 (2016) 444e451

spread in individual scores in the different categories. This means principles and the design of quality tools for educators. Eugene, OR: University of
Oregon.
that a considerable group of teachers still scores well below target,
EP-Nuffic. (2015). Education system the Netherlands. Retrieved from https://www.
specifically in the more difficult categories of teaching behavior. epnuffic.nl/en/publications/find-a-publication/education-system-the-
Further research is needed to find out whether or not for these netherlands.pdf.
teachers an extended intervention could be successful. Further- Evertson, C. M., Anderson, C. W., Anderson, L. M., & Brophy, J. E. (1980). Relation-
ships between classroom behaviors and student outcomes in junior high
more we are very curious whether the obtained results are sus- mathematics and English classes. American Educational Research Journal, 17(1),
tainable over time. This means that in a future study we could use a 43e60.
third, and possibly even a fourth observation moment, scheduled Evertson, C. M., & Weinstein, C. S. (2013). Handbook of classroom management:
Research, practice, and contemporary issues. Mahwah, NJ: Laurence Erlbaum
some time after the participating students have finished the master Associates.
course. In addition it is interesting to find out whether or not there Fischer, C., Berliner, D., Filby, N., Marliave, R., Cahen, L. S., & Dishaw, M. M. (1980).
is a transfer off skills to the participating teachers own professional Teaching behaviors, academic learning time, and student achievement: An
overview. In C. Denham, & A. Lieberman (Eds.), Time to learn (pp. 7e32).
context. In other words do the participating teachers still make use Washington DC: Department of Education and the National Institute of
of some sort of data-feedback procedures concerning their own, or Education.
their colleagues’, instructional behavior. Fullan, M. (2007). Change the terms for teacher learning. Journal of Staff Develop-
ment, 28(3), 35e36.
The results of this study lead to the conclusion that it is possible Ghaith, G. (2003). The relationship between forms of instruction, achievement and
to significantly improve relevant teacher behavior through the use perceptions of classroom climate. Educational Research, 45(1), 83e93.
of data-feedback in a cyclic procedure of data-driven teaching. It is Gummer, E. S., & Mandinach, E. B. (2015). Building a conceptual framework for data
literacy. Teachers College Record, 117(4), 1e12.
striking that these improvements occur over a relatively short
Guskey, T. R. (2002). Professional development and teacher change. Teachers and
period of time. In the meantime course evaluations have indicated Teaching: Theory and Practice, 8(3), 381e391.
that both lecturers and participating students experience positive Guskey, T. R., & Yoon, K. S. (2009). What works in professional development. Phi
developments in students’ knowledge and skills. Therefore we will Delta Kappan, 90(7), 495e500.
Haager, D., & Klingner, J. K. (2005). Differentiating instruction in inclusive classrooms:
continue to use the procedure in our master course for teachers. At The special educator’s guide. Columbus, OH: Merrill.
the same time we are convinced that novice teachers could capi- Hampton, S. E., & Reiser, R. A. (2004). Effects of a theory-based feedback and
talize on the advantages of the implementation of this procedure in consultation process on instruction and learning in college classrooms. Research
in Higher Education, 45(5), 497e527.
their current professional practice. Therefore it could be worth- Harris, M. J., & Rosenthal, R. (1985). Mediation of interpersonal expectancy effects:
while to incorporate a procedure of data-feedback in a cyclic model 31 meta-analyses. Psychological Bulletin, 97(3), 363.
of data driven teaching in novice teacher coaching programs or in Hattie, J. (2009). Visible learning: A synthesis of 800 meta-analyses on achievement.
London: Routledge.
teacher induction projects. Hattie, J., & Clinton, J. (2008). Identifying accomplished teachers: A validation study.
In L. C. Ingvarson, & J. Hattie (Eds.), Assessing teachers for professional certifica-
tion: The first decade of the National Board for Professional Teaching Standards
References (Vol. 11, pp. 313e344). Amsterdam: Elsevier Press.
Houtveen, A. (2007). Leren lezen is te leren. In Reading can be learned. Utrecht:
Anderson, L. M., Evertson, C. M., & Brophy, J. E. (1979). An experimental study of Kenniscentrum Educatie.
effective teaching in first-grade reading groups. The Elementary School Journal, Houtveen, A., Booij, N., De Jong, R., & Van de Grift, W. (1999). Adaptive instruction
79(4), 193e223. and pupil achievement. School Effectiveness and School Improvement, 10(2),
Ateah, C., & Cohen, I. (2009). School victimization and bullying experiences: Cross- 172e192.
national comparisons between Canada and the United States. Currents: Schol- Houtveen, T., & Van de Grift, W. (2001). Inclusion and adaptive instruction in
arship in the Human Services, 8(1), 1e22. elementary education. Journal of Education for Students Placed at Risk, 6(4),
Bandura, A. (1997). Self-efficacy: The exercise of self-control. New York: Freeman. 389e409.
Bennett, C. I. (1986). Comprehensive multicultural education: Theory and practice. Houtveen, A., & Van de Grift, W. (2007). Effects of metacognitive strategy instruc-
Boston: Allyn and Bacon. tion and instruction time on reading comprehension. School Effectiveness and
Brophy, J., & Good, T. L. (1986). Teacher behavior and student achievement. In School Improvement, 18(2), 173e190.
M. C. Wittrock (Ed.), Handbook of research on teaching (pp. 328e375). New York: Houtveen, A., & Van der Velde, V. (2010). Rapportage Pilots Taalbeleid onderwij-
Macmillan. sachterstanden over de schooljaren 2007-2008 en 2008-2009. Utrecht: Kennis-
Carnine, D., Dixon, R., & Silbert, J. (1998). Effective strategies for teaching mathe- centrum Educatie.
matics. In E. Kameenui, & D. Carnine (Eds.), Effective teaching strategies that Kameenui, E. J., & Carnine, D. (Eds.). (1998). Effective teaching strategies that
accommodate diverse learners (pp. 93e112). Columbus, OH: Merrill. accommodate diverse learners. Upper Saddle River, NJ: Prentice Hall.
Carroll, J. (1963). A model of school learning. The Teachers College Record, 64(8), Kidron, Y., & Lindsay, J. (2014). The effects of increased learning time on student ac-
723e723. ademic and nonacademic outcomes: Findings from a meta-analytic review (REL
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). 2014-015). Washington DC: Regional Educational Laboratory Appalachia.
Hillsdale, NJ: Lawrence Earlbaum Associates. Lang, J. W., & Kersting, M. (2007). Regular feedback from student ratings of in-
Cohen, J., McCabe, L., Michelli, N. M., & Pickeral, T. (2009). School climate: Research, struction: Do college teachers improve their ratings in the long run? Instruc-
policy, practice, and teacher education. The Teachers College Record, 111(1), tional Science, 35(3), 187e205.
180e213. Latham, G. P., & Locke, E. A. (1991). Self-regulation through goal setting. Organiza-
Cotton, K. (1995). Effective schooling practices: A research synthesis, 1995 update. tional Behavior and Human Decision Processes, 50(2), 212e247.
Portland, OR: Northwest Regional Educational Laboratory. Levine, D. U., & Lezotte, L. W. (1990). Unusually effective schools: A review and
Craig, S. D., Sullins, J., Witherspoon, A., & Gholson, B. (2006). The deep-level- analysis of research and practice. Madison, WI: National Centre for Effective
reasoning-question effect: The role of dialogue and deep-level-reasoning School Research and Development.
questions during vicarious learning. Cognition and Instruction, 24(4), 565e591. Levine, D. U., & Lezotte, L. W. (1995). Effective schools research. In J. A. Banks, &
Creemers, B. P. (1994). The effective classroom. London: Cassell. C. A. M. Banks (Eds.), Handbook of research on multicultural education (pp.
Creemers, B. P., & Reezigt, G. J. (1996). School level conditions affecting the effec- 525e547). New York: Macmillan.
tiveness of instruction. School Effectiveness and School Improvement, 7(3), Lohman, D. (1986). Predicting mathematanic effects in the teaching of higher-order
197e228. thinking skills. Educational Psychologist, 21, 191e208.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psycho- Mantione, R. D., & Smead, S. (2003). Weaving through words: Using the arts to teach
metrika, 16(3), 297e334. reading comprehension strategies. Newark, DE: International Reading
Davis, H. A. (2003). Conceptualizing the role and influence of student-teacher re- Association.
lationships on children’s social and cognitive development. Educational Psy- Marsh, J. (2012). Interventions promoting educators’ use of data: Research insights
chologist, 38(4), 207e234. & gaps. Teachers College Record, 114(11), 1e48.
Denham, C., & Lieberman, A. (1980). Time to learn. A review of the beginning teacher Marzano, R. J. (2003). What works in schools: Translating research into action.
evaluation study. Washington, DC: National Institute of Education. Alexandria, VA: ASCD.
Dixon, R., Carnine, D., Lee, D., Wallin, J., & Chard, D. (1998). Review of high quality Marzano, R. J. (2007). The art and science of teaching: A comprehensive framework for
experimental mathematics research: Report to the California state board of edu- effective instruction. Alexandria, VA: ASCD.
cation. Eugene, OR: National Center to Improve the Tools of Educators. Maulana, R., Helms-Lorenz, M., & van de Grift, W. (2015). Development and eval-
Dusek, J. B., & Joseph, G. (1983). The bases of teacher expectancies: A meta-analysis. uation of a questionnaire measuring pre-service teachers’ teaching behaviour:
Journal of Educational Psychology, 75(3), 327e346. A rasch modelling approach. School Effectiveness and School Improvement, 26(2),
Ellis, E. S., & Worthington, L. A. (1994). Research synthesis on effective teaching 169e194.
 H.T.G. Van den Hurk et al. / Teaching and Teacher Education 60 (2016) 444e451 451

Meeuwisse, M., Severiens, S. E., & Born, M. P. (2010). Learning environment, inter- Slavin, R. E. (1996). Education for all. Lisse, the Netherlands: Swets & Zeitlinger.
action, sense of belonging and study success in ethnically diverse student Slavin, R. E., & Braddock, J. H. I., II (1993). Ability grouping: On the wrong track.
groups. Research in Higher Education, 51(6), 528e545. College Board Review, 168, 11e17.
Min. OC&W. (2004). Wet op de beroepen in het onderwijs [Professions in Education Smith, T., Baker, W., Hattie, J., & Bond, L. (2008). A validity study of the certification
Act]. Zoetermeer: Ministerie van OC&W. system of the national board for professional teaching standards. In
Min. OC&W. (2013). Lerarenagenda 2012-2020. De leraar maakt het verschil [Teachers L. C. Ingvarson, & J. Hattie (Eds.), Assessing Teachers for Professional Certification:
Calender 2013-2020. The teacher makes the difference]. Zoetermeer: Ministerie The First Decade of the National Board for Professional Teaching Standards (Vol. 11,
van OC&W. pp. 345e380). Amsterdam: Elsevier Press.
Muijs, D., & Reynolds, D. (2010). Effective teaching: Evidence and practice. London: Teddlie, C., Falkowski, C., Stringfield, S., Desselle, S., & Garvue, R. (1984). The Loui-
Sage. siana school effectiveness Study: Phase two, 1982-84. Baton Rouge LO: State
Nunes, T., & Bryant, P. (1996). Children doing mathematics. Oxford: Wiley-Blackwell. Department of Education.
Osterman, K. F. (2000). Students’ need for belonging in the school community. Thapa, A., Cohen, J., Guffey, S., & Higgins-D’Alessandro, A. (2013). A review of school
Review of Educational Research, 70(3), 323e367. climate research. Review of Educational Research, 83(3), 357e385.
Pearson, P. D., & Fielding, L. (1991). Comprehension instruction. In Handbook of Timperley, H., Wilson, A., Barrar, H., & Fung, I. (2008). Teacher professional learning
reading research (Vol. 2, pp. 815e860). and development. Wellington, New Zealand: Ministry of Education. Available
Pearson, P. D., & Gallagher, M. C. (1983). The instruction of reading comprehension. from: http://www.minedu.govt.nz/goto/bestevidencesynthesis.
Contemporary Educational Psychology, 8(3), 317e344. Van de Grift, W. J. C. M. (2007). Quality of teaching in four European countries: A
Perry, K. E., & Weinstein, R. S. (1998). The social context of early schooling and review of the literature and application of an assessment instrument. Educa-
children’s school adjustment. Educational Psychologist, 33(4), 177e194. tional Research, 49(2), 127e152.
Pressley, M., Wood, E., Woloshyn, V. E., Martin, V., King, A., & Menke, D. (1992). Van de Grift, W. J. C. M. (2014). Measuring teaching quality in several European
Encouraging mindful use of prior knowledge: Attempting to construct countries. School Effectiveness and School Improvement, 25(3), 295e311.
explanatory answers facilitates learning. Educational Psychologist, 27(1), Van de Grift, W., & Lam, J. (1998). Het didactisch handelen in het basisonderwijs
91e109. [Instruction in primary education]. Tijdschrift Voor Onderwijsresearch, 23(3),
Purkey, S. C., & Smith, M. S. (1983). Effective schools: A review. The Elementary 224e241.
School Journal, 83(4), 427e452. Van de Grift, W. J. C. M., Van den Hurk, H. T. G., & Houtveen, A. A. M. (2016). Effective
Rosenshine, B. V., & Meister, C. (1997). Cognitive strategy instruction in reading. In Teaching. Empirical evidence for the construction of the ICALT observation instru-
S. A. Stahl, & D. A. Hayes (Eds.), Instructional models in reading (pp. 85e107). ment (Manuscript in preparation).
Hilsdale, NJ: Lawrence Erlbaum. Van de Grift, W., Van der Wal, M., & Torenbeek, M. (2011). Ontwikkeling in de
Rosenshine, B., & Stevens, R. (1986). Teaching functions. In M. C. Wittrock (Ed.), pedagogisch didactische vaardigheid van leraren in het basisonderwijs
Handbook of research on teaching (3rd ed., pp. 376e391). New York: Macmillan. [Development of teacher skills in primary education]. Pedagogische Studi€ en, 88,
Sammons, P., Hillman, J., & Mortimore, P. (1995). Key characteristics of effective 416e432.
schools. A review of school effectiveness research. London: OFSTED. Van den Hurk, H. T. G., Houtveen, A. A. M., Van de Grift, W. J. C. M., & Cras, D. W. P.
Scheerens, J. (1992). Effective schooling: Research, theory and practice. London: (2014). Data-feedback in teacher training. using observational data to improve
Cassell. student teachers’ reading instruction. Studies in Educational Evaluation, 42,
Scheerens, J., & Bosker, R. J. (1997). The foundations of educational effectiveness. 191e206.
Oxford: Pergamon. Walberg, H. J., & Haertel, G. D. (1992). Educational psychology’s first century. Journal
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi- of Educational Psychology, 84(1), 6e19.
experimental designs for generalized causal inference. Boston, MA: Houghton Walmsley, S. A., & Allington, R. L. (1995). Redefining and reforming instructional
Mifflin. support programs for at-risk students. In R. L. Allington, & S. A. Walmsley (Eds.),
Shagrir, L. (2012). How evaluation processes affect the professional development of No quick Fix: Rethinking literacy programs in America’s elementary schools (pp.
five teachers in higher education. Journal of the Scholarship of Teaching and 19e44). Newark, DE: International Reading Association.
Learning, 12(1), 23e35. Wayman, J. C., & Jimerson, J. B. (2014). Teacher needs for data-related professional
Shah, A. K., & Oppenheimer, D. M. (2008). Heuristics made easy: An effort-reduction learning. Studies in Educational Evaluation, 42, 25e34.
framework. Psychological Bulletin, 134(2), 2007e2022.