Journal of Education for Teaching

International research and pedagogy

ISSN: (Print) (Online) Journal homepage: www.tandfonline.com/journals/cjet20

Enhancing teachers’ STEM understanding through

observation, discussion and reflection

Xiao Huang, Sibel Erduran, Piaosa Zhang, Kangkang Luo & Chumni Li

To cite this article: Xiao Huang, Sibel Erduran, Piaosa Zhang, Kangkang Luo & Chumni Li (2022)
Enhancing teachers’ STEM understanding through observation, discussion and reflection,
Journal of Education for Teaching, 48:5, 576-591, DOI: 10.1080/02607476.2021.2006571

To link to this article: https://doi.org/10.1080/02607476.2021.2006571

© 2022 The Author(s). Published by Informa

UK Limited, trading as Taylor & Francis
Group.

Published online: 07 Feb 2022.

Submit your article to this journal

Article views: 5969

View related articles

View Crossmark data

Citing articles: 7 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=cjet20
JOURNAL OF EDUCATION FOR TEACHING
2022, VOL. 48, NO. 5, 576–591
https://doi.org/10.1080/02607476.2021.2006571

Enhancing teachers’ STEM understanding through

observation, discussion and reflection
Xiao Huanga, Sibel Erduranb, Piaosa Zhangc, Kangkang Luoc and Chumni Lid
a
College of Teacher Education, Zhejiang Normal University, Jinhua, China; bDepartment of Education,
University of Oxford, Oxford, United Kingdom; cSchool of Educational Studies, Universiti Sains Malaysia,
Minden, Malaysia; dDepartment of Physics, Beijing Normal University, Beijing, China

ABSTRACT ARTICLE HISTORY

Effective teaching of STEM has become a significant concern for Received 14 February 2021
teachers’ professional development (PD). Many teachers are not Accepted 10 August 2021
familiar with STEM strategies and do not possess the disciplinary KEYWORDS
knowledge or pedagogical strategies demanded by STEM teaching. STEM literacy; stem
The purpose of the study reported in this paper was to explore the education; classroom
path and model of Chinese teachers’ professional development observation; discussion in
about STEM understanding based on their participation in the group; video reflection; ODR
Zhejiang-Indiana STEM summer programme. This programme capi­
talises on strategies referred to as the observation-discussion-
reflection (ODR) framework. The participants were 82 teachers
who specialise in science, technology, or mathematics and who
attended the training activities. There were two weeks of training
interventions. Statistical analysis of two groups of teachers’ pre-test
and post-test, as well as the comparative analysis of two groups,
were conducted. The changes in teachers’ STEM discipline knowl­
edge as well as their abilities and attitudes were investigated.
Overall, the findings indicate variations in how teachers understand
the different components of STEM knowledge with integrated
knowledge requiring the most attention for further professional
development.

Introduction
Since the Science, Technology, Engineering, and Mathematics (STEM) Education Integration
framework (originally referred to as SEM& T) was proposed by the National Science Board
(NSB) in the USA in 1986, STEM education has become a widespread agenda globally.
Policy makers and educational leaders have argued that the key to future prosperity is
improving STEM teaching and learning opportunities (National Academy of Sciences, et
al. 2007). In 1996, the National Science Foundation (NSF) reflected on STEM education in
a ten-year review and summarised that focusing on ‘the needs of college students in
various two-year and four-year colleges in the United States’ requires ‘cultivating K-12
education system SEM&T’s faculty issues’. The Innovative America: Developing Science,
Technology, Engineering and Maths Agenda proposed by the US Governors Association
in 2007 not only explains the background, current status, problems and strategies of STEM

CONTACT Sibel Erduran [email protected]

© 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License
(http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any med­
ium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.
 JOURNAL OF EDUCATION FOR TEACHING 577

implementation but also points out the challenges teachers face in adopting pedagogical
strategies that support effective learning of STEM. One concern is that teachers have
limited background in STEM and disciplinary orientations demanded by STEM. Many
educational systems around the world currently face similar challenges, and different
educational systems advocate models of STEM teachers’ professional development
(Hayden et al. 2011; Hsu and Yeh 2020; Taningco, Mathew, and Pachon 2008).
In this paper, recent research literature on STEM education is reviewed, particularly in
relation to characterising STEM literacy and the role of teachers’ professional develop­
ment in its achievement. The research literature illustrates the components of STEM
teaching and learning, and the demands that STEM places on secondary school teachers.
The discussion is situated in the context of China where a professional development
project was carried out with the objective of improving teachers’ STEM knowledge and
attitudes towards STEM. Findings from the empirical study is reported focusing on the
impact of a professional development intervention.

Review of literature
One goal of STEM education is to cultivate STEM literacy for all learners (Bybee 2010,
2013). Different scholars have different views on the characteristics and definition of STEM
literacy. Bybee (2010) argues that STEM literacy includes conceptual understanding,
procedural skills and abilities for individuals to address STEM-related personal, social,
and global issues. Some researchers elaborated on the definition of STEM literacy as
pertaining to skills that are demanded in the 21st century which include problem-solving
skills, social communication skills, technology and engineering skills, system skills, and
time, resource, and knowledge management skills as the key competencies for STEM
workers (Morrison 2006). They proposed that STEM literacy is a compound literacy, which
is the integration and expansion of the four specialisations (i.e. scientific, technical,
engineering and mathematical literacy). According to Zollman (2012), the ultimate goal
of STEM education should change from learning STEM literacy to using STEM literacy for
learning. The implications of research and policy literature is that STEM literacy can be
divided into three dimensions: STEM knowledge, STEM ability, and STEM attitude.
STEM literacy requires students and future citizens to develop the competencies to
apply basic content and practices in STEM disciplines within the situations they encoun­
ter. In order to meet the challenges in 21st century, different scholars have given different
opinions on the abilities that students need to have. There are many similarities between
STEM literacy and STEM competencies. However, STEM competencies focus more on the
skills needed for future careers whereas STEM literacy is more comprehensive. The
promotion of STEM literacy requires explicit education, which demands teachers’ STEM
literacy. Consequently, teachers’ understanding of STEM affects students’ achievement
(Yoon et al. 2007; Capraro 2013). In order to achieve effective STEM teaching, teachers
must have content knowledge (i.e. four disciplines of STEM), pedagogical content knowl­
edge (PCK) of STEM content and pedagogical strategies (e.g. problem-solving, scientific
inquiry, engineering design). Furthermore, their attitudes are important in defining how
they will approach STEM teaching (Eckman, Williams, and Silver-Thom 2016). Moreover, to
develop students’ STEM literacy through teaching, teachers must be informed of STEM
literacy and teaching ability, which requires high-quality professional development.
578 X. HUANG ET AL.

However, many teachers are not qualified for STEM teaching and have limited under­
standing of STEM (Czerniak and Johnson 2014). In the case of science teachers, there is
a finite awareness of technology and engineering (National Governors Association 2011;
National Research Council 2011; Thibaut, et al. 2018).

Approaches to improvement of teachers’ understanding of STEM

Compared to the traditional transmission model of teaching, STEM education places
higher demands on teachers and requires teachers to teach knowledge skills as well as
skills such as critical thinking and problem-solving (Corlu, Capraro, and Capraro 2014).
However, a significant proportion of training currently received by teachers internation­
ally includes only one subject, such as science or mathematics. Hence, there is an urgency
for effective STEM PD to help teachers develop STEM literacy and improve STEM under­
standing (Ye and Yang 2018). Teacher training programmes can include mentoring,
workshops, coaching, support groups, and online training (Pfeffer and Sutton 2000).
Most PD courses focus on pedagogical content knowledge (PCK) and the attitudes of
teacher participants (National Research Council 2015). Effective PD requires training with
concrete tasks of teaching, assessment, discussion, observation and reflection (Darling-
Hammond and McLaughlin 2011). Postholm (2008) proposed that reflection is the key to
teachers’ learning and development of teaching experience and showed that teachers
can reflect before action, in action and on action, connecting theoretical concepts with
their teaching practice (p.17).
Furthermore, Shernoff et al. (2017) argued that improving teachers’ STEM understand­
ing should give in-service teachers opportunities to observe out-of-district teachers and
have more time for teacher collaboration and see examples of effective lessons. Therefore,
using observation as a form of professional development improves teaching practices and
professional competence (Alshehri 2019). Moreover, teachers’ discussion plays an impor­
tant role in teachers’ professional development training and provides opportunities for in-
depth peer-to-peer interaction. The approaches to effective STEM teacher PD include
continuous, participatory teacher training, theoretical study, field trips and listening
courses, and STEM professional training requires teachers to go through scientific
research and engineering design processes before teaching (Crippen, Biesinger, and
Ebert 2010; Brown and Crippen 2018). The observation-discussion-reflection (ODR)
approach, as an effective way to promote professional development in STEM education,
can keep teachers active in personal learning and understanding STEM and provide
a framework within which to share their own discussion, reflections and observations.
The purpose of the ODR framework is to extract the main approaches of PD training
that affect teachers' STEM understanding. The most direct and basic approach to improve
teaching practices is to allow teachers to observe high-quality courses (American
Association for the Advancement of Science, 2012). Then, observers and participants
discuss both students’ learning ways and teachers’ instructional strategies through the
STEM observation forms, lesson design, lesson core content (Borko, et al. 2011). Reflection
helps foster meaningful discussions and potentially improve teachers’ pedagogy. All
groups have experienced the process of ODR, but the specific content and task require­
ments may be different depending on the cohort of teachers. ‘The Chinese Lesson Study
 JOURNAL OF EDUCATION FOR TEACHING 579

or Parallel Classroom Study’ refers to a type of professional development that includes

certain topics, lesson plans, classroom observation, collaborative discussion, post-lesson
debriefing and reflection (Yang and Ricks 2012; Huang, et al. 2014).

STEM literacy and professional development of Chinese teachers

Despite a wealth of reports on STEM education and PD in STEM teaching in China, there is
paucity of research on how Chinese teachers are being prepared for improving STEM
teaching and few studies on teachers’ STEM literacy. One of the few empirical studies on
the status of STEM education in Chinese primary and secondary schools, Huang, et al.
(2020) reports that STEM teachers are mostly young and middle-aged and that their
understanding of STEM needs to be deepened. For behavioural intention and behaviour
attitude, the performance of science and engineering teachers is better than that of liberal
arts teachers. In terms of self-efficacy and output quality, engineering teachers are
significantly further along. Furthermore, focusing on the status of Zhejiang teachers, the
same study showed that the understanding of the discipline concept (four disciplines of
STEM) is not balanced and that teachers do not have a sufficient understanding of
mathematics in STEM. Problem-solving ability, technical application and mathematical
modelling were also limited. Moreover, the study of our team (Huang, et al. in press)
showed that the overall situation of STEM literacy of teachers in Zhejiang Province
urgently needs to be improved. Problems of preservice teachers’ STEM literacy, which
includes knowledge understanding, problem solving ability and STEM attitudes related to
different majors, grades and genders, are highlighted. Similar problems existed in the
survey of in-service teachers; they performed well in the areas of disciplinary knowledge,
but they lacked problem solving (PSA) and STEM career interest.
The professional development of STEM teachers in Zhejiang Province aims at preser­
vice teachers and in-service teachers, including expert lectures, workshops, exchanges
and exhibitions among teachers after practice and organises to visit STEM schools in the
US. During 2017–2019, ZIPCP was aimed at Zhejiang primary and secondary school
teachers for two weeks during summer vacation. In 2019, the STEM education training
conference and Zhejiang STEAM Education Conference, China-US-Canada STEAM
Advanced Training Conferences were held for improving teachers’ STEM theory and
teaching practice, especially STEM literacy and PCK for STEM. The research and practice
of STEM education in Zhejiang has also become an important concern because of its
leading economy and education system in China. In this paper, we focus on in-service
teachers’ STEM literacy improvement through the Zhejiang-Indiana Parallel Classroom
Project (ZIPCP), which emphasises the introduction of international STEM teaching para­
digms, teaching practices highlighting local culture and teachers’ reflections.

Methodology
Research questions
The empirical study was guided by the following primary research question: How does the
ODR approach impact Chinese teachers’ professional development in STEM?
580 X. HUANG ET AL.

Professional development context

The context for professional development is ZIPCP, sponsored jointly by Zhejiang
Department of Education and Indiana Department of Education. The objectives of
ZIPCP are to improve teacher’s STEM literacy and to equip teachers with appropriate
skills to design STEM education activities. Since STEM education has been practised in the
United States for a long time, STEM courses, designed by teachers from Indiana teacher
teams, were put into practice in Chinese schools in this project guided by their original
designers. As STEM education has become an important topic in compulsory education in
China, teachers from primary and secondary schools in Zhejiang province participated in
the ZIPCP program to get support for their STEM teaching. With this approach, teachers
have an opportunity to redesign STEM activities and to learn how to implement inter­
disciplinary activities in their own classrooms. Supervisors from government as well as
professors from universities facilitated the sessions. The training materials and activities
were developed collaboratively by teacher teams who had experience in STEM education
and researchers who focused on STEM education.

Design and procedures

The study was divided into four stages: pre-test, teaching intervention, post-test and
semi-structured interviews. A previous analysis was used to establish Zhejiang teachers’
STEM literacy (Huang, et al. in press). PD interventions with the process of observation,
discussion and reflection were carried out through different training sessions. In this
paper, we focus on the quantitative measures to investigate the effect of the intervention,
but we are providing an overview of the entire data set to provide the context for the data
and data analysis.

Selection of participants
The study focuses on 82 teachers from ZIPCP. Teachers come from primary and secondary
schools, and most of them have some understanding of STEM education. These teachers
had different academic backgrounds, including science, mathematics, technology and
English. Twenty-eight teachers participated in the first week of training, named Group
A. In addition, 8 teachers chosen from Group A participated in the training of the second
week. Group B had 54 teachers who only participated in the second week. Table 1 shows
the numbers and topics of different groups.

Table 1. Number of teachers and topics of different groups.

Number

Variables Category Group A Group B

Theme Building a Car 7 13
Buildings and bridges 7 13
Aeronautical Engineering 7 14
Series and parallel circuit 7 14
 JOURNAL OF EDUCATION FOR TEACHING 581

Teaching intervention
There were 4 topics of the STEM integration curriculum for every teacher to choose (see
Table 1). Building a Car required students to use their knowledge of magnetic forces to
create a car that moves through a student-created obstacle course. In Building a Bridge
courses, students explored and manipulated shapes as they relate to natural and man-
made structures. Aeronautical engineering required designing and building a tumble wing
that travels the farthest distance and establishing a paper jet model. Series and parallel
circuits required learning and creating a series or parallel circuit. All groups have experi­
enced the process of observation, discussion and reflection, but the specific content and
task requirements were different. (see Figure 1 and Figure 2)

Figure 1. Intervention between Group A and Group B.

Figure 2. The framework for observation, group discussion and reflection.

582 X. HUANG ET AL.

Group A: We divided its teaching process into 2 stages. Teachers were asked to
experience a three-step cyclic process of lesson observation (20 hours), group discussion
and video reflection in the first stage. They are asked to observe how American STEM
teachers carry out teaching and record according to the scale. A 2-hour group discussion
with no topic specified was arranged for the teachers to brainstorm and communicate
after the observation. Reflect on American STEM teachers behaviour through video clips
on the basis of classroom observation scale. The second stage is aloud reflection and
report one topic related to STEM teaching. Group B: There were similar lesson observation,
discussion and aloud reflection/report processes. The difference is that the task require­
ments are clarified, questions are clarified (8 hours) before the lesson observation
(12 hours), and the observation and group discussion revolve around designated
questions.

Instruments
The research tools used in this research are mainly the STEM Literacy Status Questionnaire,
Interview Protocol and Observation Recording Table (see Table 2).

Table 2. Research instruments.

Type Collection Means Evaluation Tool
Quantitative data Questionnaire STEM Literacy Status Questionnaire
(Pre- and post-test volume)
Qualitative data Interview Interview Outline of Teacher Interview (Groups A, B)
Interview Transcription Text
Class observation Observation record table
Discussion Instructional design
group discussion results

Questionnaires and interviews

The STEM Literacy Status Questionnaire (pre- and post-test versions) was compiled using
STEM literacy related to scientific, mathematical, technology and engineering literacy
assessment in The National Assessment of Educational Progress. The questionnaire is
mainly used to examine the ability of teachers to comprehensively apply the four con­
cepts of STEM to solve complex problems in real issues, the understanding and attitude
towards STEM education. According to Zollman (2012), the ultimate goal of STEM educa­
tion should change from learning STEM literacy to using STEM literacy for learning.
Combined with Bloom educational goal taxonomy (cognitive domain (knowledge and
process), emotional domain (attitude), psychological field), STEM literacy is divided into
the following dimensions

● STEM knowledge (e.g. Scientific Knowledge. SK; Technical Knowledge, TK;

Engineering Knowledge, EK; Mathematical Knowledge, MK; Integration Knowledge,
IK)
 JOURNAL OF EDUCATION FOR TEACHING 583

● STEM ability (e.g. problem-solving ability, PSA; scientific inquiry ability, SIA; maths
modelling ability, MMA; engineering design ability, EDS; technical application ability,
TAA)
● STEM attitude (Attitudes towards connections between STEM disciplines, ATC;
Attitudes towards effect of STEM education, AE; STEM career interest, CI)

Eighty-two valid questionnaires were used in this survey. After the questionnaires were
collected, we evaluated and coded the teachers’ answers in the questionnaires, and SPSS
was used for statistical analysis. The questionnaire used in this study included quiz and
open questions. The quiz examines the two dimensions of teachers’ STEM concept
understanding and STEM ability (see Table 3). The open questions are based on real-life
situations, such as topics on wind generators, cross-sea bridges and air capture systems. The
following example was using the topic of CO2 Collector to investigate the level of SK (see
Table 4). Different questions (including each sub-problem) have different aspects to
examine. In the pre-test questionnaire, the first question mainly examines the teacher’s
STEM understanding and ability. Considering that this question is biased towards the
investigation of STEM teaching literacy, it contrasts with the second sub-question of the
second question in the post-test questionnaire.

Table 3. Evaluation criteria for conventional questions.

Answer performance Score
The teacher’s response is in complete agreement with the point of view of the questionnaire. 3
The teacher’s answer is partially consistent with the point of view that the questionnaire is to examine. 2
The teacher’s response is not related to the point of view of the questionnaire. 1
The teacher’s answer is contrary to the point of view of the questionnaire.
Teacher did not answer 0

Table 4. Analysis of different levels of SK.

Level Requirement Example
Level Choose the place where the carbon dioxide a)understand the principles of capturing CO2 and
3 concentration is high (such as the factory, crowded know how to deal with it
places like the shopping mall, etc.); the place where b)choose a place with a large proportion of CO2 in
the air flow rate is suitable; know the principle of space, such as the place with large population;
the device usedand explain the reason accurately. c) the device can also be moved to collect CO2 more
quickly, such as near a burning site; an external
device such as an auricle that assists in collecting
gas can be installed.
Level The participants cannot clarify the reasons. 1.installed in the car exhaust where the concentration
2 of CO2 is higher.
2.installed in the factory flue, the concentration of CO2
is higher, the effect is better.
Level The reasons given are incorrect or not related to the Instal the system at the relevant place of the relevant
1 key requirement. enterprise and start the device at a fixed point and
timing according to the production situation of the
enterprise.
584 X. HUANG ET AL.

Data analysis
Data from multiple sources (pre-test, post-test and interview) were analysed. With regard
to the status of teachers’ STEM understanding, we obtained data from participants
(n = 82) in Zhejiang and used descriptive statistics to describe the significance of teachers’
STEM understanding and the aspects that should be emphasised. In addition to the
analysis of teachers’ answers concerning the way of training to their STEM understanding,
the significance of the teachers’ STEM understanding, including knowledge, ability and
attitude, was analysed.
The scoring process was carried out by 6 STEM teachers with theoretical foundations
and years of teaching experience, followed by consistency analysis. In order to analyse the
quiz and the open questions, 6 STEM researchers with theoretical foundations and years
of STEM teaching experience conducted a consensus evaluation according to the stan­
dards in Table 3 and Table 4. The Kendall’s coefficient of concordance reached 0.89 and
0.84, meaning that the results of evaluation were consistent. The consistency of each
question is 0.8 and above, indicating that the text analysis has a relatively high consis­
tency. The Likert scale of the questionnaire is a kind of scoring plus total scale with
scientific and reasonable evaluation methods and is widely used in research evaluation. In
this part, Level 5 is in favour, Level 4 is slightly in favour, Level 3 is general, Level 2 is a little
disapproval and Level 1 is disapproval. Therefore, a higher average score in a category
indicates that the dimension is well mastered.

Results and findings

The results indicate that teachers’ STEM literacy has improved, especially TK, EK and ATC,
CI, and AE. Lesson observation, group discussion and reflection are effective approaches
to improving teachers’ STEM literacy. In the following sections, we detail the evidence that
supports such observations.

Comparative analysis of the STEM knowledge, ability and attitude dimensions

We tested the knowledge, ability and attitudes of different disciplines before and after the
PD intervention, and observed the differences between the dimensions. It can be seen
from Table 5 that the average value of SK (M = −0.163) decreased in the post-test
compared with the pre-test, while the TK(M = 0.272), EK(M = 0.428), MK (M = 0.169),

Table 5. T-test in STEM knowledge and attitude dimensions.

Pre-Post Test (N = 82)

M (SD) T
Mathematical Knowledge (MK) 0.169 (0.754) 2.028*
Scientific Knowledge (SK) −0.163 (0.741) −2.002
Technical Knowledge (TK) 0.272 (0.692) 3.564**
Engineering Knowledge (EK) 0.428 (0.865) 4.482***
Problem-Solving Ability(PSA) −0.131 (0.742) −0.159
Attitudes Towards Connections (ATC) 0.339 (0.929) 3.306**
STEM Career Interest (CI) 0.700 (0.993) 6.373***
Attitudes Towards Effect of STEM Education (AE) 0.998 (2.059) 4.389***
Notes: *p < 0.05; **p < 0.01; ***p < 0.001
 JOURNAL OF EDUCATION FOR TEACHING 585

ATC(M = 0.339), CI(M = 0.700), AE(M = 0.998) increased and TK, EK, ATC, STEM CI, AE
presented p < 0.001, were significant differences. The data show that there is no obvious
effect on the understanding of teachers’ SK and PSA through training, but the under­
standing of MK, TK and EK, ATC, CI, AE has obvious effects.

STEM discipline knowledge

We compared and analysed the understanding and application of teachers’ disciplinary
knowledge in Group A and B. To comprehensively examine the data, we focused on
a statistical analysis of the mastery of the concept of participating teachers from quanti­
tative and qualitative analysis (see Table 6). The average value of Group A SK (M = −0.140)
decreased post-test, while TK (M = 0.214), MK (M = 1.153), and EK (M = 0.321) increased. EK
showed p < 0.05 and was significantly different, but SK, TK and MK were not significantly
different. From the perspective of the distribution ratio of teachers at different levels, the
proportion of the concept understanding of teachers in level 5 is mostly between 50%
and 80%. People at level 5 of EK have increased from 50% to 75%.
The average value of Group B in SK (M = −0.176) decreased, while TK (M = 0.348), EK
(M = 0.484), and MK (M = 0.177) increased. However, EK showed p < 0.01, and TK showed
p < 0.05. The results showed that after training, the EK and TK of group B was significantly
improved, while the levels of MK and SK were not significantly different from those of
group A. The proportion of teachers who had a basic understanding of TK rose from
40.74% to 62.96%. Moreover, EK improved significantly, and the proportion increased
29.63%(see Table 7).

Table 6. Comparison between Group A and B in STEM discipline knowledge.

Group A (N = 28) Group B (N = 54)

M± SD T(po-pr) M+ SD T(po-pr) T(GroupA-B)

MKpre 4.329 ± .511 1.177 4.134 ± .623 1.643 1.419
MKpost 4.482 ± .535 4.311 ± .513 1.410
SKpre 4.655 ± .532 −0.954 4.580 ± .395 −1.777 0.721
SKpost 4.515 ± .503 4.404 ± .639 0.797
TKpre 4.406 ± .409 1.921 4.050 ± .464 2.992* 3.431**
TKpost 4.620 ± .442 4.398 ± .615 \
EKpre 4.161 ± .559 2.588* 4.007 ± .659 3.716** 1.054
EKpost 4.482 ± .486 4.491 ± .608 −0.065
Notes:Pre:Pres-test; Post: Post-test.

Table 7. Analysis of STEM discipline knowledge understanding.

SK TK EK MK
Levels Pre Post Pre Post Pre Post Pre Post
A Level 5 82.1% 82.1% 75.0% 85.7% 50.0% 75.0% 67.9% 71.4%
Level 4 14.3% 14.3% 25.0% 14.3% 46.4% 25.0% 28.6% 28.6%
Level 3 3.6% 3.6% 0.0% 0.0% 3.6% 0.0% 3.57% 0.0%
B Level 5 83.3% 70.4% 40.8% 63.0% 44.4% 74.1% 51.9% 75.9%
Level 4 16.7% 24.1% 53.7% 31.5% 44.4% 22.2% 44.4% 20.4%
Level 3 0.0% 5.6% 5.6% 5.6% 9.3% 3.7% 1.9% 3.7%
Level 2 0.0% 0.0% 0.0% 0.0% 1.85% 0.0% 1.8% 0.0%
There were no members in Group A reaching level 1,2 and Group B in level 1
586 X. HUANG ET AL.

For IK, most teachers lacked an understanding of interdisciplinary concepts, and few
teachers had a good understanding of interdisciplinary concepts . Most of them had
deviations from expected outcomes. Most teachers had a vague understanding of subject
knowledge. They simply consider STEM integration as a simple superposition of multiple
disciplines. For example, a teacher with a mathematical background knows the engineer­
ing discipline contains measurements. In the teaching design process, he expressed that
he knows engineering and mathematics should be closely integrated, but he did not
know how to integrate them. Only a small number of teachers can use the knowledge of
various disciplines to solve practical problems, thus achieving cross-disciplinary
integration.
Overall, the findings are as follows: (a) both groups have improved in TK and EK, MK; (b)
Group B improved more than Group A in EK and TK; (c) These teachers still performed at a
lower level of IK which to be the most pressing aspect that need further PD.

STEM ability
Problem-solving ability (PSA)
Comparing the pre- and post-test the average value of the problem-solving ability in
Group A increased, but group B decreased, and group A (M = 0.098) was improving more
than group B (M = −0.071), and p < 0.05, there was significant a difference between Group
A and B (see Table 8). Furthermore, the percentage chart shows that most teachers in
Group A are at level 5 and level 4, and more than 80% of teachers have basic problem-
solving ability. In the open question, most teachers can provide solutions to problems
based on the comprehensive use of multidisciplinary knowledge and can optimise exist­
ing solutions and exchange design concepts. For example, Chen CY (pseudonym)
designed bridges. The 40 m bridge has a moderate span and does not require a cable-
stayed structure, but the span is too large for a single-hole arch bridge, so she combined the
advantages of both to design. However, the ability of teachers in Group B declined after
training; the proportion of teachers at level 5 dropped from 74.07% to 62.69%, but other
levels improved (see Table 9). Through the comparative analysis, Group B decreased, but
the problem-solving ability of Group A improved. Among the teachers in each group,
teachers with basic problem-solving ability accounted for more than 70% and performed
well. However, there are still some problems. Most teachers still lack awareness of
comprehensively considering the complexity of the problem.
Overall, a small number of teachers have strong problem-solving ability and a good
grasp of the problem-solving process, including describing and designing problem solu­
tions; some teachers’ understandings are incomplete and often involve only part of the
process. A small number of teachers have clear problem-solving awareness, which
includes processing the information in the material to further clarify the problem, using

Table 8. Comparison between A and B in STEM pedagogical skills.

Group A (N = 28) Group B (N = 54) T (GroupA-B)
M ± SD T(po-pr) M ± SD T(po-pr)
PSA pre 4.438 ± .331 0.990 4.330 ± .383 −0.621 1.273
PSA post 4.536 ± .415 4.259 ± .716 2.211*
 JOURNAL OF EDUCATION FOR TEACHING 587

Table 9. Analysis of pedagogical skills in STEM.

Problem-solving Ability (PSA)

Group A Group B

Pre-test Post-test Pre-test Post-test

Level 5 85.71% 78.57% 74.07% 62.69%
Level 4 14.29% 21.43% 25.93% 20.90%
Level 3 0.00% 0.00% 0.00% 7.46%

mathematical modelling and scientific inquiry to design the problem solution, and
reflecting on the problems existing to improve. Teacher Xie CY’s systematic approach
used is to a)find problems b)finding constraints c)brainstorming d)choice options e)build
prototypes f)testing g)repetitive testing h)communication and discussions. The teacher has
a good grasp of the entire problem-solving process.

Scientific inquiry ability (SIA)

Teachers, especially those with scientific backgrounds, can reflect a certain precognition
and knowledge reserve for scientific inquiry ability(SIA) in the pre-test. From the process
data, it can be seen that most teachers do not have enough understanding of SIA. In
addition, there is confusion about its use in teaching. A small number of teachers
misunderstood ‘scientific inquiry’ and considered it an ‘engineering design process’ or
‘reading material practice’. Some teachers chose a scientific question as a question of
inquiry, such as ‘the production of rockets’ and ‘the research of proportion’, suggesting
certainty for scientific inquiry of understanding and application. However, it is not enough
for the understanding and application of setting up questions in scientific inquiries.
Except for some teachers who have a high level of understanding in the background of
science education, most teachers seem to have fairly limited understanding. In SIA, there
is little difference between the two groups of teachers, which is greatly influenced by
academic background.

STEM attitude
Compared with the pre-test, the mean values of ATC (M = 0.549) and CI (M = 0.583) and AE
(M = 0.827) in group A increased, and P < 0.05 showed a significant difference (see Table
10). The mean values of CI (M = 0.761) and AE (M = 0.766) in group B increased and P
< 0.001, it showed significant difference, but ATC in Group B had no significant difference.
The understanding has been significantly improved in the STEM attitude. On the whole,
for the relationship between learning mathematics and STEM education, the teacher who
believes that ‘learning mathematics helps us to learn relevant content in engineering,
technology, science and other fields’ is approximately 80% in the pre-test and approxi­
mately 72% in the post. Teachers generally understand the relationship between learning
mathematics and STEM education. There is little difference between the two groups
(Table 10).
588 X. HUANG ET AL.

Table 10. Comparison between A and B in STEM attitude.

Group A (N = 28) Group B (N = 54)

M ± SD T(po-pr) M ± SD T(po-pr) T(Group A-B)

ATCpre 4.051 ± .633 3.729** 4.211 ± .702 1.714 −1.010
ATCpost 4.600 ± .478 4.441 ± .547 1.303
CIpre 3.953 ± .554 3.719** 3.604 ± .623 5.212*** 2.494*
CIpost 4.536 ± .782 4.365 ± .767 0.953
AEpre 3.756 ± .815 4.275*** 3.759 ± .708 6.088*** −0.021
AEpost 4.583 ±.495 4.525 ± .544 0.565

Before the intervention, most teachers agreed that the teaching of each subject should
strengthen the connection with society and life. For the overall understanding of STEM,
teachers had a vague cognition of ‘STEM is a course’ or ‘STEM is a teaching concept’. After
training, the definition of STEM and its characteristics were better understood. On the
whole, the understanding of STEM has been improved through this activity.

Conclusions and discussions

Given the challenges faced by many teachers worldwide, many teachers are lack of STEM
education experience (Awad and Barak 2018; Honey, et al. 2014). The present study
contributes to understanding how teachers’ STEM knowledge, PCK in STEM knowledge
and attitudes may be improved. Teachers should be given STEM learning experiences and
be encouraged to carry out STEM curricula by providing real situation questions to
deepen the understanding of STEM concepts and pedagogical strategies that support
STEM teaching (e.g. problem-solving ability, scientific-inquiry ability, critical thinking)
(Bybee 2013; Margot and Kettler 2019; Zollman 2012). The PD programmes prompted
teachers to explore the STEM class in depth, deepen or correct the teachers’ original
understanding of STEM and encouraged teachers to understand the STEM curriculum.
Many studies investigate how STEM PD program impact in-service teachers’ conceptua­
lisations and instruction of STEM (Akerson and Buck 2020; Berry, et al. 2019), fewer still
have discussed this impact teachers’ STEM literacy through observation, discussion and
reflection approach.
Overall, the observation, discussion and reflection approach was effective for teachers’
PD in STEM. Through this PD training, teachers’ understanding of EK, TK, MK; PSA, ATC, CI
and AE improved. Lesson Observation and Group Discussion encourages teachers to learn
STEM knowledge and pedagogical change in a timely mannerand be deepened by
discussion. Both groups have experienced Lesson Observation and Group Discussion, so
these may have a positive effect on the improvement of STEM literacy. The results of this
study indicate that the ODR approach has the potential to improve teachers’ under­
standing and attitudes towards STEM teaching. Observation encourages teachers to
understand STEM classrooms and seeks the use and integration of engineering design,
scientific inquiry, mathematical modelling and technology applications from teaching
(Shernoff et al. 2017). The discussion gives teachers the opportunity to communicate with
each other, deepen the connection between STEM, society and life, and learn to solve
problems in real situations. Video Reflection is an effective way for teachers’ reflection to
improve their in-depth thinking and problem-solving ability. This STEM PD training lasted
 JOURNAL OF EDUCATION FOR TEACHING 589

one week and had different approaches. Both Group A and B have grown in different
aspects. The teachers who participated in the two-week training have significantly
improved their knowledge, abilities and attitudes in STEM and have turned the STEM
education they have learned into STEM teaching and have begun to shift from STEM
literacy to teachers’ PCK for STEM. Teachers value PD when they can acquire the knowl­
edge and skills in order to improve their students’ STEM literacy, for example by improv­
ing their PCK (Hwang, et al. 2018). Aspects of teachers’ knowledge such as PCK have not
been investigated in the present study. Future studies could explore how the ODR
framework may support teachers’ PCK for teaching STEM.

Acknowledgments
We would like to thank the the National Social Science Fund of China for its support (BHA210121),
Department of Education of Zhejiang Province, Teaching and Research Section of Zhejiang, who
supported the training project of ZIPCP; the teachers and students involved in the research;, and the
anonymous referees who provided us with constructive comments and suggestions.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
The research was supported by the National Social Science Fund of China (Grant number:
BHA210121).

References
Akerson, V. L., and G. A. Buck. 2020. “Critical Questions in STEM Education. Contemporary Trends and
Issues in Science Educatio“. Switzerland: Springer Switzerland. doi:10.1007/978-3-030-57646-2
Alshehri, Eman. 2019. “Classroom Observation for Professional Development: Views of EFL Teachers
and Observers.” Arab World English Journal 1 (1): 57–71. doi:10.24093/awej/elt1.5.
American Association for the Advancement of Science .2012. Describing and measuring under­
graduate STEM teaching practices. https://www.uvu.edu/osp/docs/describing-and-measuring-
undergraduate-stem teaching-practices-nsf-and-aaas.pdf
Awad, N., and M. Barak. 2018. “Pre-service Science Teachers Learn a Science, Technology,
Engineering and Mathematics (STEM)-Oriented Program: The Case of Sound, Waves and
Communication Systems.” Eurasia Journal of Mathematics, Science and Technology Education 14
(4):1431–1451. doi:10.29333/ejmste/83680.
Berry, A., P. McLaughlin, and G. Cooper. 2019. “Building STEM Self-Perception and Capacity in Pre-
Service Science Teachers through a School-University Mentor Program“ In STEM Education: An
Emerging Field of Inquiry. Koninklijke Brill NV, Leiden, Netherlands.
Borko, H., K. Koellner, J. Jacobs, & N. Seago. 2011. “Using video representations of teaching in
practice-based professional development programs.” ZDM Mathematics. Education. 43, 175–187.
doi: 10.1007/s11858-010-0302-5.
Brown, J. C., and K. J. Crippen. 2018. “Designing for Culturally Responsive Science Education through
Professional Development.” International Journal of Science Education 38 (3): 470–492.
doi:10.1080/09500693.2015.1136756.
590 X. HUANG ET AL.

Bybee, R. W. 2010. “Advancing STEM Education: A 2020 Vision.” Technology and Engineering Teacher
70 (1) : 30–35. https://www.proquest.com/scholarly-journals/advancing-stem-education-2020-
vision/docview/853062675/se-2?accountid=47934
Bybee, R. W. 2013. The Case for STEM Education: Challenges and Opportunities, 4–12. Arlington, VA:
NSTA Press.
Capraro, M. M. 2013. “Interdisciplinary STEM Project-based Learning.” In STEM Project-based
Learning: An Integrated Science Technology Engineering and Mathematics (STEM) Approach, edited
by R. M. Capraro, M. M. Capraro, and J Morgan, 47–54. Rotterdam, Netherlands: Sense.
Corlu, M. S., R. M. Capraro, and M. M. Capraro. 2014. “Introducing STEM Education: Implications for
Educating Our Teachers for the Age of Innovation.” Education and Science 39 (171): 74–85. http://
repository.bilkent.edu.tr/handle/11693/13203
Crippen, K. J., K. D. Biesinger, and E. K. Ebert. 2010. “Using Professional Development to Achieve
Classroom Reform and Science Proficiency: An Urban Success Story from Southern Nevada, USA.”
Professional Development in Education 36 (4): 637–661. doi:10.1080/19415250903396026.
Czerniak, C M, and C. C Johnson. 2014. “Interdisciplinary Science and STEM Teaching.” In Handbook
of Research on Science Education (Vol. 2), edited by N. G. Lederman, and S.K Abell, 395–411, New
York, NY: Routledge.
Darling-Hammond, L., and M. W. McLaughlin. 2011. “Policies that Support Professional
Development in an Era of Reform.” Phi Delta Kappan 92 (6): 81–92. doi:10.1177/
003172171109200622.
Eckman, E.W, M. A. Williams, and M. B. Silver-Thom. 2016. “An Integrated Model for STEM Teacher
Preparation: The Value of a Teaching Cooperative Educational Experience.” Journal of STEM
Teacher Education 51 (1): 71–82. doi:10.30707/JSTE51.1Eckman.
Hayden, K., Y Ouyang, L. Scinski, B. Olszewski, and T. Bielefeldt. 2011. “Increasing Student Interest
and Attitudes in STEM: Professional Development and Activities to Engage and Inspire Learners.”
Contemporary Issues in Technology and Teacher Educationhttps://stelar.edc.org/sites/stelar.edc.
org/files/v11i1science1.pdf
Honey, M., Pearson, G., and Schweingruber, A. (2014). STEM integration in K-12 education: status,
prospects, and an agenda for research. Washington DC: National Academies Press
Hsu, Y. S., and Y. Yeh. 2020. Asia-Pacific STEM Teaching Practices: From Theoretical Frameworks to
Practices. Dordrecht: Springer.
Huang X., Erduran S., Luo K. K., Zhang P. S., and Zheng, M. Z. An Empirical Investigation of Chinese
Teachers' STEM literacy development to be published.
Huang, R., Su, H. & Xu, S. 2014. “Developing teachers’ and teaching researchers’ professional
competence in mathematics through Chinese Lesson Study.” ZDM Mathematics Education 46,
239–251. 10.1007/s11858-013-0557-8
Huang, X., Luo, K. K., and Bao, C. C. 2020. “Empirical study on pre-service teachers’ STEM literacy
development.” Teacher Education Research, 32(02): 32–38.
Hwang, M.-Y., Hong, J.-C., and Hao, Y.-W. 2018. “The value of CK, PK, and PCK in professional
development programs predicted by the progressive beliefs of elementary school teachers.”
European Journal of Teacher Education, 41(4), 448–462. doi:10.1080/02619768.2018.1471463
Margot, K.C., and Kettler, T. 2019. “Teachers’ perception of STEM integration and education: a
systematic literature review.” International Journal of STEM Education, 6, 2. doi:10.1186/s40594-
018-0151-2.
Morrison, J. 2006. TIES STEM education monograph series, attributes of STEM education. Baltimore,
MD: TIES(2)5. http://www.leadingpbl.org/f/Jans%20pdf%20Attributes_of_STEM_Education1.pdf
National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2007.
Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic
Future.Washington, DC: The National Academies Press
National Governors Association. 2011. Building a Science, Technology, Engineering, and Math
Education Agenda. An Update of State Actions. Washington, DC: NGA Center for Best Practices.
National Research Council. 2011. Successful K-12 STEM Education: Identifying Effective Approaches in
Science, Technology, Engineering, and Mathematics. Washington, DC: National Academies Press.
 JOURNAL OF EDUCATION FOR TEACHING 591

National Research Council. 2015. Guide to Implementing the Next Generation Science Standards.
Washington, DC: National Academies Press.
Pfeffer, J., and R.I Sutton. 2000. The Knowing-doing Gap. Boston, MA: Harvard Business school press.
Postholm, M.B. 2008. “Teachers Developing Practice: Reflection as Key Activity.” Teaching and
Teacher Education 24 (7): 1717–1728. doi:10.1016/j.tate.2008.02.024.
Shernoff, D.J., S. Sinha, D.M. Bressler, and L. Ginsburg. 2017. “Assessing Teacher Education and
Professional Development Needs for the Implementation of Integrated Approaches to STEM
Education.” International Journal of STEM Education 4 (1): 13. doi:10.1186/s40594-017-0068-1.
Taningco, M.T., A. B. Mathew, and H. P. Pachon. 2008. STEM Professions: Opportunities and Challenges
for Latinos in Science, Technology, Engineering and Mathematics. Los Angeles, CA: Tomas Rivera
Policy Institute.
Thibaut, L., Ceuppens, S., De Loof, H., De Meester, J., Goovaerts, L., Struyf, A., Boeve-de Pauw, J.,
Dehaene, W., Deprez, J., De Cock, M., Hellinckx, L., Knipprath, H., Langie, G., Struyven, K., Van de
Velde, D., Van Petegem, P. & Depaepe, F. 2018. “Integrated STEM Education: A Systematic Review
of Instructional Practices in Secondary Education.” European Journal of STEM Education, 3(1), 02.
doi:10.20897/ejsteme/85525.
Yang, Y., and Ricks, T. E. 2012. “How crucial incidents analysis support Chinese lesson study.”
International Journal for Lesson and Learning Studies, 1(1), 41–48. doi: 10.1108/
20468251211179696
Ye, Z. N., and Y. K. Yang. 2018. “Construction of STEM Education Curriculum——The Way for STEM
Teacher Professional Development.” The People’s Education, no. (8): 63–67. http://www.cnki.com.
cn/Article/CJFDTOTAL-RMJY201808021.htm
Yoon, K. S., T. Duncan, S. W. Y. Lee, B. Scarloss, and K. L. Shapley 2007. “Reviewing the Evidence on
How Teacher Professional Development Affects Student Achievement.“ Issues & Answers Report.
Accessed20 November 2021. http://ies.ed.gov/ncee/edlabs.
Zollman, A. 2012. “Learning for STEM Literacy: STEM Literacy for Learning.” School
Science&Mathematics 112 (1): 12–19. doi:10.1111/j.1949-8594.2012.00101.x.