Journal of Technology and Chinese Language Teaching Volume 7 Number 1, June 2016

http://www.tclt.us/journal/2016v7n1/tseng.pdf pp. 15-42

An Investigation of the Design of a Four-stage Flipped

Classroom in Mandarin Chinese
(中文翻转教室四个阶段的设计与研究)
Tseng, Miao-fen Broadstock, Maan Chen, Henny
(曾妙芬) (姜满) (陈姮良)
University of Virginia Wright State University Moreau Catholic High School
(弗吉尼亚大学) (莱特州立大学) (莫若高中)
[email protected] [email protected] [email protected]

Abstract: This paper aims to investigate the efficacy of a flipped

classroom that was created for twenty novice learners in an intensive
summer Chinese language program. The flipped classroom featured four-
stage learning: watching videos, taking notes online while watching videos,
responding to preview questions, and receiving instant feedback. Three-
factor ANOVA, one-way ANOVA, and regression analyses were used to
investigate how gender, level of Chinese, instructional topic, and note-
taking affected students' learning. Learners also completed a questionnaire
to elicit their reactions to the flipped classroom and feedback on the effect
it had on their learning. Statistical analysis indicated that the scores
differed significantly in instructional topics and that the number of notes
taken significantly decreased from the beginning toward the end of the
program. Additionally, a significant positive relationship was found
between the number of notes taken and test scores. The results of the
questionnaire at a 5-point Likert Scale showed that learners strongly
recommended the four stages of flipped design despite a slight decrease in
the second stage of learning through note-taking while watching videos.
Limitations of the study and directions for future research are discussed at
the end of the paper.

摘要：本研究旨在探讨暑期初级中文密集项目中翻转教室之有效性
以及其对学习之影响。本实验设计之翻转教室分为四个学习阶段，依
序为学生看教学视频，进而一边看教学视频一边在网上做笔记，再则
回答理解问题，最后收到立即性反馈。本研究采用三因子变异数分析、
单因子变异数分析、以及回归分析，以期探索性别，语言水平， 教
学主题，网上笔记四个因素对学生学习的影响。除此之外，本研究亦
针对翻转教室设计了一个问卷，以期了解学习者对翻转教室的反馈以
及其对参与者学习的影响。统计分析显示，学生的学习分数在不同教
学主题间有显著的差异，学生笔记次数的多寡自项目开始至终也明显
递减。同时也发现，笔记次数多寡与学习分数成正相关。李克量表问
卷结果显示，尽管翻转教室在第二个阶段一边看教学视频一边在网上
做笔记的数值不如其它三个阶段高，大多数学习者仍然普遍支持并推

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 15
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

荐此翻转教室四阶段的设计以及其未来在中文教学上的应用。本文最
后也提出此研究的限制与未来研究方向。

Keywords: Blended learning, flipped classroom, flipped learning,

teaching Chinese as a foreign language

关键词: 混成学习，翻转教室，翻转学习，对外汉语教学

1. Introduction
Flipped learning, which originated from research in STEM fields, i.e., science,
technology, engineering, and math (Berrett, 2012), has rapidly gained prominence and
popularity in innovative higher education E-learning contexts. Disciplines such as
physics (Deslauries, Schelew, & Wieman, 2011), engineering electronics and circuits
(Biemiller, 2013), statistics (Strayer, 2007), and cinema and TV arts (Enfield, 2013)
employ flipped learning. As the Internet continues to explode in the E-learning era, this
model has taken hold in secondary education in language arts (Ullman, 2013; Moran,
2014), science (Bergman & Sams, 2012), and math (Fulton, 2013). This evolving
pedagogical approach has been presented in several interchangeable terms: Inverted
classroom (Lage, Platt, & Treglia, 2000), Just-in-time teaching (Novak, 2011), Flipped
classroom (Bergmann & Sams, 2012a), or Inverted learning (Davis, 2013). Baker (2000),
who required his communication students to read PowerPoint slides before class through
an institutional new computer network, is accredited for conceptualizing the flipped
classroom paradigm (Lage & Platt, 2000; Lage, Platt, & Treglia, 2000).

According to Graham’s (2006) articulation of three levels of blends on a

continuum of technology integration, flip learning enhances blends instead of enabling
blends or transforming blends. It aims to provide “enhanced rather than equivalent
experiences of traditional classroom teaching” (Hung, 2015) in terms of technology use
in hybrid learning settings. It does not simply take place in “brick-and-mortar location”
establishments (Staker & Horn, 2012) and is built on extant mobile technological tools
that students frequently encounter. The acts of learning that have “traditionally taken
place inside the classroom now take place outside the classroom and vice versa” (Lage,
Platt, & Treglia, 2000, p. 32). In a flipped classroom, instructional order is reversed in a
way that students typically view instructional videos or other types of materials before
class and devote in-class time to applied learning activities and interactive higher-order
thinking.

1.1 Advantages of flipped classrooms

Researchers have well-recognized the advantages that this option has brought into
teaching and learning. Baker (2000) discovered that students had learned a great deal
from their peers through collaborative activities. Fulton (2012) identified that instructors
were able to allocate more time to individual students and implement the curriculum
more effectively. Other positive effects from flipped classrooms include, but are not

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 16
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

limited to, creating inquiry and problem-based learning (Bergmann & Sams, 2012a),
eliminating constant homework struggle (Strayer, 2007), and allowing students to catch
up with class content in their absence (Herreid & Schiller, 2013). In short, educational
proponents have faith in this paradigm shift that empowers them to reach “every student
in every class every day” (Bergmann & Sams, 2012a, book title). Although teachers may
feel unprepared to “wade into the digital muck of video creation” (Moran, 2014), this
technologically reliant idea has unparalleled potential to revamp classroom environments
and change how subjects are taught and learned (Bergmann & Sams, 2012a; Bergmann &
Sams, 2012b; Fulton, 2012; Tucker, 2012). Its close linkage to theoretical understandings
of active learning also “involves students in doing things and thinking about the things
they are doing (Bonnell & Eison, 1991, p. 2).

1.2 Flipped classrooms in second language acquisition

Nearly all empirical research on flipped classrooms has been conducted in STEM
classrooms. Despite increasing focused attention to flipped learning, surprisingly there
have been only a very limited number of studies on flip learning models in second
language acquisition. The following literature review offers a small glimpse of what is
available in post-secondary, but not secondary, educational settings.

Hung (2015) conducted a posttest-only quasi-experiment to investigate EFL

learners’ flip learning experiences at a Taiwan university. The experimental design of the
study divided students into a flip group with WebQuest learning, a semi-flip group
featuring TED-ED viewing, and a non-flip group. The results confirmed the value of
active learning and revealed that students in the flip and semi-flip classrooms attained
more satisfactory learning outcomes, developed more positive learning attitudes, and
devoted themselves more to the learning process than students in the non-flip group.

In his study on developing EFL writing skills through student-created digital

videos, Engin (2014) made an attempt to transform passive “sit-and-get” learners into
producers and consumers in an active peer learning process. Although student-generated
materials and input tended to lower cognitive, linguistic, and cultural load that may be a
barrier to language learning (Mercer, 2000), learners who were Arabic native speakers
expressed their preference of a teacher’s explanation over receiving a peer’s input in the
flip learning phase.

Mehring (2014) completed a qualitative study in an attempt to investigate how

EFL learners who were native speakers of Japanese reacted to the flipped classroom. It
was found that students greatly benefitted from “the greater student-centered and active
learning environment, the added amount of time participants spent preparing for the face-
to-face class, and the significantly enhanced authentic, communicative learning
environment” (p. 86).

Hojnacki & Häusler-Gross (2014) compared language gains in four language

skills through flip and non-flip sessions in elementary German in a US mid-west liberal
arts college. They summarized that learning did take place in each of the two sessions.

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 17
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

Additionally, students who had flip experiences performed slightly better than those who
did not in all language skills except for speaking. Despite the fact that no significant
difference was reported in change between the two sessions because of small sample size,
the survey showed that students who learned through the flipped classroom model were
more motivated and held a positive attitude toward the flipped format and therefore
recommended continued adoption of the flipped model in the future.

Salazar & Sinclair (2015) reported the findings of their empirical study on the
development of lexical competence among elementary learners of Spanish at an
American university. In their study, three types of class structures were under
investigation: a non-flip class, a flipped class using Quizlet, and a flipped class using
vocabulary video tutorials. The differences between the pre-post and post-test indicated
that all three designs yielded observable gains that promoted differentiated instruction.
Although a combination of Quizlet and videos was recommended, the authors reflected
upon potential future improvements to add comprehension checks and a tactile and
interactive platform for video tutorials.

The above limited studies on SLA coincidentally were completed at higher

education institutions. Whether the findings and pedagogical implications of flip learning
design work in foreign language classes at the secondary level remains unknown and
deserves further exploration. This pioneering study demonstrates the very first attempt of
its kind to investigate the design of an innovative flipped classroom model that involves
four stages of learning in Mandarin Chinese among 9th-12th graders at the novice level.

2. Design of the four-stage flipped classroom in this study

The creation and experiment of the flipped classroom began in the summer of
2014. The flipped classroom launched in the 2014 STARTALK program involved only
two stages of learning before face-to-face instruction took place in class, i.e., watching
videos and working on Google forms to check for understanding, the latter of which
provided valuable insight for teachers to incorporate collective feedback into face-to-face
instruction on the following day. It was then re-evaluated, modified, enriched, and
expanded to a four-stage innovative model to constantly and systematically monitor
learners’ progress and ensure that flipped learning achieved its proximal effects before
face-to-face learning began. Compared to the traditional flipped classroom model in
which learners watch videos at home and then participate in activities in class, the flipped
learning model specifically created for this program advanced its existing functionality
by adding three critical components: note-taking, comprehension checks, and instant
feedback, including scores, error corrections, and explanations for inaccurate items after
students completed comprehension questions.

The four stages of the flipped classroom in the 2015 STARTALK program are as
follows, described from learners’ perspective:

Stage 1: Watching videos

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 18
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

Stage 2: Taking notes online while watching videos

Stage 3: Responding to preview questions
Stage 4: Receiving instant feedback

The four-stage flipped classroom model is highlighted below.

Students Students take Teachers

Students show their
watch notes while provide
understanding via
flipped watching instant
Google forms
videos videos feedback

The four-stage model was meticulously laid out at both academic and
technological levels. It was intended to be well-sequenced and structured, so each single
stage was interconnected with its immediate next stage and the remaining three stages. It
was collaboratively created by a technology specialist, four leading teachers of the
STARTALK teacher and student programs, and a curriculum developer and key trainer.
The pre-class flipped learning components were created to strongly enhance in-class
face-to-face learning and support the fulfillment of curricular objectives as the ultimate
goal. All materials were uploaded to the program website and well tested before the onset
of the teacher and student program. The core instructional team worked tirelessly to
ensure that the online operational system remained stable and well-run during the
program. The twelve teacher trainees did not become involved in the creation of the
flipped learning process. Upon their arrival at the program, the flipped learning online
system was already completed and uploaded to the combination program website for the
purpose of teacher development.

In total, twenty-two videos were pre-recorded for eight topics to be taught on

eight days during the program. Six topics had three videos each, and two topics had two
videos each. The videos were created in full alignment with curricular goals and program
needs. Specifically, each video included essential learning materials, core sentence
structures, and cultural references in a PowerPoint for screen recording. Each video lasted
from one to less than three minutes. While a number of free screen recording tools were
available online such as Ocam, QuickTime Player Pro, and Jing, the free downloadable
version of “Screencast-O-Matic” was chosen to create program videos because of its
accessibility on both the Macintosh and Windows operating systems.

After the videos were pre-recorded, daily Google forms and spreadsheets were
then created to check for student comprehension through multiple-choice questions,
ranging from six to 9 items for each topic. The Google forms and spreadsheets had the
capacity to help educators collect, share, analyze, and graph data. A Google spreadsheet
add-on was later installed to create a MCQ test, which automatically corrected and scored
learners’ responses submitted online and sent along explanations for wrong items via
email. Compared to other web tools such as Flubaroo, Zaption and EduCanon, the
combination of Google forms and the MCQ script is free of cost, and its instant e-mail

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 19
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

feedback and explanatory notes well concluded the four-stage flipped learning process
and sufficed for the purposes of this study.

Afterwards, pre-recorded videos were uploaded by the technology specialist to

YouTube under the unlisted category and “1know.net,” an innovative online learning
management system that enables instructors to collect data and monitor students’
progress. The 1Know Learning Management System (LMS) recorded all of students’
learning data, including the number of notes taken, video and unit learning duration, and
video accessing records. The data trail allowed instructors to monitor the note-taking
process and outcomes. Instructors were able to know when students took notes while
watching the videos, what types of notes and how many notes students typed online, and
whether the quantity of notes correlated with the score that students received through
Google forms. To put it concisely, LMS generated a database through which instructors
could monitor and analyze learners’ daily work and progress, and, most importantly, best
prepare for face-to-face instruction the next day. The preparation workflow for teachers is
as follows.

Create
Learn Make Record Google Link
contents PowerPoint materials forms materials to
LMS

Upload
videos to
YouTube

When students logged into the system, they would be able to see the 8-day
assigned online tasks in order, the YouTube videos, and the Google forms on the central
1know platform. They accessed online materials all-at-once at any location that was
most convenient for them. An orientation was held on the first day of the program to help
students become familiar with the online system and successfully register for the course
via the 1Know platform. During this 1Know orientation session, students practiced
viewing the videos and taking notes at the same time. Students were also encouraged to
post their questions as notes and highlight the questions in red, so their teacher could read
their questions more easily. Before attending the program, it was assumed that students
had developed note-taking skills in their own high schools. Therefore, what to write for
online notes was not the focus of the training session and was open for them to decide.
They were advised to write whatever they felt necessary and helpful for their own self-
paced learning.

2.1 Research questions and method

The emergence of flipped learning in the STEM area has an increasing impact on
foreign language education. More and more foreign language educators have adopted the

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 20
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

concept to create flipped classrooms for students to preview lessons before class, so in-
class time can be secured for interactive and engaging discussions. But most flipped
classrooms were created without sound research to support the design and instructional
strategies (Hamden et al., 2013). More research is therefore needed to inform us of what
type of flipped design works and does not work pedagogically.

As indicated earlier, while a very limited number of flipped learning studies in

SLA have been situated in post-secondary settings, there is little research to inform us of
how flipped learning complements face-to-face instruction to strengthen the learning
process and outcomes in K-12 foreign language classes. Unlike the design of most
flipped classrooms, this study makes a breakthrough to go beyond one stage of flipped
learning and creates a four-stage learning process in order to systematically consolidate
and monitor pre-class online learning, which is expected to strengthen face-to-face
instruction in teaching Mandarin Chinese as a foreign language to high schoolers in a
summer intensive program implemented at an American university.

This study aims to investigate how gender, learners’ proficiency level (hereafter
level), instructional topics (hereafter day), and note-taking affected students’ learning
after students went through the four-stage learning process: watching videos, taking notes
online while watching videos, responding to preview questions, and receiving instant
feedback.

The research questions include the following:

1. While students were watching videos, did gender, level, or topic affect their
note- taking behavior?
2. After students went through the four-stage flipped learning, did gender, level,
topic, or note-taking affect students’ preview question scores?
3. Did students who watched videos score higher than students who did not
watch videos?

In addition to the above research questions, the study also intended to explore
students’ reactions to the four-stage flipped design and their feedback. A questionnaire
was developed according to a 5-point Likert Scale to help us understand students’
reactions to the four-stage flipped learning and their comments and recommendations for
future applications. The questionnaire was intended to gather information about students’
input on their motivation level, the effects of flipped learning on their learning outcomes,
the friendliness of technology use, and their recommendations for future applications of
the four-stage flipped learning model to Chinese language courses or programs (See
Appendix I). Learners were also encouraged to share any of their additional comments
on the design of the flipped learning model and overall reflective learning experiences.

2.2 An overview of the program

The 2015 program pilot-tested cutting-edge technology tools to create the flipped
classroom and achieve proximal learning outcomes in an innovative blended learning

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 21
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

model. The daily routine of teachers’ practicum closely followed a comprehensive cycle
that was composed of pre-class preview, during-class learning, and after-class review to
ensure closely monitored daily progress in alignment with learning objectives.

2.2.1 Student participants

Students who participated in the study learned Mandarin Chinese in the Virginia
STARTALK Chinese Student Academy in summer 2015. The program evolved from a
half-day Chinese Language and Culture Immersion Program as part of the Virginia
STARTALK Chinese Teacher Academy that was launched in 2008 through the
STARTALK initiative in the US. The Virginia STARTALK Chinese student Academy
(VSCSA) partnered with the Virginia STARTALK Chinese teacher Academy (VSCTA)
to increase capacity for learning Mandarin Chinese in K-16 educational settings. The
teacher to student ratio was 1:1 in 2015, and students frequently interacted with devoted
teachers to satisfy their individual needs on a daily basis. The program strives to create a
truly interactive, communicative, and innovative class featuring an integration of a wide
array of authentic tasks and cutting-edge instructional technology.

Participants were 20 non-heritage learners who were 9th-12th grade high schoolers
recruited from the commonwealth of Virginia and neighboring states in the US. There
were 13 female students and 7 male students in the program, including five 9th graders,
seven 10th graders, four 11th graders, and four 12th graders. Of the twenty students, seven
students had prior learning experiences. One had learned Chinese for three months in a
weekend Chinese school, four had completed Level I Chinese, one had completed Level
II Chinese, and one had completed a Virtual Chinese course for 3 years. According to the
pre-program survey and individual interviews, they were placed into two classes
depending on their prior learning experiences, learning styles, age, and gender.

Of the twenty students who participated, two were not able to access the 1Know
website. Therefore, only eighteen students watched the videos and completed the online
preview questions daily. Among these eighteen students, eleven were female and seven
were male. Eleven were novice students, and seven had studied Chinese before. Since
the students were not strictly required to complete the assignments, some students missed
one or two videos or did not complete the online quizzes. Two students did not watch the
videos except on Day One, when all students watched the video in class, but they
completed the daily quizzes.

2.2.2 Curriculum

Twenty learners participated in the 10-day non-residential program from July 13th
to July 23rd in 2015, excluding Sunday. The curriculum was created by a core
instructional team consisting of four leading teachers of the teacher and student program,
aided by a technology specialist’s expertise in instructional technology and Chinese
language instruction through 3-stages of backward design. Morning included three 50-
minute slots when students learned through face-to-face instruction in regular classrooms.
The target language was exclusively used in class, which allowed learners to gain

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 22
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

enriched immersion experiences. Afternoon sessions were reserved for students to

complete technology-related tasks, review learned materials, and interact in cultural
activities. English was mingled with Chinese in the afternoon sessions.

The program offered no credits to students. Total instructional time was 65 hours
during the day program, and 10 hours for online flip learning and self-review at home.
The targeted proficiency level was novice-mid according to ACTFL’s proficiency
guidelines, but performance level could reach up to novice-high on certain topics. The
instructor exclusively used the target language in class, and encouraged learners to do
likewise. Learners were enrolled in the virtual US-China Exchange Program to embark
on an exciting trip to Beijing. The entire student program lasted for ten days but covered
only eight topics on eight days, with the seventh day reserved for review and the last day
for an end-of-program performance and closing ceremony.

Instructional topics included the following:

Day 1: Greetings and self-introduction

Day 2: Family
Day 3: Dates and time
Day 4: Hobbies
Day 5: Vising a Chinese family
Day 6: At a Chinese restaurant
Day 7: Review
Day 8: Sightseeing in Beijing
Day 9: Shopping
Day 10: End-of-program performance and closing ceremony

2.2.3 Teachers

Instructors were two leading teachers in the student program and 12 teacher
trainees who were recruited nationwide to take a 3-credit graduate level course in
teaching Chinese as a foreign language during the program. Of the twelve teacher
trainees, ten were in-service teachers who taught in secondary or post-secondary
educational settings in the US, and two were graduate students who were about to enter
the job market to launch their teaching careers. Three teachers worked together to prepare
for team-teaching, and a total of five groups of teachers rotated to complete their teaching
practicum. The teachers prepared lesson plans according to STARTALK-endorsed
principles for effective teaching and learning as follows, while taking into account
program-specific curricular objectives on a daily basis.

 Implementing a standards-based and thematically organized curriculum

 Facilitating a learner-centered classroom
 Using target language and providing comprehensible input for instruction
 Integrating culture, content, and language in a world language classroom
 Adapting and using age-appropriate authentic materials
 Conducting performance-based assessment

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 23
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

The twelve teacher trainees went through a multi-faceted reflective

comprehensive cycle that was carefully designed for them to fulfill practicum
requirements. They worked closely with leading teachers in the teacher and student
program to complete teaching practice and were systematically supervised by the leading
teachers, the technology specialist, and the key trainer and program director before and
during the program.

3. Discussion and results

Whereas twenty students completed the questionnaire at the end of the program,
only eighteen students fully participated in the four-stage flipped learning process from
the beginning until the end of the program. Therefore the statistical analysis of the
following first section is based on data collected from eighteen students. The discussion
is divided into two sections. The first section focuses on statistical analysis of the factors
potentially affecting students’ preview questions and preview question scores. The
second section summarizes the results of the questionnaire pertaining to students’ input
and comments on the design of the flipped learning classroom.

3.1 Factors potentially affecting students’ learning in the flipped classroom

Scores on the daily comprehension questions, i.e., quiz scores, were used to
indicate students’ understanding of the videos. A statistical analysis was conducted to
investigate whether the following factors played a significant role in students’ scores and
the number of notes they took: gender, level of student’s prior Chinese knowledge, and
the number of days into the program, i.e., 8 days with 8 different topics. The effect of
note- taking on students’ quiz scores was also considered.

3.1.1 Score versus gender, Chinese level, and day

A three-factor statistical analysis of variance (ANOVA) model was used to

investigate the following research questions:

 Did female students score higher than male students?

 Did students who learned Chinese before score higher than novice students?
 Did the scores differ in eight days, i.e., eight topics?

The three factors analyzed were gender, Chinese level, and day, with day as the repeated
measurement.

Table 1 shows the results of the three-factor ANOVA on scores. The results
indicated that students’ scores did not show a significant difference based on gender, the
level of students’ prior Chinese knowledge, or the interaction between gender and level.
However, the scores showed a significant difference on day (F (7, 98) = 4.17, p< 0.001).

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 24
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

Table 1: Results of the 3-factor (gender, level, day) ANOVA on scores with day as the repeated
measurement
Tests of Hypotheses Using the ANOVA MS for ID (Gender*Level) as an Error
Term
Dependent Variable: Score
Source DF ANOVA SS Mean F Value Pr>F
Square
Gender 1 0.00042861 0.00042861 0.00 0.9555
Level 1 0.18101010 0.18101010 1.37 0.2620
Gender*Level 1 0.11533924 0.11533924 0.87 0.3667
Tests of Hypotheses Using the ANOVA MA for ID* day(Gender*Level) as an Error
Term
Dependent Variable: Score
Source DF ANOVA SS Mean F Value Pr>F
Square
Day 7 0.44336667 0.06333810 4.17 0.0005
Gender*Day 7 0.07373690 0.01053384 0.69 0.6776
Level*Day 7 0.09677619 0.01382517 0.90 0.5023
Gender* 7 0.09165201 0.01309314 0.86 0.5397
Level*Day

A plot of day versus average score by gender is shown in Figure 1, and a plot of
day versis average score by level of Chinese is shown in Figure 2. Both figures reveal
that the average scores throughout the eight days showed no significant difference based
on gender or levels.
DAY VS. SCORE BY GENDER

Female Male

1.00
0.80
SCORE

0.60
0.40
0.20
0.00
1 2 3 4 5 6 7 8
DAY

Figure 1: Day versus Score by Gender

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 25
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

DAY VS. SCORE BY LEVEL

Learned Chinese before Novice

1.00

SCORE
0.50

0.00
1 2 3 4 5 6 7 8
DAY

Figure 2: Day versus Score by Level

The results show that day is a significant factor in the scores of student
performance. A one-factor repeated measurement ANOVA was then conducted for a
detailed comparison of the scores among these eight days. The results were consistent
with the 3-factor ANOVA in that there were significant effects for day on the scores
(F(24, 119)= 7.35, p< 0.001). The average score of Day 1 and the average score of Day 7
were significantly different.

The following figure, i.e., Figure 3, shows the average scores for the eight days.

DAY VS. SCORE

1.00
0.93
0.95
0.88 0.86
0.90 0.83 0.82 0.83
0.85 0.80
SCORE

0.80
0.75 0.69
0.70
0.65
0.60
1 2 3 4 5 6 7 8
DAY

Figure 3: Day versus Score

The Day 1 score was higher than the other days. This might be due to the time
that the quiz was taken. On the first day of the program, students received in-class
instruction in the morning and then learned how to use the flipped classroom video
platform during the afternoon lab time. They watched the videos and took the quiz after
they were introduced to the 1Know website. Therefore, the scores reflected their
understanding after the in-class instruction and video watching. The score from Day 7
was significantly lower than the scores on other days. This might be due to the higher
difficulty level of the content associated with vocabulary and grammatical structures and
the design of the quiz questions. For Day 7, students watched two flipped videos and
answered six questions after watching the videos. However, two of the questions were
answered correctly only 50% of the time. One of the questions was not clearly stated,
© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 26
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

and the other question referred to grammar structure that was not explained in the video.
Besides Day 7, the rest of the scores show no significant difference among the different
days and topics. Therefore, excluding the human error in the question design on Day 7,
gender, level of Chinese knowledge, and the day of each topic did not have a significant
effect on quiz scores.

3.1.2 Note-taking versus gender, Chinese level, and day

Since gender, level, and day did not have a significant effect on students’ scores,
the effect of note-taking on students’ scores was studied. First, how gender, level of prior
Chinese knowledge, and day affected the number of notes students took while watching
the videos was studied. A three-factor ANOVA model was used to investigate the
following research questions:

 Did female students take more notes than male students?

 Did students who learned Chinese before take fewer notes than novice students?
 Did the number of notes taken change throughout the eight days (topics)?

The three factors analyzed were gender, Chinese level, and day, with day as the repeated
measurement.

Among the eighteen students who took the quiz, two students’ data were excluded
from the data analysis since they watched only the first day’s videos and did not watch
the rest. Therefore, this statistical model included only the data from sixteen students. In
total, there were 1,045 notes taken throughout the eight days of video watching. The
notes can be categorized into the following three different types.

1. Repetition of video content

Ninety percent of the notes (totally 941 notes) that students took repeated what
they learned from the videos. This includes listing new vocabulary in Pinyin and
English (for example: da lanqiu = bball), and summarizing the content they
learned in the video (for example, women jintian qu kan dian ying, hao ma?
subject-time-verb-thing).
2. Questions (94 notes)
Nine percent of the notes (totally 94 notes) that students took were questions they
had about the content. They were encouraged to ask questions and to highlight
the questions in red so the teachers could see them more easily. Their questions
included queries about language and grammar structure (for example, In Chinese,
do you ever conjugate verbs?), and questions about culture (for example, is it
disrespectful to ask an adult their age?).
3. Notes to self
Only one percent of the notes (totally 10 notes) served as notes to themselves.
They included a reminder (for example, be careful of the pinyin), and self-
reflections (for example, wo hen xi huan xue xi han yu).

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 27
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

Although the quality of the notes might be a factor that affected how students
learned Chinese language and culture, this study sorely focused on examining whether
the quantity of note-taking affected students’ learning outcomes. The quality of note-
taking is beyond the scope of this study.

Table 2 shows the results of the three-factor ANOVA on note-taking. The results
indicated that students’ scores did not show a significant difference on gender, the level
of students’ prior Chinese knowledge, or the interaction between gender and level.
However, the number of notes taken shows a significant difference on day (F (7, 84) =
5.74, p< 0.001); thus there was a significant effect of day on the number of notes taken.

Table 2: Results of the 3-factor (gender, level, day) ANOVA on number of notes taken with day as
the repeated measurement
Tests of Hypotheses Using the ANOVA MS for ID(Gender*Level) as an Error Term
Dependent Variable: Notes
Source DF ANOVA SS Mean F Value Pr>F
Square
Gender 1 646.1519805 646.1519805 2.04 0.1786
Level 1 4.5893630 4.5893630 0.01 0.9063
Gender*Level 1 282.6019129 282.6019129 0.89 0.3641
Tests of Hypotheses Using the ANOVA MA for ID* day(Gender*Level) as an Error
Term
Dependent Variable: Notes
Source DF ANOVA SS Mean F Value Pr>F
Square
Day 7 724.1919430 103.4559919 5.74 <.0001
Gender*Day 7 52.8686549 7.5526650 0.42 0.8880
Level*Day 7 123.3499991 17.6214284 0.98 0.4530
Gender* 7 25.9886940 3.7126706 0.21 0.9833
Level*Day

A plot of day versus number of notes taken by gender is shown in Figure 4 below.

DAY VS. NOTES BY GENDER

Female Male

15.00
NUMBER OF NOTES

10.00

5.00

0.00
1 2 3 4 5 6 7 8
DAY

Figure 4: Day versus Number of notes taken by Gender

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 28
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

In Figure 4, the daily average number of notes taken by the female students was
consistently higher than that of the male students. However, the results of ANOVA
showed no significant difference between genders (F(1, 12)=2.04, p=0.17). When
looking at the averages over the entire eight days for females (M=9.87, SD= 2.17) and
males (M=5.21, SD=2.57) and taking into account the standard deviations, there was no
significant difference between the genders in terms of the average number of notes taken.

Figure 5 is the plot of day versus the number of notes taken by students with
different levels of Chinese. The results of the analysis showed that the number of notes
taken by the novice students (M=8.27, SD= 2.17) and the number of notes taken by the
students who had previously learned Chinese (M=7.88, SD=2.57) were not significantly
different.

DAY VS. NOTES BY LEVEL

Learned Chinese before Novice

NUMBER OF NOTES

15.00
10.00
5.00
0.00
1 2 3 4 5 6 7 8
DAY

Figure 5: Day versus Number of notes taken by Level of Chinese

Figure 6 shows the average number of notes taken throughout the eight days. The
number of notes decreases throughout the eight days. A one-factor (day) repeated
measure ANOVA with number of notes as the dependent variable was conducted. The
result was consistent with the results from the 3-factor ANOVA in that there was a
statistically significant effect of day on the number of notes taken (F(22,105)=15.21,
p<.001). The decrease in the number of notes taken can be explained by the decreasing
level of student motivation and energy level to take notes and increasing accumulative
workload toward the end of the intensive program. Another factor must have something
to do with the design of note-taking that is one-way in nature, lacking the instructor’s
interaction with students online. Although face-to-face feedback on students’ note-taking
was given as needed during the next day, daily encouraging feedback and responses to
students’ comments, regardless of type of notes, were not given online through two-way
communication as a program feature.

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 29
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

DAY VS. AVERAGE OF NUMBER OF NOTES TAKEN

14.0011.72
11.44
12.00
8.81

NUMBER OF NOTES
10.00 8.63
6.38 6.94 6.50
8.00
5.23
6.00
4.00
2.00
0.00
1 2 3 4 5 6 7 8

DAY

Figure 6: Day vs. Number of notes taken

3.1.3 Number of notes taken vs. score

Gender and level of prior Chinese knowledge did not show significant effects on
quiz scores. A regression analysis on the average of individual students’ scores and their
average number of notes taken was used to investigate whether students who took more
notes scored higher than students who took fewer notes, and whether there was a
relationship between test scores and the number of notes taken. As indicated in Table 3,
the results of the analysis showed that there was a significant positive relationship
between the number of notes taken and score (F(1, 14)=4.69, p < .05). The coefficient of
linear fit slope is 0.00737, which showed that there is a positive relationship between the
number of notes taken and score, with a significance level of .05. However, the effect on
score is very small.

Table 3: Results of the regression analysis of the number of notes taken and quiz score
Analysis of Variance
Source DF Sum of Squares Mean Square F Value
Model 1 0.03228833 0.032288 4.6938
Error 14 0.09630542 0.006879 Prob > F
C. Total 15 0.12859375 0.0480*
Parameter Estimates
Term Estimate Std Error T Ratio Prob > |t|
Intercept 0.7944662 0.034563 22.99 <.0001*
AvgNote 0.0073768 0.003405 2.17 0.0480*
Linear Fit
AvgScore = 0.7944662 + 0.0073768 x AvgNote

3.1.4 Watching videos or not versus score

There were two students who did not watch the flipped classroom videos at home
but completed the quiz questions, except on Day 2. One of them had learned Chinese for
© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 30
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

one year, and the other did not have any prior Chinese learning. To understand how these
videos helped novice students learn, the scores of this novice student who did not watch
videos were compared with the scores of the novice students who watched the videos.
Figure 7 shows that the scores of the novice student who did not watch the videos
(M=0.40, SD= 0.15) were significantly lower than the novice students who watched the
videos (M=p.83, SD= 0.074).

DAY VS SCORE BY VIDEO WATCHING

Watch videos Did not watch videos

0.90 0.87 0.90
1.00 0.83 0.80
0.71
0.80
0.56 0.56
SCORE

0.60 0.44
0.33 0.33
0.40 0.17
0.20
0.00
1 2 3 4 5 6
DAY

Figure 7: Day versus Score by video watching and no video watching

The results of the statistical analysis showed that gender and level of Chinese
knowledge did not have a significant effect on quiz scores or on the number of notes
taken. However, day had significant effects on both scores and the number of notes
taken. The effect of day on the scores might have been caused by the question design,
which may have room for improvement. The effect of day on the number of notes taken
was significant, but the reasons for the decrease in the number of notes taken needs
further study. The regression test showed that there is a significant positive relationship
between the number of notes taken and quiz scores. The students who took more notes
scored higher on the quizzes; however, the impact on score was small. Students who did
not have prior Chinese knowledge and watched the videos scored significantly higher
than the novice student who did not watch the videos. This result indicated that watching
flipped classroom videos did help students understand the content and perform better on
the quizzes.

3.2 Results of the questionnaire on learners’ input and comments

In addition to the above statistical analysis of the impact of gender, level, topic,
and note-taking on students’ learning, this study also conducted a questionnaire with
items created based on a 5-point Likert Scale to gather learner participants’ input on the
effectiveness of the implementation of the four stages of flipped learning. The
questionnaire looked into five categories of each stage of flipped design: time
commitment, motivation, effects, technology use, and future recommendations.

As far as time commitment is concerned, when learners watched videos, they

watched videos and took notes at the same time, so time was automatically calculated

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 31
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

based on the total time spent watching videos and taking notes. There was no mean
indicative of time spent on watching videos only. On average, each learner spent 11.50
minutes watching videos and taking notes, 10.05 minutes responding to preview
questions, and 6.10 minutes viewing instant feedback, amounting to 28.1 minutes for
daily flipped learning at home. The time spent on completing flipped learning
components seemed reasonable and manageable and did not appear overwhelming as
learners were fully devoted to Chinese language learning, and no other subjects, in the
summer.

The following table summarizes the means of the remaining four categories:
motivation, effects, technology, and recommendations.

Table 4: Means of 5-point Likert Scale on four Stages of flipped Learning

Category Watching Watching Responding Viewing
videos videos/taking to preview instant
notes questions feedback
Motivation 4.15 3.45 4.15 4.25
Effects 4.55 3.30 4.40 4.45
Technology 4.65 3.95 4.50 4.50
Recommendations 4.65 3.65 4.30 4.60
Average 4.50 3.59 4.34 4.45

As the above table shows, all columns have means higher than 4 except for the
second column on watching videos and taking notes, with the mean of each cell ranging
from 3 to 4.

Looking at the figures vertically, the means from the highest to the lowest are
represented in the following order: watching videos (4.50), viewing instant feedback
(4.45), responding to preview questions (4.34), and watching videos and taking notes
(3.59). The highest mean lies in watching videos, but surprisingly when it goes with note-
taking, the means in four categories become the lowest. Responding to preview questions
and viewing instant feedback had their means very close to the highest.

Horizontally, analyzing the means in four categories, motivation overall has

slightly lower means, while effects, technology, and recommendations have higher means.
Having said that, cells that represent motivation still have means higher than 4, except for
the cell that represents watching videos and taking notes, as mentioned earlier. This
indicates that learners well-recognized the effects and values of flipped learning and the
friendliness of technology use, therefore highly recommending the future use of the four-
stage flipped learning model. The only area that did not receive the highest
recommendation is online note-taking that accompanies watching videos. Although
learners affirmed the value of watching instructional videos as a before-class preview
component, they did not hold the same high level of motivation when they were required
to do note-taking while watching videos simultaneously. Similarly, learners did not hold
the same high level of positive attitude toward note-taking as the effects of flipped
learning and the friendliness of technology use.
© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 32
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

The relatively low value of taking notes while watching videos is worth exploring
and explaining. Apparently, learners’ personal preference did not comply with the impact
of note-taking on learning outcomes, as indicated in the statistical analysis discussed
above.

As discussed in the previous section on statistical analysis, a significant positive

relationship was found between the number of notes taken and the learning outcome, i.e.,
quiz score. This can be interpreted that the more notes taken, the better the score is. The
results showed that note-taking does have a positive effect on absorbing video content,
therefore enhancing language performance with more satisfactory learning results. It may
be possible that 9-12th graders were not yet independent learners in online learning, and
were unaware of the value of note-taking and its positive impact on their overall learning.
For this group of learners, flipped learning is a brand new concept and experience. They
had never watched videos and taken notes online simultaneously, so they were not
accustomed to performing similar tasks at the same time. Another factor that may well
explain why note-taking is not a welcome option is that the flipped learning process in
this study was designed to be a one-way communication; no follow-ups or interactive
features were added between the learner and instructor or among learners themselves.
This led note-taking to be an individual task with interaction taking place between each
learner and the online platform. It is admitted that note-taking has undisputable
pedagogical value and its functionality cannot be ignored or under-stressed. It is
hypothesized that if learners had received more training on note-taking and understood its
positive effect before participating in flipped learning during the study, their input on this
aspect might have differed. This hypothesis, however, will have to be verified through
future studies.

Aside from the results summarized above based on a questionnaire at the 5-point
Likert Scale, it is worthwhile discussing additional comments that some of the learners
shared in the questionnaire. Most learners appreciated the opportunity to watch the short
videos as the videos helped them prepare for the next class, gain an idea of upcoming
content, and get ready for the full immersion class on the next day. One commented that
learners can “get a head start by watching the video,” adding that these videos are
informative and are great resources. One said positively that videos also helped explain
grammatical use, and in terms of grammar, another expressed the need to add more
grammar to the videos. Of all the added comments, one student who was not used to
online learning expressed his frustration because “video content and vocabulary were not
on student sheets.” The program actually offered a folder of study guides that included
daily materials to be learned, and whether it is necessary to create handouts to accompany
video content is a matter for flipped classroom instructors to consider further. According
to comments collected in this study, this does not stand out as a critical issue.

Unlike mostly positive comments on learners’ reactions to watching videos,

feedback on note-taking was half positive and half slightly negative.

On the positive side, note-taking helped to “retain content” and “affirm

knowledge and keep it fresh in mind.” Two learners expressed their appreciation of the

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 33
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

convenience of the technology tool that allowed the video to automatically pause when
learners started taking notes. This surprisingly made another student frustrated, revealing
completely the opposite response.

One student shared a very interesting observation about grammar covered in the
videos. She pointed out that the videos sometimes discussed grammar, whereas in class
the instructor rarely discussed it. This point was perceived by that particular student as a
drawback, but it is indeed a proof of the successful handling of grammar that was taught
through performance-based instruction with no English explanation at all. Apparently,
this particular student expected the instructor to use English to explain and analyze
grammatical structures, but this was not encouraged in a full immersion setting where
only the target language could be used as a best practice.

Two students mentioned that it was unnecessary to take notes during videos
without explicating reasons or a justification. One may have experienced technology
glitches and could not get the note-taking software to work, and another found it was hard
to access the notes online afterwards and preferred paper notes. This leaves the
researchers to ponder upon the issue of whether online notes or traditional paper-and-
pencil notes bring about more convenience and desirable outcomes. If notes were made
readily accessible afterwards, then the objection could have been easily removed.
Echoing this point of view, one student requested a physical paper with preview items on
it to aid in watching the videos. Several others who provided additional comments
unanimously shared positive feedback on preview questions. The preview questions not
only showed learners the area where they needed to improve but also helped them check
for understanding and retain knowledge. Besides, the tool used to access and respond to
preview questions was easy to use, as added by one student.

Additional comments on receiving instant feedback were overwhelmingly

positive. Time commitment to instant feedback was reasonable and manageable. The
answer keys helped learners “learn grammar,” “see mistakes,” and “understand what to
improve and why.” As further noted by three students, the process was “fast, useful, and
efficient.” While one student agreed that links to email were good, another noted that
“email feedback was less convenient; immediate feedback on the site would be better.”
No one experienced any technology glitches while receiving instant feedback via email,
and they just had to check emails.

Overall, the preview section really helped learners “feel prepared and ready for
the next day, but without being overwhelming.” The value of watching videos is well-
acknowledged by learners. Taking notes helped “make material sink in,” although several
comments demonstrate a need for further thoughts about note-taking. Responding to
preview questions helped to “get the best STARTALK experience possible,” and students
wanted to retain receiving instant feedback in the future program.

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 34
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

3.3 Pedagogical implications and applications

This study contributes to our understanding of effective flipped learning by

creating a well-sequenced and structured four-stage flip learning process. Its applicability
in secondary and post-secondary classrooms during the regular semester is perceived to
have high interest for language teachers, supervisors, and administrators. The four-stage
flipped learning model allows instructors to constantly and systematically monitor
students’ daily progress through a multi-functioned 1know centralized online operational
system that was created for the purpose of the study. Undoubtedly, the planning of each
single stage is never an easy process. It takes full devotion among teaching and
administrative staff on the team.

3.3.1 Creating the first stage of flipped learning

Most teachers tend to have reservations about the required time commitment to
create instructional videos. Flipped learning actually engages students to be exposed to a
wide variety of learning materials and does not necessarily require teachers to create
videos at the expense of their own time. There are different types of materials that can be
adopted and used for flipped learning with ease, such as YouTube videos, online films,
podcasts, newspapers, TV news, audio files, and a large variety of authentic materials
geared toward four language skills across three communicative modes. Watching videos
is just one of many effective ways for pre-class preparation, but listening to sound files,
reading an article, writing a brief email, or recording a short response are some examples
of what students can do to prepare before class. As technology continues to grow and
gain popularity, online resources have become more accessible, diversified, and versatile,
and teachers do not need to work alone. Collaboration and professional exchanges save
teachers time and help achieve mutual goals.

3.3.2 Alternatives to the online platform for flipped learning

Conceivably, creating the multi-functioned 1know central system that was

particularly developed for the purpose of this study is beyond what a foreign language
teacher can do. Most schools may have limited personnel support, lack of expertise and
knowledge, and budget constraints. A possible resolution to this situation is to use any
online platform that is available and whatever tools teachers feel familiar with and
confident about. In this regard, checking for understanding can be made possible, and it is
highly recommended that comprehension be included in whatever format to ensure that
learners do what they are expected to do and that learning does take place before class.
Providing instant feedback, if not doable through technology, can be replaced by other
convenient options that achieve similar evaluative and curricular goals, such as self-
checking or a small-scale follow-up evaluation. Alternatives to the adoption of
technology tools and for the three stages that follow video watching in order to achieve
the same curricular objectives are crucial and of great interest to language teachers. But
that is not the focus of this study and will be addressed in a separate paper.

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 35
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

3.3.3 Time allocation during regular semesters

The program was conducted in an intensive summer STARTALK combination

program when student participants did not take any course other than learning Chinese,
so they were able to fully concentrate on completing required work before and after class.
The group of learners was selected from a large group of applicants, and recruited
students were highly motivated, well-disciplined, and willing to put in their greatest effort
to meet curricular standards. One may argue that while the flipped components work well
in such an intensive summer program, they may not work well during regular semesters.
As acknowledged by many Chinese language teachers who teach in high schools in the
US, they are not allowed to give students too much homework after class and therefore
have reservations about assigning preview work in accordance with school policies. This
leads them to give little or no work for students to do before entering the classroom, and
the result is that all materials have to be taught by the instructor in class, and what can be
reasonably and readily self-learned before class is postponed to limited class time.

An allocation of less than 20 minutes for preview work prior to class time is
indeed feasible and greatly beneficial to both teachers and students. At any rate,
instructional videos are supposed to be as short as 1 to 2 minutes, and no more than 3
minutes. Adding questions for comprehension checks and instant feedback, but excluding
note-taking, the preview exercise will not exceed 20 minutes and deplete too much of
students’ time at home. Wisely arranged, pre-class materials for self-paced flipped
learning can be divided into several small segments as needed. To decrease the
complexity level on technology, a two-step design that involves videos and
comprehension questions may still fulfill pedagogical needs that instructors aim to
achieve.

3.4 Conclusion

This study yields practical applications and potential for future research, but like
many other studies on pedagogical innovation, it has perceivable limitations. The study
had a small sample size at the novice level during a short, intensive summer program. To
what extent the results of the study are applicable to other proficiency levels in different
types of educational settings is unknown. Future studies should increase the sample size,
extend to other proficiency levels, experiment in different courses and programs, and if
feasible, include both control and experimental groups to make possible a comparison of
students’ performance in light of scores of pre-program and post-program tests and other
aspects of analysis.

The pedagogical value of note-taking cannot be ignored despite the fact that
students did not strongly favor it. Online note-taking accompanying video-watching is an
area that deserves in-depth discussion. When learners move to the intermediate and
advanced levels, the content of their note-taking may change. In this study, the major
note type is the repetition of video content, which accounts for 90% of all notes, leaving
10% to be questions and reminders. The content of elementary Chinese is not composed
of a wide range of vocabulary, and the complexity of syntactic structures, functions, and

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 36
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

pragmatic use is not as high as that at the intermediate and advanced levels. It is thus
hypothesized that the composition and type of notes may alter in intermediate and
advanced Chinese. This poses another interesting research question to study further.

Online note-taking in this study was designed to be a one-way flipped learning

component, not a two-way communication. Assuming that learners benefit more from
two-way interactive communication, tweaking online note-taking to two-way interactive
learning may increase the level of engagement, but this would have become labor
intensive and requires more time commitment from instructors or teaching assistants. It is
assumed that one-on-one connections between individual students and the instructor will
motivate students to work harder, and teachers will be better able to interact with students
online. To what extent one-way or two-way note-taking affects learners’ input and
learning outcomes is another new area to explore in future studies.

Note-taking strategies are another domain to attend to as well. Although student

participants attended an orientation provided by the technology specialist and the leading
teacher of the student program, how to take notes to best retain content while watching
videos was not the focus of the orientation. If students were well guided to learn effective
note-taking strategies, such instruction could affect note-taking content. This, however,
went beyond the scope of the study but deserves further exploration.

It is hoped that the same or revised flipped learning model in the study be
duplicated, modified, and extended to different levels of Chinese language courses and
different types of Chinese language programs domestically or abroad. Further studies
with such experimental measurement can deepen our understanding of the effective
design and implementation of flipped learning in Chinese as a foreign language across
different proficiency levels.

References
Baker, J. W. (2000). The “classroom flip”: Using web course management tools to
become the guide by the side. In J. A. Chambers (Ed.), Selected papers from the
11th international conference on college teaching and learning (pp. 9-17).
Jacksonville, FL: Florida Community College at Jacksonville.
Bergmann, J., & Sams, A. (2012a). Flip your classroom: Reach every student in every
class every day. Eugene, OR: International Society for Technology in Education.
Bergmann, J., & Sams, A. (2012b). Before you flip, consider this. Phi Delta Kappan,
94(2), 25.
Berrett, D. (2012). How “flipping” the classroom can improve the traditional lecture. The
Chronicle of Higher Education, 58(25), 16-18.
Biemiller, L. (2013). San Jose State U. says replacing live lectures with videos increased
test scores. The Chronicle of Higher Education. Retrieved from
http://chronicle.com/blogs/wiredcampus/san-jose-state-u-saysreplacing-live-
lectures-with-videos-increased-test-scores/40470
Bonnell, C., & Eison, J. (1991). Active learning: Creating excitement in the classroom.
Washington, DC: George Washington University Press.
© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 37
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

Davis, C. (2013). Flipped or inverted learning: Strategies for course design. In E. G.

Smyth & J. X. Volkder (Eds.), Enhancing instruction with visual media: Utilizing
video and lecture caputer (pp. 241-265). Hershey, PA: IGI Global.
Deslauries, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-
enrollment physics class. Science, 332, 862-864. doi:10.1126/science.1201783
Enfield, J. (2013). Looking at the impact of the flipped classroom model of instruction on
undergraduate multimedia students at CSUN. TechTrends, 57 (6), 14-27. doi:
10.1007/s11528-013-0698-1
Fulton, K. P. (2012). 10 Reasons to Flip. Phi Delta Kappan, 94(2), 20-24.
Fulton, K. P. (2013). Byron’s flipped classrooms. Retrieved from
http://www.aasa.org/content.aspx?id=27488
Graham, C. R. (2006). Blended learning systems: Definitions, current trends, and future
directions. In C. J. Bonk & C. R. Graham (Eds.), The handbook of blended
learning: Global perspectives, local designs (pp. 3-21). San Francisco, CA:
Pfeiffer Publishing.
Hamden, N., McKnight, P., McKnight, K., & Arfstrom, K. (2013). The flipped learning
model: A white paper based on the literature review titled A review of flipped
learning. Retrieved from
http://www.flippedlearning.org/cms/lib07/VA01923112/Centricity/Domain/41/W
hitePaper_FlippedLearning.pdf
Herreid, C. F., & Schiller, N. A. (2013). Case studies and the flipped classroom. Journal
of College Science Teaching, 42(5), 62-66.
Hung, H. (2015). Flipping the classroom for English language learners to foster active
learning. Computer Assisted Language Learning, 28(1), 81-96,
doi:10.1080/09588221.2014.967701.
Lage, M. J., & Platt, G. (2000). The internet and the inverted classroom. Journal of
Economic Education, 31(1), 11.
Lage, M. J., Platt, G., & Treglia, M. (2000). Inverting the classroom: A gateway to
creating an inclusive learning environment. The Journal of Economic Education,
31(1), 30-43.
Mehring, J. (2014). An exploratory study of the lived experiences of Japanese
undergraduate EFL students in the flipped classroom (Unpublished doctoral
dissertation). Pepperdine University, Malibu, CA, USA.
Moran, C. (2014). Changing Paradigms: A Mixed Methods Study of Flipping the English
Language Arts Classroom (Unpublished doctoral dissertation). North Carolina
State University, Raleigh, NC, USA.
Novak, G. (2011). Just-in-time teaching. New Directions for Teaching and learning, 128,
63-73.
Ryan, B. (2013). Flipping over: Student-centered learning and assessment. Journal of
Perspectives in Applied Academic Practice, 1 (2), 30-39. doi:
10.14297/jpaap.v1i2.64
Strayer, J. F. (2007). The effects of the classroom flip on the learning environment: A
comparison of learning activity in a traditional classroom and a flip classroom
that used an intelligent tutoring system (Unpublished doctoral Dissertation).
Retrieved from
http://faculty.washington.edu/rvanderp/DLData/FlippingClassDis.pdf

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 38
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

Salazar, H., & Sinclair K. (2015, November). Improving L2 lexical competency through
flipping the classroom. Paper presented at the 2015 ACTFL Annual Convention
and World Languages Expo. San Diego, CA, USA.
Staker, H., & Horn, M. (2012). Classifying K-12 blended learning. Retrieved from
http://www.christenseninstitute.org/wp-content/uploads/2013/04/Classifying-K-
12-blended-learning.pdf.
Tucker, B. (2012). The flipped classroom: Online instruction at home frees class time for
learning. Education Next, 12(1), 82-83. Retrieved from
http://educationnext.org/files/ednext_20121_BTucker.pdf
Ullman, E. (2013). Tips and tools for the flipped classroom. Retrieved from
http://www.techlearning.com/features/0039/tools-and-tips-for-the-flipped-
classroom/53725.

Appendix 1

Questionnaire on Learners’ Input and Comments on the Four-Stage Flipped

Learning

Instructions: Thank you for participating in the 2015 Virginia STARTALK Chinese
Student Academy. This year we have pilot-tested before-class flipped classroom
components and homework components. We would like to get your feedback on these.
Circle the number that best reflects your learning experiences. Your responses will be
kept confidential and anonymous and will have no impact on your position in this
program. Thank you very much for your participation and feedback.
Before-class flipped classroom components:

1. Watching videos
2. Watching videos while taking notes
3. Responding to questions via Google Form
4. Receiving instant feedback via email

A. Motivation
1. I am motivated to watch the videos before class.
1 2 3 4 5
Least Agree Strongly agree
Additional comment:

2. I am motivated to take notes while watching videos before class.

1 2 3 4 5
Least Agree Strongly agree
Additional comment:

3. I am motivated to work on comprehension questions in multiple-choice questions

before class.

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 39
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

1 2 3 4 5
Least Agree Strongly agree
Additional comment:

4. I am motivated to read the results (score & explanation) of my responses to the

multiple-choice questions before class.
1 2 3 4 5
Least Agree Strongly agree
Additional comment:

B. Effects
5. Watching videos before class helps me prepare for the content that I will learn on the
following day.
1 2 3 4 5
Least Agree Strongly agree
Additional comment:

6. Taking notes while watching videos helps me learn the content of the videos better.
1 2 3 4 5
Least Agree Strongly agree
Additional comment:

7. Responding to comprehension questions (multiple-choice) after watching the videos

helps me understand how well I learn.
1 2 3 4 5
Least Agree Strongly agree
Additional comment:

8. The instant feedback (score and explanation) helps me better understand how well I
learn.
1 2 3 4 5
Least Agree Strongly agree
Additional comment:

C. Time commitment
Please write down how much time on average you spend on each of the following
components of the flipped classroom.

9. Taking notes while watching videos

Average time spent per day: ( )

10. Responding to comprehension questions

Average time spent per day: ( )

11. Receiving instant feedback on my responses to comprehension questions

Average time spent per day: ( )

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 40
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

D. Use of technology
Comment on the friendliness of the technology tools that the program created for the
following component.

12. Watching videos

1 2 3 4 5
Least User Friendly Very User Friendly
Additional comment:

13. Taking notes (iNote) while watching videos

1 2 3 4 5
Least User Friendly Very User Friendly
Additional comment:

14. Responding to comprehension questions

1 2 3 4 5
Least User Friendly Very User Friendly
Additional comment:

15. Receiving instant feedback on my responses to comprehension questions

1 2 3 4 5
Least User Friendly Very User Friendly
Additional comment:

E. Recommendation
Recommendation for the flipped classroom components in next year’s STARTALK
program

16. Watching videos

1 2 3 4 5
Least recommended Strongly recommended
Additional comment:

17. Taking notes while watching videos

1 2 3 4 5
Least recommended Strongly recommended
Additional comment:

18. Responding to comprehension questions

1 2 3 4 5
Least recommended Strongly recommended
Additional comment:

19. Receiving instant feedback on my responses to comprehension questions

1 2 3 4 5

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 41
Tseng, Mann, Chen An Investigation of the Design of a Four-stage Flipped Classroom

Least recommended Strongly recommended

Additional comment:

20. Please write 3-5 sentences to provide your overall comments and suggestions on the
flipped classroom components.

© 2016 The Authors. Compilation © 2016 Journal of Technology and Chinese Language Teaching 42