Pertanika J. Soc. Sci. & Hum.

27 (3): 1709 - 1725 (2019)

SOCIAL SCIENCES & HUMANITIES

Journal homepage: http://www.pertanika.upm.edu.my/

Building Higher Order Thinking Skills (Hots) for Pre-Service

Vocational Engineering Teachers by Applying a Multi-Method
Approach to the Learning Process through Lesson Study
Anis Rahmawati*, Aryanti Nurhidayati and Ida Nugroho Saputro

Building Engineering Education, Faculty of Teacher Training and Education, Universitas Sebelas Maret,
Surakarta, Central Java 57126, Indonesia

ABSTRACT
This article aims to examine the implementation of a multi-method approach in the
vocational engineering learning process through a Lesson Study to develop higher order
thinking skills (HOTS) for pre-service teachers. This is a multiphase mixed methods
design which included a series of Lesson Study activities. In the implementation phase,
the multi-method learning was started through self-directed learning, followed by
collaborative problem-solving by using a mixture of several techniques. The last phase was
a summarization session. The results of this study indicate that a multi-method approach
has the potential to develop pre-service teachers’ HOTS.

Keywords: Higher order thinking skills, lesson

study, multi-method, pre-service teacher, vocational
engineering

INTRODUCTION
ARTICLE INFO
Higher order thinking skills (HOTS) is one
of the critical abilites found in the learning
Article history:
Received: 9 December 2017 process, especially among higher education
Accepted: 1 July 2019
Published: 13 September 2019 institutions. It constitutes as the highest
level in the hierarchy of cognitive process.
It includes critical, logical, reflective,
E-mail addresses:
[email protected] (Anis Rahmawati) metacognitive, and creative thinking. Such
[email protected] (Aryanti Nurhidayati)
[email protected] (Ida Nugroho Saputro) skills enable students to face numerous
*Corresponding author
challenges; in an era when there is so much
information available, but the time to
ISSN: 0128-7702
e-ISSN: 2231-8534 © Universiti Putra Malaysia Press
 Anis Rahmawati, Aryanti Nurhidayati, and Ida Nugroho Saputro

process it is limited (Yee et al., 2015), HOTS Co-operation and Development [OECD],
encourages individuals to interpret, analyse 2015). Furthermore, vocational education
and manipulate information. and training provide more direct pathways
HOTS is more difficult to be learnt or to the labour market (OECD, 2016).
taught than other types of skills. However, The increasing number of students
it is also more valuable because it will bring at vocational institutions should be
beneficial when faced with a new different accompanied by improvements in the quality
situation compared with the learnt skill. In of vocational education. High-quality
the information era, mastery of HOTS has vocational education will be achieved if
become a necessary competency in order the process of vocational teaching and
to address a variety of information which learning is implemented effectively and met
sometimes contradicts one another (Brierton with high standard. Research has shown
et al., 2016). that the quality of teaching is a decisive
Pre-service vocational engineering factor in improving the learning outcomes
teachers are prepared to be taught in of students. The results of a recent review
vocational secondary schools or Technical of 20 of the world’s top education systems
and Vocational Education Training (TVET). concluded that the quality of the education
TVET has aims to make workers be more system could not exceed the quality of its
incisive and enable them to interact better teachers. So, it is suggested that the only
with their job duties (Yusop et al., 2015). way to improve student learning outcomes
In Indonesia, formal vocational education is to enhance the instructional process
and training are offered at the secondary managed by teachers (Barber & Mourshed,
level through SMK (Sekolah Menengah 2007). Teachers are required to be able
Kejuruan), under the management of the to develop the learning process through
Directorate General of Senior Secondary the learning objectives. In line with the
Education, which is under assitance of the objective of vocational learning, which is
Ministry of Education and Culture (MOEC). to prepare students for work, vocational
The new mission of MOEC is to increase teachers have to be able to follow the
the number of learners enrolling in SMK development of science and technology
schools to achieve a greater proportion in their field. This capability will be more
(60%) compared to those studying in easily achieved when teachers have HOTS,
academic senior secondary high schools or due to the dynamic nature of science and
Sekolah Menengah Atas (SMA). This effort technology.
is being undertaken to reduce their workers’ To enable the learner to retain HOTS
skill mismatches. Thus, it is expected to for a longer period of time, or even forever,
increase job creation, and to support the each particular skill is not applied through
productivity, competitiveness, and growth a single technique or process; rather, a
of the country (Organisation for Economic combination of all of these processes is

1710 Pertanika J. Soc. Sci. & Hum.27 (3): 1709 - 1725 (2019)
 Building Hots by Applying a Multi-Method Approach

employed to develop a unique learning Lewis et al. (2006) called for Lesson
experience. The opportunity to develop Study research in three areas: “(1)
critical thinking skills can be obtained expansion of the descriptive knowledge
if the teacher purposely and persistently base on Lesson Study; (2) explanation of
practices higher order thinking strategies in the Lesson Study’s procedure (that results
the implementation of teaching and learning, in instructional improvement); and (3)
for example through open-ended class iterative cycles of testing and refinement of
discussions, real-world problem solving, and Lesson Study”. This research was focused
encouraging inquiry-oriented experiments on repeating cycles of testing and refinement
(Miri et al., 2007). Teaching collaboratively of Lesson Study by applying a multi-method
provides the instructor with an opportunity approach, which comprised a combination
to improve their teaching practice and makes of three learning methods, i.e., self-directed
the professional development of the teacher learning, collaborative problem-solving,
more effective (Jao & Mcdougall, 2015). and summarisation, in a learning process to
Lesson Study has been used widely in develop students’ HOTS. The multi-method
Japan for many decades as a professional approach was employed to optimise the
development approach that is often credited benefits and minimise the disadvantages
for the eternal restoration of Japanese of each model, especially with the aim
learning instruction (Fernandez & Yoshida of developing the HOTS of pre-service
2004). This professional development vocational engineering teachers.
practice engages teachers in the process The research goals were to answer
of systematically examining their teaching the following questions: 1) what does the
to become more effective (Myers, 2013). implementation of Lesson Study cycles with
Lesson Study is a professional learning multi-method learning for the engineering
approach in which a group of teachers works learning process look like? 2) How effective
together to: formulate learning objectives is Lesson Study with multi-method learning
and long-term development; plan a design in contributing towards HOTS among pre-
for a research lesson as an attempt to achieve service vocational engineering teachers?
those objectives; carry out the lesson in a In this study, the development of HOTS in
classroom, with one team member teaching pre-service vocational engineering teachers
and the others gathering evidence on student through the application of multi-method
learning and development; and then discuss learning will focus on analytical and critical
and debrief the live lessons and evidence thinking.
gathered during the learning process, and
use the findings to improve the quality of METHODOLOGY
both research lessons and lessons in general Research Design
(Lewis, 2009).
This is a multiphase mixed methods design
(Creswell, 2014) which was conducted

Pertanika J. Soc. Sci. & Hum. 27 (3): 1709 - 1725 (2019) 1711
 Anis Rahmawati, Aryanti Nurhidayati, and Ida Nugroho Saputro

following a series of Lesson Study activities the next session that is a debriefing session.
adapted from Cerbin and Kopp (2011), as Its session delivers transcript as qualitative
shown in Figure 1. data, which the information within the
A multiphase mixed methods design transcript is used as the main reason to
was chosen in this research approach carry out the next cycle of the lesson study
because it is suitable with the objective of program.
this research that was to understand the need This article focuses on HOTS
for an impact of an intervention program, development in analytical and critical
i.e. the lesson study. This program consists thinking through lesson study research.
of multiple phases which may go back and Analytical skill builds as the ability to make
forth between quantitative and qualitative decisions based on the information gathered.
studies, but they build on each other to Critical thinking is described as the ability
address a program objective. in deciding what to believe or do based on
Data obtained from the plan session was reasonable reflective thinking. Pre-service
a set of the lesson plan. The information teachers’ scores for analytical and critical
obtained based on data in the plan session skill are gained from their collection of
becomes the basis of the next session, articles and the literature reviews that they
i.e. the implementation stage. At this created relating to the subject matter, the
stage, the gathered data consists of pre- observation during class discussion, and the
service teacher’s paperwork, presentation summaries they made.
materials, summaries, and the observation To find out how the pre-service teacher
sheets. The result of data analysis from the understood with the collected article, a
implementation stage gives information for rubric for doing a content analysis of the

Figure 1. Steps in lesson study

1712 Pertanika J. Soc. Sci. & Hum.27 (3): 1709 - 1725 (2019)
 Building Hots by Applying a Multi-Method Approach

pre-service teacher’ written reviews was Participants

made. The analysis itself involves an In this study, groups consisting of three
iterative process. The article and the written Steel Structure engineering instructors
reviews is read repeatedly, interpreting collaboratively attempted to enhance HOTS
and reinterpreting the written reviews, through Lesson Study research. All pre-
and searching for part of the article that service vocational engineering teachers
corresponds to written reviews. The rubric attending the Steel Structure course at a
consists of items for assessing analytical Teacher Training and Education Department
skill in terms of collecting and organising from a state university in Indonesia (n=68)
information, analysing information, reading were invited to participate. The pre-service
with comprehension, and identifying teachers were in their fourth semester of a
appropriate materials. While for assessing 4-year undergraduate pre-service teacher
critical thinking, the rubrics include items programme.
to identify the source of information,
to compare such information to prior Procedures
knowledge, and to form conclusions. Lesson Study activities was begun with
The other data regarding to analytical the formation of the team. Once the team
skill are collected from written summaries. was formed, the members formulated
Pre-service teachers forced to do such lesson objectives as listed in the syllabus.
analytical activities described above in In the next step, the research lesson was
order to make it not only brief but also planned by the team by designing the type
comprehensive one. A rubric for scoring of learning to be applied. Here, learning
analytical skill based on summaries data was designed using a combination of three
consists of several key items in the relevant learning models, i.e., self-directed learning,
topic. collaborative problem-based learning, and
The observation sheet provides data summarisation. In the implementation
regarding critical skill which is collected phase, the learning process using a multi-
during class discussion. The focus of the method approach begun with self-directed
observation is on critical skill ability in learning, in which pre-service teachers were
directed to collect information about the
terms of identifying conclusions, reasons,
subject matter from various sources using
and assumptions; the quality of an argument;
Web-based Instruction. In the next session,
develop and defend an idea; and draw
collaborative problem-based learning was
conclusions (Larsson, 2017). The number
performed. A mix of several techniques, i.e.,
of pre-service teachers that is shown those
Make a Match, Pre-service Teacher Team
abilities listed on the frequency tables as a
Achievement Division, and Number Heads
part of the observation sheet along with the
Together were used in this session. The last
description of the abilities shown. session involved each pre-service teacher
summarising the process in their own words.

Pertanika J. Soc. Sci. & Hum. 27 (3): 1709 - 1725 (2019) 1713
 Anis Rahmawati, Aryanti Nurhidayati, and Ida Nugroho Saputro

During the implementation stage, Collaborative problem-based learning

one of the team members served as a was performed by using a mix of several
teacher and the others as observers. Those techniques, i.e., Make a Match, Pre-service
tasked with observing the course of the Teacher Team Achievement Division,
learning process were to gather data using and Number Heads Together. The lesson
observation guidelines that had been plans describe clearly each step of the
prepared previously (Cerbin & Kopp, 2011). instructional design applied in this lesson
The data observation results were divided study research. Each pre-service teacher was
into two categories. The first comprised data given a random card. Their group number
that came from the pre-service teachers’ and the problems that must be solved were
assignments, i.e., paperwork, presentation listed on the card. Pre-service teachers with
materials, summaries, quizzes, and final the same group number were gathered into
tests. The remaining data came from pre- one group. There were 13 groups involved
service teacher activity observation sheets. in the learning activities with each group
In the next stage, the team analyzed the consists of 5-6 members. The problems
results of the implementation of the learning to be solved were different for each group
model and assessed the pre-service teachers’ member. All the problems were discussed
progress towards the learning goals through with team members, although each member
a debriefing session. If necessary, the team of the group was responsible for solving the
revised the lesson plan to produce the problems stated on their own card. During
optimal design based on the results of the the solving of various problems, pre-service
analysis. The transcripts from this debriefing teachers performed high order thinking
session were collected. actions, from accessing their knowledge
of law formulations and the respective
RESULTS conditions of the studied process to drawing
The Implementation of Lesson Study connections between the characteristics
Cycles with Multi-Method Learning of the processes already known and those
unknown. They then explained and made a
In the First Cycle of Lesson Study. The
quantitative assessment of the results of the
first cycle began with self-directed learning;
study (Politsinsky et al., 2015).
all pre-service teachers had collected related
All the members of a group were to
papers from several sources and produced
be ready if selected to present the results
article reviews. However, this session was
of their work in class at the end of the
not taken seriously by most pre-service
session. Assessments of discussions and
teachers. The pre-service teachers did not
presentations were carried out during the
read their articles well. This was reflected
session. Then, each of the pre-service
from the article reviews, which merely
teachers was given a quiz. This quiz was
resembled very short summaries.
performed to determine the effectiveness

1714 Pertanika J. Soc. Sci. & Hum.27 (3): 1709 - 1725 (2019)
 Building Hots by Applying a Multi-Method Approach

of the learning method, as well as forming pre-service teachers during the discussion
part of the achievements of each group. The with the group and their position in the
group with the highest scores was named summarisation session need to be set so that
the best group and given a reward. In the the purpose of each session can be achieved.
last session, each pre-service teacher had to 3) The instructor should give equal attention
provide a summary of the process in their to all pre-service teachers. Pre-service
own words, and then the final test was taken. teachers who did not actively need to be
During the collaborative problem- given more attention should be motivated to
solving method, the activeness of the pre- become more active. 4) Time management
service teachers was not evenly distributed implementation in the classroom needs to
in the classroom. Only those who already be improved. The instructor should be more
understand the purpose of learning and active in instructing pre-service teachers to
highly motivated to learn were seen to be use the time more effectively, especially
active during the learning process. The when switching between sessions.
instructor appeared in dividing their attention
throughout the class. Summarisation also In the Second Cycle of Lesson Study.The
did not receive enough attention. Pre-service second cycle again started with self-directed
teachers made their summaries carelessly; learning. This involved pre-service teachers
moreover, some of them even cheated by collected articles from the web, and then
copying the peers’ summaries. asked to create a literature review which
Based on observations made during contained a summary of the content of the
the implementation of Lesson Study in the article, the strengths, and weaknesses of
first cycle, the monitoring team then held review articles, and guidance on how to
a debriefing session. In this session, the apply the information from the article in
team analyzed the implementation of the the subject course. Through this technique,
learning process by using a multi-method the pre-service teachers were encouraged to
approach, the difficulties encountered, and read by the aim of understanding, analyzing
the achievement of the pre-service teacher and identifying information.
learning outcomes associated with learning To further streamline the discussion
objectives. They then agreed to some groups, the number of group members in
refinements for the Lesson Study process. the collaborative problem-based learning
Those refinements were: 1) The instructor sessions was reduced to 3–4 pre-service
should more actively cultivate the curiosity teachers. The class setting for discussions
of pre-service teachers with the inducement would be easier for a small group. Instructors
of questions. 2) The classroom needs to be needed to be more active in instructing
rearranged. The monitoring team needs to all pre-service teachers to move more
be positioned so as not to interfere with quickly when forming groups based on the
the mobility of teachers. The positions of distribution of the cards.

Pertanika J. Soc. Sci. & Hum. 27 (3): 1709 - 1725 (2019) 1715
 Anis Rahmawati, Aryanti Nurhidayati, and Ida Nugroho Saputro

Before starting classes, pre-service criterion was based on the applicable

teachers were asked to pose some questions standard assessment in the institution where
about aspects of the subject to be discussed this study was conducted. Pre-service
that they had not understood based on teachers were graded according to the score
the literature they had read previously. achieved. For critical thinking skill derived
Then, the pre-service teachers exchanged from the frequency tables, students’ scores
questions and answers via a group quiz are based on the number of ability criteria
game. The group who could answer the indicated during observation. Pre-service
most questions correctly would receive teachers who showed three or more abilities
a reward. The instructor acted as a judge were in the “very good” category. It revealed
and also clarified the answers given by the two other abilities in the “good category”,
pre-service teachers. With this technique, while bringing out one ability was in the fair
the questions discussed in class during category, and if it did not show any ability
the problem-based learning session were will be categorized as “poor”.
expected to focus on issues that had not The pre-service teachers’ score related
been understood by the pre-service teachers. to the HOTS from the first cycle showed
An outline for summarisation was given to that some of them reached “very good”
ensure the pre-service teachers’ summaries achievement (21.9%), while 40.6% were
were better planned and more orderly. deemed “good”. Fewer pre-service teachers
fell into the “fair” and “poor” categories
The Contribution of Multi-method (25% and 12.5% respectively). Meanwhile,
Learning in Developing HOTS
the results of the development of HOTS
Pre-service teachers’ score on 100 scales among pre-service teachers in the second
related to HOTS studied here were cycle showed that 34.4% had the very good
obtained by using the assessment rubrics. achievement and 28.1% were good, while
The final score was the average from 31.3% were fair and only 6.3% remained
all items which was observed covering poor.
the ability in collecting and organizing This research did not mention the initial
information, analyzing information, score because HOTS score here derived
reading with comprehension, identifying from data collected during the lesson study
appropriate materials, identifying the source program, which did not apply before. From
of information, comparing such information the two cycles of lesson study conducted, the
to prior knowledge, forming conclusions, number of pre-service teachers who reached
and drawing up a comprehensive summary. very good achievement was increasing
The scores then were divided into 12.5%, while the number who placed
fourth categories involving; “very good” in the poor level was decreasing 6.2%.
for score 80-100; “good” for score 70-79; However, the paired t- test analysis – found
“fair” for score 60-69; and “poor” category that there was no significant difference
for score <60. Score distribution for each

1716 Pertanika J. Soc. Sci. & Hum.27 (3): 1709 - 1725 (2019)
 Building Hots by Applying a Multi-Method Approach

(N= 68, T= -1.38, P= 0.173), between the During the collaborative problem-
first cycle score (mean= 71.56, SD= 7.69) solving method, pre-service teachers
and the second cycle score (mean= 73.31, enhanced their critical and analytical
SD=8.75). This showed that although thinking skills through problem-solving
there was an increasing level of HOTS activities, which involved exchanging
achievement but not significantly made ideas in a discussion, creating new ideas,
different. Implementing and improving and evaluating them. Moreover, they
lesson study in the next cycle needs to be taught and learned with their peers. They
done. Therefore, more significant HOTS also learned how to communicate their
enhancement can be obtained. . understanding, analyze other opinions,
HOTS was developed through a series receive the appropriate opinions, and to
of activities, started with self-directed draw suitable conclusions from the data
learning, to test pre-service teachers’ ability provided. Creating a summary aimed to
to collect, organise, and analyze information. develop the pre-service teachers’ skills in
This learning method also contributes mapping information and, in turn, improve
in developing the ability to read with their analytical and critical thinking abilities,
understanding and to identify the materials as well as to instil a deeper understanding of
needed and not. The results were seen in the the meaning of information.
articles derived from the collected materials
and in the literature reviews they produced. DISCUSSION
The pre-service teachers demonstrated their The Implementation of Lesson Study
ability to identify the source of a piece of Cycles with Multi-method Learning
information, analyze its credibility, reflect The implementation of Lesson Study
the consistency of information, and make in the daily learning process is in line
conclusion about the subject matter based with efforts being undertaken to achieve
on the information that they had collected. high-quality and sustainable professional
However, the analytical skills of the pre- development of teachers. Lesson Study can
service teachers still needed to be improved. be adapted to meet the needs of teachers
Their abilities regarding to analyzing and to satisfy certain professional learning
information, reading comprehension, and outcomes (Saito, 2012). Further, Saito
identifying the materials needed were not (2012) suggested that it was important
refined well. This can be observed from their for teachers to consider how to enhance
performance in the quiz, where most pre- their quality of learning by discarding
service teachers experienced difficulty in unnecessary activities and replacing them
solving the problems, despite the questions with activities that were more meaningful;
being related to the articles that they had this strategy must then be applied in daily
submitted. teaching.

Pertanika J. Soc. Sci. & Hum. 27 (3): 1709 - 1725 (2019) 1717
 Anis Rahmawati, Aryanti Nurhidayati, and Ida Nugroho Saputro

The multi-method approach which for the subject before entering the classroom.
was applied in this Lesson Study allowed Self-directed learning is a central element
pre-service teachers to create, integrate, in higher education and encompasses
and generalise knowledge, one of the main any learning activities that take place
attributes of learning (Bicknell-Holmes & outside of class hours or scheduled events.
Hoffman, 2000). This was an active form Independent activities may be in the form
of learning, which was based on the process of online learning, studying in the library, or
and not on facts. The pre-service teachers group work. Online learning environments
learned from their mistakes and failures help the students to gain knowledge and
during the process. Feedback was required develop their views and interests in a way
to ensure pre-service teachers gained a that enhances their novel and adaptive
deeper understanding. Such an approach thinking (Kapenieks, 2016). In self-directed
distinguishes between inventive learning learning, something about going to happen
and traditional learning (Bonwell, 1998; depends on the individual’s commitment
Mosca & Howard 1997; Papert, 2000). and initiative. Individuals select, organize
Pineteh (2012) proposed that lecturers and evaluate their learning activities, which
should be able to creatively integrate can be performed at anytime, anywhere, by
traditional teaching and learning methods any means, and at any age. Self-directed
with technological advances so that the learning activities can be carried out by
learning process would be more engaging. teachers in schools at every opportunity.
In this Lesson Study, a multi-method Collaborative problem-based learning
approach was applied to the subject of “Steel is an educational approach to teaching and
Structure” to enhance the HOTS of pre- learning which involves groups of pre-
service vocational engineering teachers. The service teachers working together to solve
multi-method approach was combined with a problem. The problem-solving process
three learning methods, i.e., self-directed provides an opportunity to understand
learning, collaborative problem-solving, the topic in more depth and to develop a
and summarisation. Through Lesson Study, comprehensive idea based on the realities
pre-service teachers were considered to encountered. The skill of problem-solving
have become more capable of listening and is the best criterion for evaluating the level
collaborating with peers, as well as having of mastery of learning the material and is
become more confident and respectful (Saito a skill that is required by any professional
& Atencio, 2013). engineer (Politsinsky et al., 2015). Yusop
Self-directed learning was intended to et al. (2015) noted that “Problem-solving
cultivate self-education among pre-service is a series of techniques, both cognitive and
teachers, in order to reduce their total behavioural that can be applied in various
dependency on the lecturers (Din et al., fields or areas to generate solutions”. This
2016) and to ensure they are better prepared definition is in line with the character of

1718 Pertanika J. Soc. Sci. & Hum.27 (3): 1709 - 1725 (2019)
 Building Hots by Applying a Multi-Method Approach

HOTS, i.e., that it does not apply only one The Contribution of Multi-method
technique, process, or skill but rather a learning in developing HOTS
combination of them. For this reason, this Learning methods that are appropriate
study employed problem-solving as part of for teaching and learning not only aid the
the multi-method approach to the learning learning process, but also contribute in
process. The results of previous researchers improving soft skills among pre-service
(Politsinsky et al., 2015) had indicated that teachers (Esa et al., 2015). To ensure that
problem-solving was an effective method these soft skills instilled in pre-service
to teach school and university pre-service teachers, appropriate and better organized
teachers the general methods of mental instructors are required to be more creative
activity, especially analysis. in devising strategies for teaching and
Summarisation is a part of the learning
learning (Morris, 2009). Kuhn (2005) stated
strategy. Several studies have shown the
that teaching focussed on improving HOTS
benefits of summarisation (Anderson &
was essential to equip pre-service teachers
Thiede, 2008; Thiede & Anderson, 2003).
to participate in and contribute to modern
Creating a summary involves a process of
democratic societies.
presenting data from many sources to the
HOTS can be conceptualized as
user to make an abstract concept easier
multiple, non-algorithmic modes of thinking
to understand (Mohan et al., 2016). The
that often deliver multiple solutions (Miri
most significant increases in pre-service
et al., 2007). Learning experiences which
teacher learning outcomes occurred when
are considered to involve HOTS include
summarisation took the form of a kind of
answering questions, decision making, and
recitation. This required VGNA (Verbal,
critical and systemic thinking (Dillon, 2002;
Graphical or Geometric, Numeric, and
Zohar & Dori, 2003; Zoller et al., 2002).
Algebraic) concept activities, in which
Formal education, from elementary school
pre-service teachers were taken out of
to postgraduate level, strives to produce
their passive learning environments and
pre-service teachers who master analytical,
integrated into environments of active
problem-solving, and critical thinking skills.
learning where, through the use of the
This multi-method approach gave
VGNA concept activities, they were able
pre-service teachers the opportunity to
to construct knowledge (Watt et al., 2014).
enhance their analytical skill through
Through analysing pre-service teacher’s
diverse activities, such as analyzing texts
summaries, instructors could pinpoint gaps
and evaluating others’ opinions. Analytical
not only in the teachers’ learning but also in
skill refers to the ability to make decisions,
their knowledge. The instructors were, thus,
in order to resolve both complex and
more adept at identifying their pre-service
uncomplicated problems, based on the
teachers’ needs in order to achieve the next
information gathered. This ability is
steps of the learning process (Limbrick et
necessary for workers to solve problems
al., 2010).
that occur in the workplace and to maintain

Pertanika J. Soc. Sci. & Hum. 27 (3): 1709 - 1725 (2019) 1719
 Anis Rahmawati, Aryanti Nurhidayati, and Ida Nugroho Saputro

productivity, as well as smooth functioning process information and apply it actively

throughout the workplace. in the learning process. Through this
Critical thinking requires a variety learning process, pre-service teachers can
of skills, such as the ability to identify apply the knowledge they have gained to
the source of information, to analyze its new situations encountered in real life, by
credibility, to compare such information to applying the skill of problem-solving. This
prior knowledge, and to form conclusions method is known as problem-based learning
(Linn, 2000). Instructors should facilitate and is usually undertaken collaboratively or
and encourage pre-service teachers’ jointly. Grover (2014) defined collaborative
creative and critical thinking skills, and not learning as “an active learning technique
concentrate too much or be too dependent where pre-service teachers work and learn
on textbooks and their schools’ aspirations, together in groups to accomplish shared
which are usually exam-oriented (Kabilan, goals”. This technique provides pre-service
2000). teachers with the opportunity to enhance
Pre-service teachers from any their critical skills through activities, such as
disciplines need to familiarise themselves exchanging ideas in a discussion, teaching,
with thinking analytically and critically. and learning with peers. Collaborative
Phang et al. (2011) stated that all engineers learning encourages the development
should be inquisitive, analytical and of critical thinking through discussion,
critical. This statement was based on the clarification of ideas, and the evaluation of
fact that all engineers had to deal with a other people’s ideas. This type of learning
variety of theoretical concepts expressed is beneficial in improving critical thinking
in models, such as physical models of and problem-solving skills (Gokhale,
building structure systems, mechanical and 1995). A pre-service teacher with a high
electrical systems, and many others. These level of critical thinking usually will be
models should be embodied in forms that more successful in defining and solving
can be easily realised into actual structures. problems (Susar et al., 2015). The ability
The significance of developing engineering to solve problems can be gained from
pre-service teachers’ critical thinking in creative thinking which, according to Jonāne
their professional careers was recognized (2015), has a powerful connection to the
by Andreu-Andres et al. (2009), Kobzeva development of meaningful learning and
(2015) and Melles (2008). transfer skills.
In the globalization era, there is rapid Research has shown that problem-based
dissemination of diverse ideas, techniques learning develops pre-service teachers’
of production, and results of production. attitudes such that they are more positive and
In relation to the world of education, this encouraging in their approach to learning
rapid process demands quick action, both and helps pre-service teachers to retain
from schools and pre-service teachers, to knowledge longer. With a problem-based

1720 Pertanika J. Soc. Sci. & Hum.27 (3): 1709 - 1725 (2019)
 Building Hots by Applying a Multi-Method Approach

learning approach in a collaborative learning the collaborative problem-based session.

setting, learners have the opportunity to This method provides an opportunity for
communicate with each other, exchange pre-service teachers to communicate their
diverse ideas and knowledge, or develop understanding, as well as to build others’
questions in a conceptual framework. ideas (Barak & Rafaeli, 2004).
Collaborative learning activities do not Summarisation is the process in which
supersede lectures by the teacher, listening, one presents a summary of a topic by
or note-taking; those activities were retained extracting important information from
and formed the basis of discussion group textual sources (Mohan et al., 2016). In
activities (Smith & Mc Gregor, 1992). this study, summarisation was found to be
Politsinsky et al. (2015) stated that helpful in developing pre-service teacher
“Problem-solving allows: - understanding HOTS, especially the skills of analyzing and
and learning the basic laws and equations, identifying information. The attainment of
forming of the idea about their peculiarities all of the pre-service teachers over time was
and boundaries of application; - developing associated with their involvement in Lesson
the skills and ability to use general Study (Had & Jopling, 2016).
laws for solving specific and practical
tasks. The success in problem-solving is CONCLUSION
determined by the skill to perform analysis Based on the above discussion, a multi-
of the engineering situation, described in the method approach to a learning process
problem, which is based on the skill to sort has the potential to develop the HOTS of
out the main thing”. pre-service teachers in terms of analytical
Activities carried out during the and critical thinking. Self-directed learning
educational process are directed to cultivate enhanced analytical and critical thinking
the ability of pre-service teachers such that skills through activities involving identifying
they are not only able to acquire knowledge the source of information, analyzing its
and understand ideas but can also synthesise credibility, reflecting the consistency of
thoughts and evaluate concepts (Brierton information, and making conclusions.
et al., 2016). One method of instilling Collaborative problem-solving similarly
HOTS among pre-service teachers is by enhanced these skills through activities
encouraging them to actively participate in which the teachers exchanged ideas in
in educational discourse. This term refers a discussion, analyzed others’ opinions,
to activities involving discussion and identified the appropriate opinions, and
the exchange of ideas, either verbal or came to a conclusion with regard to diverse
written. In educational discourse, pre- ideas. Summarisation was also found to
service teachers discuss ideas, teach their be helpful in developing skills relating to
peers, and learn with each other. In this analyze and identify information. Applying
research, this was accomplished through a multi-method approach that is tailored

Pertanika J. Soc. Sci. & Hum. 27 (3): 1709 - 1725 (2019) 1721
 Anis Rahmawati, Aryanti Nurhidayati, and Ida Nugroho Saputro

to the characteristics and conditions of the Bonwell, C. C. (1998). Active learning: Energizing
pre-service teachers in each of the learning the classroom. Green Mountain Falls, USA:
Purdue.edu.
environments will improve their HOTS.
The contribution of this article is to provide Brierton, S., Wilson, E., Kistler, M., Flowers, J.,
teachers and lecturers with information & Jones, D. (2016). A comparison of HOTSs
demonstrated in synchronous and asynchronous
on how to design and implement a multi-
online college discussion posts. North American
method approach in their classroom learning
Colleges and Teachers of Agriculture, 60(1),
to develop the HOTS of learners. 14-21.

Cerbin, W., & Kopp, B. (2011). Lesson study guide.

ACKNOWLEDGEMENT Retrieved October 11, 2015, from http://www.
The authors would like to appreciate the uwlax.edu/sotl/lsp/guide.
financial support by DIPA PNBP from Creswell, J. W. (2014). Research design: Qualitative,
Universitas Sebelas Maret [research grant quantitative, and mixed methods approaches (4th
contract number: 632/UN27.21/LT/2016]. ed.). California, USA: Sage.

Dillon, J. (2002). Perspectives on environmental

REFERENCES education-related research in science education.
Anderson, M. C. M., & Thiede, K. W. (2008). Why do International Journal of Science Education, 24,
delayed summaries improve metacomprehension 1111-1117.
accuracy? Acta Psychologica, 128, 110-118. Din, N., Haron, S., & Mohd, R. (2016). Can self-
Andreu-Andres, A., Garcia-Casas, M., & Rising, directed learning environment improve quality
B. (2009). Assessment of pre-service teacher of life ? Procedia - Social and Behavioral
participation and critical thinking in engineering Sciences, 222, 219-227. http://doi.org/10.1016/j.
pre-service teachers’ teamwork. WSEAS sbspro.2016.05.150
Transactions on Advances in Engineering Esa, A., Padil, S., Selamat, A., & Idris, M. T. M. (2015).
Education, 12(6), 464-477. SoSTeM model development for application of
Barak, M., & Rafaeli, S. (2004). Online question- soft skills to engineering students at Malaysian
posing and peer-assessment as means for Web- polytechnics. International Education Studies,
based knowledge sharing. International Journal 8(11), 204-210.
of Human-Computer Studies, 61, 84-103. Fernandez, C. & Yoshida, M. (2004). Lesson
Barber, M., & Mourshed, M. (2007). How the world’s study: A Japanese approach to improving
best-performing school systems came out on mathematics teaching and learning. Mahwah,
top (The McKinsey report). London, England: USA: Lawrence Erlbaum Associates, Publishers.
McKinsey & Company. Gokhale, A. A. (1995). Collaborative learning
Bicknell-Holmes, T., & Hoffman, P. S. (2000). Elicit, enhances critical thinking. Journal of Technology
engage, experience, explore: Discovery learning Education, 7(1), 22-30. Retrieved October 15,
in library instruction. Reference Services Review, 2015, from http://scholar.lib.vt.edu/ejournals/
28(4), 313-322. JTE/v7n1/gokhale.jte-v7n1.html.

1722 Pertanika J. Soc. Sci. & Hum.27 (3): 1709 - 1725 (2019)
 Building Hots by Applying a Multi-Method Approach

Grover, K. (2014). Teaching sustainable crop Lewis, C., Perry, R., & Murata, A. (2006). How
production through collaborative should research contribute to instructional
learning. NACTA Journal, 58(2),177-178. improvement ? The case of lesson study.
Educational Researcher, 35(3) 3-14.
Had, M., & Jopling, M. (2016). Problematizing
lesson study and its impacts : Studying a highly Limbrick, L., Buchanan, P., Goodwin, M., &
contextualised approach to professional learning, Schwarcz, H. (2010). Doing things differently:
Teacher and Teaching Education, 60, 203-214. The outcomes of teachers researching their own
practice in teaching writing. Canadian Journal
Jao, L., & Mcdougall, D. (2015). The collaborative
Of Education, 33(4), 897-924.
teacher inquiry project: A purposeful professional
development. Canadian Journal of Education, Linn, M. C. (2000). Designing the knowledge
38(1), 1-22. integration environment. International Journal
of Science Education, 22(8), 781-796.
Jonāne, L. (2015). Using analogies in teaching
physics: A study on Latvian teachers views and Mohan, M. J., Sunitha, C., Ganesh, A., & Jaya, A.
experience. Journal of Teacher Education for (2016). A study on ontology based abstractive
Sustainability, 17(2), 53-73. summarization. Procedia - Procedia Computer
Science, 87, 32-37. http://doi.org/10.1016/j.
Kabilan, M. K. (2000). Creative and critical thinking
procs.2016.05.122.
inlanguage classrooms. The Internet TESL
Journal, 6(6), 1-3. Retrieved Oktober 10, 2016, Melles, G. (2008). Teaching critical appraisal skills
from http://iteslj.org/Techniques/Kabilan- to postgraduate, English as a second language,
CriticalThinking.html engineering students. Australasian Journal of
Engineering Education, 14(2), 23-32.
Kapenieks, J. (2016). Educational action research to
achieve the essential competencies of the future. Miri, B., David, B., & Uri, Z. (2007). Purposely
Journal of Teacher Education for Sustainability, teaching for the promotion of higher-order
18(1), 95-110. thinking skills: A case of critical thinking.
Research in Science Education, 37, 353-369.
Kobzeva, N. (2015). Scrabble as a tool for engineering
http://doi.org/10.1007/s11165-006-9029-2
students’ critical thinking skills development.
Procedia - Social and Behavioral Sciences, Mosca, J., & Howard, L. (1997). Grounded learning:
182, 369-374. http://doi.org/10.1016/j. Breathing live into business education. Journal
sbspro.2015.04.791 of Education for Business, 73, 90-93.

Kuhn, D. (2005). Education for thinking. Cambridge, Morris, A. (2009). The stretched academy: The
England: Harvard University Press. learning experience of mature students from
under-represented groups. In The future of higher
Larsson, K. (2017). Understanding and teaching
education: Policy, pedagogy and the student
critical thinking - A new approach. International
experience (pp. 99-111). New York, USA:
Journal of Educational Research, 84(December
Continuum International Publishing Group.
2016), 32-42. https://doi.org/10.1016/j.
ijer.2017.05.004 Myers, J. (2013). Creating reflective practitioners with
preservice lesson study. International Journal of
Lewis, C. (2009). What is the nature of knowledge
Pedagogies and Learning, 8(1), 1-9.
development in lesson study? Educational Action
Research, 17(1), 95-110.

Pertanika J. Soc. Sci. & Hum. 27 (3): 1709 - 1725 (2019) 1723
 Anis Rahmawati, Aryanti Nurhidayati, and Ida Nugroho Saputro

Organisation for Economic Co-operation and Saito, E., & Atencio, M. (2013). A conceptual
Development. (2015). Education in discussion of lesson study from a micro-
Indonesia: Rising to the challenge. Paris, political perspective: Implications for teacher
France: OECD Publishing. http://dx.doi. development and pupil learning. Teaching
org/10.1787/9789264230750-en and Teacher Education, 31, 87-95. http://doi.
org/10.1016/j.tate.2013.01.001
Organisation for Economic Co-operation and
Development. (2016). Education at a glance Smith, B. L., & MacGregor, J. T. (1992). What
2016: OECD indicators. Paris, France: OECD is collaborative learning? In Collaborative
Publishing. http://dx.doi.org/10.1787/eag- learning: A sourcebook for higher education. New
2016-en York, USA: National Center on Postsecondary
Teaching, Learning, & Assessment, Syracuse
Papert, S. (2000). What’s the big idea?: Toward a
University.
pedagogy of idea power. IBM Systems Journal,
39(3/4), 720-729. Susar, F., Saygi, C., & Halil, I. (2015). Determine the
relationship between the disposition of critical
Phang, F. A., Ali, M. B., Bakar, M. N., Zanzali, N.
thinking and the perception about problem
A. A., Rahman, N. F. A., Mohtar, L. E., … &
solving skills. Procedia - Social and Behavioral
Puteh, M. (2011). Engineering elements profile
Sciences, 191, 657-661. http://doi.org/10.1016/j.
among first- and final- year engineering students
sbspro.2015.04.719
in Malaysia. 2011 IEEE Global Engineering
Education Conference (EDUCON) – Learning Thiede, K. W., & Anderson, M. C. M. (2003).
Environments and Ecosystems in Engineering Summarizing can improve metacomprehension
Education (pp. 70–73). Amman, Jordan: IEEE. a c c u r a c y. C o n t e m p o r a r y E d u c a t i o n a l
Psychology, 28, 129-160.
Pineteh, E. A. (2012). Using virtual interactions to
enhance the teaching of communication skills to Watt, J. X., Feldhaus, C. R., Sorge, B. H., Fore, G. A.,
information technology students. British Journal Andrew, D., & Marrs, K. A. (2014). The effects
of Educational Technology, 43(1), 85-96. http:// of implementing recitation activities on success
dx.doi.org/10.1111/j.1467-8535.2011.01193.x rates in a college calculus course. Journal of the
Scholarship of Teaching and Learning, 14(4),1-
Politsinsky, E., Demenkova, L., & Medvedev, O.
17. http://doi.org/10.14434/josotl.v14i4.12823
(2015). Ways of students training aimed at
analytical skills development while solving Yee, M. H., Yunos, J. M., Othman, W., Hassan, R., Tee,
learning tasks. Procedia - Social and Behavioral T. K., & Mohamad, M. M. (2015). Disparity of
Sciences, 206, 383-387. http://doi.org/10.1016/j. learning styles and higher order thinking skills
sbspro.2015.10.070 among technical students. Procedia - Social and
Behavioral Sciences, 20(4), 143-152. http://doi.
Saito, E. (2012). Strategies to promote lesson
org/10.1016/j.sbspro.2015.08.127
study in developing countries. Lesson study in
developing countries, 26(6), 565-576. http://doi. Yusop, M., Hadi, A., Hassan, R., Rashid, A., Razzaq,
org/10.1108/09513541211251398 A., & Zaid, M. (2015). Application of thinking
skills in career: A survey on technical and
vocational education training (tvet) qualification

1724 Pertanika J. Soc. Sci. & Hum.27 (3): 1709 - 1725 (2019)
 Building Hots by Applying a Multi-Method Approach

semi-professional job duties. Procedia - Social

and Behavioral Sciences, 211(September),
11 6 3 - 11 7 0 . h t t p : / / d o i . o r g / 1 0 . 1 0 1 6 / j .
sbspro.2015.11.155

Zohar, A., & Dori, Y. J. (2003). HOTS and low

achieving pre-service teachers: Are they mutually
exclusive? Journal of the Learning Sciences,
12(2), 145-183.

Zoller, U., Dori, Y., & Lubezky, A. (2002).

Algorithmic, LOCS and HOCS (chemistry) exam
questions: Performance and attitudes of college
pre-service teachers. International Journal of
Science Education, 24(2), 185-203.

Pertanika J. Soc. Sci. & Hum. 27 (3): 1709 - 1725 (2019) 1725