Journal of Information Systems Education, Vol.

30(4) Fall 2019

Teaching Critical Thinking, Problem Solving, and Design

Thinking: Preparing IS Students for the Future

Machdel Matthee
Marita Turpin
Department of Informatics
University of Pretoria
Private Bag X20, Hatfield 0028, South Africa
[email protected], [email protected]

ABSTRACT

Critical thinking and problem solving skills are included in the IS curriculum as foundational skills. IS education researchers
recognize the importance of these skills for future IS practitioners given the complexity of the technology based society and
economy of the future. However, there is limited work on how these skills are best taught in IS. This research reports on a course
focusing on the explicit development of critical thinking and problem solving skills of first-year IS students at the University of
Pretoria. The critical thinking part of the course focuses on the analysis, evaluation of, and response to arguments. Class discussions
and assessments are based on local, authentic arguments. In the problem solving skills component of the course, students are taught
to understand the nature of a problem and to classify it as belonging to one of three categories: puzzles, problems, and messes. For
each category, appropriate problem solving approaches are suggested and practiced. To illustrate the role of design and creativity
in problem solving, students have to create an artefact using the Maker Space of the university. They have to apply the five phases
of design thinking as suggested by the Stanford d.school design thinking approach. The course has been presented since 2016, and
feedback is collected from students annually. Based on a feedback questionnaire that the students complete at the end of each
course, we have reason to believe that they find the course valuable and consider those skills to be applicable to other courses as
well as elsewhere in their lives. They also point out the value it holds for their future as IS practitioners. As part of our ongoing
research, we are investigating ways to develop a critical disposition amongst students, an important component of critical thinking.

Keywords: Critical thinking, Problem solving, Design thinking, Computing skills, Curriculum design & development

1. INTRODUCTION (2014, p. 43), “a discipline-specific silo approach to curriculum

design falls short of preparing the type of graduate needed in
In recent years, a number of Information Systems (IS) today’s industry and society.” What employers need are
researchers pointed out the need to rethink the IS curriculum graduates with critical thinking skills, complex problem solving
(Zhao and Zhao, 2010; Pratt, Keys, and Wirkus, 2014; Topi, skills, good communication skills, and the ability to design
2019). Several reasons are given for this of which the dawning responsible systems and consider ethical implications of
Fourth Industrial revolution and its effect on society and the systems (Pratt, Keys, and Wirkus, 2014). These skills are
future of jobs is seemingly the most critical one. The indeed included in the IS2010 undergraduate IS curriculum
combination of powerful technologies and developments such under the Foundational knowledge and skills category (Topi et
as artificial intelligence, machine learning, biotechnology, 3D al., 2010) and in the 2016 graduate IS curriculum as Individual
printers, and nanotechnology will lead to an unprecedented Foundational Competencies (Topi et al., 2017), but Pratt, Keys,
revolution in nearly every aspect of human life (WEF, 2018). and Wirkus (2014) suggest a more explicit integration of these
World leaders, economists, and educationalists are scrambling skills into the IS curriculum.
to get to grips with the implication of this revolution on the At the University of Pretoria, the Department of Informatics
future of jobs. Although IS is not primarily focused on the presents the only ABET-accredited IS undergraduate degree
development of the technical component of the systems of this program in Africa. Every year, an average of 100, mostly South
new era, Topi (2019) considers IS professionals to be ideally African, students enroll for the undergraduate degree in
situated in bringing “the capabilities [of the technologies] Business Information Systems (BIS). These students live in a
together in a way that serves individual, organizational, and multi-cultural, socio-economically unequal, and complex
societal goals.” While according to Topi, the core IS society. The quality of their schooling and economic reality
competencies will remain relevant, Pratt, Keys, and Wirkus differ substantially from one student to the next. This means
(2014) argue for the alignment of the IS curriculum with liberal that students enter the BIS degree program with vastly different
arts education outcomes. According to Pratt, Keys, and Wirkus levels of knowledge, skills, and resources. The fragile South

242
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

African democracy and economy can face the realities of the domain-independent, distinct, and definable. CT teaching
fourth industrial revolution only if its youth are equipped to take therefore involves the transfer of these skills to enable students
on the role of responsible, reflective citizens. Similar to Higher to solve problems encountered in everyday life (Tiruneh,
Education Institutions worldwide but even more so, South Verburgh, and Elen, 2014).
African universities realize the role it plays in educating such Tiruneh, Verburgh, and Elen (2014) studied the
citizens. For example, included in the graduate attributes of the effectiveness of CT teaching interventions by considering
University of Pretoria is the following: literature on the topic published from 1995 to 2012. They
considered interventions where CT skills were taught separately
They conceptualize issues and synthesize knowledge from, as well as included implicitly or explicitly within, the
creatively to provide solutions for current and future- subject matter instruction. They label these two different
orientated challenges. They conduct context-focused, approaches as direct teaching versus implicit teaching. From the
solution-orientated inquiries using critical, creative and literature, compared to implicit instruction, it appears that direct
logical thinking. They use a systems approach to instruction of critical thinking skills results consistently in
manage change in complex situations, using global better critical thinking skills (especially in the case of first year
perspectives to improve understanding of causes and students) (Tiruneh, Verburgh, and Elen, 2014).
solutions related to local problems. Abrami et al. (2015) give a useful categorization of
effective instructional approaches used in both direct and
Indeed, the World Economic Forum (WEF) emphasizes the implicit approaches:
importance of problem solving and critical thinking as essential
21st century skills (WEF, 2018). It is acknowledged that critical Category 1: Individual study. This refers to students’
thinking and problem solving are implicitly taught in the IS individual work by engaging, reading, and solving
curriculum, but in support of Pratt, Keys, and Wirkus (2014), problems on their own.
and given the South African context, the Department of
Informatics decided to introduce a course for first-year IS Category 2: Dialogue. This refers to learning through
students focusing explicitly on the development of critical discussion. There are numerous ways in which
thinking and problem solving skills. discussion can be facilitated. Some examples include a
The objective of this paper is therefore to give an overview formal debate in class, whole-class discussion led by
of this course as well as students’ reaction to it over the last the teacher, group discussions, and student dyads.
three years. The next sections provide background on how
critical thinking and problem solving skills are being taught Category 3: Authentic or anchored instruction. The
after which we focus on IS education researchers’ approaches focus here is on presenting students with real problems
to developing these skills. This is followed by an overview of or problems to which they can relate. Examples of
and reflection on the course. relevant approaches include case studies, simulation,
role-play, applied problem solving, and games.
2. THE TEACHING OF CRITICAL THINKING
Category 4: Mentoring. This emphasizes one-on-one
Tiruneh, Verburgh, and Elen (2014) define critical thinking as modeling of critical thinking disposition and skills by
the ability to analyze and evaluate arguments according to their teachers or peers. From the meta-analysis on critical
soundness and credibility, respond to arguments, and reach thinking strategies, Abrami et al. (2015) found that the
conclusions through deduction from given information. most effective dialogue strategies are whole class
Halpern (1998, p. 450) gives a broader definition by discussion, teacher led group discussions, and teachers
considering critical thinking as the use “of cognitive skills or posing questions. Also, exposing students to authentic
strategies that increase the probability of a desirable outcome.” problems seems to be effective, particularly if applied
Halpern considers critical thinking skills as crucial in solving problem solving or role-playing is used. Lastly, their
complex and ill-defined problems. A taxonomy of critical research indicates that the combination of dialogue and
thinking skills as suggested by Halpern includes the following authentic instruction is particularly effective, and even
skills: 1) verbal reasoning skills, 2) argument analysis skills, 3) more so when combined with mentorship.
skills in thinking as hypothesis testing, 4) dealing with
likelihood and uncertainties, and 5) decision making and 3. THE TEACHING OF PROBLEM SOLVING
problem solving skills. Problem solving skills in Halpern’s
writing refers to those skills used to judge the quality of a The design and development of an information system is
solution or decision, as well as the thought processes needed to essentially a problem solving exercise (Turpin, Matthee, and
reach the solution. Kruger, 2015). However, what does problem solving entail, and
Researchers are in agreement that apart from cognitive how is it taught?
skills, critical thinking (CT) requires a disposition towards According to Butterworth and Thwaites (2013), problem
being critical (e.g., begin open-minded, curious, truth-seeking) solving refers to the thinking and planning required to achieve
(Halpern, 1998; Tiruneh, Verburgh, and Elen, 2014; Abrami et a particular outcome. The less familiar one is with the problem
al., 2015). context, the more thinking and planning are required. Turban,
Although some researchers consider critical thinking skills Aronson, and Liang (2004) define problem solving in a decision
to be domain-specific only (Abrami et al., 2015), the general support system (DSS) context and state that problem solving
assumption is that CT skills are cognitive skills which are entails a process that starts with an initial state and from there

243
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

moves to search through a problem space in order to reach a (e.g., Kimmel, Kimmel, and Deek (2003)), and computer
desired goal. Turban, Aronson, and Liang (2004) regard programming (e.g., de Raadt, Watson, and Toleman (2006)).
problem solving and decision-making to be similar. Gammack, These fields have the analytical as well as design aspects of
Hobbs, and Pigott (2011) believe that, in order to solve a problem solving in common with IS.
problem, one needs to understand the nature of the problem As with the teaching of critical thinking, studies on teaching
first. This is in accordance with Pólya’s (1957) four steps of problem solving also concern themselves with the question of
solving a problem, namely: understand the problem, devise a whether the material should be taught implicitly or explicitly.
plan or strategy, implement the plan, and reflect on the With an implicit approach, the assumption is that through the
outcome. Gammack, Hobbs, and Pigott (2011) and Pidd (2003) normal teaching of domain subjects, problem solving skills will
believe that Ackoff’s (1978) work on classifying problems as emerge. However, “implicit instruction on solving a problem
puzzles, problems, or messes is an essential departure point to has been shown to result in poor learning outcomes” (de Raadt,
solving problems. Ackoff (1978) classified problems in terms Watson, and Toleman, 2006). Kimmel, Kimmel, and Deek
of their complexity. According to Ackoff, a puzzle is a situation (2003) note that while problem solving skills are acknowledged
where there is no ambiguity. There are clear rules to follow in by engineering educators to be important, “unless it is included
order to arrive at a solution, there is only one solution, and, as in the course objectives and specifically identified as a skill
with a jigsaw puzzle, it is possible to know that one has arrived students are expected to master in the course, it is, at best, given
at the correct answer (Pidd, 2003). Many problems in the field perfunctory attention in the classroom” (2003, p. 810). The
of mathematics can be classified as puzzles. studies go further to say that students need to be taught specific
Problems are more complicated than puzzles since there are strategies that link to specific classes of problems (English and
multiple possible solutions depending on the circumstances, the Sriraman, 2010). There appears to be a consensus that the
constraints, and the assumptions made. The field of operations explicit teaching of problem solving is better, not just by
research concerns itself with methods to address problems in a recognizing problem solving as an explicit skill to be taught, but
real-world context, such as vehicle scheduling and the placing also by giving guidance in the form of specific problem solving
of a new factory. Messes are to the opposite extreme of puzzles methods and the problem situations to which these apply.
in that they are totally ill-structured. In messy situations, it is
not even clear what the problem is that needs to be solved, let 4. THE IS CURRICULUM, CRITICAL THINKING,
alone what process to follow. Messes are also known as wicked AND PROBLEM SOLVING
problems and are characterized by uncertainty, complexity, as
well as multiple views on the situation by the various The Foundational Knowledge and Skills category of the IS2010
stakeholders (Rosenhead and Mingers, 2001). The field of curriculum includes the sub-theme “analytical and critical
study dedicated to resolving messes is known as “soft” thinking, including creativity and ethical analysis.” Problem
operations research since it concerns itself with soft, people solving is considered part of this category (Topi et al., 2010). It
issues more so than hard, technical issues. It should be clear that can be seen that some authors assume problem solving to be
problem solving in the IS field has to deal with the part of critical thinking, and others, critical thinking to be part
characteristics of puzzles (logical and analytical thinking), of problem solving. Admittedly, to solve a problem, one needs
problems (socio-technical, open-ended real world systems), as comprehension and reasoning capabilities (Kimmel, Kimmel,
well as messes (multiple role-players that may have and Deek, 2003) in order to understand the problem and reason
incompatible perspectives). Indeed, many problem situations through strategies. Comprehension and reasoning skills are part
that appear like problems also have messy aspects (Pidd, 2003). of the critical thinking skills set. To think critically, one needs
This is why Gammack, Hobbs, and Pigott (2011) propose that an analytical mind set which in turn forms part of the ability to
in the field of IS, problem solving skills need to include dealing solve problems. According to Butterworth and Thwaites
with puzzles, problems, as well as messes. (2013), critical thinking is often associated with verbal texts,
An aspect of problem solving that is not explicitly while problem-solving is usually associated with contexts that
addressed in the body of work above is that of creativity. Evans involve numbers or other mathematical language. However, the
(1992) argues that complex, real-world problems call for a underlying thinking skills are “quite similar and certainly
larger solution space than what can be arrived at by analytical complementary” (Butterworth and Thwaites, 2013, p. 13). For
reasoning alone. He argues for the enlargement of the solution the purpose of this paper (and the course on which we report),
space by means of creativity (Evans, 1992, in Turpin, Matthee, we regard critical thinking and problem solving as
and Kruger, 2015). Ackoff (1978) also holds that creativity is a interdependent thinking skills.
key ingredient to problem solving. Gammack, Hobbs, and Since the designing and building of information systems are
Pigott (2011) promote creativity as a means to provide different always done to solve a problem, the assumption is that
and new views on a problem situation. They believe it is a outcomes of an IS program will be analytical, critical, and
valuable skill in systems analysis and design. creative thinking skills, whether taught in a formal way or not.
Up to here, we have discussed important work that has been Indeed, Agerfalk, Sjostrom, and Tuunanen (2017) go as far as
done on problem solving, taking into account the nature of using the California Critical Thinking test to evaluate the
problems and means to address different aspects of problems. effectiveness of a new IS curriculum introduced after a merger
Now, we will consider some previous work on the teaching of between two universities. They found that the critical thinking
problem solving in the classroom. While little has been skills of students improved because of the IS curriculum.
published on the teaching of problem solving in IS (see section Considering the teaching of critical thinking and problem
4), the topic has been more widely studied in the teaching of solving in the IS curriculum, almost no evidence could be found
mathematics (e.g., English and Sriraman (2010)), engineering of direct instruction of these skills. The closest to direct

244
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

teaching is where students are given a problem to solve and the assessment and optional extra tutor classes. Summative
solution is assessed according to rubrics based on the assessment takes place through a written exam at the end of the
components of higher order thinking (e.g., analysis, evaluation, semester. Through the study guide and regular communication
synthesis, and creativity). For example, Mukherjee (2004) via Blackboard technology, their educational pathway is
promoted higher-order thinking skills in the teaching of communicated. The course consists of two parts: Critical
decision support systems by asking students to analyse, Thinking and Problem Solving. Students attend two 50-minute
evaluate, and respond to case studies on decision making. lectures per week. Critical thinking as taught in this course
Similarly, Pratt, Keys, and Wirkus (2014) show how critical entails the identification, analysis, and evaluation of arguments
thinking can be developed by using a rubric based on critical and responding with further argument. In the problem solving
thinking elements to assess all presentations across the IS part of the course, students are exposed to different types of
courses. Other researchers focus on specific teaching strategies problems and problem solving techniques for structured and
with the assumption that it will result in the development of unstructured problems. Design thinking as a creative problem
higher-order thinking skills. Jones (2015) implements a BIS solving approach is also introduced to the students. Each sub-
honors course based on Neumeier’s metacognitive framework section is discussed in more detail below.
using high-impact teaching practices (collaborative learning,
learning communities, and writing intensive exercises) and 5.1 Critical Thinking
flipped classrooms. Saundage et al. (2016) use interactive Seven weeks of the 14-week semester are used for this part of
visual narratives to teach business analytics to IS students and the syllabus. The study material is based on selected parts of the
measure the effectiveness according to Bloom’s higher-order book by Butterworth and Thwaites (2013). Despite the diverse
thinking skills. In fact, implementing experiential learning South African student group, we find the examples in the
(Riordan, Hine, and Smith, 2017), problem-based learning textbook sufficiently applicable to illustrate the principles.
(Taipalus, Seppänen, and Pirhonen, 2018), game-based Table 1 presents the learning outcomes and objectives of the CT
learning, and flipped classrooms (Caceffo, Gama, and Azevedo, part of the course.
2018) in IS education are generally considered conducive for
cultivating critical thinking and problem solving skills but are Learning
seldom implemented with that explicit objective. Outcome Learning Objectives
The study by Steyn, Matthee, and Turpin (2013), although Understand  Understand what is meant by
focusing on teaching creativity skills, was the only one to be what critical thinking skills;
found using a direct way of teaching these skills: creativity thinking  Understand what is meant by
techniques (including de Bono’s Six Hat technique, Do-IT, and entails critical thinking; and
brainstorming) were taught in a first-year system analysis and  Understand why and when critical
design course. Students had to use these methods in solving an thinking is necessary.
authentic problem after which they had to produce a video to Analyse an  Understand what the different
illustrate the way in which they used a creativity technique to argument types of claims are;
reach a solution. Turpin, Matthee, and Kruger (2015) show that  Judge a claim;
there is a lack of emphasis on fostering creative thinking skills  Understand what an argument is;
in South African IS degree programs. Their findings show the  Know how to analyse an
importance of thorough domain knowledge and presenting argument;
authentic problems to students to foster creative thinking skills.  Identify a conclusion; and
It is clear from the above discussion that IS educators are  Understand what diffuse
aware of the importance of the development of the foundational conclusions are.
skills. However, the above studies are highly innovative, but Critically  Understand what a flawed
mostly ad hoc interventions. The study by Pratt, Keys, and evaluate an argument is;
Wirkus (2014) is one exception. Pratt and colleagues went argument  Know the different types of
through a re-curriculation exercise to align course-level fallacies;
outcomes with university-level liberal education learning  Identify the flaws in arguments;
outcomes that include critical thinking and problem solving and
skills.  Understand how graphs and
The Department of Informatics at the University of Pretoria statistics are used in arguments.
went through a similar process, but it resulted in the Respond to an  Develop a new line of argument
identification of an extra course. In an attempt to address the argument by with its own conclusion;
unequal level of these crucial cognitive skills among the first- developing  Understand the use of counter-
year IS students, and given the importance of these skills for the further examples; and
future, the department opted to develop a course through which argument  Anticipate counter arguments.
these skills and aptitude are taught directly and explicitly. Table 1. Learning Outcomes and Objectives of the
Critical Thinking Section
5. THE COURSE
5.1.1 Instruction strategy. Using the terminology as suggested
The semester-long course has been presented annually since by Abrami et al. (2015), we make use extensively of dialogue:
2016. Students are supported in their learning by a structured lecturers pose questions, lecturer-led whole-class discussions
teaching and assessment plan that includes regular formative take place, and group discussions are encouraged. In addition,

245
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

lecturers model critical thinking skills and aptitudes. The focus space explain what services and courses they provide. The
is also on authentic public discourses and problems. It makes a Maker Space is a fun, colourful, and creative work space. The
student relate to the problem at hand and, therefore, feel Maker Space houses technology such as 3D printers and
included. The importance of authentic content has been a programmable microchip kits, and it provides free training and
finding of the authors’ own previous research and supported by assistance with these technologies. Table 2 gives an overview
Abrami et al. (2015). One of the objectives of the course is to of the learning outcomes and objectives of this part of the
develop informed, engaged, and accountable citizens beyond course.
the classroom context. The researchers, therefore, put a lot of
effort into finding South African, context-specific examples for Learning
class exercises and assignments. Outcome Learning Objectives
Identify basic  Understand how problems are
5.1.2 Assessment. The assessments include identification of problem types defined and characterized;
arguments, analysis of arguments, identifying flaws in and problem  Differentiate between problems,
arguments, evaluating the credibility of an argument, and solving symptoms and problem
building further argument. As mentioned above, the focus is on approaches situations;
arguments from authentic, relevant local content. A few  Understand the difference
examples of arguments used in assessments are given below (all between puzzles, problems and
adapted from articles in the popular press): messes; and
 Be able to identify the
• By legalising rhino horn, rhino poaching will be curbed. appropriate methods to deal with
• The towing of icebergs from Antartica to Cape Town puzzles, problems and messes.
will solve the water crisis in Cape Town (in 2018, the Apply problem  Understand the characteristics of
city faced the possibility of running out of water due to structuring a messy problem;
a long term drought). methods to  Understand problem structuring
• The introduction of a sugar tax will not solve the messy problems methods;
obesity problem of South Africans.  Understand Checkland’s Soft
• Race relations in South Africa are not as bad as Systems Methodology (SSM);
everyone says (during the #FeesMustFall – the demand and
for free education – crisis at South African universities  Apply CATWOE mnemonic to
in 2016). develop root definitions as part
of the SSM process.
Arguments closer to the study field of IS: Apply design  Define design thinking;
thinking to  Contrast design thinking with
• Artificial Intelligence can be our friend, despite the design an problem-based thinking;
fears about its adverse effects on society. artefact  Know the generic steps of a
• Business leaders, educators, and governments all need design thinking process;
to be proactive in up-skilling and retraining people to  Identify the pitfalls of design
prepare for the Fourth Industrial Revolution. thinking; and
 Apply design thinking by
In addition, as an individual assignment in 2016, students had following the Stanford
to argue how a self-driving car should be programmed to make University’s design thinking
a moral decision if faced with difficult choices. process to design and
manufacture an artefact.
5.2 Problem Solving Table 2. Learning Outcomes and Objectives of the
The other part of the course focuses on problem solving skills. Problem Solving Part
To make sense of the vast set of such skills, the categorization
of problems as suggested by Gammack, Hobbs, and Pigott Puzzles are taught by referring to problem solving
(2011) is used. Problems are labeled according to their strategies as proposed by Posamentier and Krulik (2015). These
complexity: puzzles refer to well defined problems with strategies include: pattern recognition, working backwards,
specific solutions, while problems are partly structured with adopting a different point of view, considering extreme cases,
multiple possible solutions. The most complex type of problems solving a simpler version, organizing the data, drawing/visual
are referred to as messes. Messes are unstructured and often not presentation, considering all possibilities, and informed
solvable. The best one can do is hope to get a better guessing. Each of these strategies is illustrated with examples.
understanding of the problem and structure, and solve parts of Relating to problems (semi-structured problems), students are
it. In the course, approaches to solve each of these types of presented with decision analysis type problems where they are
problems are introduced to students. In addition, design exposed to handling constraints, assumptions, and trade-offs.
thinking (Hasso Plattner Institute of Design, n.d.) is introduced As an example, students have to consider their employment
as a creative problem solving technique. One lecture is used to options after graduation and identify appropriate decision
introduce students to a maker who explains the Maker criteria for comparing these options. To deal with messes,
Movement and what it entails. Students are then accompanied Checkland’s Soft System Methodology (SSM) (Checkland,
to the university’s Maker Space where the assistants of the 2000) is presented as a suitable approach.

246
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

5.2.1 Instruction strategy. Similar to the critical thinking part printed. The glass top is a recycled coffee jar that screws into
of the course, active class participation by students is the base.
encouraged. Also, lecturers model problem solving behavior. In The students can only complete the assignment by sourcing
addition, more emphasis is placed on group work in this part of additional knowledge, and they are free to use any means to do
the course. For example, the Jigsaw collaborative learning so, including the internet and asking friends or family. In this
method (Doymus, 2008) is used when applying SSM. Messes way, students are prepared for the world of work where one is
typical to the South African context like youth unemployment not given a recipe for completing a task. Further, the open-
are used in a class group assignment. The class is divided into ended nature of this project serves as an enrichment opportunity
stakeholder groups for the role-play (in the case of youth for exceptional students.
unemployment: the government, youth, private sector, and
Department of Education). Each of these groups has to draw a
rich picture of the problem situation and develop a root
definition from their perspective by using the CATWOE
mnemonic. Once this is done, students have to regroup into new
groups consisting of at least one representative of each
stakeholder group. Using the rich pictures and the root
definitions developed during the previous round of the Jigsaw
by the stakeholder groups, the new groups have to decide on a
possible solution, create a conceptual diagram, and create a plan
of action. This exercise clearly illustrates the contention that a
mess cannot be solved, only better understood and managed.
The instruction of design thinking relies heavily on project-
based learning, as discussed in the next section.

5.2.2 Assessment. Different group assignments are used during

the formative assessment. To establish an understanding of the Figure 1. Pencil Holder
different categories of problems, in a group, students have to
identify the category to which a problem belongs (instead of
solving it). When practicing strategies to solve puzzles, students
are provided with a number of puzzles which they have to first
solve and, second, identify the strategy that was used (e.g.,
pattern recognition). The approach to messes is illustrated by
having a group assignment where students have to structure and
understand a mess by using SSM (discussed above).
Lastly, project-based learning is used to illustrate design
thinking. Student groups have to use design thinking (as a
creative problem solving method) to design and build an
artefact using the Maker Space of the university. Students use
the Stanford d.school’s approach to go through the design
process and have to present evidence of this through a blog. The
design approach consists of five phases: empathize, define,
ideate, prototype, and test.
As preparation for the Maker Space group assignment,
students are given the opportunity to practice the five design
thinking steps by means of a class exercise. During this Figure 2. Candy Dispenser
exercise, by going through the design phases, students have to
use recycled material to create an artefact for a classmate to 6. FEEDBACK
address a problem he/she experiences with accommodation.
Over the past few years, students have created prototypes for Since the start of the course, we asked for feedback on the
study areas, multifunctional furniture, mechanisms to block different parts of the course. The feedback is mainly of a
noise, and apps to assist with time management, to name a few. qualitative nature. A total of 292 students completed the
The artefact that has to be created using the Maker Space is questionnaire over the three years from 2016-2018. Ethics
limited by type, cost, and size. Examples of artefacts that had to approval for collecting and disseminating student data was
be created over the past few years include a kitchen utensil, a obtained from the university. Informed consent was received
corporate gift, a container, and an educational toy. Students from the students who participated.
have emerged with highly innovative artefacts. Two corporate
gift prototypes resulting from this project are given below. 6.1 Feedback on the Critical Thinking Part
Figure 1 shows a pencil holder that can be assembled and Only one open-ended question was asked about this part of the
disassembled whereas Figure 2 is a mini candy dispenser. As course: Please provide feedback about the critical thinking part
can be seen in Figure 2, the base of the candy dispenser is 3D of the course.
Fifty nine students mentioned that this part of the course
helped them to start thinking critically. Thirty three students

247
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

stated that they found it enjoyable. More importantly, students and SSM exercise part of the course. The scale was: 1) Strongly
mentioned that enhanced critical thinking skills helped them to: disagree, 2) Somewhat disagree, 3) Neither agree nor disagree,
4) Somewhat agree, and 5) Strongly agree. Each of the
• Interpret and write narratives in use cases: “The critical questions and the responses are given below.
thinking part of the course was interesting and valuable
to other modules such as INF171 [system analysis and Question 1: I found the theory lecture on approaches to
design]. The extensive class assignments, activities and address different kinds of problems valuable. Over the
exercises also helped because they allowed me to fully three years, 35% strongly agreed whereas 58% agreed
understand the concepts.” “Critical thinking helps a somewhat. Figure 3 shows the comparison between the
lot, now that we are doing use cases in INF 171 it helps three years. Although it appears that more students
me think of what is being said, how it’s being said and agreed in 2018 that the theory lectures are valuable, the
what I can take from this passage or rather case study difference between the three years is not statistically
I have been given.” significant.
• Interpret exam and test questions better: “[Critical
thinking] assisted with tests and assignments. Being Question 2: I found the class activities valuable where
able to break down a question or statement in order to we had to apply strategies for problem solving, such as
understand what exactly is required from you as a pattern recognition and visual representation. This
student. It was very helpful.” question was only asked in 2018, and of the 132
• Prepare them for the future: “It was really challenging students that responded, 37% strongly agreed with the
but at the end of the day it equipped me with skills I can statement and 52% somewhat agreed. It replaced a
use in the near future; [Critical thinking] is an differently formulated question covering the same topic
invaluable skill because in the line of informatics that was asked in 2016 and 2017.
graduates, problems and situations are not always as
they may appear.” Question 3: I found the theory lecture on messy
problems and SSM valuable. Over the three years, 32%
6.2. Feedback on the Problem Solving Part strongly agreed whereas 49% agreed somewhat. The
The questions on this part of the course include Likert scale comparison over the three years showed no significant
questions as well as open-ended questions. difference.

6.2.1 Feedback on problem categories and messy problems.

Four Likert scale questions were asked on the problem strategy

Figure 3. Comparing Frequencies of Responses to Question 1

248
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

Question 4: I found the practical SSM class assignment problem solving and messy problems part of the course,” some
valuable. Over the three years, 36% strongly agreed students mentioned the complex nature of the SSM and the
whereas 48% agreed somewhat. Again no significant difficulty they have understanding it. Other interesting aspects
difference between the three groups was noted. identified from the responses include the transferability of skills
– a number of students mentioned that these problem solving
Aspect Quotes skills are applicable to other subjects, everyday life, and their
Transferability “The problem solving and messy personal life as well. In addition, some students mentioned the
of skills problems part was valuable and value of these skills in mastering the material in their other BIS
necessary because I won't only use what courses. Students also find these skills important for the future.
I have learned in class because it is One student considered the formal way of teaching problem
knowledge I can use to solve any solving as beneficial to the students with poor schooling (which
problem I might face in my life even was one of the reasons why we used this approach). Table 3
after graduating”; summarizes these aspects. Note that the student feedback in
“I found that helpful as a lot [of] the italics are in the students’ own words, and no spelling or
problems in our day to day lives as grammar corrections have been made.
students aren't always ones we can
solve systematically. Personally, I have 6.2.2 Feedback on design thinking. Regarding the design
made use of this method in my personal thinking project, three Likert scale questions were asked. The
issues after learning and fully scale used was: 1) Not at all useful, 2) Slightly useful, 3)
understanding ”t”; Moderately useful, 4) Very useful, and 5) Extremely useful.
“With regards to problem solving and The following responses were noted:
messy problems helps us apply the
problem solving methods inside and Question 5: How useful did you find the theory lecture
outside of our course”; and slides on design thinking? Over the three years,
“I think it was great learning 24% found it extremely useful whereas 54% found it
experience to get to know how to very useful. Again, no significant difference between
classify problems, as this will allow me the three years was noted.
to know which problem is feasible for
me to try and solve.” Question 6: How useful did you find the class
Valuable to IS “I have learnt about types of problems assignment where you had to design and build a
studies and how to identify these problems, and prototype for student accommodation? Over the three
how to identify practical and viable years, 33% found it extremely useful whereas 33%
solutions for these problems. These found it very useful. A significant difference was found
skills are invaluable in the line of work between the three groups. On closer inspection, the
which the BCom informatics degree difference was between the 2016 and 2018 groups, but
prepares students for”; with the adjusted results, the difference is just above the
“The problem solving and messy significance cut off.
problems part of the course is very
interesting and it provides a building Question 7: How useful did you find the Maker Space
block for the Informatics subjects which group assignment? Over the three years, 40% found it
is valuable.” extremely useful whereas 31% found it very useful.
Important “It was really interesting and would Again no significant difference between the three years
skills for the help people (including myself) solve was noted.
future problem in this unpredictable world of
today.” Open-ended questions were asked on each of the phases of
Addressing “Since problem solving skills are the Stanford d.school design thinking methodology regarding
unequal expected of a student to have developed their use of the Maker Space. It is evident that students used the
schooling through primary and secondary Maker Space extensively in the prototyping phase while
schooling, not all were given the same friends, parents, and other people in their social networks
circumstances to be on equal plains; played an important role in the ideate phase.
therefore, the abstract approach Students also had to report on the influence of the Maker
provided by the module and exposure to Space environment on their assignment. Apart from the services
different kinds of problems allowed provided at the Maker Space, they found it conducive to
some, if not most, to further develop creativity and describe the space as inspirational, exciting, and
their level of problem solving skills” enjoyable. It also provided them a tangible way to test ideas and
Table 3. Feedback on the Problem Solving Part of the do prototyping. As one student put it: “it provides a platform
Course for us to formulate our thoughts.”
When asked to provide feedback about the design thinking
When adding up the percentages, it is clear that the part of the course, students picked up that the objective of this
feedback from the students is predominantly positive. However part of the course was to teach them creative problem solving:
in the open ended question, “Please provide feedback about the “It has made me feel like a problem solver, innovator, and felt

249
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

like preparation for the working environment; it helped me really improved because of this course? This year we will
come up with creative ways to solve problems.” They also attempt to answer this question by administering a pre- and
mentioned the transferability of the skills: “I did not know about post-test to the students, hoping that this research will shed
the Design Thinking topic until it was taught to me in the INF some light on the effectiveness of the course.
113 class. I do see myself applying Design Thinking in my As researchers, we are interested in other related topics. For
future courses (e.g., programming courses) as well as my future example, will students be in a better position to judge the
endeavours.” Another student stated: credibility of online news (identify fake news) as a result of this
course? Also, what is the relationship between meta-cognition
This provided different ways of looking at problems and critical thinking skills, and what is the effect of a growth
and ways of finding solutions. The Maker Space group mind-set on critical thinking skills? These questions provide
assignment was a worthwhile experience because I was interesting avenues for future research.
introduced to 3D printing and what actually goes into
the design requirements. All in all, the entire 8. CONCLUSION
assignment was a thought provoking experience. A lot
of time and effort goes into satisfying a customer/client. This research describes a course presented to first-year IS
students where critical thinking and problem solving skills are
In a previous paper (Matthee, Turpin, and Kriel, 2017), we taught in a formal way. Preliminary qualitative results give us
argue that exposure to the Maker Space and 3D printers gives reason to believe that both planned and unexpected outcomes
students an idea of what it means to be part of the Maker of the course are contributing towards the development of some
Movement. Using the three components of the Maker of the IS professionals foundational skills – according to Topi
Movement proposed by Halverson and Sheridan (2014), we et al. (2010, p. 369):
provide evidence that some students understand that the action
of making an artefact is at the core of the movement, that a IS professionals must have strong analytical and critical
creative space (communities of practice) enables the action, and thinking skills to thrive in a competitive global
that the community consists of makers (individuals that take on environment. Students must, therefore, be problem
this identity). A number of students were eager to create their solvers and critical thinkers and use systems concepts
own artefacts using the Maker Space. for understanding and framing problems.

7. REFLECTION In the survey that was run over the three years of teaching
the course, a majority of students found the problem solving
The student feedback (Table 3 as well as section 6.2.2) clearly activities valuable for each activity on which feedback was
indicates the value that the students perceived to have gained requested. The qualitative feedback on the critical thinking
from the explicit teaching of problem solving skills and specific component of the course shows evidence of students who were
problem solving strategies. This finding is in line with the able to apply their newly gained argument analysis skills in
literature where the explicit teaching of problem solving other subjects as well as when completing tests and
alongside with specific problem solving strategies was assignments. The feedback on the problem solving component
advocated (Kimmel, Kimmel, and Deek, 2003; de Raadt, also showed students who believed that their newly gained
Watson, and Toleman, 2006; English and Sriraman, 2010). problem solving skills equipped them to better deal with
A lot of thought went into the improvement of the course problems presented elsewhere in their degree program as well
over the years. For example, we introduced more interim as in everyday life. In the context of the socio-economically
deliverables in the design thinking Maker Space project to unequal background of the South African students, it was
prevent last minute delays with long 3D print job queues. A encouraging to hear from a student who believed that the
section on misleading graphs and statistics was added to the problem solving skills assisted to address an unequal schooling
critical thinking part of the course based on the book by Levitin background. In addition, feedback from the students points
(2016). This year, we added, as another problem solving toward their perceived value of these skills for their future as IS
approach, a lecture on computational thinking. Although all practitioners.
students in this course also take a programming course, we In the context of the lack of studies reporting on the explicit
believe that the placing of computational thinking among other teaching of the foundational skills of critical thinking and
problem solving skills will improve the transferability of this problem solving to IS students, this study makes a contribution.
skill to not only their programming course but also to other The study is limited by the self-reported nature of the feedback
courses as well. received from students. Future research includes aspects such
An important aspect of critical thinking that we do not as investigating the relationship between developing critical
address explicitly is the critical disposition. The only way we thinking and meta-cognition and better assessment of the
teach it is by modelling this behavior. More thought is needed effectiveness of our teaching with pre- and post-assessments.
on ways to address this shortcoming.
Up to now, the only evidence we have of the effectiveness 9. REFERENCES
of the course is the mainly qualitative, self-reported feedback
we get from the questionnaires every year. The average marks Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington,
of the groups are relatively high, and the pass rate is good (the D. I., Wade, C. A., & Persson, T. (2015). Strategies for
pass rate was 89% in 2018). But what additional confirmation Teaching Students to Think Critically: A Meta-Analysis.
do we have that the students’ critical and problem solving skills Review of Educational Research, 85(2), 275–314.

250
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

Ackoff, R. L. (1978). The Art of Problem Solving. New York: Mukherjee, A. (2004). Promoting Higher Order Thinking in
John Wiley & Sons. MIS/CIS Students using Class Exercises. Journal of
Ågerfalk, P., Sjostrom, J., & Tuunanen, T. (2017). Evaluation Information Systems Education, 15(2), 171–179.
of IS Curriculum Design: A Pilot Study using the California Pidd, M. (2003). Tools for Thinking: Modelling in Management
Critical Thinking Skills Test. Presented at ICIS 2017, Seoul, Science. Retrieved from https://www.wiley.com/en-
South Korea. us/Tools+for+Thinking%3A+Modelling+in+Management+
Butterworth, J. & Thwaites, G. (2013). Thinking Skills: Critical Science%2C+3rd+Edition-p-9780470721421.
Thinking and Problem Solving (2 ed.). Cambridge: Pólya, G. (1957). How to Solve It. Princeton: Princeton
Cambridge University Press. University Press.
Caceffo, R., Gama, G., & Azevedo, R. (2018). Exploring Posamentier, A. S. & Krulik, S. (2015). Problem-Solving
Active Learning Approaches to Computer Science Classes. Strategies in Mathematics: From Common Approaches to
Proceedings of the 49th ACM Technical Symposium on Exemplary Strategies. New Jersey: Wspc.
Computer Science Education, 922–927. Pratt, J. A., Keys, A., & Wirkus, T. (2014). Preparing
Checkland, P. (2000). Soft Systems Methodology: A Thirty Information Systems Graduates for a Complex Society:
Year Retrospective. Systems Research and Behavioral Aligning IS Curricula with Liberal Education Learning
Science, 17(S1), S11-S58. Outcomes. Journal of Information Systems Education, 25(1),
de Raadt, M., Watson, R., & Toleman, M. (2006). Chick Sexing 35-44.
and Novice Programmers: Explicit Instruction of Problem Riordan, R. J., Hine, M. J., & Smith, T. C. (2017). An Integrated
Solving Strategies. Proceedings of the 8th Australasian Learning Approach to Teaching an Undergraduate
Conference on Computing Education, 52, 55–62. Information Systems Course. Journal of Information Systems
Doymus, K. (2008). Teaching Chemical Equilibrium with the Education, 28(1), 59–69.
Jigsaw Technique. Research in Science Education, 38(2), Rosenhead, J. & Mingers, J. (2001). Rational Analysis for a
249–260. Problematic World Revisited: Problem Structuring Methods
English, L. & Sriraman, B. (2010). Problem Solving for the 21st for Complexity, Uncertainty and Conflict (2 ed.). New York:
Century. In Theories of Mathematics Education (pp. 263– Wiley.
290). Saundage, D., Cybulski, J. L., Keller, S., & Dharmasena, L.
Evans, J. R. (1992). Creativity in MS/OR: Improving Problem (2016). Teaching Data Analysis with Interactive Visual
Solving Through Creative Thinking. Interfaces, 22(2), 87– Narratives. Journal of Information Systems Education, 27(4),
91. 233–247.
Gammack, J. G., Hobbs, V., & Pigott, D. (2011). The Book of Steyn, A., Matthee, M. C., & Turpin, M. (2013). Teaching
Informatics. Cengage Learning. Creativity to First Year Students to Solve Relevant Problems
Halpern, D. F. (1998). Teaching Critical Thinking for Transfer – A Success Story. Proceedings of the 2013 International
across Domains: Disposition, Skills, Structure Training, and Conference SIGEd:IAIM, Milan, Italy.
Metacognitive Monitoring. American Psychologist, 53(4), Taipalus, T., Seppänen, V., & Pirhonen, M. (2018). Coping
449–455. with Uncertainty in an Agile Systems Development Course.
Halverson, E. R. & Sheridan, K. M. (2014). The Maker Journal of Information Systems Education, 29(2), 117–126.
Movement in Education. Harvard Educational Review, Tiruneh, D. T., Verburgh, A., & Elen, J. (2014). Effectiveness
84(4), 495–504. of Critical Thinking Instruction in Higher Education: A
Hasso Plattner Institute of Design. (n.d.). An Introduction to Systematic Review of Intervention Studies. Higher
Design Thinking Process Guide. Retrieved from Education Studies, 4(1), 1-17.
https://dschool- Topi, H. (2019). Reflections on the Current State and Future of
old.stanford.edu/sandbox/groups/designresources/wiki/3687 Information Systems Education. Journal of Information
3/attachments/74b3d/ModeGuideBOOTCAMP2010L.pdf?s Systems Education, 30(1), 1-9.
es-sionID=573efa71aea50503341224491c862e32f5edc0a9. Topi, H., Karsten, H., Brown, S. A., Carvalho, J. Á., Donnellan,
Jones, B. M. (2015). Re-Engineering Introductory Information B., Shen, J., Tan, B and Thouin, M. (2017). MSIS 2016
Systems Course for the 21st Century. Proceedings of the Global Competency Model for Graduate Degree Programs in
2015 International Conference SIGEd:IAIM, Ft. Worth, TX. Information Systems. Communications of the Association for
Kimmel, S. J., Kimmel, H. S., & Deek, F. P. (2003). The Information Systems, 40(1).
Common Skills of Problem Solving: From Program Topi, H., Valacich, J. S., Wright, R. T., Kaiser, K., Nunamaker,
Development to Engineering Design. International Journal J. F., Sipior, J. C., & Vreede, G. J. de. (2010). IS 2010:
of Engineering Education, 19(6), 810–817. Curriculum Guidelines for Undergraduate Degree Programs
Levitin, D. J. (2016). A Field Guide to Lies and Statistics. UK: in Information Systems. Communications of the Association
Penguin Random House. for Information Systems, 26(1), 359–428.
Matthee, M., Turpin, M., & Kriel, D. (2017). Introducing the Turban, E., Aronson, J. E., & Liang, T.-P. (2004). Decision
Maker Movement to Information Systems Students. In T.-C. Support Systems and Intelligent Systems (7 ed.). Upper
Huang, R. Lau, Y.-M. Huang, M. Spaniol, & C.-H. Yuen Saddle River, NJ: Pearson, Prentice-Hall, Inc.
(Eds.), Emerging Technologies for Education, Lecture Notes Turpin, M., Matthee, M. C., & Kruger, A. (2015). The Teaching
in Computer Science (pp. 161–169). Springer International of Creativity in Information Systems Programmes at South
Publishing. African Higher Education Institutions. African Journal of
Research in Mathematics, Science and Technology, 19(3),
278–288.

251
 Journal of Information Systems Education, Vol. 30(4) Fall 2019

WEF. (2018). The World Economic Forum Future of Jobs

Report 2018. WEF.
Zhao, J. J. & Zhao, S. Y. (2010). The Impact of IQ+EQ+CQ
Integration on Student Productivity in Web Design and
Development. Journal of Information Systems Education,
21(1), 43–53.

AUTHOR BIOGRAPHIES

Machdel Matthee is an associate professor at the department

of informatics at the University of
Pretoria in South Africa. She is
interested in developing thinking
skills such as critical thinking,
problem solving, and
computational thinking needed in a
technology-driven world. Other
research interests include
educational technologies, privacy,
cyber safety, fake news, and user
experience. She is currently an associate editor of the South
African Computer Journal (SACJ) and a member of the
editorial board of the Journal of Information Systems Education
(JISE).

Marita Turpin is a senior lecturer at the department of

informatics at the University of Pretoria
in South Africa. She teaches thinking
skills (including critical thinking,
problem solving, systems thinking, and
design thinking) to a variety of
audiences, ranging from high school
learners and undergraduate IS students
to industry professionals. Her research
related to the teaching of thinking skills
in IS has been disseminated in a variety
of international outlets. She is also an
active researcher in the field of ICT for socio-economic
development (ICT4D).

252
Copyright of Journal of Information Systems Education is the property of Journal of
Information Systems Education and its content may not be copied or emailed to multiple sites
or posted to a listserv without the copyright holder's express written permission. However,
users may print, download, or email articles for individual use.