See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/333844306

Participatory Design: an Approach for Involving Teachers as Design Partners

Thesis · June 2019

CITATIONS READS

3 605

1 author:

Ari Tuhkala
University of Jyväskylä
14 PUBLICATIONS   86 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Participatory Design: an Approach for Involving Teachers as Design Partners View project

All content following this page was uploaded by Ari Tuhkala on 18 June 2019.

The user has requested enhancement of the downloaded file.

JYU DISSERTATIONS 92

Ari Tuhkala

Participatory Design:
an Approach for Involving
Teachers as Design Partners
 JYU DISSERTATIONS 92

Ari Tuhkala

Participatory Design:
an Approach for Involving
Teachers as Design Partners

Esitetään Jyväskylän yliopiston informaatioteknologian tiedekunnan suostumuksella

julkisesti tarkastettavaksi yliopiston vanhassa juhlasalissa S212
kesäkuun 14. päivänä 2019 kello 11.

Academic dissertation to be publicly discussed, by permission of

the Faculty of Information Technology of the University of Jyväskylä,
in building Seminarium, auditorium S212, on June 14, 2019 at 11 o’clock.

JYVÄSKYLÄ 2019
Editors
Timo Männikkö
Faculty of Information Technology, University of Jyväskylä
Ville Korkiakangas
Open Science Centre, University of Jyväskylä

Cover photo by Ari Tuhkala.

Copyright © 2019, by University of Jyväskylä

Permanent link to this publication: http://urn.fi/URN:ISBN:978-951-39-7782-5

ISBN 978-951-39-7782-5 (PDF)

URN:ISBN:978-951-39-7782-5
ISSN 2489-9003
ABSTRACT

Tuhkala, Ari
Participatory Design: an Approach for Involving Teachers as Design Partners
Jyväskylä: University of Jyväskylä, 2019, 73 p. (+included articles)
(JYU Dissertations
ISSN 2489-9003; 92)
ISBN 978-951-39-7782-5 (PDF)
Finnish summary

Teachers play an important role in preparing students for a fast-moving, glob-

alised, and digitalised world. Research to date on teachers and technology has
focused on teachers’ skills in using technology and integrating it into learning
and teaching. However, the issue is that teachers are often seen as implementors,
but are denied the opportunity to influence w hat i s b eing i m plemented. Thus,
teachers may perceive that they are being forced to adopt technology without
proper cause.
To address this issue, this dissertation examines participatory design as an
approach for involving teachers as design partners. Previous participatory de-
sign studies in educational contexts have focused on students. To expand such
research to include teachers, the present dissertation examines the following: For
what purpose have participatory design studies examined teachers? How are
teachers’ goals and concerns manifested through participatory design? What is-
sues can be observed when involving teachers in participatory design? The re-
search design comprises three parts: systematic literature mapping of participa-
tory design studies involving teachers; a project involving teachers from a special
education school to design a learning space system; and a project piloting a new
subject as an elective course in Danish lower secondary education.
The systematic mapping produced an overview of the current state of re-
search into participatory design involving teachers. Teachers’ goals and con-
cerns demonstrated how participatory design contributes to the development of
a shared pedagogical vision and communication between political decisionmak-
ers and local teachers. The identified i ssues i n t he t wo p rojects w ere connected
to findings from the literature mapping to propose three building blocks of par-
ticipatory design for involving teachers: identifying roles, needs, rights, and re-
sponsibilities, positioning participation as a possibility instead of an obligation,
and clarifying an agenda for sustainable outcomes. This dissertation thus serves
as a foundation for future efforts in involving teachers in decision-making, for
example when introducing digital technologies in education.

Keywords: participatory design, design, design-based research, teachers, educa-

tors, education, school, technology, digital technologies, digitalisation
Author Ari Tuhkala
Faculty of Information Technology
University of Jyväskylä
Finland

Supervisors Professor Tommi Kärkkäinen

Faculty of Information Technology
University of Jyväskylä
Finland

Professor Ole Sejer Iversen

Center for Computational Thinking and Design
Aarhus University
Denmark

Senior Lecturer Hannakaisa Isomäki

Faculty of Information Technology
University of Jyväskylä
Finland

Reviewers Associate Professor Teemu Leinonen

School of Arts, Design, and Architecture
Aalto University
Finland

Postdoctoral Researcher Nicolai Brodersen Hansen

Department of Industrial Design
Eindhoven University of Technology
Netherlands

Opponent Professor Arnold Pears

Department of Learning in Engineering Sciences
KTH Royal Institute of Technology
Sweden
TIIVISTELMÄ (FINNISH ABSTRACT)

Tuhkala, Ari
Osallistava suunnittelu opettajien kanssa tehtävässä kehittämistyössä
Jyväskylä: University of Jyväskylä, 2019, 73 s. (+artikkelit)
(JYU Dissertations
ISSN 2489-9003; 92)
ISBN 978-951-39-7782-5 (PDF)

Opettajilla on merkittävä rooli kun lapset ja nuoret valmistautuvat yhä nopeam-

min muuttuvaan, globalisoituvaan ja digitalisoituvaan maailmaan. Aikaisempi
opettajia ja digitaalisia teknologioita käsittelevä tutkimus on keskittynyt lähin-
nä opettajien taitoihin käyttää teknologiaa ja sisällyttää sitä oppimiseen. Opetta-
jat nähdäänkin usein toimeenpanijoina, joilla ei välttämättä ole mahdollisuuksia
osallistua päätöksentekoon siitä, miten teknologia kouluihin tuodaan. Opettajat
voivat kokea, että teknologian omaksumisesta on tullut heille pakko, jolle ei ole
selkeitä perusteluja.
Väitöskirjassa tutkitaan osallistavaa suunnittelua lähestymistapana opetta-
jien kanssa tehtävässä kehittämistyössä. Aiemmat aihepiirin tutkimukset on teh-
ty pääsääntöisesti oppilaiden kanssa, minkä vuoksi väitöskirjassa keskitytään
opettajiin. Väitöskirjan tutkimuskysymykset ovat: millaisia tuloksia aiemmat opet-
tajien kanssa tehdyt osallistavan suunnittelun tutkimukset sisältävät, miten opet-
tajien tavoitteet ja huolenaiheet ilmenevät kehittämistyön aikana ja mitä ongel-
mia opettajien osallistamiseen liittyy? Väitöskirjan tutkimusasetelma sisältää kol-
me osaa: systemaattisen kirjallisuuskartoituksen opettajien kanssa tehdyistä osal-
listavan suunnittelun tutkimuksista, oppimistilajärjestelmän kehittämishankkeen
Valteri-koulu Onervassa sekä kehittämishankkeen, jossa tutkittiin uutta teknolo-
giaymmärrys nimistä oppiainetta tanskalaisissa yläkouluissa.
Systemaattisella kirjallisuuskartoituksella saatiin ajankohtainen ja kattava
kuvaus opettajia koskevasta osallistavan suunnittelun tutkimuskentästä. Opetta-
jien tavoitteiden ja huolenaiheiden tarkastelun perusteella osallistava suunnitte-
lu edesauttoi pedagogisen vision muodostamisessa ja toimi viestintäväylänä pai-
kallisten opettajien sekä poliittisten päätöksentekijöiden välillä. Kehittämishank-
keissa tehdyt havainnot yhdistettiin kirjallisuuskartoituksen tuloksiin, joiden pe-
rusteella väitöskirja ehdottaa kolmea toimenpidettä opettajien osallistamiseksi.
Ensiksikin, eri osallistujaryhmien roolien, tarpeiden, oikeuksien ja velvollisuuk-
sien tunnistaminen. Toiseksi, osallistumisen perusteleminen opettajille mahdol-
lisuutena velvollisuuden sijaan. Kolmanneksi, kehittämishankkeen tavoitteiden
täsmentäminen kestävien tulosten aikaansaamiseksi. Väitöskirja muodostaa pe-
rustan opettajien osallistamiseksi päätöksentekoon, esimerkiksi kun teknologiaa
otetaan käyttöön kouluissa.

Avainsanat: osallistava suunnittelu, design, design-tutkimus,

opettajat, kouluttajat, koulutus, koulu,
teknologia, digitaaliset teknologiat, digitalisaatio
ACKNOWLEDGEMENTS

I am sincerely grateful to the Doctoral School of the Faculty of Information Tech-

nology for funding the long and winding road of my PhD journey. Thanks to
you, I was able to fully concentrate on my work, which is a privilege in the aca-
demic world nowadays. As well, I owe gratitudes to Pekka Neittaanmäki for
arranging financial support in the early stage of my career. My dissertation was
supervised by highly proficient people: Tommi Kärkkäinen, your endless source
of enthusiasm keeps me going on, even when papers get rejected, doors closed,
or applications turned down. Ole Sejer Iversen, simply put, I have been honored
to learn from the best. Hannakaisa Isomäki, you helped me in getting through
difficult times, thank you. Finally, a dissertation would not be a proper disser-
tation without reviewers and an opponent: Nicolai Brodersen Hansen, Teemu
Leinonen, and Arnold Pears, I appreciate your effort.
I was delighted to meet a number of brilliant academic people during my
PhD journey, from old-school masterminds to junior researchers like myself. Ole
Sejer Iversen, Toni Robertson, Kim Halskov, Jesper Simonsen, Susanne Bødker,
David Lamas, Nicolai Brodersen Hansen, Michail Giannakos, Christopher Frauen-
berger, Netta Iivari, Marianne Kinnula, Helena Karasti, Rachel Charlotte Smith,
Rachael Luck, Eva Eriksson, and other pioneering researchers who I have met,
but whose name I do not recall. Marie-Louise Wagner, Sofia Papavlasopoulou,
Peter Börjesson, Amanda Geppert, Linus Kendall, Linda Tonolli, Amanuel Zewge,
Peter Lyle, Matti Nelimarkka, Maria-Cristina Ibarrah, Mariana Costard, Kasper
Skov Christensen, Lindsay MacDonald Vermeulen, Jo Vermeulen, Noora Salo-
nen, Maarten Van Mechelen, Gökçe Elif Baykal, Laura Popplow, Seray Ibrahim,
Mela Bettega, Hasselt sisters (Katrien Dreessen and Selina Schepers), Niels Hen-
driks, Banu Saatçi, Julie Inger Goodall, Sarah Thiel, Kati Pitkänen, Hans Põl-
doja, María Menéndez, doppelganger (Henrik Korsgaard), Mitra Azar, and other
young scholars who are going to change the world for better. The next ones go to
Jeanette Falk Olesen, Raune Frankjaer, Rui Xu, Mads Möller Jensen, Ask Risom
Bøge, Gabriel Pereira, Marie Louise Juul Søndergaard, Caroline Hols Lundqvist,
Ida Larsen-Ledet, Winnie Soon, Amanda Karlsson, Sigrid Nielsen Saabye, Naomi
Cherie Lipke, Ane Kathrine Lolholm Gammelby, Peter Dalsgaard, Jonas Frich
Pedersen, Kasper Ostrowski, og alla andre rasende unge forskere i kosegruppa,
længe leve Sandbjerg Gods!
There are a bunch of colleagues who make Agora worth visiting. Antti-
Jussi Lakanen, Ville Isomöttönen, Toni Taipalus, Sami Kollanus, Paavo Niemi-
nen, Laura Mononen, Piia Perälä, Mariia Gavriushenko, Manja Nikolovska, Kati
Clements, Juuli Lintula, Dicle Köse, Denis Kotkov, Naomi Woods, Gaurav Pandey,
Hadi Ghanbari, Arttu, Teija Palonen, Johanna Silvennoinen, Hilkka Grahn, Tuomo
Kujala, Tuomas Kari, Markus Salo, Panu, Tapio Tammi, Ville Seppänen, Leena
Hiltunen, Sisko Minkkinen, Sanna Juutinen, Antti Ekonoja, Tommi Lahtonen,
Jonne Itkonen, Antti-Juhani Kaijanaho, Vesku, Markus Hartikainen, Tero Tuovi-
nen, Mirka Saarela, Ville Heilala, Pieta Sikström, Mikko Siponen, Ari Viinikainen,
Ville Tirronen, Sanna Mönkölä, Mikko Rönkkö, and of course Jorma Kyppö. Many
of these people are also fellow combatants from the IT-faculty floorball, which
is by far the most amusing sports event there can be. Warm thanks to the ad-
ministrative staff at the IT-faculty for helping me out in several practical prob-
lems along the way: Eija Hatanpää, Katja Tynkkynen, Eeva Pöykkö, Kati Valpe,
Elisa Rahikainen, Annemari Auvinen, Niina Ormshaw, Jaana Markkanen, Jari
Rahikainen, Pasi Tyrväinen, Timo Tiihonen, and many others. Last, but foremost,
Pirjo – everyday you feed me up, among Joona Muotka, Milja Parviainen, Juho
Strömmer, Iida Honkavaara, Jari Kurkela, Elina Auvinen, Kaisa Lohvansuu, and
many others who often join me to enjoy the fruits of your kitchen.
Never needless to say, thank you mother, father, family, and relatives. I
would gladly visit you more often, but the distance between is the one and only
major drawback of my current home city. Dear friends, there are so many of you,
and I am happy for every single encounter with you. Some special thanks deserve
to be noted. Juho Strömmer, Juho Salonen, and Sampo Taskinen – A poem, if I
may: being with you, in the woods, is a life of, very good. Tommi Liinalampi,
a member of the parliament to be appointed. Miikka Voutilainen, a professor to
be appointed. Ville Jaatinen (Mustikkamäki) and Markku Lehtonen, there are so
many songs to play and sing, but we keep choosing Battle Hymn. Tommi Kallela, a
brother. Teemu Tirkkonen, an aviator. Zachris Haaparinne, Juho Saksholm, Jesse
Kananen, the other Savonian (Janne Tuomenlehto), Konsta Kalenius, as well as
other members in the one and only kisastudio, never forget the real Kemi.
Emmi Ulvinen, there are no such words to write so that they could describe
how much you mean to me. We have come a long way together, ups and downs,
and always towards better. After a couple of years, one special (also known as
a bebe) can read these words and smile. Finally, even though she cannot read or
speak – despite the fact that her emotional intelligence is well above most humans
– I want to show my gratitude to the dearest friend Roope. Because of you all, not
a single day goes by without a glance of deep warm happy feeling.

Jyväskylä, 24. May 2019

Ari Tuhkala
LIST OF FIGURES

FIGURE 1 Landscape of human-centred design research (adapted from

Sanders and Stappers, 2008, p. 6). ........................................... 18
FIGURE 2 Relationships between research and design (adapted from Sanders
and Stappers, 2014, p. 27)....................................................... 21
FIGURE 3 Systematic literature mapping protocol. .................................. 25
FIGURE 4 A draft of the Valteri School Onerva building from the con-
struction project brochure. ..................................................... 27
FIGURE 5 The school concept (Nevari, 2013)........................................... 27
FIGURE 6 Interrelations between dissertation findings. ........................... 49
FIGURE 7 Conclusion in Finnish............................................................ 56
LIST OF TABLES

TABLE 1 Overview of publications and their relevance to the study. ....... 16

TABLE 2 Timeline of the dissertation work............................................ 24
TABLE 3 Design stage activities in the ONSPACE project. ...................... 28
TABLE 4 Development stage activities in the ONSPACE project. ............. 29
TABLE 5 Activities in the Technology Comprehension project. ............... 30
TABLE 6 Summary of methods in this dissertation ................................ 31
TABLE 7 Summary of participatory design studies involving teachers
(n = 72). .............................................................................. 33
TABLE 8 Teachers in participatory design of environments (n = 15). ....... 33
TABLE 9 Teachers in participatory design of practices (n = 29). .............. 35
TABLE 10 Teachers in participatory design of technologies (n = 28).......... 37
CONTENTS

ABSTRACT
TIIVISTELMÄ (FINNISH ABSTRACT)
ACKNOWLEDGEMENTS
LIST OF FIGURES
LIST OF TABLES
CONTENTS
LIST OF INCLUDED ARTICLES

1 INTRODUCTION ............................................................................ 13

2 PARTICIPATORY DESIGN ................................................................ 17

2.1 Foundations of participatory design ........................................... 19
2.2 Involving users as design partners ............................................. 20
2.3 Constructing knowledge through design .................................... 21
2.4 From designing objects to building communities ......................... 22

3 RESEARCH DESIGN........................................................................ 24
3.1 Systematic literature mapping ................................................... 24
3.2 Designing learning space system for open and adaptable school ... 26
3.3 Piloting Technology Comprehension as an elective course ............ 28
3.4 Summary of methods in this dissertation .................................... 30

4 RESULTS ......................................................................................... 32
4.1 Environments, practices, and technologies .................................. 32
4.2 Teachers’ goals and concerns ..................................................... 39
4.3 Issues in involving teachers ....................................................... 42

5 DISCUSSION AND CONCLUSION................................................... 45

5.1 Discussion of results ................................................................. 45
5.2 Conclusion............................................................................... 48
5.3 Quality and limitations ............................................................. 50
5.4 Ethical considerations ............................................................... 52
5.5 Author’s contributions.............................................................. 53

YHTEENVETO (FINNISH SUMMARY) ..................................................... 55

REFERENCES.......................................................................................... 58

INCLUDED ARTICLES
LIST OF INCLUDED ARTICLES

PI Tuhkala Ari, Kärkkäinen Tommi, and Nieminen Paavo. Semi-automatic

Literature Mapping of Participatory Design Studies 2006–2016. Proceedings
of the 15th Participatory Design Conference: Short Papers, Situated Actions, and
Workshops - PDC ’18, 2018.

PII Tuhkala, Ari. Participatory Design with Teachers - Systematic Mapping of

Studies. Submitted, 2019.

PIII Tuhkala Ari, Isomäki Hannakaisa, Hartikainen Markus, Cristea Alexan-

dra, and Alessandrini Andrea. Design of a Learning Space Management
System for Open and Adaptable School Facilities. Communications in Com-
puter and Information Science, vol. 865, 2018.

PIV Tuhkala Ari, Wagner Marie-Louise, Nielsen Nils, Iversen Ole Sejer, and
Kärkkäinen Tommi. Technology Comprehension – Scaling Making into a
National Discipline. Proceedings of the Conference on Creativity and Making in
Education - FabLearn Europe’18, 2018.

PV Tuhkala Ari, Wagner Marie-Louise, Iversen Ole Sejer, and Kärkkäinen

Tommi. Technology Comprehension – Combining Computing, Design,
and Societal Reflection as a National Subject. International Journal of Child-
Computer Interaction, 2019.
1 INTRODUCTION

Digital technologies such as computers, mobile devices, and software applica-

tions are developing rapidly (hereinafter referred to as technology) and have a
strong impact on societies. This development becomes actualised when tasks
that once required human involvement become automated through digital form.
A typical example is how the transportation industry is changing dramatically
due to the introduction of automatic vehicles. Reducing human error in the op-
eration of aeroplanes, ships, and cars makes transferring goods and people more
efficient. Even though this is a positive direction, it raises new challenges: for
example, algorithms must now accommodate decisions previously made by pro-
fessional drivers, the outcome of which can directly impact human life.
Similarly, it is said that education is going through a digital reform. Tech-
nology presents both challenges and possibilities for education. The benefits of
computers in the classroom, for example, include: significantly increased academic
achievement in science, writing, math, and English; increased technology use for varied
learning purposes; more student-centered, individualized, and project-based instruction;
enhanced engagement and enthusiasm among students; and improved teacher–student
and home–school relationships (Zheng et al., 2016, p. 24). Realising these benefits,
however, does not happen automatically. Rather, it relies greatly on how teachers
choose to introduce and implement technology (see Owston, 2007; Wastiau et al.,
2013; OECD, 2015; European Commission, 2018).
Previous research on teachers and technology has focused on teachers’ abil-
ity to use and integrate technology in teaching and learning (Cox, 2013). These
skills are defined in technological competence frameworks, such as Technolog-
ical Pedagogical Content Knowledge (Mishra and Koehler, 2006), digital liter-
acy (Hall et al., 2014), and digital competency (Krumsvik, 2014; Røkenes and
Krumsvik, 2016; Kelentric et al., 2017). Some researchers have examined teachers’
proficiency in integrating technology (Hsu, 2017), their adoption of new technolo-
gies (Aldunate and Nussbaum, 2013; Salinas et al., 2016), perceived usefulness
of technology (Scherer et al., 2015; Tondeur et al., 2016), motivations for using
technology (Uluyol and Şahin, 2016), and pedagogical beliefs about technology
(Ertmer et al., 2012; Mama and Hennessy, 2013; Admiraal et al., 2017).
14

According to these studies, teachers need to be provided with high quality

training, enough time to get adapted to technology, and enough support to peda-
gogically utilise technology. Moreover, teachers’ willingess to use technology in-
creases, if technology gives them positive experiences and aligns with their peda-
gogical beliefs. In contrast, insufficient time to learn new technologies, the lack of
opportunity for professional development, lack of support from colleagues and
administration, and negative attitudes towards technology are well known barri-
ers (Owston, 2007; Lawrence and Calhoun, 2013; Mama and Hennessy, 2013; Ng,
2015; Vrasidas, 2015).
Despite the research into the subject, realising the benefits of technology has
proven challenging. In Finland, for example, where teachers are highly educated,
students perform comparatively well in international tests (OECD, 2016), and
people, in general, have a high technology adoption level (Cruz-Jesus et al., 2016).
However, there are signs that the negative impacts of technology are overcoming
the benefits: as a teacher, I have had enough of digital hype by consults1 ; majority of stu-
dents, parents, and teachers are getting frustrated with the drawbacks of digitalisation2 ;
and the use of technology is impairing learning results3 .
While these reports demonstrate anecdotal evidence, they nonetheless in-
dicate that current ways of introducing technology in schools can be improved.
Teachers should be more involved in the processes in which decisions about tech-
nology integration in schools are made (Cober et al., 2015, p. 204). A systematic
literature review of studies in Computers and Education, the highest ranked journal
about the pedagogical use of technology, shows that of 352 studies, only 30 per-
cent involved teachers as study participants and only 24 percent involved stake-
holders in co-designing technology, whether they were teachers or not (Pérez-
Sanagustín et al., 2017, p. A11). A large-scale survey of teachers in Cyprus shows
that almost half (43 percent) of 531 teachers experienced having no influence on
how technology is introduced in schools (Vrasidas, 2015). Hence, teachers are
often seen as implementors, but are denied the opportunity to influence what is
being implemented (see Cviko et al., 2014, p. 69 and Kyza and Nicolaidou, 2017,
p. 263).

Research objective

A Finnish government report named The current status of the digitalisation of learn-
ing environments in basic education and the readiness of teachers to utilise digital learn-
ing environments calls for involving all stakeholders to make decisions about tech-
nology (Tanhua-Piiroinen et al., 2016, p. 65). This dissertation answers this call by
examining participatory design (PD) as a potential approach for involving teachers
as design partners. PD is a design and research approach, of which the partic-
ular aim is to involve stakeholders directly in the design process (Schuler and
Namioka, 1993; Bergvall-Kåreborn and Ståhlbrost, 2008; Simonsen and Robert-

1 Helsingin sanomat 27.9.2017

2 Helsingin sanomat 4.10.2017
3 Helsinging sanomat 18.11.2018
 15

son, 2013). PD is well-known in the field of Human–Computer Interaction, and it

has recently gained attention in the education field. For example, in a recent book
called Participatory Design for Learning, PD was proposed as a suitable approach
to improve the development, implementation, and sustainability of learning in-
novations (DiSalvo et al., 2017, p. 5).
PD has been previously examined in the educational context, most notably
when developing technology for children with special needs (e.g. Druin, 2002;
Iversen and Dindler, 2013). However, few studies exist in which teachers have
been the main stakeholders (also noted by Kyza and Georgiou, 2014, p. 60).
Thus, the present research objective is to examine PD in the educational context
and from the teacher perspective. This objective is executed with the following
research questions:

RQ1: For what purpose have PD studies examined teachers?

RQ2: How are teachers’ goals and concerns manifested through PD?
RQ3: What issues can be observed when involving teachers in PD?

The research design comprises three parts. The first question is addressed by
conducting a systematic literature mapping of PD studies that have involved
teachers. The second question examines teachers’ goals and concerns across two
projects. The first project, called ONSPACE, involves the development of a learn-
ing space system for a new special education school in Finland. The author en-
gaged in the project as a research assistant and later as a software developer.
The second project, Technology Comprehension, piloted a new subject for Dan-
ish lower secondary education as an elective course. The author engaged in the
project during a one-year research visit at Aarhus University. The third question
is answered by analysing the issues observed in the two projects and connecting
them with the findings from the systematic mapping.
The included publications are presented in Table 1. The semi-automatic lit-
erature mapping of PD studies from 2006 to 2016 was published in the Proceed-
ings of the Participatory Design Conference (PI). The systematic mapping of the PD
studies involving teachers is completed and the draft is submitted to Instructional
Science (PII). The findings from the ONSPACE project were presented in the Inter-
national Conference of Computer Supported Education and published in the Springer
book series called Communications in Computer and Information Science (PIII). The
findings from the Technology Comprehension project were published in the Pro-
ceedings of the FabLearn Europe Conference (PIV) and expanded as a journal arti-
cle for the special issue of the International Journal of Child-Computer Interaction
(PV). In addition, the author has published research, not included to this disserta-
tion, about supporting students’ interactions with technology in higher education
(Tuhkala and Kärkkäinen, 2018) and intelligent learning systems (Gavriushenko,
Khriyenko and Tuhkala, 2017).
The contributions of this dissertation are three-fold. First, it contributes to
the research in PD: a comprehensive sample of PD studies has been collected
and organised into thematic structures with semi-automatic methods (PI) and
16

TABLE 1 Overview of publications and their relevance to the study.

Research Publication Relation to the research question

question
RQ1 PI Locate thematic structures in PD studies through semi-automatic
mapping
PII Examine the research purposes of PD studies involving teachers
RQ2 PIII Summarise the goals and concerns related to the new school and
the learning space system
PIV and PV Summarise the goals and concerns related to the implementation
of the new elective course
RQ3 Dissertation Analyse issues observed in the two projects and connect them to
the systematic mapping

studies that involved teachers identified, categorised, and synthetised (PII). Sec-
ond, the ONSPACE project contributes to the research on developing technology-
enhanced learning environments, and the Technology Comprehension project
informs digital fabrication and making in formal education. Furthermore, the
projects have practical impact by developing a new system still in use and by
reporting the status of educational reform to the Danish Ministry of Education.
Third, the dissertation is written in the form of a self-contained publication and
contributes to the research on PD in a formal education context. Thus, it does
not only summarise the published articles but outlines how to scaffold PD with
teachers (Section 5.2).
The dissertation is organised as follows. Chapter 2 introduces PD through
four topics: foundations (Section 2.1), user as a design parter (Section 2.2), gen-
erating knowledge through design (Section 2.3), and from designing objects to
building communities (Section 2.4). Chapter 3 specifies the research design and
describes the systematic literature mapping (Section 3.1), design of the learning
space system (Section 3.2), piloting of the new elective course (Section 3.3), and
methods in this dissertation (Section 3.4). Chapter 4 presents the results, includ-
ing a synthesis of the mapped PD studies (Section 4.1), a summary of teachers’
goals and concerns (Section 4.2), and issues in the two projects (Section 4.3).
Chapter 5 presents a discussion of the results (Section 5.1), conclusion (Section
5.2), strengths and limitations (Section 5.3), ethical considerations (Section 5.4),
and author’s contributions (Section 5.5).
2 PARTICIPATORY DESIGN

There are many design-related research paradigms. Well-known examples, espe-

cially in the educational context, include design-based research (Wang and Han-
nafin, 2005), educational design research (Reeves, 2015), human-centred design or
user-centred design (see Iivari and Iivari, 2006; Steen, 2011), and design science
(see Cross, 2001). Common to these paradigms is to consider design, practice,
and research together: design as envisioning new objects and representing them
by drawing, modifying tangible materials, or utilising digital tools; practice as
solving concrete problems through iterative cycles of design, development, en-
actment, and analysis; and research as collecting and analysing data to examine
the outcomes.
In this chapter, PD is positioned in relation to the landscape of human-
centred design research that has been defined by Sanders and Stappers (2008).
A simplified version of this landscape is presented in Figure 1. The horizontal
axis has two edges: user as an informant and user as a design partner. It depicts
two different ways to configure user participation in design. In the former, users
are investigated – how they perceive and react to different materials, forms, and
stimuli – to modify the functionality of the designed object. In the latter, users
are active agents who influence and make decisions in design. The edges on the
vertical axis are design driven and research driven. This depicts which outcome is
the more predominant purpose of user participation: design objects or construct
knowledge. For example, involving users for purely artistic purposes would be
design-dominant, and involving users only for data collection purposes would
be research-dominant.
Sanders and Stappers (2008) define User-centred design as a broad category
for approaches in which the user is the object of investigation. In usability testing,
users test new designs and the objects are modified based on these findings. This
can also be based on existing knowledge, such as when designing furniture based
on human anatomy, as in ergonomics. In contextual inquiry, design is informed by
interviewing users about use practices, wherein applied ethnography the users are
observed. As such, all these approaches are research-driven. For example, it is
difficult to see how contextual inquiry could be separated from the user inter-
18

Design
driven

Participatory design
User-centred design

Contextual
Usability testing
inquiry

Scandinavian
Applied participatory
Ergonomics design
Research ethnography
driven
User as an User as a design
informant partner

FIGURE 1 Landscape of human-centred design research (adapted from Sanders and

Stappers, 2008, p. 6).

views without losing its very purpose.

Similarly, PD can be understood as a broad category. The reason for com-
paring these two categories is to highlight the differences: while the focus in user-
centered design is to investigate users to develop new solutions, the focus in PD
is to create a space where users, designers, and researchers explore the problem
and envision the solutions together (Leong and Iversen, 2015). This dissertation
focuses specifically on Scandinavian participatory design. It can be characterised as
an action-research inspired approach, which means that both design and research
are driven by local accountability: to support the local stakeholders instead of just
producing objects and knowledge (see Simonsen and Robertson, 2013, p. 44).
The next sections describe PD within this framework. Section 2.1 provides
a brief overview of the PD history, as the topic is thoroughly addressed in Her-
itage, having a say (Kensing and Greenbaum, 2013), A brief overview of the history of
participatory design (Halskov and Hansen, 2015), What is participatory design: his-
tory (Spinuzzi, 2005), Participatory design research overview (Pilemalm, 2018), and
History (Bødker and Kyng, 2018). Section 2.2 elaborates on what users as design
partners means. Section 2.3 concentrates on the idea of constructing knowledge
through design. Section 2.4 presents the recent shift of focus from designing ob-
jects to building communities.
 19

2.1 Foundations of participatory design

The roots of PD are in the workplace democracy movement in Scandinavia that

emerged in the late 1960s and 1970s. The movement addressed concerns over em-
ployees’ opportunities to influence how, and with what implications, computer
systems were introduced at their workplaces (Beck, 2002). The foundations were
built in projects that engaged in this movement, such as the Iron and Metal Work-
ers Union project in Norway, Democratic Control and Planning in Working Life in
Sweden, and Democracy, Development, and Electronic Data Processing System
in Denmark (see Bjerknes and Bratteteig, 1995; Bødker and Kyng, 2018). These
projects demanded more democratic working conditions by increasing workers’
influence on the use of technology, developing new technology and work prac-
tices, and developing new design methods (Gregory, 2003; Iversen et al., 2012;
Bødker and Kyng, 2018).
An important stepping stone for PD was the publication of a book called
Computers and Democracy: a Scandinavian challenge (Bjerknes et al., 1987). Suchman
(1988) described the book as a genuinely human-centered alternative to technol-
ogy design. This alternative was based on two notions. First, design addresses
the contradiction between tradition and transcendence – what is and what could
be (Ehn, 1988). Design can either support old practices, values, and power struc-
tures or aim to change them. Second, people who are affected by design should be
involved in making decisions about it (Greenbaum, 1993; Simonsen and Robert-
son, 2013). These ideas were disseminated by pioneer researchers like Susanne
Bødker (1987) with Through the Interface: a Human Activity Approach to User Inter-
face Design, Pelle Ehn (1988) with Work-Oriented Design of Computer Artifacts, Joan
Greenbaum and Morten Kyng (1991) with Design at Work: Cooperative Design of
Computer Systems, and Tone Bratteteig (2003) with Making change: Dealing with
relations between design and use.
Nowadays, PD researchers and practitioners form an established and mul-
tidisciplinary community that is more or less committed to the ideals of the Scan-
divanian tradition (see Vines et al., 2015). Topics like democratic decision-making
and empowering marginalised people are still in current debate, especially in the
community’s main venue: the biannual Participatory Design Conference. At the
same time, involving stakeholders has become a mainstream practice in the soft-
ware industry and the pragmatic side of PD, that is developing techniques for
involving stakeholders, has pervaded other design approaches as well (Bødker
et al., 2000; Spinuzzi, 2002; Bødker and Kyng, 2018). This has led to an intensive
discussion of what are, or should be, the contemporary characteristics of PD (see
Halskov and Hansen, 2015; Smith et al., 2017; Bødker and Kyng, 2018; Pilemalm,
2018).
20

2.2 Involving users as design partners

The notion of a design partner comes from the works of Druin (2002, p. 3), who
analysed the ways children take part in design and defined four roles: user, tester,
informant, and design partner. Users are the main audience of an existing tech-
nology, whose practices are investigated to improve the technology. Testers use
technology that is not yet released for commercial use, and the aim is to develop
the technology for a larger audience. Informants have an active part throughout
the design and provide input before, during, and after the technology is devel-
oped. Partners are acknowledged as legitimate decision-makers and promoted
with an equal role with designers and researchers. This typology of roles was
later expanded with two roles, co-researcher and protagonist (Iversen et al., 2017).
Co-researchers take part in gathering and analysing data to investigate the use
context side by side with researchers (cf. Duarte et al., 2018), and protagonists
develop an ownership to the design project and take the responsibility to pursue
it further (cf. Bødker and Kyng, 2018).
This kind of change in participants’ role is referred to as genuine participa-
tion (Bødker et al., 2004, p. 58). In Routledge International Handbook of Participatory
Design, genuine participation is defined as a fundamental transcendence of the users’
role from being merely informants to being legitimate and acknowledged participants in
the design process (Simonsen and Robertson, 2013, p. 5). Accordingly, this kind of
participation actualises when participants are not just answering questions but
drawing, sketching, and using other ways to explicate their perspectives. This
highlights how PD is about enabling participants to realise that there are alter-
native choices, to negotiate what they care about most in these choices, and to
influence how these choices are pursued (Bødker, 2003; Iversen et al., 2012; Bød-
ker and Kyng, 2018). Establishing genuine participation requires that participants
are provided with access to relevant information, an independent position, and a
right to take part in decision-making, and that there are appropriate design meth-
ods and enough organisational flexibility (Clement and Van den Besselaar, 1993;
Kensing and Blomberg, 1998).
However, establishing genuine participation is challenging because there is
no single right way to do it (Schuler and Namioka, 1993). Even when the partic-
ipants are constantly involved in design, they can held back from influencing on
any actual decisions (Bratteteig and Wagner, 2012; Frauenberger et al., 2015). For
example, this may be the case when the participants are present in design meet-
ings, but lack the proper concepts and language to state their opinions (Bødker,
2003; Simonsen and Robertson, 2013). As a general rule, instead of being satisfied
with the fact that the stakeholders are involved, there is a need for cricital reflec-
tion: questioning who initiates and directs participation, why certain participants
are involved, who these participants are, and who benefits from the outcomes of
participation (Vines et al., 2013; Frauenberger et al., 2015; Halskov and Hansen,
2015; Smith et al., 2017; Saad-Sulonen et al., 2018).
 21

2.3 Constructing knowledge through design

It was already introduced in the beginning of the chapter that PD is driven by

both design and research. However, there are different ways to understand how
these relate. Figure 2 presents four ways in which design and research interrelate:
Design and research separated represents when industrial design and academic re-
search need to be completely separated from each other. Research for design refers
to the practice of carrying out research to design better solutions (also known as
research-based design). Design research refers to research on how to conduct de-
sign, such as methods, tools, or principles (cf. science of design in Cross, 2001, p.
53). In research through design, design activities have a formative role in generating
knowledge (see Zimmerman et al., 2007, 2010).

Research and Research for Research through

Design research
design separated design design

Research Research Design

Design Research Design Research

Design

FIGURE 2 Relationships between research and design (adapted from Sanders and Stap-
pers, 2014, p. 27).

PD is considered in relation to research through design. This implies that

design activities constitute pre-conditions for generating knowledge (see Frauen-
berger et al., 2015, p. 98). Practical work establishes a dialogic space, where new
knowledge can be constructed through interactions between different stakehold-
ers (Iversen and Dindler, 2013; Bannon et al., 2018). This new knowledge can be
about 1) the social context in which the work was carried out, 2) the design out-
comes, 3) the methods of conducting and analysing design, and 4) the concepts
and frameworks related to design (based on Frauenberger et al., 2015).
Knowledge about the social context comprises, for example, participant
conceptualisations of their working and living practices. This knowledge is con-
structed in a bi-directional process, where the participants reflect their ways of
working and designers and researchers aim to understand the context. The goal
here is mutual learning, which means that while the designers and researchers
aim to understand the participants, the participants become aware of their cur-
rent practices and envision ways to improve them (Kensing and Blomberg, 1998).
In this sense, PD relates to etnographical research: understanding participants’
practices requires understanding the context, and merely asking is not enough,
because what the participants say is not necessarily consistent with what they do
(see Pontual-Falcão et al., 2018; Lindtner and Lin, 2017; Grönvall and Kyng, 2013).
The outcomes of design, such as new technologies, concepts, and build-
22

ings, often result from creative and uncontrollable activity. Thus, describing the
process of producing these outcomes in a scientifically rigorous way can be chal-
lenging (Cross, 2001). However, these outcomes are representations of knowl-
edge, into which the decisions and considerations are embedded (Frauenberger
et al., 2015). The knowledge these objects manifest is related to the concept of
tacit knowledge – people may not be able to articulate their skills with words,
but instead by demonstrating them (Spinuzzi, 2005). Similarly, taking the de-
signed objects into use may lead to a better understanding than would reading
descriptions of these objects.
Knowledge about methods is constructed by applying, adapting, and de-
veloping tools and means to better involve stakeholders. This has been the goal
in many PD projects (e.g. Triantafyllakos et al., 2011; Muller, 1991). For exam-
ple, Andersen et al. (2015) proposed Actor-Network Theory as an analytical tool,
Barcellini et al. (2015) developed Actual Role Analysis in Design approach, Ma-
linverni et al. (2016) proposed a multimodal analysis to evaluate participants’
actions, and Bratteteig and Wagner (2016) developed a decision-making frame-
work to analyse participation from political perspective. Exploring ways to in-
volve stakeholders is still relevant, as these means are constantly challenged and
enhanced due to technological and societal development (Halskov and Hansen,
2015).
Design can also lead to new concepts and frameworks. While these can also
be referred to as theories, they are considered more as generative rather than fal-
sifiable (see Gaver, 2012, p. 938). This means that these concepts and frameworks
are appraised based on their ability to inspire new insights and raise new issues,
instead of being general rules that can be tested. In this sense, these concepts
and frameworks focus on describing and explaining certain phenomena instead
of predicting. Still, insights regarding practices of design are often referred to as
design principles.

2.4 From designing objects to building communities

Identifying and defining participation in a modern technological and societal

context is complicated (Bergvall-Kåreborn and Ståhlbrost, 2008; Smith et al., 2017).
For example, in open-source software development communities, the boundary
between designers and users has become blurred. Because the development work
can be distributed all over the world, participation does not necessarily mean
physical presence in design activities, but rather influencing from a distance by
proposing changes, pointing out problems, and making recommendations. Fur-
thermore, participation is not limited to single-encounter design activities, but
develops over time and changes its form (Saad-Sulonen et al., 2018).
To take these new kind of communities into account, infrastructuring has be-
come a popular concept (see Bjögvinsson et al., 2012; Dantec and DiSalvo, 2013;
Karasti, 2014). It means that the open-source development community, for ex-
 23

ample, is understood as an infrastructure that connects technical, social, and

organisational aspects and includes people, technology, standards, procedures,
and practices (Karasti, 2014). Hence, infrastructuring is an action of building
these socio-material assemblies (Bjögvinsson et al., 2012). Bødker et al. (2017, p.
269) put this concept into the PD context and defined participatory infrastructing.
The concept implies that participants are involved in creating the structures, net-
works, and agreements necessary for sustainable outcomes. The shift from de-
signing objects to building communities brings the focus to 1) how a PD project
is configured 2) what happens in the background of PD work, and 3) how the
outcomes of PD are sustained (see Iversen and Dindler, 2014; Vines et al., 2015;
Bødker and Kyng, 2018; Saad-Sulonen et al., 2018; Smith and Iversen, 2018).
A critical stage of PD is to configure the project. This refers to exploring the
project context, anticipating who may be the potential participants, and defin-
ing the project’s agenda (Vines et al., 2015; Smith and Iversen, 2018). Smith and
Iversen (2018, p. 12) conceptualised these actions as Scoping, to define participants
in ways that allow for the flexible engagement and agency of diverse stakeholders over
time, as well as configuring participation in ways that enable people, practices and net-
works to (co)evolve. Thus, the fundamental idea here is to open up the project for
potential partners instead of closing it for pre-defined participants.
The second matter is to identify the back-stage work behind the front-stage ac-
tivities (see Bødker et al., 2017). Front-stage activities refer to workshops, design
sessions, and other arrangements typically reported in research papers. However,
most of the actual decisions are made back-stage, such as when the project goals
are negotiated, outcomes evaluated, and conclusions derived (Frauenberger et
al., 2015). Opening up the back-stage to an external audience increases trans-
parency as to how the project unfolds and how the participants influenced deci-
sions (cf. Frauenberger et al., 2015).
Third, when the PD project is over, the challenge is to sustain the outcomes
of the PD project. It is critical that the project has developed resources for its
stakeholders by this point, so that they may take ownership and responsibility
for continuing the project (Iversen and Dindler, 2014; Bødker and Kyng, 2018).
Iversen and Dindler (2014) define the forms of sustainability as: maintaining, scal-
ing, replicating, and evolving. Maintaining refers to integrating the outcomes of a
PD project into existing practices, so that the initiative itself does not stop even
though the project ends. Scaling refers to the effort of expanding outcomes into
larger contexts and communities without losing the essence of the initiative’s
agenda. Smith and Iversen (2018, p. 20) describe this further as establishing
networks of people and organisations, knowledge frameworks, and visions for
developing and implementing long-term strategies. The idea of replicating is
somewhat similar, but with the aim of transferring the initiative to other contexts
instead of expanding. Finally, evolving means that the developed ideas in the
project can change and that abstract ideas may actualise into concrete outcomes
later on.
3 RESEARCH DESIGN

A timeline of the dissertation work is presented in Table 2. As can be noted, the

research design consists of three parts: a systematic literature mapping and two
projects. Section 3.1 describes how the articles for the systematic literature map-
ping were collected. The reason for this part was to establish a strong theoretical
background: PD is a multi-disciplinary field so research is often published in
discipline-specific journals and conferences and locating relevant body of knowl-
edge can be challenging. The mapping examined ten years of PD studies involv-
ing teachers, and the findings served as a basis for analysing the two other parts.
TABLE 2 Timeline of the dissertation work.

Dates Dissertation stage

8/2014 – 12/2014 Research assistant in ONSPACE project
1/2015 – 12/2015 Software developer in ONSPACE project
01/2016 – 5/2017 Systematic literature mapping
8/2017 – 5/2018 Researcher in Technology Comprehension project
6/2018 – 6/2019 Final analysis and finishing dissertation

The two other parts are the research projects that the author was involved
during the dissertation work. The first project is called ONSPACE, which in-
volved teachers to design the learning space system, and the second one is called
Technology Comprehension. Sections 3.2 and 3.3 summarise how these projects
unfolded, what research activities were carried out, and what data was collected.
Section 3.4 describes how the findings in this dissertation were constructed.

3.1 Systematic literature mapping

Data collection took place over 14 search engines and databases to gather a wide
sample of PD literature: ACM Digital library, Bielefeld Academic Search Engine,
EBSCOhost Research Databases, ERIC Institute of Education Sciences search, IEEE
 25

Xplore Digital library, JSTOR, ProQuest, SAGE Journals, ScienceDirect, Scopus,

SpringerLink, Taylor and Francis Online, Wiley Online Library, and Thomson
Reuters Web of Science. Criteria for database selection were the possibility to
1) export multiple references and 2) export references in Mendeley supported
format (RIS, Bibtex, Endnote XML, or Zotero). Thus, Google Scholar, Semantic
Scholar, and CiteSeerX were excluded.
References where participatory design appeared in title, abstract, or keywords,
and which were published between 2006 and 2016 were collected. The references
from different search engines were imported to the Mendeley reference manage-
ment tool to build a single reference database (2943 articles). Because two refer-
ences could have different meta-information and still point to the same source,
the duplicates were removed by using the duplicate identification tool.
The article database was exported to a Python preprocessor (developed in
Nieminen et al., 2013). The preprocessor created a word matrix, which was used
in the semi-automatic clustering of the articles. After the clusters were produced,
the findings were illustrated as a cluster map. To validate the mapping results, six
education-related clusters were identified and the word portions analysed. The
findings and the clustering procedure are described more in detail in PI.
This work continued in PII. The reference database was updated by repeat-
ing the data collection procedure for the year 2017, and the references from 2006
were removed. From the reference database, the references that included the
word teacher either in title, abstract, or keywords were extracted. In total, 191
references included both teacher and participatory design.
The data refinement process is illustrated in Figure 3. In the first exclusion
stage, the references were screened and those not published in a journal or con-
ference, written in English, or where there was no access to full text in any of the
14 databases were removed (55 articles). The excluded references consisted of
workshop desriptions, research proposals, posters, extended abstracts, introduc-
tions to special issues, and editorial notes.

Original data (191)

1) participatory design and teacher appear in title, abstract, or keywords.
2) published between 2007 and 2017

Exclusion 1: not journal or conference articles, not in English, no access to full text (55)

Exclusion 2: includes the word teacher, but does not consider teachers (17)

Exclusion 3: teachers were not participants in the study (38)

Exclusion 4: if conference and journal paper about same study, conference removed (9)

Mapped data (72 studies)

| author(s) | year | venue | teachers | education level | domain | location | research objective |
methods | findings |

FIGURE 3 Systematic literature mapping protocol.

Before the second exclusion stage, full text articles for each remaining ref-
26

erence were downloaded. Articles that included the words participatory design
but defined the study as action research, user-centred design, or another research
approach were eliminated for being beyond the scope of the present study (17
articles).
In the third exclusion stage, the articles that did not involve teachers as par-
ticipants were removed (37 articles). For example, Hussain (2010) stated that
valuable user perspectives are lost if only information from adult carers such as teach-
ers and parents are included in the design process but did not consider teachers any
further. Studies that did not define the participants in detail, but obviously con-
sidered teachers, were included. The final exclusion stage (9 articles) removed
conference articles where the same research was reported in a journal article (cf.
Fage et al., 2014 and Fage et al., 2016). As a result, there were 72 PD studies that
had involved teachers.

3.2 Designing learning space system for open and adaptable school

Valteri School Onerva is part of the national Valteri Centre for Learning and Con-
sulting Centre in Finland. The school provides pre-primary, basic, and voluntary
additional basic education and is specialised in meeting student needs related to
vision, hearing, language, and interaction 1 . The school constructed a new build-
ing, which was put into use in 2016.
A draft of the new school building is seen in Figure 4. The theoretical idea
for the building was based on Marko Kuuskorpi’s dissertation (Kuuskorpi, 2012),
and the school concept was designed by Julianna Nevari (Nevari, 2013). The
concept is illustrated in Figure 5. The learning areas comprise of parks, fountains,
and dens. A park is an open space, which can be easily modified for group work,
presentations, and physical activities. A fountain is a partially open space for
collaborative learning, which can be divided into different areas. A den is a quiet,
individual, and closed space that is used for focusing on tasks.
The purpose of the new school building is to encourage teachers to utilise
new spaces for pedagogical activities. This means that learning activities happen
in the space most suitable for current needs, instead of gathering learning re-
sources into the classroom. The practical problem of the school was to figure out
how the teachers would organise their activities without information on which
spaces are available.

Design stage

The ONSPACE project was initiated on May 1 2014 to address how the teachers
could organise their activities by using modern technology. The project was car-
ried out collaboratively between the Faculty of Information Technology (the Uni-
versity of Jyväskylä) and the Valteri School Onerva. The participants included a

1 Official website of the Valteri School Onerva

 OIVALLUS
Open and adaptable learning and working environment
27

AN INNOVATIVE LEARNING AND WORKING ENVIRONMENT

One of the aims of the project is to create a new kind of learning and working environment that enables functionality, activeness
and the application of new technology. New space arrangements and new ways to use spaces have been created for the
learning and working premises, in compliance with modern ideas on learning. Based on these aims, some spaces that have
traditionally not been utilised in teaching are harnessed for educational use. Examples of this are the stairs of learning, which offer
the opportunity to functionally practise the key skills of learning: reading, calculation and perception of time. We hope that the
building will be a model for a new way to construct schools – taking into account pupils’ special needs both within the building
and in its surroundings. The integration and presence of new technology in daily school life supports pupils’ networking with peer
groups and promotes the utilisation of information networks in teaching.
Dynamic multipurpose learning spaces
The new building is optimal for action-based learning because its premises and furniture can be adapted to different purposes.
In addition, the target of the project is to utilise the premises efficiently, making the common spaces available to the various func-
tions, in various ways, from morning to evening. Restaurant Omppu (Finnish for apple) is the heart of the entire building and, at
the same time, a meeting place. Both staff and pupils can work together with others in the open working area and, for instance,
FIGURE 4 inAthedraft
with a peer ofworking
intensive the Valteri School
area, or alone Onerva
in the silent building
working area. fromworking
These different the areas
construction project
are called the park,
fountain and den, adapting Julianna Nevari’s learning space concept. In compliance with the space concept, the project is
brochure.
called ‘Oivallus’ (Finnish for ‘insight’).
Accessibility and multi-sensory impact in the learning environment
The design process has been based on pupils’ needs. Accessibility refers to the suitability of the premises for everyone, irre-
spective of the nature of support needed. The spaces and routes are clear, barrier-free and safe, and the perception of spaces
is facilitated by limiting and outlining different spaces with contrasts. Good acoustics are an important factor that promotes
learning. The spaces can also be lit in various ways, which is important from the viewpoint of vision as well as of concentration.

INCREASED COOPERATION

OPEN INTENSIVE SILENT

WORKING AREA WORKING AREA WORKING AREA

‘PARK’ ‘FOUNTAIN’ ‘DEN’

INCREASED CONCENTRATION

Group work zone

Small-group work zone
Meeting zone
Multifunction zone
Performance zone
Instructors’ team zone
The new learning and working environment provides different working areas.
Action-based learning is enabled by multi-functional, adaptable premises and furniture.

FIGURE 5 The school concept (Nevari, 2013).

researcher team, special education teachers, occupational therapists, visual sense

specialists, and technical staff. The participants expressed that they were not
content with the outdated facility management system of the previous school. To
develop a system that would better meet the teachers’ needs, the purpose of the
design stage was to elicit the features of a learning spaces system.
The design stage lasted until December 15 2014, and the main activities are
presented in Table 3. All activities were carried out in the old school’s facilities
to give participants familiar surroundings and to give researchers a better under-
standing of the context. The meetings focused on eliciting needs regarding the
new school and envisioning ways to meet those needs.
28

TABLE 3 Design stage activities in the ONSPACE project.

Date Participants Activity Materials

14.5.2014 8 teachers, 1 technical Group discussions, ex- Recorded discussions,

staff, 3 researchers amining a 3D-model of conceptual map of the
the building building
24.8.2014 6 teachers, 4 researchers Eliciting participants’ Recorded meeting, list
needs about the build- of needs and hopes
ing
27.9.2014 6 teachers, 3 researchers Presenting and dis- Recorded meeting, use
cussing initial use cases cases, initial require-
and requirements ments
10.11.2014 Technical administra- Semi-structured inter- Recorded interview,
tor, 1 researcher view technological specifica-
tion
12.12.2014 6 teachers, 2 instructors, Group evaluation of the Recorded evaluation,
3 researchers design stage outcomes requirements specifica-
tions

Development stage

The outcomes of the design stage served as a basis for developing the system. The
sequel project, ONSPACE2, was scheduled to run between May 1 and December
31, 2015. Most of the participants had also served in the design stage. During the
development stage, monthly design meetings were held to report on the develop-
ment work and allow participants to give their feedback and recommendations
on how to proceed (Table 4). The aim was to enable the participants to influence
how their expectations were transformed into a working system.
The new school building opened in January 2016, and the ownership of the
learning space system (source code, documentation, intelligent property rights)
was transferred to the Valteri School Onerva. However, the system still lacked
two critical requirements: the option to make recurring reservations and the abil-
ity to log into the system using existing user accounts. To implement the missing
features, the school recruited a freelance software developer in March 2016. The
system was put into use in late Spring 2016 2 .

3.3 Piloting Technology Comprehension as an elective course

As a part of educational reform in Denmark, the Ministry of Education initiated a

new subject for Danish lower secondary education (13–15 y.o. students). The sub-
ject, Technology Comprehension consisted of three learning objectives: to develop
basic skills in computing, such as programming, algorithms, pattern recognition,
2 ONSPACE web interface, accessed 15.3.2019
 29

TABLE 4 Development stage activities in the ONSPACE project.

Date Participants Activity Materials

17.6.2015 Principal investigator Project initiation: Official project docu-

and head of the school timetables, goals, and ments
contracts
27.8.2015 8 teachers and 3 re- User interface design User interface sketch,
searchers (including 2 session recording
developers)
21.10.2015 8 teachers and 3 re- Development meeting, Meeting recording
searchers (including 2 review of the develop-
developers) ment status
19.11.2015 Same as previous Same as previous Same as previous
17.12.2015 Same as previous but Final meeting where the Meeting recording, sys-
including technical staff ownership of the sys- tem documentation
tem was transferred

and abstraction; skills to specify and articulate a problem and utilise an iterative
design process to develop a digital solution; and skills to reflect and evaluate the
digital solution, its applicability, impact, and ethical concerns with reference to
the broader socio-political context within which it is applied.
To support the implementation of Technology Comprehension across 13
schools, the ministry commissioned a research project with the Centre for Com-
putational Thinking and Design in Aarhus University. The head of the centre (one
of the dissertation supervisors) invited the author of the present dissertation to
join the project as a visiting researcher. The project goals aligned with the present
research objective, as the research in the centre was grounded in Scandinavian
participatory design (see Smith et al., 2015, p. 22).
The project activities are summarised in Table 5. The research started by
sending an electronic survey to the participating schools. The survey asked about
each teacher’s professional background, anticipated challenges regarding Tech-
nology Comprehension, and outlined expectations of being part of the project.
During winter 2017, the project involved school visits and semi-structured in-
terviews with 14 teachers. The interviews examined the expectations of those not
yet teaching the course, and the experiences of those already conducting Technol-
ogy Comprehension classes. Technology Comprehension lessons were observed
during these visits if the schools allowed.
A one-day workshop was held to develop support for the teachers of the
course. The workshop was held once in Aarhus and once in Copenhagen. The
workshop programme allowed the teachers to get to know one another, examine
the Technology Comprehension learning goals, introduce computational think-
ing and design, and program with Micro:bits. The workshop also included theme
discussions about the following topics: what is Technology Comprehension as an
elective course for you; how do you incorporate Technology Comprehension in
your current teaching; how do you perceive the competency goals; and what
30

TABLE 5 Activities in the Technology Comprehension project.

Date Participants Activity Materials

Autumn Principal investigator, Project initiation Learning objectives,

2017 ministry curriculum documents
15.10.2017 13 schools Electronic survey Survey answers
1.11.2017 – 14 teachers, researchers School visits 14 recorded interviews,
31.1.2018 field notes
19.2.2018 8 teachers, 1 pedagog- Workshop in Aarhus Workshop recording,
ical consultant, 3 re- self-assessment, feed-
searchers back, theme discussion
21.22018 7 teachers, 2 principals, Workshop in Copen- Same as in Aarhus
3 researchers hagen
25.4.2018 Teachers and educa- FablearnDK conference Posters, field notes
tional professionals in Kolding

should Technology Comprehension be in future.

At the beginning of the workshop, the teachers filled a self-assessment ques-
tionnaire. The questionnaire included Likert-scale questions about perceived
competencies in using digital tools and in teaching design and computing topics,
as well as open questions about learning methods, positive or negative experi-
ences, and the skills needed to teach Technology Comprehension. The teachers
were tasked to utilise the workshop ideas in their teaching and produce a poster
about these activities with students. At the end of the workshop, the teachers
filled a feedback questionnaire. The developed posters were published in the
FablearnDK conference in Kolding 3 .

3.4 Summary of methods in this dissertation

Previous sections described what data was collected during the dissertation work.
Table 6 presents data and the analysis methods that were used to construct the
findings of this dissertation. Most of data was already pre-processed for the pur-
poses of the individual publications: the design meeting recordings were tran-
scribed and survey answers in Danish translated to English. It needs to be noted
that, however, the analysis process was not straightforward, but rather continous
interaction between literature mapping findings and insights from the projects.
The analysis of the articles was carried out by the author and consisted of
three stages. First, the articles were screened through to gather basic information.
From each article, the following information was extracted to an Excel sheet: au-
thor(s), year, venue, participants, number of teachers as participants, other stake-
holders, education level, and geographical location. In the second stage, the ar-
3 FabLearnDK 2018 website, accessed 20.12.2018
 31

TABLE 6 Summary of methods in this dissertation

Research Data Analysis

question

RQ1 72 PD studies that have involved Organise the studies into categories ac-
teachers cording to the research purpose
RQ2 ONSPACE: eight recorded design Research question based coding of de-
meetings sign meetings
Technology Comprehension: survey Theme-building based on research
answers, interview and field notes, questions
workshop recordings
RQ3 ONSPACE: eight recorded design Identify issues within the project and
meetings, interview with technical connect them to the RQ1 findings
staff, project documentation, develop-
ment documents
Technology Comprehension: survey Identify issues within the project and
answers, interview and field notes, connect them to the RQ1 findings
workshop recordings

ticles were scrutinised more in detail to identify research objectives, methods,

and main findings for each article. The Excel sheet was examined to define high
level categories for the articles. The categories that emerged were environments,
practices, and technologies. After the studies were organised into these three cat-
egories, they were further refined into sub-categories. A synthesis of the mapped
studies was written based on the developed categories.
The author exported the design meeting transcriptions into Atlas.TI to ex-
amine the teachers’ goals and concerns within the ONSPACE project. Utterances
that referred to something that the teachers found valuable, or worrying, were
marked as quotations and assigned with a code goal or concern. The quotations
were further assigned with a code that referred to the object of goal or concern.
The codes were scrutinised to develop answers for the research question. When
regarding the Technology Comprehension project, examining the goals and con-
cerns had been already done within the project (see PV). Data had been analysed
with the other project researchers by watching the recorded theme discussion
together, negotiating potential themes, and riching the themes with survey an-
swers.
For the third question, all materials from the two projects were examined
by the author. The aim was to identify what issues the projects had. However,
it needs to be emphasised that the identified issues represent the author’s per-
spective, as discussed in Section 5.3. Furthermore, the fact that the author had
been acquainted with PD literature clearly had effect on how the projects were
reflected. After the issues were recapitulated, the findings for the RQ1 were ex-
amined again. The purpose was to examine how the identified issues have been
addressed in previous PD literature that has involved teachers.
4 RESULTS

Section 4.1 presents the categories produced through the systematic literature
mapping and overviews the studies within these categories. Section 4.2 describes
the teachers’ goals and concerns regarding the open and adaptable school in the
ONSPACE project and the new curriculum in the Technology Comprehension
project. Section 4.3 presents the issues regarding identifying roles, needs, rights,
and responsibilities, positioning participation as a possibility instead of an obli-
gation, and clarifying an agenda for sustainable outcomes.

4.1 Environments, practices, and technologies

A summary of the studies involving teachers in PD is presented in Table 7. Most

of the studies were published in journals (51). The studies considered several
education levels: pre-primary, primary, secondary, and higher education. More-
over, seven studies were about PD in teacher education, and six investigated PD
in more than one education level. Most of the studies were small-scale investi-
gations such as case studies, with no more than five teacher participants. Seven
large-scale studies consulted 20 or more teachers. However, some studies did
not specify the exact number of teacher participants. Over half the studies were
conducted in the United Kingdom, Netherlands, United States, Australia, and
Finland, and five studies covered more than one geographical location.

Environments

Table 8 presents the studies related to learning environments. In the largest cate-
gory, School buildings, teachers were involved in envisioning a new school concept
or re-design of an existing concept (Burke and Könings, 2016; Koutamanis et al.,
2017; Könings et al., 2017; van Merriënboer et al., 2017; Woolner et al., 2007, 2010).
These studies approached the school building as a whole, including furniture,
(technological) equipment, materials, and structures, whereas two of the studies
 33

TABLE 7 Summary of participatory design studies involving teachers (n = 72).

Venue Education level Number of teachers Location

Journal: 51 Pre-primary: 2 Not defined: 22 United Kingdom: 16
Conference: 21 Primary: 22 1 – 5: 25 Netherlands: 10
Secondary: 19 6 – 10: 9 United States: 7
Higher: 16 11 – 20: 9 Australia: 5
Teacher education: 7 Over 20: 7 Finland: 4
Several levels: 6 Rest of Europe: 16
Asia: 6
Rest of the world: 3
Multiple locations: 5

focused on specific facilities: the university cafeteria (Lundström et al., 2016), and
library learning commons (Somerville and Collins, 2008). In the second category,
Technology-enhanced learning spaces, the studies focused not only on the physical
space but also on how technology is integrated into the learning environment.

TABLE 8 Teachers in participatory design of environments (n = 15).

Category Articles

School buildings Burke and Könings (2016), Koutamanis et al. (2017), Könings et
al. (2017), Lundström et al. (2016), van Merriënboer et al. (2017),
Somerville and Collins (2008), Woolner et al. (2007), Woolner et
al. (2010)
Technology-enhanced Bossen et al. (2010), Casanova and Mitchell (2017), Cober et al.
learning spaces (2015), Joyce et al. (2014), Kreitmayer et al. (2013), Otero et al.
(2013), Stephen et al. (2014)

Regarding the studies in the School buildings category, Burke and Könings
(2016) examined how a school’s history inspired the participants’ design imagi-
nation. They present an example from the Netherlands, De Werkplaats, a school
that was re-designed according to the educational thinking of Kees Boeke. They
point out how a historical narrative can be utilised as a positive agent for change,
but also that previous traditions from more conservative schools can limit and
hinder the potential for design innovations.
Two other studies took place at De Werkplaats. As reported by Koutama-
nis et al. (2017), visual information technology (Building Information Model) was
utilised as a collaborative tool during the building’s lifecycle. The tool served as
a knowledge repository and a communication service, which enabled the par-
ticipants to engage in decision-making. van Merriënboer et al. (2017), in turn,
addressed the relationship between pedagogy and physical spaces. They framed
a three-stage design process: specifying the pedagogy, aligning the chosen ped-
agogy with seating arrangements and physical learning spaces, and realising the
school building. They found PD especially beneficial when teachers’ pedagogical
needs and architects’ non-pedagogical needs (resources, cost-effectiveness) con-
tradicted (also in Woolner et al., 2007). van Merriënboer et al. (2017) conclude
34

that the PD of school buildings is not about the building per se, but negotiating a
shared pedagogical vision and establishing a commitment to this vision.
Two methodological contributions deserve to be highlighted. Woolner et al.
(2010) accounted for using visual tools, such as photo elicitation, diamond map-
ping, and map-based activities, to gather perceptions of various participants and
to improve the learning environment. They concluded that the visual methods
produced rich understandings of the current school environment and enabled the
triangulation of participants’ different perceptions. The second study developed
an interdisciplinary model of practice for participatory building design (Könings
et al., 2017). The model integrates an action research cycle, stakeholder analy-
sis model, ladder of participation tool, and participation matrix to address the
complexity of involving several different stakeholder roles.
In the Technology-enhanced learning spaces category, three studies developed
technologies to be integrated into classrooms: an Internet of Things ecosystem
(Joyce et al., 2014), UniPad application (Kreitmayer et al., 2013), and digital dis-
plays (Otero et al., 2013). Two studies involved teachers in designing new learn-
ing environments where technology plays a major part. Casanova and Mitchell
(2017) provided participants with two provocative design space concepts, which
were then re-designed. This process resulted in rich data about how the partici-
pants conceptualised the learning spaces and the value of technology. Similarly,
Stephen et al. (2014) involved teachers and students to design technology-rich
classrooms as community spaces that are owned and maintained together.
Two studies are described in detail because they pay specific attention to
teachers’ participation. Bossen et al. (2010) accounted for a large PD project,
iSchool, which was about envisioning new learning spaces and opportunities of
pervasive technology. They interviewed the teachers three years after the project
ended and examined what they gained from the project. According to the teach-
ers, the most satisfying experiences were: reflecting with professionals from other
backgrounds, the enthusiasm of the students towards technology, and gaining
experience from using modern technology. Moreover, the teachers expressed
four types of gains: opportunity to reflect on teaching methods, to develop skills
and understandings about technology, to have leverage to influence technology-
related decisions, and to advance their own interest in technology.
Cober et al. (2015) analysed teacher engagement in two case studies and in-
vestigated what supports teacher participation. The teachers engaged in: theory-
driven discussions with researchers and developers to ground design work and
understand each others’ perspectives; design partnerships by providing input,
guidance, and ideas; reflecting on the innovations from a pedagogical perspec-
tive and evaluating the potential impact for students; and adjusting implementa-
tion enactments. Regarding the conditions supporting the teachers, the authors
highlighted a combination of highly facilitated conditions and flexibility, an at-
mosphere of trust and partnership, and designing with contextual knowledge
about the physical environment, students, and potential technologies.
 35

Practices

The studies in the Learning practices category were about establishing professional
communities, intstructional design, and professional development programmes
(Table 9).

TABLE 9 Teachers in participatory design of practices (n = 29).

Category Articles

Communities Booker and Goldman (2016), Duell et al. (2014), Farooq et al.
(2007), Ishimaru and Takahashi (2017), Karimi et al. (2017), Pol-
lock and Amaechi (2013), Selwyn et al. (2017), Tammets et al.
(2011), Vakil et al. (2016)
Instructional design Anderson and Östlund (2017), Barbera et al. (2017), Gros and
López (2016), Harrison et al. (2017), Janssen et al. (2017), Kuure
et al. (2016), Könings et al. (2011), Könings et al. (2010), Könings
et al. (2007, A), Könings et al. (2007, B), Prins et al. (2016)
Professional develop- Al-Eraky et al. (2015), Goeze et al. (2014), Janssen et al. (2014),
ment Kyza and Georgiou (2014), Kyza and Nicolaidou (2017), Põldoja
et al. (2014), Rodrigo and Ramírez (2017), So et al. (2009), Tulinius
et al. (2012)

In the Professional communities category, two studies examined online com-

munities. Duell et al. (2014) established a yearly ambassador programme for in-
troducing design thinking as a general competency in K-12 education in Aus-
tralia. In this study, PD was undertaken to create an online design education
platform and to increase teachers’ capacity to teach creativity and design. Fa-
rooq et al. (2007) developed an online environment for a diverse community of
distributed education professionals. The project drew on PD and included four
design interventions. The study proposed that the interventions were success-
ful because the community members developed ownership over the online en-
vironment and kept using it for long-term professional development and social
networking. Karimi et al. (2017) organised a hackathon workshop for teachers,
where the teachers experimented with technology and designed learning activi-
ties. Because the teachers faced challenges implementing the digital technology
projects, it provided an honest experience of exploring novel technologies and
demystifying some aspects of technical practices.
The other studies in this sub-category were about empowering local com-
munities. Booker and Goldman (2016) examined PD as an approach to tackle
math fears in families by restoring epistemic authority. Pollock and Amaechi
(2013) explored how texting supports rapid and individualised communication
with vulnerable youth. Selwyn et al. (2017) explored the possibilities of making
existing school data available in digital form for teachers, students, and admin-
istrators. This study revealed technical, informational, organisational, and social
issues in democratising data engagement within school settings. Tammets et al.
(2011) examined a teacher accrediation programme that requires the teachers to
36

be involved in community of practices, collaborative learning, and knowledge

building. Finally, Vakil et al. (2016) used the notion of politiced trust to anal-
yse how political and racial solidarity was established, contested, and negotiated
throughout two PD projects.
The Instructional design category consists of studies about designing learn-
ing practices and curricula. Most of the studies were conducted by the same
researchers from the Netherlands. Könings et al. (2007, A) aimed to reduce dis-
crepancies between students’ and teachers’ perceptions on appropriate learning
environments and to collaboratively design these environments. Könings et al.
(in 2007, B) expanded this work by focusing on teachers. In two other studies
(Könings et al., 2010, 2011), the authors invited students to collaborate with teach-
ers. Both the teachers and students found PD appealing in this context, but with
several challenges: PD takes too much time, students underestimate their capa-
bility to decide educational issues, teachers doubt students’ willingness to take
part in PD, and PD outcomes were perceived positively by the students involved
but not by the rest of the class.
Janssen et al. (2017) defined participatory educational design and conducted a
study with three aims: to view classroom teaching as bounded rational design,
develop a tool that supports participants in mapping and sharing their goals,
and develop another tool that helps participants to explore practical and effective
possibilities for designing learning environments. This study demonstrated the
use of tools for improving the quality and usability of learning environments and
stated that even participants with similar backgrounds benefitted from learning
about each others’ practices and goals.
The remaining studies in this sub-category considered a variety of topics.
Barbera et al. (2017) developed learning scenarios to identify moments of change
and describe causes and agents that motivate these changes. Gros and López
(2016) examined the Learning Centric Ecology of Resources model to facilitate co-
design processes. Two studies considered assessment in teaching: Harrison et al.
(2017) explored how to redesign a summative assessment culture that takes into
account students’ post-assessment feedback, and Anderson and Östlund (2017)
considered assessment practices of students who attend special schools. Janssen
et al. (2014) developed a PD-based teacher training trajectory for guided discov-
ery learning (GDL) lessons in biology. Accordingly, the teachers were willing
and capable of implementing GDL, utilised the heuristics that were developed
by experienced teachers, and valued GDL at a higher level than regular lessons.
Kuure et al. (2016) supported English teachers in a Finnish university to become
designers of language learning with new technologies, and Prins et al. (2016) de-
veloped an instructional framework that provided educational designers with a
set of prescriptive guidelines for transforming authentic modeling practices.
Two studies in the Professional development category were about improving
professional development through PD. Kyza and Georgiou (2014) examined PD
for promoting teachers’ sense of ownership towards inquiry-based learning mod-
ules. Accordingly, the teachers perceived PD as a collaborative and supportive
framework that enables the exchange of different perspectives, encourages criti-
 37

cal constructivism, and facilitates new teaching methods and technologies. Con-
versely, the time-consuming nature of PD, communication problems, and partici-
pants’ unequal contributions were identified as the main disadvantages. Despite
this, all teachers preferred designing the teaching module over using pre-made
modules. Kyza and Nicolaidou (2017) conclude that iterative design facilitated
teachers’ professional development because it enabled teachers to reflect on in-
quiry learning and teaching.
The remaining studies in this category were about training programmes.
Al-Eraky et al. (2015) involved teachers in designing a faculty development pro-
gramme for teaching professionalism in medical education, Rodrigo and Ramírez
(2017) developed a master course for online teaching, and Tulinius et al. (2012) de-
signed a programme for teachers to obtain critical appraisal skills and higher aca-
demic capacity. Four studies were about developing digital platforms for profes-
sional development: Goeze et al. (2014) examined how video case-based learning
could promote teachers’ analytical competence to become immersed and to adopt
multiple perspectives, to apply conceptual knowledge, and to describe pedagog-
ical situations. Similarly, Põldoja et al. (2014) addressed the design challenges of
a software solution for self- and peer-assessing teachers’ digital competencies. Fi-
nally, So et al. (2009) designed an online platform where teachers can share vivid
images of their practices with their peers.

Technologies

The studies regarding learning technologies were assigned to the following cat-
egories: Assessment and monitoring tools, Educational games, Learning and teaching
applications, safety and security tools, and Technology for special needs (Table 10).

TABLE 10 Teachers in participatory design of technologies (n = 28).

Category Articles

Assessment and monitor- Gillies et al. (2015), Rodríguez-Triana et al. (2012), Siozos et al.
ing tools (2009)
Educational games Hoda et al. (2014), Klonari and Gousiou (2014)
Learning and teaching Carmichael (2015), Cramer and Hayes (2013), Girard and John-
applications son (2010), Hannon et al. (2012), Kalra et al. (2007), Pedersen et
al. (2012), Rahamat et al. (2011), Song and Oh (2016), Su et al.
(2010), Triantafyllou and Timcenko (2013)
Safety and security Ervasti et al. (2016), Jutila et al. (2015), Pantsar-Syväniemi et al.
(2015)
Technology for special Abdullah and Brereton (2015), Bossavit and Parsons (2016), Br-
needs ereton et al. (2015), Medeiros-Braz et al. (2017), Fage et al. (2016),
Herstad and Holone (2012), Lingnau and Lenschow (2010), Par-
sons et al. (2011), Parsons and Cobb (2014), Zainuddin et al.
(2010)
38

In the Assessment and monitoring tools category, Gillies et al. (2015) devel-
oped an application for giving feedback about students’ playing posture in mu-
sic education. They created a prototype, then asked teachers for feedback before
developing next version for evaluation. Rodrigo and Ramírez (2017) developed
computer-supported collaborative learning scenarios for monitoring students’ in-
teractions. Siozos et al. (2009) reported positive outcomes after involving teachers
and students in designing computer-based assessment tools: both teachers and
students perceived PD as an opportunity to re-conceptualise existing pedagogies
and that PD supported locality, diversity, participation, and attitudes that counter
impassivity and homogenisation.
Two of the studies were about Educational games. Hoda et al. (2014) involved
teachers as part of a multidisciplinary team that designed a game for supporting
reciprocal teaching and collaboration with children. They evaluated and refined
the game through functional testing, teacher trials, and children–teacher trials.
As an outcome, the game was perceived as engaging and easily understood by
young children. Klonari and Gousiou (2014) described a game for helping teach-
ers to become aware of their pedagogical choices. The game itself was described
in detail, but it remained unclear how the teachers engaged in the design of the
game.
The studies in the Learning and teaching applications category designed tech-
nology for dance and environmental education (Carmichael, 2015), financial ed-
ucation (Cramer and Hayes, 2013), STEM education (Hannon et al., 2012; Su et
al., 2010), mathematics (Pedersen et al., 2012; Triantafyllou and Timcenko, 2013),
and literature (Rahamat et al., 2011). Three studies examined the development
of tutoring systems (Girard and Johnson, 2010; Kalra et al., 2007; Song and Oh,
2016). The studies in this sub-category focused on the technologies themselves.
An exception was the study by Carmichael (2015), which criticised the assump-
tions behind education technology development. That is, it concerned the risks
of designing educational technology based on stereotypical views, such as digital
natives, and losing sight of practice-based knowledge.
All three studies in the Safety and security category related to designing a
situation-aware safety service. Jutila et al. (2015) examined the technological en-
ablers and requirements for building a safety system, Pantsar-Syväniemi et al.
(2015) analysed the design process itself, and Ervasti et al. (2016) analysed the
feedback from children, parents, and teachers. Even though the design clearly fo-
cused on children, these studies were able to bring together various perspectives
from teachers and parents as well.
The largest category was Technology for special needs. The studies considered
various special needs, such as Autism Disorder, language delays, and cognitive
and sensory impairments. Some studies focused on identifying requirements for
technology design (Lingnau and Lenschow, 2010; Zainuddin et al., 2010) or de-
scribing how technology can support these needs (Abdullah and Brereton, 2015;
Fage et al., 2016; Herstad and Holone, 2012; Parsons et al., 2011). Medeiros-Braz
et al. (2017) emphasised that teachers have valuable knowledge about students’
special needs and can envision technologies to support students’ abilities and
 39

learning possibilities. In contrast, Brereton et al. (2015) noted that teachers (and
other adults) have their own needs, and these can be different than the actual
objectives of the students who have special needs.
Finally, two methodological contributions stand out in this category. Bossavit
and Parsons (2016) utilised a stakeholder analysis framework to reflect the design
process of an educational game. The framework was grounded in PD literature
and used to map stakeholder roles, levels of engagement, design tools, and deci-
sions. Parsons and Cobb (2014) addressed the complexity of involving multiple
stakeholders: in this case, teachers and children with special needs. They dis-
cussed how the key challenge is to prioritise different stakeholders and decisions.
They argue that prioritising each stakeholder equally is impossible and question
if an outcome-focused agenda, which aims for efficient technology development,
is even possible to combine with the empowering approach of PD.

4.2 Teachers’ goals and concerns

This section summarises the goals and concerns of the teachers involved in the
two projects.

The open and adaptable school

The main goal of the teachers from Valteri School Onerva was to harness a new
perspective on learning spaces: In the previous school, we are used to certain conven-
tions. They are like unwritten rules that certain spaces are only for certain people and for
certain use. We want to renew the whole culture of using learning spaces. This goal was
further exemplified by a teacher: Crafting spaces are normally only meant for crafting
lessons. However, these spaces are often free, and they could be used for teaching some
mathematical concepts, such as measuring. But how do you communicate these kind of
needs and possibilities during the hectic days?
This new perspective was related to creative use of space, especially in the
context of supporting action-based learning activities. The teachers pointed out
that their student groups are smaller than class sizes in basic education, which
makes it easier for the group to move between spaces. In this sense, the teachers
emphasised that they want to get away from the desks and start moving: What
could be more natural way of learning than going to an environment where you can
learn something by actually doing it? However, the differing levels of students’
perceptional abilities were noted as a challenge to creative use of spaces because
blind students, for example, need to be able to navigate in the building without
assistance.
The open school represented not only open physical spaces, but also the
opening up of the work practices for increased collaboration. In the previous
school, when a certain space was reserved for a certain group, it was difficult for
other groups to ask if they could join and use the same space. The teachers noted
40

that they hoped the new school to inspire them to find better ways to communi-
cate and initiate collaboration: Hopefully it could enhance collaboration if we are able
to show that others are welcome to the space we have reserved.
One concern that the teachers expressed was related to privacy and security
in the new school. They emphasised that even in the open school, it is necessary
to have spaces for private conversations. This includes both audible and visual
privacy, because communicating in sign language is a normal occurrence: Despite
the fact that there are open spaces, there needs to be spaces for private communication.
We have lots of confidential conversations with parents and other stakeholders. In the
current school, if the phone rings and students are present, we may use cleaning closet –
or whatever place – to talk. From the perspective of security, the teachers pointed
out what kind of practical problems working in special education has: Many of
my students are almost or completely deaf. If I give them a task, where they can choose the
place where they work, how am I suppose to reach them in this open school environment.
Even though there are communication devices, organinsing a group of students
is more complicated in the large school than in a single classroom.
The teachers outlined their another main concern with the new school as
follows: When we are acting during a normal hectic day, how are we supposed to know
what spaces are free when we need them? This is the main problem we try to resolve
with technology. Without permanent classrooms and learning activities scheduled
around the school, preventing work from becoming chaotic becomes a difficult
task. Thus, the teachers hoped for: A system that would be available in the new school,
so that it makes possible to get up and go actually doing something. Still, the teachers
stressed that they are committed to the idea of open and adaptable school, and
that the system must not become a barrier for this vision: The system needs to
be fast and easy, otherwise we will not use it. We want to concentrate on our work,
not on managing learning spaces. Where [pedagogical activity] happens, should be a
matter of pedagogical choices, not technological. This demonstrates that the teachers
understood the project to not only focus on developing new technology, but also
on supporting new workplace practices.

Technology Comprehension

Piloting Technology Comprehension as an elective course revealed a crucial is-

sue. The teachers did not have a shared understanding of Technology Compre-
hension as a learning subject, although the subject was already defined in the
national curriculum. Instead, the teachers interpreted the subject through their
personal beliefs, experiences, and interests. Teachers with design backgrounds
emphasised design goals, teachers with computing backgrounds valued com-
puting goals, and teachers with humanistic backgrounds tended societal goals.
Hence, the learning goals of Technology Comprehension became individual skill
sets for integration into other subjects, instead of a combination to form a distinct
subject.

We created a programming and math course, which starts in the first grade and runs
through all grades. Programming is okay, but should not be a standalone subject, it
 41

should be part of the other subjects. A tool.

I think a lot about how it can be part of natural science subjects. Currently, I am also
teaching crafts, where I think that it could fit in. But, as I said, I also think that having
it as a part of natural sciences would be very exciting for me.
I tend to focus on the design part of the subject because that is what I find awesome,
this entrepreneurship, and I try to give tasks like ’Design a logo’, ’Find a company
name’, ’Create a business model’.
This new thing that is starting, I think about it as part of the existing subjects.

The teachers’ main concern was how to engage all students in Technology Com-
prehension. They worried whether the students would possess the required skills
so that the subject could be taught in a meaningful way. If the subject were intro-
duced in the seventh grade, for example, the prerequisites would need to be very
low. Otherwise, any lack of basic skills, such as basic computer use, would pre-
vent the students from focusing on the actual learning activities. Further, teaching
the basic skills during the class would leave little time for other learning objec-
tives. Another concern was that students have different needs regarding structure
and guidance. Some students want to be challenged and to be provided with less
guidance, while others are incapable of acting without clear instructions.
I have some boys in my elective course and even before I started the teaching they had
downloaded the files we should use. At the same time, I had a girl who did not know
what a file is. The students had very different skills for participating in this field.
I would like to be better at presenting the students with a problem as a starting point,
where they can analyse, design and develop. Currently, they have mostly worked with
learning the different technologies.
Some of them expect to be challenged, some of them expect to get everything served
on a silver plate. That is one of the biggest concerns I have to get them to be better.
A lot of students want to participate in 4-6th grade, in 7-9th grade, it is primarily boys.

To engage students in Technology Comprehension, the teachers proposed student-

centred learning activities that focused on topics relevant for the students. For
example, a teacher described how ninth graders developed a sense of ownership
towards the design task. The students’ task was to make math games for first
graders, and the students used their breaks to test the games to assure that the
games were not too difficult.
We were making math games with Scratch, it was obvious that older students had a
sense of ownership to this assignment. The 9th graders were supposed to make a math
game for 1st graders. The day before, 9th graders used their lunch break to go to 1st
graders and check if the level was too hard. Then 9th graders went back and adjusted
the games. That’s very uncommon to 9th graders to do something like that in own
time.
My focus is to have a starting point that the students can relate to, for example, in
the Odense municipality we are establishing the new light-rail. The students were
concerned about what if a blind person should cross the light-rail, can we be sure that
the train will stop. So they tried to build some censors with Micro:bits. This was the
classical problem-solving setting that the students could relate to.

The teachers appreciated that the subject combined computing, design, and so-
cietal reflection. They stated that computing-related curricula are often designed
42

by people with computing backgrounds, and as such learning objectives are often
restricted to computing goals. Technology Comprehension opens up to holistic
learning goals when it is designed by stakeholders with various backgrounds.

4.3 Issues in involving teachers

This section presents the issues that were identified during the two projects.

Identifying roles, needs, rights, and responsibilites

The first issue involved identifying roles, needs, rights, and responsibilities. In
the ONSPACE project, the main roles were teachers, researchers, developers, and
technical staff. The teachers needed to solve the problem that the new school
had: how to organise everyday life in the new school without any dedicated
classrooms. The researchers’ need was to study the project and obtain interna-
tional funding. The developers needed to deliver a working system that met the
defined requirements, whereas the technical staff needed to make sure that the
security of the system could not be compromised. Issue arose when these needs
conflicted. First, ignoring the needs of administrative staff resulted in serious de-
ficiencies when putting the system into production. This was solved by assigning
a freelance software developer to implement the missing features. Second, after
the design stage, not enough attention was paid to deciding what features would
be realistic to develop. This resulted in expectations that were too optimistic for
the resources available for development.
In the Technology Comprehension project, the main stakeholder roles were
teachers, project staff, and researchers. In this case, the teachers did not have a
single common need. Some were enthusiastic about developing the subject, some
hoped to learn about the subject topics (e.g. programming), and some called for
support to organise the elective course. The project staff needed to deliver the
outcomes that the Ministry of Education had defined. When regarding the re-
searcher role, the author’s need was to conduct research for the dissertation. In
the end, there appeared a contradiction between the author’s need to conduct
research and the project staff’s need to minimise pressure on the teachers. The
author was supposed to interview the teachers for the dissertation, but the teach-
ers signaled that the interviews would take more time than they had available for
the project. Thus, it was agreed to leave the interviews out, so that the teachers
did not become overburdened.
In addition to identifying the roles and needs, it is crucial to negotiate the
rights and responsibilities of project outcomes, such as produced technologies
or data. There was a misunderstandment regarding the system maintenance re-
sponsibility in the ONSPACE project. The technical staff assumed that they were
to install the system, and the faculty members were responsible for fixing pos-
sible errors and updates. However, transferring the system rights meant that
 43

faculty were no longer responsible for the system. In the end, the problem was
solved when the developers agreed to remain available for consultation should
problems appear when deploying the system in the school.

Participation as a possibility instead of an obligation

This dissertation focuses on teachers, so participation was examined from the

teacher perspective. A teacher’s core work includes lesson planning, teaching,
and assessing. In addition, teachers have a lot of out-of-classroom work: solv-
ing disputes between students, communicating with parents, and organising ac-
tivities such as trips and sports events. Teachers are also required to take part
in school boards, parent evenings, multidisciplinary expert groups, and school
welfare groups. Thus, any additional project may appear as something that just
takes time and attention away from the most essential – being with students. PD
requires a lot of time and resources, such as being present in design activities
and familiarising with new people, technologies, and concepts. It is important to
make sure that the PD initiative does not exploit the responsibility and account-
ability often observed in teachers.
The ONSPACE project was justified for teachers by emphasising that the
system would be owned by the school and not by the university or some other
party. Moreover, the project sought to develop a system based on teachers’ needs
instead of asking teachers to adapt their work practices. However, how the teach-
ers would benefit from participating in the project was not considered directly; it
was assumed that developing the system would automatically lead to outcomes
that benefit teachers.
The Technology Comprehension project had a two-fold goal: to examine
the implementation of the new subject and to envision support for teachers. The
practical implication was that the project activities were configured from a premise
that the teachers should receive personal benefits from participating in the project.
During the interviews and the school visits, the teachers expressed that they
would benefit from professional training about the learning objectives of Technol-
ogy Comprehension. The workshop programme was thus added to include lec-
tures about design, computational thinking, and hands-on tasks with Micro:bits.
This allowed the teachers to get something useful out of providing information.

Clarifying an agenda for sustainable outcomes

The final issue to overcome is to construct compelling arguments as to why it is

necessary to allocate resources for PD. If the agenda is left vague, the feasibility
of the project is easily questioned. For example, one might ask why the learning
space management system was not simply ordered from a professional software
company, or why the best practices for Technology Comprehension are not de-
veloped in teacher education departments?
In the ONSPACE project, the key problem was to anticipate how to organise
work practices in the new school without traditional classrooms. Developing the
44

system was not so much a necessity itself, but rather the most promising way to
solve the problem. In the Technology Comprehension project, the agenda was
to examine the implementation of the new subject. This included understanding
teachers’ work conditions, challenges, and capabilities.
Despite the fact that both of the two projects put significant effort on sus-
tainability, they ended up in opposite trajectories. The ONSPACE project aimed
for international scope and applied for Horizon2020 Research and Innovation Ac-
tion funding. The project was showcased in the Horizon2020 Proposers Day and
received partnership proposals from several universities, companies, and non-
governmental organisations. The established international initiative included
three universities, two special education schools (including the Valteri School On-
erva), and four companies. However, after the funding request from Horizon2020
was rejected both in 2014 and 2015, the international initiative was closed down.
The Technology Comprehension project has evolved in that the new subject
is being integrated as a mandatory programme for lower secondary education
in Denmark. The learning objectives comprise four competence areas: digital
design and design processes, computational thinking, digital empowerment, and
technological abilities. Between 2019 and 2022, 46 Danish schools will experiment
with the programme either by integrating it into existing subjects or establishing
it as an independent subject 1 ; and these experiments will be further investigated
by the project.
These opposite trajectories occurred for two reasons. First, the ONSPACE
project was not able to communicate the value of its agenda. Even though the
focus of the project was on what the teachers want to achieve, instead of what the sys-
tem should do, it appeared too much as a standard software development project.
Because the research was published in technically-oriented venues, the contem-
plated pedagogical possibilities of the new spaces, and how to dissemenate good
pedagogical practices, were left in the background. The value of the project was
in pedagogical deliberations with the teachers, but the communication pointed
to non-significant technology. In comparison, the Technology Comprehension
project was able to justify its existence by asserting that all schools are different,
so there exists no one-size-fits-all solution. Rather, it is necessary to investigate
the subject in various contexts.
Second, the ONSPACE project offered no opportunity for potential com-
munities to emerge. Configuring the project according to open-source principles
could have attracted other potential stakeholders, such as other schools, research
organisations, or software companies. When the system was developed with the
resources of a single organisation, there was a much higher threshold for mak-
ing the system publicly available. To the author’s knowledge, the learning space
system has been further developed within Valteri School Onerva, but without
collaboration with other stakeholders.

1 The Danish Learning Portal, accessed 15.3.2019

5 DISCUSSION AND CONCLUSION

Section 5.1 summarises the findings and connects them to previous PD research.
Section 5.2 concludes the dissertation by considering PD as an approach for in-
volving teachers as design partners. Section 5.3 examines the strengths and limi-
tations of the dissertation. Section 5.4 reviews the dissertation in relation to ACM
Ethical Codes, the Finnish National Board on Research Integrity, and responsible
conduct within PD. Section 5.5 specifies the author’s contributions in the individ-
ual studies.

5.1 Discussion of results

The three research questions were: For what purpose have studies in PD exam-
ined teachers? How are teachers’ goals and concerns manifested through PD?
What issues can be observed when involving teachers in PD?

RQ1: Current state of research in participatory design involving teachers

Systematic mapping produced an overview of the current research on partici-

patory design involving teachers. The mapped studies were assigned into three
categories: environments, practices, and technologies. The studies in the environ-
ments category involved teachers in designing physical buildings, as well as tech-
nologies integrated into the environment. The studies in the practices category
considered professional communities, instructional planning, and professional
development programmes. The studies in the technologies category designed
assessment and monitoring tools, educational games, learning and teaching ap-
plications, and technology for special needs.
Previous mapping and systematic review studies about PD exhibit a nar-
rower focus. Halskov and Hansen (2015, p.83) limited the scope to the Proceed-
ings of the Participatory Design Conference between 1990 and 2012, and Nunes et al.
(2016, p.408) limited theirs to the IEEE database and ACM Digital Library. Hence,
46

PI is the first known attempt to systematically map the whole PD field and organ-
ise the studies into a thematic structure. Furthermore, the study contributed to
the use of computational methods for literature mapping and demonstrated how
these methods could assist researchers. PII revealed that many PD studies have
been published in venues other than in Human–Computer Interaction journals
and conferences. Thus, the mapping provides a basis for multidisciplinary re-
search that accounts for the research corpus both in learning sciences and PD, as
recommended by DiSalvo et al. (2017).
There are several opportunities for continuing the work in PI and PII. The
clusters in PI were analysed based on descriptive information, such as the most
frequent words. A more in-depth examination of the articles within the clusters
would produce a better view of contemporary PD research. Furthermore, the col-
lected references can be disseminated for the PD research community by devel-
oping a web interface based on Mendeley API. As for PII, analyses of the studies
focused on mapping the research objectives, and the methods used in each study
were tabulated. However, these data have yet to be synthesised and reported.

RQ2: Developing a pedagogical vision and communicating between political

and local levels

Teachers’ goals and concerns regarding the open and adaptable school and Tech-
nology Comprehension can be expressed according to two themes: PD for de-
veloping a pedagogical vision, and PD for communicating between political and
local levels. A central notion in PD is that design can either support old practices
or aim to change them (Ehn, 1988). The motivation in PD is to involve partic-
ipants in envisioning alternative choices and influencing how these choices are
pursued (Bødker, 2003; Iversen et al., 2012; Simonsen and Robertson, 2013; Bød-
ker and Kyng, 2018). The teachers were involved in designing the learning space
system before the new school was put into use. Hence, the new building repre-
sented an opportunity for change, and the teachers were enthusiastic about the
possibilities that the school could provide. Moving to the new school was also a
chance to identify challenges in the work practices of the old school and engage
in negotiations to address these challenges.
When the teachers engaged in designing the system, they attempted to de-
velop a pedagogical vision for the open and adaptable school. This vision was
manifested by negotatiating the future practices for the new school. The vision
included, for example, the goal of rejecting the convention of viewing certain
spaces only for certain purposes, such as viewing sports facilities only for teach-
ing sports. When the teachers negotiatied with other stakeholders, such as the
researchers and developers, the vision served as a guidepost: conflicting propo-
sitions, opinions, and decisions were evaluated from the premises of the vision.
This finding corroborates the works of Iversen et al. (2012), who outlined PD as a
trajectory where participants emerge, develop, and ground their values.
The teachers’ goals and concerns in the Technology Comprehension project
point to a crucial issue regarding implementing curriculum reform in Denmark.
 47

The policy-level vision in the curriculum contradicted that of the local teachers.
Although the three learning objectives were combined as a single subject, the
teachers based their visions for implementation on individual preferences, em-
phasising the objectives that related to their own backgrounds. Another concern
was how to engage all students, with different backgrounds, skills, and needs,
in Technology Comprehension. This concern related to the political decision as
to in what grade the subject should be introduced. The Technology Compre-
hension project exemplifies an important aspect of participatory infrastructuring:
the project served as a communication channel between political and local lev-
els (Bødker et al., 2017; Smith and Iversen, 2018). By engaging with teachers,
the project provided knowledge for politicians about the teachers’ goals and con-
cerns and, on the other hand, disseminated guidance and training about the new
subject for the teachers.

RQ3: Building blocks of involving teachers as design partners

Analysing the issues in the two projects produced three themes: identifying roles,
needs, rights, and responsibilies; participation as a possibility instead of an obli-
gation; and setting an agenda for sustainable outcomes. These themes are com-
mon in PD literature, as can be seen in Chapter 2. The contribution here is that
this discussion is contextualised within education, and specifically in terms of
collaboration with teachers. Here, these themes are connected to the mapped
studies to provide building blocks of involving teachers as design partners.
Identifying roles, needs, rights, and responsibilities is essential for PD (Brat-
teteig and Wagner, 2012; Iversen and Dindler, 2014; Barcellini et al., 2015). The
ONSPACE project demonstrated that the more roles in the project, the larger the
potential for conflicting needs, rights, or responsibilities. Although conflicts do
not necessarily appear, anticipating this possibility prevents ending up in a sit-
uation where the stakeholders steer the project into opposite directions. The
systematic mapping provides some valuable assets: visual tools for triangulat-
ing participants’ different perspectives (Woolner et al., 2010), stakeholder analy-
sis models (Könings et al., 2017), a tool for mapping and sharing stakeholders’
goals (Janssen et al., 2017), and the stakeholder analysis framework (Bossavit and
Parsons, 2016). When regarding the involvement of teachers and students, some
studies recommended involving them together, while others proposed separating
them. Casanova and Mitchell (2017) stated that dividing students and teachers
into separate groups offers a more pleasant environment for both. Woolner et al.
(2007) warned that the teachers perspective may be pushed to the background if
students and teachers are in the same group. This would be an interesting point
to explore further when investigating PD in an educational context.
Positioning participation as a possibility instead of an obligation involves
anticipating the benefits of PD projects from the participants’ perspectives (Vines
et al., 2013; Frauenberger et al., 2015; Halskov and Hansen, 2015; Smith et al.,
2017; Saad-Sulonen et al., 2018). Based on the literature mapping, the advantages
of PD from the teacher perspective include: for example, opportunity to reflect
48

teaching methods (Bossen et al., 2010), development of teacher ownership (Kyza

and Georgiou, 2014), conditions that support teachers (Cober et al., 2015), and
teacher’s professional development (Kyza and Nicolaidou, 2017). Furthermore,
some studies proposed PD as a way to deal with contradictions between teach-
ers’ pedagogical needs and other stakeholders’ needs (van Merriënboer et al.,
2017; Woolner et al., 2007) and to reduce discrepancies between teachers and stu-
dents (Könings et al., 2007, 2010, 2011). Most of the mapped studies relied on the
assumption that PD would indirectly benefit teachers, such as through building a
better learning environment or developing better practices and technologies. Re-
liance on indirect benefits was observed in the ONSPACE project, whereas direct
benefits for teachers were built into the Technology Comprehension project.
Sustaining outcomes is a central topic in contemporary PD research (Iversen
and Dindler, 2014; Vines et al., 2015; Bødker et al., 2017; Bødker and Kyng, 2018;
Saad-Sulonen et al., 2018; Smith and Iversen, 2018). In the ONSPACE project,
the agenda became blurred because the pedagogical deliberations fell behind the
technological interests, whereas the Technology Comprehension initiative was
able to justify its importance. The mapping revealed examples of a clear agenda,
such as promoting social mobility and greater professionalism in public schools
(Bossen et al., 2010), facilitating teachers’ professional development and social
networking (Farooq et al., 2007), and developing teachers’ ownership towards ed-
ucational reforms (Kyza and Nicolaidou, 2017). The other issue in the ONSPACE
project was that access to outcomes was closed, which did not allow communi-
ties to emerge. In the literature mapping, the studies that had long-lasting and
wide impact were grounded in research collaboration (Könings et al., 2007, 2010,
2011; Cober et al., 2015; Burke and Könings, 2016; Könings et al., 2017; Janssen et
al., 2017; Koutamanis et al., 2017, see), technological infrastructure (nine years of
research about Tapped In Farooq et al., 2007), or pedagogical ideas (see De Werk-
plaats Burke and Könings, 2016; Koutamanis et al., 2017; van Merriënboer et al.,
2017).

5.2 Conclusion

This dissertation examined PD as an approach for involving teachers as design

partners. PD has become a well-known approach, especially when involving
users in designing new technologies. PD is not a strictly defined framework,
or a collection of methods, but rather a way of designing that incorporates own
principles and values established in the PD community. The purpose of involving
users is not only to inform design, but to give users agency and influence on what
kind of products, services, and practices are designed. As such, users should have
equal possibilities to take part in decision-making, among researchers, designers,
developers, and other stakeholders.
PD was examined by producing an overview of the current state of research
into studies involving teachers, and by taking part in two projects. The findings
 49

of these three parts are summarised in Figure 6.

Environments Technologies Practices

School buildings Assessment and monitoring tools Professional communities
Technology-enhanced Educational games Instructional design
learning spaces Learning and teaching apps Professional development
Safety systems
Technology for special needs

ONSPACE Technology Comprehension

Developing a pedagogical vision and Participatory design as a communication
establishing a shared commitment to channel between educational policy and
pursue this vision. local implementation.

Building blocks of participatory design with teachers

1. Identifying roles, needs, rights, and responsibilities
2. Positioning participation as a possibility instead of an obligation
3. Clarifying an agenda for sustainable outcomes

FIGURE 6 Interrelations between dissertation findings.

Systematic mapping produced three main areas in which PD advocates teacher

involvement: designing learning environments, practices, and technologies. Taken
together, these categories relate to two themes how technology pervades educa-
tion. First, learning environments are expanding from classrooms to technology-
enhanced spaces that are modifiable, flexible, and merge formal and informal
learning (Kuuskorpi and Gonzalez, 2014). Second, education needs to provide
skills not only to use technology but also understand how it works so that stu-
dents develop the capacity to act as democratic, responsible, and critical indi-
viduals (Iversen et al., 2018). The implications of these two topics appear in the
educational policies of the Nordic countries: governments are investing in new
kinds of school facilities and implementing curriculum reforms that oblige teach-
ers to incorporate computing subjects (Berge, 2017).
Involving teachers in the design of the learning space system for the open
and adaptable school demonstrated how PD supports in developing a pedagog-
ical vision and a shared commitment to pursue this vision (van Merriënboer et
al., 2017, p. 266). Involving the whole school community in anticipating what
changes the new environment might bring, and negotiating how the working
and learning practices should be organised, requires a common ground where
different needs are put into perspective. Of course, not all needs can be met, and
developing the vision aids communication between stakeholders so that different
options can be evaluated. The vision also prevents the introduction of technology
that would become a barrier for the intended change. The technology must aid
the objective of the project and operate well in the context for which it was devel-
oped. The project vision can be used to assess whether the technology supports
or restricts the goals set for the new school.
50

The Technology Comprehension project demonstrated how PD served as a

communication channel between local teachers and politicians. This relates to the
three implementation strategies for educational reforms, presented by Pietarinen
et al. (2017, p.25). In the top-down approach, school and teachers are considered
as implementors of policy level decisions, while the bottom-up approach relies
on the capacity of schools and teacher communities to develop these practices.
The Technology Comprehension project examplified the interactive strategy: it
brought together policy-level guidelines, resources, and networks with a local-
level capacity to develop, evaluate, and provide knowledge. The new subject
aimed to integrate computing and design skills as a means for understanding
technology rather than learning outcomes. Engaging with teachers to pilot the
subject demonstrated how the project prioritised their goals and concerns and
provided information and training to best equip the teachers for success.
This dissertation proposes three building blocks of involving teachers as de-
sign partners based on the issues observed in the two projects and previous PD
research with teachers. The first building block – identifying stakeholder roles,
needs, rights, and responsibilities – aims to make transparent the reasons behind
different decisions and to anticipate possible conflicts between different stake-
holders. The second block aims to establish engagement by anticipating possible
benefits from teacher’s perspective and defining beforehand how these benefits
are evaluated. The third block aims to clarify an agenda for sustainable outcomes,
so that potential communities can emerge and continue the efforts of a PD project.
There are no silver bullets for how to involve teachers in design. That
taken, PD still seems to provide an encouraging framework for conducting de-
sign projects in educational environments. For example, when schools are intro-
ducing new innovations, such as applications, devices, and services, a PD ap-
proach could be used to ensure that such innovations are designed to take teach-
ers’ working environment into account. Involving teachers in decision-making
could prevent unwanted situations, in which innovations and teacher’s every-
day needs contradict and teachers need to change their practices to utilise these
new innovations.

5.3 Quality and limitations

The quality assessment criteria for the literature mapping findings (RQ1) differ
substantially from those of the two projects (RQ2 and RQ3). Literature reviews
and mapping differ in purpose: mapping aims to gather a representative set of
studies using inclusion and exlusion criteria, and then tabulates these studies
into specific categories (see Kitchenham et al., 2010, p. 793 and Kitchenham and
Charters, 2007, p. 44). From this angle, the quality of the study can be assessed
based on how expansive and rigourous the mapping protocol is. For example,
data collection in PI and PII should yield a similar collection of studies when
executing the same search parameters and inclusion/exclusion criteria.
 51

Both in PI and PII, the literature was collected from 14 academic search en-
gines and databases, which is a very wide scope for a single mapping study. The
data collection procedure can be repeated, which increases the reliability of the
study. If data were to be collected now (2019), some studies would appear that
are not in the original data. This is because new studies have been published
since data collection for the present study was concluded in 2018. The exclusion
criteria for PII have been explained in full and the reasons for exclusion recorded
for each study. This makes external validation of the mapping possible.
However, when regarding the reliability of the results in PI and PII, it needs
to be noted that producing the categories is not deterministic. In the former,
the number of clusters in each clustering round was decided by the researcher,
even though the Clustering Validation Indice recommends a set amount. The re-
searcher also interpreted the cluster content, even though the algorithm provided
information such as the most frequent words in a cluster. In the latter mapping,
some of the studies could have been assigned to any of the three main categories.
For example, So et al. (2009) examined teachers’ professional development under
the category of practices. This study could easily have incorporated the other two
categories as well, as it involved online environments and the development of an
online video platform.
Assessing the quality of the findings from the two projects is more compli-
cated. These findings are strongly connected to the context in which the research
was conducted (Spinuzzi, 2005; Frauenberger et al., 2015). Frauenberger et al.
(2015) discuss the accountability and rigour of such results within the PD frame-
work. Accountability is the transparency of expressing how PD led to certain
outcomes: what happened and how it ended up. Rigour is the internal validi-
tity of the statements derived through reflection and debate. Assessing trust-
worthiness is based on credibility, dependability, transferability, conformability
(Frauenberger et al., 2015). Credibility refers to a participant’s internal accep-
tance of outcomes, as well as the external believability of the outcomes based on
provided evidence. Transefarability and dependability imply that evaluating the
applicability of findings requires an understanding of how the findings depend
on context, whereas conformability asks whether the provided evidence can be
confirmed.
Findings related to RQ2 stem from interpretations of the teachers’ interac-
tions during the two projects. As such, the presented goals and concerns repre-
sent a consensus rather than unique opinions, the latter of which could have been
better examined by surveying each teacher individually. The findings exemplify
the knowledge construction in research through design (Zimmerman et al., 2007,
2010). The goals and concerns emerged when the teachers engaged in deliber-
ating problems that were relevant to themselves and in their current context (cf.
dialogic space in Iversen and Dindler, 2013; Bannon et al., 2018). These goals
and concerns have been internally validated, as the other researchers in the two
projects participated in analysing and confirming the findings. In comparison,
the identified issues for the third research question emerged from the author’s
personal experiences. While events like the rejection of Horizon2020 funding can
52

be confirmed, the credibility of the offered reasons and evidence must be left for
others to evaluate.
There are three main limitations. First, because of the practical reasons re-
lated to funding, moving from one project to another limited the possibility for
long-term research. Second, narrowing the scope to teachers kept the disserta-
tion focused, but was artificial at times: there are no teachers without students.
Neither of the projects involved students, as this could have been problematic
with the chosen scope. Third, defining what is PD and is not can be somewhat
ambigious: for some, each workshop where stakeholders take part in design is
PD, and for others, only projects with a political mission genuinely represent PD.

5.4 Ethical considerations

The ethical aspects of the dissertation are reviewed here in relation to the guide-
lines for responsible conduct of research by the Finnish National Board on Re-
search Integrity (2018, p. 30–31), Code of Ethics and Professional Conduct of the
Association for Computing Machinery 1 , and the ethical considerations within
the PD research community. When regarding the published articles, all authors
contributed in conducting and reporting research. For every publication, each
author signed the agreement for publishing, and that they are accountable for all
aspects of the work. Those who assisted in conducting research, as well as the
study participants, were acknowledged in each publication.
The ethical aspects of the literature mapping relate most notably to the
data collection and analysis. The search parameters, used search engines and
databases, and data refinement protocols in PI and PII are desrcibed in detail. The
collected references and articles were stored in a private database. Furthermore,
the applied software tools and program code were described in both publications,
and data is available on request.
In the ONSPACE project, the data management plan and consent docu-
ments were made according to the guidelines of the University of Jyväskylä.
The data management plan informed what data was collected from the project
into a register, who was responsible for the register, how data in the register was
anonymised, where the register was located, and how the register was secured.
Consent to conduct research was collected from each participant. The consent
documents provided information about the research organisation, the purpose
of the study, methods, possible benefits and disadvantages for participants, par-
ticipants’ rights, and asked participants to register their agreement. The project
funding was described in the publication acknowledgements. In the Technol-
ogy Comprehension project, data about study participants was anonymised and
stored securely by the project staff. Each participant formally agreed to the re-
search, and the participants were provided with information about the research
purposes, methods, and participants’ rights.
1 Association for Computing Machinery, accessed 17.3.2019
 53

The development of the learning space system in the ONSPACE project fol-
lowed recommended practices in software engineering. The use of open-source
libraries is documented in the system specifications. The development process
was documented in a private version control system. The system was tested in
a private server, and no data about the project participants was used in the test-
ing stage. The system rights transfer agreement was made between stakeholder
organisations. When the system rights were transfered, all development material
was provided to the owner.
Part of pursuing the Doctor of Philosophy degree is to learn the academic
conventions for responsible conduct of research, such as abstaining from fabrica-
tion, misrepresentation, plagiarism, or misappropriation. Furthermore, responsi-
ble conducting in PD requires accountability of design and research for local com-
munities and pursuing professional growth (Robertson and Wagner, 2012; Steen,
2013). The author endeavoured to engage in two projects, become acquainted
with PD literature, and network with other senior and junior researchers in the
PD field. Valuable learning points included the doctoral course Values-led Par-
ticipatory Design with Children (2015), the doctoral colloquium in Participatory
Design Conference (2016), the doctoral course Designing Human Technologies
(2017), and the annual summer meeting of Participatory Information Technology
research group (2017).

5.5 Author’s contributions

The following contributions have been reviewed based on the recommendations

for authorship agreement in the (Finnish National Board on Research Integrity,
2018) and discussed in relation to research conception, data collection, and data
analysis. The author was the corresponding author in each publication and had
the most substantial role in drafting the research report.

PI: The author developed the idea and the scope for the mapping. The author
executed the literature collection. The pre-processor was developed by Paavo
Nieminen, which the author modified to work with the collected data. The clus-
tering method was developed and executed by Tommi Kärkkäinen (Section 2.3 in
PI). The author produced the cluster map and analysed the six education-related
clusters. The author presented the article at the Participatory Design Conference.

PII: The study was conducted alone by the author.

PIII: The principal investigator of the project was Hannakaisa Isomäki. The au-
thor had no part in establishing the project or in developing the idea of utilising
value-focused thinking as a requirement elicitation method. In the design stage,
the author was involved in orchestrating the design activities and produced the
system requirements specification. In the development stage, the author devel-
54

oped the system together with a project researcher, Toni Taipalus, and organised
the design meetings. The author was responsible for data collection, except in
the first meeting (Table 2 in PIII), and analysed data (Section 5.4 in PIII). The
author presented the article at the International Conference of Computer Sup-
ported Education and was responsible for revising and extending the article for
the Springer book Communications in Computer and Information Science.

PIV: The principal investigator of the project was Ole Sejer Iversen. The research
design was based on the goals that were set for the project. The first survey and
the interviews were conducted by the project researcher, Marie-Louise Wagner,
as they were in Danish. The author was involved in some of the school visits and
class observations. The workshops were facilitated by the project researchers, and
the author recorded the workshops. The data collection and analysis regarding
the self-assessment and feedback questionnaire were made by the author. The
author designed the protocol for analysing the workshop recordings and was in-
volved in the analysis with the other project members. The author presented the
article at the Conference on Creativity and Making in Education – FabLearn Eu-
rope.

PV: When the article was revised for the special issue of the International Journal
of Child-Computer Interaction, the author’s responsibility was to extend the litera-
ture review (Section 2 in PV) together with Tommi Kärkkäinen. The additional
findings (Section 5.3 in PV) were produced by Ole Sejer Iversen and Marie-Louise
Wagner.
 55

YHTEENVETO (FINNISH SUMMARY)

Väitöskirjassa tutkittiin osallistavaa suunnittelua opettajien kanssa tehtävän ke-

hittämistyön lähestymistapana. Osallistavasta suunnittelusta on tullut suosittu
lähestymistapa käyttäjien mukaan ottamiseksi erilaisten tuotteiden ja palvelui-
den suunnittelussa. Osallistava suunnittelu ei kuitenkaan ole tiukasti määritelty
viitekehys, joka sisältäisi ennakkoon määrättyjä menetelmiä, vaan pikemminkin
tutkimusyhteisössä muodostunut tapa tehdä suunnittelua ja kehittämistä omi-
ne periaatteineen ja arvopohjineen. Käyttäjien osallistamisen tarkoituksena ei ole
ainoastaan saada tietoa käyttäjistä kehittämistyötä varten, vaan antaa käyttäjille
mahdollisuus vaikuttaa siihen, millaisia tuotteita, palveluita ja käytäntöjä toteu-
tetaan: käyttäjillä tulisikin olla tasavertaiset mahdollisuudet osallistua päätöksen-
tekoon tutkijoiden, suunnittelujoiden ja sovelluskehittäjien kanssa. Selvimmiten
tämä periaate on näkynyt tutkimuksissa, joissa vähemmistöedustajia, kuten maa-
hanmuuttajanuoria, on otettu mukaan tekemään päätöksiä heille suunniteltavis-
ta tuotteista ja palveluista.
Väitöskirja sisältää kolme osaa. Ensimmäinen osa on systemaattinen kirjal-
lisuuskartoitus opettajia koskevista osallistavan suunnittelun tutkimuksista vii-
meisen kymmenen vuoden ajalta. Neljästätoista eri hakukoneesta kerätyt tutki-
mukset analysoitiin ja sijoitettiin kolmeen eri kategoriaan: oppimisympäristöt,
käytännöt ja teknologiat. Oppimisympäristöt sisälsivät kaksi osa-aluetta: kou-
lurakennusten suunnittelu ja oppimisympäristöön integroitavien teknologioiden
kehittäminen. Käytännöt liittyivät ammatillisten yhteisöjen muodostamiseen se-
kä opetusmenetelmien ja ammatillisten koulutusohjelmien suunnitteluun. Kol-
mannen kategorian teknologiat olivat arviointityökaluja, oppimispelejä, oppimis-
ja opetusohjelmia, turvallisuusteknologioita sekä työkaluja erityistarpeita varten.
Toinen osa on Jyväskylän yliopiston informaatioteknologian tiedekunnan
ja oppimis- ja ohjauskeskus Valteri Onervan kanssa yhteistyössä toteutettu kehit-
tämishanke ONSPACE. Hankkeessa toteutettiin Valteri-koulu Onervan vuonna
2016 valmistuneeseen uuteen toimipisteeseen oppimistilojen hallintaan tarkoitet-
tu mobiilisovellus. Sovelluksen tarkoituksena on tukea koulun Oivallus – avau-
tuva oppimistila -tilakonseptia ja mahdollistaa oppimistilojen varaaminen kart-
tapohjaisella visuaalisella käyttöliittymällä. Hankkeen aikana järjestettiin ennen
uuden koulun käyttöön ottoa kahdeksan osallistavan suunnittelun työpajaa, jois-
sa opettajat pohtivat pedagogisia tarpeitaan ja miten niihin voitaisiin vastata ke-
hitettävällä teknologialla. Keskustelujen analysoinnin perusteella opettajat eivät
ainoastaan osallistuneet sovelluksen vaatimusten määrittelyyn, vaan myös neu-
vottelivat koulun uudesta toimintakulttuurista ja tilojen käyttämisen periaatteis-
ta. Osallistavan suunnittelun työpajat olivatkin opettajille mahdollisuus neuvo-
tella yhteisestä pedagogisesta visiosta sekä ennakoida uuden koulun mahdolli-
suuksia ja ongelmia.
Kolmas osa toteutettiin Aarhusin yliopistossa Tanskassa, vuoden mittaisen
tutkimusvaihdon aikana. Teknologiaymmärrys -nimisessä kehittämishankkeessa ko-
keiltiin uutta valinnaista oppiainetta kolmessatoista tanskalaisessa yläkoulussa.
56

Tanskan opetusministeriön määrittelemän uuden oppiaineen tavoitteena on op-

pia suunnittelemaan digitaalisia tuotteita, toteuttamaan niitä ohjelmoimalla ja ar-
vioimaan niiden yhteiskunnallisia vaikutuksia. Hankkeessa suoritettiin koulu-
vierailuja, oppiaineelle nimettyjen opettajien haastatteluja ja työpajoja. Työpajat
sisälsivät täydennyskoulutusta opettajille sekä uuden oppiaineen toteuttamiseen
liittyviä teemakeskusteluja. Hankkeen analysoinnin perusteella osallistava suun-
nittelu toimi viestintäkanavana opettajien ja poliittisten päättäjien välillä: opetta-
jat välittivät huolenaiheitaan ja toiveitaan päätöksentekijöille ja toisaalta opettajat
saivat tietoa ja koulutusta poliittisella tasolla määritellyistä vaatimuksista.
Kuviossa 7 väitöskirjan tulosten perusteella ehdotetaan kolmea toimenpi-
dettä opettajien osallistamiseksi.

Oppimisympäristöt Teknologiat Käytännöt

Koulurakennukset Arviointityökalut Ammatillisten yhteisöjen
Oppimisympäristöön Oppimispelit muodostaminen
integroitavat teknologiat Oppimis- ja opetusohjelmat Opetusmenetelmät
Turvallisuusteknologia Ammatilliset koulutus-
Työkalut erityistarpeisiin ohjelmat

ONSPACE Teknologiaymmärrys
Yhteisen pedagogisen vision Osallistava suunnittelu viestintäkanavana
muodostaminen toimintakulttuurista ja paikallisten opettajien ja poliittisten
tilojen käytön periaatteista neuvottelemalla päätöksentekijöiden välillä

Toimenpiteet opettajien osallistamiseksi

1. Erilaisten roolien, tarpeiden, oikeuksien ja velvollisuuksien tunnistaminen
2. Osallistumisen perustelu mahdollisuutena velvollisuuden sijaan
3. Tavoitteiden täsmentäminen kestävien tulosten aikaansaamiseksi

FIGURE 7 Conclusion in Finnish.

Ensiksikin, tulisi tunnistaa osallistujien erilaiset roolit, tarpeet, oikeudet ja

velvollisuudet. Mitä useampia rooleja hankkeessa on sitä todennäköisempää että
osallistujien tarpeet, oikeudet ja velvollisuudet ajautuvat keskenään ristiriitaan.
Tämän ennakoiminen voi ehkäistä tilanteita, joissa eri osallistujaroolit ohjaavat
hanketta vastakkaisiin suuntiin. Väitöskirjan kirjallisuuskartoituksella löydettiin-
kin erilaisia työkaluja näiden tekijöiden tunnistamiseksi. Toiseksi, hankkeeseen
osallistuminen tulisi voida perustella opettajille mahdollisuutena velvollisuuden
sijaan. Opettajien työnkuvaan kuuluu ennestäänkin paljon osallistumista, esimer-
kiksi oppilashuoltoryhmiin, vanhempainiltoihin ja luokkaretkien suunnitteluun.
Kehittämishankkeeen ei pitäisi olla opettajalle vain uusi velvollisuus, vaan opet-
tajille tulisi perustella kuinka he voivat hyötyä osallistumisestaan. Tähän voidaan
pyrkiä kutsumalla opettajat ennakoimaan hankkeen mahdollisia hyötyjä ja miten
hyödyt arvioidaan hankkeen aikana. Kolmanneksi, kehittämishankkeen tavoit-
teet tulisi täsmentää kestävien tulosten aikaansaamiseksi. Käytännössä tämä tar-
koittaa sitä, että hankkeen tuloksena syntyy yhteisöjä, joissa hankkeen tavoitteita
jatketaan ja joihin voi osallistua uusia toimijoita.
 57

Opettajien osallistamiseen ei ole olemassa yhtä ainoaa oikeaa ratkaisua. Väi-

töskirjassa tutkittu osallistava suunnittelu näyttäisi kuitenkin tarjoavan mahdol-
lisen viitekehyksen kouluympäristössä tehtäville kehittämishankkeille. Kun kou-
luissa otetaan käyttöön uusia innovaatioita, kuten sovelluksia, laitteita ja palve-
luja, osallistavan suunnittelun lähtökohtana olisi varmistaa, että nämä innovaa-
tiot suunnitellaan yhteensopiviksi opettajien työskentely-ympäristöön. Ottamal-
la opettajat mukaan päätöksentekijöiksi voidaan ehkäistä tilanteita, joissa uudet
innovaatiot ja opettajan arjen tarpeet eivät kohtaa, jolloin opettajat joutuvat mu-
kauttamaan työtapojaan innovaatioiden käyttämiseksi.
58

REFERENCES

Abdullah, M. H. L. & Brereton, M. 2015. MyCalendar: Fostering Communication

for Children with Autism Spectrum Disorder Through Photos and Videos. In
Proceedings of the Annual Meeting of the Australian Special Interest Group
for Computer Human Interaction on - OzCHI ’15. New York, New York, USA:
ACM Press, 1–9. doi:10.1145/2838739.2838785.

Admiraal, W., Louws, M., Lockhorst, D., Paas, T., Buynsters, M., Cviko, A.,
Janssen, C., de Jonge, M., Nouwens, S., Post, L., van der Ven, F. & Kester,
L. 2017. Teachers in school-based technology innovations: A typology of
their beliefs on teaching and technology. Computers & Education 114, 57–68.
doi:10.1016/j.compedu.2017.06.013.

Al-Eraky, M. M., Donkers, J., Wajid, G. & van Merriënboer, J. J. 2015. Faculty
development for learning and teaching of medical professionalism. Medical
Teacher 37 (sup1), S40–S46. doi:10.3109/0142159X.2015.1006604.

Aldunate, R. & Nussbaum, M. 2013. Teacher adoption of technology. Computers

in Human Behavior 29 (3), 519–524. doi:10.1016/j.chb.2012.10.017.

Andersen, L. B., Danholt, P., Halskov, K., Hansen, N. B. & Lauritsen, P. 2015.
Participation as a matter of concern in participatory design. CoDesign 11 (3-4),
250–261. doi:10.1080/15710882.2015.1081246.

Anderson, L. & Östlund, D. 2017. Assessments for learning in grades 1-9 in a

special school for students with intellectual disability in Sweden. Problems of
Education in the 21st Century 75 (6), 508–524.

Bannon, L., Bardzell, J. & Bødker, S. 2018. Introduction. ACM Transactions on

Computer-Human Interaction 25 (1), 1–8. doi:10.1145/3177794.

Barbera, E., Garcia, I. & Fuertes-Alpiste, M. 2017. A Co-Design Process Microanal-

ysis: Stages and Facilitators of an Inquiry-Based and Technology-Enhanced
Learning Scenario. The International Review of Research in Open and Dis-
tributed Learning 18 (6), 104–126. doi:10.19173/irrodl.v18i6.2805.

Barcellini, F., Prost, L. & Cerf, M. 2015. Designers’ and users’ roles in participatory
design: What is actually co-designed by participants? Applied Ergonomics 50,
31–40. doi:10.1016/j.apergo.2015.02.005.

Beck, E. E. 2002. P for Political: Participation is Not Enough. Scandinavian Journal

of Information Systems 14 (1), 77–92.

Berge, O. 2017. Rethinking Digital Literacy in Nordic School Curricula. Nordic

Journal of Digital Literacy 12 (01-02), 5–7. doi:10.18261/issn.1891-943x-2017-
01-02-01.
 59

Bergvall-Kåreborn, B. & Ståhlbrost, A. 2008. Participatory design: one step back

or two steps forward? In Proceedings of the Tenth Anniversary Conference on
Participatory Design 2008, 102–111. doi:10.1145/1795234.1795249.

Bjerknes, G. & Bratteteig, T. 1995. User participation and democracy: a discus-

sion of Scandinavian research on system development. Scandinavian Journal
of Information Systems 7 (1), 73–98.

Bjerknes, G., Ehn, P. & Kyng, M. 1987. Computers and Democracy: A Scandina-
vian Challenge. Brookfield: Gower Press, 455.

Bjögvinsson, E., Ehn, P. & Hillgren, P.-a. 2012. Design Things and Design Think-
ing: Contemporary Participatory Design Challenges. Design Issues 28 (3), 101–
116.

Booker, A. & Goldman, S. 2016. Participatory Design Research as a Practice for

Systemic Repair: Doing Hand-in-Hand Math Research with Families. Cogni-
tion and Instruction 34 (3), 222–235. doi:10.1080/07370008.2016.1179535.

Bossavit, B. & Parsons, S. 2016. Designing an Educational Game for and with
Teenagers with High Functioning Autism. In Proceedings of the 14th Partici-
patory Design Conference: Full Papers - Volume 1. New York, NY, USA: ACM.
PDC ’16, 11–20. doi:10.1145/2940299.2940313.

Bossen, C., Dindler, C. & Iversen, O. S. 2010. User gains and PD aims. In Proceed-
ings of the 11th Biennial Participatory Design Conference on - PDC ’10. New
York, New York, USA: ACM Press, 141. doi:10.1145/1900441.1900461.

Bratteteig, T. & Wagner, I. 2012. Disentangling power and decision-making in par-

ticipatory design. In Proceedings of the 12th Participatory Design Conference
on Research Papers: Volume 1 - PDC ’12. New York, New York, USA: ACM
Press, 41.

Bratteteig, T. & Wagner, I. 2016. Unpacking the Notion of Participation in Partic-

ipatory Design. Computer Supported Cooperative Work (CSCW) 25 (6), 425–
475. doi:10.1007/s10606-016-9259-4.

Bratteteig, T. 2003. Making change: Dealing with relations between design and
use. University of Oslo. Ph. D. Thesis, 265.

Brereton, M., Sitbon, L., Abdullah, M. H. L., Vanderberg, M. & Koplick, S.

2015. Design after design to bridge between people living with cognitive
or sensory impairments, their friends and proxies. CoDesign 11 (1), 4–20.
doi:10.1080/15710882.2015.1009471.

Burke, C. & Könings, K. D. 2016. Recovering lost histories of educational design:

a case study in contemporary participatory strategies. Oxford Review of Edu-
cation 42 (6), 721–732. doi:10.1080/03054985.2016.1232244.
60

Bødker, K., Kensing, F. & Simonsen, J. 2004. Participatory IT Design - Design-

ing for Business and Workplace Realities (First edition). Cambridge, Mas-
sachusetts: The MIT Press, 341.
Bødker, S., Dindler, C. & Iversen, O. S. 2017. Tying Knots: Participatory Infras-
tructuring at Work. Computer Supported Cooperative Work (CSCW) 26 (1-2),
245–273. doi:10.1007/s10606-017-9268-y.
Bødker, S., Ehn, P., Sjögren, D. & Sundblad, Y. 2000. Co-operative Design — per-
spectives on 20 years with ‘ the Scandinavian IT Design Model ’. In NordiCHI
2000, 1–9.
Bødker, S. & Kyng, M. 2018. Participatory Design that Matters—Facing the
Big Issues. ACM Transactions on Computer-Human Interaction 25 (1), 1–31.
doi:10.1145/3152421.
Bødker, S. 1987. Through the Interface - a Human Activity Approach to User In-
terface Design. University of Aarhus. Ph. D. Thesis.
Bødker, S. 2003. A for Alternatives. Scandinavian Journal of Information Systems
15 (2002), 87–89.
Carmichael, P. 2015. Not just about gadgets: Habit, innovation and change in the
design of learning technologies. E-Learning and Digital Media 12 (3-4), 279–
294. doi:10.1177/2042753015571052.
Casanova, D. & Mitchell, P. 2017. The cube and the poppy: Participatory ap-
proaches for designing technology-enhanced learning spaces. Journal of Learn-
ing Spaces 6 (3), 1–12.
Clement, A. & Van den Besselaar, P. 1993. A retrospective look at PD projects.
Communications of the ACM 36 (6), 29–37.
Cober, R., Tan, E., Slotta, J., So, H.-J. & Könings, K. D. 2015. Teachers as participa-
tory designers: two case studies with technology-enhanced learning environ-
ments. Instructional Science 43 (2), 203–228. doi:10.1007/s11251-014-9339-0.
Cox, M. J. 2013. Formal to informal learning with IT: research challenges and
issues for e-learning. Journal of Computer Assisted Learning 29 (1), 85–105.
doi:10.1111/j.1365-2729.2012.00483.x.
Cramer, M. & Hayes, G. R. 2013. The digital economy. In Proceedings of the 12th
International Conference on Interaction Design and Children - IDC ’13. New
York, New York, USA: ACM Press, 431–434. doi:10.1145/2485760.2485832.
Cross, N. 2001. Designerly Ways of Knowing: Design Discipline Versus Design
Science. Design Issues 17 (3), 49–55.
Cruz-Jesus, F., Vicente, M. R., Bacao, F. & Oliveira, T. 2016. The education-related
digital divide: An analysis for the EU-28. Computers in Human Behavior 56,
72–82.
 61

Cviko, A., McKenney, S. & Voogt, J. M. 2014. Teacher roles in designing

technology-rich learning activities for early literacy: A cross-case analysis.
Computers & Education 72, 68–79. doi:10.1016/j.compedu.2013.10.014.

Dantec, C. A. L. & DiSalvo, C. 2013. Infrastructuring and the formation of publics

in participatory design. Social Studies of Science 43 (2), 241–264.

DiSalvo, B., Yip, J., Bonsignore, E. & DiSalvo, C. 2017. Participatory Design for
Learning: Perspectives from Practice and Research (First edition). New york:
Routledge, 268.

Druin, A. 2002. The role of children in the design of new technology. Behaviour
& Information Technology 21 (1), 1–25. doi:10.1080/01449290110108659.

Duarte, A. M. B., Brendel, N., Degbelo, A. & Kray, C. 2018. Participatory Design
and Participatory Research. ACM Transactions on Computer-Human Interac-
tion 25 (1), 1–39.

Duell, C., Wright, N. & Roxburgh, J. 2014. Developing "Design Minds" for the 21st
Century through a Public Sector Initiated Online Design Education Platform.

Ehn, P. 1988. Work-Oriented Design of Computer Artifacts. Umeå Universitat.

Ph. D. Thesis.

Ertmer, P. a., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E. & Sendurur, P.

2012. Teacher beliefs and technology integration practices: A critical relation-
ship. doi:10.1016/j.compedu.2012.02.001.

Ervasti, M., Laitakari, J. & Hillukkala, M. 2016. ‘I want to know where

my child is at all times’ – field study of a location-aware safety service
for schoolchildren. Behaviour & Information Technology 35 (10), 833–852.
doi:10.1080/0144929X.2016.1201144.

Fage, C., Pommereau, L., Consel, C., Balland, E. & Sauzéon, H. 2016. Tablet-
Based Activity Schedule in Mainstream Environment for Children with Autism
and Children with ID. ACM Transactions on Accessible Computing 8 (3), 1–26.
doi:10.1145/2854156.

Fage, C., Pommereau, L., Consel, C., Balland, É. & Sauzéon, H. 2014. Tablet-based
activity schedule for children with autism in mainstream environment. In Pro-
ceedings of the 16th international ACM SIGACCESS conference on Computers
& accessibility - ASSETS ’14. New York, New York, USA: ACM Press, 145–152.

Farooq, U., Schank, P., Harris, A., Fusco, J. & Schlager, M. 2007. Sustaining a
Community Computing Infrastructure for Online Teacher Professional Devel-
opment: A Case Study of Designing Tapped In. Computer Supported Cooper-
ative Work (CSCW) 16 (4-5), 397–429. doi:10.1007/s10606-007-9049-0.
62

Frauenberger, C., Good, J., Fitzpatrick, G. & Iversen, O. S. 2015. In pursuit

of rigour and accountability in participatory design. International Journal of
Human-Computer Studies 74, 93–106. doi:10.1016/j.ijhcs.2014.09.004.

Gaver, W. 2012. What should we expect from research through design? In Pro-
ceedings of the 2012 ACM annual conference on Human Factors in Computing
Systems - CHI ’12. New York, New York, USA: ACM Press, 937.

Gavriushenko, M., Khriyenko, O. & Tuhkala, A. 2017. An Intelligent Learning

Support System. In Proceedings of the 9th International Conference on Com-
puter Supported Education, Vol. 1. SCITEPRESS - Science and Technology Pub-
lications, 217–225.

Gillies, M., Brenton, H., Yee-King, M., Grimalt-Reynes, A. & D’Inverno, M. 2015.
Sketches vs skeletons. In Proceedings of the 2nd International Workshop on
Movement and Computing - MOCO ’15, Vol. 14-15-Augu. New York, New
York, USA: ACM Press, 104–111. doi:10.1145/2790994.2790995.

Girard, S. & Johnson, H. 2010. Designing affective computing learning compan-

ions with teachers as design partners. In Proceedings of the 3rd international
workshop on Affective interaction in natural environments - AFFINE ’10. New
York, New York, USA: ACM Press, 49. doi:10.1145/1877826.1877840.

Goeze, A., Zottmann, J. M., Vogel, F., Fischer, F. & Schrader, J. 2014. Getting im-
mersed in teacher and student perspectives? Facilitating analytical competence
using video cases in teacher education. Instructional Science 42 (1), 91–114.
doi:10.1007/s11251-013-9304-3.

Greenbaum, J. & Kyng, M. (Eds.) 1991. Design at Work: Cooperative Design of

Computer Systems. Hillsdale, NJ, USA: L. Erlbaum Associates Inc.

Greenbaum, J. 1993. PD a personal statement. Communications of the ACM 36

(6), 47. doi:10.1145/153571.214816.

Gregory, J. 2003. Scandinavian Approaches to Participatory Design. International

Journal of Engaging Education 19 (1), 62–74.

Gros, B. & López, M. 2016. Students as co-creators of technology-rich learning

activities in higher education. International Journal of Educational Technology
in Higher Education 13 (1), 28. doi:10.1186/s41239-016-0026-x.

Grönvall, E. & Kyng, M. 2013. On participatory design of home-based healthcare.

Cognition, Technology & Work 15 (4), 389–401.

Hall, R., Atkins, L. & Fraser, J. 2014. Defining a self-evaluation digital literacy
framework for secondary educators: the DigiLit Leicester project. Research in
Learning Technology 22. doi:10.3402/rlt.v22.21440.
 63

Halskov, K. & Hansen, N. B. 2015. The diversity of participatory design research

practice at PDC 2002–2012. International Journal of Human-Computer Studies
74, 81–92. doi:10.1016/j.ijhcs.2014.09.003.

Hannon, D., Danahy, E., Schneider, L., Coopey, E. & Garber, G. 2012. Encouraging
teachers to adopt inquiry-based learning by engaging in participatory design.
In IEEE 2nd Integrated STEM Education Conference. Department of Mechan-
ical Engineering/Human Factors, Tufts University, United States: IEEE, 1–4.
doi:10.1109/ISECon.2012.6204169.

Harrison, C. J., Könings, K. D., Schuwirth, L. W. T., Wass, V. & van der
Vleuten, C. P. M. 2017. Changing the culture of assessment: the dominance
of the summative assessment paradigm. BMC Medical Education 17 (1), 73.
doi:10.1186/s12909-017-0912-5.

Herstad, J. & Holone, H. 2012. What we talk about when we talk about co-
creative tangibles. In Proceedings of the 12th Participatory Design Confer-
ence on Exploratory Papers Workshop Descriptions Industry Cases - Vol-
ume 2 - PDC ’12. New York, New York, USA: ACM Press. PDC ’12, 109.
doi:10.1145/2348144.2348179.

Hoda, R., Henderson, A., Lee, S., Beh, B. & Greenwood, J. 2014. Aligning tech-
nological and pedagogical considerations: Harnessing touch-technology to en-
hance opportunities for collaborative gameplay and reciprocal teaching in NZ
early education. International Journal of Child-Computer Interaction 2 (1), 48–
59. doi:10.1016/j.ijcci.2014.06.001.

Hsu, S. 2017. Developing and validating a scale for measuring changes in teach-
ers’ ICT integration proficiency over time. Computers & Education 111, 18–30.
doi:10.1016/j.compedu.2017.04.001.

Hussain, S. 2010. Empowering marginalised children in developing coun-

tries through participatory design processes. CoDesign 6 (2), 99–117.
doi:10.1080/15710882.2010.499467.

Iivari, J. & Iivari, N. 2006. Varieties of User-Centeredness. In Proceedings of the

39th Annual Hawaii International Conference on System Sciences (HICSS’06),
Vol. 8. IEEE, 176a–176a. doi:10.1109/HICSS.2006.530.

Ishimaru, A. M. & Takahashi, S. 2017. Disrupting Racialized Insti-

tutional Scripts Toward Parent-Teacher Transformative Agency for
Educational Justice. Peabody Journal of Education 92 (3), 343–362.
doi:10.1080/0161956X.2017.1324660.

Iversen, O. S. & Dindler, C. 2013. A Utopian agenda in child-computer in-

teraction. International Journal of Child-Computer Interaction 1 (1), 24–29.
doi:10.1016/j.ijcci.2012.08.002.
64

Iversen, O. S. & Dindler, C. 2014. Sustaining participatory design initiatives.

CoDesign 10, 153–170. doi:10.1080/15710882.2014.963124.

Iversen, O. S., Halskov, K. & Leong, T. W. 2012. Values-led participatory design.

CoDesign 8 (2-3), 87–103. doi:10.1080/15710882.2012.672575.

Iversen, O. S., Smith, R. C. & Dindler, C. 2017. Child as Protagonist: Expanding

the Role of Children in Participatory Design. Proceedings of the 2017 Confer-
ence on Interaction Design and Children - IDC ’17, 27–37.

Iversen, O. S., Smith, R. C. & Dindler, C. 2018. From computational thinking to

computational empowerment. In Proceedings of the 15th Participatory Design
Conference on Full Papers - PDC ’18. New York, New York, USA: ACM Press,
1–11.

Janssen, F., Könings, K. D. & van Merriënboer, J. J. 2017. Participatory educational

design: How to improve mutual learning and the quality and usability of the
design? European Journal of Education 52 (3), 268–279. doi:10.1111/ejed.12229.

Janssen, F. J. J. M., Westbroek, H. B. & van Driel, J. H. 2014. How to make guided
discovery learning practical for student teachers. Instructional Science 42 (1),
67–90. doi:10.1007/s11251-013-9296-z.

Joyce, C., Pham, H., Stanton Fraser, D., Payne, S., Crellin, D. & McDougall, S.
2014. Building an internet of school things ecosystem. In Proceedings of the
2014 conference on Interaction design and children - IDC ’14. New York, New
York, USA: ACM Press, 289–292. doi:10.1145/2593968.2610474.

Jutila, M., Strömmer, E., Ervasti, M., Hillukkala, M., Karhula, P. & Laitakari,
J. 2015. Safety services for children: a wearable sensor vest with wire-
less charging. Personal and Ubiquitous Computing 19 (5-6), 915–927.
doi:10.1007/s00779-015-0838-z.

Kalra, N., Lauwers, T., Dewey, D., Stepleton, T. & Dias, M. B. 2007. Iterative de-
sign of a Braille writing tutor to combat illiteracy. In 2007 International Con-
ference on Information and Communication Technologies and Development.
RAND corporation, United States: IEEE, 1–9. doi:10.1109/ICTD.2007.4937386.

Karasti, H. 2014. Infrastructuring in participatory design. In Proceedings of the

13th Participatory Design Conference on Research Papers - PDC ’14. New York,
New York, USA: ACM Press, 141–150.

Karimi, A., Worthy, P., McInnes, P., Bodén, M., Matthews, B. & Viller, S. 2017.
The Community Garden Hack: Participatory Experiments in Facilitating Pri-
mary School Teacher’s Appropriation of Technology. In Proceedings of the 29th
Australian Conference on Computer-Human Interaction. New York, NY, USA:
ACM. OZCHI ’17, 143–151. doi:10.1145/3152771.3152787.

Kelentric, M., Helland, K. & Arstorp, A.-T. 2017. Professional Digital Competence
Framework for Teachers.
 65

Kensing, F. & Blomberg, J. 1998. Participatory Design: Issues and Con-

cerns. Computer Supported Cooperative Work (CSCW) 7 (3-4), 167–185.
doi:10.1023/A:1008689307411.

Kensing, F. & Greenbaum, J. 2013. Heritage - Having a say. In J. Simonsen &

T. Robertson (Eds.) Routledge International Handbook of Participatory Design.
New York: Routledge, 21–36. doi:10.4324/9780203108543.ch2.

Kitchenham, B. & Charters, S. 2007. Guidelines for performing Systematic Litera-

ture reviews in Software Engineering, 57. doi:10.1145/1134285.1134500.

Kitchenham, B., Pretorius, R., Budgen, D., Brereton, O. P., Turner, M., Niazi, M.
& Linkman, S. 2010. Systematic literature reviews in software engineering-
A tertiary study. Information and Software Technology 52 (8), 792–805.
doi:10.1016/j.infsof.2010.03.006.

Klonari, A. & Gousiou, A. 2014. Encouraging teachers’ reflection using a card

game: The game of consequences. In Proceedings of the European Confer-
ence on Games-based Learning, Vol. 1. Dept of Geography, University of the
Aegean, Mytilene, Greece: , 279–285.

Koutamanis, A., Heuer, J. & Könings, K. D. 2017. A visual information tool for
user participation during the lifecycle of school building design: BIM. Euro-
pean Journal of Education 52 (3), 295–305. doi:10.1111/ejed.12226.

Kreitmayer, S., Rogers, Y., Laney, R. & Peake, S. 2013. UniPad: Orchestrating Col-
laborative Activities Through Shared Tablets and An Integrated Wall Display.
In Proceedings of the 2013 ACM international joint conference on Pervasive
and ubiquitous computing - UbiComp ’13. New York, New York, USA: ACM
Press, 801. doi:10.1145/2493432.2493506.

Krumsvik, R. J. 2014. Teacher educators’ digital competence. Scandinavian Jour-

nal of Educational Research 58 (3), 269–280. doi:10.1080/00313831.2012.726273.

Kuure, L., Molin-Juustila, T., Keisanen, T., Riekki, M., Iivari, N. & Kinnula, M.
2016. Switching perspectives: from a language teacher to a designer of lan-
guage learning with new technologies. Computer Assisted Language Learning
29 (5), 925–941. doi:10.1080/09588221.2015.1068815.

Kuuskorpi, M. & Gonzalez, N. C. 2014. Physical learning environments: learning

in the future. In M. Kuuskorpi (Ed.) Perspectives from finland towards new
learning environment. Tampere: Juvenes Print, Suomen Yliopistopaino Oy, 63–
77.

Kuuskorpi, M. 2012. Future physical learning environment. User oriented exible

and changeable teaching spaces. University of Turku. Dissertation.

Kyza, A. E. & Georgiou, Y. 2014. Developing In-Service Science Teachers’ Owner-

ship of the Profiles Pedagogical Framework through a Technology-Supported
66

Participatory Design Approach to Professional Development. Science Educa-

tion International 25 (2), 21.

Kyza, E. A. & Nicolaidou, I. 2017. Co-designing reform-based online inquiry

learning environments as a situated approach to teachers’ professional devel-
opment. CoDesign 13 (4), 261–286. doi:10.1080/15710882.2016.1209528.

Könings, K. D., Bovill, C. & Woolner, P. 2017. Towards an interdisciplinary model

of practice for participatory building design in education. European Journal of
Education 52 (3), 306–317. doi:10.1111/ejed.12230.

Könings, K. D., Brand-Gruwel, S. & van Merriënboer, J. J. 2007. Teachers’ per-

spectives on innovations: Implications for educational design. Teaching and
Teacher Education 23 (6), 985–997. doi:10.1016/j.tate.2006.06.004.

Könings, K. D., Brand-Gruwel, S. & van Merriënboer, J. J. 2010. An approach

to participatory instructional design in secondary education: an exploratory
study. Educational Research 52 (1), 45–59. doi:10.1080/00131881003588204.

Könings, K. D., Brand-Gruwel, S. & van Merriënboer, J. J. 2011. Participa-

tory instructional redesign by students and teachers in secondary education:
effects on perceptions of instruction. Instructional Science 39 (5), 737–762.
doi:10.1007/s11251-010-9152-3.

Könings, K. D., van Zundert, M. J., Brand-Gruwel, S. & van Merriënboer, J. J.

2007. Participatory design in secondary education: is it a good idea? Students’
and teachers’ opinions on its desirability and feasibility. Educational Studies
33 (4), 445–465. doi:10.1080/03055690701423648.

Lawrence, S. A. & Calhoun, F. 2013. Exploring Teachers ’ Perceptions of Literacy

and Use of Technology in Classroom Practice : Analysis of Self-Reported Prac-
tice in One School District. Journal of Literacy and Technology 14 (1), 51–71.

Leong, T. W. & Iversen, O. S. 2015. Values-led Participatory Design as a

Pursuit of Meaningful Alternatives. In Proceedings of the Annual Meet-
ing of the Australian Special Interest Group for Computer Human Inter-
action on - OzCHI ’15. New York, New York, USA: ACM Press, 314–323.
doi:10.1145/2838739.2838784.

Lindtner, S. & Lin, C. 2017. Making and its promises. CoDesign 13 (2), 70–82.

Lingnau, A. & Lenschow, H. 2010. Scenarios for computersupported learning

in a special needs classroom. Journal of Assistive Technologies 4 (2), 26–35.
doi:10.5042/jat.2010.0279.

Lundström, A., Savolainen, J. & Kostiainen, E. 2016. Case study: developing cam-
pus spaces through co-creation. Architectural Engineering and Design Man-
agement 12 (6), 409–426. doi:10.1080/17452007.2016.1208077.
 67

Malinverni, L., Mora-Guiard, J. & Pares, N. 2016. Towards methods for evaluat-
ing and communicating participatory design: A multimodal approach. Inter-
national Journal of Human-Computer Studies 94, 53–63.

Mama, M. & Hennessy, S. 2013. Developing a typology of teacher beliefs and

practices concerning classroom use of ICT. Computers & Education 68 (0), 380–
387. doi:10.1016/j.compedu.2013.05.022.

Medeiros-Braz, L., Souza Ramos, E., Benedetti, M. L. P. & Hornung, H. 2017. Par-
ticipatory Design of Technology for Inclusive Education: A Case Study. In Lec-
ture Notes in Computer Science, Vol. 10279 LNCS. Campinas: Springer Verlag,
168–187. doi:10.1007/978-3-319-58700-4.

van Merriënboer, J. J., McKenney, S., Cullinan, D. & Heuer, J. 2017. Aligning ped-
agogy with physical learning spaces. European Journal of Education 52 (3),
253–267. doi:10.1111/ejed.12225.

Mishra, P. & Koehler, M. 2006. Technological Pedagogical Content Knowledge:

A Framework for Teacher Knowledge. Teachers College Record 108 (6), 1017–
1054.

Muller, M. J. 1991. PICTIVE - An exploration in participatory design. In Proceed-

ings of the SIGCHI conference on Human factors in computing systems Reach-
ing through technology - CHI ’91. New York, NY, USA: ACM. CHI ’91, 225–231.

Nevari, J. 2013. Oivaltamo: avautuva oppimistila. Lahti University of Applied

Sciences. Ph. D. Thesis.

Ng, W. 2015. Change and Continuity in Educational Uses of New Digital Tech-
nologies. In New Digital Technology in Education. Cham: Springer Interna-
tional Publishing, 3–23.

Nieminen, P., Pölönen, I. & Sipola, T. 2013. Research literature clustering using
diffusion maps. Journal of Informetrics 7 (4), 874–886.

Nunes, E. P. S., Luz, A. R., Lemos, E. M. & Nunes, C. 2016. Approaches of Partici-
patory Design in the Design Process of a Serious Game to Assist in the Learning
of Hospitalized Children. doi:10.1007/978-3-319-39513-5.

OECD 2015. Education Policy Outlook 2015: Making Reforms Happen.

OECD 2016. PISA 2015 Results (Volume I): Excellence and Equity in Education.

Otero, N., Alissandrakis, A., Müller, M., Milrad, M., Lencastre, J. A., Casal, J. &
José, R. 2013. Promoting secondary school learners’ curiosity towards science
through digital public displays. In Proceedings of International Conference on
Making Sense of Converging Media - AcademicMindTrek ’13. New York, New
York, USA: ACM Press, 204–210. doi:10.1145/2523429.2523475.
68

Owston, R. 2007. Teachers can make a difference : Professional development as

a policy option for improving student learning with ICT. In CERI-KERIS Inter-
national Expert Meeting On ICT and Educational Performance, 1–6.

Pantsar-Syväniemi, S., Ervasti, M., Karppinen, K., Väätänen, A., Oksman, V. &
Kuure, E. 2015. A situation-aware safety service for children via participatory
design. Journal of Ambient Intelligence and Humanized Computing 6 (2), 279–
293. doi:10.1007/s12652-014-0225-z.

Parsons, S. & Cobb, S. 2014. Reflections on the role of the ‘users’: challenges in a
multi-disciplinary context of learner-centred design for children on the autism
spectrum. International Journal of Research & Method in Education 37 (4), 421–
441. doi:10.1080/1743727X.2014.890584.

Parsons, S., Millen, L., Garib-Penna, S. & Cobb, S. 2011. Participatory design in the
development of innovative technologies for children and young people on the
autism spectrum: the COSPATIAL project. Journal of Assistive Technologies 5
(1), 29–34. doi:10.5042/jat.2011.0099.

Pedersen, J. B., Andersen, A. S. Ö. & Majgaard, G. 2012. Design of trigonome-

try apps for vocational education. In Proceedings of the IASTED International
Conference on Human-Computer Interaction, HCI 2012. Itai a/s, Ndr. Lande-
vej 2a, DK-6270 T{ø}nder, Denmark: , 226–232. doi:10.2316/P.2012.772-020.

Pietarinen, J., Pyhältö, K. & Soini, T. 2017. Large-scale curriculum reform in Fin-
land – exploring the interrelation between implementation strategy, the func-
tion of the reform, and curriculum coherence. The Curriculum Journal 28 (1),
22–40.

Pilemalm, S. 2018. Participatory Design in Emerging Civic Engagement Initia-

tives in the New Public Sector. ACM Transactions on Computer-Human Inter-
action 25 (1), 1–26. doi:10.1145/3152420.

Pollock, M. & Amaechi, U. 2013. Texting as a channel for personalized youth

support: participatory design research by city youth and teachers. Learning,
Media and Technology 38 (2), 128–144. doi:10.1080/17439884.2013.756516.

Pontual-Falcão, T., Mendes de Andrade e Peres, F., Sales de Morais, D. C. & da

Silva Oliveira, G. 2018. Participatory methodologies to promote student en-
gagement in the development of educational digital games. Computers & Ed-
ucation 116, 161–175. doi:10.1016/j.compedu.2017.09.006.

Prins, G., Bulte, A. & Pilot, A. 2016. An Activity-Based Instructional Frame-

work for Transforming Authentic Modeling Practices into Meaningful Con-
texts for Learning in Science Education. Science Education 100 (6), 1092–1123.
doi:10.1002/sce.21247.
 69

Pérez-Sanagustín, M., Nussbaum, M., Hilliger, I., Alario-Hoyos, C., Heller, R. S.,
Twining, P. & Tsai, C.-C. 2017. Research on ICT in K-12 schools – A review of ex-
perimental and survey-based studies in computers and education 2011 to 2015.
Computers & Education 104, A1–A15. doi:10.1016/j.compedu.2016.09.006.

Põldoja, H., Väljataga, T., Laanpere, M. & Tammets, K. 2014. Web-based self- and
peer-assessment of teachers’ digital competencies. World Wide Web 17 (2), 255–
269. doi:10.1007/s11280-012-0176-2.

Rahamat, R., Shah, P. M., Nor Puteh, S., Din, R. & Karim, A. A. 2011. End-users’
involvement in the development of web-based learning resources for English
literature. 3L: The Southeast Asian Journal of English Language Studies 17
(SPEC. ISSUE), 5–18.

Reeves, T. C. 2015. Educational design research: Signs of progress. Australasian

Journal of Educational Technology 31 (5), 613–620.

Robertson, T. & Wagner, I. 2012. Ethics: engagement, representation and politics-

in-action. In J. Simonsen & T. Robertson (Eds.) Routledge International Hand-
book of Participatory Design. Routledge, 64–85.

Rodrigo, R. & Ramírez, C. D. 2017. Balancing Institutional Demands with Effec-

tive Practice: A Lesson in Curricular and Professional Development. Technical
Communication Quarterly 26 (3), 314–328. doi:10.1080/10572252.2017.1339529.

Rodríguez-Triana, M. J., Martínez-Monés, A., Asensio-Pérez, J. I. & Dimitri-

adis, Y. 2012. Towards a Monitoring-Aware Design Process for CSCL Scripts.
doi:10.1007/978-3-642-33284-5.

Røkenes, F. M. & Krumsvik, R. J. 2016. Prepared to teach ESL with ICT? A study of
digital competence in Norwegian teacher education. Computers & Education
97, 1–20. doi:10.1016/j.compedu.2016.02.014.

Saad-Sulonen, J., Eriksson, E., Halskov, K., Karasti, H. & Vines, J. 2018. Unfolding
participation over time: temporal lenses in participatory design. CoDesign 14
(1), 4–16.

Salinas, Á., Nussbaum, M., Herrera, O., Solarte, M. & Aldunate, R. 2016. Fac-
tors affecting the adoption of information and communication technologies in
teaching. Education and Information Technologies, 1–22. doi:10.1007/s10639-
016-9540-7.

Sanders, E. B.-N. & Stappers, P. J. 2008. Co-creation and the new landscapes of
design. CoDesign 4 (1), 5–18. doi:10.1080/15710880701875068.

Sanders, E. B.-N. & Stappers, P. J. 2014. From designing to co-designing to collec-

tive dreaming. Interactions 21 (6), 24–33.
70

Scherer, R., Siddiq, F. & Teo, T. 2015. Becoming more specific: Mea-
suring and modeling teachers’ perceived usefulness of ICT in the con-
text of teaching and learning. Computers & Education 88, 202–214.
doi:10.1016/j.compedu.2015.05.005.

Schuler, D. & Namioka, A. (Eds.) 1993. Participatory design: Principles and prac-
tices. New Jersey: Lawrence Erlbaum Associates, Inc., 319.

Selwyn, N., Henderson, M. & Chao, S.-H. 2017. The possibilities and limitations
of applying ‘open data’ principles in schools. Cambridge Journal of Education
47 (2), 167–187. doi:10.1080/0305764X.2016.1143449.

Simonsen, J. & Robertson, T. (Eds.) 2013. Routledge International Handbook of

Participatory Design. New York: Routledge, 295.

Siozos, P., Palaigeorgiou, G., Triantafyllakos, G. & Despotakis, T. 2009. Computer

based testing using “digital ink”: Participatory design of a Tablet PC based
assessment application for secondary education. Computers & Education 52
(4), 811–819. doi:10.1016/j.compedu.2008.12.006.

Smith, R. C., Bossen, C. & Kanstrup, A. M. 2017. Participatory design in an era of

participation. CoDesign 13 (2), 65–69. doi:10.1080/15710882.2017.1310466.

Smith, R. C., Iversen, O. S. & Hjorth, M. 2015. Design thinking for digital fab-
rication in education. International Journal of Child-Computer Interaction 5,
20–28.

Smith, R. C. & Iversen, O. S. 2018. Participatory design for sustainable social

change. Design Studies, 1–28.

So, H.-J., Lossman, H., Lim, W.-Y. & Jacobson, M. J. 2009. Designing an online
video based platform for teacher learning in Singapore. Australasian Journal
of Educational Technology 25 (3), 440–457.

Somerville, M. M. & Collins, L. 2008. Collaborative design: a learner-

centered library planning approach. The Electronic Library 26 (6), 803–820.
doi:10.1108/02640470810921592.

Song, D. & Oh, E. Y. 2016. A Participatory Design Approach for a Mobile App-
Based Personal Response System. Journal of Educational Technology Systems
44 (3), 346–361. doi:10.1177/0047239515618465.

Spinuzzi, C. 2002. A Scandinavian challenge, a US response. In Proceed-

ings of the 20th annual international conference on Computer documen-
tation - SIGDOC ’02. New York, New York, USA: ACM Press, 208–215.
doi:10.1145/584955.584986.

Spinuzzi, C. 2005. The methodology of participatory design. Technical Commu-

nication 52 (2), 163–174.
 71

Steen, M. 2011. Tensions in human-centred design. CoDesign 7 (1), 45–60.

doi:10.1080/15710882.2011.563314.

Steen, M. 2013. Virtues in Participatory Design: Cooperation, Curiosity, Creativ-

ity, Empowerment and Reflexivity. Science and Engineering Ethics 19 (3), 945–
962. doi:10.1007/s11948-012-9380-9.

Stephen, M. L., Locke, S. M. & Bracey, G. L. 2014. Using a Participatory Design

Approach to Create and Sustain an Innovative Technology-rich STEM Class-
room - One School’s Story. In Proceedings of the 6th International Conference
on Computer Supported Education, Vol. 3. Center for Science, Technology, En-
gineering and Mathematics Research, Education, and Outreach, Southern Illi-
nois University, Edwardsville, IL, United States: SCITEPRESS - Science and
and Technology Publications, 30–38. doi:10.5220/0004849900300038.

Su, C., Chiu, C. & Wang, T. 2010. The development of SCORM-conformant

learning content based on the learning cycle using participatory design.
Journal of Computer Assisted Learning 26 (5), 392–406. doi:10.1111/j.1365-
2729.2010.00355.x.

Suchman, L. 1988. Designing with the user: book review of Computers and
democracy: a Scandinavian challenge, G. Bjerknes, P. Ehn, and M. Kyng, Eds.
Gower Press, Brookfield, VT, 1987. ACM Transactions on Information Systems
6 (2), 173–183. doi:10.1145/45941.383895.

Tammets, K., Pata, K., Laanpere, M., Tomberg, V., Gašević, D. & Siadaty, M.
2011. Designing the competence-driven teacher accreditation. doi:10.1007/978-
3-642-25813-8.

Tanhua-Piiroinen, E., Viteli, J., Syvänen, A., Vuorio, J., Hintikka, K. A. & Saira-
nen, H. 2016. Perusopetuksen oppimisympäristöjen digitalisaation nykytilanne
ja opettajien valmiudet hyödyntää digitaalisia oppimisympäristöjä. Valtioneu-
voston kanslia.

Tondeur, J., van Braak, J., Siddiq, F. & Scherer, R. 2016. Time for a
new approach to prepare future teachers for educational technology use:
Its meaning and measurement. Computers & Education 94, 134–150.
doi:10.1016/j.compedu.2015.11.009.

Triantafyllakos, G., Palaigeorgiou, G. & Tsoukalas, I. A. 2011. Designing

educational software with students through collaborative design games:
The We!Design&Play framework. Computers & Education 56 (1), 227–242.
doi:10.1016/j.compedu.2010.08.002.

Triantafyllou, E. & Timcenko, O. 2013. Developing digital technologies for uni-

versity mathematics by applying participatory design methods. In 2013 24th
EAEEIE Annual Conference (EAEEIE 2013). Dept. of Media Technology, Aal-
borg University Copenhagen, A.C. Meyers Vaenge 15, DK-2450 Copenhagen
SV, Denmark: IEEE, 82–85. doi:10.1109/EAEEIE.2013.6576507.
72

Tuhkala, A. & Kärkkäinen, T. 2018. Using Slack for computer-mediated communi-

cation to support higher education students’ peer interactions during Master’s
thesis seminar. Education and Information Technologies 23 (6), 2379–2397.

Tulinius, C., Brit, A., Nielsen, S., Hansen, L. J., Hermann, C., Vlasova, L. & Dal-
sted, R. 2012. Increasing the general level of academic capacity in general prac-
tice: Introducing mandatory research training for general practitioner trainees
through a participatory research process. Quality in Primary Care 20 (1), 57–67.

Uluyol, Ç. & Şahin, S. 2016. Elementary school teachers’ ICT use in the classroom
and their motivators for using ICT. British Journal of Educational Technology
47 (1), 65–75. doi:10.1111/bjet.12220.

Vakil, S., McKinney de Royston, M., Suad Nasir, N. & Kirshner, B. 2016. Rethink-
ing Race and Power in Design-Based Research: Reflections from the Field. Cog-
nition and Instruction 34 (3), 194–209. doi:10.1080/07370008.2016.1169817.

Vines, J., Clarke, R., Light, A. & Wright, P. 2015. The beginnings, middles and
endings of participatory research in HCI: An introduction to the special issue
on ‘perspectives on participation’. International Journal of Human-Computer
Studies 74, 77–80. doi:10.1016/j.ijhcs.2014.11.002.

Vines, J., Clarke, R., Wright, P., McCarthy, J. & Olivier, P. 2013. Configuring partic-
ipation: On How We Involve People In Design. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems - CHI ’13, Vol. 20. New
York, New York, USA: ACM Press, 429.

Vrasidas, C. 2015. The rhetoric of reform and teachers’ use of ICT. British Journal
of Educational Technology 46 (2), 370–380. doi:10.1111/bjet.12149.

Wang, F. & Hannafin, M. J. 2005. Design-based research and technology-enhanced

learning environments. Educational Technology Research and Development 53
(4), 5–23. doi:10.1007/BF02504682.

Wastiau, P., Blamire, R., Kearney, C., Quittre, V., Van de Gaer, E. & Monseur, C.
2013. The Use of ICT in Education: a survey of schools in Europe. European
Journal of Education 48 (1), 11–27. doi:10.1111/ejed.12020.

Woolner, P., Clark, J., Hall, E., Tiplady, L., Thomas, U. & Wall, K. 2010. Pic-
tures are necessary but not sufficient: Using a range of visual methods to en-
gage users about school design. Learning Environments Research 13 (1), 1–22.
doi:10.1007/s10984-009-9067-6.

Woolner, P., Hall, E., Wall, K. & Dennison, D. 2007. Getting together to improve
the school environment: user consultation, participatory design and student
voice. Improving Schools 10 (3), 233–248. doi:10.1177/1365480207077846.

Zainuddin, N. M. M., Zaman, H. B. & Ahmad, A. 2010. A Participatory De-

sign in Developing Prototype an Augmented Reality Book for Deaf Students.
 73

In 2010 Second International Conference on Computer Research and Devel-

opment. Department of Information Science, Universiti Kebangsaan Malaysia,
Bangi Selangor, Malaysia: IEEE, 400–404. doi:10.1109/ICCRD.2010.55.

Zheng, B., Warschauer, M., Lin, C.-H. & Chang, C. 2016. Learning in One-to-
One Laptop Environments: A Meta-Analysis and Research Synthesis. Review
of Educational Research 86 (4), 1052–1084.

Zimmerman, J., Forlizzi, J. & Evenson, S. 2007. Research through design as a

method for interaction design research in HCI. In Proceedings of the SIGCHI
conference on Human factors in computing systems - CHI ’07. New York, New
York, USA: ACM Press, 493.

Zimmerman, J., Stolterman, E. & Forlizzi, J. 2010. An analysis and critique of

Research through Design. In Proceedings of the 8th ACM Conference on De-
signing Interactive Systems - DIS ’10. New York, New York, USA: ACM Press,
310.

European Commission 2018. Communication on the Digital Education Action

Plan. European Comission, 11.

Finnish National Board on Research Integrity 2018. Agreeing on authorship rec-

ommendation for research publications. The Finnish National Board on Re-
search Integrity.
 ORIGINAL PAPERS

PI

SEMI-AUTOMATIC LITERATURE MAPPING OF

PARTICIPATORY DESIGN STUDIES 2006–2016

by

Tuhkala Ari, Kärkkäinen Tommi, and Nieminen Paavo 2018

Proceedings of the 15th Participatory Design Conference: Short Papers, Situated

Actions, and Workshops - PDC ’18
 Semi-Automatic Literature Mapping of Participatory Design
Studies 2006-2016
Ari Tuhkala Tommi Kärkkäinen Paavo Nieminen
University of Jyväskylä University of Jyväskylä University of Jyväskylä
P.O. Box 35 (Agora) P.O. Box 35 (Agora) P.O. Box 35 (Agora)
Jyväskylä FI-40014, Finland Jyväskylä FI-40014, Finland Jyväskylä FI-40014, Finland
[email protected] [email protected] [email protected]

ABSTRACT extended from mere offices to schools, hospitals, and other contexts.
The paper presents a process of semi-automatic literature mapping Consequently, Participatory Design Conference proceedings repre-
of a comprehensive set of participatory design studies between sent only a fraction of all PD research and a vast amount of research
2006-2016. The data of 2939 abstracts were collected from 14 aca- is published in discipline-specific journals and conferences. Thus,
demic search engines and databases. With the presented method, researchers need to use several search engines and go through a
we were able to identify six education-related clusters of PD arti- large number of studies. This raises a question: could computational
cles. Furthermore, we point out that the identified clusters cover methods assist researchers in mapping previous knowledge and
the majority of education-related words in the whole data. This locating relevant literature?
is the first attempt to systematically map the participatory design This paper describes the process of a semi-automatic literature
literature. We argue that by continuing our work, we can help to mapping and demonstrates the applicability of our method. First,
perceive a coherent structure in the body of PD research. we built a comprehensive set of all PD literature, that is published
between 2006-2016, by systematically collecting studies from 14
CCS CONCEPTS different search engines and databases. Then, we conducted a semi-
automatic mapping, similarly to the approach in Nieminen et al.
• Information systems → Clustering; Clustering and classifica-
[21], of this set of PD studies. We applied unsupervised learning
tion; • Human-centered computing → Participatory design; •
to automatically find a division of topics and a thematic structure
Computing methodologies → Natural language processing;
in a body of literature. Although, preprocessing the article dataset,
KEYWORDS determining the number of article clusters in recursive clustering,
and interpreting the article clusters were done semi-automatically
Systematic literature mapping, clustering, participatory design by the authors. To demonstrate the applicability of our method, we
ACM Reference format: scrutinised the clusters to identify education related PD literature.
Ari Tuhkala, Tommi Kärkkäinen, and Paavo Nieminen. 2018. Semi-Automatic In addition, we analysed the proportions of education-related words
Literature Mapping of Participatory Design Studies 2006-2016. In Proceed- within these clusters. With our method, we managed to identify six
ings of Proceedings of the 15th Participatory Design Conference - Volume 2,
education related clusters that covered the majority of education-
Hasselt and Genk, Belgium, August 20–24, 2018 (PDC ’18), 5 pages.
related words.
https://doi.org/10.1145/3210604.3210621
This is the first attempt to systematically map PD research lit-
erature. For example, Halskov and Hansen [14, p.83] focused on
1 INTRODUCTION Participatory Design Conference proceedings between 1990-2012.
The number of published scientific articles is increasing with high Nunes et al. [22, p.408] focused on IEEE and ACM databases. In
pace and new publication venues, such as journals, conferences, line with Halskov and Hansen [14], we propose that systematic
and open access archives, are emerging [19, 28]. To base research mapping of PD literature provides better understanding of where,
on existing body of knowledge, researchers use academic search how, and why PD has been carried out. This helps PD adjuncts to
engines and databases, such as ACM Digital Library. However, the build more solid bases for their work by locating relevant studies
problem is that there is no single search engine that would cover and serving as a pre-stage for the actual literature review. For this,
all different publication venues. Consequently, staying up to date our study is an encouraging step. However, we remind that the
with all published research has become arduous. quality, or relevance to a certain topic, of individual studies should
This problem accumulates in interdisciplinary fields, such as par- not be based solely on the mapping. Instead, the method is useful
ticipatory design (PD). The domains, where PD is carried out, have when perceiving structure in a large amount of studies.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed 2 METHOD
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM 2.1 Data Collection
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a The data collection took place in January 2017 and encompassed 14
fee. Request permissions from [email protected]. databases: ACM Digital library, Bielefeld Academic Search Engine,
PDC ’18, August 20–24, 2018, Hasselt and Genk, Belgium EBSCOhost Research Databases, ERIC Institute of Education Sci-
© 2018 Association for Computing Machinery.
ACM ISBN 978-1-4503-5574-2/18/08. . . $15.00 ences search, IEEE Xplore Digital library, JSTOR, ProQuest, SAGE
https://doi.org/10.1145/3210604.3210621 Journals, ScienceDirect, Scopus, SpringerLink, Taylor and Francis
PDC ’18, August 20–24, 2018, Hasselt and Genk, Belgium A. Tuhkala et al.

Online, Wiley Online Library, and Thomson Reuters Web of Sci- 2.3 Data clustering
ence. Criteria for database selection were the possibility to 1) export Clustering means unsupervised classification of observations into
multiple references, 2) export references in Mendeley supported groups with a twofold aim: observations within a cluster should be
format, and 3) include abstracts in meta data. Thus, we excluded similar to each other, and dissimilar to observations in other clusters
Google Scholar, Semantic Scholar, and CiteSeerX. [16]. There are various clustering methods and approaches, such
From the selected databases, we extracted references where ti- as density-based, probabilistic, grid-based, and spectral clustering
tle, abstract, or keywords contained keyword pair "participatory [1]. The most common clustering methods are hierarchical and
design", publication year was between 2006-2016, and publication prototype-based clustering, of which the basic form of hierarchical
type was either journal or conference article. We excluded book clustering is not scalable to the large volume of data because of
chapters, reports, reviews, theses, lectures, and patents, if it was the pairwise distance matrix requirement [32]. Moreover, many
possible in the search engine. clustering algorithms, including the hierarchical clustering, can
We combined the references in Mendeley to build a PD reference produce clusters of arbitrary shape in the data space, which is
database. First, we removed the references that were obviously difficult or even impossible to interpret [25]. Thus, we avoided any
faulty, such as references with only empty meta-information. We dimension reduction technique (see [21]), and used a prototype-
inspected all references to ensure that the needed metadata (title, based clustering method with a global distance measure to identify
abstract, author, year, and publication venue) were included. If the groups of similar documents. When each cluster is characterised by
reference had missing fields, we retrieved them manually. If it was a prototypical document, the cluster centroid, it is straightforward
not possible to retrieve the missing metadata with any of the search to determine a set of most typical documents of a cluster, closest to
engines, we removed the reference. Some journals, such as MIT the centroid, for analysis and interpretation.
Design Issues, did not provide abstracts in meta data, but we could Well-known iterative relocation algorithms, most prominently
retrieve them from full texts. the classical K-means [16, 20], approach clustering in two main
We removed initial duplicates with the Mendeley’s "Check for steps: i) initial generation of K prototypes, ii) local search (refine-
Duplicates" tool. We found that the tool cannot identify a duplicate, ment) of the initial prototypes. In general, initial prototypes should
if an author’s name is written with non-latin characters in one be well separated from each other without being outliers [16, 18].
reference and with latin characters in another. We resolved these Lately, the K-means++ algorithm [4], where the random initialisa-
cases manually. After the duplicates were removed, the database tion is based on a density function favouring distinct prototypes,
consisted of 2939 articles. has become the most popular variant. However, because the search
phase of these algorithms is locally exploitative, they need repeated
restarts in initial prototype regeneration to address global explo-
ration of the best clustering structure [15].
2.2 Data processing Each document is represented as a bag-of-words (BOW) vector
with the number of occurrences of each stemmed word. Because
We exported the references as a single RIS file, which is a stan-
the analysed documents arised from titles, abstracts, and keywords
dard tag format by Research Information Systems. We exported
of the articles, we used the so-called inverse document frequency
the file to our preprocessor, which splits the titles and abstracts
(idf) transformation [6] with the scaling function log(N /d f ), where
of each reference into words around whitespace boundaries and
N denotes the number of documents and d f the overall word fre-
converts the words to lowercase. We removed the common Eng-
quency. Even when such scaling changes the original data type of
lish language stopwords by using the default English stopwords
integers into real numbers, we still had a strictly discrete set of
of the Natural Language Tool Kit (NLTK) corpus and stemmatised
values with uniform quantization error [26]. Hence, we used the
the words with NLTK Snowball stemmer [7]. The data contained
l 1 /Cityblock distance, favorable compared in [11], as the distance
310013 non-stopword stems, of which 10194 were unique. Hereby,
function and to define the clustering error criterion. This means
we refer to the word stem simply as word.
that we used as the actual clustering algorithm the K-medians++,
The top ten frequent words in data were: design (12826), partici-
which is an initialization strategy with the l 1 -distance for the den-
patori (4177), use (3660), user (3468), develop (3280), process (2713),
sity function [4], and median as the document subset prototype
research (2525), particip (2330), paper (2142), and system (2073). As
within each cluster.
can be seen, manual addition of stopwords was needed to guide
After few initial tests, we noticed the need for recursive appli-
the algorithm to produce clusters based on the content words, not
cation of the document clustering. This was suggested in [29] and
on the format. Thus, we removed the words from the query string
successfully used for other application of clustering with c. 500
(participatory design): design (12826), participatori (4177), particip
000 observations by [27]. We hypothesised that this is due to the
(2330), part (376), pd (844), redesign (100), and codesign (54). We also
different shapes and scales of document clusters, as illustrated in [5,
removed 1515 words based on the knowledge that they are typical
Figure 5]. Similarly to the dimension reduction approach in [21], we
research parlance and have no discriminative power, such as: use
re-applied the idf -scaling at each level of recursive clustering and
(1480), develop (1282), paper (1251), process (1155), user (1130), studi
removed the non-informative words with at most one occurrence
(979), research (949), approach (892), base (848), and method (774).
within the analysed subset of documents. For the original document
After adding the manual stopwords, the word matrix was ready for
set, we had occurrences of 8672 words, which were then reduced
clustering the abstracts.
to 4760 words of at least two occurrences. During the course of

2
PD Literature Mapping PDC ’18, August 20–24, 2018, Hasselt and Genk, Belgium

Algorithm 1 Hierarchical application of prototype-based docu- as boxes, where the number represents the size of a cluster (number
ment clustering of articles). The clusters containing more than 300 articles, and thus
Input: Set of documents with bag-of-words encoding. chosen for re-clustering, are emphasised with bold lines. Unique
Output: Set of prototypes and cluster labels for different refinement levels, articles, that were not assigned to any cluster, are emphasised with
through dotted lines. The clusters that are vertically aligned belong to the
1: Remove words with at most one instance and apply idf -scaling. same refinement level. For example, clustering of the original docu-
2: Cluster the current document set using K-medians++ with 1000 restarts ment set provided six clusters of sizes 141, 123, 1984, 125, 103, and
for K = 2, . . . , 10. Check CVAIs and select number of clusters accord- 377, of which the third and the sixth were reclustered.
ing to the recommendation by one or more cluster indices.
3: Recursively recluster those document clusters which contain more than
300 documents. 3.1 Education-related clusters
Interpretation of document clusters was based on interactive expert
analysis of the most frequent words and the most representative
recursive clustering, we fixed the threshold of 300 documents as documents. First we assessed the 20 most common words of a clus-
the minimum size of a cluster, which was exposed to refinement. ter, how much of the total occurrences in the whole document set
In an unsupervised clustering scenario, the number of clusters (K) they cover (in percentages). Then, similarly to the BOW duplicate
is unknown and needs to be estimated. The quality of the clustering detection, we scanned through the titles, abstracts, and keywords
result can be measured with the so-called Clustering Validation of ten documents in the cluster, closest to the cluster prototype. In
Indices (CVAI), which can be divided into three categories [32]: this way, out of the whole set, we identified six clusters related to
internal, external, and relative. Internal CVAIs, which do not utilise the joint theme of PD in education (Table 1).
any external knowledge, typically measure the compactness and
separation of clusters. To estimate the number of clusters, we used 3.2 Word portions of education-related clusters
suggestions from the l 1 -distance modified set of the best internal The preprocessor provided a list of all words in the data, sorted
CVAIs, as identified in the tests in [15, 17]: K times the clustering from the highest frequency to the lowest. From this list, we selected
error (kCE) [17], Wemmert-Ganciarsky (WG) [12, 13], Ray-Turi (RT) education related words with the frequency of 30 or higher. Table
[24], Calinski-Harabasz (CH) [10], and Pakhira-Bandyopadhyay- 2 presents these words (Word), word frequency in the whole data
Maulik (PBM) [23]. As an example, CH and RT suggested six clusters (Freq), and number of articles that include the word at least once
for the original document set. (AF). Then, the table displays the word portions of all six education
The experiments were carried out in the Matlab-environment, by related clusters. The number shows how many percentages the
using the available kmeans clustering algorithm with the ‘Cityblock’ cluster covers from the total word frequency. Thus, the final column
distance and 1000 repetitions. The overall document clustering (Total) shows how many percents the six education related clusters
method is summarised in Algorithm 1. cover from total frequency all together.
2.4 Duplicate identification
4 DISCUSSION
After the clustering, we assessed the BOW representation of the
Due to the restricted space of a short paper, this study concentrated
documents to identify possible duplicates. We studied the closest
on demonstrating the semi-automatic clustering method for the sys-
document match of the first 100 documents with the Euclidean
tematic literature mapping of PD studies. In the future, we provide
distance by manually checking whether this was a duplicate. In
more detailed analysis of the clusters and the most representative
this way, we detected the first three duplicates, along with their
articles. Furthermore, we analyse articles that the algorithm could
distances to the ’host’ document. We used the maximum of these
not assign to any cluster, because they may represent some ex-
distances, 3.5, as the basis of threshold 3.6 when scanning through
ceptional studies in PD field. In addition, we provide an access to
all 2939 documents. If we detected a document, of which the dis-
the collected literature by implementing a web interface that uses
tance was less or equal than the threshold, we checked manually
Mendeley API 1 .
both the titles and the abstracts of such documents.
The main challenge of the systematic mapping was that data
Even when we had used the Mendeley duplicate removal tool
collection stage took a lot of manual work. This was due to the
in data collection stage, we now identified 86 duplicates. Majority
faced usability problems in the used academic search engines. For
of duplicates occurred because two different search engines had
illustration: ACM Digital Library did not provide abstracts when
exported titles in a slightly different form. Another source of differ-
exporting multiple references, so we copied them manually. In
ence, that was not detected by Mendeley but revealed here, was that
EBSCOhost, references needed to be moved to a folder (50 at a time)
the title or abstract of the same article was written in two different
and downloaded, with a limit of 150 references at a time. In JSTOR,
languages. However, and slightly alarmingly, we identified cases
references needed to be selected by clicking a checkbox, one by one,
where the same abstract, or almost the same, was used in a different
and then exported. To overcome this kind of defiencies, academic
article, such as: [2] and [3], [8] and [9], [30] and [31].
search engines should improve and standardise their database meta-
information fields, search protocols, and exporting features.
3 FINDINGS
Figure 1 shows an overview of the total number of 53 clusters and
1 Mendeley
49 unique articles on 21 refinement levels. The clusters are marked API: http://dev.mendeley.com, retrieved 15.1.2018
3
PDC ’18, August 20–24, 2018, Hasselt and Genk, Belgium A. Tuhkala et al.

Figure 1: Cluster map

1
EDU 1
DATA
141 535 466 1
2853

123 5 1 464 1
EDU 3

1984 60 5 66 1 417 361 1 1 1 353 1

EDU 2 EDU 4
EDU 6 125 66 1 1 1 1 1 1 1 1 351 1

103 104 950 913 1 99 1 30 55 359 357 355 1 1 344 342 8

195 377 161 115 2 48 1 1 1 1 1 1

146 1 124 1 732 85 12 1 1 1 1 33

32 87 275 33 33 1 1 1 1 294

1 1505 41 1 69 1 1 331 329 1

1 1 1

1 28 EDU 5 1

1 1

Table 1: PD in education clusters

Cluster Articles Journal / conference Name

EDU 1 141 79 / 62 PD in learning, learning technology including edugames, and learning environment design
EDU 2 125 59 / 66 Designing with children and for children (also with special needs)
EDU 3 60 29 / 31 Game design and gaming in design
EDU 4 66 38 / 28 Teaching PD and PD in learning design
EDU 5 28 18 / 10 PD with students and for educational activities
EDU 6 195 100 / 95 Educational and assistive technology design

Table 2: Word portions of the PD in education clusters

Word Freq AF EDU 1 EDU 2 EDU 3 EDU 4 EDU 5 EDU 6 Total

learn 1372 590 55.4 5.4 1.7 4.3 0.6 4.2 71.6
educ 814 392 16.8 5.4 3.3 13.2 9.0 12.4 60.1
student 791 265 24.9 0.8 1.8 41.3 1.7 5.1 75.6
school 418 184 16.3 15.3 4.1 10.5 4.8 8.2 59.2
teacher 390 151 38.8 7.1 4.5 11.6 2.4 9.2 73.6
teach 172 101 27.9 1.2 4.2 19.4 1.2 6.1 60.0
learner 138 70 56.3 3.7 2.2 5.2 1.5 4.4 73.3
classroom 115 55 36.9 5.8 1.0 12.6 8.7 6.8 71.8
instruc 120 64 32.2 2.5 3.4 11.9 2.5 2.5 55.0
pedagog 45 33 42.2 13.3 4.4 8.9 0.0 15.6 84.4
curriculum 42 30 26.2 0.0 7.1 11.9 4.8 0.0 50.0
pedagogi 30 24 42.9 3.6 0.0 3.6 0.0 0.0 50.1
Mean 370.6 163.3 34.7 5.4 3.2 12.9 3.1 6.2 65.4

4
PD Literature Mapping PDC ’18, August 20–24, 2018, Hasselt and Genk, Belgium

REFERENCES Proceedings of the 8th International Conference on Educational Data Mining (EDM
[1] Charu C Aggarwal and Chandan K Reddy. 2013. Data clustering: algorithms and 2015). 156–163.
applications. CRC press. [28] Richard Van Noorden. 2014. Global scientific output doubles every nine years.
[2] Meghan Allen, Joanna McGrenere, and Barbara Purves. 2007. The Design and (2014).
Field Evaluation of PhotoTalk: A Digital Image Communication Application for [29] P. Wartiainen and T. Kärkkäinen. 2015. Hierarchical, prototype-based clustering
People. In Proceedings of the 9th International ACM SIGACCESS Conference on of multiple time series with missing values. In Proceedings of the European Sym-
Computers and Accessibility (Assets ’07). ACM, New York, NY, USA, 187–194. posium on Artificial Neural Networks, Computational Intelligence and Machine
https://doi.org/10.1145/1296843.1296876 Learning - ESAINN 2015. 95–100.
[3] Meghan Allen, Joanna McGrenere, and Barbara Purves. 2008. The Field Evaluation [30] Y. Yu and Z. Liu. 2006. Integrated scenario-based design method for interac-
of a Mobile Digital Image Communication Application Designed for People tive online teaching system. In 2006 International Conference on Computational
with Aphasia. ACM Trans. Access. Comput. 1, 1, Article 5 (May 2008), 26 pages. Inteligence for Modelling Control and Automation and International Conference
https://doi.org/10.1145/1361203.1361208 on Intelligent Agents Web Technologies and International Commerce (CIMCA’06).
[4] David Arthur and Sergei Vassilvitskii. 2007. K-means++: The advantages of 153–153. https://doi.org/10.1109/CIMCA.2006.128
careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on [31] Y. Yu and Z. Liu. 2006. Research on a user-centered design method for interac-
Discrete algorithms. Society for Industrial and Applied Mathematics, 1027–1035. tive online teaching system. In 2006 International Conference on Communication
[5] Sami Äyrämö. 2006. Knowledge Mining Using Robust Clustering. Jyväskylä Studies Technology. 1–4. https://doi.org/10.1109/ICCT.2006.341739
in Computing, Vol. 63. University of Jyväskylä. [32] Mohammed J Zaki and Wagner Meira Jr. 2014. Data Mining and Analysis: Funda-
[6] Ricardo Baeza-Yates, Berthier Ribeiro-Neto, et al. 1999. Modern information mental Concepts and Algorithms. Cambridge University Press.
retrieval. Vol. 463. ACM press New York.
[7] Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing
with Python: analyzing text with the natural language toolkit. O’Reilly Media, Inc.
[8] Tone Bratteteig and Ina Wagner. 2010. Spaces for Participatory Creativity. In
Proceedings of the 11th Biennial Participatory Design Conference (PDC ’10). ACM,
New York, NY, USA, 51–60. https://doi.org/10.1145/1900441.1900449
[9] Tone Bratteteig and Ina Wagner. 2012. Spaces for participatory creativity. CoDe-
sign 8, 2-3 (2012), 105–126. https://doi.org/10.1080/15710882.2012.672576
[10] Tadeusz Caliński and Jerzy Harabasz. 1974. A dendrite method for cluster analysis.
Communications in Statistics-theory and Methods 3, 1 (1974), 1–27.
[11] John Collins and Kazunori Okada. 2012. A Comparative Study of Similarity
Measures for Content-Based Medical Image Retrieval. In CLEF (Online Working
Notes/Labs/Workshop).
[12] Bernard Desgraupes. 2013. Clustering indices. University of Paris Ouest-Lab
Modal’X 1 (2013), 34.
[13] Germain Forestier, Pierre Gançarski, and Cédric Wemmert. 2010. Collaborative
clustering with background knowledge. Data & Knowledge Engineering 69, 2
(2010), 211–228.
[14] Kim Halskov and Nicolai Brodersen Hansen. 2015. The diversity of participatory
design research practice at PDC 2002–2012. International Journal of Human-
Computer Studies 74 (feb 2015), 81–92. https://doi.org/10.1016/j.ijhcs.2014.09.003
[15] Joonas Hämäläinen, Susanne Jauhiainen, and Tommi Kärkkäinen. 2017. Com-
parison of Internal Clustering Validation Indices for Prototype-Based Clustering.
Algorithms 10, 3 (2017), 1–14.
[16] Anil K. Jain. 2010. Data clustering: 50 years beyond K-means. Pattern Recognition
Letters 31, 8 (2010), 651–666.
[17] Susanne Jauhiainen and Tommi Kärkkäinen. 2017. A Simple Cluster Validation
Index with Maximal Coverage. In Proceedings of the European Symposium on
Artificial Neural Networks, Computational Intelligence and Machine Learning -
ESAINN 2017. 293–298.
[18] Shehroz S Khan and Amir Ahmad. 2013. Cluster center initialization algorithm for
K-modes clustering. Expert Systems with Applications 40, 18 (2013), 7444–7456.
[19] Peder Olesen Larsen and Markus von Ins. 2010. The rate of growth in scientific
publication and the decline in coverage provided by Science Citation Index. Sci-
entometrics 84, 3 (sep 2010), 575–603. https://doi.org/10.1007/s11192-010-0202-z
[20] S. Lloyd. 1982. Least squares quantization in PCM. IEEE Transactions on Informa-
tion Theory 28, 2 (1982), 129–137.
[21] Paavo Nieminen, Ilkka Pölönen, and Tuomo Sipola. 2013. Research literature
clustering using diffusion maps. Journal of Informetrics 7, 4 (oct 2013), 874–886.
https://doi.org/10.1016/j.joi.2013.08.004
[22] Eunice P. S. Nunes, Alessandro R. Luz, Eduardo M. Lemos, and Clodoaldo Nunes.
2016. Approaches of Participatory Design in the Design Process of a Serious Game to
Assist in the Learning of Hospitalized Children. Springer International Publishing,
Cham, 406–416. https://doi.org/10.1007/978-3-319-39513-5_38
[23] Malay K Pakhira, Sanghamitra Bandyopadhyay, and Ujjwal Maulik. 2004. Validity
index for crisp and fuzzy clusters. Pattern recognition 37, 3 (2004), 487–501.
[24] Siddheswar Ray and Rose H Turi. 1999. Determination of number of clusters in
k-means clustering and application in colour image segmentation. In Proceedings
of the 4th international conference on advances in pattern recognition and digital
techniques. Calcutta, India, 137–143.
[25] Mirka Saarela, Joonas Hämäläinen, and Tommi Kärkkäinen. 2017. Feature Rank-
ing of Large, Robust, and Weighted Clustering Result. In Pacific-Asia Conference
on Knowledge Discovery and Data Mining. Springer, 96–109.
[26] Mirka Saarela and Tommi Kärkkäinen. 2015. Analysing Student Performance
using Sparse Data of Core Bachelor Courses. JEDM-Journal of Educational Data
Mining 7, 1 (2015), 3–32. http://www.educationaldatamining.org/JEDM/index.
php/JEDM/article/view/JEDM056
[27] Mirka Saarela and Tommi Kärkkäinen. 2015. Do country stereotypes exist in
educational data? A clustering approach for large, sparse, and weighted data.. In

5
 PII

INVOLVING TEACHERS IN PARTICIPATORY DESIGN:

SYSTEMATIC MAPPING OF STUDIES
BETWEEN 2007 AND 2017

by

Tuhkala, Ari 2019

Submitted
Involving Teachers in Participatory Design: Systematic Mapping of Studies
between 2007 and 2017

Ari Tuhkala∗
University of Jyväskylä, P.O. Box 35 (Agora), FI-40014 Jyväskylä, Finland

Abstract
Participatory design has been proposed as a potential approach to involve teachers in designing learning
innovations. This study examines this topic by conducting a systematic literature mapping of participatory
design studies between 2007 and 2017 that have involved teachers. A representative set of studies was
collected from 14 search engines and databases. The studies were mapped into specific categories and syn-
thesised to produce an overview of the current research. The main categories were environments, practices,
and technologies. The studies in the environments category involved teachers in designing physical build-
ings, as well as technologies integrated into the environment. The practices category considered professional
communities, instructional planning, and professional development programmes. The technologies category
considered assessment and monitoring tools, educational games, learning and teaching applications, security
and safety technology, and technology for special needs. This mapping serves as a solid basis for future
research on involving teachers in participatory design.
Keywords: participatory design, systematic literature mapping, teachers, education

1. Introduction

Realising the possible benefits of educational innovations relies greatly on teachers (OECD, 2015; Euro-
pean Commission, 2018). However, while teacher’s role is acknowledged as implementer, teachers are often
denied the opportunity to influence what is being implemented (Cviko et al., 2014; Kyza and Nicolaidou,
2017). For example, when regarding technological innovations in education, a systematic literature review in
Computers and Education shows that of 352 studies, only 30 percent involved teachers, and only 24 percent
involved stakeholders in co-designing technology, whether they were teachers or not (Pérez-Sanagustı́n et al.,
2017, p. A11).
A recent book, Participatory Design for Learning, presented participatory design (PD) as an untapped
resource for improving the development, implementation, and sustainability of learning innovations through
direct input from educational stakeholders (DiSalvo et al., 2017). PD is a well-known approach that advo-
cates for involving stakeholders as decision-makers in design, instead of mere informants, through collab-
oration and mutual learning (Kensing and Blomberg, 1998; Simonsen and Robertson, 2013; Halskov and
Hansen, 2015; Frauenberger et al., 2015). PD is most familiar in the field of Human–Computer Interaction,
but it has been found appealing also in educational context. This has resulted in special issues about PD
in Instructional Science (Könings et al., 2014), Cognition and Instruction (Bang and Vossoughi, 2016), and
European Journal of Education (Könings and McKenney, 2017).
Previous investigations about PD in education have focused on students, especially when developing
technology for students with special needs (Druin, 2002; Iversen and Dindler, 2013; Benton and Johnson,
2015). While there are studies that focus on teachers, the findings are scattered across several publication

∗ Corresponding author
Email address: [email protected] (Ari Tuhkala )

Preprint submitted to Elsevier May 27, 2019

venues and from different fields. Therefore, this study examined: For what purpose have PD studies
involved teachers? This question was answered by conducting a systematic mapping of all PD studies
between 2007 and 2017 that have involved teachers as main stakeholders. A comprehensive set of PD
studies was collected from 14 academic search engines and databases [blinded reference]. The studies were
scrutinised with rigorous inclusion and exclusion criteria and tabulated into specific categories (Kitchenham
and Charters, 2007; Kitchenham et al., 2010). These categories were synthesised to produce an overview
of the current body of knowledge regarding teacher involvement in PD. Research purposes, methods, and
main findings of individual studies are included in the appendix.

2. Participatory design

There are many design-related research paradigms. Well-known examples, especially in the educational
context, include design-based research (Wang and Hannafin, 2005), educational design research (Reeves,
2015), and human-centred design or user-centred design (Iivari and Iivari, 2006; Steen, 2011). Common to
these paradigms is to consider design, practice, and research together: design as envisioning new objects and
representing them by drawing, modifying tangible materials, or utilising digital tools; practice as solving
concrete problems through iterative cycles of design, development, enactment, and analysis; and research as
collecting and analysing data to examine the outcomes.
PD is positioned here in relation to the framework of design paradigms that has been defined by Sanders
and Stappers (2008). This framework is presented in Figure 1. The horizontal axis has two edges: user as
an informant and user as a design partner. It depicts two different ways to configure user participation in
design. In the former, users are investigated – how they perceive and react to different materials, forms,
and stimuli – to modify the functionality of the designed object. In the latter, users are active agents
who influence and make decisions in design. The edges on the vertical axis are design driven and research
driven. This depicts which outcome is the more predominant purpose of user participation: design objects
or construct knowledge. For example, involving users for purely artistic purposes would be design-dominant,
and involving users only for data collection purposes would be research-dominant.

Figure 1: Landscape of design paradigms (adapted from Sanders and Stappers, 2008, p. 6).

Design
driven

Participatory design
User-centred design

Contextual
Usability testing
inquiry

Scandinavian
Applied participatory
Ergonomics design
Research ethnography
driven
User as an User as a design
informant partner

User-centred design is a broad category for approaches in which the user is the object of investigation.
In usability testing, users test new designs and the objects are modified based on these findings. This can
also be based on existing knowledge, such as when designing furniture based on human anatomy, as in

2
ergonomics. In contextual inquiry, design is informed by interviewing users about use practices, wherein
applied ethnography the users are observed. As such, all these approaches are research-driven. For example,
it is difficult to see how contextual inquiry could be separated from the user interviews without losing its
very purpose. Similarly, PD can be understood as a broad category. The reason for comparing these two
categories is to highlight the differences: while the focus in user-centered design is to investigate users to
develop new solutions, the focus in PD is to create a space where users, designers, and researchers explore
the problem and envision the solutions together (Leong and Iversen, 2015).
Scandinavian participatory design can be characterised as an action-research inspired approach, which
means that both design and research are driven by local accountability: to support the local stakeholders
instead of just producing objects and knowledge (see Simonsen and Robertson, 2013, p. 44). This approach
was rooted in the workplace democracy movement in Scandinavia that emerged in the late 1960s and 1970s
(Bjerknes and Bratteteig, 1995; Simonsen and Robertson, 2013; Bødker and Kyng, 2018). The movement
addressed concerns over employees’ opportunities to influence how, and with what implications, computer
systems were introduced at their workplaces (Beck, 2002). The foundations were built in early action
research projects that demanded more democratic working conditions and developed new technology, work
practices, and design methods (Gregory, 2003; Iversen et al., 2012; Bødker and Kyng, 2018).
Nowadays, PD researchers and practitioners form an established and multidisciplinary community that
is more or less committed to the ideals of the Scandinavian tradition (Vines et al., 2015). These ideals are
familiar from a book called Computers and Democracy: a Scandinavian challenge (Bjerknes et al., 1987),
which was described as a genuinely human-centered alternative to technology design (Suchman, 1988): First,
design addresses the contradiction between tradition and transcendence – what is and what could be – so
design can either support old practices, values, and power structures or aim to change them (Ehn, 1988).
Second, people who are affected by design should be involved in making decisions about it (Greenbaum,
1993; Simonsen and Robertson, 2013). Topics like democratic decision-making and empowering marginalised
people are still in current debate, especially in the community’s main venue: the biannual Participatory
Design Conference. At the same time, the pragmatic side of PD, that is developing techniques for involving
stakeholders, has pervaded other design approaches as well (Bødker et al., 2000; Spinuzzi, 2002; Bødker
and Kyng, 2018). This has led to an intensive discussion of what are, or should be, the contemporary
characteristics of PD (Halskov and Hansen, 2015; Smith et al., 2017; Bødker and Kyng, 2018; Pilemalm,
2018).

3. Systematic mapping process

Data collection took place over 14 search engines and databases to gather a wide sample of PD literature:
ACM Digital library, Bielefeld Academic Search Engine, EBSCOhost Research Databases, ERIC Institute of
Education Sciences search, IEEE Xplore Digital library, JSTOR, ProQuest, SAGE Journals, ScienceDirect,
Scopus, SpringerLink, Taylor and Francis Online, Wiley Online Library, and Thomson Reuters Web of
Science. Criteria for database selection were the possibility to 1) export multiple references and 2) export
references in Mendeley supported format (RIS, Bibtex, Endnote XML, or Zotero). Thus, Google Scholar,
Semantic Scholar, and CiteSeerX were excluded.
References where participatory design appeared in title, abstract, or keywords, and which were published
between 2007 and 2017 were collected. The references from different search engines were imported to the
Mendeley reference management tool to build a single reference database (2943 articles). Because two
references could have different meta-information and still point to the same source, the duplicates were
removed by using the duplicate identification tool. From the reference database, the references that included
the word teacher either in title, abstract, or keywords were extracted. In total, 191 references included both
teacher and participatory design.
The data refinement process is illustrated in Figure 2. In the first exclusion stage, the references were
screened and those not published in a journal or conference, written in English, or where there was no access
to full text in any of the 14 databases were removed (55 articles). The excluded references consisted of
workshop descriptions, research proposals, posters, extended abstracts, introductions to special issues, and
editorial notes.
3
 Figure 2: Systematic literature mapping protocol.

Original data (191)

1) participatory design and teacher appear in title, abstract, or keywords.
2) published between 2007 and 2017

Exclusion 1: not journal or conference articles, not in English, no access to full text (55)

Exclusion 2: includes the word teacher, but does not consider teachers (17)

Exclusion 3: teachers were not participants in the study (38)

Exclusion 4: if conference and journal paper about same study, conference removed (9)

Mapped data (72 studies)

| author(s) | year | venue | teachers | education level | domain | location | research objective |
methods | findings |

Before the second exclusion stage, full text articles for each remaining reference were downloaded. Articles
that included the words participatory design but defined the study as action research, user-centred design,
or another research approach were eliminated for being beyond the scope of the present study (17 articles).
In the third exclusion stage, the articles that did not involve teachers as participants were removed (37
articles). For example, Hussain (2010) stated that valuable user perspectives are lost if only information from
adult carers such as teachers and parents are included in the design process but did not consider teachers
any further. Studies that did not define the participants in detail, but obviously considered teachers, were
included. The final exclusion stage (9 articles) removed conference articles where the same research was
reported in a journal article.
The following basic information was then extracted to an Excel sheet after reading all the articles: au-
thor(s), year, venue, participants, amount of teachers as participants, other stakeholders, education level,
and geographical location. The articles were scrutinised in detail and mapped according to the research ob-
jectives, methods, and main findings. These results were analysed to assign the articles into three categories:
environments, practices, and technologies. These categories were further refined into sub-categories.

4. Results

A summary of the studies involving teachers in PD is presented in Table 1. Most of the studies were
published in journals (51). The studies considered several education levels: pre-primary, primary, secondary,
and higher education. Moreover, seven studies were about PD in teacher education, and six investigated PD
in more than one education level. Most of the studies were small-scale investigations such as case studies, with
no more than five teacher participants. Seven large-scale studies consulted 20 or more teachers. However,
some studies did not specify the exact number of teacher participants. Over half the studies were conducted
in the United Kingdom, Netherlands, United States, Australia, and Finland, and five studies covered more
than one geographical location.

Environments

Table 2 presents the studies related to learning environments. In the largest category, School buildings,
teachers were involved in envisioning a new school concept or re-design of an existing concept (Burke and
Könings, 2016; Koutamanis et al., 2017; Könings et al., 2017; van Merriënboer et al., 2017; Woolner et al.,
2007, 2010). These studies approached the school building as a whole, including furniture, (technological)
equipment, materials, and structures, whereas two of the studies focused on specific facilities: the university
4
 Table 1: Summary of participatory design studies involving teachers (n = 72).
Venue Education level Number of teachers Location

Journal: 51 Pre-primary: 2 Not defined: 22 United Kingdom: 16

Conference: 21 Primary: 22 1 – 5: 25 Netherlands: 10
Secondary: 19 6 – 10: 9 United States: 7
Higher: 16 11 – 20: 9 Australia: 5
Teacher education: 7 Over 20: 7 Finland: 4
Several levels: 6 Rest of Europe: 16
Asia: 6
Rest of the world: 3
Multiple locations: 5

cafeteria (Lundström et al., 2016), and library learning commons (Somerville and Collins, 2008). In the
second category, Technology-enhanced learning spaces, the studies focused not only on the physical space
but also on how technology is integrated into the learning environment.

Table 2: Teachers in participatory design of environments (n = 15).

Category Articles

School buildings Burke and Könings (2016), Koutamanis et al. (2017), Könings et al. (2017), Lundström
et al. (2016), van Merriënboer et al. (2017), Somerville and Collins (2008), Woolner et al.
(2007), Woolner et al. (2010)
Technology-enhanced learning Bossen et al. (2010), Casanova and Mitchell (2017), Cober et al. (2015), Joyce et al.
spaces (2014), Kreitmayer et al. (2013), Otero et al. (2013), Stephen et al. (2014)

Regarding the studies in the School buildings category, Burke and Könings (2016) examined how a school’s
history inspired the participants’ design imagination. They present an example from the Netherlands, De
Werkplaats, a school that was re-designed according to the educational thinking of Kees Boeke. They point
out how a historical narrative can be utilised as a positive agent for change, but also that previous traditions
from more conservative schools can limit and hinder the potential for design innovations.
Two other studies took place at De Werkplaats. As reported by Koutamanis et al. (2017), visual infor-
mation technology (Building Information Model) was utilised as a collaborative tool during the building’s
life-cycle. The tool served as a knowledge repository and a communication service, which enabled the par-
ticipants to engage in decision-making. van Merriënboer et al. (2017), in turn, addressed the relationship
between pedagogy and physical spaces. They framed a three-stage design process: specifying the peda-
gogy, aligning the chosen pedagogy with seating arrangements and physical learning spaces, and realising
the school building. They found PD especially beneficial when teachers’ pedagogical needs and architects’
non-pedagogical needs (resources, cost-effectiveness) contradicted (Woolner et al., 2007). van Merriënboer
et al. (2017) conclude that the PD of school buildings is not about the building per se, but negotiating a
shared pedagogical vision and establishing a commitment to this vision.
Two methodological contributions deserve to be highlighted. Woolner et al. (2010) accounted for using
visual tools, such as photo elicitation, diamond mapping, and map-based activities, to gather perceptions of
various participants and to improve the learning environment. They concluded that the visual methods pro-
duced rich understandings of the current school environment and enabled the triangulation of participants’
different perceptions. The second study developed an interdisciplinary model of practice for participatory
building design (Könings et al., 2017). The model integrates an action research cycle, stakeholder analysis
model, ladder of participation tool, and participation matrix to address the complexity of involving several
different stakeholder roles.
In the Technology-enhanced learning spaces category, three studies developed technologies to be inte-
grated into classrooms: an Internet of Things ecosystem (Joyce et al., 2014), UniPad application (Kreit-
5
mayer et al., 2013), and digital displays (Otero et al., 2013). Two studies involved teachers in designing new
learning environments where technology plays a major part. Casanova and Mitchell (2017) provided partici-
pants with two provocative design space concepts, which were then re-designed. This process resulted in rich
data about how the participants conceptualised the learning spaces and the value of technology. Similarly,
Stephen et al. (2014) involved teachers and students to design technology-rich classrooms as community
spaces that are owned and maintained together.
Two studies are described in detail because they pay specific attention to teachers’ participation. Bossen
et al. (2010) accounted for a large PD project, iSchool, which was about envisioning new learning spaces and
opportunities of pervasive technology. They interviewed the teachers three years after the project ended and
examined what they gained from the project. According to the teachers, the most satisfying experiences were:
reflecting with professionals from other backgrounds, the enthusiasm of the students towards technology,
and gaining experience from using modern technology. Moreover, the teachers expressed four types of gains:
opportunity to reflect on teaching methods, to develop skills and understandings about technology, to have
leverage to influence technology-related decisions, and to advance their own interest in technology.
Cober et al. (2015) analysed teacher engagement in two case studies and investigated what supports
teacher participation. The teachers engaged in: theory-driven discussions with researchers and developers
to ground design work and understand each others’ perspectives; design partnerships by providing input,
guidance, and ideas; reflecting on the innovations from a pedagogical perspective and evaluating the potential
impact for students; and adjusting implementation enactments. Regarding the conditions supporting the
teachers, the authors highlighted a combination of highly facilitated conditions and flexibility, an atmosphere
of trust and partnership, and designing with contextual knowledge about the physical environment, students,
and potential technologies.

Practices

The studies in the Learning practices category were about establishing professional communities, intstruc-
tional design, and professional development programmes (Table 3).

Table 3: Teachers in participatory design of practices (n = 29).

Category Articles

Communities Booker and Goldman (2016), Duell et al. (2014), Farooq et al. (2007), Ishimaru and
Takahashi (2017), Karimi et al. (2017), Pollock and Amaechi (2013), Selwyn et al. (2017),
Tammets et al. (2011), Vakil et al. (2016)
Instructional design Anderson and Östlund (2017), Barbera et al. (2017), Gros and López (2016), Harrison
et al. (2017), Janssen et al. (2017), Kuure et al. (2016), Könings et al. (2011), Könings
et al. (2010), Könings et al. (2007b, A), Könings et al. (2007a, B), Prins et al. (2016)
Professional development Al-Eraky et al. (2015), Goeze et al. (2014), Janssen et al. (2014), Kyza and Georgiou
(2014), Kyza and Nicolaidou (2017), Põldoja et al. (2014), Rodrigo and Ramı́rez (2017),
So et al. (2009), Tulinius et al. (2012)

In the Professional communities category, two studies examined online communities. Duell et al. (2014)
established a yearly ambassador programme for introducing design thinking as a general competency in K-12
education in Australia. In this study, PD was undertaken to create an online design education platform
and to increase teachers’ capacity to teach creativity and design. Farooq et al. (2007) developed an online
environment for a diverse community of distributed education professionals. The project drew on PD and
included four design interventions. The study proposed that the interventions were successful because the
community members developed ownership over the online environment and kept using it for long-term
professional development and social networking. Karimi et al. (2017) organised a hackathon workshop for
teachers, where the teachers experimented with technology and designed learning activities. Because the
teachers faced challenges implementing the digital technology projects, it provided an honest experience of
exploring novel technologies and demystifying some aspects of technical practices.
6
 The other studies in this sub-category were about empowering local communities. Booker and Goldman
(2016) examined PD as an approach to tackle math fears in families by restoring epistemic authority. Pollock
and Amaechi (2013) explored how texting supports rapid and individualised communication with vulnerable
youth. Selwyn et al. (2017) explored the possibilities of making existing school data available in digital form
for teachers, students, and administrators. This study revealed technical, informational, organisational, and
social issues in democratising data engagement within school settings. Tammets et al. (2011) examined
a teacher accreditation programme that requires the teachers to be involved in community of practices,
collaborative learning, and knowledge building. Finally, Vakil et al. (2016) used the notion of politicised
trust to analyse how political and racial solidarity was established, contested, and negotiated throughout
two PD projects.
The Instructional design category consists of studies about designing learning practices and curricula.
Most of the studies were conducted by the same researchers from the Netherlands. Könings et al. (2007b,
A) aimed to reduce discrepancies between students’ and teachers’ perceptions on appropriate learning en-
vironments and to collaboratively design these environments. Könings et al. (in 2007a, B) expanded this
work by focusing on teachers. In two other studies (Könings et al., 2010, 2011), the authors invited students
to collaborate with teachers. Both the teachers and students found PD appealing in this context, but with
several challenges: PD takes too much time, students underestimate their capability to decide educational
issues, teachers doubt students’ willingness to take part in PD, and PD outcomes were perceived positively
by the students involved but not by the rest of the class.
Janssen et al. (2017) defined participatory educational design and conducted a study with three aims: to
view classroom teaching as bounded rational design, develop a tool that supports participants in mapping
and sharing their goals, and develop another tool that helps participants to explore practical and effective
possibilities for designing learning environments. This study demonstrated the use of tools for improving the
quality and usability of learning environments and stated that even participants with similar backgrounds
benefited from learning about each others’ practices and goals.
The remaining studies in this sub-category considered a variety of topics. Barbera et al. (2017) developed
learning scenarios to identify moments of change and describe causes and agents that motivate these changes.
Gros and López (2016) examined the Learning Centric Ecology of Resources model to facilitate co-design
processes. Two studies considered assessment in teaching: Harrison et al. (2017) explored how to redesign
a summative assessment culture that takes into account students’ post-assessment feedback, and Anderson
and Östlund (2017) considered assessment practices of students who attend special schools. Janssen et al.
(2014) developed a PD-based teacher training trajectory for guided discovery learning (GDL) lessons in
biology. Accordingly, the teachers were willing and capable of implementing GDL, utilised the heuristics
that were developed by experienced teachers, and valued GDL at a higher level than regular lessons. Kuure
et al. (2016) supported English teachers in a Finnish university to become designers of language learning with
new technologies, and Prins et al. (2016) developed an instructional framework that provided educational
designers with a set of prescriptive guidelines for transforming authentic modelling practices.
Two studies in the Professional development category were about improving professional development
through PD. Kyza and Georgiou (2014) examined PD for promoting teachers’ sense of ownership towards
inquiry-based learning modules. Accordingly, the teachers perceived PD as a collaborative and support-
ive framework that enables the exchange of different perspectives, encourages critical constructivism, and
facilitates new teaching methods and technologies. Conversely, the time-consuming nature of PD, com-
munication problems, and participants’ unequal contributions were identified as the main disadvantages.
Despite this, all teachers preferred designing the teaching module over using pre-made modules. Kyza and
Nicolaidou (2017) conclude that iterative design facilitated teachers’ professional development because it
enabled teachers to reflect on inquiry learning and teaching.
The remaining studies in this category were about training programmes. Al-Eraky et al. (2015) involved
teachers in designing a faculty development programme for teaching professionalism in medical education,
Rodrigo and Ramı́rez (2017) developed a master course for online teaching, and Tulinius et al. (2012)
designed a programme for teachers to obtain critical appraisal skills and higher academic capacity. Four
studies were about developing digital platforms for professional development: Goeze et al. (2014) examined
how video case-based learning could promote teachers’ analytical competence to become immersed and
7
to adopt multiple perspectives, to apply conceptual knowledge, and to describe pedagogical situations.
Similarly, Põldoja et al. (2014) addressed the design challenges of a software solution for self- and peer-
assessing teachers’ digital competencies. Finally, So et al. (2009) designed an online platform where teachers
can share vivid images of their practices with their peers.

Technologies

The studies regarding learning technologies were assigned to the following categories: Assessment and
monitoring tools, Educational games, Learning and teaching applications, safety and security tools, and
Technology for special needs (Table 4).

Table 4: Teachers in participatory design of technologies (n = 28).

Category Articles

Assessment and monitoring Gillies et al. (2015), Rodrı́guez-Triana et al. (2012), Siozos et al. (2009)
tools
Educational games Hoda et al. (2014), Klonari and Gousiou (2014)
Learning and teaching applica- Carmichael (2015), Cramer and Hayes (2013), Girard and Johnson (2010), Hannon et al.
tions (2012), Kalra et al. (2007), Pedersen et al. (2012), Rahamat et al. (2011), Song and Oh
(2016), Su et al. (2010), Triantafyllou and Timcenko (2013)
Safety and security Ervasti et al. (2016), Jutila et al. (2015), Pantsar-Syväniemi et al. (2015)
Technology for special needs Abdullah and Brereton (2015), Bossavit and Parsons (2016), Brereton et al. (2015),
Medeiros-Braz et al. (2017), Fage et al. (2016), Herstad and Holone (2012), Lingnau and
Lenschow (2010), Parsons et al. (2011), Parsons and Cobb (2014), Zainuddin et al. (2010)

In the Assessment and monitoring tools category, Gillies et al. (2015) developed an application for giving
feedback about students’ playing posture in music education. They created a prototype, then asked teachers
for feedback before developing next version for evaluation. Rodrigo and Ramı́rez (2017) developed computer-
supported collaborative learning scenarios for monitoring students’ interactions. Siozos et al. (2009) reported
positive outcomes after involving teachers and students in designing computer-based assessment tools: both
teachers and students perceived PD as an opportunity to re-conceptualise existing pedagogies and that PD
supported locality, diversity, participation, and attitudes that counter impassivity and homogenisation.
Two of the studies were about Educational games. Hoda et al. (2014) involved teachers as part of
a multidisciplinary team that designed a game for supporting reciprocal teaching and collaboration with
children. They evaluated and refined the game through functional testing, teacher trials, and children–
teacher trials. As an outcome, the game was perceived as engaging and easily understood by young children.
Klonari and Gousiou (2014) described a game for helping teachers to become aware of their pedagogical
choices. The game itself was described in detail, but it remained unclear how the teachers engaged in the
design of the game.
The studies in the Learning and teaching applications category designed technology for dance and envi-
ronmental education (Carmichael, 2015), financial education (Cramer and Hayes, 2013), STEM education
(Hannon et al., 2012; Su et al., 2010), mathematics (Pedersen et al., 2012; Triantafyllou and Timcenko,
2013), and literature (Rahamat et al., 2011). Three studies examined the development of tutoring systems
(Girard and Johnson, 2010; Kalra et al., 2007; Song and Oh, 2016). The studies in this sub-category focused
on the technologies themselves. An exception was the study by Carmichael (2015), which criticised the
assumptions behind education technology development. That is, it concerned the risks of designing edu-
cational technology based on stereotypical views, such as digital natives, and losing sight of practice-based
knowledge.
All three studies in the Safety and security category related to designing a situation-aware safety service.
Jutila et al. (2015) examined the technological enablers and requirements for building a safety system,
Pantsar-Syväniemi et al. (2015) analysed the design process itself, and Ervasti et al. (2016) analysed the
8
feedback from children, parents, and teachers. Even though the design clearly focused on children, these
studies were able to bring together various perspectives from teachers and parents as well.
The largest category was Technology for special needs. The studies considered various special needs,
such as Autism Disorder, language delays, and cognitive and sensory impairments. Some studies focused
on identifying requirements for technology design (Lingnau and Lenschow, 2010; Zainuddin et al., 2010) or
describing how technology can support these needs (Abdullah and Brereton, 2015; Fage et al., 2016; Herstad
and Holone, 2012; Parsons et al., 2011). Medeiros-Braz et al. (2017) emphasised that teachers have valuable
knowledge about students’ special needs and can envision technologies to support students’ abilities and
learning possibilities. In contrast, Brereton et al. (2015) noted that teachers (and other adults) have their
own needs, and these can be different than the actual objectives of the students who have special needs.
Finally, two methodological contributions stand out in this category. Bossavit and Parsons (2016) utilised
a stakeholder analysis framework to reflect the design process of an educational game. The framework was
grounded in PD literature and used to map stakeholder roles, levels of engagement, design tools, and
decisions. Parsons and Cobb (2014) addressed the complexity of involving multiple stakeholders: in this
case, teachers and children with special needs. They discussed how the key challenge is to prioritise different
stakeholders and decisions. They argue that prioritising each stakeholder equally is impossible and question
if an outcome-focused agenda, which aims for efficient technology development, is even possible to combine
with the empowering approach of PD.

5. Conclusion and discussion

This systematic mapping produced an overview of the current research on PD involving teachers. The
mapped studies were assigned into three categories: environments, practices, and technologies. The studies
in the environments category involved teachers in designing physical buildings, as well as technologies in-
tegrated into the environment. The studies in the practices category considered professional communities,
instructional planning, and professional development programmes. The studies in the technologies category
designed assessment and monitoring tools, educational games, learning and teaching applications, security
and safety technology, and technology for special needs. However, it needs to be noted that producing the
categories is not deterministic. Some of the studies could have been assigned to any of the three main
categories. For example, So et al. (2009) examined teachers’ professional development under the category
of practices and could have incorporated the other two categories as well, because it involved online envi-
ronments and the development of an online video platform.
The mapping revealed valuable assets for conducting PD in educational context: visual tools for trian-
gulating participants’ different perspectives (Woolner et al., 2010), stakeholder analysis models (Könings
et al., 2017), a tool for mapping and sharing stakeholders’ goals (Janssen et al., 2017), and the stakeholder
analysis framework (Bossavit and Parsons, 2016). When regarding the involvement of teachers and students,
some studies recommended involving them together, while others proposed separating them: Casanova and
Mitchell (2017) stated that dividing students and teachers into separate groups offers a more pleasant envi-
ronment for both, whereas Woolner et al. (2007) warned that the teacher perspective may be pushed to the
background if students and teachers are in the same group. This would be an interesting topic to explore
further.
Based on the mapping, the advantages of PD for teachers include: possibility to reflect on teaching
through PD (Bossen et al., 2010), development of teacher ownership (Kyza and Georgiou, 2014), develop-
ment of supporting design conditions for teachers (Cober et al., 2015), and teacher’s professional development
(Kyza and Nicolaidou, 2017). These findings relate to the similar kind of benefits in research about collabo-
rative curriculum design in teacher design teams (Voogt et al., 2011). Another standpoint was how PD was
proposed as a way to deal with contradictions between teachers’ pedagogical needs and other stakeholders’
needs (van Merriënboer et al., 2017; Woolner et al., 2007) and to reduce discrepancies between teachers
and students (Könings et al., 2007b, 2010, 2011). However, especially the studies in technology category
relied on the assumption that PD would eventually benefit teachers through better learning environments,
practices, and technologies, instead of paying specific attention on what are the benefits for teachers.

9
 Literature review and mapping differ in purpose: mapping aims to gather a representative set of studies
using inclusion and exclusion criteria and tabulate these studies into specific categories (Kitchenham and
Charters, 2007; Kitchenham et al., 2010). The quality of this study can be assessed based on how expansive
and rigorous the mapping protocol is. The literature was collected from 14 academic search engines and
databases, which is a very wide scope for a single mapping study. The data collection procedure can be
repeated, which increases the reliability of the study. If the data were to be collected now (2019), some
studies would appear that are not in the original data. This is because new studies have been published since
data collection for the present study was concluded in 2018. The exclusion criteria for have been explained
in full and the reasons for exclusion recorded for each study, which makes external validation of the mapping
possible. This mapping provides a basis for multidisciplinary research about PD in educational context, as
been called for by DiSalvo et al. (2017).

10
References
Abdullah, M.H.L., Brereton, M., 2015. MyCalendar: Fostering Communication for Children with Autism Spectrum Disorder
Through Photos and Videos, in: Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer
Human Interaction on - OzCHI ’15, ACM Press, New York, New York, USA. pp. 1–9. doi:10.1145/2838739.2838785.
Al-Eraky, M.M., Donkers, J., Wajid, G., van Merriënboer, J.J., 2015. Faculty development for learning and teaching of medical
professionalism. Medical Teacher 37, S40–S46. doi:10.3109/0142159X.2015.1006604.
Anderson, L., Östlund, D., 2017. Assessments for learning in grades 1-9 in a special school for students with intellectual
disability in Sweden. Problems of Education in the 21st Century 75, 508–524.
Bang, M., Vossoughi, S., 2016. Participatory Design Research and Educational Justice: Studying Learning and Relations
Within Social Change Making. Cognition and Instruction 34, 173–193. doi:10.1080/07370008.2016.1181879.
Barbera, E., Garcia, I., Fuertes-Alpiste, M., 2017. A Co-Design Process Microanalysis: Stages and Facilitators of an Inquiry-
Based and Technology-Enhanced Learning Scenario. The International Review of Research in Open and Distributed Learning
18, 104–126. doi:10.19173/irrodl.v18i6.2805.
Beck, E.E., 2002. P for Political: Participation is Not Enough. Scandinavian Journal of Information Systems 14, 77–92. URL:
http://folk.uio.no/eevi/research/pub-papers/Beck2002aSJIS.PforPol.ps.
Benton, L., Johnson, H., 2015. Widening participation in technology design: A review of the involvement of chil-
dren with special educational needs and disabilities. International Journal of Child-Computer Interaction 3-4, 23–40.
doi:10.1016/j.ijcci.2015.07.001.
Bjerknes, G., Bratteteig, T., 1995. User participation and democracy: a discussion of Scandinavian research on system
development. Scandinavian Journal of Information Systems 7, 73–98.
Bjerknes, G., Ehn, P., Kyng, M., 1987. Computers and Democracy: A Scandinavian Challenge. Gower Press, Brookfield.
Bødker, S., Ehn, P., Sjögren, D., Sundblad, Y., 2000. Co-operative Design — perspectives on 20 years with ‘ the Scandinavian
IT Design Model ’, in: NordiCHI 2000, pp. 1–9.
Bødker, S., Kyng, M., 2018. Participatory Design that Matters—Facing the Big Issues. ACM Transactions on Computer-Human
Interaction 25, 1–31. doi:10.1145/3152421.
Booker, A., Goldman, S., 2016. Participatory Design Research as a Practice for Systemic Repair: Doing Hand-in-Hand Math
Research with Families. Cognition and Instruction 34, 222–235. doi:10.1080/07370008.2016.1179535.
Bossavit, B., Parsons, S., 2016. Designing an Educational Game for and with Teenagers with High Functioning Autism, in:
Proceedings of the 14th Participatory Design Conference: Full Papers - Volume 1, ACM, New York, NY, USA. pp. 11–20.
doi:10.1145/2940299.2940313.
Bossen, C., Dindler, C., Iversen, O.S., 2010. User gains and PD aims, in: Proceedings of the 11th Biennial Participatory Design
Conference on - PDC ’10, ACM Press, New York, New York, USA. p. 141. doi:10.1145/1900441.1900461.
Brereton, M., Sitbon, L., Abdullah, M.H.L., Vanderberg, M., Koplick, S., 2015. Design after design to bridge between people liv-
ing with cognitive or sensory impairments, their friends and proxies. CoDesign 11, 4–20. doi:10.1080/15710882.2015.1009471.
Burke, C., Könings, K.D., 2016. Recovering lost histories of educational design: a case study in contemporary participatory
strategies. Oxford Review of Education 42, 721–732. doi:10.1080/03054985.2016.1232244.
Carmichael, P., 2015. Not just about gadgets: Habit, innovation and change in the design of learning technologies. E-Learning
and Digital Media 12, 279–294. doi:10.1177/2042753015571052.
Casanova, D., Mitchell, P., 2017. The cube and the poppy: Participatory approaches for designing technology-enhanced learning
spaces. Journal of Learning Spaces 6, 1–12.
Cober, R., Tan, E., Slotta, J., So, H.J., Könings, K.D., 2015. Teachers as participatory designers: two case studies with
technology-enhanced learning environments. Instructional Science 43, 203–228. doi:10.1007/s11251-014-9339-0.
Cramer, M., Hayes, G.R., 2013. The digital economy, in: Proceedings of the 12th International Conference on Interaction
Design and Children - IDC ’13, ACM Press, New York, New York, USA. pp. 431–434. doi:10.1145/2485760.2485832.
Cviko, A., McKenney, S., Voogt, J.M., 2014. Teacher roles in designing technology-rich learning activities for early literacy: A
cross-case analysis. Computers & Education 72, 68–79. doi:10.1016/j.compedu.2013.10.014.
DiSalvo, B., Yip, J., Bonsignore, E., DiSalvo, C., 2017. Participatory Design for Learning: Perspectives from Practice and
Research. First ed., Routledge, New york.
Druin, A., 2002. The role of children in the design of new technology. Behaviour & Information Technology 21, 1–25.
doi:10.1080/01449290110108659.
Duell, C., Wright, N., Roxburgh, J., 2014. Developing ”Design Minds” for the 21st Century through a Public Sector Initiated
Online Design Education Platform. URL: http://ojs.lboro.ac.uk/ojs/index.php/DATE/article/view/1926/1968.
Ehn, P., 1988. Work-Oriented Design of Computer Artifacts. Ph.D. thesis. Umeå Universitat. URL:
http://www.diva-portal.org/smash/record.jsf?pid=diva2:580037&dswid=-8358.
Ervasti, M., Laitakari, J., Hillukkala, M., 2016. ‘I want to know where my child is at all times’ – field study of a location-aware
safety service for schoolchildren. Behaviour & Information Technology 35, 833–852. doi:10.1080/0144929X.2016.1201144.
European Commission, 2018. Communication on the Digital Education Action Plan. Technical Report. European Comission.
Brussels. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=COM:2018:22:FIN.
Fage, C., Pommereau, L., Consel, C., Balland, E., Sauzéon, H., 2016. Tablet-Based Activity Schedule in Mainstream
Environment for Children with Autism and Children with ID. ACM Transactions on Accessible Computing 8, 1–26.
doi:10.1145/2854156.
Farooq, U., Schank, P., Harris, A., Fusco, J., Schlager, M., 2007. Sustaining a Community Computing Infrastructure for Online
Teacher Professional Development: A Case Study of Designing Tapped In. Computer Supported Cooperative Work (CSCW)
16, 397–429. doi:10.1007/s10606-007-9049-0.

11
Frauenberger, C., Good, J., Fitzpatrick, G., Iversen, O.S., 2015. In pursuit of rigour and accountability in participatory design.
International Journal of Human-Computer Studies 74, 93–106. doi:10.1016/j.ijhcs.2014.09.004.
Gillies, M., Brenton, H., Yee-King, M., Grimalt-Reynes, A., D’Inverno, M., 2015. Sketches vs skeletons, in: Proceedings of
the 2nd International Workshop on Movement and Computing - MOCO ’15, ACM Press, New York, New York, USA. pp.
104–111. doi:10.1145/2790994.2790995.
Girard, S., Johnson, H., 2010. Designing affective computing learning companions with teachers as design partners., in:
Proceedings of the 3rd international workshop on Affective interaction in natural environments - AFFINE ’10, ACM Press,
New York, New York, USA. p. 49. doi:10.1145/1877826.1877840.
Goeze, A., Zottmann, J.M., Vogel, F., Fischer, F., Schrader, J., 2014. Getting immersed in teacher and student perspectives? Fa-
cilitating analytical competence using video cases in teacher education. Instructional Science 42, 91–114. doi:10.1007/s11251-
013-9304-3.
Greenbaum, J., 1993. PD a personal statement. Communications of the ACM 36, 47. doi:10.1145/153571.214816.
Gregory, J., 2003. Scandinavian Approaches to Participatory Design. International Journal of Engaging Education 19, 62–74.
Gros, B., López, M., 2016. Students as co-creators of technology-rich learning activities in higher education. International
Journal of Educational Technology in Higher Education 13, 28. doi:10.1186/s41239-016-0026-x.
Halskov, K., Hansen, N.B., 2015. The diversity of participatory design research practice at PDC 2002–2012. International
Journal of Human-Computer Studies 74, 81–92. doi:10.1016/j.ijhcs.2014.09.003.
Hannon, D., Danahy, E., Schneider, L., Coopey, E., Garber, G., 2012. Encouraging teachers to adopt inquiry-based learning by
engaging in participatory design, in: IEEE 2nd Integrated STEM Education Conference, IEEE, Department of Mechanical
Engineering/Human Factors, Tufts University, United States. pp. 1–4. doi:10.1109/ISECon.2012.6204169.
Harrison, C.J., Könings, K.D., Schuwirth, L.W.T., Wass, V., van der Vleuten, C.P.M., 2017. Changing the culture of assessment:
the dominance of the summative assessment paradigm. BMC Medical Education 17, 73. doi:10.1186/s12909-017-0912-5.
Herstad, J., Holone, H., 2012. What we talk about when we talk about co-creative tangibles, in: Proceedings of the 12th
Participatory Design Conference on Exploratory Papers Workshop Descriptions Industry Cases - Volume 2 - PDC ’12, ACM
Press, New York, New York, USA. p. 109. doi:10.1145/2348144.2348179.
Hoda, R., Henderson, A., Lee, S., Beh, B., Greenwood, J., 2014. Aligning technological and pedagogical considerations: Har-
nessing touch-technology to enhance opportunities for collaborative gameplay and reciprocal teaching in NZ early education.
International Journal of Child-Computer Interaction 2, 48–59. doi:10.1016/j.ijcci.2014.06.001.
Hussain, S., 2010. Empowering marginalised children in developing countries through participatory design processes. CoDesign
6, 99–117. doi:10.1080/15710882.2010.499467.
Iivari, J., Iivari, N., 2006. Varieties of User-Centeredness, in: Proceedings of the 39th Annual Hawaii International Conference
on System Sciences (HICSS’06), IEEE. pp. 176a–176a. doi:10.1109/HICSS.2006.530.
Ishimaru, A.M., Takahashi, S., 2017. Disrupting Racialized Institutional Scripts Toward Parent-Teacher Transformative Agency
for Educational Justice. Peabody Journal of Education 92, 343–362. doi:10.1080/0161956X.2017.1324660.
Iversen, O.S., Dindler, C., 2013. A Utopian agenda in child-computer interaction. International Journal of Child-Computer
Interaction 1, 24–29. doi:10.1016/j.ijcci.2012.08.002.
Iversen, O.S., Halskov, K., Leong, T.W., 2012. Values-led participatory design. CoDesign 8, 87–103.
doi:10.1080/15710882.2012.672575.
Janssen, F., Könings, K.D., van Merriënboer, J.J., 2017. Participatory educational design: How to improve mutual learning
and the quality and usability of the design? European Journal of Education 52, 268–279. doi:10.1111/ejed.12229.
Janssen, F.J.J.M., Westbroek, H.B., van Driel, J.H., 2014. How to make guided discovery learning practical for student teachers.
Instructional Science 42, 67–90. doi:10.1007/s11251-013-9296-z.
Joyce, C., Pham, H., Stanton Fraser, D., Payne, S., Crellin, D., McDougall, S., 2014. Building an internet of school things
ecosystem, in: Proceedings of the 2014 conference on Interaction design and children - IDC ’14, ACM Press, New York, New
York, USA. pp. 289–292. doi:10.1145/2593968.2610474.
Jutila, M., Strömmer, E., Ervasti, M., Hillukkala, M., Karhula, P., Laitakari, J., 2015. Safety services for children: a wearable
sensor vest with wireless charging. Personal and Ubiquitous Computing 19, 915–927. doi:10.1007/s00779-015-0838-z.
Kalra, N., Lauwers, T., Dewey, D., Stepleton, T., Dias, M.B., 2007. Iterative design of a Braille writing tutor to combat
illiteracy, in: 2007 International Conference on Information and Communication Technologies and Development, IEEE,
RAND corporation, United States. pp. 1–9. doi:10.1109/ICTD.2007.4937386.
Karimi, A., Worthy, P., McInnes, P., Bodén, M., Matthews, B., Viller, S., 2017. The Community Garden Hack: Participatory
Experiments in Facilitating Primary School Teacher’s Appropriation of Technology, in: Proceedings of the 29th Australian
Conference on Computer-Human Interaction, ACM, New York, NY, USA. pp. 143–151. doi:10.1145/3152771.3152787.
Kensing, F., Blomberg, J., 1998. Participatory Design: Issues and Concerns. Computer Supported Cooperative Work (CSCW)
7, 167–185. doi:10.1023/A:1008689307411.
Kitchenham, B., Charters, S., 2007. Guidelines for performing Systematic Literature reviews in Software Engineering. Technical
Report. doi:10.1145/1134285.1134500, arXiv:1304.1186.
Kitchenham, B., Pretorius, R., Budgen, D., Brereton, O.P., Turner, M., Niazi, M., Linkman, S., 2010. System-
atic literature reviews in software engineering-A tertiary study. Information and Software Technology 52, 792–805.
doi:10.1016/j.infsof.2010.03.006.
Klonari, A., Gousiou, A., 2014. Encouraging teachers’ reflection using a card game: The game of consequences, in: Proceedings
of the European Conference on Games-based Learning, Dept of Geography, University of the Aegean, Mytilene, Greece. pp.
279–285.
Könings, K.D., Bovill, C., Woolner, P., 2017. Towards an interdisciplinary model of practice for participatory building design
in education. European Journal of Education 52, 306–317. doi:10.1111/ejed.12230.

12
Könings, K.D., Brand-Gruwel, S., van Merriënboer, J.J., 2007a. Teachers’ perspectives on innovations: Implications for
educational design. Teaching and Teacher Education 23, 985–997. doi:10.1016/j.tate.2006.06.004.
Könings, K.D., Brand-Gruwel, S., van Merriënboer, J.J., 2010. An approach to participatory instructional design in secondary
education: an exploratory study. Educational Research 52, 45–59. doi:10.1080/00131881003588204.
Könings, K.D., Brand-Gruwel, S., van Merriënboer, J.J., 2011. Participatory instructional redesign by students and teachers in
secondary education: effects on perceptions of instruction. Instructional Science 39, 737–762. doi:10.1007/s11251-010-9152-3.
Könings, K.D., McKenney, S., 2017. Participatory design of (built) learning environments. European Journal of Education 52,
247–252. doi:10.1111/ejed.12232.
Könings, K.D., Seidel, T., van Merriënboer, J.J., 2014. Participatory design of learning environments: integrating perspectives
of students, teachers, and designers. Instructional Science 42, 1–9. doi:10.1007/s11251-013-9305-2.
Könings, K.D., van Zundert, M.J., Brand-Gruwel, S., van Merriënboer, J.J., 2007b. Participatory design in secondary education:
is it a good idea? Students’ and teachers’ opinions on its desirability and feasibility. Educational Studies 33, 445–465.
doi:10.1080/03055690701423648.
Koutamanis, A., Heuer, J., Könings, K.D., 2017. A visual information tool for user participation during the lifecycle of school
building design: BIM. European Journal of Education 52, 295–305. doi:10.1111/ejed.12226.
Kreitmayer, S., Rogers, Y., Laney, R., Peake, S., 2013. UniPad: Orchestrating Collaborative Activities Through Shared Tablets
and An Integrated Wall Display, in: Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous
computing - UbiComp ’13, ACM Press, New York, New York, USA. p. 801. doi:10.1145/2493432.2493506.
Kuure, L., Molin-Juustila, T., Keisanen, T., Riekki, M., Iivari, N., Kinnula, M., 2016. Switching perspectives: from a language
teacher to a designer of language learning with new technologies. Computer Assisted Language Learning 29, 925–941.
doi:10.1080/09588221.2015.1068815.
Kyza, A.E., Georgiou, Y., 2014. Developing In-Service Science Teachers’ Ownership of the Profiles Pedagogical Framework
through a Technology-Supported Participatory Design Approach to Professional Development. Science Education Interna-
tional 25, 21.
Kyza, E.A., Nicolaidou, I., 2017. Co-designing reform-based online inquiry learning environments as a situated approach to
teachers’ professional development. CoDesign 13, 261–286. doi:10.1080/15710882.2016.1209528.
Leong, T.W., Iversen, O.S., 2015. Values-led Participatory Design as a Pursuit of Meaningful Alternatives, in: Proceedings of
the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction on - OzCHI ’15, ACM Press,
New York, New York, USA. pp. 314–323. doi:10.1145/2838739.2838784.
Lingnau, A., Lenschow, H., 2010. Scenarios for computersupported learning in a special needs classroom. Journal of Assistive
Technologies 4, 26–35. doi:10.5042/jat.2010.0279.
Lundström, A., Savolainen, J., Kostiainen, E., 2016. Case study: developing campus spaces through co-creation. Architectural
Engineering and Design Management 12, 409–426. doi:10.1080/17452007.2016.1208077.
Medeiros-Braz, L., Souza Ramos, E., Benedetti, M.L.P., Hornung, H., 2017. Participatory Design of Technology for Inclusive
Education: A Case Study, in: Lecture Notes in Computer Science, Springer Verlag, Campinas. pp. 168–187. doi:10.1007/978-
3-319-58700-4.
van Merriënboer, J.J., McKenney, S., Cullinan, D., Heuer, J., 2017. Aligning pedagogy with physical learning spaces. European
Journal of Education 52, 253–267. doi:10.1111/ejed.12225.
OECD, 2015. Education Policy Outlook 2015: Making Reforms Happen. Technical Report.
Paris. URL: http://www.oecd-ilibrary.org/education/education-policy-outlook-2015 9789264225442-en,
doi:10.1787/9789264225442-en.
Otero, N., Alissandrakis, A., Müller, M., Milrad, M., Lencastre, J.A., Casal, J., José, R., 2013. Promoting secondary
school learners’ curiosity towards science through digital public displays, in: Proceedings of International Conference
on Making Sense of Converging Media - AcademicMindTrek ’13, ACM Press, New York, New York, USA. pp. 204–210.
doi:10.1145/2523429.2523475.
Pantsar-Syväniemi, S., Ervasti, M., Karppinen, K., Väätänen, A., Oksman, V., Kuure, E., 2015. A situation-aware safety
service for children via participatory design. Journal of Ambient Intelligence and Humanized Computing 6, 279–293.
doi:10.1007/s12652-014-0225-z.
Parsons, S., Cobb, S., 2014. Reflections on the role of the ‘users’: challenges in a multi-disciplinary context of learner-
centred design for children on the autism spectrum. International Journal of Research & Method in Education 37, 421–441.
doi:10.1080/1743727X.2014.890584.
Parsons, S., Millen, L., Garib-Penna, S., Cobb, S., 2011. Participatory design in the development of innovative technologies for
children and young people on the autism spectrum: the COSPATIAL project. Journal of Assistive Technologies 5, 29–34.
doi:10.5042/jat.2011.0099.
Pedersen, J.B., Andersen, A.S.Ö., Majgaard, G., 2012. Design of trigonometry apps for vocational education, in: Proceedings
of the IASTED International Conference on Human-Computer Interaction, HCI 2012, Itai a/s, Ndr. Landevej 2a, DK-6270
T{ø}nder, Denmark. pp. 226–232. doi:10.2316/P.2012.772-020.
Pérez-Sanagustı́n, M., Nussbaum, M., Hilliger, I., Alario-Hoyos, C., Heller, R.S., Twining, P., Tsai, C.C., 2017. Research
on ICT in K-12 schools – A review of experimental and survey-based studies in computers and education 2011 to 2015.
Computers & Education 104, A1–A15. doi:10.1016/j.compedu.2016.09.006.
Pilemalm, S., 2018. Participatory Design in Emerging Civic Engagement Initiatives in the New Public Sector. ACM Transactions
on Computer-Human Interaction 25, 1–26. doi:10.1145/3152420.
Põldoja, H., Väljataga, T., Laanpere, M., Tammets, K., 2014. Web-based self- and peer-assessment of teachers’ digital
competencies. World Wide Web 17, 255–269. doi:10.1007/s11280-012-0176-2.
Pollock, M., Amaechi, U., 2013. Texting as a channel for personalized youth support: participatory design research by city

13
 youth and teachers. Learning, Media and Technology 38, 128–144. doi:10.1080/17439884.2013.756516.
Prins, G., Bulte, A., Pilot, A., 2016. An Activity-Based Instructional Framework for Transforming Authentic Modeling Practices
into Meaningful Contexts for Learning in Science Education. Science Education 100, 1092–1123. doi:10.1002/sce.21247.
Rahamat, R., Shah, P.M., Nor Puteh, S., Din, R., Karim, A.A., 2011. End-users’ involvement in the development of web-based
learning resources for English literature. 3L: The Southeast Asian Journal of English Language Studies 17, 5–18.
Reeves, T.C., 2015. Educational design research: Signs of progress. Australasian Journal of Educational Technology 31,
613–620.
Rodrigo, R., Ramı́rez, C.D., 2017. Balancing Institutional Demands with Effective Practice: A Lesson in Curricular and
Professional Development. Technical Communication Quarterly 26, 314–328. doi:10.1080/10572252.2017.1339529.
Rodrı́guez-Triana, M.J., Martı́nez-Monés, A., Asensio-Pérez, J.I., Dimitriadis, Y., 2012. Towards a Monitoring-Aware Design
Process for CSCL Scripts. doi:10.1007/978-3-642-33284-5.
Sanders, E.B.N., Stappers, P.J., 2008. Co-creation and the new landscapes of design. CoDesign 4, 5–18.
doi:10.1080/15710880701875068.
Santally, M.I., Cooshna-Naik, D., Conruyt, N., Wing, C.K., 2015. A Social Partnership Model to Promote Educators’ Devel-
opment in Mauritius through Formal and Informal Capacity-Building Initiatives. Journal of Learning for Development 2,
9.
Selwyn, N., Henderson, M., Chao, S.H., 2017. The possibilities and limitations of applying ‘open data’ principles in schools.
Cambridge Journal of Education 47, 167–187. doi:10.1080/0305764X.2016.1143449.
Simonsen, J., Robertson, T., 2013. Routledge International Handbook of Participatory Design. Routledge, New York.
Siozos, P., Palaigeorgiou, G., Triantafyllakos, G., Despotakis, T., 2009. Computer based testing using “digital ink”: Partici-
patory design of a Tablet PC based assessment application for secondary education. Computers & Education 52, 811–819.
doi:10.1016/j.compedu.2008.12.006.
Smith, R.C., Bossen, C., Kanstrup, A.M., 2017. Participatory design in an era of participation. CoDesign 13, 65–69.
doi:10.1080/15710882.2017.1310466.
So, H.J., Lossman, H., Lim, W.Y., Jacobson, M.J., 2009. Designing an online video based platform for teacher learning in
Singapore. Australasian Journal of Educational Technology 25, 440–457.
Somerville, M.M., Collins, L., 2008. Collaborative design: a learner-centered library planning approach. The Electronic Library
26, 803–820. doi:10.1108/02640470810921592.
Song, D., Oh, E.Y., 2016. A Participatory Design Approach for a Mobile App-Based Personal Response System. Journal of
Educational Technology Systems 44, 346–361. doi:10.1177/0047239515618465.
Spinuzzi, C., 2002. A Scandinavian challenge, a US response, in: Proceedings of the 20th annual international conference on
Computer documentation - SIGDOC ’02, ACM Press, New York, New York, USA. pp. 208–215. doi:10.1145/584955.584986.
Steen, M., 2011. Tensions in human-centred design. CoDesign 7, 45–60. doi:10.1080/15710882.2011.563314.
Stephen, M.L., Locke, S.M., Bracey, G.L., 2014. Using a Participatory Design Approach to Create and Sustain an Innovative
Technology-rich STEM Classroom - One School’s Story, in: Proceedings of the 6th International Conference on Computer
Supported Education, SCITEPRESS - Science and and Technology Publications, Center for Science, Technology, Engineering
and Mathematics Research, Education, and Outreach, Southern Illinois University, Edwardsville, IL, United States. pp. 30–
38. doi:10.5220/0004849900300038.
Su, C., Chiu, C., Wang, T., 2010. The development of SCORM-conformant learning content based on the learning cycle using
participatory design. Journal of Computer Assisted Learning 26, 392–406. doi:10.1111/j.1365-2729.2010.00355.x.
Suchman, L., 1988. Designing with the user: book review of Computers and democracy: a Scandinavian challenge, G. Bjerknes,
P. Ehn, and M. Kyng, Eds. Gower Press, Brookfield, VT, 1987. ACM Transactions on Information Systems 6, 173–183.
doi:10.1145/45941.383895.
Tammets, K., Pata, K., Laanpere, M., Tomberg, V., Gašević, D., Siadaty, M., 2011. Designing the competence-driven teacher
accreditation. doi:10.1007/978-3-642-25813-8.
Triantafyllou, E., Timcenko, O., 2013. Developing digital technologies for university mathematics by applying partic-
ipatory design methods, in: 2013 24th EAEEIE Annual Conference (EAEEIE 2013), IEEE, Dept. of Media Tech-
nology, Aalborg University Copenhagen, A.C. Meyers Vaenge 15, DK-2450 Copenhagen SV, Denmark. pp. 82–85.
doi:10.1109/EAEEIE.2013.6576507.
Tulinius, C., Brit, A., Nielsen, S., Hansen, L.J., Hermann, C., Vlasova, L., Dalsted, R., 2012. Increasing the general level of
academic capacity in general practice: Introducing mandatory research training for general practitioner trainees through a
participatory research process. Quality in Primary Care 20, 57–67.
Vakil, S., McKinney de Royston, M., Suad Nasir, N., Kirshner, B., 2016. Rethinking Race and Power in Design-Based Research:
Reflections from the Field. Cognition and Instruction 34, 194–209. doi:10.1080/07370008.2016.1169817.
Vines, J., Clarke, R., Light, A., Wright, P., 2015. The beginnings, middles and endings of participatory research in HCI: An
introduction to the special issue on ‘perspectives on participation’. International Journal of Human-Computer Studies 74,
77–80. doi:10.1016/j.ijhcs.2014.11.002.
Voogt, J.M., Westbroek, H., Handelzalts, A., Walraven, A., McKenney, S., Pieters, J., de Vries, B., 2011. Teacher learning in
collaborative curriculum design. Teaching and Teacher Education 27, 1235–1244. doi:10.1016/j.tate.2011.07.003.
Wang, F., Hannafin, M.J., 2005. Design-based research and technology-enhanced learning environments. Educational Technol-
ogy Research and Development 53, 5–23. doi:10.1007/BF02504682, arXiv:arXiv:1011.1669v3.
Woolner, P., Clark, J., Hall, E., Tiplady, L., Thomas, U., Wall, K., 2010. Pictures are necessary but not sufficient: Using a
range of visual methods to engage users about school design. Learning Environments Research 13, 1–22. doi:10.1007/s10984-
009-9067-6.
Woolner, P., Hall, E., Wall, K., Dennison, D., 2007. Getting together to improve the school environment: user consultation,

14
 participatory design and student voice. Improving Schools 10, 233–248. doi:10.1177/1365480207077846.
Zainuddin, N.M.M., Zaman, H.B., Ahmad, A., 2010. A Participatory Design in Developing Prototype an Augmented
Reality Book for Deaf Students, in: 2010 Second International Conference on Computer Research and Development,
IEEE, Department of Information Science, Universiti Kebangsaan Malaysia, Bangi Selangor, Malaysia. pp. 400–404.
doi:10.1109/ICCRD.2010.55.

15
Appendix

16
Reference Research purpose Analysis / validation method Main findings
Abdullah and Brereton Support children with Autism Disor- Thematical analysis (Boyatzis, 1998; Teachers could model positive be-
(2015) der and language delays. Braun and Clarke, 2006) of video clips haviours and to scaffold more rel-
and observational notes on interactions evant and meaningful learning op-
between teachers and children. portunities by relating them to the
children’s lives.
Al-Eraky et al. (2015) Design a faculty development pro- Kirkpatrick’s (1994) model for pro- 1) developing vignettes on pro-
gramme on professionalism. gramme outcomes evaluation 1) reac- fessionalism is difficult 2) some
tion, 2) learning, 3) behaviour, and 4) vignettes help learners prioritise
results. their lives, 3) different versions
from a vignette. Teachers became
more knowledgeable on profession-
alism and how it can be taught.
Anderson and Östlund Analyze teachers and paraprofession- Research is based on the four teams’ Contributes to the assessment of
(2017) als’ work and reflections in AfL during documentation (written texts) of their learners attending special schools,
a professional development project in work and analyzed using a qualita- an assessment practice that is still
a special school. tive content analysis (Graneheim and in its infancy and needs to be fur-
Lundman, 2004). ther developed through extensive
studies.
Barbera et al. (2017) Identify the moments of change that Descriptions of agents’ main roles and Developing contextualised theory
occur during the co-design process and changes between three different co- on the domain of IBL and TEL
secondly, to describe the causes and designed versions of learning scenarios. pedagogical models as well as on
agents that motivate them. the co-design process itself. A
qualitative approach was used for
data collection, analysis, and inter-
pretation.
Booker and Goldman Tackle math fears by restoring families’ Four years of design, facilitation, 1) open dialogue about what
(2016) epistemic authority. and dissemination of workshops, take- counts as the phenomenon of in-
home materials, and family case stud- terest, 2) simultanoeus positioning
ies. to learners and authors in ways
of knowing, 3) collaborative data
analysis, 4) removal of individual
and cultural deficit.
Bossavit and Parsons Design an educational game to learn Stakeholders contribute their own Mapping of design sessions: 1)
(2016) about Geography via the use of Natu- spheres of expertise and equity in de- stakeholder role, 2) level of engage-
ral User Interfaces. sign partnership is not about all part- ment, 3) design tools, 4) made de-
ners sharing all decisions, but about cisions, 5) power-sharing.
respectfully managing the different ex-
pertise that each partner brings.
Bossen et al. (2010) What participants gain from PD Single-person, semi-structured inter- 1) opportunity to reflect on teach-
projects? views (Kvale, 1996) with a focus on ing and to communicate across
exploring the participants’ experiences professional borders, 2) skilled
and gains from the project, 1) influ- in understanding technology, 3)
ence on project, 2) satisfying and frus- higher influence on related to tech-
trating experiences, 3) personal gain, nology.
4) new possibilities and areas of com-
petence.
Brereton et al. (2015) Finding possibilites for teachers to fa- Elaborating the design after design Utilising prototypes fostered new
cilitate communication with persons process when the prototypes were forms of social interaction and ex-
with intellectual disability or form of utilised. pression between teachers and per-
cognitive or sensory impairment. sons with impairments.
Burke and Könings Describe how designing in education Reflecting the history of De Werk- The power of past experience and
(2016) context is informed, inspired, and in- plaats and how it was present in de- the vision of its founder can be
fluenced by the past practices, experi- signing the new school. recognised as a powerful force in
ences, and mythologies. the participatory design process of
the new building, which revitalises
the educational principles that the
school strives for.

17
Reference Research purpose Analysis / validation method Main findings
Carmichael (2015) Reflecting technology design in terms Reflecting observations from PD work- Syntheses of time involves conver-
of Deleuze’s dimensions of time. shops in two case studies. sations not just about creating a
technology-rich educational utopia
or constantly specifying new gad-
gets, but the troubling of existing
pedagogical practices and the mul-
tiplication of perspectives and sub-
jectivities.
Casanova and Mitchell Investigates new ways of designing Establishing a creative space (sandpit) Found participatory design as an
(2017) learning spaces that are future-proof wherein students and teachers criti- effective method to anticipate the
and relevant for users and discusses the cally reflect on the purpose and value learning and teaching spaces of the
purpose and value of technologies in of technologies and co-create new pro- next decade.
such learning spaces. posals for technology-enhanced learn-
ing spaces. Qualitative analysis of
recordings.
Cober et al. (2015) In what ways teachers participated in Ppen coding of design documentation Participation: engaging in theoret-
the design process and what conditions and teacher interviews and utilising ical discussions, active participa-
supported them in their design work? discourse analysis using the codes from tion in a design partnership, re-
Muller and Kuhn (1993). flection about pedagogy and prac-
tice, and experimenting with en-
actment. Support: supporting
emergent processes, cultivating an
atmosphere of trust and partner-
ship, and designing with contex-
tual knowledge.
Cramer and Hayes How empirical work at elementary Analysing and coding classroom ob- Three guidelines for designing for
(2013) school informs the design of mobile ap- servations, semi-structured interviews, classrooms: classroom flow, indi-
plications for teachers to manage finan- and field notes. vidual assesment, and peer groups.
cial education?
Medeiros-Braz et al. Investigates the hypothesis that Par- The interpretation of video transcrip- Participation benefits the process
(2017) ticipatory Design with inclusive edu- tions and material created during the of inclusion in the classroom of
cation teachers facilitates the creation workshops was done by the HCI re- the regular school, since it enables
of technology for inclusion in the class- searcher. thinking about how to eliminate
room (Tangible User Interfaces). barriers that prevent students from
access to knowledge, the school en-
vironment, and the people there.
Duell et al. (2014) A framework for incorporating design Validation is discussed in future re- Establishment of Design Minds
thinking as a generic capability in K-12 search, but not done in the study community and yearly ambassador
education in regional areas of Australia program.
Ervasti et al. (2016) 1) learn about needs and perspectives User experience testing, interviews, Teachers experienced that the
to provide a location-aware safety ser- feedback questionnaires, log data from safety service did not bring real
vice that is perceived positive and en- the safety service system. added value and use to their prac-
abling, 2) test the service, and 3) anal- tical work, mostly due to the fact
yse service experience and value for that there had been only minor
end-users. exceptions (absences and late ar-
rivals) on normal schooldays dur-
ing the trial period.
Fage et al. (2016) Design of a tablet-based application Data from application, questionnaires, 1) children with ID are not au-
to support activity schedules for both comparison between two children tonomous in the use of the applica-
classroom and verbal communication groups. tion at the end of the intervention.
routines 2) both groups exhibited the same
benefits on classroom routines. 3)
children with ID improve signifi-
cantly less their performance on
verbal communication routines.
Farooq et al. (2007) How to design and develop sustainable Case descriptions (Yin 2013), dis- Presenting the design interventions
community computing infrastructures: course analysis (Schlager et al., 2002), as a measure of success in itera-
nine years of design experience with collective reflecting of data interpreta- tively designing the artefact and
Tapped In – an online community of tions keeping the community members
practice for education professionals interested in using the infrastruc-
ture for online teacher professional
development and social network-
ing.

18
Reference Research purpose Analysis / validation method Main findings
Gillies et al. (2015) Technology to support music teachers Piloting the developed system with The problem of skeletal motion
in giving feedback on students’ posture teachers and having a group-feedback capture for feedback was that it
and the effectiveness of motion capture session. leaved out cues that are required
in music teaching. to detect posture problems.
Girard and Johnson Design computational models of emo- Teachers were asked to rate the use- Not enough data was collected to
(2010) tions for use in computational agents fulness of this emotional response for present viable results.
embodied in intelligent tutoring sys- the pedagogical role within the soft-
tems. ware and provide with a description of
how such responses could be better as-
sociated with the pedagogical goal.
Goeze et al. (2014) How video case-based learning could A coding scheme for measuring the an- Training teachers to become im-
promote the analytical competence of alytical competence of the participants mersed in the student perspective
teachers 1) to become immersed and and statistical comparison of experi- can be considered as an alterna-
to adopt multiple perspectives, 2) to mental groups. tive for direct involvement of stu-
apply conceptual knowledge, and 3) to dents in the design process. Usage
describe pedagogical situations of cases plays a role in facilitating
analytical skills, and that instruc-
tional support is crucial, but only if
hyperlinks to multiple perspectives
or conceptual knowledge were pro-
vided.
Gros and López (2016) Is the Learner Centric Ecology of Re- Interviews with teachers and students Framework was identified by the
sources model useful to support the in order to identify the main charac- participants as a good facilita-
selection of the resources using a co- teristics of the participants, partici- tor of the co-design process as it
design methodology and how are stu- pant observation and audio recordings helped to identify the main re-
dents influenced by participating in a of joint work sessions, and question- sources for the different scenarios
co-design process to build a learning naires addressed to both teachers and and encouraged dialogue and coop-
environment supported by technology. students after each work session. eration among the participants.
Hannon et al. (2012) Develop classroom educational tech- Classroom observations, task analysis, By participating in a technology-
nology tools for promoting collabora- PD workshops, but the results are not design project, teachers are experi-
tive inquiry-based learning. validated. encing the inquiry process as well
as developing tools that will facil-
itate using inquiry-based methods
in their classrooms.
Harrison et al. (2017) Explore an institution’s readiness to Participatory redesing meetings and 1) the need for more authentic as-
adopt initial changes which would help follow-up interviews. Data were anal- sessment, 2) the potential to give
an organisation move towards an as- ysed from a sociocultural perspective, feedback without (or before) the
sessment for learning culture: 1) op- using Johnson’s cultural web as a lens issuing of grades, and 3) the role
portunities and challenges, 2) individ- , in order to understand aspects of the of one-to-one mentoring to support
ual beliefs. organisational culture as well as indi- the interpretation of the feedback.
vidual beliefs.
Herstad and Holone Use co-creative tangibles to improve Prototyping. The co-creative tangibles in the
(2012) health for persons with severe disabili- RHYME project opens up new
ties. ways of participation within the
fields of Universal Design and tan-
gible interaction.
Hoda et al. (2014) Design and develop an engaging soft- 1) functional testing and informal feed- Interaction with the game was
ware solution that would preserve the back, 2) teacher trials and refinements, engaging and easily understood
principles of reciprocal teaching and and 3) children-teacher trials. Analy- by young children, provides ev-
support collaborative gameplay among sis: thematic analysis and open cod- idence that the design features
teachers and children. ing sought evidence of reciprocal teach- of themed content and mutual
ing and collaboration, while remaining awareness, availability of informa-
open to other emerging patterns. tion, and control support recip-
rocal teaching, collaboration, and
collaborative gameplay.

19
Reference Research purpose Analysis / validation method Main findings
Ishimaru and Taka- Illustrate how codesign concepts and Methodological approach is situated Authors suggest that bringing to-
hashi (2017) practices that draw from participatory within an emerging set of approaches gether concepts from organiza-
design research methodologies (Bang referred to as participatory design- tional theory and sociocultural
and Vossoughi, 2016) might disrupt based research study. learning theory can deepen our
and shift the racialized institutional understandings and strengthen so-
scripts about nondominant parents to cial change interventions. These
build transformative agency between concepts address the moment-
them and teachers. to-moment interactions that are
shaped by – and also have un-
tapped ability to transform –
the broader institutional struc-
tures that reinforce oppression in
schools and school systems.
Janssen et al. (2017) 1) view classroom teaching as bounded Demonstrate how these tools can be Even participants with a similar
rational design, 2) develop a laddering used in a design process to improve the background need tools to learn
tool that supports participants to map quality and usability of learning envi- more about each other’s practice
and share their multiple goals related ronments, including physical learning and goals, as well as to co-design
to the design of learning environments, spaces that better support learning. possible ways of teaching that suit
and 3) develop a building block tool their needs. Participatory de-
that helps participants to explore prac- sign of the learning environment
tical and effective possibilities for the is an important prerequisite for a
design of the learning environment. productive (re-)design of a shared
classroom.
Janssen et al. (2014) Develop a teacher training trajectory Identified the lesson segments and Biology student teachers 1) proved
for developing guided discovery learn- heuristics, and estimated the expected to be willing and capable to imple-
ing lessons in biology. values and underlying motivational be- ment GDL aspects in their lessons,
liefs. 2) increasingly used the heuristics
developed by experienced teachers
for designing GDL lessons, 3) will-
ingness to use the heuristics for de-
signing GDL lessons increased af-
ter the intervention, 4) both the
estimated desirability and proba-
bility of GDL lessons increased,
and 5) most student teachers the
expected value of GDL after the
trajectory was higher than the
expected value of their regular
lessons.
Joyce et al. (2014) Design an Internet of Things based Eight-month design and pilot phase, Schools were willing to adopt
ecosystem for schools. including open-ended interviews, fo- the IoT technology within certain
cus groups, training sessions, class- bounds. Outline best practices
room observation, and shadowing. uncovered when introducing IoT
technologies to schools.
Jutila et al. (2015) Technological enablers and require- PD workshops, piloting, and evalu- Insights regarding the monitoring
ments for building a complete end-to- ation. User questionnaire, technical of the child on a situation ba-
end energy-efficient safety system. evaluation. sis. A novel energy-efficient solu-
tion through the designed sensor
vest with wireless integrated charg-
ing and related end-to-end service
applications.
Kalra et al. (2007) Develop Braille, a prototype writing Six week pilot study, learning measure- The Braille Writing Tutor has
tutor system to tackle illiteracy ments, acceptance questionnaire, and great potential to inexpensively
usability observations and effectively aid the education of
a large number of blind students in
the developing world.
Karimi et al. (2017) Involving teachers in the design pro- A theme identification based on The teachers’ primary takeaway
cess as agile appropriators who outlook analysing data from the transcribed was the pedagogical concept of
technologies as working material and workshop feedback. collaborative problem solving with
examine how designers can enable a technology and they found ways to
space for this involvement and engage- appropriate the process to provide
ment to co-create and co-explore tech- similar experiences for their stu-
nology possibilities with and for teach- dents.
ers.

20
Reference Research purpose Analysis / validation method Main findings
Klonari and Gousiou Develop a Game of Consequences to Game description, but no analy- Game 1) outlines a social script for
(2014) increase of teachers’ awareness on the sis/validation. participants to come together and
consequences of their choices for differ- process materials about challeng-
ent stakeholders/aspects by reflecting ing situations and dilemmas that
on several dilemmas related to school could encountered at their working
reality spaces/schools, 2) encourages and
facilitates reflection in a meaning-
ful way, and 3) encourages critical
thinking.
Koutamanis et al. Elaborate on an example of a visual Elaborating how BIM was used in four A comparison of the potential
(2017) information technology tool - Building stages of building design: 1) initiative, of digital, integrated information
Information Modelling (BIM) – in the 2) development, 3) realization, and 4) tools, such as in BIM, and a suc-
new context of participatory design of operation. cessful conventional participatory
the built learning environment. design project shows that appro-
priate information technologies can
empower users such as school man-
agement, teachers and students to
become full participants in the en-
tire life cycle of a school building.
Kreitmayer et al. Encourage students to talk, collabo- Participatory design with expert fi- The UniPad set-up was able to en-
(2013) rate, and make decisions together in nance educators and then evaluating courage much switching between
real time by switching between work- the UniPad application at a school. small and large group discussions.
ing on shared small group devices and In particular, the students were
a whole classroom public display (Uni- highly engaged with each other
Pad). when taking part in the financial
decision-making activity.
Kuure et al. (2016) Support students of English in a Two qualitative approaches: 1) the The PD approach applied in the
Finnish university in switching their research strategy of nexus analysis project was obviously not enough
career perspective: they were to be- (Scollon and Scollon, 2004) and, 2) in helping the language students
come not only language teachers but design-oriented cultural–historical ac- see themselves as designers for the
also designers of language learning tivity theory (Kuutti, 1994, 2005; future. Therefore, the students
with new technologies. Molin-Juustila, 2006). largely failed to engage their pupils
as crucial participants in the design
process.
Kyza and Georgiou PD as a bottom-up approach for pro- 1) open ended questionnaire analysed The use of technological media-
(2014) moting teachers’ sense of ownership by employing the Attride-Stirling’s tion and the combination of tools
of inquiry-based learning and teaching (2001) thematic network analysis, 2) that can support teachers’ asyn-
approach. data corpus was analyzed in a two chronous and synchronous commu-
phase analysis (Patton, 2000), focused nication, authoring tools to sup-
on teachers’ utterances and used an port the design process, and hu-
open coding approach without any pre- man scaffolding of teachers’ dis-
determined categories, and 3) quanti- cussions and enactment processes
tative data with the MoLE motivation could contribute to the develop-
instrument (Bolte, 2012). ment of a better understanding
of the inquiry process and could
lead to motivating learning envi-
ronments for students.
Kyza and Nicolaidou Examine co-design as an informal con- Thematic analysis, open coding, non- Co-design is a viable approach for
(2017) text for teachers’ professional devel- parametric and parametric tests of stu- transformative teacher professional
opment on reform initiatives such as dents’ pre- and post-tests. development, which can support
inquiry-based learning teachers in developing knowledge
and skills to address their just-in-
time needs.
Könings et al. (2017) Explore how architects, educational Workshop consiting of 1) forming het- A new Interdisciplinary Model of
designers, teachers, and students can erogeneous groups, 2) envisioning a Participatory Building Design that
collaborate in the design of educational participatory design process focused is based on four workshop proposi-
buildings. on developing a new educational build- tions of participatory building de-
ing and how to involve different stake- sign processes.
holders, 3) presenting the outcomes,
and 4) analysing the presentations.

21
Reference Research purpose Analysis / validation method Main findings
Könings et al. (2011) Examines participatory design as a Two quantitative surveys: 1) In- The findings indicate that the
strategy for taking student perceptions ventory of Perceived Study Environ- effects of participatory de-
into account in instructional re/design. ment Extended (IPSEE) to measure sign on students’ perceptions,
students’ perceptions of a particular perceived-desired discrepancies,
learning environment and their de- and teacher–student disagreement
sires with regard to the design of that show some positive effects for
environment. 2) Inventory of Per- the co-designers, but limited or
ceived Study Environment Extended- negative effects for the rest of the
Teacher Version (IPSEE-T) in which class.
some items are reformulated to reflect
the teacher’s perspective.
Könings et al. (2010) Develop an approach for student par- For evaluating the participatory meet- Both teachers and co-designing
ticipation in instructional design, and ing, the teachers, co-designing students students were largely satisfied with
to evaluate how students and teachers and the remainder of the class were the meeting. The atmosphere
experience the discussion about possi- asked open questions about the quality was experienced predominantly as
ble changes in the design and how they of the meeting and/or the agreement comfortable and enough opportu-
co-operate in designing lessons. with its proposed changes. nities were provided to express
thoughts and ideas. Teachers
stated that the usability of stu-
dents’ suggestions was good. The
students predominantly agreed on
the proposed changes. No differ-
ences were found between the eval-
uation scores of students of differ-
ent courses.
Könings et al. (2007b) Teachers’ perceptions of an innovative Inventory of Perceived Study Envi- Study showed that the implemen-
learning environment ”Second Phase” ronment Extended-Teacher (IPSEE- tation of the innovative learn-
T): teacher’s perceptions of a particu- ing environment only partly suc-
lar learning environment and their de- ceeded and that more coopera-
sires with regard to the design of a tion between educational design-
learning environment. ers and teachers is needed to cre-
ate more congruence between the
educational design and the factual
learning environment in the class-
room.
Lingnau and Lenschow How teachers use a computerised Analysing teachers’ contributions in a The Learning Chest software is
(2010) learning environment to teach pupils Wikipage that was established for the an example of a software develop-
with special educational needs. development project. ment process where teachers be-
come proactive contributors. Au-
thors believe that encouraging the
teachers to actively participate in
the development process was a key
factor for the success of the soft-
ware development.
Lundström et al. Examine the relationship between user Video observations, project document 1) PD provided a positive impact
(2016) needs and the service level in con- analysis, and survey questionnaires on the resulting premises, even
struction projects through a case study though every part of the project
where a university cafeteria was reno- may not be successful. 2) the
vated using a PD method called char- accomplishments can be undone
rette. in the later phases of the project
if collaboration is not extended
through the entire project. 3) the
study revealed a framework of user
needs that can be used in design
management in order to enhance
the user perspective.

22
Reference Research purpose Analysis / validation method Main findings
Otero et al. (2013) Utilising digital public displays to take The findings reported are based on teachers were able to generate sce-
into consideration educational goals a qualitative data analysis approach narios that take advantage of the
where the design team discussed to- possibilities offered by digital pub-
gether the data collected and created lic displays to stimulate learning
meaningful interpretations of it. processes. However, there are
several crucial elements regarding
management and control of con-
tent that need to be carefully de-
signed in order to accommodate
each schools’ organizational issues.
Pantsar-Syväniemi Design a situation-aware safety service 1) brainstortming workshop 2) Open Design process proved to be power-
et al. (2015) for children with a unique combination Web Lab to gather people’s thoughts ful and enabled the gathering and
of novel participatory tools, a brain- and ideas, 3) Service Innovation Cor- receiving of valuable feedback from
storming workshop, and scenario writ- ner (SINCO) 4) Scenario writing. both end users and the local soci-
ing. ety.
Parsons and Cobb Examine the complexity of navigat- Derives findings for the research objec- Complexity as a triple-decker
(2014) ing and involving different user groups tive by reflecting multiple previous re- ’sandwich’ representing Theory,
in the context of multi-disciplinary re- search projects. Technologies and Thoughts and
search projects. argue that all three layers need
to be appropriately aligned for a
good quality product or outcome.
The challenge lies in navigating
and negotiating all three layers at
the same time, including the views
and experiences of the learners.
Parsons et al. (2011) Develop specific interactive technolo- Pilot testing, but not described how Initial impressions from pilot test-
gies for school settings that may help these were analysed. ing are that both prototypes are
to promote learning and understanding motivating and enjoyable to use
of social skills. for typically developing children
and children with Autism Spec-
trum Conditions
Pedersen et al. (2012) how mathematics teaching can be en- Three user testing iterations. The application we developed en-
riched by apps using smartphone sen- riched mathematics teaching by in-
sors such as gyroscope, compass, cam- troducing several modes of partic-
era, and touch screen in a gaming con- ipation: 1) Physical activity in the
text. real world, 2) Individual and col-
laborative interaction, 3) Gaming
elements, 5) Aural and visual feed-
back, and 5) Social media.
Põldoja et al. (2014) Addresses design challenges related to Four stages: 1) contextual inquiry, 2) Teachers in the validation exper-
a software solution for self- and peer- participatory design, 3) product de- iment were satisfied with Web-
assessment of teachers’ digital compe- sign, and 4) production of software as based self- and peer-assessment of
tencies. hypothesis. A small-scale validation teachers’ digital competencies and
experiment. how it was implemented in the de-
sign of the DigiMina tool.
Pollock and Amaechi PD research on texting as a channel for Analyzed the texting record during Text messaging could rapidly
(2013) personalized youth support. and after the pilot, using basic prin- deepen student–teacher support
ciples of grounded theory (Charmaz relationships, with effects on
2006) and thematic coding (Lofland student–teacher bonding, caring,
and Lofland 1995). and student motivation.

23
Reference Research purpose Analysis / validation method Main findings
Prins et al. (2016) Presents an activity-based instruc- 1) an exemplary version of the cur- An instructional framework that
tional framework that assists educa- riculum, guided by the initial instruc- provides educational designers
tional designers in transforming au- tional framework, 2) the pedagogic de- with a set of prescriptive guide-
thentic scientific practices for the pop- cisions were conceptualized as design lines for transforming authentic
ulation of students in science educa- guidelines, 3) the exemplary curricu- modeling practices into contexts
tion. lum unit was piloted in four different for learning, while maintaining
classes resulting in a second version, the coherency between scientific
5) second version was enacted in two activities, content, and tools.
different classes and studied in-depth
to reveal the experienced coherency in
learners, and 6) the results were used
to revise and enrich the initial instruc-
tional framework, yielding an activity-
based instructional framework.
Rahamat et al. (2011) Develop and evaluate the usability of Questionnaire of Usability Evaluation The findings of formative evalu-
a web-based learning resource fo the of Website (QUEW). ation gave insight into a things
English Literature Component. needed in developing web-based
teaching material.
Rodrigo and Ramı́rez incorporating the use of ’master’ on- Authors have been developing and The developed master course can
(2017) line template courses as a part of the updating the course curriculum for be used as model that new online
initial and continued professional de- the professional and technical writing TPC instructors can use to learn
velopment processes of online TPC in- (P&TW) traditional face-to-face and about teaching online and promote
structors. newly emerging online class offerings the reason to continuously prompt
and developing a four-course certificate reflection and innovation through a
program in P&TW. user-focused iterative revision pro-
cess.
Rodrı́guez-Triana et al. Which aspects should be considered at 1) co-design process between teacher Three dimensions that influence
(2012) design in order to monitor the learning and researcher to obtain a monitorable the configuration of the monitor-
scenario. CSCL script, 2) putting the script into ing process in pattern-based ap-
practice, 3) results were triangulated proaches: the design pattern, the
with data coming from observations specific features of each activity,
carried out by the teacher during the and the teacher’s choices about
face-to-face sessions, 4) two question- specific issues.
naires handed to the students about
their work in groups, and 5) and sev-
eral interviews to the teacher during
and after the learning situation.
Santally et al. (2015) Develop a social partnership model Reflections from several projects. The model tries to embody the new
based on the living lab concept to pro- open innovation concepts as pro-
mote the professional development of posed by the Living Lab ecosystem
educators through formal and informal and can serve in the longer term as
capacity-building initiatives. an example for other initiatives.
Selwyn et al. (2017) explores the possibilities of making ex- PD was used as a research methodol- PD interventions in both schools
isting school data openly available in ogy to investigate the realities of the quickly encountered compromises
digitised form for teachers, administra- two school contexts as places of data and barriers – all of which illus-
tors and students to access, interpret use, and to highlight where it might be trated the relatively closed nature
and use. possible to align more democratically of school data practices and, it fol-
the use of school data with the values, lows, the practical limitations of
history and context of the whole school the open data philosophy.
community.
Siozos et al. (2009) Identify challenges faced by computer- 1) Multiple iterations of the same con- Designing effective CBA applica-
based assessment (CBA) in secondary cise and highly structured collabora- tions can be realized by actively
education and put forward a frame- tive design session with different stu- involving students and teachers in
work of design considerations. dents, 2) designers systematically an- the design process. Both students
alyze and integrate student products and teachers were excited about
from the different sessions into a final their participation in the design
application. sessions, and they asserted that
they would rely more on educa-
tional software designed using this
approach.

24
Reference Research purpose Analysis / validation method Main findings
So et al. (2009) Design an online platform where teach- 1) large scale survey to understand cur- Teachers perceived their school
ers can share vivid images of their rent situation 2) describe PD work- culture to be collaborative, but col-
teaching practices with other teachers. shops in two schools laboration across schools is rarely
encouraged. Most professional de-
velopment programs are conducted
as a pre-packaged format in a face
to face mode. Workshop partici-
pants expressed that video technol-
ogy is seldom used in professional
development programs, and that
there is a strong need to have ac-
cess to an online video based com-
munity for teachers.
Somerville and Collins Collaborative planning of library Reflects several design projects with Inviting and enabling user input
(2008) learning commons and campus learn- various methods. from the start offers a fruitful plan-
ing spaces. ning approach in which campus li-
brarians, stakeholders, and benefi-
ciaries “learn their way” to appro-
priate library (re)design decisions.
Song and Oh (2016) Design of a mobile app-based personal 1) interviews, an exploratory evalua- The participants showed strong
response system tion of the prototype using a Likert agreement with the use of
scale questionnaire. EnClicker increased their enjoy-
ment of lectures and participation
level in the questioning activity.
Stephen et al. (2014) Design of a technology-rich STEM Data sources include guided and open- Using a PD process that included
classroom. ended interviews with teachers and students as well as teachers has led
administrators, student focus groups, to a sense of ownership of the room
and observation of sessions in both by both groups. Teachers and
the STEM classroom and in regular students depict traditional class-
classrooms. Additional data that in- rooms as ’teacher space’ while they
form the study were gathered from sur- view the STEM classroom as ’com-
vey instruments including pre and post munity space’ with both teachers
student and teacher questionnaires. and students equally responsible
for maintaining the room.
Su et al. (2010) Investigate the possibility of apply- Study used triangulation to inspect the The presented results contribute to
ing the engagement, exploration, ex- data collected from the weekly worker the integration of 5E learning cy-
planation, elaboration, and evaluation records, written journals and meet- cle into SCORM-conformant mate-
model (5E) to science e-learning mate- ing observations for examining docu- rials and provide concrete recom-
rials. ments such as technology standardiza- mendations for how to develop ef-
tion and instructional strategy. fective e-learning materials in the
future.
Tammets et al. (2011) Examine the applicability of an or- 1) A web-based Likert-scale survey, 2) The study showed how to techno-
ganizational knowledge-management design interview, and 3) evaluation of logically support maintaining and
model extended by the principles of the service with nine Estonian teach- pursuing professionalism through
self-regulated learning. ers. teachers’ participation in the com-
munity of practice, through col-
laborative learning and knowledge
building activities.
Triantafyllou and Tim- Explores the challenges of integrating 1) prototyping with two professors, 2) Professors, teaching assistants and
cenko (2013) digital technologies to support math- focus group discussions with students, students do not always share
ematics teaching and learning at uni- 3) group interviews with teaching as- same perceptions about how the
versity level. sistants, and 4) observation of seven mathematics curriculum should be
lectures. taught and which parts of it are
challenging. While professors fo-
cus on visualizing mathematical
concepts, students and teaching as-
sistants stress the importance of fo-
cusing on basic mathematics first
and also presenting applications of
mathematics in Media Technology.

25
Reference Research purpose Analysis / validation method Main findings
Tulinius et al. (2012) 1) create a link between general prac- 1) two ’think tanks’, 2) 14 interviews, It was possible to overcome several
titioner (GP) researchers and the GP 3) 119 hours of observations, 4) 583 of the previously reported obsta-
training community and 2) create an written evaluation forms, written and cles in critical appraisal training of
awareness of critical appraisal in surg- oral evaluations, 5) 13 additional inter- GP trainees. However, the study
eries training GPs, allowing GP train- views. is also an illustration of an inbuilt
ers to experience the relevance of crit- obstacle to any attempt to build
ical appraisal for their own clinical bridges between the clinical world
practice. and academia.
Vakil et al. (2016) Examine how race and power mediate Using the notion of politicized trust Making visible how race and power
relationships between researchers and as a conceptual lens, authors reflect mediate relationships in design re-
communities in ways that significantly on two distinct participatory design search is critical for engaging in
shape the process of research. projects to explore how political and ethical and sociopolitically con-
racial solidarity was established, con- scious relationships with commu-
tested, and negotiated throughout the nity partners and developing the-
course of the design process. oretical and practical knowledge
about the repertoires of practice,
tasks, and sociocultural competen-
cies demanded of university re-
searchers.
van Merriënboer et al. Addresses the need to align peda- Reflects the proposed model through Distinguished three phases in such
(2017) gogies and physical learning environ- two case studies. a design process: the specification
ments and describes a participatory of a pedagogy, which can be de-
design process to help to create phys- scribed as an interplay of four basic
ical spaces and school buildings that educational components; the align-
optimally support specific visions on ment of the chosen pedagogy with
learning and pedagogy. seating arrangements and physical
learning spaces; and the realisation
of the school building.
Woolner et al. (2010) Explore the views of a diverse sample Using visual research methods, authors Methods facilitated the engage-
of individuals from a school commu- explored their experiences of the exist- ment of a broad range of people
nity and so develop understanding of ing school environment together with from the school community. Fur-
the learning environment. aspirations for the future, when the thermore, it was observed that the
school would be rebuilt: 1) photo elici- differing views of those with dif-
tation, 2) diamond ranking, 3) picture ferent roles produced a more com-
sorting, 4) map-based activities. plete understanding of the complex
functioning of the school and the
potential influences of this setting
on learning.
Woolner et al. (2007) Exploring consultation within the 1) The teachers completed lesson pro- The message which is heard by de-
modern context of participatory school formas which investigated the use signers and architects is no more
design and student voice. made of the classroom in terms of lay- certain to lead to a complete design
out and lesson structure and percep- solution and still runs the risk of
tions of the quality of the teaching and being unrepresentative of the full
learning in the session, as well as be- range of relevant views. The diffi-
haviour management. 2) The teachers culty of deciding whom to consult
were also interviewed by the university in order to reveal to architects and
team regarding their experiences of the designers the needs of education, is
design process. not completely solved by empha-
sizing the involvement of students.
Zainuddin et al. (2010) Design of an augmented reality book Ethnographic method via observation The study identified the criteria
for deaf students. to discover what participants thought, in designing an AR-Book for deaf
their interaction with the researcher, students, which will be used in
and their reflection on choosing AR- the prototype development of AR-
Book sets during the observation and Book Science in Deaf, called as
interaction process. AR-SiD.

26
 PIII

DESIGN OF A LEARNING SPACE MANAGEMENT SYSTEM

FOR OPEN AND ADAPTABLE SCHOOL FACILITIES

by

Tuhkala Ari, Isomäki Hannakaisa, Hartikainen Markus, Cristea Alexandra, and

Alessandrini Andrea 2018

Communications in Computer and Information Science, vol. 865

https://doi.org/10.1007/978-3-319-94640-5_2
Design of a Learning Space Management System for Open and
Adaptable School Facilities

Tuhkala Ari1 , Isomäki Hannakaisa1 , Hartikainen Markus1 , Cristea Alexandra2 , and Alessandrini
Andrea3
1
Faculty of Information Technology, University of Jyväskylä, Finland
2
Department of Computer Science, University of Warwick, United Kingdom
3
Duncan of Jordanstone College of Art and Design, University of Dundee, Scotland

Abstract. In this design-based research project, a learning space management system was devel-
oped for the Valteri School Onerva in Central-Finland. The school represents a modern educational
environment with open and adaptable learning spaces. The goal was to develop a software to sup-
port the stakeholders in organising flexible pedagogical activities and sharing pedagogical practices.
To reach this goal, we utilised value-focused thinking as a requirements elicitation method, to iden-
tify the objectives that the stakeholders associate with the new environment. In the implementation
phase, we organised participatory design workshops, to involve the stakeholders in decision-making,
to ensure that the prototype development was proceeding according to their needs. As a result,
we elaborate how we utilised value-focused thinking, what were the objectives that were identified,
and how they were transformed into system requirements. Finally, we describe the first prototype
of the learning space management system, which was developed using these requirements.

Keywords: Classroom Management, Learning Spaces, Educational Technology, Special Educa-

tion, Value-focused Thinking.

1 Introduction

The traditional classroom setting of children sitting on benches and patiently listening to a teacher is not
easily applicable in special education. The classrooms are often considered less inspiring, and an activity-
driven approach is more appropriate [38]. For example, children with hearing and vision problems can
benefit from visual and physical stimulations and moving between different spaces, and children with
autism disorders benefit from the use of technologies and digital artefacts that promote collaborative
educational activities and attentional exercises [2]. To overcome these issues, a new school was recently
created in Finland, the Valteri School Onerva, which was just finished in early 2016. Its stated goal was
to enable functionality, physical activity, and the application of new technologies. The idea of an open
and adaptable school was a focus from the planning and construction stages of the school. Under this
concept, all physical spaces are understood as potential spaces for learning, not just the classroom, and
the environment is dynamically adapted to the needs of the practised pedagogy. A simple example is
using stairways as an active learning space: children might physically move from one stairway step to
another, while learning the number line, months, or weekdays (Ikkelä-Koski, personal communication,
May 5, 2014).
However, the activities in the modern school environment of the Valteri School Onerva must be
supported with modern technology. In the stairway example, in a regular educational setting, with
the current level of support, it would not be possible to know if the stairway was already in use, as
the stairway is a non-traditional learning space and would not be considered by any scheduling tool.
The lack of such critical information prevents teachers from implementing such new pedagogical ideas,
even simple ones, due to the time costs if the targeted space is not available and the whole class must
2 Tuhkala, Isomäki, Hartikainen, Cristea & Alessandrini

return to the classroom. Moreover, not all teachers have the time and resources to develop new ideas
and surely are not aware of all the available possibilities. Unfortunately, we find the current facility
(or classroom) management systems not suitable for use in this dynamic environment. The systems for
commercial or non-commercial organisations seem to be developed mainly for standard administrative
needs. Instead of traditional facility management features, teachers need a tool that supports them in
organising flexible pedagogical activities and sharing pedagogical practices. To successfully develop a
learning space management system, we need to carefully examine the objectives that teachers associate
with the open and adaptable environment.
As a result, the requirements for a learning space management system were produced in the ONSPACE
research project between May 1 and December 5, 2014, before the building construction even started [50].
Requirements elicitation is one of the most critical activities of software development and is known to be
a major reason for project failures [39]. We grounded our research in two assumptions: First, to have a
better understanding of teachers’ work, we needed to involve our stakeholders and arrange user-centred
workshops, based on participatory design principles. Participatory design emphasises shared decision-
making, which is crucial when different stakeholders are involved [11]. Second, traditional requirements
elicitation concentrates on identifying system’s goals, functionality, and limitations [39]. While this is
fundamental, we argue that the stakeholders’ objectives need to be defined more holistically than just
considering the actual system. Therefore, we applied a method developed by Professor Ralph Keeney and
proposed in the book Value-focused Thinking: A Path to Creative Decision Making [22]. The method
offers systematic guidelines, which are described in a later section, for identifying objectives for the
defined decision problem.
This study has both methodological and practical contributions. Value-focused thinking has been
applied in multiple domains, but less in the context of requirements elicitation, especially as they relate
to education. Learning space management systems are currently gaining attention as modern schools
increasingly adjust to the idea of open and adaptable learning environments [43]. The identified objectives
were used during the implementation of the learning space management prototype. To fulfill our goals,
we framed the following research questions:
– How can value-focused thinking be implemented and applied to the requirements elicitation context?
– What are the objectives associated with an open and adaptable school environment?
For the first question, we describe in detail how we applied the method, and for the second, we
interpret recordings from the workshops and present the identified objectives, with the help of teachers,
administrative personnel and rehabilitation instructors. The original requirements specification document
is in Finnish and consists of 32 pages; therefore, its full inclusion is beyond the scope of this paper.
Instead, we highlight and discuss the process of extracting the objectives, followed by a discussion of the
objectives. The prototype of the system was developed in 2015, in the sequel project called ONSPACE2,
and the objectives were used as guiding evaluation principles by software engineers during development
of the learning space management system.

2 Towards novel school facilities

Facility management considers assets that are not the primary activity for the organisation, but essential
to function [26]. These assets are typically buildings and properties, while information systems, human
resources, and finance are understood as separate areas. In education context, assets may include school
buildings, accessories, vehicles, and permissions to certain services, like a healthcare specialist or a school
psychiatrist. While efficient management of these assets can make a significant difference regarding finan-
cial cost and value in any organisation, the physical environment itself, like the school building, can have
an enormous impact on facilitating learning [26, 29, 49]. A good physical environment gives resources, in-
spiration, and motivation for learning. Moreover, the physical environment can be enhanced with virtual
 Learning Space Management System 3

properties, by using information technology [25]. The virtual resources can extend the social action, by
allowing distant interactions and enabling learning activities for students with perception problems. In
addition to physical and virtual aspects, there is a social dimension: how interactions are made possible
between people within organisation culture.
Lievonen and Kinnunen [31] point out how information technology extends learning situations beyond
school buildings. Therefore, spatial navigation, spatial control, interpersonal communication and collabo-
ration need to be considered. Kumpulainen and Mikkola [28] even argue that learning 21st-century skills,
like critical thinking, problem-solving or media literacy, are challenging to promote in an educational
environment that is restricted to a particular space and time. They call for learning environments that
reconfigure spaces for learning, because there is an increasing number of students who feel disengaged
and disconnected from formal education.
The principal stakeholder of this study, the Valteri School Onerva, has utilised modern ideas based
on Kuuskorpi’s [30] Doctoral Thesis in the construction of their new building. Kuuskorpi defines the
dimensions for a physical learning environment as: societal orientation, individual orientation, informal
learning processes and formal teaching (Figure 1), and learning spaces can be positioned according to
these dimensions. In the Valteri School Onerva, this means that the physical space and its furniture can
be adapted to different purposes and the common spaces are made available to various functions.

Society
orientation

Formal teaching Physical learning Informal learning

process environment process

Individual
orientation

Fig. 1. Dimensions for a physical learning environment [30].

Nevari [37] has designed the concept for the new building, illustrated in Figure 2. The areas are
called parks, fountains, and dens. A park is an open space, which can be easily modified for group work,
presentations and physical activities. A fountain is a partially open space for collaborative learning,
which can be divided into different areas, to suit both formal and informal learning situations. A quiet,
individual and closed space is called a den, and it can be used for focusing on a task, individually or
in small groups. In addition to the novel concept of physical space, there are no prearranged spaces
for teachers or students. The idea is that the learning and working activities will happen in the space
that is most suitable at the current moment, instead of staying in classrooms or offices. This idea is the
underlying motivation for our study, to develop technology that could give the best support for organising
activities in this unique environment.

3 Approaches for user participation and involvement

User participation and involvement are considered essential for success in system development [3, 17, 32],
as they improve the quality of the system, by generating more precise requirements [16] and tend to lead
 Dynamic multipurpose learning spaces
The new building is optimal for action-based learning because its premises and furniture can be adapted to different purposes.
In addition, the target of the project is to utilise the premises efficiently, making the common spaces available to the various func-
tions, in various ways, from morning to evening. Restaurant Omppu (Finnish for apple) is the heart of the entire building and, at
the same time, a meeting place. Both staff and pupils can work together with others in the open working area and, for instance,
with a peer in the intensive working area, or alone in the silent working area. These different working areas are called the park,
fountain and den, adapting Julianna Nevari’s learning space concept. In compliance with the space concept, the project is
called ‘Oivallus’ (Finnish for ‘insight’).
Accessibility and multi-sensory impact in the learning environment
The design process has been based on pupils’ needs. Accessibility refers to the suitability of the premises for everyone, irre-
4 spective of theHartikainen,
Tuhkala, Isomäki, nature of support needed. The spaces
Cristea and routes are clear, barrier-free and safe, and the perception of spaces
& Alessandrini
is facilitated by limiting and outlining different spaces with contrasts. Good acoustics are an important factor that promotes
learning. The spaces can also be lit in various ways, which is important from the viewpoint of vision as well as of concentration.

INCREASED COOPERATION

OPEN INTENSIVE SILENT

WORKING AREA WORKING AREA WORKING AREA

‘PARK’ ‘FOUNTAIN’ ‘DEN’

INCREASED CONCENTRATION

Group work zone

Small-group work zone
Meeting zone
Multifunction zone
Performance zone
Instructors’ team zone
The new learning and working environment provides different working areas.
Action-based learning is enabled by multi-functional, adaptable premises and furniture.

Fig. 2. Concept of the Valteri School Onerva [37].

to a positive attitude and perceived usefulness among users [1, 34]. Participation refers to assignments,
activities and behaviours that users engage in during the system development process and involvement
is a psychological state of the individual, defined as the importance and relevance of a system to a
user [3]. User involvement can also be seen as a broader concept, in which users are somehow involved
in the system’s development process, whereas user participation refers to more active and intentional
involvement [18]. Kujala [27] has presented methodological approaches to achieving participation and
involvement (Table 1).

Table 1. Methodological approaches to achieve participation and involvement [27].

Participatory design User-centered design Ethnography Contextual design

Democratic participation Usability Social aspects of work Context of work
Workshops, prototyping Task analysis, prototyping, Observation, video anal- Contextual inquiry, pro-
usability evaluations ysis totyping

In user-centred design, ethnology, and contextual design, participation can be characterised as an

approach by the designer to gain information from participants. The fundamental difference in partici-
patory design is that it encourages participants to actively take part in the decision making and creative
processing of the solution [11]. The goal of participatory design is not just to empirically understand the
design activity (or users, as in user-centred design), but to simultaneously envision, shape, and transcend
it to benefit the participants [48].
The ideological grounding of participatory design emerged from Scandinavian workplace democracy,
to ensure that people who are affected by technology can also participate in making decisions about it [6,
10, 15, 36]. In participatory design, the following statements are understood as guiding principles: partici-
pants from diverse backgrounds are seen as experts in how they live their lives and design in collaboration
with other professionals [41, 42], participants have the right to influence technological decisions affecting
their private and professional lives [5], and especially, participatory design is seen as appropriate in the
context of special needs [4, 12, 14, 33]. Thus, we have based our workshops on participatory design, to
adopt these principles, and we have implemented value-focused thinking as a requirements elicitation
technique.
 Learning Space Management System 5

4 Value-focused thinking

Value-focused thinking (VFT) comes from the operational research field and has been applied to decision
problems in multiple domains, such as defence, environment, energy, government, corporations, and
intelligence [40]. The underlying principle of VFT is that, when faced with a decision problem, participants
should first examine their values. In general, values are core concepts within individuals and society [51].
Values are desirable and trans-situational goals that serve as principles that guide one’s lives [13, 44].
Keeney [22, 23] employs values as principles for evaluation of actual or potential consequences of action
or inaction, of proposed alternatives, and of made decisions. In VFT, decision makers reflect what they
want to achieve, instead of immediately comparing alternative solutions. Values, moreover, can be made
explicit for examination, by associating them with a specific statement of objectives, which are in the
form of a verb, followed by an objective [22, 24].
The basic steps of the VFT process are as follows: develop a list of values, convert values to objectives,
and classify them, as a means-ends objective network [46, 47]. The starting point is the statement of
the problem to be solved. The definition of the problem must be made carefully, to ensure a shared
understanding of the situation. Participants are asked to make a list of anything that they hope to
achieve, by solving the problem being addressed. This is done without any restrictions or constraints in
reflection, to reach the different dimensions that participants find valuable. After generating the initial
list, participants are encouraged to extend the list, by using different mind-probing techniques (Table 3
in [23]). For example, participants can be asked to review each item and articulate why they care about
it, which in turn might lead to new items. This phase of producing a comprehensive list requires intensive
thinking and discussion, and it will most likely take several iterations.
The list is considered as completed, when participants cannot find any new information about the
problem. Then, each list item is translated into the format of objectives (Keeney defines this phase as
converting values into a common form). For example, if the participants expressed that the school day
is too busy, the item might be ”rush”, and the objective would be ”reduce rush”. This might raise a
discussion, for instance, on why there is rush and how it could be reduced. This, in turn, may generate
new items and objectives. Finally, the list needs to be examined for possible redundancies, which have
to be eliminated.
The next phase is to classify objectives as fundamental - or means objectives. Fundamental objectives
characterise the essential interests in the decision situation, representing the goals that participants value.
Means objectives are of interest due to their implications for the degree to which fundamental objectives
can be achieved. For example, if reducing rush is a fundamental objective, the respective means objectives
could be about having the needed accessories available. Finally, the structure of these objectives is
illustrated, by building a means-ends network, which demonstrates how the different objectives are related
to each other. The process of structuring objectives results in a deeper and more accurate understanding
of what one cares about and helps to clarify the decision context and enhances the quality of decisions
[23].

5 Methodology

5.1 The stakeholder organisation

The Valteri School Onerva is one of the six learning and consulting centres for Valteri schools that operate
under the Finnish National Board of Education. The school provides services that support learning and
school attendance, in order to implement general, intensified, and special support. In the school, education
is combined with rehabilitation and guidance that support learning, to form a seamless whole. The school
has expertise particularly in supporting needs relating to vision, hearing, language, and interaction. The
school’s mission is to increase the accessibility of support services and promote the neighbourhood school
6 Tuhkala, Isomäki, Hartikainen, Cristea & Alessandrini

principle. The school aims to realise this, by making their operations more effective, creating new action
models and innovations, and utilising new technology. The aim is to develop solutions for learning and
rehabilitation that support learning for individuals. The school’s activities are guided by a development-
oriented approach and the utilisation of research and networking.

5.2 The data gathering process

To gather the data for the extraction of our requirements rooted in the theory as explained above, we
have organised four workshops (Table 2) in collaboration with the school’s staff. The data was collected
by recording the workshops with a video camera or mobile phone; the researchers also took notes.
The participants were special education teachers, occupational therapists, visual sense specialists, and
researchers. The researchers who participated in the workshops were all from the University of Jyväskylä,
Faculty of Information Technology. There was some variation between workshops: in the first workshop
there was one person from the technical staff, and in the last workshop, there were two members of the
instructional staff, but otherwise, the membership stayed constant. All of the workshops were held in the
old school’s facilities, to help researchers understand the context at the given time and provide teachers
and staff members with familiar surroundings.

Table 2. Value-focused thinking workshops [50].

1. 14.5.2014 8 teachers, 1 technical staff, 3 researchers Video

2. 24.8.2014 6 teachers, 4 researchers Video
3. 27.9.2014 6 teachers, 3 researchers Audio
4. 12.12.2014 6 teachers, 2 instructors, 3 researchers Video

The first workshop acquainted the participants with one another and familiarised everyone with the
context of our study. Informal group discussions were conducted, during which we asked questions about
the plans for the new school building, elicited their ideas of an open and adaptable environment, and dis-
cussed the initial needs for the learning space management system. The technical staff member presented
a three-dimensional (3D) model of the new school building, and researchers analysed it, together with the
participants. The researchers produced conceptual maps of the building, to gain a better understanding
of the new environment. Finally, the participants discussed the initial desired functions and the possible
users of the system.
In the second workshop, the participants were asked to provide ideas that they considered important
for the open and adaptable environment. This triggered intensive discussions, which resulted in a list of
words (items) which described anything that the participants perceived as valuable in the school context.
The list was reviewed and discussed again, and the participants were asked to classify these ’raw’ words,
by defining higher level categories to encompass all of them. Finally, the participants transformed the
items into objectives, which represented their shared understanding of how each item could be achieved.
Next, the objectives were examined as a whole, with the goal of removing redundancy and disentangling
abstract objectives, transforming them into more concrete ones. The emerging objectives were scrutinised
again, by asking the participants ”why is this item important?”. The goal of this iterative, cyclical process
was to encourage more elaboration of the objectives, as well as good grounding and justification for each
emerging objective, and finally, good placement in the overall context. Because time was limited during
the workshop itself, the participants were asked to finish the task by themselves afterwards, and they
sent the final document by email to the researchers.
 Learning Space Management System 7

5.3 Extracting functional requirements

The analysis of the first two workshops was based on the VFT methodology. First, the recordings were
checked, to ensure that there was no missing data, and to verify the notes the researchers took during the
discussion. The resulting document then included a full list of objectives. When analysing the objectives,
we found that some of them were directly related to the actual system, whilst others related to the
whole organisation. Therefore, the objectives were divided into two categories: system objectives and
organisational objectives. From the system objectives, we derived the requirements that defined the
initial functions of the system and illustrated them as a use case diagram. Every use case was then
described in use case scenarios, which detailed how the user interacts with the system.
In the third workshop, the researchers described to participants how the use case diagram were
constructed and how the system would be used, by describing the use case scenarios. Furthermore, the
researchers presented initial user profiles, system architecture, and non-functional requirements. The
participants then discussed the requirements and gave feedback on how they could be enhanced. After
the workshop, the requirements were updated, based on the feedback from the participants. The final
version of the document was sent to the participants two weeks before the final meeting, in December
2014. In the last meeting, participants evaluated the outcomes and validated the produced requirements.
The participants appreciated the transparency of the design process and how researchers were able to
communicate using language they understood. Finally, researchers thanked the participants for their
collaboration and discussed future plans for the prototype development.

5.4 Identifying objectives

The recordings of the four workshops were transcribed completely, in order to gain an overall view of
the data, which were then exported into the ATLAS-ti software, for a more detailed interpretation. Data
was analysed through a process of open coding [9], to develop a list of utterances that are related to the
objectives of the group, that is, what the group considered important, or the way they thought that the
desired situation could be achieved. All 153 utterances were examined one by one and assigned at least
one code. The coding process was overlapping: a single utterance could be connected to many different
codes and vice-versa. If it was impossible to connect an utterance with any of the previous codes or
imagine a new code, the utterance was removed as an irrelevant phrase. Finally, 133 utterance remained
that had been assigned at least one code. The rejected utterances were examined again, to ensure that
no relevant data was removed by chance or mistake.
The utterances inside the codes were refined, to ascertain that the codes had a coherent structure. The
codes, including the assigned utterances, were analysed, to differentiate between fundamental objectives
and means objectives. If the assigned utterance expressed an essential objective, it became a candidate for
a fundamental objective. If the assigned utterance expressed something that was important because of its
implications for some other objective, it was a candidate for a means objective. Finally, the transcriptions
were read through again, to validate the structure of the objectives.

6 The resulting objectives

The fundamental objectives regarding an open and adaptable environment were identified as follows:
improving communication, strengthening the community, increasing efficiency, enabling functionality,
taking special needs into account, and ensuring privacy (Table 3). These are further discussed one by
one. Each of these fundamental objectives were allocated means objectives, in order to bring these
objectives closer to the actual implementation. Moreover, means objectives were further classifed into
organisational means and technological means. Organisational level means represent the social actions
that contribute towards the fundamental objective. System level means were defined as those features of
the system that could possibly contribute to an associated fundamental objective.
8 Tuhkala, Isomäki, Hartikainen, Cristea & Alessandrini

Table 3. Summary of identified objectives: fundamental objectives and means objectives (organisational and
technological).

Fundamental objective Organisational means Technological means

Improve Communication culture; Access with mobile devices;
communication Discuss conflicting reservations; Automatic conflict handling;
Information about reserved spaces;
Purpose for reservations;
Owner of a reservation;
Strengthen Responsible use of resources; A “right of way” feature;
community Negotiated rules and norms; Reservation status;
Open discussion;
Increase Planned behaviour; Visual information;
efficiency Real-time information;
Usability;
Mobile use;
Enable Think differently; Recommends suitable spaces;
Functionality Functional pedagogy; Shows accessories;
Creative use of spaces; Shows the purpose of a space;
Shows size of a space;
Accessible from different locations and
with different devices
Pedagogical Empower students; Authentication policy;
use Guide to responsive use of ICT; Generic student accounts;
Take account of special needs;
Accesible user interface;
Ensure privacy and security Respect privacy of others; Critical information on dedicated
servers

6.1 Improved communication

The first fundamental objective regarded improved communication. The hope was that teachers, staff,
and students would not be isolated in the classrooms and this would encourage more communication
between people. We thus interpreted communication as a central objective, even though it often appeared
implicitly in the data, because it is strongly connected to other objectives. For example, the connection
with privacy appears as a need to have spaces available for private conversations between teachers,
students, and other stakeholders. Participants emphasised that, regardless of the features or possibilities
of the system, there is a need for a culture of open communication. It is unavoidable that conflicts
will occur when adjusting to a new environment. Participants agreed that the responsibility for solving
conflicts cannot be outsourced entirely to technology. Even when a mechanism for automatically resolving
reservation-related conflict would exist, the prioritisations policy must be determined by the people.
Communication can be improved in many ways at the system level. The primary feature required
was that the system could be accessed by mobile devices. The participants stressed that they do not
have time to look for a desktop computer during the day. One proposition was that there could be tablet
devices ported near the learning spaces, making it easy to check the status of the space and make a
reservation. The participants brought up the issue that information related to reservations needed to
be easily accessed and needed to contain some mandatory fields: contact information of the person who
made the reservation and the purpose of the reservation. From a pedagogical perspective, there should
also be features allowing for commenting, rating and sharing knowledge about the learning possibilities
of spaces.
 Learning Space Management System 9

6.2 Strengthened community

A strong community was conceptualised as a situation wherein the whole school community is able to
negotiate shared goals among stakeholders and work together towards them. As discussed before, the
participants emphasised the need for a culture of open communication. The participants concluded that
they needed to learn ways to co-operate in an open and adaptable environment: the actions are less
confined to classrooms, and possible conflicting encounters need to be negotiated. It is not just the
policies and rules that need to be negotiated with the school staff, the whole operational culture of the
school needs shifting.
The participants proposed an interesting feature for the system, which was named ”right of way”.
The idea was that the system could understand if someone had privileges to certain spaces and auto-
matically reorganise the reservations, based on these privileges. This raised an intense discussion about
what constituted privileges and whether this idea conflicted with the open and adaptable environment.
Moreover, this feature would be rather complicated to implement, technically.
An essential method of strengthening the community was found to be the possibility of marking
reservations with open or closed status. An open reservation means that the space is reserved for certain
people, but others are still welcome to use it at the same time. Some spaces are divided into smaller
rooms or areas, which could be used in parallel. For example, two classes of deaf children, communicating
via sign language, could share the same room, as long as they would use the separating curtains available
in the room. This feature was appreciated by the participants, because it further supported the idea of
collaboration and more efficient use of facilities.

6.3 Increased efficiency

The participants extensively discussed how everyday life would be organised in the new environment.
The idea of not having their own classrooms was both fascinating and frightening. The main expectation
from the technological tool was that it would help to organise the school activities. This is a crucial
issue and affects the whole work community, as one teacher commented: ”I think, it [the system] would
help to sort things out, without unnecessary hassle. It is something that would have a great impact on
our work atmosphere”. We interpret that time is the most limited resource the participants have, and
it is extremely important that using the developed technology does not waste it. The participants also
emphasised how the ability to plan activities beforehand will make the working day more tranquil.
When considering the actual system, the participants described that efficiency was about getting real-
time information that could be used everywhere and that was easy to use. They also noted the possibility
of having visual information. A concrete example of the relationship between ease of use and efficiency
being discussed was based on their previous experiences with a facility management system which had
a complicated function for removing reservations and resulted in too many ”no-show” reservations. A
visual view (visual interface) of the building was important for the participants. They were used to
perceiving the dimensions of the new building on the map. The possibility of making reservations with
a visual picture was thought to be more accessible than, for example, a list of available spaces. Mobile
access was again mentioned, because it supported the idea of an open and adaptable environment, by
encouraging people to move around.

6.4 Enabling functionality

The participants shared the view that action-based learning has a very important role in special education;
therefore, enabling functionality is one of the main goals of the open and adaptive environment, and so,
it seemed rather self-evidently to qualift as a functional objective. Functionality was conceptualised
as a vision where activities are always happening in the space that is most suitable for the intended
10 Tuhkala, Isomäki, Hartikainen, Cristea & Alessandrini

pedagogical practice and that is available at the current moment. The participants hoped that a more
functional environment would lead to more creative pedagogy, due to the possibilities the new learning
spaces are offering. However, creativity was seen as a challenge: how to question the old practices (and
think differently) and pedagogically combine the needs of the students and new learning spaces?
The main question at a system level was what spaces are made available for reservation. There seemed
to be contradictory views between the new way of understanding all spaces as ”open learning spaces”
and the need for individual and private spaces for certain tasks. This discussion resulted in interesting
observations; for instance: if there is a room with several workstations, does the reservation apply to
the whole room, or is it possible to reserve only a single workstation? Solving these issues led to a
clearer understanding of the level on which the decisions are made: between people, pre-programmed in
technological systems, or as institutional policies. According to the participants, the following features of
the system would enable functionality: the system is able to recommend the most suitable spaces based
on certain criteria, it is easy to see important information in the system, and the system can be accessed
from any location in the school, with most used devices.

6.5 Pedagogical use

The students of the school have a wide range of special needs. Different perceptional abilities present
a challenge between the creative and dynamic use of learning spaces and the need for structure and
formality. For example, it is essential for blind students to learn how to navigate through the building
and find the necessary accessories inside the learning spaces on their own. The school introduced several
guides for this, including typical tracks for blind people, but also innovative uniquely textured walls,
which helped identify the respective spaces, as well as a novel sound-based guidance system (specific
intersections emitting different little tunes, to be uniquely identifiable).
The participants, however, discussed that the world itself is not structured for the needs of blind
people, and an important aim is to teach students to act independently outside the school. This reflects
the idea that using the system should be one way to facilitate the students’ independence. The system was
seen as an opportunity to enhance responsibility, by empowering students to reserve learning spaces for
themselves and by guiding students towards responsible use of information and communication technology
(ICT). The participants noted that permitting students to use the system could result in accidental or
intentional misuse, but they seemed to agree that, despite the possible unwanted scenarios, it is important
to accustom students to ICT.
An important issue was to decide on user policies and authentication within the system. One possibil-
ity was to create user accounts for every student, but this would raise challenges related to security and
technical implementation. Information related to students has high-security classification, which would
mean tight restrictions in the system. The participants proposed the possibility of making generic user
accounts for students, so their personal information could not be revealed. Special needs were to be taken
into account in system development, to make pedagogical use of the system possible.

6.6 Ensuring privacy and security

An important matter of discussion was how privacy could be ensured in the open and adaptable environ-
ment. The participants emphasised the need for private spaces, to have conversations with stakeholders
and how this privacy needs to be respected. They also commented that visual positioning information
about staff or students could be very useful, but that it raises many privacy-related problems. However,
participants explained that they have actually had emergency situations during which a student has been
completely lost.
From a technical perspective, the discussion focused around how the current technological infras-
tructure is connected to the system and what security vulnerabilities it might cause. The participants
 Learning Space Management System 11

concluded that critical student information is stored in dedicated servers and that access to the system
should be restricted.

7 The resulting learning space management system prototype

In order to implement the objectives that resulted from the VFT workshops, firstly, the researchers
created a use-case diagram. This use case diagram of the prototype is presented in Figure 3. The functions
within Onspace mobile are optimised for mobile devices (responding to the fundamental objective of
increased efficiency, via the technological means of ’mobile use’, as well as the fundamental objective
of enabling functionality, via the technological means of accessibility from different locations and via
different devices). The administration functions Onspace web are only available from the administrative
interface (responding to the fundamental objective of ensuring privacy and security, viathe technological
means of having critical information on dedicated servers).
These two different interfaces reflect the fact that the prototype allows for two user roles: user and
admin(istrator). In addition to the user functions, the administrator can edit all reservations, user infor-
mation, and learning spaces. The server functionality is developed with the Django Web framework, the
PostgreSQL open source database, and HTTP servers Nginx and Gunicorn. The user interface is built
on a variety of open source Javascript libraries.

Fig. 3. Use case diagram.

The main use scenario, user logging in and making a reservation with the map-based interface, is
presented in Figure 4: 1) The user logs in with his credentials. 2) The prototype automatically assigns
the current date and the next rounded half hour as the starting time for a reservation and displays
the first floor of the building. If the user changes the parameters, the map under the search view is
immediately updated. 3) The map shows the areas in the single floor of the building and how many free
learning spaces the area has. 4) The user chooses the area C from the first floor and can now see the map
of the area, which has currently four available learning spaces. 5) The user clicks on the desired learning
12 Tuhkala, Isomäki, Hartikainen, Cristea & Alessandrini

space, and the reservation model opens. The learning space information includes a description and the
accessories it has. The user can write the purpose of the reservation and needs to choose whether the
reservation is open or closed for other people. 6) The user can see all the reservations as a list or on a
calendar. The reservations can be edited and removed, by clicking.

7.1 Prototype development and additional participatory design workshops

The prototype implementation was carried out by the first author and a project researcher during May -
December 2015. During the development phase, we organised another set of monthly participatory design
workshops (Table 4). The first meeting included mainly decision-making stakeholders, such as the Head
of the School, and was focused more on project practicalities, such as timetables, responsibilities, and
contracts. The school’s ICT staff manager was also present, to describe the current technical infrastruc-
ture. We agreed to develop a responsive web application that could be used by different computers and
mobile devices, due to the fact that the school staff uses a broad range of mobile devices, from different
manufacturers.

Table 4. Participatory design workshops during the implementation stage.

Initiation 17. June, 2015

1. workshop 27. August, 2015
2. workshop 17. September, 2015
3. workshop 21. October, 2015
4. workshop 19. November, 2015
Final meeting 17. December, 2015

The workshops represented iterative cycles of negotiation, development, and demonstration. The
participants of the workshops were: special education teachers, occupational therapists, visual sense
specialists, researchers, and developers - similar to the set of design workshops, which defined the initial
set of objectives. This allowed for them to follow the transformation of their objectives into practical
features of a running software system. The implementation had a modular nature, in order to be able to
add features in an incremental way, as well as to easily rectify individual features, based on the workshop
feedback from the participants. Every workshop began with an explanation on what features we had been
working on since the previous workshop. Then, the problem was approached with different techniques: by
presenting questions, having group discussions, and using a sketching tool. Feedback from the workshop
helped in refining the features and deciding on priorities. The prototype was ready by the end of 2015. In
the final meeting, the prototype was presented and validated. We made an agreement that the developers
will produce a documentation for the school’s ICT staff and help them with technical issues. This enabled
the school to continue the development of the prototype, according to any further needs. This was made
especially easy by the modular approach of our implementation, which allows further extensions, based
on the growing needs of the school.

8 Discussion
The design process described is an example of re-imagining a rather typical information management
system, but for a completely new environment, represented by the open and adaptable school. It was
clear from the beginning that we needed to reinvent the characteristics of facility management systems. In
practice, we needed to encourage the participants to reflect on the new surroundings and their everyday
work, to frame what was important to them and to clarify what they wanted to achieve. To reach this
 Learning Space Management System 13

Fig. 4. Use case of logging in and making a reservation.

14 Tuhkala, Isomäki, Hartikainen, Cristea & Alessandrini

goal, we organised four workshops, during which we applied value-focused thinking, to identify objectives
for a learning space management system for an open and adaptable environment.
Our utilisation of value-focused thinking had two stages: first, we needed to analyse the workshops
from the perspective of requirements specification, in order to establish necessary attributes of the system
- that is, functions, a use case diagram, and use case scenarios. After the workshops, we made a more
in-depth investigation of the data, using an open coding analysis. This two-staged analysis was used to
verify our results. As Morse et al. [35] state, data may demand to be treated in different ways, so the
analytic procedure should match the research questions. The first analysis stage was more practical and
straightforward, while the second stage required a more reflective strategy and critical discussions about
the project, between the authors of this paper.
The fundamental objectives identified were, as described above: improving communication, strength-
ening community, increasing efficiency, enabling functionality, pedagogical use of the system, and ensuring
privacy and security. These fundamental objectives, as well as the means to achieve them, are described
at a system - as well as at an organisational level. We argue that this approach helped and will help
developers, in general, to take a more holistic view in the development phase. The functions and features
of the system need to be considered together with organisational level means, and they should be in
line with approved fundamental objectives. The results render a more in-depth representation about the
context, people, and environment for which the system is developed.
We implemented a prototype of the system and involved our stakeholders in monthly participatory
design workshops. The participants had a real opportunity to influence the prototype development and
there was strong collaboration between researchers and participants. Researchers were able to learn about
the work and the new environment of the participants and the researchers were able to share knowledge
about technical possibilities, as well as restrictions. The workshops were advantageous, because the
stakeholders continuously discussed the underlying philosophy of the new school and how the prototype
should support it. The concept of the ”old way” was used to describe the traditional school, where teachers
have their own classrooms and learning activities are pre-programmed in timetables. The prototype that
supports the ”new way” would enable the dynamic and creative use of learning spaces and prevent
teachers from returning to the habit of reserving a single space for extensive periods of time. This
method additionally helped the developers to understand the most important objectives and optimise
their resources to meet then.
We also collided with various issues during the design process. VFT does not put emphasis on the
complex power relationships participants may have. The method assumes that people are able to commu-
nicate their thoughts, regardless of the social hierarchies that may constrain the discussion. Furthermore,
VFT examines the identified objectives as a whole, while the objectives between different stakeholders
might be very conflicting. The question is, whose objectives are we supposed to meet? As an example,
the requirements did not include a feature to make recurring reservations. However, when the implemen-
tation phase was ending, the administrative staff was disappointed, because of the lack of that feature.
They need to rent certain assets for other organisations and the contracts are made for long time periods,
and manually inputting and updating these reservations with the prototype would be an arduous task.
Furthermore, the rationale of VFT is that decision-making is based on the values of decision makers,
rather than just comparing possible alternatives. The concept of value is very challenging, because of
the different definitions of value in different research fields and even among individuals. Keeney’s [22, 23]
definition is very general, and the difference between the concepts of value and objective is not completely
clarified. To underline the point, for some people, value is about currency or efficiency and for others it
is about ethical questions. As an anecdote, Cockton [7, 8] changed the name of the concept from value
to worth after struggling with the same issue. It may seem appealing to use a pre-defined set of values,
as in Schwartz’s [45] theory of basic values, which provides more depth to the contents and structure of
values, but as Isomursu et al. [20] discussed, using a pre-defined framework to analyse and interpret the
findings can lead to confirmation bias.
 Learning Space Management System 15

Even if we embrace Keeney’s definition, the question arises of how to reach abstract constructions that
may be difficult to form as statements. For example, Iversen et al. [21] pointed out that values are not
static entities that are waiting for researchers and developers to collect them, but more like changing,
complex and abstract ways of being and thinking. Keeney seems to take it for granted that decision
makers are automatically people who are able to express what is important to them. For example, when
designing with children, there should be more appropriate methods than just asking ”what it is that one
cares for”. People’s values tend to emerge, change and even conflict, and researchers should carefully
consider who is answering these questions and what they mean.

9 Conclusion
In this paper we have described how the objectives for a learning space management system for a very
special type of adaptive school were identified with the value-focused thinking method and how the system
prototype was developed. We find it of utmost importance to understand the participants as a human
agents, with intentions, feelings, and attitudes, instead of contenting ourselves with a generic definition
of users with shared goals [18, 19]. Different stakeholders consider the confronted design problems from
their own perspectives and it needs to be acknowledged that the goals of the stakeholders can conflict.
Concludingly, we found value-focused thinking as an applicable method, allowing for a holistic approach to
requirements elicitation. However, at the same time, we found that more focus on the different stakeholder
roles is needed. We have implemented a system prototype which instantiates the identified objectives.
However, more data is needed to investigate the outcomes of the developed system prototype and the
impact of using value-focused thinking.

Acknowledgements
We would like to express our gratitude to the Valteri School Onerva’s personnel for participating in the
research project. We also thank Kirsi Heinonen, M.Sc., for assisting in the data collection. This research
has been funded by the Valteri School Onerva and the University of Jyvaskyla, Faculty of Information
Technology, under the projects entitled ONSPACE and ONSPACE2. The research did not receive any
specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

References
1. Abelein, U., Paech, B.: Understanding the influence of user participation and involvement on system
success – a systematic mapping study. Empirical Software Engineering 20(1), 28–81 (feb 2015), http:
//link.springer.com/10.1007/s10664-013-9278-4
2. Alessandrini, A., Cappelletti, A., Zancanaro, M.: Audio-augmented paper for therapy and educational in-
tervention for children with autistic spectrum disorder. International Journal of Human-Computer Studies
72(4), 422–430 (apr 2014), http://linkinghub.elsevier.com/retrieve/pii/S10715819
3. Barki, H., Hartwick, J.: Measuring user participation, user involvement, and user attitude. MIS Quarterly
18(1), 59–82 (1994)
4. Benton, L., Vasalou, A., Khaled, R., Johnson, H., Gooch, D.: Diversity for design. In: Proceedings of the
32nd Annual ACM Conference on Human Factors in Computing Systems - CHI ’14. pp. 3747–3756. ACM
Press, New York, New York, USA (2014), http://dl.acm.org/citation.cfm?id=2556288.2557244http:
//dl.acm.org/citation.cfm?doid=2556288.2557244
5. Bergvall-Kåreborn, B., Ståhlbrost, A.: Participatory design: one step back or two steps forward? In: Proceed-
ings of the Tenth Anniversary Conference on Participatory Design 2008. pp. 102–111 (2008)
6. Bjerknes, G., Bratteteig, T.: User participation and democracy: a discussion of Scandinavian research on
system development. Scandinavian Journal of Information Systems 7(1), 73–98 (1995), http://citeseerx.
ist.psu.edu/viewdoc/download?doi=10.1.1.91.6452{\&}rep=rep1{\&}type=pdf
16 Tuhkala, Isomäki, Hartikainen, Cristea & Alessandrini

7. Cockton, G.: Value-centred HCI. In: Proceedings of the Third Nordic Conference on Human-Computer In-
teraction - NordiCHI ’04. pp. 149–160. ACM Press, New York, New York, USA (2004), http://dl.acm.org/
citation.cfm?id=1028038http://portal.acm.org/citation.cfm?doid=1028014.1028038
8. Cockton, G.: Designing worth is worth designing. In: Proceedings of the 4th Nordic Conference on Human-
Computer Interaction Changing Roles - NordiCHI ’06. pp. 165–174. No. October, ACM Press, New York,
New York, USA (2006), http://portal.acm.org/citation.cfm?doid=1182475.1182493
9. Corbin, J.M., Strauss, A.: Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative
Sociology 13(1), 3–21 (1990)
10. Ehn, P.: Work-Oriented Design of Computer Artifacts. Ph.D. thesis, Umeå Universitat (1988), http://www.
diva-portal.org/smash/record.jsf?pid=diva2:580037{\&}dswid=-8358
11. Frauenberger, C., Good, J., Fitzpatrick, G., Iversen, O.S.: In pursuit of rigour and accountability in partic-
ipatory design. International Journal of Human-Computer Studies 74, 93–106 (feb 2015), http://dx.doi.
org/10.1016/j.ijhcs.2014.09.004http://linkinghub.elsevier.com/retrieve/pii/S1071581914001232
12. Frauenberger, C., Good, J., Keay-Bright, W.: Designing technology for children with special needs: bridging
perspectives through participatory design. CoDesign 7(1), 1–28 (mar 2011), http://www.tandfonline.com/
doi/abs/10.1080/15710882.2011.587013{\%}5Cnhttp://dx.doi.org/10.1080/15710882.2011.587013
13. Friedman, B.: Value-sensitive design. Interactions 3(6), 16–23 (dec 1996), http://portal.acm.org/citation.
cfm?doid=242485.242493
14. Guha, M.L., Druin, A., Fails, J.A.: Designing with and for children with special needs. In: Proceedings of the
7th International Conference on Interaction Design and Children - IDC ’08. p. 61. ACM Press, New York,
New York, USA (2008), http://dl.acm.org/citation.cfm?id=1463719http://portal.acm.org/citation.
cfm?doid=1463689.1463719
15. Halskov, K., Hansen, N.B.: The diversity of participatory design research practice at PDC 2002–2012. In-
ternational Journal of Human-Computer Studies 74, 81–92 (feb 2015), http://linkinghub.elsevier.com/
retrieve/pii/S1071581914001220
16. Harris, M.A., Weistroffer, H.R.: A new look at the relationship between user involvement in systems devel-
opment and system success. Communication of the Association for Information Systems 24(1), Article 42
(2009)
17. He, J., King, W.R.: The role of user participation in information systems development: implica-
tions from a meta-analysis. Journal of Management Information Systems 25(1), 301–331 (jul 2008),
http://search.ebscohost.com/login.aspx?direct=true{\&}db=buh{\&}AN=33245055{\&}lang=de{\&
}site=ehost-livehttp://www.tandfonline.com/doi/full/10.2753/MIS0742-1222250111
18. Iivari, J., Isomäki, H., Pekkola, S.: The user - the great unknown of systems development: reasons, forms,
challenges, experiences and intellectual contributions of user involvement. Information Systems Journal 20(2),
109–117 (mar 2010), http://doi.wiley.com/10.1111/j.1365-2575.2009.00336.x
19. Isomäki, H.: The Human Modes of Being in Investigating User Experience, pp. 191–207. Springer London,
London (2009), http://dx.doi.org/10.1007/978-1-84800-385-9_10
20. Isomursu, M., Ervasti, M., Kinnula, M., Isomursu, P.: Understanding human values in adopting new technol-
ogy - a case study and methodological discussion. International Journal of Human-Computer Studies 69(4),
183–200 (2011), http://dx.doi.org/10.1016/j.ijhcs.2010.12.001
21. Iversen, O.S., Halskov, K., Leong, T.W.: Values-led participatory design. CoDesign 8(2-3), 87–103 (jun 2012),
http://www.tandfonline.com/doi/abs/10.1080/15710882.2012.672575
22. Keeney, R.L.: Value-focused thinking: a path to creative decisionmaking. Harvard University Press, Cam-
bridge, Massachusetts (1992)
23. Keeney, R.L.: Value-focused thinking: identifying decision opportunities and creating alternatives. Euro-
pean Journal of Operational Research 92(3), 537–549 (aug 1996), http://www.sciencedirect.com/science/
article/pii/0377221796000045http://linkinghub.elsevier.com/retrieve/pii/0377221796000045
24. Keeney, R.L.: Identifying, prioritizing, and using multiple objectives. EURO Journal on Decision Processes
1(1-2), 45–67 (jun 2013), http://link.springer.com/article/10.1007/s40070-013-0002-9http://link.
springer.com/10.1007/s40070-013-0002-9
25. Keppell, M., Riddle, M.: Distributed Learning Spaces: Physical, Blended and Virtual Learning Spaces in
Higher Education. In: Physical and Virtual Learning Spaces in Higher Education, pp. 1–20. IGI Global (2012),
http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/978-1-60960-114-0.ch001
 Learning Space Management System 17

26. Kok, H.B., Mobach, M.P., Omta, O.S.: The added value of facility management in the educational environ-
ment. Journal of Facilities Management 9(4), 249–265 (2011), http://www.researchgate.net/publication/
235297138{\%}5CnThe
27. Kujala, S.: User involvement: a review of the benefits and challenges. Behaviour & Information Technology
22(1), 1–16 (2003)
28. Kumpulainen, K., Mikkola, A.: researching learning across space and time in extended learning environments
(2014)
29. Kumpulainen, K., Rajala, A.: Negotiating time-space contexts in students’ technology-mediated interaction
during a collaborative learning activity. International Journal of Educational Research (2015) (2015), http:
//dx.doi.org/10.1016/j.ijer.2016.05.002
30. Kuuskorpi, M.: Future physical learning environment. User oriented exible and changeable teaching
spaces. Dissertation, University of Turku (2012), http://www.doria.fi/bitstream/handle/10024/76724/
vaitoskirja2012Kuuskorpi.pdf
31. Lievonen, M., Kinnunen, P.: Main features of an ideal learning space: A user-based description. In: Proceedings
of the 6th Annual Architectural Research Symposium in Finland. p. 8 (2014), http://ojs.tsv.fi/index.
php/atut/article/download/47183/13933
32. Mahmood, A.M., Burn, J., Gemoets, L., Jacquez, C.: Variables affecting information technology end-user
satisfaction: a meta-analysis of the empirical literature. International Journal of Human-Computer Studies
52(4), 751–771 (apr 2000), http://linkinghub.elsevier.com/retrieve/pii/S1071581999903531
33. Malinverni, L., MoraGuiard, J., Padillo, V., Mairena, M.a., Hervás, A., Pares, N.: Participatory design
strategies to enhance the creative contribution of children with special needs. In: Proceedings of the
2014 Conference on Interaction Design and Children - IDC ’14. pp. 85–94. ACM Press, New York, New
York, USA (2014), http://dl.acm.org.libgate.library.nuigalway.ie/citation.cfm?id=2593981{\&
}CFID=469514897{\&}CFTOKEN=53769239http://dl.acm.org/citation.cfm?doid=2593968.2593981
34. McGill, T., Klobas, J.: User developed application success: sources and effects of involvement. Behaviour
& Information Technology 27(5), 407–422 (sep 2008), http://www.tandfonline.com/doi/abs/10.1080/
01449290601110715
35. Morse, J.M., Barrett, M., Mayan, M., Olson, K., Spiers, J.: Verification strategies for establishing reliability
and validity in qualitative research. International Journal of Qualitative Methods 1(2), 13–22 (2002), https:
//ejournals.library.ualberta.ca/index.php/IJQM/article/view/4603
36. Muller, M.J., Kuhn, S.: Participatory design. Communications of the ACM 36(6), 24–28 (jun 1993), http:
//doi.acm.org/10.1145/153571.255960
37. Nevari, J.: Oivaltamo: avautuva oppimistila. Ph.D. thesis, Lahti University of Applied Sciences (2013)
38. Özen, A., Ergenekon, Y.: Activity-based intervention practices in special education. Educational Sciences:
Theory and Practice 11(1), 359–362 (2011)
39. Pacheco, C., Garcia, I.: A systematic literature review of stakeholder identification methods in requirements
elicitation. Journal of Systems and Software 85(9), 2171–2181 (2012), http://dx.doi.org/10.1016/j.jss.
2012.04.075
40. Parnell, G.S., Hughes, D.W., Burk, R.C., Driscoll, P.J., Kucik, P.D., Morales, B.L., Nunn, L.R.: Invited
review-survey of value-focused thinking: applications, research developments and areas for future research.
Journal of Multi-Criteria Decision Analysis 20(1-2), 49–60 (jan 2013), http://dx.doi.org/10.1002/mcda.
442http://doi.wiley.com/10.1002/mcda.1483
41. Sanders, E.B.N., Brandt, E., Binder, T.: A framework for organizing the tools and techniques of participatory
design. In: Proceedings of the 11th Biennial Participatory Design Conference. p. 195. PDC ’10, ACM Press,
New York, New York, USA (2010), http://portal.acm.org/citation.cfm?doid=1900441.1900476
42. Sanoff, H.: Special issue on participatory design. Design Studies 28(3), 213–215 (may 2007), http://www.
sciencedirect.com/science/article/pii/S0142694X07000178
43. Sanoff, H., Walden, R.: School environments. In: The Oxford Handbook of Environmental and Conservation
Psychology, pp. 276–294 (2012), http://oxfordhandbooks.com/view/10.1093/oxfordhb/9780199733026.
001.0001/oxfordhb-9780199733026-e-15
44. Schwartz, S.H.: Universals in the content and structure of values: theoretical advances and empirical tests in
20 countries. Advances in Experimental Social Psychology 25(C), 1–65 (1992)
45. Schwartz, S.H.: An overview of the schwartz theory of basic values. Online Readings in Psychology and
Culture 2, 1–20 (2012)
18 Tuhkala, Isomäki, Hartikainen, Cristea & Alessandrini

46. Sheng, H., Nah, F., Siau, K.: Strategic implications of mobile technology: a case study using value-focused
thinking. The Journal of Strategic Information Systems (2005), http://www.sciencedirect.com/science/
article/pii/S0963868705000405
47. Sheng, H., Siau, K., Nah, F.F.H.: Understanding the values of mobile technology in education. ACM SIGMIS
Database 41(2), 25 (may 2010), http://dl.acm.org/citation.cfm?id=1795380http://portal.acm.org/
citation.cfm?doid=1795377.1795380
48. Spinuzzi, C.: The methodology of participatory design. Technical Communication 52(2), 163–174 (2005),
http://www.ingentaconnect.com/content/stc/tc/2005/00000052/00000002/art00005
49. Todhunter, B.J.: An examination of the views of key stakeholders on the development of learning spaces at a
regional university. Journal of Facilities Management 13(2), 204–222 (2015), http://www.emeraldinsight.
com/doi/abs/10.1108/JFM-06-2014-0016
50. Tuhkala, A., Isomäki, H., Hartikainen, M., Cristea, A., Alessandrini, A.: Identifying Objectives for a Learn-
ing Space Management System with Value-focused Thinking. In: Escudeiro, P., Costagliola, G., Zvacek, S.,
Uhomoibhi, J., McLaren, B. (eds.) Proceedings of the 9th International Conference on Computer Supported
Education (CSEDU). SCITEPRESS - Science and and Technology Publications, Porto, Portugal (2016)
51. Williams, R.M.: Change and stability in values and value systems: A sociological perspective. Understanding
Human Values: Individual and Societal Values 1, 5–46 (1979)
 PIV

TECHNOLOGY COMPREHENSION – SCALING MAKING

INTO A NATIONAL DISCIPLINE

by

Tuhkala Ari, Wagner Marie-Louise, Nielsen Nils, Iversen Ole Sejer, and
Kärkkäinen Tommi 2018

Proceedings of the Conference on Creativity and Making in Education -

FabLearn Europe’18
 National Perspectives of Making FabLearn Europe 2018, June 2018, Trondheim, Norway

Technology Comprehension - Scaling Making into a

National Discipline
Ari Tuhkala Marie-Louise Wagner Nick Nielsen
University of Jyväskylä Aarhus University Aarhus University
Faculty of Information Center for Computational Center for Computational
Technology P.O. Box 35 Thinking and Design Thinking and Design
Jyväskylä, Finland Helsingforsgade 14 Helsingforsgade 14
[email protected] Aarhus, Denmark Aarhus, Denmark

Ole Sejer Iversen Tommi Kärkkäinen

Aarhus University University of Jyväskylä
Center for Computational Faculty of Information
Thinking and Design Technology P.O. Box 35
Helsingforsgade 14 Jyväskylä, Finland
Aarhus, Denmark

ABSTRACT ACM Classification Keywords

We account for the first research results from a government- K.3.1. Computer Uses in Education; K.3.2 Computer and
initiated experiment that scales Making to a national disci- Information Science Education
pline. The Ministry of Education, in Denmark, has intro-
duced Technology Comprehension as a new discipline for Author Keywords
lower secondary education. Technology Comprehension is technology comprehension; computational thinking; design;
first experimented as an elective subject in 13 schools. The education; teachers; national; scaling; making; discipline
discipline combines elements from computing, design, and
the societal aspect of technology and, thus, resonates with the
INTRODUCTION
existing FabLearn and Making initiatives in Scandinavia. We
We present the opportunities and challenges of scaling Making
report the identified opportunities and challenges based on
into a nationwide discipline, which is called as "Technology
interviews, surveys, and a theme discussion with experienced
Comprehension" (translated from "Teknologi Forståelse", here-
teachers from the 13 schools. The main takeaways are: First,
after referred as TC). The new discipline is initiated by the
the teachers did not perceive Technology Comprehension as
Danish Ministry of Education and will be incorporated into
a distinguished discipline, which calls for more research on
the national lower secondary education curriculum (13-15
how Making is scaled into a national discipline. Second, Tech-
y.o. students). TC comprehends three main learning objec-
nology Comprehension opens up for interdisciplinary and
tives: Students understand the core concepts in computing,
engaging learning activities, but teachers need scaffolding and
such as programming, algorithms, pattern recognition, and
support to actualise these opportunities. Third, Technology
abstraction. Students specify and articulate a problem and
Comprehension challenges teachers’ existing competencies in
utilise an iterative design process to develop a digital solu-
relation to the discipline and students’ prerequisites and needs.
tion. Students reflect and evaluate the problem solution, its
Teachers need pedagogical means to take the societal aspect
applicability, impact, and ethical concerns, from the societal
into account within the discipline. Finally, we argue for fur-
perspective. During the implementation of the new discipline,
ther research on supporting teachers when scaling Technology
we conducted interviews, surveys, and a theme discussion
Comprehension on a national level.
with teachers from 13 Danish secondary schools to investi-
gate how the discipline is actualising and what opportunities
and challenges the teachers perceive. For this purpose, we
Permission to make digital or hard copies of all or part of this work for personal or address two research questions: How is Technology Compre-
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation hension perceived as a discipline by experienced teachers
on the first page. Copyrights for components of this work owned by others than the and what opportunities and challenges teachers face when
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
introducing TC in lower secondary education?
and/or a fee. Request permissions from [email protected].
Making considers hands-on activities, collaboration, prototyp-
FabLearn Europe’18, FabLearn Europe 2018: Conference on Creativity and Fab-
rication in Education, June 18, 2018, Trondheim, Norway ing, and iterative design to create digital and physical artefacts
© 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. and to promote self-cultivation and democratisation [2, 31,
ISBN 978-1-4503-5371-7/18/06. . . 15.00$ 4, 5, 33]. Thus, Making is strongly connected with previous
DOI: 10.1145/3213818.3213828

72
 National Perspectives of Making FabLearn Europe 2018, June 2018, Trondheim, Norway

research on design of technology and learning activities with process with an interdisciplinary approach. [22] proposes
children [34, 30, 19, 27]. Much of the research, that is carried that Making provides sophisticated tools for students to build
out particularly under the concept of Making, examines oppor- and think and a tolerant environment for experimenting, play,
tunities and challenges of Making activities in out-of-school and making errors. [8] found that students acquired, through
context [36, 29]. However, more research in formal educa- scaffolding, sufficient technical skills, mental models related
tion context is emerging [3, 17, 8, 35, 13]. Hence, our study to troubleshooting and problem decomposition, and under-
contributes to reseach in formal education context by scaling standing the possibilities and practices of sharing ideas and
Making into national policy level through TC. responsibilities.
TC is inspired by extracurricular activities in Denmark, such as Despite the opportunities, unfolding Making is challenging
Fablab at School [32], Coding Class [16], and Coding Pirates due to the incompatibility between obscure processes and
[25]. Consequently, we consider TC as a Scandinavian alterna- the formality of educational settings [32, 33]. [17] point out
tive for CS4all1 , Code.org2 , and the curriculum defined by the that teachers need to be able to navigate a complex process,
Computer Science Teacher Association3 . These are heavily manage digital and analogue materials, and balance different
grounded on computational thinking, which was made popu- modes of teaching. [33] emphasise understanding Making
lar by [40], and later clarified by [1] as: "thought processes technologies as reflective tools instead of outcomes, develop-
involved in formulating problems so their solutions can be rep- ing language to express the quality in Making, and creating
resented as computational steps and algorithms". In national means to handle insecurity, contingency, and possible lack of
educational policies, such as in the United Kingdom, computa- authority. Consequently, teachers are too often left alone after
tional thinking is positioned as a necessity for students’ future the first stage of introducing Making for schools [5].
careers [37]. In contrast, TC combines computing, design, and
a societal aspect as an interdisciplinary discipline, where the [13] examined a national level distributed Makerspace project
learning goals are understood as means, instead of end-goals, as a single case study by using thematic analysis of a set of
to engage creatively in technology development, understand heterogeneous material. They summarise five main consid-
the role of technology in society, and critically reflect the role erations: Procurement practices to identify tools, materials,
of technology in one’s life. and kits in partnership. Teacher and leader perspectives em-
phasizing professional training and knowledge sharing with
The paper reads as follows: Section 2 overviews the current mutual understanding between teachers and school leaders.
research about Making in formal education context. Section Informing national policy-making to support general manage-
3 presents the goals and learning objectives of TC, as defined ment, for example, of joint teaching material and curriculum
by the Danish Ministry of Education. Section 4 describes the development. Creating equal opportunities on both Making
research context, research questions, and the data collection and computing for both genders, especially for girls. And
and analysis methods. We present and discuss the findings in finally, creating initial interest, later supported by knowledge
sections 5 and 6. Finally, we conclude the findings, present creation, through challenging and more advanced projects.
the limitations, and propose further research.
Despite the previous findings, research on Making in the for-
mal education setting is still scarce [3, 15, 22]. Furthermore, a
MAKING IN EDUCATION great extent of studies considers Making in the STEM, Com-
Making has gained a lot of attention in recent years [4, 21, puter Science, or Natural Sciences [2, 14, 35, 36]. Only a
26, 22]. Making refers to the process of adopting a "maker few studies have examined Making in an up-scaled version,
mindset" through the creation of meaningful, significant, and which reaches beyond a municipality or a school district [13,
shareable artefacts [15, 22]. Maker mindset relates to the defi- 6]. Hence, there is a crucial need for examining Making as a
nition of technologically fluent: developing adaptive skills in part of an established and nationwide discipline.
technology and computation to empower people to manipu-
late the medium to their advantage and to handle unexpected
TECHNOLOGY COMPREHENSION
problems [24]. Making manifests Dewey’s democratisation,
Papert’s constructionism, and Freire’s critical pedagogy by in- The Danish Ministry of Education initiated TC as a new dis-
corporating democratisation and empowerment into classical cipline for lower secondary education. The curriculum was
learning-by-doing approaches, such as project-based, student- formulated by three experts representing teaching and research.
centred, and constructionist learning [4]. TC was first piloted in 13 schools as an elective course during
fall 2017. The teachers, who are assigned to teach TC, had
The possibilities of Making are recognised in education con- not received supplementary or in-service training to teach the
text. [21] argues that Making provides opportunities to in- discipline. Between summer 2018 and 2021, TC will be exper-
teract with concrete objects-to-think-with, link students’ per- imented in over 40 schools to investigate three implementation
sonal interests and learning activities, and develop self-efficacy options: i) an independent subject running from first to ninth
through affecting the surrounding environment. [4] states that grade, ii) an integrated discipline to existing subjects or iii)
Making provides an environment for working in a design combination of both, where TC is integrated into other sub-
jects between first to sixth grades and then as an independent
1 CS4All, www.cs4all.io, retrieved 14.3.2018 subject from third to ninth grades.
2 Code.org, code.org, retrieved 14.3.2018
3 Computer Science Teacher Association, www.csteachers.org, re- The ministry [11] has defined four mandatory topics that TC
trieved 14.3.2018 needs to address: i) The implications of technology and au-

73
 National Perspectives of Making FabLearn Europe 2018, June 2018, Trondheim, Norway

tomation on society, including an understanding of security, Figure 1. Specifications of TC in lower secondary education.
ethical and consequences of digital technologies. ii) Com- - Trial period from 2017 to 2020
puting as a knowledge area, including basic knowledge of - 13 public schools in Denmark, selected by the
networks, algorithms, programming, logic and algorithmic Ministry of Education to represent different
thinking, abstraction and pattern recognition, data modelling geographical areas and socio-economic
as well as testing. iii) Iterative design process as an interaction diversities
between gaining an understanding of the world that is being - 20 teachers and 303 students in the first year of
designed to and gaining an understanding of the digital tech- the project
nologies that are being designed with. iv) Complex problem - In 2019, 45 schools will have TC as a
solving, where children create new digital solutions and, hence, compulsory program
learn to argue for their relevance through an understanding of - In 2021, TC will be compulsory for K-9 students
design processes.
The ministry [12] has also defined three learning objectives
for TC: i) Students learn to produce and analyse digital prod- Table 1. Participants’ teaching background (n = 18)
ucts. ii) Students learn to develop, modify, evaluate and refine
digital products through work with remixing, refinement, and # Subjects Teaching ex-
perience
production. iii) Students learn the possibilities and role of
informatics as a catalyst for changes in society, in order to Workshop 1 - Aarhus
1 IT pedagogy over 10
strengthen the capabilities for acting in a meaningful way in 2 Math, physics, chemistry, history 3-5
a democratic and digital society, including constructive and 3 History, societal, physics, chemistry, IT over 10
critical contribution in shaping the digital society. 4 Math, physics, chemistry, TC over 10
5 Languages, math, sports, household, nature and 3-5
TC has some intersections with computational thinking and technology
computational concepts, practices, and perspectives [40, 7, 6 Math, sports, IT/Fablab over 10
20]. However, TC differs significantly from computational 7 Math, nature and technology, religion, crafts and over 10
design, TC
thinking in the following areas: Firstly, it treats computing 8 Danish, music, fablab over 10
and design as equal competency areas. Secondly, these two 9 Math, nature and technology, science over 10
areas are dependent on each other, in order to develop students’ Workshop 2 - Copenhagen
capabilities to analyse, design, and develop digital products. 10 English, history, crafts and design 5-10
Thirdly, it integrates the societal aspect, meaning the critical 11 Nature and technology, TC 0-2
12 Music, english, TC 0-2
reflection of the societal impact of technology, as a part of 13 Danish, religion, sports, music, TC 0-2
the learning objectives. In this sense, TC is related to [31]’s 14 History, religion, nature and technology, biology 0-2
"Bildung", as a way of considering complex and sustainable 15 Math, physics, chemistry, TC 5-10
learning. These three standpoints are all related to, but dif- 16-18 Unknown unknown
ferent from similar initiatives, such as CS4all in the United
States, CoolThink in Hongkong, and Computing in the United
Kingdom. We designed a six-hour workshop to provide support for the
teachers and to collect data for the study. At the beginning
METHOD of the workshop, we informed all participants about the data
The research is carried out as a co-financed research project, in collection. The workshop was executed two times at different
collaboration between the Center for Computational Thinking regions of Denmark, once in Aarhus (19th February, 2018)
and Design at Aarhus University and the Danish Ministry of and once in Copenhagen (21st February, 2018). In Aarhus,
Education, initiated in October 2017 (Figure 1). The overall there were nine participants from seven schools: eight teachers
goal is to investigate whether TC is appropriate for Danish and one pedagogical consultant. In Copenhagen, we also had
lower secondary education. The research centre will develop nine participants, of which seven were teachers and two school
support for the projects’ schools and teachers, in order to principals (Table 1).
pursue successful implementation of TC. This support includes
establishing an understanding of professional competencies Our first research question holds the assumption that the work-
of TC and facilitating peer support among the teachers. Thus, shop participants have previous knowledge and expertise in
the research perspective is focused on teachers’ perceptions. TC related contents, even though they have not received spe-
cific training for teaching TC. Thus, we needed to validate this
During winter 2017, we familiarized with the study context assumption. We designed a self-assessment questionnaire and
by conducting semi-structured interviews with 14 teachers, by provided it to the teachers at the beginning of the workshop.
following classroom activities of TC in 12 schools, and by The questionnaire consisted of four Likert scale question sets
sending a survey about the support that teachers would need [9]. To find how the participants perceive their competence
[23]. Based on these preliminary investigations, we defined in using digital tools in education, we accustomed the first
the following research questions: i) "How is Technology Com- question set from the digital competence model [28]. The
prehension perceived as a discipline by experienced teachers" digital competence model does not consider programming,
and ii) "What opportunities and challenges teachers face when thus, the second question set examined participants perceived
introducing TC in lower secondary education?" programming skills. Two final question sets examined the par-

74
 National Perspectives of Making FabLearn Europe 2018, June 2018, Trondheim, Norway

ticipants’ perceived capability to teach design and computing the participants (10) had no competence in programming with
related concepts. text-based language. This reveals that, while the participants
considered themselves as digitally competent, most had ex-
Answers to the Likert scale questions were analysed with pertise only in using visual programming language. We also
IBM SPSS Statistics. First, "I don’t know" answers were asked how the participants perceive their competence to teach
excluded as missing answers. Then, frequencies, frequency TC concepts. Over 60% of the answers to teaching the design
distributions, and portions were calculated. The question sets concepts were at least competent. In contrast, only 31.1%
were combined into four Likert scale constructs to calculate were at least competent in computing concepts. Consequently,
the means and standard deviations. However, the internal the participants perceived their competence to teach design
consistency of the constructs could not be verified, due to the concepts higher than computing concepts.
small sample size. We also compared the two workshops using
the Mann-Whitney test and found no statistically significant Besides the presented competencies, it is worth noting that
differences. two of the participants had been part of the expert group in
Danish Ministry of Education, which had formulated the exact
The qualitative data consisted of open questions in the self- competency areas, competency goals, proficiency goals and
assessment survey and a theme discussion during the work- knowledge goals for TC. As a conclusion, the participants
shop. The open questions were about learning methods, pos- perceived high digital competence and most of the participants
itive or negative experiences, needed skills, and contents of had a lot of teaching experience and from a broad range of sub-
TC. The theme discussion was arranged within the workshop
jects. The participants considered themselves more competent
and it served as our main data source. The topics of the theme
in teaching design concepts than computing concepts.
discussion were: What is TC as an elective course for you,
how do you incorporate TC in your current teaching, how do
you perceive the competence goals, and what should TC be in Participants’ perceptions of TC as a discipline
future? For the theme discussion, we supplied the participants Two of the Danish Government’s implementation options po-
with a handout of TC learning objectives. The discussion sitions TC as an individual discipline. Despite this, the partici-
was moderated by one of the authors and recorded with two pants addressed TC as a part of some other subject. For exam-
video-cameras. ple, a participant reduced TC as mere programming or other
separate skill: “We created programming and math course
The theme discussion was translated and transcribed into En- that starts in the first grade and runs through all grades. Pro-
glish because all authors are not fluent in Danish. To answer gramming is okay, but should not be a standalone subject, it
the research questions, we conducted a collaborative content should be part of the other subjects. A tool.” Likewise, when
analysis [9]. First, we negotiated the high-level objective of the participants discussed TC in the context of integrating it
the analysis. Then, we watched the discussion on a video and into other subjects, they considered TC as a tool for learning
made individual notes. After watching the video, we discussed other subjects’ content: "I think a lot about how it can be part
different interpretations and constructed themes that answer of the natural science subjects. Currently, I am also teaching
the research questions. Finally, we went back to the answers in crafts, where I think that it could fit in. But, as I said, I also
the self-assessment survey and further developed the themes. think that having it as a part of the natural science would
be very exciting for me." This also became apparent when
RESULTS the participants talked about the tasks that they involve the
We start by describing the competency of the participants students in, as noted by a participant: “they [students] created
and then continue by reporting the findings for the research math games." Another participant had integrated other sub-
questions. jects, such as entrepreneurship, into TC: “I tend to focus on the
Design part of the subject because that is what I find awesome,
Participants’ perceived competency this entrepreneurship and I try to keep asking the students
15 participants answered the self-assessment questionnaire. questions if they claim that they are done ’Design a Logo’,
As can be found in Table 1, seven participants had more than ’Find a company name’, ’Create a business model’.” As a con-
ten years, four participants had three to ten years, and four clusion, these perceptions indicate that the participants lacked
participants had less than two years of teaching experience. formalised ways of addressing TC as a distinct discipline, as
The participants had taught the following subjects: TC, Math, explicitly coined by one of the participants: “This new thing
Physics, Chemistry, History, Crafts and Design, Social studies, that is starting, I think about it as part of the existing subjects.”
IT, Danish, Sports, Fablab, Nature and Technology, Religion,
As indicated by the previous examples, the participants ex-
German, Music, Biology and Food Literacy.
plicated mainly episodic knowledge of TC. The participants’
The participants perceived their competence to use digital arguments derived from their own, or others, practices of using
tools in education high (Table 2). 90.7% of the answers to technology in education. Even when the learning goals of TC
the five questions were either competent or highly competent. were handed out to the participants, the arguments manifested
As mentioned earlier, the digital competence model does not personal beliefs, experiences, and interests. When considering
include questions about computing skills. In programming the learning objectives of TC, the participants with design
competence questions, Almost all (14) participants answered background emphasised design goals, participants with com-
that they had at least some competence with visual program- puting background computing goals, and participants with
ming languages, such as Scratch. On the other hand, most of humanistic background societal goals. Thus, the participants

75
 National Perspectives of Making FabLearn Europe 2018, June 2018, Trondheim, Norway

Table 2. Perceived competencies of the participants (n = 15).

1 2 3 4 ∑ Mean SD
Digital competence f 0 7 26 42 75 3.47 .48
5 questions f /n (%) 0.0 9.3 34.7 56.0 100

Programming competence f 18 13 9 2 42 1.93 .64

3 questions f /n (%) 40.0 28.9 20.0 4.4 93.3

Teaching design concepts f 3 11 27 1 42 2.62 .60

3 questions f /n (%) 6.7 24.4 60.0 2.2 93.3

Teaching computing concepts f 27 30 28 2 87 1.76 .68

6 questions f /n (%) 30.0 33.3 31.1 2.2 96.6
Scale: 1=not competent, 2=little competent, 3=competent, 4=highly competent

did not have a mutual understanding what TC is currently, or focus is to have a starting point that the students can relate to,
what it should be in the future, but instead relied on personal for example, in the Odense municipality we are establishing
preferences and episodic knowledge. the new light-rail. The students were concerned about what if
a blind person should cross the light-rail, can we be sure that
Opportunities of TC the train will stop. So they tried to build some censors with
Most of the discussed opportunities were confused with Micro:bits. This was the classical problem-solving setting that
technology-supported education. The participants referred the students could relate to." These examples demonstrate the
to examples how technology can support students: "I can have opportunity to actualise TC as a design process that integrates
students that are creating a paper booklet, and right next to computing, the societal aspect, and problem-solving.
them another group that works with creating a blog. Both As illustrated by the previous quotes, the participants appreci-
make equal sense. Then you do have students that are able ated the fact that TC combines computing, design, the societal
to concentrate for more than 25 minutes because you have aspect, and problem-solving together. They pointed out that,
access to different technologies." Another example was using normally, computing-related curricula are designed by people
technology to engage students with special needs: "I had an with the computing background. Hence, the learning objec-
experience, where mother of a dyslexic child contacted me. tives tend to address mere computing goals. TC opens up for
Usually, when the girl had to make presentations, she was holistic discipline goals when it is designed by stakeholders
embarrassed by doing it. She used the computer to make the from various disciplines.
presentation and felt more capable of presenting due to the
auto-correction tools." In addition, the participants brought up
Challenges of TC
the opportunity to motivate students, who use digital tools in
Several challenges emerged from the data. The participants’
spare time or to concretise abstract topics, such as perceiving
conceptions indicated uncertainty about the meaning of the
the coordinate system using Scratch.
societal aspect in TC. The participants discussed the societal
The participants presented several narratives how TC engages aspect primarily as a means to contextualise the subject with
different students. For example, a participant said: "We have real-world problems. They referred to topics that were familiar
had some boys that were very hard to engage in other subjects, from previous teaching experiences in other subjects, such
that have been very engaged and therefore also very coopera- as "fake news" in social media, election meddling, and the
tive on this matter." This was followed up by an example in earlier example about the light-rail track in Odense. Thus, the
special education context: "I have never gotten them to focus participants considered the societal aspect of TC as a means
for more than 25 minutes, now they have been working for 90 to contextualise classroom activities, instead of a learning
minutes". In general, the participants agreed on the fact that objective as such.
TC is an engaging discipline, as concluded by a participant:
"[TC] is fun, and a lot of students get engaged by it." The participants proposed students’ varying skills as a major
challenge. A participant told that: "I have some boys in my
A recurring theme was that TC opens up for more student- elective course and even before I started the teaching they
centred learning. A participant elaborated on the 9th graders’ had downloaded the files we should use. At the same time, I
sense of ownership towards making Math games to 1st graders: had a girl who did not know what a file is. The students had
"We were making math games with Scratch, it was obvious that very different skills for participating in this field." Another
older students had a sense of ownership to this assignment. participant noted that if TC is first introduced in seventh grade,
The 9th graders were supposed to make a math game for 1st the prerequisites of the subject are necessarily low. Otherwise,
graders. The day before, 9th graders used their lunch break to lack of basic skills, such as basic computer use, will prevent
go 1st graders and check if the level was too hard. Then 9th those students to pursue the actual learning goals: "I would
graders went back and adjusted the games. That’s very un- like to be better at presenting the students with a problem as
common to 9th graders to do something like that in own time." a starting point, where they can analyse, design and develop.
This was also exemplified by another participant, highlighting Currently, they have mostly worked with learning the different
how TC integrates topics that are relevant to students: "My technologies.". A participant concluded that if the basic com-

76
 National Perspectives of Making FabLearn Europe 2018, June 2018, Trondheim, Norway

puting skills are to be taught in TC, it leaves less space and TC opens up for interdisciplinary and engaging learning
time for other learning objectives. activities. As stated above, TC has similarities with current
research incorporating computing and design into curriculum
Another challenge was that students have different needs re- based and formal education [32]. We found that teachers iden-
garding the structure and guidance of TC learning activities. tified several opportunities in implementing TC: it encourages
Some students want to be challenged and to be provided with children to be creative with digital technology, to work with
less guidance, while others are incapable of acting without authentic and complex problems and to take responsibility for
clear structure and instructions: "Some of them expect to be their learning process. Moreover, the participants thought that
challenged, some of them expect to get everything served on a students perceive TC learning activities as engaging, inspiring,
silver plate. That is one of the biggest challenges I have to get and fun. TC shares [31]’s reasons for introducing making
them to be better." This indicates that TC, as a new discipline,
in curriculum-based education: developing skills related to
calls for high level of individual differentiation of the learning
computing and computational thinking, but also to digital citi-
activities.
zenship, in relation to a digitized and post-modern society. In
Finally, other identified challenges were: gender issue, teach- this way, TC embraces digital competencies as well as critical
ers’ need for time and peer support. As manifested by the and reflective personal skills that relate to [31]’s “Bildung”,
earlier quotes about students’ varying skills, gender issue is Iversen et al. (2018, in press) Computational empowerment,
an existing topic also in TC. A participant stated: "A lot of [4]’s empowerment, and also [7]’s and [20]’s computational
students want to participate in 4-6th grade, in 7-9th grade, it is perspectives. The opportunities to address digital technology
primarily boys." The participants’ conceptions distinguished from a critical and societal point of view are discussed by
between boys, as being interested and knowledgeable, and the teachers. However, the teachers do not feel capable of
girls as not necessarily interested, or engaged, in TC. The bridging between hands-on activities and more abstract discus-
participants pointed out that teachers need more time, peer sion of computational perspectives. To fulfil the opportunities,
support, and scaffolded teaching instructions to be able to scaffolding activities such as in-service training of teachers,
implement TC as a new discipline. As concluded by one par- development of textbook material, and online resources are
ticipant: " [TC] is a new subject and a new way of thinking required to support this effort.
in primary school. It requires more preparation time than TC challenges teachers’ existing competencies in relation
the ’normal subjects’, where you can adopt a lot of existing to the discipline and students’ prerequisites and needs. We
teaching material from various learning portals into your own identified the following challenges: lack of shared understand-
work."
ing of the meaning of the societal aspect in TC, students’
varying skills and needs, and paradox between instructional
DISCUSSION
structure and freedom, and lack of girls’ involvement. The
challenge of balancing between creative Making activities and
Our findings derive from the first year of scaling TC into a
formal education’s structure is already known in research con-
national initiative. Despite the fact that we are early in the
sidering Making in education [32, 33, 17]. Furthermore, some
project, the teachers provided us with important practice-based
of the challenges, such as the need for teachers’ support to use
knowledge for scaling TC and, thus, to our research questions:
digital tools and to teach computing concepts, students’ vary-
How is TC perceived as a discipline by experienced teachers
ing skills, and gender issues, are well known in other fields
and what opportunities and challenges teachers face when in-
[39, 18, 38, 10]. Our study contributes by pointing out the im-
troducing TC in lower secondary education? Consequently,
minent need for considering how the societal aspect, including
our empirical findings suggest that: i) teachers do not per-
topics like ethics, empathy, responsibility, and accountability,
ceive TC as a distinguished discipline, ii) TC opens up for
are defined as concrete learning objectives to provide teachers
interdisciplinary and engaging learning activities, and iii) TC
with tools to assess how they are being met.
challenges teachers’ existing competencies in relation to the
discipline and students’ prerequisites and needs.
CONCLUSION
Teachers do not perceive TC as a distinguished discipline.
This study contributed to the FabLearn community by report-
Technology Comprehension is a term coined by the Ministry ing the first research results from the government-initiated
of Education and, thus, not well-known among teachers or research about introducing Technology Comprehension into a
in research. Our research results reveal that teachers do not national curriculum. Based on interviews, surveys, and theme
see TC as a distinct discipline, but rather as a set of skills that discussion with highly experienced teachers, we found that
can be integrated into other disciplines. This finding can be
teachers do not perceive TC as a distinguished discipline, TC
related to what [33] considered as the impediments to inte- opens up for interdisciplinary and engaging learning activities,
grating making into K12-education. Whereas [33] found that
and TC challenges teachers’ existing competencies in relation
teachers generally lack a sufficient understanding of digital
to the discipline and students’ prerequisites and needs.
technology and complex problem solving, our study indicates
that teachers do not possess an understanding of the disciplines We identify the following shortcomings in our study: Our
(computing/design/societal aspect) related to digital technol- findings derive solely from interactions with teachers and does
ogy. The insufficient understanding of the disciplines is not not include principals, policy-makers, or students. We have
a challenge to the teachers, but rather, it is a challenge to the not taken into consideration that many teachers will ultimately
entire TC initiative and ultimately to research. teach TC without prior experiences or any compulsory educa-

77
 National Perspectives of Making FabLearn Europe 2018, June 2018, Trondheim, Norway

tion in TC. This will inarguably further emphasise the need York, USA, 613. DOI:
for better means to support teachers. Due to the scope of this http://dx.doi.org/10.1145/2485760.2485884
paper, we have not conducted an in-depth and systematic liter-
6. Susanne Bødker, Christian Dindler, and Ole Sejer Iversen.
ature review that goes beyond the recent literature in Fablearn
2017. Tying Knots: Participatory Infrastructuring at
and Making. A next step would be to systematically survey
Work. Computer Supported Cooperative Work (CSCW)
on literature within Computer Science Education, Technology
26, 1-2 (feb 2017), 245–273. DOI:
Design with Children, and other relevant research communi-
http://dx.doi.org/10.1007/s10606-017-9268-y
ties to identify what could be gained from similar studies in
these areas of research. 7. Karen Brennan and Mitchel Resnick. 2012. New
frameworks for studying and assessing the development
The challenges of implementing the political agenda to offer
of computational thinking. annual American Educational
TC to all students in Denmark, even by the highly experienced
Research Association meeting, Vancouver, BC, Canada
teachers, should be addressed merely as a general lack of
(2012), 1–25. DOI:http://dx.doi.org/10.1.1.296.6602
research about TC. Consequently, our study raises several
questions for researchers within Fablearn and Making fields: 8. Sharon Lynn Chu, Francis Quek, Elizabeth Deuermeyer,
What is TC as a discipline and how do we merge previous and Rachel Martin. 2017. From Classroom-Making to
research on computing and design education to develop TC? Functional-Making: A Study in the Development of
How do we develop a curriculum and supplementary training Making Literacy. In Proceedings of the 7th Annual
for pre-service and in-service teachers to support their TC Conference on Creativity and Fabrication in Education
teaching practices? How do we incorporate the critical and (FabLearn ’17). ACM, New York, NY, USA, Article 3, 8
societal approach of TC as concrete learning objectives, of pages. DOI:http://dx.doi.org/10.1145/3141798.3141802
which accomplishment could be assessed? How do we balance
9. Louis Cohen, Lawrence Manion, and Keith Morrison.
between the obscure structures of Making, computing, and
2013. Research methods in education. Routledge.
design with the formal curriculum in education?
10. M. J. Cox. 2013. Formal to informal learning with IT:
Acknowledgments research challenges and issues for e-learning. Journal of
We want to thank the Danish Ministry of Education and, espe- Computer Assisted Learning 29, 1 (feb 2013), 85–105.
cially, the teachers who participated in the study. DOI:
http://dx.doi.org/10.1111/j.1365-2729.2012.00483.x
REFERENCES
1. Alfred V. Aho. 2011. Ubiquity symposium: Computation 11. Danish Ministry of Education. 2018a. Kommissorium for
and Computational Thinking. Ubiquity 2011, January den rådgivende ekspertskrivegruppe for forsøgsprogram
(jan 2011), 1. DOI: for styrkelse af teknologiforståelse i folkeskolens
http://dx.doi.org/10.1145/1922681.1922682 obligatoriske undervisning. (2018). https:
//www.uvm.dk/-/media/filer/uvm/udd/folke/pdf18/jan/
2. Douglas Ball, Colby Tofel-Grehl, and Kristin A. Searle. 180124-kommissorium-for-raadgivende-ekspertskrivegruppe-teknologi
2017. Sustaining Making in the Era of Accountability: pdf
STEM Integration Using E-Textiles Materials in a High
School Physics Class. In Proceedings of the 7th Annual 12. Danish Ministry of Education. 2018b.
Conference on Creativity and Fabrication in Education Teknologiforståelse valgfag – Fælles mæl og læseplan.
(FabLearn ’17). ACM, New York, NY, USA, Article 2, 7 (2018). https://www.emu.dk/modul/teknologiforst
pages. DOI:http://dx.doi.org/10.1145/3141798.3141801 13. Eva Eriksson, Carl Heath, Peter Ljungstrand, and Peter
3. Alexander Berman, Brittany Garcia, Beth Nam, Sharon Parnes. 2017. Makerspace in school—Considerations
Chu, and Francis Quek. 2016. Toward a Making from a large-scale national testbed. International Journal
Community of Practice: The Social Aspects of of Child-Computer Interaction (2017), 1–7. DOI:
Elementary Classroom-Based Making. In Proceedings of http://dx.doi.org/10.1016/j.ijcci.2017.10.001
the 6th Annual Conference on Creativity and Fabrication 14. Deborah A. Fields, Yasmin B. Kafai, Tomoko Nakajima,
in Education (FabLearn ’16). ACM, New York, NY, and Joanna Goode. 2017. Teaching Practices for Making
USA, 9–16. DOI: E-Textiles in High School Computing Classrooms. In
http://dx.doi.org/10.1145/3003397.3003399 Proceedings of the 7th Annual Conference on Creativity
4. Paulo Blikstein. 2013. Digital Fabrication and ‘Making’ and Fabrication in Education (FabLearn ’17). ACM,
in Education: The Democratization of Invention. In New York, NY, USA, Article 5, 8 pages. DOI:
FabLabs: Of Machines, Makers and Inventors, http://dx.doi.org/10.1145/3141798.3141804
J. Walter-Herrmann and C. Buching (Eds.). Bielefield: 15. Michail N. Giannakos, Monica Divitini, and Ole Sejer
Transcript Publishers, 1–21. DOI: Iversen. 2017. Introduction for the Special issue on
http://dx.doi.org/10.1080/10749039.2014.939762 ‘Maker technologies to foster engagement and creativity
5. Paulo Blikstein and Dennis Krannich. 2013. The makers’ in learning’. Entertainment Computing 18 (jan 2017),
movement and FabLabs in education. In Proceedings of 143–144. DOI:
the 12th International Conference on Interaction Design http://dx.doi.org/10.1016/j.entcom.2016.11.001
and Children - IDC ’13. ACM Press, New York, New

78
 National Perspectives of Making FabLearn Europe 2018, June 2018, Trondheim, Norway

16. Mikala Hansbøl and Stine Ejsing-Duun. 2017. 26. Sofia Papavlasopoulou, Michail N. Giannakos, and
Hovedrapport: Coding Class - Documentation og Letizia Jaccheri. 2017. Empirical studies on the Maker
Evaluering. (2017). Movement, a promising approach to learning: A literature
https://itb.dk/sites/default/files/Rapport review. Entertainment Computing 18 (jan 2017), 57–78.
DOI:http://dx.doi.org/10.1016/j.entcom.2016.09.002
17. Mikkel Hjorth, Rachel Charlotte Smith, Daria Loi,
Ole Sejer Iversen, and Kasper Skov Christensen. 2016. 27. Jennifer A. Rode, Mark Stringer, Eleanor F. Toye,
Educating the Reflective Educator: Design Processes and Amanda R. Simpson, and Alan F. Blackwell. 2003.
Digital Fabrication for the Classroom. In Proceedings of Curriculum-focused Design. In Proceedings of the 2003
the 6th Annual Conference on Creativity and Fabrication Conference on Interaction Design and Children (IDC
in Education (FabLearn ’16). ACM, New York, NY, ’03). ACM, New York, NY, USA, 119–126. DOI:
USA, 26–33. DOI: http://dx.doi.org/10.1145/953536.953553
http://dx.doi.org/10.1145/3003397.3003401 28. Fredrik Mørk Røkenes and Rune Johan Krumsvik. 2016.
18. Shihkuan Hsu. 2017. Developing and validating a scale Prepared to teach ESL with ICT? A study of digital
for measuring changes in teachers’ ICT integration competence in Norwegian teacher education. Computers
proficiency over time. Computers & Education 111 (aug & Education 97 (jun 2016), 1–20. DOI:
2017), 18–30. DOI: http://dx.doi.org/10.1016/j.compedu.2016.02.014
http://dx.doi.org/10.1016/j.compedu.2017.04.001 29. Jean J. Ryoo, Lianna Kali, and Bronwyn Bevan. 2016.
19. Netta Iivari and Marianne Kinnula. 2016. Inclusive or Equity-Oriented Pedagogical Strategies and Student
Inflexible: A Critical Analysis of the School Context in Learning in After School Making. In Proceedings of the
Supporting Children’s Genuine Participation. In 6th Annual Conference on Creativity and Fabrication in
Proceedings of the 9th Nordic Conference on Education (FabLearn ’16). ACM, New York, NY, USA,
Human-Computer Interaction (NordiCHI ’16). ACM, 49–57. DOI:http://dx.doi.org/10.1145/3003397.3003404
New York, NY, USA, Article 63, 10 pages. DOI: 30. Michael Scaife, Yvonne Rogers, Frances Aldrich, and
http://dx.doi.org/10.1145/2971485.2971531 Matt Davies. 1997. Designing for or Designing with?
20. Yasmin B. Kafai, Eunkyoung Lee, Kristin Searle, Informant Design for Interactive Learning Environments.
Deborah Fields, Eliot Kaplan, and Debora Lui. 2014. A In Proceedings of the ACM SIGCHI Conference on
Crafts-Oriented Approach to Computing in High School: Human Factors in Computing Systems (CHI ’97). ACM,
Introducing Computational Concepts, Practices, and New York, NY, USA, 343–350. DOI:
Perspectives with Electronic Textiles. Trans. Comput. http://dx.doi.org/10.1145/258549.258789
Educ. 14, 1, Article 1 (March 2014), 20 pages. DOI: 31. Heidi Schelhowe. 2013. Digital realities, physical action
http://dx.doi.org/10.1145/2576874 and deep learning-FabLabs as educational environments.
21. Eva-Sophie Katterfeldt, Nadine Dittert, and Heidi FabLab: Of Machines, Makers and Inventors (2013),
Schelhowe. 2015. Designing digital fabrication learning 93–103.
environments for Bildung: Implications from ten years of 32. Rachel Charlotte Smith, Ole Sejer Iversen, and Mikkel
physical computing workshops. International Journal of Hjorth. 2015. Design thinking for digital fabrication in
Child-Computer Interaction 5 (sep 2015), 3–10. DOI: education. International Journal of Child-Computer
http://dx.doi.org/10.1016/j.ijcci.2015.08.001 Interaction 5 (sep 2015), 20–28. DOI:
22. Lee Martin. 2015. The Promise of the Maker Movement http://dx.doi.org/10.1016/j.ijcci.2015.10.002
for Education. Journal of Pre-College Engineering 33. Rachel Charlotte Smith, Ole Sejer Iversen, and Rune
Education Research (J-PEER) 5, 1 (apr 2015). DOI: Veerasawmy. 2016. Impediments to Digital Fabrication in
http://dx.doi.org/10.7771/2157-9288.1099 Education: A Study of Teachers’ Role in Digital
23. Sharan B. Merriam. 2009. Qualitative Research - A Fabrication. International Journal of Digital Literacy and
Guide to Design and Implementation (3rd ed.). Digital Competence 7, 1 (jan 2016), 33–49. DOI:
Jossey-Bass, San Francisco. 320 pages. http://dx.doi.org/10.4018/IJDLDC.2016010103

24. National Research Council. 1999. Being Fluent with 34. Elliot Soloway, Mark Guzdial, and Kenneth E. Hay. 1994.
Information Technology. National Academies Press, Learner-centered design: the challenge for HCI in the
Washington, D.C. DOI:http://dx.doi.org/10.17226/6482 21st century. interactions 1, 2 (apr 1994), 36–48. DOI:
http://dx.doi.org/10.1145/174809.174813
25. Rikke Toft Nørgård and Rikke Berggreen Paaskesen.
2016. Open-ended education: how open-endedness might 35. Edna Tan and Angela Calabrese Barton. 2017. Designing
foster and promote technological imagination, for Rightful Presence in STEM-rich Making: Community
enterprising and participation in education. Conjunctions : Ethnography As Pedagogy. In Proceedings of the 7th
transdisciplinary journal of cultural participation 3, 1 Annual Conference on Creativity and Fabrication in
(2016), 1–25. Education (FabLearn ’17). ACM, New York, NY, USA,
Article 8, 8 pages. DOI:
http://dx.doi.org/10.1145/3141798.3141807

79
 National Perspectives of Making FabLearn Europe 2018, June 2018, Trondheim, Norway

36. Edna Tan, Angela Calabrese Barton, Myunghwan Shin, for using ICT. British Journal of Educational Technology
and Carmen Turner. 2016. Probing Participatory 47, 1 (2016), 65–75. DOI:
Partnerships: Equitably-consequential Making by, for and http://dx.doi.org/10.1111/bjet.12220
with Marginalized Youth. In Proceedings of the 6th
39. Patricia Wastiau, Roger Blamire, Caroline Kearney,
Annual Conference on Creativity and Fabrication in
Valerie Quittre, Eva Van de Gaer, and Christian Monseur.
Education (FabLearn ’16). ACM, New York, NY, USA,
2013. The Use of ICT in Education: a survey of schools
1–8. DOI:http://dx.doi.org/10.1145/3003397.3003398
in Europe. European Journal of Education 48, 1 (mar
37. The Royal Society. 2012. Shut down or restart? The way 2013), 11–27. DOI:
forward for computing in UK schools. Technical Report. http://dx.doi.org/10.1111/ejed.12020
1–122 pages.
40. Jeannette M Wing. 2006. Computational thinking.
38. Çelebi Uluyol and Sami Şahin. 2016. Elementary school Commun. ACM 49, 3 (mar 2006), 33. DOI:
teachers’ ICT use in the classroom and their motivators http://dx.doi.org/10.1145/1118178.1118215

80
 PV

TECHNOLOGY COMPREHENSION – COMBINING

COMPUTING, DESIGN, AND SOCIETAL REFLECTION AS A
NATIONAL SUBJECT

by

Tuhkala Ari, Wagner Marie-Louise, Iversen Ole Sejer, and Kärkkäinen Tommi
2019

International Journal of Child-Computer Interaction

 International Journal of Child-Computer Interaction 20 (2019) 54–63

Contents lists available at ScienceDirect

International Journal of Child-Computer Interaction

journal homepage: www.elsevier.com/locate/ijcci

Technology Comprehension — Combining computing, design, and

societal reflection as a national subject
∗
Ari Tuhkala a , , Marie-Louise Wagner b , Ole Sejer Iversen b , Tommi Kärkkäinen a
a
University of Jyväskylä, Faculty of Information Technology, Finland
b
Aarhus University, Centre for Computational Thinking and Design, Denmark

article info a b s t r a c t

Article history: This article considers the implementation of a new learning subject "Technology Comprehension" into
Received 29 October 2018 lower secondary schools in Denmark, as part of an initiative by the Danish Ministry of Education.
Received in revised form 12 February 2019 The subject consists of learning objectives related to computing, design, and societal reflection and
Accepted 24 March 2019
was first introduced as an elective course in 13 schools to investigate how it could be integrated
Available online 27 March 2019
into the Danish education system. We present four key findings based on school visits, interviews,
Keywords: an electronical survey, two questionnaires, and workshops including theme discussions: (1) teachers
Technology comprehension did not perceive Technology Comprehension as a distinct subject, but rather as a set of skills that
Digital fabrication can be integrated into other subjects; (2) teachers pointed out that Technology Comprehension opens
Making up for interdisciplinary and engaging learning activities, but they need more scaffolding and support;
Design
(3) Technology Comprehension challenges teachers’ existing competencies and there is a need for
Computing
a framework that takes into account computing, design, and societal reflection as a whole; (4)
Computational thinking
Education Technology Comprehension appealed to various kind of students, not only those who are enthusiastic
Teachers about technical matters. This study contributes to the previous research on making and digital
National fabrication by addressing how these endeavours are implemented on a national level through engaging
Scaling with local teachers. We call for more research on scaffolding and supporting teachers to orchestrate
Subject meaningful learning activities to successfully integrate Technology Comprehension into the Danish
national education.
© 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND
license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction applicability, impact, and ethical concerns with reference to the

broader socio-political context within which it is applied.
This paper considers the implementation of a new subject – The research project was initiated in October 2017 in a col-
Technology Comprehension – across lower secondary schools in laboration between the Centre for Computational Thinking and
Denmark. Technology Comprehension (TC) is a translation from Design at the Aarhus University and the Danish Ministry of Educa-
the Danish word ‘‘Teknologiforståelse’’ and consists of learning tion (Fig. 1). The research objective was to examine the response
objectives related to computing, design, and societal reflection. to the implementation of TC, and support its implementation
A shorter version of the paper was presented in the Fablearn across 13 Danish lower secondary education schools. In partic-
Europe conference and this version expands the original paper by ular, we focused on three questions: (1) how teachers perceived
carrying out a more in-depth literature review and by presenting their competency in teaching TC; (2) what opportunities and chal-
additional results in Section 5.3 [1]. lenges teachers experienced when teaching TC; and (3) what type
TC was initiated by the Danish Ministry of Education and it is of students participated in TC. The research questions were inves-
currently offered as an elective course to Danish lower secondary tigated through school visits, interviews, an electronical survey,
education schools (13–15 y.o. students). TC is composed of the two questionnaires, and workshops including theme discussions.
three major learning objectives to develop (1) basic skills in com- Digital literacy is essential for preparing children for the op-
puting, such as programming, algorithms, pattern recognition, portunities and challenges of a fast-moving, globalised, and in-
and abstraction; (2) skills to specify and articulate a problem and creasingly digitalising world [2]. Several initiatives have been
utilise an iterative design process to develop a digital solution; established to support the development of students’ digital liter-
and (3) skills to reflect and evaluate the digital solution, its acy, including CS4all,1 Code.org,2 and Computer Science Teacher

∗ Corresponding author. 1 CS4All, www.cs4all.io, retrieved 14.3.2018.

E-mail address: [email protected] (A. Tuhkala). 2 Code.org, code.org, retrieved 14.3.2018.

https://doi.org/10.1016/j.ijcci.2019.03.004
2212-8689/© 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
 A. Tuhkala, M.-L. Wagner, O.S. Iversen et al. / International Journal of Child-Computer Interaction 20 (2019) 54–63 55

examining national curricula of three other Nordic countries. Sec-

tion 2.3 focuses on the teachers’ skills and competence because
teachers were the main stakeholders in the study. Finally, in
order to consider the impact of TC, Section 2.4 reviews previous
literature about student engagement.

2.1. Making and digital fabrication in formal education

Fig. 1. Specifications of implementing TC in Denmark. Making and digital fabrication have gained a lot of atten-
tion in recent years [see 13,24–26]. This endeavour was driven
by a shift in focus away from mere skills to use technology
Association3 These initiatives are inspired by computational think- towards digital literacy, need for creative and design skills in
ing, which was first popularised by Wing [3], and later defined engineering, and the increased availability of prototyping equip-
by Aho [4] as: ‘‘thought processes involved in formulating problems ment [13,14]. The idea is grounded in Dewey’s democratisation,
so their solutions can be represented as computational steps and Papert’s constructionism, and Freire’s critical pedagogy, which are
algorithms’’. For example, computational thinking has been just actualised when connecting democratisation and empowerment
recently integrated into the national curriculum in the United
with learning-by-doing approaches [13]. These activities are car-
Kingdom [5].
ried out at makerspaces or fablabs, where children aim to create
Through TC, we introduce an alternative approach to devel-
meaningful and shareable artefacts [26].
opment of these skills by (1) integrating computing and design
Several scholars describe the benefits of making and digi-
skills into the learning process as means, rather than viewing
these skills as mere learning outcomes; (2) supporting creativ- tal fabrication for learning. Katterfeldt et al. [24] argue that it
ity through the development of technology to understand the provides opportunities to interact with concrete objects-to-think-
impacts of technology; and (3) to critically reflect the role of with, to link personal interests and learning activities, and to
technology in the society more broadly. Thus, TC is grounded in develop self-efficacy. Blikstein [13] states that it provides an
the values commonly associated with Scandinavian participatory environment for working in the design process with an interdis-
design [6], such as democracy and empowerment, and draws ciplinary approach. Martin [26] proposes that it provides sophis-
from previous research on the integration of design competences ticated tools to build and think, and a tolerant environment for
in educational contexts [e.g. 7,8]. Furthermore, TC builds on the experimenting, playing, and making errors. Chu et al. [27] found
previous Danish initiatives designed to promote digital literacy that through making children acquired technical skills, mental
development, such as Fablab at School [7], Coding Class [9], and models about troubleshooting and problem decomposition, as
Coding Pirates [10]. TC is strongly connected to making and digital well as means to share ideas and responsibilities. However, it has
fabrication, as it involves computing activities, collaboration, pro- been criticised that these benefits are over-romanticised and do
totyping, and iterative design to create digital artefacts and aims not necessarily actualise in the formal classroom context [e.g. 28].
to promote self-cultivation and democratisation [8,11–14].
Research on making and digital fabrication has recently been
The paper is structured as follows: Section 2 provides back-
expanded into formal education contexts [e.g. 26,27,29–31]. Ac-
ground on four topics of TC from within a Nordic educational
cording to these studies, there are several areas that require
context. Section 3 describes TC more in detail, including goals
and learning objectives that were defined by the Danish Min- greater attention. Berman et al. [29] question how making could
istry of Education. Section 4 explains how the study was carried evolve beyond individual sessions to established, sustainable
out, including research questions, data collection procedures, and practices in the classroom, Martin [26] questions how making
analysis methods. Finally, Section 5 presents the results, Section 6 can align with the goals and needs of schools, whilst Eriksson
discusses four themes that were derived from the results, and et al. [31] call for an examination of the relationship between
Section 7 concludes the paper and addresses study limitations pedagogical practices and making activities. Furthermore, there is
and future research. also need for research to explore applications of making in non-
engineering contexts (e.g. social sciences and arts), because stud-
2. Background ies of making and digital fabrication occur predominantly in the
context of STEM, Computer Science, or Natural Sciences [e.g. 11,
The three core components of TC – computing, design, and 30,32,33].
societal reflection – have relevance to several research disciplines, The most seminal work related to implementation of TC was
including Educational Research, Human–Computer Interaction,
conducted by Eriksson et al. [31]. They investigated a national
and Computer Science. Computing and computational thinking
distributed Makerspace project and derived five main consider-
are a particular focus in the fields of Computer Science Education
ations for initiating and running a large-scale national project:
and Computing Education Research [e.g. 15,16]. Approaches to
(1) procurement practices, such as identifying appropriate tools
designing with children, also in educational context, is a focus
of the Interaction Design and Children community [e.g. 17–21]. and materials for schools, partnering up with companies to de-
Furthermore, societal reflection is related to research on empow- velop educational materials, and standardising maker kits for
erment and democracy in technology design [e.g. 22,23]. As the education; (2) professional training and knowledge sharing with
theme of this special issue is making and digital fabrication, this mutual understanding between teachers and school leaders; (3)
literature review focuses on these two topics. the need for informing national policy-making to support general
Section 2.1 introduces making and digital fabrication in the management, for example, of joint teaching material and cur-
formal education. In Section 2.2, we build context for TC by riculum development; (4) creating equal opportunities for both
genders, especially for girls; and (5) creating initial interest with
3 Computer Science Teacher Association, www.csteachers.org, retrieved 14.3. simple activities and progressing towards more challenging and
2018. advanced projects.
56 A. Tuhkala, M.-L. Wagner, O.S. Iversen et al. / International Journal of Child-Computer Interaction 20 (2019) 54–63

2.2. National curricula in the Nordic countries Teachers’ technological skills and knowledge have been con-
sidered through the concept of digital competence. Based on a
As the context of this study is the Danish national education, large meta-analysis, Ilomäki et al. [39] defined that teachers’ dig-
we consider the national curricula on three neighbouring Nordic ital competence consists of the technical competence, the ability
countries of Denmark: Sweden, Norway, and Finland. Note that to use digital technologies in a meaningful way, the ability to
the curricula in these countries do not use making and digital evaluate digital technologies critically, and the motivation to par-
fabrication specific concepts, but discuss digitalisation, digital ticipate in the digital culture. Røkenes and Krumsvik also applied
technology, and information and communication technology in meta-analysis and defined four teachers’ competence levels: basic
more general level. digital skills, didactic competence, learning strategies, and digital
The Swedish K-12 curriculum [34] was recently (2017, in Bildung [40,41]. While both of the digital competence frameworks
action since June 2018) changed towards a stronger emphasis take into account the societal impact of technology and its devel-
on utilisation of digital technology, opportunities and risks of opment, neither of them includes computing-specific skills, such
digitalisation, and how digitalisation is affecting the development as programming, or design-specific skills.
of society. In Norway, the curriculum addresses educational goals To orchestrate making and digital fabrication learning activ-
in a high abstraction level [35]. Technology, in the sense of its ities, teachers need knowledge in technical matters as well as
development and importance, is discussed in relation to a work- the ability to foster learning through design. According to Smith
ing human-being, ecological sustainability, and societal impact. In et al. [8], teachers need to (1) foster students’ reflection and
Finland, the new curriculum positions information and communi- knowledge construction, instead of focusing on functionality or
cation technology (ICT) as one of the seven embedded transversal aesthetics; (2) to view technology as flexible processes and ma-
competencies [36]. For the ICT competence, students should re- terials instead of fixed products; and (3) to pivot between the
flect how technology is related to sustainable development, to roles of classroom teacher, design facilitator, and coach. Teacher
everyday communication and interaction, and even to political support can be provided in the form of real-world examples,
influence. As seen, these curricula have a holistic approach re- emphasis on problem-solving and hands-on experience, scaffold-
garding the use of technology in education. Technology-related ing, peer mentoring, and collaboration [16,42]. Teachers may also
skills and knowledge are conceived as fundamental for well- benefit from access to makerspaces, separated from students,
being and self-realisation, instead of mere future working life where they could explore and learn without having to fear losing
capabilities. control or authority in front of their students [43].
When it comes to computing skills, Finland and Sweden differ
from Norway. In Sweden, algorithmic thinking and programming
2.4. Student engagement categories
are introduced in mathematics in the primary school — ba-
sics of programming for the grades 1–3, and introduction of
The importance of engagement to counterbalance low levels
algorithms and visual programming environments for the grades
of academic motivation and achievement is well understood in
4–6. In Finland, the embedded transversal competence of ICT is
the literature [44]. Engagement can be divided to behavioural,
defined for grades 1–2, 3–6, and 7–9. Accordingly, students get
emotional, and cognitive engagement, and thus, its application in
acquainted with programming as early as in the first year of
research should be accompanied with a clear definition. [45]. For
school and develop their computing skills from an abstract level
example, the contradictory role of student engagement in a fablab
towards writing programming languages. In turn, the Norwegian
context was depicted by Blikstein [13, Section 4.1]. Finn and Zim-
curriculum currently lacks the computing specific perspective,
mer [46] found that engagement can be facilitated through didac-
but there is an ongoing effort of examining how it is incorporated
tic arrangements that foster cooperative student–student interac-
into education [37].
tion, in-depth inquiry and meta-cognitive actions, and authentic
Swedish K-12 curriculum stands out from the two others with
design-related skills, such as identifying problems and utilising instruction to construct meaning beyond the classroom can fa-
a design process to develop alternative solutions. In contrast, cilitate student engagement. The engagement behaviours and
Finnish and Norwegian curricula concentrate on the perspective profiles in the learning environment are responsive to teachers’
of using technology for learning purposes, such as searching, and schools’ practices [46].
processing, producing, communicating, and judging information In relation to student outreach, one can investigate how get-
in a digital form. ting acquainted with Computer Science through making events,
As a conclusion, the three Nordic countries share the view such as coding clubs and game programming workshops, raise
that learning technology should be firmly embedded within a interest and engagement in the subject [e.g. 47]. Based on the
broader socio-political context. Sweden and Finland specify that four-phase model of interest development as defined by Hidi and
it includes at least the basics in computing, such as programming, Renninger [48], Lakanen and Kärkkäinen [49] identified four K12-
whereas only Sweden outlines that technology education should student categories and pathways characterising the longer-term
provide opportunities to create new solutions and artefacts. impact of the computing activities. However, in this work, we fo-
cus on a short-term assessment of TC. In order to depict different
2.3. Teachers’ digital competence types of engagement in learning through design, we utilise the
framework that was recently developed in the context of FabLab
The Fablearn Fellows Program is an important channel for schools in Denmark. There, Smith and Iversen [6] defined five
disseminating making and digital fabrication to schools. It pro- archetypical student categories based on interviews and surveys
vides open source curricular materials, guidelines, and support with the students.
for schools [38]. Utilising these resources, however, still relies on In the design competent category, students demonstrate the
individual teachers. The large survey among the teachers who be- development of language, repertoire, and design literacies
long to the Computing At School (CAS) network in the UK shows through problem-solving. The student profile in this category
that after significant curriculum change, teachers encounter a closely resembles Category I—‘‘Confirmed career option’’ —in [49].
diversity of challenges, including teachers’ limited technological The tech-savvy students in the second category are engaged with
skills and knowledge, technical problems, didactic differentiation, technical challenges that, for example, programming provides
a limited ability of students to understand the content, and lack (cf. [47], Category II: ‘‘Novel career option’’ in [49]). The well-
of students’ general willingness or ability to solve problems [16]. schooled category represents students who have no troubles in
 A. Tuhkala, M.-L. Wagner, O.S. Iversen et al. / International Journal of Child-Computer Interaction 20 (2019) 54–63 57

meeting the learning objectives, but who do not show interest 4. Methods
in technology-related topics (cf. Category III in [49]: ‘‘Stick to
other plan’’). The undecided students are not convinced by the An overview of the research process is shown in Table 1. After
relevance of technology or design, other than part of school work, the project was initiated, we started the research by sending
whereas the not (yet) motivated students feel that technology- an electronic survey to the participating schools. The survey
related activities have little sense for them (cf. Category IV in [49]: asked about the teacher’s professional background, anticipated
‘‘Confirmed not interested’’). We apply these archetypical cate- challenges regarding TC, and expectations of being part of the
gories to consider what kind of students TC elective courses can project. During winter 2017, we visited the schools and carried
reach, serving as a pre-step for investigating the impact of TC on out semi-structured interviews with 14 teachers [54]. The in-
terviews explored the teachers’ expectations (if not yet teached
students’ perceptions in the future.
TC) and experiences (if already teached TC) regarding TC. For
the teachers who had already teached TC, we handed the ‘‘five
3. Technology comprehension archetypical student categories’’ [6], described in Section 2.4. We
presented these categories to the teachers, discussed what the
We now draw the attention on the current initiative in Den- categories stand for, and asked them to assign their students
mark, where the Danish Ministry of Education defined TC as a within these categories. We also observed TC teaching activi-
new subject for the lower secondary education. The curriculum ties during the school visits, if it was possible. Based on these
was formulated by three experts in education. TC was first piloted preliminary investigations, we clarified the research questions as:
in 13 schools as an elective course during fall 2017. The teachers,
RQ1: How the teachers perceived their competence to teach TC?
who were assigned to teach TC, had not received supplementary
or in-service training to teach the subject. During 2018–2021, TC RQ2: What opportunities and challenges the teachers perceived
will be experimented in over 40 schools to investigate the three when introducing TC?
implementation options: An independent subject running from
first to ninth grade; an integrated subject to existing subjects; or RQ3: What form of student engagement the teachers recognised
combination of both, where TC is integrated into other subjects in TC classes?
from first to sixth grades and then thought as an independent
subject from third to ninth grades. Because one goal of the project was to support the teachers,
The Danish Ministry of Education [50] has defined that TC we decided to arrange a one day workshop. The workshop had
three-fold purpose: to provide support and training for the teach-
needs to consider: (1) The implications of technology and au-
ers; to involve the teachers to discuss possibilities and challenges
tomation on society, including an understanding of security,
of TC; and to gather data for this study. Before the workshop
ethics, and consequences of digital technologies; (2) Computing
started, we informed all teachers about our data collection pur-
as a knowledge area, including basic knowledge of networks,
poses and provided them with a self-assessment questionnaire.
algorithms, programming, logic and algorithmic thinking, ab- The workshop program consisted of familiarising with each oth-
straction and pattern recognition, as well as data modelling and ers, examining the TC learning goals, and practical hands-on
testing. (3) Iterative design process as an interaction between tasks with Micro:bits. After the workshop, the teachers answered
gaining an understanding of the world that is being designed to a feedback questionnaire (e.g. what did they learn in the
to and gaining an understanding of the digital technologies that workshop).
are being designed with. (4) Complex problem solving, where During the workshop, we also arranged a theme discussion
children create new digital solutions and, hence, learn to argue about TC. The topics of the theme discussion were: What is TC
for their relevance through an understanding of design processes. as an elective course for you, how do you incorporate TC in your
Consequently, TC includes [51] three major learning objec- current teaching, how do you perceive the competency goals,
tives: (1) Students learn how to produce and analyse digital and what should TC be in future? For the theme discussion, we
products. (2) Students learn to develop, modify, evaluate, and supplied the teachers with a handout of TC learning objectives.
refine digital products through work with remixing, refinement, The discussion was moderated by one of the researchers and
and production. (3) Students learn the possibilities and role of recorded with two video-cameras.
informatics as a catalyst for changes in the society, in order to The workshop was executed two times in different regions
strengthen the capabilities for acting in a meaningful way in a of Denmark, once in Aarhus and once in Copenhagen (see Ta-
democratic and digital society, including constructive and critical ble 2). In Aarhus, there were nine participants from seven schools
contribution in shaping the digital society. (eight teachers and one pedagogical consultant). In Copenhagen,
TC has some intersections with computational thinking and there was also nine participants (seven teachers and two school
principals).
especially with computational concepts, practices, and perspec-
For the RQ1, the self-assessment questionnaire consisted of
tives [as defined in 3,52,53]. However, TC differs significantly
four Likert scale question sets [55]. The first set was accustomed
from computational thinking in the following areas: Firstly, it
from the digital competence framework [41], which examines
treats computing and design as equal competency areas. Sec-
teachers’ general competences regarding digital tools: using dig-
ondly, these two areas are dependent on each other, in order
ital tools in spare time (e.g. online banking), using digital tools
to develop students’ capabilities to analyse, design, and develop in work (e.g. office tools and presentation), using digital tools in
digital products. And thirdly, it integrates the societal reflection, instruction (e.g. learning resources in web), guiding students to
meaning the critical reflection of the societal impact of tech- improve their learning strategies with digital tools (e.g. reading
nology, as a part of the learning objectives. In this sense, TC is screen-based text, note-making, mind-maps), and guiding stu-
related to [12]’s ‘‘Bildung’’, as a way of considering complex and dents in ethical matters related to digital tools (e.g. plagiarism,
sustainable learning. These three standpoints are all related to, social media). Because this question set does not consider pro-
but different from similar initiatives, such as CS4all in the United gramming, we added three questions about programming compe-
States, CoolThink in Hongkong, and Computing in the United tence: visual programming language (e.g. Scratch), programming
Kingdom. (e.g. Javascript), and debugging. The two other sets examined
58 A. Tuhkala, M.-L. Wagner, O.S. Iversen et al. / International Journal of Child-Computer Interaction 20 (2019) 54–63

Table 1
Technology comprehension research process.
Date Participants Activity Data
October 2017 Representatives from the Aarhus university Project initiation Official documents (e.g. learning objectives
and the Danish Ministry of Education and official documentation of TC)
October 2017 14 teachers from 13 schools Electronic survey for schools Survey answers
November 14 teachers from 13 schools and researchers School visits 14 recorded interviews, field notes, student
2017–January categories form
2018
19th February 8 teachers, 1 pedagogical consultant, and 3 Workshop in Aarhus (including theme Workshop recording, self-assessment
2018 researchers discussion) questionnaire, and feedback questionnaire
21 February 2018 7 teachers, 2 principals, 3 researchers Workshop in Copenhagen (including theme Same as in Aarhus
discussion)

Table 2 Table 3
Teachers’ teaching background (n = 18). Perceived competencies of the teachers (n = 15).

# Subjects Teaching 1 2 3 4 Mean SD
experience (years) Digital competence f 0 7 26 42 75 3.47 .48
Workshop in Aarhus 5 questions f /n (%) 0.0 9.3 34.7 56.0 100
1 IT pedagogy Over 10
Programming competence f 18 13 9 2 42 1.93 .64
2 Math, physics, chemistry, history 3–5
3 questions f /n (%) 40.0 28.9 20.0 4.4 93.3
3 History, societal, physics, chemistry, IT Over 10
4 Math, physics, chemistry, TC Over 10 Teaching design concepts f 3 11 27 1 42 2.62 .60
5 Languages, math, sports, household, 3–5 3 questions f /n (%) 6.7 24.4 60.0 2.2 93.3
nature and technology
6 Math, sports, IT/Fablab Over 10 Teaching computing concepts f 27 30 28 2 87 1.76 .68
7 Math, nature and technology, religion, Over 10 6 questions f /n (%) 30.0 33.3 31.1 2.2 96.6
crafts and design, TC Scale: 1 = not competent, 2 = little competent, 3 = competent, 4 = highly
8 Danish, music, fablab Over 10 competent.
9 Math, nature and technology, science Over 10
Workshop in Copenhagen
10 English, history, crafts and design 5–10
11 Nature and technology, TC 0–2 that determining the category that represented each student was
12 Music, English, TC 0–2
challenging and should be considered as only a rough estimate.
13 Danish, religion, sports, music, TC 0–2
14 History, religion, nature and technology, 0–2
biology
5. Results
15 Math, physics, chemistry, TC 5–10
16–18 Unknown Unknown Here we present the findings to the three research questions.

5.1. RQ1: Teachers’ perceived competency

the teachers’ perceived capability to teach design and comput- Fifteen teachers answered to the self-assessment question-
ing related concepts. The design concepts were idea generation, naire. As can be seen in Table 2, seven teachers had more than
fabrication, and societal significance and the computing concepts ten years, four teachers had three to ten years, and four teachers
were patterns, algorithms, data structures, coding, programming had less than two years of teaching experience. The teachers
languages, and testing. had taught the following subjects: TC, math, physics, chemistry,
Answers to the Likert scale questions were analysed with IBM history, crafts and design, social studies, IT, Danish, sports, Fablab,
SPSS Statistics. All ‘‘I don’t know’’ answers were treated as missing nature and technology, religion, German, music, biology, and food
answers, and excluded from the analysis. The first analysis in- Literacy.
volved the calculation of frequencies, frequency distributions, and The teachers perceived their competence to use digital tools
portions. The four question sets were transformed into four Likert in education high (Table 3). Altogether 90.7% of the answers to
scale constructs to calculate the means and standard deviations. the five questions were either competent or highly competent. In
However, the internal consistency of the constructs could not the programming competence questions, almost all (14) teachers
be verified due to the small sample size. We also compared answered that they had at least some competence with visual
the two workshops using the Mann–Whitney test and found no programming languages, such as Scratch. On the other hand,
most of the teachers (10) had no competence in programming
statistically significant differences between the workshops.
with a text-based language. This reveals that while the teachers
For the RQ2, we first translated and transcribed the theme
considered themselves as digitally competent, most had only
discussion in English, because all authors are not fluent in Dan-
expertise in using visual programming languages. We also asked
ish. Then, we carried out a collaborative content analysis of the
how the teachers perceive their competence to teach TC concepts.
theme discussion [55]: we negotiated the high-level objective of Over 60% of the answers to teaching the design concepts were
the analysis, watched the discussion recording, and made notes competent or highly competent. In contrast, only 31.1% were
individually. Then we discussed different interpretations and con- competent or highly competent regarding computing concepts.
structed themes based on the research question. We triangulated Besides the presented competencies, it is worth noting that
the findings by analysing the answers to the electronic survey, two of the teachers had been part of the expert group in the
self-assessment questionnaire, and feedback questionnaire. Danish Ministry of Education. The group had formulated the
For the RQ3, we analysed the student category forms, which exact competency areas, competency goals, proficiency goals,
were filled during the school visits and, in some cases, provided and knowledge goals for TC. As a conclusion, the teachers per-
to us after the workshop. It needs to be noted that the student ceived high digital competence and most of the teachers had a
frequencies in these categories do not represent the students as lot of teaching experience and from a broad range of subjects.
such, but rather the teachers perceptions how they would define The teachers considered themselves more competent in teaching
the students who took part in TC. Moreover, the teachers stated design concepts than computing concepts.
 A. Tuhkala, M.-L. Wagner, O.S. Iversen et al. / International Journal of Child-Computer Interaction 20 (2019) 54–63 59

5.2. RQ2: Opportunities and challenges of TC as a subject had a sense of ownership to this assignment. The 9th graders were
supposed to make a math game for 1st graders. The day before, 9th
Two of the Danish Government’s implementation options po- graders used their lunch break to go to 1st graders and check if
sition TC as an individual subject. Despite this, the teachers ad- the level was too hard. Then 9th graders went back and adjusted
dressed TC as a component of some other subjects. For example, the games. That’s very uncommon to 9th graders to do something
a teacher considered TC as mere programming skills: ‘‘We created like that in own time’’. This was also exemplified by another
programming and maths course that starts in the first grade and teacher, highlighting how TC integrates topics that are relevant to
runs through all grades. Programming is okay, but should not be a students: ‘‘My focus is to have a starting point that the students can
standalone subject, it should be part of the other subjects. A tool’’. relate to, for example, in the Odense municipality we are establishing
Likewise, the teachers considered TC as a tool for learning other the new light-rail. The students were concerned about what if a blind
subjects: ‘‘I think a lot about how it can be part of natural science person should cross the light-rail, can we be sure that the train will
subjects. Currently, I am also teaching crafts, where I think that it stop. So they tried to build some censors with Micro:bits. This was the
could fit in. But, as I said, I also think that having it as a part of classical problem-solving setting that the students could relate to’’.
natural science would be very exciting for me’’. This also became ap- These examples demonstrate the opportunity to actualise TC as a
parent when the teachers talked about the tasks that they involve design process that integrates computing, the societal reflection,
the students in, as noted by a teacher: ‘‘they [students] created and problem-solving.
math games’’. Another teacher had integrated other subjects, such As illustrated by the previous quotes, the teachers appreciated
as entrepreneurship, into TC: ‘‘I tend to focus on the Design part of the fact that TC combines computing, design, and societal reflec-
the subject because that is what I find awesome, this entrepreneur- tion together. They pointed out that, normally, computing-related
ship and I try to keep asking the students questions if they claim curricula are designed by people with the computing background.
that they are done ‘Design a Logo’, ‘Find a company name’, ‘Create a Hence, the learning objectives tend to address mere computing
business model’’’. As a conclusion, these perceptions indicate that goals. TC opens up for holistic subject goals when it is designed
the teachers lacked formalised ways of addressing TC as a distinct by stakeholders from the various subjects.
subject, as explicitly coined by one of the teachers: ‘‘This new thing Several challenges emerged from the data. The teachers’ con-
that is starting, I think about it as part of the existing subjects’’. ceptions indicated uncertainty about the meaning of the societal
As indicated by the previous examples, when considering TC, reflection in TC. The teachers discussed the societal reflection
the teachers displayed mainly episodic knowledge: the teachers’ primarily as a means to contextualise the subject with real-
arguments derived from their own, or others, practices of using world problems. They referred to topics that were familiar from
technology in education. Despite the fact that the learning goals previous teaching experiences in other subjects, such as ‘‘fake
of TC were provided to the teachers, the arguments for the subject news’’ in social media, election meddling, and the earlier example
reflected their personal beliefs, experiences, and interests. For ex- about the light-rail track in Odense. Thus, the teachers considered
ample, the teachers with design background emphasised design the societal reflection of TC as a means to contextualise classroom
goals, teachers with computing background computing goals, and activities, instead of a learning objective as such.
teachers with humanistic background societal goals. Thus, the The teachers described the varying digital skills of students as
teachers did not have a mutual understanding of what TC was, a major challenge to teaching. A teacher told that: ‘‘I have some
or what it should be in the future, but instead relied on personal boys in my elective course and even before I started the teaching they
preferences and episodic knowledge. had downloaded the files we should use. At the same time, I had a
Most of the discussed opportunities were confused with girl who did not know what a file is. The students had very different
technology-supported education. The teachers referred to exam- skills for participating in this field’’. Another teacher noted that if
ples how technology can support students: ‘‘I can have students TC is first introduced in the seventh grade, the prerequisites of
that are creating a paper booklet, and right next to them another the subject are necessarily low. Otherwise, lack of basic skills,
group that works with creating a blog. Both make equal sense. such as basic computer use, will prevent those students to pursue
Then you do have students that are able to concentrate for more the actual learning goals: ‘‘I would like to be better at presenting
than 25 min because you have access to different technologies’’. the students with a problem as a starting point, where they can
Another example was using technology to engage students with analyse, design and develop. Currently, they have mostly worked
special needs: ‘‘I had an experience, where mother of a dyslexic child with learning the different technologies’’.. One teacher concluded
contacted me. Usually, when the girl had to make presentations, that if the basic computing skills are to be taught in TC, it leaves
she was embarrassed by doing it. She used the computer to make less space and time for other learning objectives.
the presentation and felt more capable of presenting due to the Another challenge was that students have different needs
auto-correction tools’’. In addition, the teachers brought up the regarding the structure and guidance of TC learning activities.
opportunity to motivate students, who use digital tools in spare Some students want to be challenged and to be provided with
time or to concretise abstract topics, such as perceiving the less guidance, while others are incapable of acting without clear
coordinate system using Scratch. structure and instructions: ‘‘Some of them expect to be challenged,
The teachers presented several narratives how TC engages some of them expect to get everything served on a silver plate. That
different students. For example, a teacher said: ‘‘We have had is one of the biggest challenges I have to get them to be better’’.
some boys that were very hard to engage in other subjects, that This indicates that TC, as a new subject, calls for a high level of
have been very engaged and therefore also very cooperative on this individual differentiation of the learning activities.
matter’’. This was followed up by an example in special education Finally, we identified further challenges related to the gender
context: ‘‘I have never gotten them to focus for more than 25 min, disparities and teachers’ need for time and peer support. As
now they have been working for 90 min’’. In general, the teachers demonstrated by the earlier quotes, there is a gender disparity
agreed on the fact that TC is an engaging subject, as concluded by around students skills and interest in TC. For example, a teacher
a teacher: ‘‘[TC] is fun, and a lot of students get engaged by it’’. stated: ‘‘A lot of students want to participate in 4–6th grade, in 7–
A recurring theme was that TC opens up for more student- 9th grade, it is primarily boys’’. The teachers’ conceptions distinct
centred learning. A teacher elaborated on the 9th graders’ sense of between boys, as being interested and knowledgeable, and girls
ownership towards making math games to 1st graders: ‘‘We were as not necessarily interested, or engaged, in TC. The teachers
making math games with Scratch, it was obvious that older students pointed out that teachers need more time, peer support, and
60 A. Tuhkala, M.-L. Wagner, O.S. Iversen et al. / International Journal of Child-Computer Interaction 20 (2019) 54–63

scaffolded teaching instructions to be able to implement TC as a 6. Discussion

new subject. As concluded by one teacher: ‘‘[TC] is a new subject
and a new way of thinking in primary school. It requires more Our findings derive from the first year of implementing TC
preparation time than the ’normal subjects’, where you can adopt as a national subject. Despite the fact that we are early in the
a lot of existing teaching material from various learning portals into process, the teachers provided us with important practice-based
your own work’’. knowledge about TC and its further development. To discuss our
findings, we have developed four themes: (1) teachers did not
perceive TC as a distinct subject; (2) TC opened up for inter-
5.3. RQ3: Student engagement in TC disciplinary and engaging learning activities; (3) TC challenged
teachers’ existing competencies; and (4) TC appealed to various
kinds of students.
Teachers from 11 out of 13 participating schools reported Teachers did not perceive TC as a distinct subject. Our re-
back on their perception of students’ characteristics based on search is associated with current efforts of incorporating com-
five archetypical student categories [8]. Two schools chose not to puting and design in formal education [7,27,31,42,53]. Our main
participate in this part of the interview or were unable to account finding was that teachers did not see TC as a distinct subject,
for the student profiles. The number of students from each of the but rather as a set of skills that can be integrated into other
five archetypical categories is accounted in Table 4 in relation to subjects. This may be due to the fact that TC is a subject coined
the total number of students on each school. by the Danish Ministry of Education and, thus, not well-known
Generally, each course had a very limited number (1–3 stu- among teachers or in research. This relates to what Smith and
dents pr. class) of design competent students. This is consistent Iversen [8] considered as the impediments of digital fabrication
with our hypothesis as technology comprehension is a new sub- in education. Whereas they pointed out that teachers generally
ject matter. Hence, students do not have any prior formal training lack a sufficient understanding of design processes and complex
in technology comprehension. However, there are three schools problem solving, our study showed that teachers did not perceive
computing, design, and societal reflection together as a distinct
with significantly more students in this category; 6 design com-
subject. Eriksson et al. [31] points to the need for informing
petent students at schools 7 and 9, which could relate to the
national policy-making to support general management of joint
teacher’s lack of knowledge of the students (one teacher had
digital fabrication teaching materials and curriculum develop-
only known the students for a few weeks at the time of the ment. Similarly, developing TC as a subject should not be left only
interviews) and 15 design competent students at school 10 re- to the teachers’ responsibility, but rather, to the responsibility of
lating to the teachers own interpretation of TC as a subject (the the entire TC initiative and ultimately to research.
teacher stressed that entrepreneurship and innovation was a part TC opened up for interdisciplinary and engaging learning
of the curriculum which it is not formally. The number of tech- activities. We found that teachers identified several opportu-
savvy students was also very limited in the TC courses. Only 2–3 nities in TC: it encourages children to be creative with digital
students per class were characterised by these competencies. The technology, to work with authentic and complex problems and
relatively low number of tech-savvy students came as surprise to take responsibility for their learning process [cf. 24]. More-
to the researchers. We expected to find a large number of tech- over, the teachers thought that students perceive TC learning
savvy students, who would sign up for the elective course, as activities as engaging, inspiring, and fun [cf. 27]. TC shares [12]’s
this would allow them to work with digital technology inside the reasons for introducing digital fabrication in curriculum-based
formal education). education: developing computing skills together with cultivating
a digital citizenship. Hence, TC promotes digital competencies as
The number of well-schooled students diverges significantly
critical and reflective personal skills that relate to Bildung [12,
from school to school. Schools 3 and 6 reported that none of their
40], computational empowerment [22], democracy and empow-
students in the TC course can be considered as well-schooled,
erment [13], and also to the computational perspectives in [52].
whereas school 4 categorised 10 (out of 31) and school 10 cat-
While the teachers discussed how addressing digital technology
egorised 10 (out of 25) as well-schooled students. The undecided from a critical and societal point of view is an opportunity, they
students were generally well-presented in the TC course. In some did not feel capable of bridging between hands-on activities and
schools (1,4,5,11), teachers had done extra effort to recruit stu- more abstract discussion of computational perspectives. To fulfil
dents from this category to the TC course. The teacher from the opportunities of TC, there is a need for scaffolding activities,
school 5 emphasised that the reason for the large proportion of such as in-service training, development of learning materials,
undecided students in the class (30 students) is directly linked to and online resources [31,43,56].
his own inability to explain to the students why TC is important, TC challenged teachers’ existing competencies in relation to
which inevitably makes the overall purpose with the subject the subject and students’ prerequisites and needs. We identi-
unclear and difficult to engage in. In some of the schools, almost fied the following challenges: the lack of shared understanding
half of the course participants can be categorised as undecided. about the meaning of the societal reflection in TC, students’ vary-
Finally, Table 3 accounts for the number of not-yet-motivated. ing skills and needs, and the paradox between instructional struc-
ture and freedom, and the lack of girls’ involvement. Some of the
Aside from schools 5 and 6, all TC courses engaged a number of
challenges are already known in research, for example, balancing
not-yet-motivated students. In the school 4, 30 out of 40 students
between creative digital fabrication activities and formal educa-
represented this category. The teacher from that school reported
tional structure [8,31,42,56] and the gender imbalance [31]. We
that the course description was designed to accommodate the
argue that a crucial challenge is the lack of teachers’ competency
interest of this particular category of students, which explains framework that considers computing, design, and societal reflec-
the high number of students from this category. Exactly the same tion as a whole. Current national and international frameworks
phenomenon has been visible in the student outreach — related understand digital competence as a capacity to use technology
profiles (see [49] and Section 2.4). Overall, it must be considered for other learning purposes. For example, Norwegian framework
that TC is an elective subject, and some students did not pick it includes mere digital skills, consisting of searching and processing
as their first priority, but as a second or even third priority. information from digital resources [57] and European Framework
 A. Tuhkala, M.-L. Wagner, O.S. Iversen et al. / International Journal of Child-Computer Interaction 20 (2019) 54–63 61

Table 4
Teachers’ perceptions of the student engagement categories in TC course.
School Design competent Tech-savvy Well-schooled Undecided Not (yet) motivated Total
School 1 1 3 3 12 6 25
School 2 3 2 4 4 1 14
School 3 2 3 0 5 5 15
School 4 0 1 10 10 10 31
School 5 3 1 6 30 0 40
School 6 3 3 0 4 0 10
School 7 6 2 3 5 6 22
School 8 1 2 4 6 5 18
School 9 6 2 7 2 3 20
School 10 15 3 5 5 2 30
School 11 1 2 10 10 2 25
Total 41 24 52 93 40 250
Total % 16.4 9.6 20.8 37.2 16.0 100.0

for the Digital Competence of Educators consider teachers’ use focused on literature in making and digital fabrication, and future
of digital tools only for communication, collaboration, and pro- research should take into account other fields as well, such as
fessional development [58]. Hence, we call for defining subject Computing Education Research and Learning Sciences.
knowledge for teachers to educate digital technologies through The challenges of implementing the political agenda to offer
learning activities that utilise design process, entail computing TC for all students in Denmark should be addressed merely as a
skills, and aim for personal empowerment. general lack of research about TC. Thus, our study raises several
TC appealed to various types of students. With reference to questions: How do we draw from previous research on com-
the archetypical categories [6], we found that TC appealed to var- puting and design education to consider computing, design, and
ious types of students in Denmark. This was due to two factors: societal reflection as a combined subject? How do we develop
First, many schools in the Danish project made a deliberate choice supplementary training for pre-service and in-service teachers
to target undecided or not-yet motivated students, not only the to support their TC teaching practices? How do we develop as-
tech-savvy, in their TC elective course. The teachers accounted for sessment strategies that take into account the development of
this choice by arguing that digital technology is first and foremost computing and design skills with the capability of critically reflect
a democratic matter. Accordingly, schools must provide every technology as a whole?
student with an opportunity to prosper and actively engage in
a digitalised society. Second, the schools adjusted content from Acknowledgements
computer science, digital fabrication, digital design, and human-
ities to customise a curriculum that would attract students with We want to thank the Danish Ministry of Education and,
less prior knowledge or interest in programming and algorithms. especially, the teachers who participated in the study. We also
As such, this implementation strategy of TC resonates well with thank Bronwyn Cumbo for proof-reading assistance.
the general Nordic approach, which has a strong emphasis on
Conflict of interest
understanding the opportunities and risks of digitalisation from
the societal reflection [34–37].
The authors declare that there is no conflict of interest in this
paper.
7. Conclusion
References
Educational researchers have warned that implementing ed-
ucational reforms with purely top-down approach can lead to [1] A. Tuhkala, M.-L. Wagner, N. Nielsen, O.S. Iversen, T. Kärkkäinen, Tech-
failure [59]. This study contributes to the previous research on nology comprehension - scaling making into a national discipline, in:
making and digital fabrication by exploring how these endeav- Proceedings of the Conference on Creativity and Making in Education -
ours can be implemented on a national level by engaging with FabLearn Europe’18, no. June, ACM Press, New York, New York, USA, 2018,
pp. 72–80, http://dx.doi.org/10.1145/3213818.3213828.
teachers on a local level. Based on interviews, surveys, ques- [2] European Commission, Communication on the Digital Education Action
tionnaires, and the theme discussion with highly experienced Plan, Tech. Rep., European Comission, Brussels, 2018, p. 11.
teachers, we found both possibilities and challenges in imple- [3] J.M. Wing, Computational thinking, Commun. ACM 49 (3) (2006) 33, http:
menting the new subject: Teachers did not yet perceive TC as //dx.doi.org/10.1145/1118178.1118215.
[4] A.V. Aho, Ubiquity symposium: computation and computational think-
a distinct and legitimate subject, TC opened up for interdisci- ing, Ubiquity 2011 (January) (2011) 1, http://dx.doi.org/10.1145/1922681.
plinary and engaging learning activities, TC challenged teachers’ 1922682.
existing competencies in relation to the subject and students’ [5] The Royal Society, Shut Down or Restart? The Way Forward for Computing
prerequisites and needs, and TC appealed to various types of in UK Schools, Tech. Rep., 2012, pp. 1–122.
[6] R.C. Smith, O.S. Iversen, Participatory design for sustainable social change,
students. Taking these findings into account helps the Danish
Des. Stud. (2018) 1–28, http://dx.doi.org/10.1016/j.destud.2018.05.005.
stakeholders to better understand the impacts of educational [7] R.C. Smith, O.S. Iversen, M. Hjorth, Design thinking for digital fabrication
reforms from teachers’ perspectives. Hopefully, these findings can in education, Int. J. Child-Comput. Interact. 5 (2015) 20–28, http://dx.doi.
be also applied in other contexts, by informing what kind of issues org/10.1016/j.ijcci.2015.10.002.
[8] R.C. Smith, O.S. Iversen, R. Veerasawmy, Impediments to digital fabrication
may emerge in this kind of initiatives.
in education: A study of teachers’ role in digital fabrication, Int. J. Digit. Lit.
We identify the following shortcomings in our study: Our Digit. Competence 7 (1) (2016) 33–49, http://dx.doi.org/10.4018/IJDLDC.
findings derive solely from collaboration with teachers, which 2016010103.
leaves out crucial stakeholders, such as policy-makers, parents, [9] M. Hansbøl, S. Ejsing-Duun, Hovedrapport: Coding class - documentation
and students. We have not taken into consideration that many og evaluering, in: Coding Class Report, Aalborg University, 2017.
[10] R.T. Nørgård, R.B. Paaskesen, Open-ended education: how open-endedness
teachers will ultimately teach TC without prior experiences or might foster and promote technological imagination, enterprising and
any compulsory education in TC, which may exacerbate the need participation in education, Conjunctions : Transdiscipl. J. Cult. Particip. 3
to better support teachers. Finally, as already stated, our study (1) (2016) 1–25.
62 A. Tuhkala, M.-L. Wagner, O.S. Iversen et al. / International Journal of Child-Computer Interaction 20 (2019) 54–63

[11] D. Ball, C. Tofel-Grehl, K.A. Searle, Sustaining making in the era of [31] E. Eriksson, C. Heath, P. Ljungstrand, P. Parnes, Makerspace in school—
accountability: Stem integration using e-textiles materials in a high school considerations from a large-scale national testbed, Int. J. Child-Comput.
physics class, in: Proceedings of the 7th Annual Conference on Creativity Interact. (2017) 1–7, http://dx.doi.org/10.1016/j.ijcci.2017.10.001.
and Fabrication in Education, in: FabLearn’17, ACM, New York, NY, USA, [32] D.A. Fields, Y.B. Kafai, T. Nakajima, J. Goode, Teaching practices for making
2017, http://dx.doi.org/10.1145/3141798.3141801, 2:1–2:7. e-textiles in high school computing classrooms, in: Proceedings of the
[12] H. Schelhowe, Digital realities, physical action and deep learning-fablabs as 7th Annual Conference on Creativity and Fabrication in Education, in:
educational environments, in: FabLabs: Of Machines, Makers and Inventors, FabLearn’17, ACM, New York, NY, USA, 2017, http://dx.doi.org/10.1145/
2013, pp. 93–103. 3141798.3141804, 5:1–5:8.
[13] P. Blikstein, Digital fabrication and ‘making’ in education: The democra- [33] E. Tan, A. Calabrese Barton, M. Shin, C. Turner, Probing participatory part-
tization of invention, in: J. Walter-Herrmann, C. Buching (Eds.), FabLabs: nerships: Equitably-consequential making by, for and with marginalized
Of Machines, Makers and Inventors, Bielefield: Transcript Publishers, 2013, youth, in: Proceedings of the 6th Annual Conference on Creativity and
pp. 1–21, http://dx.doi.org/10.1080/10749039.2014.939762.
Fabrication in Education, in: FabLearn’16, ACM, New York, NY, USA, 2016,
[14] P. Blikstein, D. Krannich, The makers’ movement and fablabs in education,
pp. 1–8, http://dx.doi.org/10.1145/3003397.3003398.
in: Proceedings of the 12th International Conference on Interaction Design
[34] Swedish National Agency for Education, Curriculum for The Compulsory
and Children - IDC’13, no. April, ACM Press, New York, New York, USA,
School, Preschool Class and School-Age Educare - Revised 2018, 2018, p.
2013, p. 613, http://dx.doi.org/10.1145/2485760.2485884.
303.
[15] D. Passey, Computer science (CS) in the compulsory education curriculum:
[35] Norwegian Board of Education, Core Curriculum for Primary, Secondary,
implications for future research, Educ. Inf. Technol. 22 (2) (2017) 421–443,
http://dx.doi.org/10.1007/s10639-016-9475-z. and Adult Education in Norway, 1997, p. 40.
[16] S. Sentance, A. Csizmadia, Computing in the curriculum: Challenges and [36] Finnish National Board of Education, National Core Curriculum of Basic
strategies from a teacher’s perspective, Educ. Inf. Technol. 22 (2) (2017) Education 2014, 2014, p. 473.
469–495, http://dx.doi.org/10.1007/s10639-016-9482-0. [37] O. Berge, Rethinking digital literacy in nordic school curricula, Nordic
[17] E. Soloway, M. Guzdial, K.E. Hay, Learner-centered design: the challenge J. Digit. Lit. 12 (01–02) (2017) 5–7, http://dx.doi.org/10.18261/issn.
for HCI in the 21st century, Interactions 1 (2) (1994) 36–48, http://dx.doi. 1891-943x-2017-01-02-01.
org/10.1145/174809.174813. [38] P. Blikstein, S.L. Martinez, H.A. Pang, Meaningful Making: Projects and In-
[18] M. Scaife, Y. Rogers, F. Aldrich, M. Davies, Designing for or designing with? spirations for Fab Labs and Makerspaces, Constructing Modern Knowledge
informant design for interactive learning environments, in: Proceedings of Press, Torrance, CA USA, 2016.
the ACM SIGCHI Conference on Human Factors in Computing Systems, in: [39] L. Ilomäki, S. Paavola, M. Lakkala, A. Kantosalo, Digital competence –
CHI’97, ACM, New York, NY, USA, 1997, pp. 343–350, http://dx.doi.org/10. an emergent boundary concept for policy and educational research,
1145/258549.258789. Educ. Inf. Technol. 21 (3) (2016) 655–679, http://dx.doi.org/10.1007/
[19] J.A. Rode, M. Stringer, E.F. Toye, A.R. Simpson, A.F. Blackwell, Curriculum- s10639-014-9346-4.
focused design, in: Proceedings of the 2003 Conference on Interaction [40] F.M. Røkenes, R.J. Krumsvik, Development of student teachers’ digital
Design and Children, in: IDC’03, ACM, New York, NY, USA, 2003, pp. competence in teacher education - A literature review, Nordic J. Digit. Lit.
119–126, http://dx.doi.org/10.1145/953536.953553. (04) (2014) 250–280.
[20] N. Iivari, M. Kinnula, Inclusive or inflexible: A critical analysis of the school [41] F.M. Røkenes, R.J. Krumsvik, Prepared to teach ESL with ICT? A study
context in supporting children’s genuine participation, in: Proceedings of of digital competence in norwegian teacher education, Comput. Educ. 97
the 9th Nordic Conference on Human-Computer Interaction, in: NordiCHI (2016) 1–20, http://dx.doi.org/10.1016/j.compedu.2016.02.014.
’16, ACM, New York, NY, USA, 2016, http://dx.doi.org/10.1145/2971485.
[42] T. Bekker, S. Bakker, I. Douma, J. van der Poel, K. Scheltenaar, Teaching
2971531, 63:1–63:10.
children digital literacy through design-based learning with digital toolkits
[21] O.S. Iversen, R.C. Smith, C. Dindler, Child as protagonist: Expanding the role
in schools, Int. J. Child-Comput. Interact. 5 (2015) 29–38, http://dx.doi.org/
of children in participatory design, in: Proceedings of the 2017 Conference
10.1016/j.ijcci.2015.12.001.
on Interaction Design and Children - IDC ’17, 2017, pp. 27–37, http://dx.
[43] T.A. Dousay, An evolving makerspace for teacher education, Int. J. Des.
doi.org/10.1145/3078072.3079725.
Learn. 8 (1) (2017) http://dx.doi.org/10.14434/ijdl.v8i1.22672.
[22] O.S. Iversen, R.C. Smith, C. Dindler, From computational thinking to
computational empowerment, in: Proceedings of the 15th Participatory [44] J.A. Fredricks, P.C. Blumenfeld, A.H. Paris, School engagement: Potential of
Design Conference on Full Papers - PDC’18, ACM Press, New York, New the concept, state of the evidence, Rev. Educ. Res. 74 (1) (2004) 59–109.
York, USA, 2018, pp. 1–11, http://dx.doi.org/10.1145/3210586.3210592. [45] T. Virtanen, Student Engagement in Finnish Lower Secondary School (Ph.D.
[23] N. Iivari, M. Kinnula, Empowering children through design and making – thesis), Jyväskylä studies in education, psychology and social research 562,
towards protagonist role adoption, in: PDC’18: Proceedings of the 15th 2016.
Participatory Design Conference - Volume 1, Vol. 1, 2018, http://dx.doi. [46] J.D. Finn, K.S. Zimmer, Student engagement: What is it? Why does it
org/10.1145/3210586.3210600. matter?, in: Handbook of research on student engagement, Springer, 2012,
[24] E.-S. Katterfeldt, N. Dittert, H. Schelhowe, Designing digital fabrication pp. 97–131.
learning environments for bildung: Implications from ten years of physical [47] A.-j. Lakanen, V. Isomöttönen, V. Lappalainen, Five years of game program-
computing workshops, Int. J. Child-Comput. Interact. 5 (2015) 3–10, http: ming outreach, in: Proceedings of the 45th ACM Technical Symposium on
//dx.doi.org/10.1016/j.ijcci.2015.08.001. Computer Science Education - SIGCSE’14, ACM Press, New York, New York,
[25] S. Papavlasopoulou, M.N. Giannakos, L. Jaccheri, Empirical studies on the USA, 2014, pp. 647–652, http://dx.doi.org/10.1145/2538862.2538914.
maker movement, a promising approach to learning: A literature review, [48] S. Hidi, K.A. Renninger, The four-phase model of interest development,
Entertain. Comput. 18 (2017) 57–78, http://dx.doi.org/10.1016/j.entcom. Educ. Psychol. 41 (2) (2006) 111–127.
2016.09.002. [49] A.-J. Lakanen, T. Kärkkäinen, Identifying pathways to computer science:
[26] L. Martin, The promise of the maker movement for education, J. Pre-Coll. Long-term impact of short-term game programming outreach intervention,
Eng. Educ. Res. (J-PEER) 5 (1) (2015) http://dx.doi.org/10.7771/2157-9288. ACM Trans. Comput. Educ. (2018) http://dx.doi.org/10.1145/3283070, (in
1099. press).
[27] S.L. Chu, F. Quek, E. Deuermeyer, R. Martin, From classroom-making to [50] Danish Ministry of Education, Kommissorium for Den Rådgivende Ekspert-
functional-making: A study in the development of making literacy, in: skrivegruppe for Forsøgsprogram for Styrkelse Af Teknologiforståelse I
Proceedings of the 7th Annual Conference on Creativity and Fabrication
Folkeskolens Obligatoriske Undervisning, 2018.
in Education, in: FabLearn’17, ACM, New York, NY, USA, 2017, http://dx.
[51] Danish Ministry of Education, Teknologiforståelse valgfag – Fælles mæl og
doi.org/10.1145/3141798.3141802, 3:1–3:8.
læseplan, 2018.
[28] S. Nemorin, The frustrations of digital fabrication: an auto/ethnographic
[52] K. Brennan, M. Resnick, New frameworks for studying and assessing the
exploration of ‘3D making’ in school, Int. J. Technol. Des. Educ. 27 (4)
development of computational thinking, in: Annual American Educational
(2017) 517–535, http://dx.doi.org/10.1007/s10798-016-9366-z.
Research Association meeting, Vancouver, BC, Canada, 2012, pp. 1–25,
[29] A. Berman, B. Garcia, B. Nam, S. Chu, F. Quek, Toward a making community
of practice: The social aspects of elementary classroom-based making, in: doi:10.1.1.296.6602.
Proceedings of the 6th Annual Conference on Creativity and Fabrication [53] Y.B. Kafai, E. Lee, K. Searle, D. Fields, E. Kaplan, D. Lui, A crafts-oriented ap-
in Education, in: FabLearn’16, ACM, New York, NY, USA, 2016, pp. 9–16, proach to computing in high school: Introducing computational concepts,
http://dx.doi.org/10.1145/3003397.3003399. practices, and perspectives with electronic textiles, Trans. Comput. Educ.
[30] E. Tan, A.C. Barton, Designing for rightful presence in stem-rich mak- 14 (1) (2014) http://dx.doi.org/10.1145/2576874, 1:1–1:20.
ing: Community ethnography as pedagogy, in: Proceedings of the 7th [54] S.B. Merriam, Qualitative Research - A Guide to Design and Implementa-
Annual Conference on Creativity and Fabrication in Education, in: Fa- tion, third ed., Jossey-Bass, San Francisco, 2009, p. 320.
bLearn’17, ACM, New York, NY, USA, 2017, http://dx.doi.org/10.1145/ [55] L. Cohen, L. Manion, K. Morrison, Research Methods in Education,
3141798.3141807, 8:1–8:8. Routledge, 2013.
 A. Tuhkala, M.-L. Wagner, O.S. Iversen et al. / International Journal of Child-Computer Interaction 20 (2019) 54–63 63

[56] M. Hjorth, R.C. Smith, D. Loi, O.S. Iversen, K.S. Christensen, Educating [58] C. Redecker, Y. Punie, European Framework for the Digital Competence of
the reflective educator: Design processes and digital fabrication for the Educators: DigCompEdu, Tech. Rep., Publications Office of the European
classroom, in: Proceedings of the 6th Annual Conference on Creativity and Unioon, Luxemburg, 2017, pp. 1–2, http://dx.doi.org/10.2760/159770.
Fabrication in Education, in: FabLearn’16, ACM, New York, NY, USA, 2016, [59] J. Pietarinen, K. Pyhältö, T. Soini, Large-scale curriculum reform in Finland –
pp. 26–33, http://dx.doi.org/10.1145/3003397.3003401. exploring the interrelation between implementation strategy, the function
[57] M. Kelentric, K. Helland, A.-T. Arstorp, Professional Digital Competence of the reform, and curriculum coherence, Curric. J. 28 (1) (2017) 22–40,
Framework for Teachers, Norwegian Centre for ICT in Education, 2017. http://dx.doi.org/10.1080/09585176.2016.1179205.

View publication stats