Creative Practice as an Evaluation Method: A Case Study

with a Movement-based Musical Instrument

Doga Cavdir
Digital Design
IT University of Copenhagen
Copenhagen, Denmark
[email protected]

ABSTRACT approach within these user studies [9, 18]. Bossen et al.
emphasize the importance of studying creative practice and
The creative process with technology requires experimenta- constructing creative artifacts [5]. He states that regardless
tion, exploring affordances and limitations, and evaluation of participant background, a performative task supports de-
of one’s process of different learning stages. Movement- veloping personalized movement and sound vocabulary and
based digital musical instruments (MDMIs) offer many op- organizing expressive ideas. This approach can highlight
portunities to study performers’ creative processes since the aesthetic experience of DMIs, supporting a longitudinal
performers can artistically explore both the familiar and performance practice or sustained use [41].
unfamiliar interactions with the instruments. In this re- In this paper, we study evaluating digital musical instru-
search, we integrate the creative process as a performance- ments beyond their technical qualities by integrating cre-
based, qualitative evaluation method into studying perform- ative practice into user evaluation. To encourage creative
ers’ interactions. While these processes are often non-linear practice both in music and movement expressivity, we ask
and iterative, we observe how creativity, through sonic and participants to develop a compositional practice in these
movement interaction, impacts participants’ learning pro- two domains using Bodyharp. Bodyharp is a movement-
cesses. We study these processes with participants from based musical instrument (MDMI) that integrates perform-
music and/or dance dance backgrounds and report on their ers’ bodies directly into the physical interface, extending the
experiences. performers’ bodies beyond the instrument [11, 10]. This
dual interaction allows us to explore creative practices in
music and dance through their shared and distinct qualities.
Author Keywords Our study asked participants to develop creative artifacts
as research outcomes, encouraging the participants to focus
on developing their creative practices. We collected partici-
creative practice, aesthetics-based evaluation, movement- pants’ experiences through semi-structured interviews with
based musical instruments, embodied interaction twenty artists whose backgrounds are in music, movement,
or both.
Our contribution is threefold. Through systematically
CCS Concepts integrating creative practice into user evaluation, we bet-
ter understand the barriers and supporting factors in de-
•Human-centered computing → User studies; •Applied com- veloping a potentially ongoing instrumental practice with
puting → Performing arts; Sound and music computing; movement-based musical instruments. Second, we offer an
evaluation approach highlighting the performer’s bodily, felt
experiences through closely linked music and movement ex-
1. INTRODUCTION pressions and we incorporate embodied, soma-based eval-
Digital musical instruments (DMIs) extend the creative space uation methods [27]. Third, we provide criteria to aes-
of musical performance where the interface is more com- thetically and more holistically evaluate new musical in-
monly evaluated based on technical specifications and de- struments that would support their sustained use beyond
sign considerations [48, 34] or based on performance from user studies.
the audience’s perspectives [39]. O’Modrain emphasizes
the different perspectives (audience’s, performer’s, and de-
signer’s) involved in evaluating these interactions. However, 2. RELATED WORK
the evaluation methods and performance practices remain
separate or the studies that focus on “performability” re- 2.1 Movement-based Interaction
port from a subjective and retrospective account of a single
performer. Similarly, the creative practice with digital in- Movement-based interaction introduces an approach that
terfaces is much less frequently studied as an evaluation focuses more on the interaction of the moving body as an
integral part and less on the interface [30, 33]. Various
recent design and research frameworks [32] on movement-
based interaction are dedicated to the role of the body and
bodily movement. However, as Moen emphasizes, “we still
lack the tools, knowledge, and vocabulary to discuss the
Licensed under a Creative Commons Attribution movement and the experience of movement” [37]. To extend
4.0 International License (CC BY 4.0). Copyright her argument, this section emphasizes that we need not only
remains with the author(s). the tools to experience movement but also the interaction
NIME’24, 4–6 September, Utrecht, The Netherlands. modalities to reveal the motivation and drive to move in
response to music that is not often visible to the observer
or even to the performer.
Gesture-based and movement interactions have been an
interest to NIME research since the design of new instru-
ments [23, 8, 47]. However, body and body movement were
more strongly integrated into instrument design with wear-
able instruments. These wearable technologies were initially
developed as hand or glove controllers by musicians, specifi-
cally to create customized interfaces such as Waisfisz’s “The
Hands” [44], Sonami’s “Lady’s Gloves” [42], and Tanaka’s
Biomuse [43]. Following Tanaka’s work, Donnarumma also
adopts a performance-based approach where he focuses on
describing gestures through muscle sensing [15].
Similarly, in NIME research, dance and music interaction
closely influenced each other as much as interface design.
These interfaces, or installations, tend to generate sound
output based on the dance or body movements as main
sound-producing gestures. From earlier examples of dance-
music interactions [24, 2] to more recent developments [19,
12, 36, 45, 38, 1], researchers explored musical interactions
through sensory technologies. Mainsbridge focuses on body
movements as a non-tactile interaction mechanism to con-
trol sound parameters through improvisation [36].

2.2 Creative Evaluation

Creative practice supports the making process beyond cre-
ating a new artifact, leading to conceiving and realizing
ideas in numerous forms [9]. These artifacts can vary from
designs to music composition and performances. Candy and
Edmonds characterize this creative practice “not only by a
focus on creating something new but also by the way that
the making process itself leads to a transformation in the
ideas” that contributes back to the creative artifacts [17,
18].
In music and movement research, as creative artifacts,
new musical instruments lead to new creative practices such
as music composition, performance, and choreography. Be-
cause of the artistic practice behind these research out-
comes, research assessments that involve qualitative, ex-
ploratory, and aesthetics-based evaluations are needed [26].
Such evaluation offers distinct advantages to studying cre-
ative artifacts and the insights and reflections from their
creation process. It provides “the possibility of taking ac-
count of context” such as the inclusion of participants in the
creative practice, the ability to describe as it is perceived Figure 1: Bodyharp, as a wearable string instrument, allows
from different observer perspectives such as combining first performers to play with larger-scale gestures (such as arm
and third-person approaches, and “strong process orienta- movements) and small, nuanced gestures for finer sound con-
tion” such as learning and creation processes [22]. trol. The instrument is flexible in shape and size, extending
The outcomes of creative practice can serve as assess- the performers’ range of movement and space.
ment tools. Ramsay and Rockwell state that creative ar-
tifacts are “tools that show us something in a new light”
[40]. Both from first and third-person observations, body participation” to understand technology’s effects on music-
movement during music-making reveals inner intentions of making [25]. Donnarumma highlights the importance of
music-related gestures and expressions [13, 14, 31, 21, 46]. performance “as a site where to inhabit hybrid forms of
In NIME research, Jack et al. discuss how Digital Musical human-machine embodiment” [16], to understand body- tech-
Instruments (DMIs) can be considered as research products nology relationships based on Berliner’s approach that eval-
[28]. However, they exclude creative artifacts such as mu- uates instrumental practice by adapting “successful perfor-
sical pieces or gestural vocabularies from the category of mance” as criteria [4]. These researchers emphasize the role
research products, although they consider performance as of performance in new instrumental design. However, we
research outcome. still lack methodologies that encourage creative practice as
Mainsbridge conducts performance-led research through an evaluation method for new musical interfaces.
performative inquiry and performance ethnography to cap-
ture the first-person moment and draw reflection on the
practice [36]. Through reflection-in-action in live perfor-
mance, she studies specific design values of her gesture- 3. FRAMING THE CURRENT WORK
based instrument: values of agency, autonomy, empathy,
and transparency [35]. Hayes encourages “creative musical 3.1 Previous Work
This study focuses on how integrating creative practice con- formers had artistic practice in other creative fields in addi-
tributes to evaluating new digital musical instruments, specif- tion to their music or movement backgrounds such as the-
ically wearable, body- and movement-based instruments. ater, poetry, photography, and playwriting. Twelve partic-
Bodyharp (see Figure 1) was previously studied to under- ipants had experience in both music and movement. Their
stand how performers engaged with Body and Space use in levels of experience varied from professional artists to self-
their interaction [10]. These qualities were quantitatively taught, self-exploratory backgrounds. We reported partic-
analyzed based on Laban Movement Analysis [29, 20]. The ipants’ experiences according to their self-reports in Table
results of the previous study inspired this study to investi- 1. Although P10 reported their primary artistic background
gate the creative practice in evaluating the music-movement was in music, they chose not to disclose their age and experi-
interaction with the instrument. In this study, we focus ence in years. Only P7 did not report any information about
on how creative practice affects participants’ learning and their artistic background. The overall participant group
practicing processes when integrated into the user study. showed a diverse distribution of age, gender (optional), and
The details and evaluation of playing techniques and sound experience.
mapping are beyond the scope of this research but can be
accessed in [10]. We report how creativity can be utilized to
evaluate MDMIs beyond technical, practical, and quantita- 4.2 Study Design
tive measures. We also investigated performance possibili- The participants learned the instrument step-by-step based
ties, suggested by participants and later realized in real-life on different categories of gestural interaction in 2-hour indi-
scenarios. vidual sessions. In Step 1, they played the instrument only
by larger-scale gestures (such as arm movements) whereas,
in Step 2, they interacted with the instrument only through
3.2 Playing Bodyharp small-scale, nuanced gestures. In Step 3, they were able to
Bodyharp consists of an instrument body and wearable parts combine both gestural spaces.
including a hand controller and an attachment to the per- After practicing with the instrument through three learn-
former’s arm, connecting the strings to the arm (see Fig- ing sessions (Steps 1-3), the participants created a short
ure 1). The instrument is played by plucking the string or musical statement in Step 4. This prompt only directed the
moving the attached arm to initiate sound production [11]. participants to create a piece based on their explorations
The performer later interacts with tactile and motion sen- from the earlier stages within a short period of time, limited
sors in the hand controller to control sound effects. These to approximately five minutes. However, their interaction
controls include changing the chord progressions (with push was not timed or interrupted by the researchers. We did
buttons), filter quality factor (with circular pressure sensor– not pose any other limitations. Until this stage, the partic-
FSR), gain and note duration (with sliders), and filter drive ipants explored different possibilities with the instrument,
factor (with square FSR). The sound mapping was imple- understanding its affordance and limitation and extending
mented in ChucK audio programming language 1 to receive these inherent interactions. They were later asked to fo-
and process the sensor data, control string physical models, cus on creating a cohesive composition that allowed them
and record audio output of performances. The perform- to reflect on their learning and practicing processes. After
ers can further interact with the string interface by pluck- this step, the participants created short movement compo-
ing, stretching, moving their arms, or isolating individual sitions/choreographies without the musical instrument in
strings. Expressive interactions can provide new affordance Step 5 while listening to their musical composition from
although the sound excitation mechanism remains the same the previous step. This prompt was asked to allow par-
[10]. ticipants to express their musical composition using body
movements beyond the interface’s limitations and reflect on
the sound-movement relationships.
4. METHODOLOGY Their creative interactions in these steps are video and
The experiment focused on the process of learning through audio recorded. The outcomes of the study prompts were
gestural/movement exploration, practicing nuanced control, stored for further analysis in video and audio formats. After
and developing creative artifacts through sound-movement the creative practice, in a semi-structured interview, the
interaction. We investigated their learning process and com- participants were asked to reflect on their experience during
pared this process to their creative practice with the instru- the performance practice and to brainstorm on new uses
ment. The creative practice included creating musical and of the instrument for performance scenarios in the wild.
movement compositions. First, participants learned the in- Along with the participants, the recordings of their creative
struments through the linked music-movement interactions. artifacts, specifically the video recordings, were returned to
Later, the two domains were isolated to unpack how partic- discuss specific moments or patterns of interaction.
ipants reflected on their coupled and decoupled music and
movement interactions.
4.3 Analysis
The participants’ reflections were recorded through written
4.1 Participants questionnaires and audio recordings of the interview discus-
The study recruited twenty participants with artistic back- sions. Their responses were transcribed using Otter.ai 2 for
grounds in sound, movement, or both via email. All par- thematic analysis to identify common and unique themes
ticipants provided informed oral consent before the study. among different participants, similar to Braun and Clarke’s
Participants with backgrounds in sound (nineteen of the 20 reflexive approach [6, 7]. In addition to reported experi-
participants) engaged in music performance, composition, ences, participants’ process of making some artistic choices
instrument design, and audiovisual performance. Partici- was documented through researcher observations, discus-
pants with backgrounds in movement (thirteen of the 20 sions with participants, and reflections on both sound- and
participants) performed as dancers or worked as choreogra- movement-based creative artifacts. Completion of the mu-
phers, performer artists, or contact improvisers. Most per- sical and movement statements, attentive listening of the
1 2
https://chuck.cs.princeton.edu https://otter.ai
 Table 1: Participant Demographics of the First Case Study
P Age Music Movement Dominant P Age Music Movement Dominant
1 35-40 1 35 Movement 11 45-50 30 30 Movement
2 50-55 2 15 Movement 12 30-35 10 11 Music-Movement
3 25-30 10 5 Music 13 20-25 14 0 Music
4 25-30 5 16 Movement 14 20-25 22 12 Music-Movement
5 45-50 41 6 Music 15 25-30 25 1 Music
6 30-35 27 0 Movement 16 20-25 15 3 Music-Movement
7 70-75 - - - 17 35-40 30 0 Music
8 65-70 30 0 Music 18 20-25 20 3 Music
9 20-25 20 22 Music-Movement 19 20-25 18 6 Music-Movement
10 - - - Music 20 35-40 26 0 Music

sound and movement, and other factors like repeatabil- the participants to think creatively and develop an evolving
ity and originality were taken into account in the analysis. creative practice with the instrument.
Codes and themes were extracted using NVivo 12 software3 . Transitioning from exploration of movement to repetition
of sounds occurred throughout the learning process. How-
ever, participants’ progress varied since some preferred dif-
5. DEVELOPING CREATIVE PRACTICE ferent gestural interactions over others. For many partici-
The participants’ experiences in developing a creative prac- pants, learning how to control the sounds in a more nuanced
tice with Bodyharp were supported in three aesthetically- way developed over time in these steps. For example, P12, a
driven processes from learning to creating and from reflect- dancer and a musician, expressed that she initially couldn’t
ing to recollecting expressive ideas. Although we focus on play rhythms until she “found the [pressure] sensor on the
the creative process during this study, the learning experi- side which gave a rhythmic element to the music.” Some
ences remained connected to the creative process and influ- participants, including P2, reported challenges in control-
enced the resulting creative artifacts. The first steps of the ling the dynamic range of the instrument. However, they
study encouraged participants to learn how to play the in- stated that their control improved during the learning pro-
struments, existing gestures, and affordances through sonic cess.
and movement explorations. After the exploratory stages,
the following steps investigated their creative process when
they were asked to create a musical statement and move- 5.2 Creative Process
ment improvisations based on their own compositions. We
After exploring the instrument, in Step 4, where partici-
observed the evolution of how the participants developed a
pants composed musical statements, the participants be-
creative practice with a new musical instrument. Although
came more focused on how their movement created specific
they had never played the instrument before, their pro-
sounds. P1, a dancer and a choreographer, reported that
cess of adapting the instrument to their artistic preferences
“being asked to create [a musical statement] makes me much
and intentions was clearly observed across the experimental
more aware of the sounds that my movements are creating.”
steps. In this section, we report participants’ experience of
In this step, participants returned to learned movement and
developing a creative practice at three stages: (1) the learn-
sonic expressions. For example, P1 said “I become much
ing process, (2) the creative process, and (3) the reflective
more aware of the sounds that I am creating, not so much
process.
of exploring but composition, having repetitions, going back
to the themes I had, and repeating the same movements with
5.1 Learning Process my body.”
The experiment allowed the participants to individually ex- We asked the participants to articulate their composi-
plore the instrument’s affordances by learning them at each tional ideas. Most participants started this step with an
step based on a specific gestural interaction. P10 expressed idea of a compositional structure. However, even partic-
that this learning method helped with “remembering differ- ipants with experience in composition and choreography
ent elements of the interaction.” This step-by-step learning momentarily returned to exploration. P11, a dancer and
helped the participants learn different gestural affordances a choreographer, stated that although “sometimes I forgot
gradually but also encouraged them to “understand different that it was a performance,” they further expressed, “the per-
possibilities that would be left unexplored otherwise” within formance kept me present.” These experiences showed that
these gestural frameworks, as P11 highlighted. integrating creative practice into the user study supported
Throughout the learning process, participants realized participants’ presence and focus. P4, a dancer, shared her
how to access different movement expressions. P2 shared experiences with focusing on the sound-creation process by
how he used musical gestures and body movements: “the saying “The sound carried through my body [...], I also felt
majority of my mind was on how it works, [...] using the very focused [...]. There was an interesting process of atten-
dance gesture as an interference while using musical ges- tive listening.”
tures as the main control.” As highlighted by P6, the par- Bodyharp not only facilitates sonic creativity but also
ticipants were able to isolate certain gestures to create the simultaneously supports movement expressivity. To bet-
desired sound. She explained how this learning experience ter understand the sound-movement correspondences, we
supported developing a creative process: “As I was learning asked the participants to improvise their composition with
how to play the instrument one element at a time, I was body movements without the instrument. When partic-
also learning how to compose and perform with it. [...] It ipants created movement compositions without the musi-
was very intuitive.” Their exploration, in a way, prepared cal instrument, their movement response to music provided
them with an opportunity to reflect on the sound-movement
3
https://lumivero.com/products/nvivo/ relationship. P4 explained “[composition and movement im-
Figure 2: The moments from (a) music composition (Stage 4) and (b) movement improvisation (Stage 5) are demonstrated,
respectively, for P12 (musician and dancer), P11 (dancer and choreographer), P6 (musician and composer), and P1 (dancer
and choreographer).

provisation] captured this intermediate explicit type of my part of the piece since you are attached to it. ” She also
movements and the sounds patterns that I create.” Some expressed that the prompt of “create a musical statement”
participants realized corresponding expressions. P12 ex- made her focus on something that she “had control over.”
pressed that “since I knew how I wanted to create musically, One of the reasons why participants stayed present and
I also knew how I wanted to move to the music.” She further focused on their interaction was because the prompt, asking
elaborated that she tried to “closely reflect the music in- them to create a sound/movement artifact, shaped the par-
stead of doing something different.” Similarly, P4 expressed ticipants’ approach to their interaction in a more structured
“After I explored the sound and when it was played back, I way, just as P6 expressed. P12 stated that “I did have an
found myself doing things in line with the [music] recording idea of what I wanted to compose but of course, I couldn’t
but not intentionally.” P11 commented on a similar expe- make it exactly how I wanted to. But I still had some sort
rience that they were “surprised how much [of movement] of structure.” She further described her compositional plan
they remembered from the practicing with the instrument.” as she “wanted to start from the bottom and start from the
The relationship between the two stages of the creative low register of the notes and kind of expand something big-
practice (music composition and movement interpretation) ger and go back there for a second.” Similarly, P15 started
varied for participants with different backgrounds. Figure composing with a structure and he expressed “I gave a little
2 shows moments from their (a) music composition (Stage bit of thought, ahead of time, about what the general shape
4) and (b) movement improvisation (Stage 5), respectively, I wanted it to be, but it was still mostly improvised.”
for P12 (musician and dancer), P11 (dancer and choreog- This creative process affected how the participants con-
rapher), P6 (musician and composer), and P1 (dancer and sidered articulating their artistic intentions. Although some
choreographer). participants needed a longer exploration period to feel com-
fortable with the instrument, the creative practice affected
their intentionality. P9 reported “When I was composing, it
5.3 Reflective Process was still more on the learning side, but I tried to be more in-
After creating sound and movement compositions, the par- tentional about the sounds I was making.” Even participants
ticipant reflected on their creative process as part of the with experience in composition and choreography momen-
semi-structured interview. They addressed the specific mo- tarily returned to a state of exploration. P11 shared that “It
ments in their composition, drawing relationships between was very wild to start making a composition with this brand-
music and movement compositions. The following five themes new relationship with the instrument. At the same time, I
emerged after analyzing the participants’ reflections. As a am very comfortable with performing in an unfamiliar sit-
direct result of the creative practice, these themes allowed uation. Sometimes, I forgot that it was a performance, but
us to extract criteria to evaluate movement-based musical it was because of how exciting it was to explore.”
instrument interaction.

5.3.2 Performance as an End Goal

5.3.1 Increased Focus on Creativity Although we value the creative process over the outcome
Directing participants to develop creative artifacts with and in this research, many participants, especially those with
without the instrument led them to articulate their artis- composition and choreography backgrounds, reported that
tic explorations in more focused, concise, and clear interac- as they were interacting, they had real-life performance pos-
tions. P1 expressed how the creative practice affected her sibilities in mind. P1 stated that “[...] since the beginning, I
attention: am already thinking of performance. How do I perform with
this?”
“When you say, create a piece, I become much Although each participant performed with the instrument
more aware of the sounds that I was creating and solo, we discussed different performance scenarios that the
my composition. I was thinking not so much of participants imagined playing the instrument in. Many
the exploring, but the composition.” participants reported that the instrument can be played
as a collaborative or duo performance of a musician and
Similarly, P6 reported how her awareness of body move- a dancer. Some imagined it performed solo or in music en-
ments increased. She said “It made me very aware of un- sembles with other string instruments. Other performance
intentional movements. It made me very aware of getting settings reported by the participants included collaborative
into position or moving because every movement you do is dance and theatre pieces, duo performances with dancers,
musicians mimicking dancers in interdisciplinary settings, new patterns of movement expressions. This decoupling
and accompanying other musicians in ensembles. Collabo- is specifically important since the instrument intertwines
rative or duo performance suggestions ranged from multiple sound-making and movement expression.
people simultaneously interacting with Bodyharp to sharing After the composition practice with the instrument in
the same stage with other artists to create interdisciplinary Step 4, some participants utilized the movement improvisa-
work where Bodyharp’s mapping not only controls sound tion without the instrument in Step 5 as a way to explore
but also other media. P5 imagined it as a contact impro- new movement patterns. P10 saw this process as an iter-
visation or a clowning instrument and P11 suggested using ative tool to explore new ways of playing the instrument
Bodyharp as a movement-based art piece that supports “the through movement improvisation. She stated “It seems like
creative process in returning to a place of not knowing” in a good way of iterating how to adjust gestures. In the dance,
dance teaching. I started to think of different ways of moving and I wanted
Some of the feedback from participants was employed in to go back to the instrument.” P18 also reported similar
real-life performance situations. Figure 3 shows four case experiences that during the movement improvisation, she “
studies of performance with Bodyharp, following the results noticed that when [she] was just responding to the music,
of the user study: Bodyharp was played in (a) a duo per- [she] can access a whole new set of movements that is also
formance with a flautist, (b) a quartet with three dancers responsible for creating music.”
where dancers used Bodyharp’s movement patterns as cues Participants’ movement engagement showed that reflect-
for choreographic events, and (c) a duo performance with a ing on their creative practice through movement not only
dancer, interacting together with the string and touch sen- supported the exploration of developing new movement in-
sors, (d) as part of an audio-visual installation performance. teraction but also revealed their interpretation between sonic
The study of these performance experiences in the wild is interaction and body movements. An example of this in-
beyond the scope of this paper. However, they show the terpretation was observed in interpreting reverberation and
potential of the creative practice with Bodyharp. beating effects with finger or hand-waving gestures.

5.3.3 Repeatability 5.3.5 Sonic Awareness and Active Listening

The repeatability of sonic and gestural motifs, phrases, and “The sound carried through my body [...], I also
sequences appeared as a common theme across almost all felt very focused [...]. There was an interest-
participants. During the creative practice, we observed that ing process of attentive listening. I had to listen
the participants frequently returned to repetition of a subset more.”
of gestures. Some of these gestures were unique to their
performers. As P4 expressed how sound-making with body move-
Participants also reported that finding sonic and gestu- ments affected her focus, many participants experienced a
ral motifs that they can rely on their repeatability helped shift between their sonic outcome and their body move-
both their learning and creative process. P1, a dancer and ments. The participants guided their movements based
a choreographer, reflected on her experience with repeti- on the sound they were creating. P11 reported that they
tion: “[...] like having repetition, going back to the motif were aware of “making choices of letting the sound process
or themes that I found. Also repeating the same movement through [them] and following the sound with movements;
of my body with the sounds that I was creating. [...] I re- other times, doing the opposite of [what] the sound offers.”
member the feeling [of one gesture] and repeat it so that [it] Similarly, P6 was focusing her attention to the sound-
sticks to the brain, that sound, that gesture.” P12 carried making: “I was making big gestures but I wasn’t thinking of
repeatable phrases to her composition as she expressed “In- the gestures, but I was thinking about how it sounded. It was
stead of moving in space, I started to stay in one place, and making me move without really letting me think about how
I was trying to repeat musical phrases. I did try to do that I was moving.” Participants’ focus on their listening also
[repetition of musical/gestural phrases] in the musical com- increased the expensiveness in their movement, potentially
position.” For P18, repeatability created a stronger sense of supporting their movement engagement.
control and achievement. She reported: This active listening process also helped the participants
with connecting to their bodies. P15 expressed “I felt like
“I felt accomplished when there was some motif the movements that I made when I was making the musical
that I could repeat. I felt a musical intuition, statement were more connected to the sound [...] than the
coming from my ability to play this instrument, movements when I was interpreting the musical statement.
that was nice. Finding a motive that way made I connected more physically when I was also making sound
me feel like I was playing a piece that I could than I wasn’t.”
come back to. Having some repetition material
that is actually interesting to listen to. Same
way with moving.” 6. CROSS-DISCIPLINARY INTERACTIONS
We observed that the creative practice allowed participants
We noticed that encouraging creative practice during the to refer back to their artistic backgrounds while creating
study supported participants in developing a sonic/gestural sound and movement compositions. The interactions be-
vocabulary that they could return to to create musical and tween the sound and movement domains allowed partici-
movement compositions. It helped them to reflect on what pants from either discipline to transfer their experience in
they have learned with the instrument through repetitions artistic practice into movement-based music-making while
and use such material to develop creative artifacts. learning from the other artistic domain.

5.3.4 Movement Engagement 6.1 Choreographic Tools

Although the instrument was originally designed to increase Choreographers reported their considerations for the au-
musicians’ movement engagement, decoupling the musical dience and how the performance can be perceived from
and movement creativity allowed participants to explore their perspective. P1, a dancer and a choreographer, stated
Figure 3: Bodyharp’s performance practices were developed based on the participant’s feedback and reflections, showing (a) a
duo performance with a flautist, (b) a quartet with three dancers, and (c) an interactive duo performance with a dancer, (d) an
audio-visual installation performance. In dance performances, the instrument was both played collaboratively with a dancer,
i.e., musician and dancer interacting with the sensor and the string interfaces, and played with four dancers, i.e., the musician
offering choreographic cues to the dancers with her body movements.

“since the beginning, I am already considering performance. uating their interaction: increased focus on creativity, per-
How do I perform with this? How does the performance look formance as an end goal, repeatability, movement engage-
from outside?” Similarly, another choreographer, dancer, ment, and sonic awareness and active listening.
and improviser (P11), reflected on their creative practice
and expressed “I was considering what makes a composition
good composition or whether I wanted to create a good com- 7.1 Limitations and Future Work
position.” where she applied some of the choreographic tools The creative practice encouraged future collaborations be-
to playing the instrument: changing height levels, adjusting tween the participants and researchers (see Figure 3). These
the direction to face the body, and following the sound or performances were co-developed based on participants’ re-
opposing what the sound suggests. flections following the study and show that the evolution of
creative practice positively impacted realizing real-life per-
formance opportunities. We recognize that a longitudinal
6.2 Compositional Tools study, similar to Reimer’s research [41], would provide a
P15, a composer, shared composing with Bodyharp was dif- more in-depth understanding of how integrating creative
ferent from his regular composition process: “I was think- practice as an evaluation method supports sustained use
ing of different things when I do not normally think about in and long-term artistic practice with Bodyharp. Moving for-
composition.” Instead of considering “notes, harmonies, and ward, we plan to study artists’ long-term practice with the
specific rhythms” as compositional materials, he stated that instrument and compare how their practice evolves. The re-
he was “almost entirely thinking of gestures, both musical searcher’s role should be acknowledged due to their active
gestures and physical gestures. It wasn’t so much of specific participation in the design process. Although they remained
sounds [...], but the shapes of those sounds.” He further ex- as an objective observer during the study, some participants
pressed that he would like to incorporate this approach to also occasionally needed clarifications from the researcher.
his compositional process. We also recognize the laboratory setting for the current
A dancer who is also a musician, P12 described her vision study’s creative practice development as a limitation com-
for the performance which included “starting with loudness pared to performance in the wild cases [3]. However, we
and craziness and then quite and then one string. You just note that integrating creative practice can still offer more
focus on that one string. And it expands from there again naturalistic, aesthetically informed, and holistic evaluation
and goes away. That came to me in that moment.” Her ap- methods for NIME researchers who wish to extend their
proach showed that creative practice in one domain can sup- qualitative and quantitative evaluation approaches.
port another domain when the connection is built through
the instrument. She reported that “for the dance, knowing
what I wanted to do musically, I knew how I wanted to move Ethical Standards
[to the dance].”

All participants provided informed oral consent and volun-

7. CONCLUDING REMARKS tarily participated in the study. The oral consent is col-
lected according to Stanford University’s necessary IRB ap-
In this paper, we study how creative practice can be utilized
provals on nonmedical human subject studies with the e-
as an evaluation method and how developing this practice as
protocol number 59665. The participants were informed
part of the user study affects participants’ interaction, cre-
about how we plan to confidentially use the data for aca-
ativity, and experience, in the context of movement-based
demic and artistic purposes. The research is partially funded
musical interaction. This musical practice aims to encour-
by the Center for Computer Music Research and Acoustics’
age the creative thinking process for increased longevity,
research funds and Graduate Research Opportunity funds
exploration, and creative utility.
from Stanford University.
We studied this creative practice by guiding the partici-
pants through learning and practicing with the instruments,
allowing them to construct creative artifacts (as music and
movement compositions, sonic and gestural vocabularies), 8. REFERENCES
and supporting their reflection on the practice through em- [1] L. Andersson López, T. Svenns, and A. Holzapfel.
bodied, soma-based evaluation methods such as defamiliar- Sensitiv – designing a sonic co-play tool for
ization, movement explorations, or non-verbal recollection. interactive dance. In Proceedings of the International
Our analysis revealed five main themes for understanding Conference on New Interfaces for Musical Expression,
participants’ creative process and for aesthetics-based eval- Shanghai, China, June 2021.
 [2] C. Bahn, T. Hahn, and D. Trueman. Physicality and R. Knott, and S. Pauletto. The studio as laboratory:
feedback: a focus on the body in the performance of Combining creative practice and digital technology
electronic music. In Proceedings of International research. International Journal of Human-Computer
Computer Music Conference, 2001. Studies, 63(4):452–481, 2005. Computer support for
[3] S. Benford, C. Greenhalgh, A. Crabtree, M. Flintham, creativity.
B. Walker, J. Marshall, B. Koleva, [19] C. Erdem, K. H. Schia, and A. R. Jensenius. Vrengt:
S. Rennick Egglestone, G. Giannachi, M. Adams, A Shared Body-Machine Instrument for Music-Dance
N. Tandavanitj, and J. Row Farr. Performance-led Performance. In Proceedings of the International
research in the wild. 20(3), jul 2013. Conference on New Interfaces for Musical Expression,
[4] P. F. Berliner. Thinking in jazz: The infinite art of pages 186–191, 2019.
improvisation. University of Chicago Press, 2009. [20] S. Fdili Alaoui, J. Françoise, T. Schiphorst, K. Studd,
[5] C. Bossen, C. Dindler, and O. S. Iversen. Evaluation and F. Bevilacqua. Seeing, sensing and recognizing
in participatory design: a literature survey. In laban movement qualities. In Proceedings of the 2017
Proceedings of the 14th Participatory Design CHI Conference on Human Factors in Computing
Conference: Full papers-Volume 1, pages 151–160, Systems, CHI ’17, page 4009–4020, New York, NY,
2016. USA, 2017. Association for Computing Machinery.
[6] V. Braun and V. Clarke. Reflecting on reflexive [21] S. Fdili Alaoui, T. Schiphorst, S. Cuykendall,
thematic analysis. Qualitative Research in Sport, K. Carlson, K. Studd, and K. Bradley. Strategies for
Exercise and Health, 11(4):589–597, 2019. embodied design: The value and challenges of
[7] V. Braun, V. Clarke, N. Hayfield, L. Davey, and observing movement. In Proceedings of the 2015 ACM
E. Jenkinson. Doing Reflexive Thematic Analysis, SIGCHI Conference on Creativity and Cognition,
pages 19–38. Springer International Publishing, pages 121–130, New York, NY, USA, 2015.
Cham, 2022. Association for Computing Machinery.
[8] A. Camurri, S. Hashimoto, M. Ricchetti, A. Ricci, [22] V. P. Glăveanu. Creativity in context: The ecology of
K. Suzuki, R. Trocca, and G. Volpe. Eyesweb: creativity evaluations and practices in an artistic
Toward gesture and affect recognition in interactive craft. Psychological Studies, 55(4):339–350, 2010.
dance and music systems. Computer Music Journal, [23] R. I. Godøy and M. Leman. Musical gestures: Sound,
24(1):57–69, 2000. movement, and meaning. Routledge, 2010.
[9] L. Candy and E. Edmonds. Practice-based research in [24] T. Hahn and C. Bahn. Pikapika–the collaborative
the creative arts: Foundations and futures from the composition of an interactive sonic character.
front line. Leonardo, 51(1):63–69, 2018. Organised Sound, 7(3):229–238, 2002.
[10] D. Cavdir and S. Dahl. Performers’ use of space and [25] L. Hayes. Enacting musical worlds: Common
body in movement interaction with a movement-based approaches to using nimes within both performance
digital musical instrument. In Proceedings of the 8th and person-centred arts practices. In Proceedings of
International Conference on Movement and the International Conference on New Interfaces for
Computing, MOCO ’22, New York, NY, USA, 2022. Musical Expression, NIME 2015, page 299–302, Baton
Association for Computing Machinery. Rouge, Louisiana, USA, 2015. The School of Music
[11] D. Cavdir, R. Michon, and G. Wang. The bodyharp: and the Center for Computation and Technology
Designing the intersection between the instrument (CCT), Louisiana State University.
and the body. In Proc. of the 15th International [26] L. Hayes and A. Marquez-Borbon. Nuanced and
Conference on Sound and Music Computing (SMC, interrelated mediations and exigencies (nime):
2018), Limassol, Cyprus, 2018. Addressing the prevailing political and
[12] P. Dahlstedt and A. S. Dahlstedt. Otokin: Mapping epistemological crises. In R. Michon and F. Schroeder,
for sound space exploration through dance editors, Proceedings of the International Conference
improvisation. In Proceedings of the 19th on New Interfaces for Musical Expression, pages
International Conference on New Interfaces for 428–433, Birmingham, UK, July 2020. Birmingham
Musical Expression (NIME-19), pages 156–161, 2019. City University.
[13] J. W. Davidson. Visual perception of performance [27] K. Höök, B. Caramiaux, C. Erkut, J. Forlizzi,
manner in the movements of solo musicians. N. Hajinejad, M. Haller, C. Hummels, K. Isbister,
Psychology of music, 21(2):103–113, 1993. M. Jonsson, G. Khut, et al. Embracing first-person
[14] J. W. Davidson. What does the visual information perspectives in soma-based design. Informatics,
contained in music performances offer the observer? 5(1):8, 2018.
some preliminary thoughts. In Music and the mind [28] R. Jack, J. Harrison, and A. McPherson. Digital
machine, pages 105–113. Springer, 1995. musical instruments as research products. In
[15] M. Donnarumma. Configuring Corporeality: R. Michon and F. Schroeder, editors, Proceedings of
Performing bodies, vibrations and new musical the International Conference on New Interfaces for
instruments. PhD thesis, Goldsmiths, University of Musical Expression, pages 446–451, Birmingham, UK,
London, 2016. July 2020. Birmingham City University.
[16] M. Donnarumma. Beyond the cyborg: performance, [29] R. Laban and L. Ullmann. The mastery of movement.
attunement and autonomous computation. 1971.
International Journal of Performance Arts and [30] A. T. Larssen, T. Robertson, L. Loke, and
Digital Media, 13(2):105–119, 2017. J. Edwards. Introduction to the special issue on
[17] E. A. Edmonds. Human Computer Interaction, Art movement-based interaction. Personal and Ubiquitous
and Experience, pages 11–23. Springer International Computing, 11(8):607, 2007.
Publishing, Cham, 2014. [31] M. Leman. Embodied music cognition and mediation
[18] E. A. Edmonds, A. Weakley, L. Candy, M. Fell, technology. MIT press, 2007.
[32] L. Loke, A. T. Larssen, T. Robertson, and
J. Edwards. Understanding movement for interaction
design: frameworks and approaches. Personal and
Ubiquitous Computing, 11(8):691–701, 2007.
[33] L. Loke and T. Robertson. Moving and making
strange: An embodied approach to movement-based
interaction design. ACM Transactions on
Computer-Human Interaction (TOCHI), 20(1):1–25,
2013.
[34] T. Magnusson. Designing constraints: Composing and
performing with digital musical systems. Computer
Music Journal, 34(4):62–73, 2010.
[35] M. Mainsbridge. Feeling movement in live electronic
music: An embodied autoethnography. In Proceedings
of the 8th International Conference on Movement and
Computing, MOCO ’22, New York, NY, USA, 2022.
Association for Computing Machinery.
[36] M. Mainsbridge and K. Beilharz. Body as
instrument–performing with gestural interfaces. In
Proceedings of the international conference on new
interfaces for musical expression, 2014.
[37] J. Moen. KinAesthetic movement interaction:
designing for the pleasure of motion. PhD thesis,
KTH Royal Institute of Technology, 2006.
[38] e. moraitis. Symbiosis: a biological taxonomy for
modes of interaction in dance-music collaborations. In
R. Michon and F. Schroeder, editors, Proceedings of
the International Conference on New Interfaces for
Musical Expression, pages 551–556, Birmingham, UK,
July 2020. Birmingham City University.
[39] S. O’Modhrain. A framework for the evaluation of
digital musical instruments. Computer Music Journal,
35(1):28–42, 2011.
[40] S. Ramsay and G. Rockwell. Developing things:
Notes toward an epistemology of building in the
digital humanities. Debates in the digital humanities,
pages 75–84, 2012.
[41] P. Reimer. Longitudinal Evaluation of User
Experience with Digital Musical Instruments:
Development and Demonstration of a New Method.
PhD thesis, McGill University (Canada), 2023.
[42] L. Sonami. On my work. 2006.
[43] A. Tanaka and M. Donnarumma. The body as
musical instrument. The Oxford Handbook of Music
and the Body, 1, 2018.
[44] G. Torre, K. Andersen, and F. Baldé. The hands: The
making of a digital musical instrument. Computer
Music Journal, 40(2):22–34, 2016.
[45] J. N. Tragtenberg, F. Calegario, G. Cabral, and G. L.
Ramalho. Towards the concept of digital dance and
music instruments. In M. Queiroz and A. X. Sedó,
editors, Proceedings of the International Conference
on New Interfaces for Musical Expression, pages
89–94, Porto Alegre, Brazil, June 2019. UFRGS.
[46] F. J. Varela and J. Shear. First-person methodologies:
What, why, how. Journal of Consciousness studies,
6(2-3):1–14, 1999.
[47] M. M. Wanderley and P. Depalle. Gestural control of
sound synthesis. Proceedings of the IEEE,
92(4):632–644, 2004.
[48] M. M. Wanderley and N. Orio. Evaluation of input
devices for musical expression: Borrowing tools from
hci. Computer Music Journal, 26(3):62–76, 2002.