21 ST CENTURY MAKERS AND MATERIALITIES

Proceedings of the 2nd Biennial

Research Through Design Conference | RTD 2015

Andersen, K., and Gibson, D. 2015. The Instrument as the Source of new in new Music. In:
Proceedings of the 2nd Biennial Research Through Design Conference, 25-27 March 2015,
Cambridge, UK, Article 26. DOI: 10.6084/m9.figshare.1327992.
 The Instrument as the Source of new
in new Music

Kristina Andersen and Dan Gibson

the source of new in new music. We would like to present an instrument

STEIM,
Amsterdam, The Netherlands design process conducted with a musician as visionary and agenda setter.
As the instrument grows and evolves through the various stages, it remains
[email protected] playable and faithful to the desire to make music. The resulting objects are
[email protected] experimental prototypes of technological matter, which allow analysis and
meaning to be specified through physical and tactile interaction with the
object itself. At RTD2015 we will present a range of these intermediate
Abstract: Like any traditional instrument, the potential of a new instrument
prototypes and play the finished instrument.
and its possible music can only be revealed through playing. How can we
treat technological matter as yet another material from which our notions Keywords: Instrument; embodied; making; improvisation; performance,
of possible future instruments can be constructed, intrinsically intertwined prototype.
with and informed by a practise of performance? Our approach to
developing instruments for musical performance strives to not just make
technology, but aesthetic and cultural objects. A musical instrument is not
an interface and should not be designed as such, instead, instruments are
2
Andersen and Gibson | The Instrument as the Source of new in new Music
 available technologies and techniques to facilitate artists in building new
‘The musical instrument is a vehicle for the desire to make music. It instruments for their professional musical practise. Our community tends
is both something that must be internalized, incorporated and made to focus on improvised and experimental electronic music centered on
flesh and something other, without which we could not get to that live performance. Within this setting it is natural for us to make use of a
sound from the world beyond’ Joel Ryan (Ryan and Andersen, 2014). broad range of techniques from theatre, dance and performance. We take
inspiration from the experimental theatre work of Boal (1992), the art
practice of estrangement as described by Shklovsky (1917) and Dewey’s
Introduction
notion of experience as a process of becoming (1958).
This paper describes an embodied design process conducted as a
collaboration between a musician and a researcher. It takes its origin in the This type of work is similar to critical design methods such as Placebo
musician’s desire to rebuild and re-imagine a cello as the site, not only for Designs (Dunne and Raby 2002), practical methodologies like the
the generation of musical sound, but also for the complex modification and Future Technology Workshops (Vavoula et al, 2002), and techniques for
processing of that sound through software. The underlying concern is to inspiration and understanding of users like the Cultural Probes (Gaver et
not just improve on an existing instrument or make it suitable for electronic al 1999). Techniques of ideation can also draw on traditions of craft, and
music, but rather to create structures that might fit better with the artist’s technology as a ‘’magical unknown’’ has been used as the jumping off point
mental images of the music. As such, we are not directly concerned for a range of workshop techniques that begin with material exploration
with traditional usability issues such as potential increases in efficiency, (Andersen 2012, 2013, 2014). In this case we specifically make use of the
but rather we aim for increased flow, intuitive modes of expression and making of non-functional mock-ups as a way to imagine a technological/
unbroken periods of concentration within the experience of playing music. musical element that is yet to be build. These then act as props, through
which the participant can express their intuitions and concerns with a given
Methodology and Context technological or musical notion.
This project is conducted within an art setting at a studio for electro-
acoustic music. The focus of the work done at the studio is to use any
4
Making New Instruments creative expression, we build tentative and hypothetical instruments
as non-functional physical mock-ups and prototypes. Each resulting
A new instrument should provide an intuitive interface between gesture
hypothetical instrument reveals new strengths and weaknesses in a
and sound, which allows for the development of virtuosity. In this way,
proposed instrument design, but more importantly the process temporarily
an instrument should provide a long lasting and fulfilling interaction that
frees us from considering practical and technical limitations and concerns.
exceeds the novelty of its modifications and extensions. Similarly, the new

Andersen and Gibson

instrument should be capable of surprising the performer and allowing the
continual renewal of musical possibilities through exploration, practice and
Non-functional Prototypes
discovery. A new instrument also requires a certain level of stability and We make use of physical making as a way of allowing us to think with
durability, providing the longevity suitable for traveling and performing. It our hands and then letting the resulting object support the imagining
should also provide a sense of reliability, allowing the performer to act with and talking about that, which is yet to be fully understood or even
immediacy, focusing on aesthetic and musical considerations rather than imagined. The resulting objects can be seen as active props, animated and
the technical details that lie behind the interface. investigated through playing. The prop allows us to fantasise and guess
about its functionality and use. Certain features will be accidental, dictated
When it comes to playing an instrument, whether it is traditional or new, by the limited material available and the short time in which it was built,
the body of the performer has always been in the centre of things, with but as the maker stands up to explain and “play”, all features (intended and
the control and sound properties of an instrument intimately linked to it’s unintended) work together to inspire and inform the presentation. In that
acoustical properties, technical execution, size, weight and construction moment of heightened alertness, the prop itself instructs us as to how it
(see also Gibson’s theory of affordances (1977)). Considering new should be handled and used.
technology for creative expression, through the framework of objects and
instruments, allows us to address the potential design and solution space This process makes use of mechanics similar to pretend play: the ability to
through the performing body of the musician. represent one object as two things at once, the ability to see one object as
representing another, and the ability to represent mental representations
To facilitate an initial embodied conversation about instruments for (Lillard, 1993). The child knows that the stick in his hand is a stick and yet

5
 Process Notes:
“My experiences with improvising and playing
acoustic instruments led me to an interest in real-
time electronic music. The detailed and expressive
control of the cello is well suited to this and provides
a wide range of possibilities for transformation using
software.”

“Processing electronic sound provides the potential

for creating more dynamic transitions and
transformations to the cello sound.”

“I used to achieve this by using a separate

commercial interface with sliders and switches to
modify the sound from the cello. As the cello usually
requires two hands to fully articulate the acoustic
sound, reaching over to the controller creates a break
in the flow of the performance - both cognitively
and physically. This is problematic when improvising,
as it becomes difficult to act spontaneously and
intuitively.”

“My first explorations of the cello involved developing

an understanding of the instrument’s sonic and
physical properties through playing, practice and
reflection.”

6
at the same time, within the game, it is also a sword. We can hold these continuous and discontinuous sounds. The use of the bow also amplifies
two realities at once, they are both in some sense true simultaneously. the performer’s movements - allowing for subtle manipulations to be made
This mirrors Augusto Boal’s games where educated, highly attuned forms to the sound.
of listening and paying attention allows the players to gather knowledge
through improvisation (1992) and is in turn reminiscent of Callois’ game The large size of the fret board provides a high resolution for the control

Andersen and Gibson

spaces (1961): as in a game we erect a tent in time where experimentation of pitch, allowing very small movements to make large tonal differences.
can take place. In addition to this, the amount of finger and bow pressure applied to the
strings results in diverse differences in the density/spectral content of the
The Cello sound.

The cello was chosen as point of departure due to both its physical form The quieter tapping and scratching sounds, produced by playing the bridge
and tonal qualities. For the purpose of this project it acts as both sound directly, have a more direct/closer quality, picking up more detail, which
source and physical interface for the real-time creation of improvised compliments the thicker/more full sounds of the strings. The complexity
electronic music. A long list of qualities makes the cello uniquely suited for and interdependence of these aspects lead to a deeply engaging and
such an exploration. responsive interaction with the instrument, whilst retaining intuitive control
of the sound.
The cello offers a wide range of sounds and intuitive control over subtle
transformations of the sound’s timbre and texture. More specifically, the Understanding the Unmodified Instrument
range of percussive and tonal sounds, that can be made using the bow
We start by reconsidering the qualities of the un-modified cello. Through
and fingers offer many possibilities for producing sound: plucking, bowing,
examining and exploring the traditional playing techniques and sonic
scratching, bouncing, tapping, scraping.
qualities, an understanding develops regarding what is and is not possible
The different materials, shapes and edges of the bow allow for different with the instrument in it’s initial state. In particular, the relationship
playing techniques, producing a wide range of percussive and tonal, between the bodies of the cello and performer are considered with
issues of balance and stability revealing themselves as significant. Whilst
7
8
Process Notes:
“By building a series of conceptual
models of the cello, we reduced it to its
fundamental components, so that we
could gain a better understanding of
the functionality and physical processes
involved in the instrument.”

“We experimented with resonance,

Andersen and Gibson

vibration, and physical stance by
constructing and playing cello-like devices
made from cups, cans, string and elastics.
We did this to explore the instrument
from different angles and reveal the extent
that each orientation between the cello
and the performer constricts or frees the
experience of playing.”

“The cello revealed itself through different

perspectives/modes of interaction and
this helped me build an embodied and
conceptual knowledge of the instrument’s
affordances and limitations. This process
focused on stability, balance and symmetry
between the body of the instrument and
my own.”

9
 playing in the traditional posture, the performer is required to support
the instrument. This means that, the instrument must be stable enough
to support the pressures exerted by the performer and allow freedom of
movement around the cello body.

The movements made by the performer and their resultant sound are
analysed in order to gain a better understanding of the limits of stability
and the connections between sound and gesture. The cello is a relatively
large instrument, this is advantageous as it provides a high-resolution
control of pitch, but the physical distance between the note positions on
the cello body require significant time for each transition. The instrument
also requires both hands in order to be played in the traditional manner,
with the control and articulation severely compromised when using only
one hand.

Processed Sound
The use of electronics frees the performer from the need to continuously
transfer energy into the instrument in order to produce sound. The use
of delays and sampling allows the creation of layers, which can be used to
create iterative transformations, build momentum and references in time,
which in turn manipulate the sense of direction in the music. This allows
the creation and manipulation of sound in multiple time domains and
playing modes.
10
Process Notes:
“Draw the sound you want to be able to make.
Drawing directly on the hand is a gesture similar
to a musical gesture in improvised music. There
is no room for hesitation and the result is both
surprising and well known. Abstract conceptions
of time can now be examined in the physical
domain, gaining perspective and developing

Andersen and Gibson

strategies.”

“The elasticity, malleability and structure of

interwoven rubber bands helped explore the ways
in which we could expand/translate this into the
digital/mappings. In this way, the models focus
on the experiential aspects of this interaction and
generate techniques for navigating the complexity
of the software through intuition and gesture.”

“The instrument integrates acoustic and digital

elements using a combination of audio analysis
and interrelated mappings. This allows the
software to create either complimentary or
contrasting reactions in the resultant sound
through the introduction of noise and probabilistic
functions.”

11
 Amplification allows us to magnify the input energy provided by the origins of its physical form. These simplified instruments allow us to twist,
performer and draw attention to the quieter sounds and textural details. bend and manipulate the different elements, exploring the affordances and
Similarly, compression and distortion add weight, impact, and density limitations of the shapes and materials.
to the sound. In comparison, the use of filtering allows for reductions to
be made in the spectrum of the sound, accentuating the tonal/textural Software and Hardware
differences that can already be made with the bow. Through this process
The new hardware interfaces function as physical representations for
the instrument is understood and approached as a platform or interface for
the state of the software, allowing the performer to navigate using visual,
exploring the combination, connections and interaction between acoustic
physical, and sonic feedback. In addition to this, the sound produced
and electronic sounds. by the cello is analysed by the software in an attempt to determine the
intentions of the performer and create complimentary and contrasting
By focusing on the transition between gestures and playing modes as well transformations to the resultant sound. In this way, the electronic
as the various ways the cello can be played as an object, removed from its parameters are influenced by interactions with the acoustic interface and
historical context, possible directions for modifications and extensions can create an interdependence between the two domains.
be imagined and developed. Similarly, through the exploration of different
sound making techniques and the possibilities of combining them in a The introduction of noise and probabilistic functions into this data stream
musically interesting way a new vocabulary of gestures can be developed. allows the performer to control the influence and stability/predictability
of the instrument’s sonic reactions. This miscommunication with the
Making it Malleable instrument can be used to manipulate the sense of continuity within the
music, surprising the performer and allowing for discovery and exploration
In order to identify the functionality and properties of the cello’s
of new sounds.
fundamental components, a series of explorative conceptual models are
created. These are simple string instruments that mimic aspects of size,
movement and material qualities of the cello. Through this process the
instrument becomes malleable and we gain a clearer understanding of the
12
Process Notes:

“The functionality of the software is transferred to

the cello body using sensors to provide precise and
direct manipulation of the software parameters. At
first we achieved this with cardboard representing the
unfinished state of the instrument and providing a sense
freedom when making modifications.”

Andersen and Gibson

“As a dialogue develops between me and the
instrument, extensions and transformations to the
sound and gestures can be imagined. In this way, the
act of playing remains the main source of momentum
and evaluation for the project and ensures an
integration between the new and the old vocabularies of
technique.”

“We focus on the subtle and detailed control over a

wide range of sounds made possible with the bow,
including adding extra strings to the cello and cutting
notches into the bow itself. By experimenting with
different materials for the strings, I became aware of
their different abilities and the subtleties of friction and
pressure that are involved in controlling the sound.”

13
 The Roles of the Hands abilities and imagining what could be learned. In this way, each parameter
evolves its own unique character, informed by its sonic qualities and the
Whilst making modifications our main concern remains the physical
physical properties and associations it conveys.
relationship between the cello and the performing body in order to remain
alert to issues of balance, counter-balance and symmetry. From this, the
placement and mapping of sensors are organised around the bimanual
What Remains
playing techniques of the cello and guitar (left side of body = pitch/ Throughout this process we have considered the instrument in the light
timbre, right side = duration/ volume). This is particularly effective, as it of a desire for make a new kind of music. At this point, each function and
compliments the characteristics of the cello whilst incorporating previously element has found it’s place on the modified body of the cello. The work
obtained instrument techniques. currently in progress is centered on the physical remaking of the cello
into a new unified body, where the new and old physical aspects of the
This can also be seen in the vertical placement of sliders, complimenting instruments are joined into a new whole.
the linear and vertical movements of the left hand, whilst providing a
visual representation of the software parameters for the musician. The In retrospect, the process can be described like this: The cello initially
placement of the sliders also acts as fret markers, facilitating orientation presents as a well known whole, a finished instrument. Through the
and accessibility with the fretting positions of the left hand. Furthermore, disassembling and adding of new powers and functions, it metaphorically
bow’able dials were added, expanding the capabilities of the bowing hand breaks apart and becomes fractured, and we get the opportunity to
and allowing for fast, unbroken transitions between controlling digital and consider it in its elements and units. Suddenly each string and fret has to
acoustic elements of the instrument. argue for its own inclusion, just as each new software component, we add
to it. As this process moves along, the cello is played and experimented
After each modification, time is spent playing the instrument in order to with continuously, but where the experiments fragments the cello, the
develop the technical skills necessary to determine the suitability of the playing acts as the driver to allow the instrument to come together again
changes. This process provides direction for the development of the new as a new whole. This push-pull process continues until the cello can re-
instrument by revealing what is physically possible, building on pre-existing emerge as a fundamentally changed instrument, stable in its new form and
14
 Process Notes:
“The position and mappings of the
additional sensors are influenced by the
bimanual techniques of the cello.”

“The vertical placement of sliders

compliments the vertical movements of

Andersen and Gibson

the left hand while playing. The sliders
also provide instant visual feedback
regarding the state of the software
parameters.”

“Similarly, the bow’able dial allows the

bow to be used to control the software
directly in a fluid and non-disruptive
manner.”

“The material of the old instrument begins to meet, foreshadowing the body of the new cello.”

“Through the iterative process of playing, reflection and modification, a new hybrid character
emerges from the combination of gestural and sonic possibilities. In this way, the act of playing
provided the momentum and direction for the development process of the instrument.”

15
 functionality. At this point it will become the main performing vehicle for Andersen, K. and Wilde, D., 2012. ‘Circles and Props - Making Unknown
the musician. Technology’. interactions, 2012/5/1, 60--65. ACM Press

Boal, A., 1992. Games For Actors and Non-Actors. Routledge, London
And then the performance itself becomes the test bed, where the
unexpected curves into your plans and you are forced to reconsider Caillois, R. Man, Play, Games. (M Barash, Trans.). Champaign: University
everything; from your premises to the gage of wire in your cables. (Ryan of Illinois Press, 1961.
and Andersen 2014)
Dewey, J., 1958. Art as Experience. New York. Putnam.

Acknowledgements Dunne, A., Raby, F., 2002. ‘The Placebo Project’. DIS 2002 London,
England. pp.9-12
The authors would like to acknowledge support from the GiantSteps
project, which has received funding from the European Community`s Gaver, W., Dunne, A., & Pacenti, E. 1999. ‘Cultural Probes’. Interactions
Seventh Framework Programme FP7/2007-2013 under grant agreement 6(1), 21–29.
no. 610591.
Gibson, J. J., 1977. ‘The theory of affordances’. In R. Shaw and J.
Bransford, editors, Perceiving, Acting, and Knowing: Toward an Ecological
All photos by Dan Buzzo.
Psychology, pages 67–82. Erlbaum, L., Hillsdale, NJ.

References Lillard, A. S. 1993. ‘Pretend play skills and the child’s theory of mind’. Child
development. 1993 Apr;64(2):348-71. Stanford University, CA.
Andersen, K., 2013.’Making Magic Machines in Proceedings of Crafting
the Future’. In: 10th European Academy of Design Conference. Göteborg. Ryan, J.; Andersen, K. 2014 (forthcoming). ‘821 words and 20 images’,
special edition, Neural Magazine.
Andersen, K., 2014. ‘The Deliberate Cargocult’. In Proc. of DIS2014, ACM
Press. Shklovsky, V., 1917. ‘Art as Technique’. In: Russian Formalist Criticism:
Four Essays, 1965. Trans. Lee T. Lemon and Marion J. Reis. University of
16
Nebraska Press, Lincoln.

Vavoula, G. N., Sharples, M., & Rudman, P. D., 2002. Developing the
‘Future Technology Workshop’ method. In Proceedings from IDC 2002,
ACM Press New York.

Andersen and Gibson

17