This article was downloaded by: [New York University]

On: 23 July 2015, At: 15:54

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: 5 Howick
Place, London, SW1P 1WG

Journal of New Music Research

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/nnmr20

The Importance of Parameter Mapping in Electronic

Instrument Design
Andy Hunt , Marcelo M. Wanderley & Matthew Paradis
Published online: 09 Aug 2010.

To cite this article: Andy Hunt , Marcelo M. Wanderley & Matthew Paradis (2003) The Importance of Parameter Mapping in
Electronic Instrument Design, Journal of New Music Research, 32:4, 429-440

To link to this article: http://dx.doi.org/10.1076/jnmr.32.4.429.18853

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of
the Content. Any opinions and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied
upon and should be independently verified with primary sources of information. Taylor and Francis shall
not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other
liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or
arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
 Journal of New Music Research 0929-8215/03/3204-429$16.00
2003, Vol. 32, No. 4, pp. 429–440 © Swets & Zeitlinger

The Importance of Parameter Mapping in Electronic

Instrument Design

Andy Hunt1, Marcelo M. Wanderley2 and Matthew Paradis3

1
Media Engineering Group, Electronics Department, University of York, Heslington, York, UK; 2Sound Processing and
Control Laboratory, Faculty of Music, McGill University, Montreal, Canada; 3Music Technology Group, Music Department,
University of York, Heslington, York, UK
Downloaded by [New York University] at 15:54 23 July 2015

Abstract
This paper presents a review of a series of experiments which is changed. Moreover, the emotional response elicited from
have contributed towards the understanding of the mapping the performer is shown to be determined to a great degree by
layer in electronic instruments. It challenges the assumption the mapping. Whereas the input devices establish the physi-
that an electronic instrument consists solely of an interface cality of the system and the synthesis methods govern the
and a sound generator. It emphasises the importance of the sound quality, the mapping somehow affects how the player
mapping between input parameters and sound parameters, reacts psychologically and musically to the instrument.
and suggests that this can define the very essence of an
instrument. The terms involved with mapping are defined,
and existing literature reviewed and summarised. A model 2. The importance of mapping
for understanding the design of such mapping strategies for
In this section we emphasise the dramatic effect that the style
electronic instruments is put forward, along with a roadmap
of mapping can have on “bringing an interface to life”. We
of ongoing research focussing on the testing and evaluation
focus on our own experience in designing digital musical
of such mapping strategies.
instruments and comment on several previous designs. An
extensive review of the available literature on mapping in
computer music has been presented by the authors (Hunt,
1. Introduction: Electronic instruments and 2000; Wanderley, 2001) and (Hunt & Wanderley, 2002). A
the mapping layer special issue of the journal Organised Sound has also been
In an acoustic instrument, the playing interface is inherently recently devoted to Mapping Strategies in Computer Music
bound up with the sound source. A violin’s string is both (Wanderley, 2002) and we refer the reader to it for a varied
part of the control mechanism and the sound generator. Since review of different approaches to mapping.
they are inseparable, the connections between the two are
complex, subtle and determined by physical laws. With elec-
2.1 Informal observations
tronic and computer instruments, the situation is dramatically
different. The interface is usually a completely separate piece The first author has carried out a number of experiments into
of equipment from the sound source. This means that the mapping. The more formal of these have been presented in
relationship between them has to be defined (see Fig. 1). detail (Hunt & Kirk, 2000; Hunt, 2000), and are summarised
The art of connecting these two, traditionally inseparable, later in this paper. We begin with some rather simpler, pre-
components of a real-time musical system (an art known as viously unpublished, observations that originally sparked
mapping) is not trivial. Indeed this paper hopes to stress that interest in this subject. We have retained the first person
by altering the mapping, even keeping the interface and writing style to denote that these are informal, personal
sound source constant, the entire character of the instrument reflections.

Accepted: 26 February, 2003

Correspondence: Andy Hunt, Media Engineering Group, Electronics Department, University of York, Heslington, York YO10 5DD,
United Kingdom. Tel.: +44 1904 432375; Fax: +44 1904 432335; E-mail: [email protected]
 430 Andy Hunt et al.

Amp

Osc

Freq

Fig. 2. Simple Mapping for Experiment 1.

Downloaded by [New York University] at 15:54 23 July 2015

Rate of
change Amp
Scale
Fig. 1. A generic model of the mapping problem. Osc
+
Invert Freq
2.1.1 The accidental theremin
Several years, ago I was invited to test out some final uni-
versity projects in their prototype form in the lab. One of
them was a recreation of a Theremin with modern electronic Fig. 3. Complex Mapping for Experiment 2.
circuitry. What was particularly unusual about this was that
a wiring mistake by a student meant that the “volume”
antenna only worked when your hand was moving. In other I let several test subjects play freely with the instrument,
words the sound was only heard when there was a rate-of- and talked to them afterwards. In the second experimental
change of position, rather than the traditional position-only run, the interface was re-configured to emulate the above-
control. It was unexpectedly exciting to play. The volume mentioned “accidental Theremin.” One slider needed to be
hand needed to keep moving back and forth, rather like moved in order to make sound; the rate of change of move-
bowing an invisible violin. I noted the effect that this had on ment controlled the oscillator’s amplitude. But I decided to
myself and the other impromptu players in the room. Because complicate matters (on purpose!) to study the effect that this
of the need to keep moving, it felt as if your own energy was had on the users. The pitch, which was mainly controlled by
directly responsible for the sound. When you stopped, it the first slider, operated “upside-down” to most people’s
stopped. The subtleties of the bowing movement gave a expectations (i.e., pushing the slider up lowered the pitch).
complex texture to the amplitude. We were “hooked.” It took In addition the second slider (being moved for amplitude
rather a long time to prise each person away from the instru- control) was used to mildly offset the pitch – i.e., it was cross-
ment, as it was so engaging. I returned a week later and noted coupled to the first slider (Fig. 3).
with disappointment that the “mistake” had been corrected, A remarkable consistency of reaction was noted over the
deleted from the student’s notes, and the traditional form of six volunteers who tried both configurations. With Experi-
the instrument implemented. ment 1, they all commented within seconds that they had dis-
covered how the instrument worked (almost like giving it a
mental “tick”; “yes, this is volume, and this is pitch”). They
2.1.2 Two sliders and two sound parameters
half-heartedly tried to play something for a maximum of two
The above observation caused me to think about the psy- minutes, before declaring that they had “finished.” Problem
chological effect on the human player of “engagement” with solved.
an instrument. To investigate this further, I constructed a With Experiment 2, again there was a noted consistency
simple experiment. The interface for this experiment con- of response. At first there were grumbles. “What on earth is
sisted of two sliders on a MIDI module, and the sound source this doing?” “Hey – this is affecting the pitch” (implied cries
was a single oscillator with amplitude and frequency con- of “unfair,” “foul play”). But they all struggled with it – inter-
trols. In the first run of the experiment the mapping was estingly for several more minutes than the total time they
simply one-to-one, i.e., one slider directly controlled the spent on Experiment 1. After a while, their bodies started to
volume, and the other directly controlled the pitch (Fig. 2). move, as they developed ways of offsetting one slider against
 Mapping in electronic instruments 431

Mouse cursor
On-screen on a blank
sliders: one screen
for each
parameter Sliders on a
MIDI fader Mouse
box
Mouse

Fig. 6. The “multi-parametric” interface.

Fig. 4. The “mouse” interface.

On-screen
sliders: one
for each
Downloaded by [New York University] at 15:54 23 July 2015

parameter
Score

Sliders on a
MIDI fader
box

Fig. 5. The “sliders” interface.

the other, while wobbling the first to shape the volume.

Nearly all the subjects noted that somehow this was reward- Practice
ing; it was “like an instrument.” Yet in both cases the inter- time Test complexity
face (two sliders) and the sound source (a single oscillator)
were identical. Only the mapping was altered, and this had a Fig. 7. Results for the “mouse” interface.
psychological effect on the players.
The third interface (Fig. 6) consisted of a series of multi-
3. Mapping experiments parametric cross-mappings, and, like the accidental Theremin
mentioned above, required constant movement from the user
Several formal investigations have been carried out by the
to produce sound.
authors in order to explore the essence and the effect of this
Users attempted to copy (using each interface) a series of
mysterious mapping layer.
sounds produced by the computer. The accuracy of repro-
duction was recorded for each user, over several attempts,
3.1 Complex mapping for arbitrary interfaces
spread out over a number of weeks. Results were gathered
The first author carried out an investigation into the psy- numerically, and plotted on a series of graphs to compare the
chology and practicality of various interfaces for real-time effect, over time, of each interface.
musical performance. The main part of this study took the The graphs are in the form of three-dimensional plots
form of a major series of experiments to determine the effect which summarise the results for the three interfaces, by aver-
that interface configuration had on the quality and accuracy aging together the responses from all users. What results is
of a human player’s performance. The full thesis is available a representation of the overall performance trend on each
for download online (Hunt, 2000) and the details of the interface– plotted over time, and against test complexity.
theory, experiments and results have been published (Hunt & The test scores are generated by a human marker (mod-
Kirk, 2000). They are summarised here, in order to give an erated by another marker) and are plotted on the vertical axis,
overview of their implications for mapping strategies. with a “perfect” score of 100% being at the top. The axis
Three interfaces were used, and these are now described. labelled Test Complexity represents the test number in
The first interface (Fig. 4) represented a typical computer increasing order of parameter complexity. The axis labelled
music editing interface with on-screen sliders connected Practice Time is the session number and thus represents
one-to-one to each sound parameter. increasing user contact time with the interface.
The second (Fig. 5) involved physical sliders (on a MIDI The test results from all the human test subjects
module) again connected in a one-to-one manner to the were averaged into the following three graphs (Figs. 7, 8,
synthesis unit. and 9).
 432 Andy Hunt et al.

• There is a rapid learning curve for the simpler tests which

means that at the final session the score is much higher
than for the mouse.
• The sliders perform slightly better than the mouse for the
more complex tests.
• The learning curve is only slight for the more complex
tests.
Score
This indicates that the “sliders” interface is more of a chal-
lenge than the “mouse” to start with, but a significant amount
of learning can take place for the simpler tests only.
Figure 9 shows the same type of plot for the multipara-
metric interface.
This shows a very different response. The angle of the
surface indicates that a very different experience occurred for
all users with the multiparametric interface. The following
Practice points of comparison with the previous two graphs are noted:
Downloaded by [New York University] at 15:54 23 July 2015

time Test complexity

• For the simplest test the scores are always lower than those
for the mouse or sliders, but they improve over time.
Fig. 8. Results for the “Sliders” interface.
• The scores get better for more complex tests and are much
higher than the other two interfaces.
• There is a good improvement over time across all test com-
plexities.
The upward tilt of the plane towards the far-right corner is
the most striking feature of the graph. It demonstrates that,
on average, the multiparametric interface allows the best per-
formance on the complex tests whilst also allowing a general
all-round improvement on tests of all complexities. Neither
Score
of the other interfaces had this characteristic.
Studies taken qualitatively (by interviewing each candi-
date after each test) show clearly that users actively enjoyed
this interface, and commented that it was “more like playing
an instrument” than operating a conventional computer
interface.
The above results can be summarised for the multipara-
metric interface as follows:
Practice • The test scores in general were much higher than those for
time Test complexity the other two interfaces, for all but the simplest tests.
• There was a good improvement over time across all test
Fig. 9. Results for the “Multiparametric” interface. complexities.
• The scores got better for more complex tests!
This last result may seem rather counter-intuitive at first
Figure 7 shows the first of the 3D plots. This graph encap-
sight; that people performed better on the harder tasks.
sulates all the data collected for the mouse interface during
However, this brings into question the definition of a “hard
the longitudinal tests. This graph shows that the mouse does
task.” If an interface allows the simultaneous control of many
indeed give a reasonably “flat” response over all the tests.
parameters, maybe it really is easier to perform the more
There are signs of a very small improvement over time. Note
complex tasks, and harder to accurately isolate individual
the small upward bend in the plane which indicates that the
parameters. A similar finding was presented by Jacob et al.
best scores are for the simplest few tests.
(1994), as explained in section 4 below.
Figure 8 shows a similar plot for the sliders interface. The
A range of qualitative results was also gathered by inter-
angle of the surface allows some immediate comparisons to
viewing the test subjects to establish their subjective experi-
be made with the mouse interface plot above.
ence of using each interface. They all concluded that the
• For the simplest tests the initial scores are lower for the “mouse” interface was the most limited – as they could see
sliders than for the mouse. how impossible it would be to operate more than one para-
 Mapping in electronic instruments 433

meter simultaneously. Surprisingly perhaps, they were users’ (subjective) views on the comparative expressivity
nearly all extremely frustrated by the four physical sliders. and learnability of each mapping, and the quality of musical
Comments abounded such as “I should be able to do this, control that could be achieved.
technically, but I can’t get my mind to split up the sound into Whilst users were experimenting with these instruments
these four finger controls.” Some users actually got quite an interesting trend was observed. The first (one-to-one) test
angry with the interface and with themselves. The multi- did not seem to hold the users’ attention. They treated it as a
parametric interface, on the other hand, was warmly received “task” to complete rather than an opportunity to experiment
– although not at the very beginning. At first it seemed a with a musical instrument. The second test received a more
difficult interface for most users, but they rapidly warmed favorable response. The third test seemed to inspire the users
to the fact that they could use complex gestural motions to to experiment and explore the interface to the point of com-
control several simultaneous parameters without having to posing and performing their own short melodies and musical
“de-code” them into individual streams. Many users re- gestures.
marked how “like an instrument” it was, or how “expressive” Users were asked to fill in a questionnaire relating to their
they felt they could be with it, but that it needed practice. feelings towards each experiment. The questionnaire asked
each user to score the instruments in different areas such as:
3.2 Focusing purely on the effect of mapping “How easy did you find it to control individual parameters?”
Downloaded by [New York University] at 15:54 23 July 2015

“How expressive could you be with the interface?”

In the above experiment several factors may have affected
“How did you feel your performance with the interface improved
the results. For instance, the multiparametric interface used
over time?”
cross-coupled parameters in addition to the user’s energy. It
also reduced reliance on visual feedback, and provided a two- Most of the users found the first interface to be very “dull,”
handed input, all of which may have contributed in varying or not very stimulating, and were not willing to play it for
degrees to the interface’s effectiveness. more than a few minutes before moving on to the next exper-
An additional experiment was subsequently carried out by iment. Users commented that the simple division of parame-
the third author, designed to focus purely on the mapping ters was not musically interesting.
layer of electronic musical instruments. Comments from the second user test suggested that the
These tests utilised three contrasting mapping strategies, incorporation of user energy presents a more engaging
with a fixed user interface (a MIDI slider box) and synthesis natural instrument, allowing the user to feel that they have
algorithm (a simple FM instrument). Users were asked to connected with the instrument. The simple division of para-
explore 3 different instruments for as much or as little time meters not longer seemed to be such an issue.
as they wished. In each phase of the experiment the interface The final test presented much more of a challenge to
looked exactly the same to the user, however the sonic the users, with some of them spending several minutes just
response to the user’s interaction was different in each case. trying to make it produce sound. As time progressed the users
The mapping employed for each phase was as follows: became more and more immersed in what they were doing
musically and forgot about the technical aspects of the exper-
1. A simple one-to-one mapping where input parameters
iment. The key to this interface’s appeal seemed to be that it
were directly linked to audio controls
felt more like a traditional instrument and promoted holistic
2. A one-to-one mapping as in instrument 1 but also requir-
thinking rather than forcing the users to analyze exactly what
ing the user to input energy to the system in order to make
each of the controls did. This allowed the users to step back
the instrument sound. This was implemented by requiring
from the technical issues relating to the instrument and con-
the user to constantly move one of the sliders in a bowing-
centrate on their musical output from the system.
like action. Users experienced with traditional instru-
ments would be familiar with this concept of having to
constantly inject energy to a system. 3.3 Learning from acoustic instruments
3. A many-to-many mapping in which parameters are cross
In terms of designing mapping strategies, much can be
coupled and modified to build complex relationships
gained from the analysis of the way existing acoustic instru-
between certain input gestures and audio output. This
ments work. Several acoustic instruments present complex
mapping also required the user’s energy as in 2.
relationships between physical control variables and the
These mappings allowed the user to control the parameters resulting sound. These relationships can serve as inspiration
of a stereo FM synthesis algorithm, including amplitude, fre- for the design of expressive new digital musical instruments.
quency, panning, modulation ratio and modulation index. Rovan et al. (1997), used a Yamaha WX7 wind controller
In each phase of the experiment, users were asked to play as the input device, and generated sound using additive syn-
with the interface until they felt they had a good sense of how thesis models of clarinet sounds in IRCAM’s FTS environ-
to “drive it” to perform musical gestures. No time limit was ment (later in jMax). The idea behind the project was simple;
given to this process; the users were encouraged to explore many wind instrument performers complained that MIDI
the possibilities of each set-up. Data was collected on the wind controllers tend to lack expressive potential when com-
 434 Andy Hunt et al.
Downloaded by [New York University] at 15:54 23 July 2015

Fig. 10. The three mappings used in the clarinet simulation presented in Rovan (1997).

pared to acoustic instruments such as the clarinet or saxo- pendently in the MIDI controller, whereas they are cross-
phone. A common path to solving this problem involves coupled in acoustic single-reed instruments. This natural
improving the design of the controller by adding extra cross-coupling is the result of the physical behaviour of the
sensors. Rovan et al. (1997) discussed the fact that by alter- reed, and variables were simply independent since the equiv-
ing the mapping layer in a digital musical instrument and alent “reed” in the controller was a plastic piece that did not
keeping the interface (an off-the-shelf MIDI controller) and vibrate, and moreover, was not coupled to an air column.
sound source unchanged, the essential quality of the instru- Based on these decisions and facts, the authors proposed
ment is changed regarding its control and expressive different mappings between the WX7 variables and the syn-
capabilities. Previous studies, notably by Buxton (1986a), thesis parameters. The first was basically a one-to-one rela-
presented evidence that input devices with similar character- tionship, where variables were independent. The second was
istics (e.g., number of degrees of freedom) could lead to very a model where the “virtual airflow” through the reed (loud-
different application situations depending on the way these ness) was a function of both the breath and lip pressure
characteristics were arranged in the device. In that study, (embouchure), as in an acoustic instrument. The third was a
however, the devices were not the same mechanically (one model that took into account both the “virtual airflow” and
had two separate one-dimensional controllers and the other the relationship between spectrum content to breath and
one two-dimensional controller), so the situation is different embouchure; a model that would match even more closely
to using exactly the same input devices with different the real behaviour of the acoustic instrument.
mapping strategies. The two most important consequences of this work were:
Another point became clear in this process; even if
the WX7 was a reasonably faithful model of a saxophone 1. By just changing the mapping layer between the controller
providing the same types of control variables (breath, lip and the synthesis algorithm, it was indeed possible to
pressure and fingering), these variables worked totally inde- completely change the instrumental behaviour and thus
 Mapping in electronic instruments 435

the instrument’s feel to the performer. Depending on the acting in a unified manner. It is also associated with feelings
performer’s previous experience and expectations, differ- of success and being uplifted – which certainly concurs with
ent mappings were preferred. the experiences of the users of the multiparametric interface.
2. By deconstructing the way that the reed actually works, it This contrasts greatly with the anger and frustration expe-
was noted that the choice of mapping could be important rienced by users of the parallel sliders interface. This may
as a pedagogical variable. Indeed, in stark contrast with come as a surprise to some, especially considering that such
acoustic instruments where the dependencies between banks of physical sliders are known the world over as the
parameters are unchangeable, cross-coupling between basis of the mixing desk. Fitzmaurice et al. (1997) conclude
variables can easily be created or destroyed in digital that slider banks (and other space-multiplexed graspable
musical instruments. This means that performers could interfaces) outperform time-multiplexed interfaces, such as
focus on specific aspects of the instrument by explicitly the mouse. Does our work contradict their findings? No,
defining its behaviour. but it does qualify them. One possible interpretation of
Fitzmaurice et al. is that we should abandon the mouse and
provide separate graspable interfaces, but our results show
this is not necessarily true. Where a mouse is used to time-
4. Analysis of experiments
multiplex (e.g., move sequentially between different on-
In this section we set the results of the above music-interface screen areas such as icons and menus) then it will be less
Downloaded by [New York University] at 15:54 23 July 2015

experiments in the context of work by other authors and from efficient than an interface where the same parameters are
mainstream Human Computer Interaction literature. physically graspable. Our work supports this, as for the sim-
The experiments in section 3.1 reveal results which are at plest tests – the physical sliders outperform the time-multi-
once challenging, and yet understandable. Experience with plexed “mouse” interface. However, in our final interface the
acoustic musical instruments, and their in-built complex mouse is used in a much more creative way – to provide con-
cross-couplings of control parameters, would tend to support tinuous multiparametric control. Users of the sliders inter-
our results that the complex, multi-parametric interface face had to mentally split down the holistic sound into its
performed significantly better. Our natural expectations of four sonic components before being able to derive the motor
having to practice hard to play an instrument are reflected in controls for each individual slider. Users were happy to do
the multiparametric interface’s slow start; it takes time to get this when only one parameter changed. For two simultane-
used to a complex mapping. However, it is notable that ous parameters it was much more difficult, but still just pos-
nobody questions the need to practice an instrument, yet in sible. With three and four parameters most people simply
the same breath everybody demands “easy-to-use” computer gave up, or tried until they were angry with frustration. This
interfaces. Much of the HCI literature concentrates on would seem to imply that space-multiplexed systems begin
making interfaces more direct, and less hard to learn. At first to fail when more than two parameters need to be changed
glance this would seem a laudable goal, even an obvious one. simultaneously.
But are we perhaps throwing out the proverbial baby with the Somehow the multiparametric interface is aiding the users
bathwater (in this case the bathwater represents interface to cope with handling simultaneous sonic variables. The
complexity and learning time, and the baby represents the work by Jacob et al. (1994) on Integrality and Separability
rewards of a flexible interface)? of input devices sheds light on how this may be happening.
So, why should a complex and cross-coupled interface Jacob’s work shows that an interface system work best where
completely outperform a more standard sliders-only one? the structure of the input devices matches the perceptual
Most users of the multiparametric interface describe a structure of the task being performed by the user. In other
moment when they “stopped thinking” about the interface words it is not just about which device is used (e.g., a mouse)
and began to “just play” with it – almost as if their body was but how that device is mapped onto the system parameters,
doing it for them, and their conscious mind was somehow and how well that helps the users to think about the task. In
“not in control.” This is remarkably close to what acoustic our case the users were hearing a holistic sound, yet the
musicians describe as the “flow” experience. In a fascinating sliders interface forced them to decode the sound into four
web-site for musicians, violinist and psychologist A. Burzik separate control streams – a very hard and frustrating task.
(2002) describes the four principles of practicing in flow as: In contrast the multiparametric interface allowed users to
think spatially about the simultaneous control of pitch,
1. A special physical contact with the instrument;
timbre and volume. Users reported listening to a sound and
2. The development of a subtle feeling for sound;
“just knowing what shape that was” in terms of their required
3. A feeling of effortlessness in the body;
movements on the interface.
4. A playful and free-spirited handling of the material
A further contribution to the success of the multipara-
studied.
metric interface could be the way in which the user’s two
“Flow” is the term coined by M. Csikszentmihalyi (1975) hands are used. Kabbash et al. (1994) describe how a well-
to describe a holistic state of consciousness where actions designed bimanual input can greatly outperform a single-
happen continuously, accompanied by sharpened senses handed control, but one hand is usually the dominant control
 436 Andy Hunt et al.

and the other more of a modifier (Guiard, 1987). This is

exactly the situation in the multiparametric interface, where
the main timbral parameters are all available to the right
hand, and the left hand controls sliders which offset them and
control the panning. Kabbash et al. also describe the converse
situation, where an inappropriately designed two-handed
interface can be significantly worse than one hand. This is
what we believe was happening with the physical sliders
interface. It also seems that the structure of the multipara-
metric interface was allowing the user to break down the
sound into more useable sized “chunks” (Simon, 1996;
Buxton, 1986b) of information, whereas the task of breaking
down the sounds into a form usable by the physical sliders
interface was just too much for the users.
Finally, there is the result common to all of the above
experiments that the users report a better response when con-
tinuous input of energy is required from them. Again, we
Downloaded by [New York University] at 15:54 23 July 2015

should perhaps expect this from our experience with acoustic

musical instruments, but this is often ignored because com-
puters are self-powered devices which do not require our
energy to operate. This point has been studied by C. Cadoz,
who proposed the notion of an energy continuum between Fig. 11. Two-layer mapping model showing intermediate layer of
performer actions and the sound being produced in most abstract parameters.
acoustic instruments (Cadoz, 1999). In fact, part of the mus-
cular energy spent by the performer is transformed into
acoustic energy in the resulting sound through excitation ges- models that can facilitate mappings other than simple one-
tures. This is known as the ergotic function of the gestural to-one relationships.
channel (Cadoz & Wanderley, 2000). In trying to answer this question of how to extend a spe-
We seem to be discovering that we need to use our energy cific mapping solution to a more general case, a model of
to operate effectively. The response from nearly every user mapping for digital musical instruments was proposed in
in all of the above experiments was that the interface per- (Wanderley et al., 1998). It was based on the separation of
ceptually transformed into an instrument when the user’s the mapping layer into two independent layers, coupled by
continuous energy was required. Maybe this is another an intermediate set of user-defined (or “abstract”) parame-
requirement of “flow”? – that we need to continuously put ters (see Fig. 11).
energy into a system, and to steer its flow through the system. This model was presented in the framework of a set of
Returning to the “accidental theremin” of 2.1.1, once the extensions to jMax later known as ESCHER (actually, a set
energy requirement was removed and the system played of objects developed at Ircam to perform interpolation using
continuous sound without movement, the users who were additive models).
so engaged, lost interest rapidly. Could it be that there is an This idea is based on previous works, such as those
in-built requirement of human beings to use their own ener- of Mulder et al. (1997a), Métois (1996), and Wessel (1979).
getic control to transmit that energy into the world around A similar direction was presented by Mulder et al.
them? (1997b), Mulder and Fels (1998), and later by Garnett and
Taking the above thoughts, it is important to think about Goudeseune (1999), and Goudeseune (2002). Basically, all
how to design mappings that will engage the user, and also be these works have used higher levels of abstraction as control
perceptible by an audience. In the following section we review structures instead of raw synthesis variables such as ampli-
the work done on mapping and contribute to the debate about tudes, frequencies and phases of sinusoidal sound partials.
how to design creative instruments for live performance. The main point made in (Wanderley et al., 1998) was to
explicitly think about two separate mapping layers and the
strategies to implement these, and not on the choice of
intermediate parameters themselves, whether perceptive,
5. Models and guidelines for mapping geometrical or “abstract” (Wanderley & Depalle, 1999).
Since there will not always be ready models for inspiration The intrinsic advantage of this model is its flexibility.
when designing new digital musical instruments, the design Indeed, for the same set of intermediate parameters and syn-
of mapping strategies becomes more complex. We therefore thesis variables, the second mapping layer is independent of
need to propose guidelines for mapping strategies and devise the choice of controller being used. The same would be true
 Mapping in electronic instruments 437

in the other sense: for the same controller and the same set
of parameters, multiple synthesis techniques could be used
by just adapting the second mapping layer, the first being Controller Parameters
held constant. Specifically in this case, the choice of synthe-
sis algorithm is transparent for the user.
The two-layered model has been expanded to include
three mapping layers (Hunt & Wanderley, 2002; Arfib et al.,
2002). Figure 12 depicts a model containing these three
mapping layers.
These works support the idea that, by using multi-
layered mappings, one can obtain a level of flexibility in Meaningful Parameters
the design of instruments and that moreover, these models
can indeed accommodate the control of different media, such
as sound and video, in a coherent way. Section 5.1 describes
in more detail the justification for a multi-layer mapping
model.
Downloaded by [New York University] at 15:54 23 July 2015

5.1 One-to-one mappings using multiple layers Abstract Parameters

We have noted that there is a tendency for designers to make
one-to-one mappings when constructing an interface. We can
use this tendency to improve the mapping process if we
utilise the many layered models outlined above. The follow-
ing scenario may illustrate this:
Imagine a system whose interface inputs included “button
1,” “button 2,” “slider 1,” “slider 2,” “mouse x” and “mouse
Synthesis Parameters
y.” Let us suppose that the synthesis system was a Frequency
Modulation module with inputs such as “carrier frequency,”
“carrier amplitude,” “modulation frequency” etc. Now con-
sider the two possibilities below: Fig. 12. Three-layer mapping model.
Case 1: let us consider a designer working to connect the
above inputs to the above outputs. We are quite likely to see
arbitrary connections such as “mouse x controls carrier
frequency,” and “slider 1 controls modulation frequency”
6. Towards designing, testing and evaluating
(similar to those shown in Fig. 1). These give us the often-
mapping strategies
encountered one-to-one mappings. The experiments outlined above have proven the importance
Case 2: let us imagine that a mapping layer has already of the mapping layer in electronic musical instruments.
been devised to abstract the inputs to parameters such as However the question remains, how do we efficiently and
“energy,” “distance between sliders,” “wobble” etc. Also let effectively design, test and evaluate a mapping strategy?
us imagine that there is a mapping layer before the FM syn- The development of mapping strategies for “known inter-
thesis unit, providing higher-level control inputs such as faces” (those which inherit their properties from existing
“brightness,” “pitch,” “sharpness” etc. This gives us a situa- interfaces) has a head start as there are plenty of successful
tion similar to that found in Figure 12. Now we can picture examples already widely used in the form of traditional
the designer making a relationship such as “energy controls musical instruments. However, fewer examples exist for
brightness.” On the surface this may appear to be yet another novel interfaces – newer designs that have no traditional
one-to-one mapping. Indeed it is – at the conceptual level. counterparts.
However, when you consider how “energy” is calculated Novel sensor based interfaces, MIDI-slider boxes and
from the given inputs, and how “brightness” has to be con- graphics tablets all have the potential to be employed as very
verted into the FM synthesis primitives, you will notice how expressive musical devices, as long as the mapping is imple-
many of the lower-level parameters have been cross-coupled. mented correctly.
Thus the many-level mapping models are a way of sim- The third author is carrying out ongoing research in an
plifying the design process, and of helping the designer to attempt to further our knowledge of the mapping layer. The
focus on the final effect of the mapping, as well as provid- goal of this research is to develop techniques and models
ing a convenient method of substituting input device or syn- which allow researchers to evaluate the performance of a
thesis method. given mapping strategy, and to provide focussed environ-
 438 Andy Hunt et al.

ments for the construction of such mappings by answering collection and analysis of data over time relating to the accu-
the following questions. racy, speed, expressive control and suitability of interfaces
for specific tasks.
How do we define the mapping layer?
How can we measure the performance of a
In order to fully understand the workings of the mapping
mapping strategy?
layer it must firstly be fully defined. This will give researchers
a clear structure and vocabulary with which to refer to ele- How can the subjective data taken from a user’s experience
ments of a system. Current research concentrates on the of an interface be represented with actual performance data
mapping layer as a high level entity. Strategies are defined (data that can be recorded whilst a user is carrying out a spe-
(one-to-one etc.) along with layers which can carry out spe- cific task)? Can trends be observed in objective data to rep-
cific functions. It is proposed that a mapping layer should be resent Expressivity, or the enjoyment that a user experiences
studied at a much lower level in order to fully understand the in their interaction?
implications of a given parameter configuration. In all interface design the end result is ultimately a user
experience. Mapping strategies can provide a tailored expe-
rience dedicated to the task in hand, rather than using a global
What effect does a specific treatment of parameters
parameter association as can often be found with devices
Downloaded by [New York University] at 15:54 23 July 2015

have on the user experience?

such as the mouse. Simply trying to extend the position of a
It is important for interface designers to understand how user’s hand into a cursor on the computer screen is neglect-
specific operations affect the overall user experience that a ing a great deal of the sophisticated interaction that users are
system offers. For example (in a musical context) is there capable of in an attempt to provide a global solution for all
a different psychological effect from adding parameters computing tasks.
together rather than averaging parameters? One would expect
that this would create a very different interface, but which
method is best for the task in hand? 7. Future discussion of mapping
At a lower level, a mapping layer can be examined in great
From the evidence presented above in both informal and
detail. As with any complex system certain points in the layer
controlled experiments, there is definitely a need to find
will contribute to the overall output of the system in differ-
better-designed mappings than simple (engineering style)
ent ways. It is proposed that these “areas of activity” can be
one-to-one relationships. General models of mappings have
thought of as mapping nodes which are examined and clas-
been proposed and expanded to incorporate multimedia
sified according to their function within the network of para-
control, but also to fit several levels of performance, from
meter relationships. At this level of detail it becomes possible
beginners to highly skilled players.
to use specific mapping operations such as addition, division,
One attempt to foster the discussion in this direction has
averaging, rate of change or acceleration (to name a few) to
been initiated in the context of the ICMA/EMF Working
build up nodes of parameter modifications which can influ-
Group on Interactive Systems and Instrument Design in
ence the interaction model in different ways.
Music (Wanderley, 2000). Readers may also wish to refer to
the special issue on “Mapping Strategies for Real-time Com-
How can a system be designed to give the appropriate puter Music” guest-edited by the second author (Wanderley,
balance of accuracy in real time control and expressive 2002).
intuitive interaction? We therefore welcome comments and criticism on issues
related to mapping so as to push the discussion on this essen-
Different interaction tasks require different interaction
tial, although often ignored, topic.
models. A task such as entering text into a word processor
requires a very accurate interface, but it is not concerned with
creative expression. Inaccuracies cannot be tolerated as they
instantly render the output from the system unusable. A
8. Conclusions
musical instrument however may not require the same degree The mapping “layer” has never needed to be addressed
of accuracy in order to create usable output. Examples of this directly before, as it has been inherently present in acoustic
can be found most traditional instruments. Players have to instruments courtesy of natural physical phenomena. Now
rely on skills built up over many years of practice before they that we have the ability to design instruments with separable
can control an instrument accurately and this in turn pro- controllers and sound sources, we need to explicitly design
motes a greater understanding of the interface by the user. the connection between the two. This is turning out to be a
Recent informal experiments indicate that accurate interfaces complex task.
do not tend to promote expressive output from a user and This paper has indicated that mapping can define the very
vice-versa. In order to understand this relationship, a set of essence of an instrument and that complex mappings can
tests has been proposed as ongoing research to enable the provide quantifiable performance benefits and improved
 Mapping in electronic instruments 439

interface effectiveness. We experimented with a multi-layer Hunt, A. (2000). Radical user interfaces for real-time musical
model to help designers to implement complex but usable control. DPhil thesis, University of York, UK. Available:
mappings, and are carrying out a programme of research to http://www-users.york.ac.uk/~elec18/download/adh_thesis/
look into the testing and evaluation of mapping strategies. Hunt, A., & Wanderley, M.M. (Eds.) (2000). Mapping of
We are still in the early stages of understanding the com- control variables to musical variables. Interactive systems
plexities of how the mapping layer affects the perception (and and instrument design in music working group. Website:
the playability) of an electronic instrument by its performer. www.igmusic.org
What we know is that it is a very important layer, and Hunt, A., & Kirk, R. (2000). Mapping strategies for musical per-
one that must not be overlooked by the designers of new formance. In: M. Wanderley, & M. Battier (Eds.), Trends in
instruments. Gestural Control of Music. IRCAM – Centre Pompidou.
Hunt, A., Wanderley, M.M., & Kirk, R. (2000). Towards a model
for instrumental mapping in expert musical interaction. In
Proc. of the 2000 International Computer Music Conference.
Acknowledgements San Francisco, CA: International Computer Music Associa-
Many thanks are due to the following people for their tion, pp. 209–211.
collaboration in the work described in this paper: Ross Hunt, A., & Wanderley, M.M. (2002). Mapping performance
Kirk, Butch Rovan, Shlomo Dubnov, Philippe Depalle, parameters to synthesis engines. Organised Sound, 7,
Downloaded by [New York University] at 15:54 23 July 2015

and Norbert Schnell. 103–114.

Garnett, G., & Goudeseune, C. (1999). Performance factors in
control of high-dimensional spaces. Proc. 1999 International
Computer Music Conference. San Francisco, CA: Interna-
References tional Computer Music Association, pp. 268–271.
Arfib, D., Couturier J., Kessous L., & Verfaille V. (2002). Strate- Goudeseune, C. (2002). Interpolated mappings for musical
gies of mapping between gesture data and synthesis model instruments, Organised Sound, 7, 85–96.
parameters using perceptual spaces, Organised Sound, 7, Jacob, R., Sibert, L., McFarlane, D., & Mullen, M. (1994). Inte-
127–144. grality and separability of input devices. ACM Transactions
Burzik, A. (2002). A Holistic Method of Practice for all Instru- on Computer-Human Interaction, 1, 3–26.
mentalists. Available: http://www.practising-in-flow.de/ Last Kabbash, P., Buxton, B., & Sellen, A. (1994). Two handed input
visited on 10/04/03. in a compound task. Proc SIGCHI Conf. Human Factors in
Buxton, W. (1986a). There’s more interaction than meets the eye: computing systems, pp. 417–423.
Some issues in manual input. In: D. Norman, & S.W. Draper Métois, E. (1996). Musical Sound information: Musical gestures
(Eds.), User Centered System Design: New Perspectives on and embedding systems. PhD Thesis. MIT Media Lab.
Human-Computer Interaction. Hillsdale, N.J.: Lawrence Mulder, A., Fels, S., & Mase, K. (1997). Empty-handed gesture
Erlbaum Associates, pp. 319–337. analysis in Max/FTS. In Kansei, The Technology of Emotion.
Buxton, W. (1986b). Chuncking and phrasing and the design of Proceedings of the AIMI International Workshop, A. Camurri
human-computer dialogues. In: H.J. Kugler (Ed.), Informa- (Ed.), Genoa: Associazione di Informatica Musicale Italiana,
tion Processing ’86, Proceedings of the IFIP 10th World October 3–4, pp. 87–91.
Computer Congress, North Holland, pp. 475–480. Mulder, A., Fels, S., & Mase, K. (1997). Mapping virtual object
Cadoz, C. (1999) Continuum energetique du geste au son. Sim- manipulation to sound variation. In: T. Rai, & R. Basset
ulation multisensorielle interactive d’objets physiques. In: (Eds.), IPSJSIG notes, 97, 63–68.
H. Vinet, & F. Delalande (Eds.), Interfaces Homme-Machine Mulder, A., & Fels, S. (1998). Sound sculpting: Manipulating
et Création Musicale. Paris: Hermès Science Publishing, sound through virtual sculpting. Proc. 1998 Western Com-
pp. 165–181. puter Graphics Symposium, pp. 15–23.
Cadoz, C., & Wanderley, M. (2000) Gesture-Music. In: M. Rovan, J.B., Wanderley, M.M., Dubnov, S., & Depalle, P. (1997).
Wanderley, & M. Battier (Eds.), Trends in Gestural Control Instrumental gestural mapping strategies as expressivity
of Music. Paris: Ircam – Centre Pompidou, pp. 71–93. determinants in computer music performance. In Kansei,
Csikszentmihalyi, M. (1975). Beyond boredom and anxiety: The Technology of Emotion. Proceedings of the AIMI
Experiencing flow in work and play. (reprint, Jossey Bass International Workshop, A. Camurri (Ed.), Genoa: Associ-
Wiley; 2000 ISBN: 0787951404). azione di Informatica Musicale Italiana, October 3–4, pp.
Fitzmaurice, G., & Buxton, B. (1997). An empirical evalua- 68–73.
tion of graspable user interfaces: Towards specialized, space- Simon, H. (1996). The Sciences of the artificial. MIT Press; 3rd
multiplexed input. Proc. ACHCHI 1997 pp. 43–50. edition, ISBN 0262691914.
http://www.acm.org/sigchi/chi97/proceedings/paper/gf.htm Wanderley, M.M., Schnell, N., & Rovan, J. (1998). Escher –
Last visited on 10/04/03. Modeling and performing “composed instruments” in
Guiard, Y. (1987) Assymetric division of labor in human skilled real-time. Proc. IEEE International Conference on Systems,
bimanual action: The kinematic chain as a model. Journal of Man and Cybernetics (SMC’98), San Diego, CA, pp.
Motor Behaviour, 19, 486–517. 1080–1084.
 440 Andy Hunt et al.

Wanderley, M.M., & Depalle, P. (1999). Contrôle gestuel de la Wanderley, M.M. (2001). Performer-instrument interaction.
synthèse sonore. In: H. Vinet, & F. Delalande (Eds.), Inter- Application to gestural control of sound synthesis. PhD
faces Homme-Machine et Creation Musicale – Hermes Thesis. University Paris VI, France.
Science Publishing, pp. 145–163. Wanderley, M.M. (Ed.) (2002). Mapping strategies for real-time
Wanderley, M.M. (Ed.) (2000). Interactive systems and instru- computer music. Organised Sound, 7.
ment design in music workgroup. Website: www.igmusic.org Wessel, D. (1979). Timbre space as a musical control structure.
Last visited on 10/04/03. Computer Music Journal, 3, 45–52.
Downloaded by [New York University] at 15:54 23 July 2015