School of Music & Sonic Arts

Faculty of Arts, Humanities and Social Sciences

Queens University Belfast
Thesis submission for the degree PhD in Music (Music Technology)
Composing Music by Composing Rules:
Design and Usage of a Generic Music Constraint System
Torsten Anders
13th February 2007
Abstract
This research presents the design, usage, and evaluation of a highly generic music con-
straint system called Strasheela. Strasheela simplies the denition of musical constraint
satisfaction problems (CSP) by predening building blocks required for such problems.
At the same time, Strasheela preserves a high degree of generality and is reasonably
ecient.
Strasheela is highly generic, because it is highly programmable. In particular, three
fundamental components are more programmable in Strasheela compared with existing
music constraint systems: the music representation, the rule application mechanism, and
the search process.
Strasheela features an expressive symbolic music representation. This representation sup-
ports explicitly storing score information by sets of score objects, their attributes, and
their hierarchic nesting. Any information available in the score is accessible from any
object in the score and can be used to obtain derived information. The representation
is complemented by the notion of variables: score information can be unknown and such
information can be constrained.
This research proposes a rule formalism which combines convenience and full user con-
trol to express which score variable sets are constrained by a given rule. A rule is a
rst-class function. A rule application mechanism is a higher-order function. A rule ap-
plication mechanism traverses the score in order to apply a given rule to variable sets.
This text presents rule application mechanisms suitable for a large set of musical CSPs
and reproduces important mechanisms of existing systems.
Strasheela is founded on a constraint programming model which makes the search process
programmable at a high-level. The Strasheela user can optimise the search for a partic-
ular constraint satisfaction problem by programming a distribution strategy (a dynamic
variable and value ordering) independent of the problem denition. Special distribution
strategies for eciently solving various musical CSPs including complex polyphonic
problems are presented.
iii
iv
Acknowledgements
First and foremost I want to thank my supervisors, Michael Alcorn and Christina Anag-
nostopoulou, for their guidance and support throughout this research study. I am grateful
to Michael for giving me the opportunity to work at the Sonic Arts Research Centre.
It was a privilege and fun to work in a place with such a stimulating atmosphere. I am
indebted to Christina who helped me to clarify the views presented in this thesis.
I am grateful to many people with whom I have discussed by PhD work: Stefan Bilbao,
Ludger Br ummer, Darrel Conklin, Mikael Laurson, Camilo Rueda,

Orjan Sandred and
Alan Smaill. I have greatly beneted from their experience and constructive criticism.
Kilian Sprotte was the rst who dared to actually use Strasheela.
I would like to thank the Oz community. Denys Duchier answered countless questions I
had when I began with this research. Christian Schulte and Raphael Collet helped me
to better understand Oz constraint programming model. Many more members of the
Oz mailing list were always willing to help.
I was very happy being surrounded by so many friendly and helpful fellow PhD stu-
dents: Tom Davis, David Fee, Jason Geistweidt, Katarzyna Glowicka, Banu G unel, Huss
Hachabiboglu, Rachel Holstead, Ravi Kuber, Christopher McClelland, Erdem Motuk,
Emma Murphy, Damian Ryan, Philip Strain, Evren Tekin, and Henry Vega. I thank the
Queens University for funding my PhD research.
For detailed feedback on drafts of this work I am grateful to Christina Anagnostopoulou,
Eoin Brazil, Cornelius Popel, and Chris Share. Chris Share also answered many questions
I had on English language usage.
Finally, I would like to thank my wife Mechthild for all her support throughout these
years.
v
vi
Contents
1. Introduction 1
1.1. Computer-Aided Composition . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2. Constraint-Based Computer-Aided Composition . . . . . . . . . . . . . . 2
1.3. Requirements for a Generic Music Constraint System . . . . . . . . . . . . 4
1.4. The Approach Taken . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5. Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2. Survey I: Issues in Music Representation for Computer-Aided Composition 11
2.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.1. What is a Music Representation? . . . . . . . . . . . . . . . . . . . 12
2.1.2. The Broader Perspective . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2. A Basic Representation: Event List . . . . . . . . . . . . . . . . . . . . . . 14
2.3. Parameter Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.4. Topologies of Hierarchic Representations . . . . . . . . . . . . . . . . . . . 15
2.4.1. Two-Dimensional Representation . . . . . . . . . . . . . . . . . . . 17
2.4.2. Representation with Specialised Containers in a Tree . . . . . . . . 18
2.4.3. Representation with Generic Containers in a Tree . . . . . . . . . 19
2.4.4. Representation with Containers in an Acyclic Graph . . . . . . . . 20
2.4.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.5. Extensibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.6. Textual Representation vs. Data Abstraction . . . . . . . . . . . . . . . . 24
2.7. Class Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.8. Higher-Order Programming for Score Processing . . . . . . . . . . . . . . 30
2.9. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3. Survey II: Composing with Constraint Programming 35
3.1. What is Constraint Programming? . . . . . . . . . . . . . . . . . . . . . . 35
3.2. Constraint-Based Composition . . . . . . . . . . . . . . . . . . . . . . . . 37
3.2.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.2.2. A First Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2.3. Musical Constraint Satisfaction Problems . . . . . . . . . . . . . . 41
3.3. Generic Music Constraint Systems . . . . . . . . . . . . . . . . . . . . . . 46
3.3.1. PWConstraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.3.2. Situation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3.3. OMClouds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
vii
Contents
3.3.4. Other Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.4. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4. Motivation and System Overview 65
4.1. Limited Generality of Existing Systems . . . . . . . . . . . . . . . . . . . 65
4.1.1. Music Representation . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.1.2. Rule Formalism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.1.3. Search Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2. The Research Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.3. Strasheela Design Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.3.1. Search Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.3.2. Rule Formalism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3.3. Music Representation . . . . . . . . . . . . . . . . . . . . . . . . . 76
5. The Strasheela Music Representation 79
5.1. Overview: Score Object Classes . . . . . . . . . . . . . . . . . . . . . . . . 79
5.2. Class Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.3. Textual Representation and Data Abstraction . . . . . . . . . . . . . . . . 83
5.4. Hierarchic Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.4.1. The Parameter Class: Variables in the Representation . . . . . . . 85
5.4.2. The Event Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.4.3. The Container Class . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.4.4. Temporal Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.5. The Data Abstraction Interface . . . . . . . . . . . . . . . . . . . . . . . . 97
5.5.1. Basic Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.5.2. Generic Interface for Hierarchic Structures . . . . . . . . . . . . . . 99
5.6. Score Contexts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
5.6.1. Principles for Expressing Score Information . . . . . . . . . . . . . 102
5.6.2. User-Dened Score Contexts . . . . . . . . . . . . . . . . . . . . . 104
5.6.3. The Generality of the Strasheela Score Context Formalism . . . . 108
6. The Strasheela Rule Formalism 111
6.1. Constraints and Rules are First-Class Functions . . . . . . . . . . . . . . . 112
6.2. Programmable Rule Application . . . . . . . . . . . . . . . . . . . . . . . 113
6.2.1. Direct Rule Application to a Single Object . . . . . . . . . . . . . 114
6.2.2. Applying a Rule to Every Element in a List . . . . . . . . . . . . . 114
6.2.3. Applying a Rule to Neighbours in a List . . . . . . . . . . . . . . . 115
6.2.4. User-Dened Rule Application Mechanisms . . . . . . . . . . . . . 115
6.2.5. Index-Based Rule Application . . . . . . . . . . . . . . . . . . . . . 116
6.2.6. Rule Application with a Pattern Matching Language . . . . . . . . 117
6.2.7. Rule Application to Selected Objects in a Score Hierarchy . . . . . 118
6.2.8. Implicit Rule Application . . . . . . . . . . . . . . . . . . . . . . . 120
6.2.9. Rule Application to Arbitrary Score Contexts . . . . . . . . . . . . 121
6.3. Constraining Inaccessible Score Contexts . . . . . . . . . . . . . . . . . . . 122
viii
Contents
6.3.1. The Inaccessible Score Context Problem . . . . . . . . . . . . . . . 122
6.3.2. Delayed Rule Application . . . . . . . . . . . . . . . . . . . . . . . 123
6.3.3. Reformulating the Problem Denition . . . . . . . . . . . . . . . . 124
6.3.4. Using Logical Connectives . . . . . . . . . . . . . . . . . . . . . . . 125
6.3.5. Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7. How Strasheela Finds a Solution 129
7.1. The Underlying Programming Model . . . . . . . . . . . . . . . . . . . . . 129
7.2. The Constraint Model Based on Computational Spaces . . . . . . . . . . 130
7.2.1. Propagate and Search . . . . . . . . . . . . . . . . . . . . . . . . . 131
7.2.2. An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
7.2.3. Features of the Space-Based Constraint Model . . . . . . . . . . . 135
7.3. Specialising the Constraint Model for Music . . . . . . . . . . . . . . . . . 144
7.3.1. The Basic Idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
7.3.2. Score Distribution Strategies . . . . . . . . . . . . . . . . . . . . . 147
8. Strasheela Examples 155
8.1. Fuxian First Species Counterpoint . . . . . . . . . . . . . . . . . . . . . . 156
8.1.1. The Music Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
8.1.2. The Formal Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
8.1.3. Search Process and Results . . . . . . . . . . . . . . . . . . . . . . 163
8.1.4. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
8.2. Florid Counterpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
8.2.1. The Music Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
8.2.2. Search Process and Results . . . . . . . . . . . . . . . . . . . . . . 169
8.3. Constraining the Shape of the Temporal Hierarchy . . . . . . . . . . . . . 171
8.3.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
8.3.2. Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
8.3.3. Elimination of Symmetries . . . . . . . . . . . . . . . . . . . . . . 172
8.3.4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
9. The Strasheela Prototype 177
9.1. Relation to the Model Presented . . . . . . . . . . . . . . . . . . . . . . . 177
9.1.1. Input and Output . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
9.1.2. Extended Strasheela Core Music Representation . . . . . . . . . . 178
9.1.3. Extensions for Specic CSP Classes . . . . . . . . . . . . . . . . . 178
9.2. Programming Language Issues . . . . . . . . . . . . . . . . . . . . . . . . 182
9.3. Limitations of the Prototype . . . . . . . . . . . . . . . . . . . . . . . . . 183
10. Comparison and Evaluation 187
10.1. Music Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
10.1.1. Representation Format . . . . . . . . . . . . . . . . . . . . . . . . . 189
10.1.2. Constrainable Aspects . . . . . . . . . . . . . . . . . . . . . . . . . 194
10.2. Rule Formalism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
ix
Contents
10.3. Search Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
10.4. Can Strasheela be Further Generalised? . . . . . . . . . . . . . . . . . . . 200
10.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
11. Conclusion 203
11.1. Main Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
11.2. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
A. Notational Conventions 209
A.1. Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
A.2. Functional Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
A.3. Control Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
A.4. Values and Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . 211
A.5. Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
B. Additional Denitions 213
C. The Strasheela Website 217
D. Source Material 219
x
1. Introduction
1.1. Computer-Aided Composition
Computers nowadays play an important role in many areas of music composition and
its production. In particular, sound synthesis and eects processing plays an important
role. By contrast, computer-aided composition utilises the computer in order to create
symbolic scores. It focuses on higher-level musical concepts such as notes, chords, motifs,
or the musical texture.
In the eld of computer-aided composition (CAC, also known as algorithmic composi-
tion
1
) composers formalise their musical intentions and implement these formal speci-
cations as computer programs. These programs output music which the composer then
uses in the composition process. For writing such programs, composers use either general-
purpose programming languages or special composition systems with programming sup-
port. Particularly widely-used composition systems include PatchWork [Laurson and
Duthen, 1989; Laurson, 1996], the two PatchWork successors OpenMusic [Assayag and
Agon, 1996; Assayag et al., 1999] and PWGL [Laurson and Kuuskankare, 2002], and
Common Music [Taube, 1991, 2004]. Also the sound synthesis systems SuperCollider
[McCartney, 1996, 2002] and Max [Puckette, 1991, 2002] (including its relatives jMax
and Pure Data) support CAC very well. Ariza [2006] provides an extensive list of com-
position systems.
A rich set of composition techniques have been proposed, including mathematical mod-
els, models based on transforming existing data, and models which implement already-
existing compositional strategies. Examples of mathematical models are stochastic mod-
els [Xenakis, 1992] or self-similar systems [Supper, 2001]. The compositional work of
Gerard Grisey or Tristan Murail exemplify transformations of spectral analysis data
into instrumental music. Koenigs software Project One and Project Two implemented
concepts which stem from serial and chance composition [Laske, 1981].
Practitioners of computer-aided composition use various ways of working. In a prag-
matic approach which is often applied, the composer delegates certain subtasks of the
composition process to some software. The software takes over tedious subtasks, but
the composer never surrenders control over the composition process itself. For example,
the composer will often manually edit the output of the software and integrate it into
the growing piece. Nevertheless, the delegation of composition subtasks to the computer
1
Ariza [2005] proposes the term computer-aided algorithmic composition (CAAC) for this eld, but his
term has not yet become widely accepted.
1
1. Introduction
often results in a dierent musical output than purely manually composed music. For ex-
ample, self-similar systems often lead to highly complex but musically convincing results
which are dicult to compose by hand.
Systematic surveys of techniques in CAC with historical annotations are provided
by Roads [1996], Miranda [2001], and Taube [2004]. Assayag [1998] outlines the history
of computer-aided composition. Papadopoulos and Wiggins [1999] oer a systematic
overview with a focus on systems based on techniques from Articial Intelligence.
1.2. Constraint-Based Computer-Aided Composition
Rule-based approaches for CAC emulate the way in which music theories are typically
expressed.
2
For centuries, compositional rules have been an established device for express-
ing explicit compositional knowledge. For instance, rules on how to create an organum
(an early polyphonic form) are already described in the treatise Musica enchiriadis,
written about 900 (usually attributed to Hucbald of St. Amand).
Because compositional rules are such a long-established concept, rule-based program-
ming approaches have attracted much attention among composers and scholars. This
approach allows a user to implement explicit compositional knowledge expressed by
compositional rules in a high-level way. A computer program can then use this knowl-
edge when generating music. The usage of rules for computational models of composition
will be motivated further in Sec. 3.2.1.
Constraint programming has proven to be a particularly successful programming para-
digm for realising ruled-based systems. Many compositional tasks have been addressed
by constraint programming. In addition to tasks inspired by traditional music theory
such as the generation of harmonic progressions [Pachet and Roy, 2001] or counterpoint
[Schottstaedt, 1989; Laurson, 1996], examples include purely rhythmical tasks [Sandred,
2003], Ligeti-like textures [Laurson and Kuuskankare, 2001], or instrumentation [Laurson
and Kuuskankare, 2001].
The attraction of constraint programming is easily explained. Constraint programming
allows users to model complex problems in a simple way. A problem is stated by a set
of variables (unknowns) and constraints (relations) between these variables. A compo-
sitional task is stated by (i) a music representation in which some musical aspects are
unknown and therefore represented by variables and (ii) compositional rules which
impose constraints on these variables. For instance, a chord can be expressed by an event
list and the chord pitches can be variables. Some harmonic rules may specify how the
chord pitches are related to each other. In the terminology of constraint programming,
the modelled problem or task is referred to as a constraint satisfaction problem (CSP).
2
In this text, the term rule (and rule-based) primarily denotes the musical concept of a compositional
rule. In particular, this term does not implicitly refer to any specic programming technique (e.g.
the term does not implicitly refer to a Prolog rule [Bratko, 2001] or a condition-action rule [Russell
and Norvig, 2002]). Instead, the text explains how the musical concept is realised as a programming
concept in dierent systems.
2
1.2. Constraint-Based Computer-Aided Composition
In a solution to a CSP, every variable has a value which is consistent with all its con-
straints. For example, the solution of a musical CSP is a fully-determined score (or score
excerpt such as a pure chord progression) consistent with all constraints expressed by the
rules. Existing constraint programming systems (abridged: constraint systems) can e-
ciently solve a CSP a fact which has greatly contributed to the popularity of constraint
programming.
A musical CSP implements a music theory model as a computer program: when the
program is executed, it generates music which complies with the modelled theory. The
present text denes a music theory model as a model of musical concepts and how these
concepts are related to each other. A music theory model implemented by a musical CSP
must be fully formalised (e.g. fully expressible in mathematical notation). However, in
the context of the present text a music theory model does not necessarily need to be
consistent with any existing music. For instance, a composer may develop some musical
CSP (and implicitly dene a theory model) ad-hoc to generate some subpart of a new
composition.
A musical CSP can always be dened from scratch in a general constraint system. For
instance, such a CSP can be dened in a regular programming language with support for
constraint programming. However, subject-specic CSPs share a considerable amount of
subject-specic knowledge: all musical CSPs require modelling of musical knowledge.
For instance, concepts such as note, pitch, or voice are required in a large number of
musical CSPs. Whenever a musical CSP is dened from scratch, all this knowledge
must be modelled anew. Whats more, any subject-specic optimisation of the search
process must also be carried out again (if the chosen constraint system supports such
optimisations at all).
Several generic music constraint systems have been proposed. A generic music constraint
system predenes general musical knowledge and building-blocks shared by many musical
CSPs and so highly simplies the denition of such problems. For example, such a
system may provide a specic music representation, templates to simplify the denition
of compositional rules, or mechanisms to conveniently control how a rule is applied to
the score. Examples of such systems are PWConstraints [Laurson, 1996], and Situation
[Rueda et al., 1998; Bonnet and Rueda, 1999]. A pioneering system is Carla [Courtot,
1990]. Further examples include the aggregation of the music representation MusES with
the constraint system BackTalk [Pachet and Roy, 1995], OMRC [Sandred, 2000a, 2003]
3
,
Arno [Anders, 2000], OMBacktrack [Truchet, b], and OMClouds [Truchet et al., 2001,
2003]. The number of existing systems underlines the high level of interest in music
constraint programming.
The constraint programming paradigm is well-suited to the needs of computer-aided
composition. Composers often prefer a way of working which is situated somewhere
between composing by hand and formalising the composition process such that it can be
3
OMRC is dened on top of OMCS, which in turn is a port of the PWConstraints subsystem PMC
from its host composition system PatchWork [Laurson, 1996] to the descendant OpenMusic [Assayag
et al., 1999].
3
1. Introduction
delegated to the computer (cf. [Laske, 1981]). Constraint programming supports this way
of working very well. For example, the composer can determine some aspects of the music
(e.g. certain pitches) by hand and constrain other aspects by rules. Alternatively, the
composer may specify the high-level structure (e.g. the formal structure) manually and
let the computer ll in the details. Furthermore, composers usually do not fully formalise
certain aspects of the composition process before they start composing. Instead, the
formalisation is often an integral aspect of the composition process itself. A compositional
task dened by means of constraint programming can be shaped in a highly exible way
during the composition process by the adding, removing and changing of individual rules.
Established composers have already made extensive use of constraint programming.
These include Antoine Bonnet (e.g. for

Epitaphe for 8 brass instruments, 2 pianos, or-
chestra and electro-acoustics, 19921994, using Situation [Bresson et al., 2005]), Mag-
nus Lindberg (Engine for chamber orchestra, 1996, using PWConstraints [Rueda et al.,
1998]), Georges Bloch (Palm Sax for seven saxophones, using Situation [Rueda et al.,
1998]),

Orjan Sandred (Kalejdoskop for clarinet, viola and piano, 1999, using OMRC,
[Sandred, 2003]), Jacopo Baboni Schilingi (Concubia nocte, in memoria di Luciano Be-
rio for soprano and live computer, 2003, using OMCS)
4
, Johannes Kretz (second horizon
for piano and orchestra, 2002, using both OMRC and OMCS [Kretz, 2003]), and Hans
Tutschku (Die S usse unserer traurigen Kindheit, music theater, 2005, using OMRC)
5
.
1.3. Requirements for a Generic Music Constraint System
Generic music constraint systems dier in their degree of generality. A system is more
generic if more musical CSPs can be implemented in this system. A most generic music
constraint system would allow its user to implement any music theory model conceiv-
able. From the perspective of the present research, such a system is an ideal system (as
long as eciency concerns are ignored). For example, composers usually prefer to make
compositional decisions themselves; only with reservations do they accept a composition
system which restricts their compositional freedom. A most generic music constraint
system would not restrict its users to any set of specic CSP classes.
The following paragraphs state a few conditions a most generic music constraint system
would have to meet. The present research does not propose such a most generic system
for eciency reasons. Nevertheless, these conditions are important as a guideline for the
design of a new system.
A most generic system allows the user to leave arbitrary information about the music
unknown such unknown information is represented by variables which can be freely
constrained. For example, the rhythmical structure of the music can be unknown and
freely constrained in such a system (e.g. the metric structure or the duration of notes).
4
Personal communication at PRISMA (Pedagogia e Ricerca sui Sistemi Musicali Assistiti) meeting,
January 2004 at Centro Tempo Reale in Florence.
5
Personal communication at PRISMA neeting, June 2006 at IRCAM, Paris.
4
1.3. Requirements for a Generic Music Constraint System
Similarly, the musical texture can be constrained (e.g. whether the music is homophonic,
polyphonic, or some piano-like texture where the number of voices is constantly changing,
etc.). The pitch structure of the music can be unknown and constrained (e.g. the CSP
implements a conventional theory of harmony, a dodecaphonic theory, or a microtonal
theory of harmony). Also, the instrumentation, or sound synthesis details (e.g. envelopes
for various parameters) can be unknown and constrained. A most generic system allows
for an extreme case (which is only theoretical), where all information about the music
is unknown in the CSP denition. The set of solutions for this extreme CSP contains
any conceivable score. The user can freely constrain any unknown information in order
to reduce the set of solutions as desired.
A most generic system provides access to arbitrary musical information required for the
denition and application of compositional rules in order to support arbitrary musical
CSPs. For example, traditional counterpoint rules require detailed information which
is deduced from the information explicitly represented in common music notation. For
instance, a common contrapuntal rule permits dissonant note pitches in situations where
several conditions are met which involve various musical aspects: a note may be dissonant
in cases where it is a passing note on an weak beat and below a certain duration. This
rule requires harmonic information which is deduced from the pitches of simultaneous
notes (whether a certain note is dissonant), melodic information which is deduced from
the pitches of notes in the same voice (whether this note is a passing note), metric
information which is deduced from the position of the note in a measure (whether this
note is on an weak beat), and rhythmical information (the notes duration).
Existing generic music constraint systems, however, are designed to cover specic ranges
of musical CSPs. These systems support the formalisation of certain music theory models
very well, but other theory models are hard or even impossible to dene. For example,
OMRC is designed solely for solving rhythmical CSPs, and Situation is best suited to
harmonic CSPs. PWConstraints subsystem score-PMC is designed to solve polyphonic
CSPs, but score-PMC requires a fully determined rhythmical structure in the problem
denition (i.e. only note pitches can be constrained).
A most generic system not only supports rhythmical, harmonic, as well as polyphonic
CSPs etc. Such a system also supports complex CSPs which constrain all these aspects
(and more) at the same time. Existing systems are not well-suited for addressing such
complex CSPs.
Nevertheless, to a certain extent existing systems are already generic. That is, they
support a considerable number of musical CSPs. These systems are generic because
the allow the user to program a CSP. In particular, existing systems support the free
denition of compositional rules: compositional rules can make use of arbitrarily complex
expressions.
Still, only certain aspects of a musical CSP can be programmed in these systems. Other
aspects cannot be changed or these systems only oer a limited set of selectable options.
For example, the music representations of many existing systems predene common mu-
sical concepts such as notes, pitches and durations which greatly simplies the denition
5
1. Introduction
of many musical CSPs. However, the user has only limited inuence on the form of
these representations. Therefore, the representations are only well suited for a limited
set of problems. In particular, the representations of existing systems limit the explicit
score information that can be stored and what derived information can be accessed. For
example, many systems provide a sequential music representation and primarily sup-
port deriving information from sets of score object which are positionally related (e.g.
neighbouring notes or chords in a sequence). Access to other derived information (e.g.
whether a note is on an weak beat, or whether a note is dissonant with respect to the
chord expressed by its surrounding notes in a polyphonic texture) is restricted which
clearly aects the set of CSPs which can be dened in these systems.
In addition, the search strategy of existing systems (i.e. how these systems nd a solution)
is usually optimised for specic classes of musical CSPs. In eect, systems sometimes even
purposefully restrict their users to CSPs which they can solve eciently. For instance,
score-PMC does not allow the user to constrain the temporal structure of music, because
a determined temporal structure is required by the polyphonic music search approach
of score-PMC to compute an ecient static search order [Laurson, 1996]. Similarly, the
search strategy of Situation (which performs a consistency enforcing technique to reduce
the search space) is optimised for its specic music representation format [Rueda et al.,
1998].
The present research proposes a highly generic music constraint system. This system
allows the user to dene and solve a large set of musical CSPs and is well suited for com-
plex musical CSPs (i.e. CSPs which depend on much musical information). At the same
time, this system performs reasonably eciently even for problems which were hard
or even impossible to solve in previous systems due to their computational complexity
(e.g. polyphonic CSPs in which both the rhythmical structure and the pitch structure is
constrained). The design of the system is outlined in Sec. 1.4.
This research originates from the eld of computer-aided composition. Nevertheless, its
results are applicable to areas outside this eld. A composer can apply a generic music
constraint system as an assistant in the composition process, a music theorist can use
it as a testbed to evaluate a music theory, an analyst can conduct a rule-based analysis,
and a teacher can demonstrate the eect of dierent compositional rules to students.
1.4. The Approach Taken
The present research proposes to make music constraint systems more generic by making
them more programmable. This proposal is exemplied in the design of the generic
music constraint system Strasheela.
6
When comparing Strasheela with previous systems,
6
Strasheela is also the name of an amicable and stubby scarecrow in the childrens novel The Wizard
of the Emerald City by Alexander Volkov [Wolkow, 1939] in which the Russian author retells The
Wonderful Wizard of Oz by Baum [1900]. The latter inspired the name for the programming language
Oz [van Roy and Haridi, 2004], which forms the foundation for the prototype of the Strasheela
6
1.4. The Approach Taken
three important aspects are (more) programmable: the music representation, the rule
application to the score, and the search strategy.
Strasheelas music representation aims to conveniently provide any information required
to express musical CSPs. To this end, the representation is highly extendable. Repre-
sentation building blocks required for many CSPs are ready-made, but Strasheela addi-
tionally predenes building blocks which assist the user in extending the representation
according to their needs.
Strasheela denes a novel music representation in the spirit of CHARM (Common Hi-
erarchical Abstract Representation for Music, [Harris et al., 1991]). Two principles have
been adopted from CHARM. Like CHARM, Strasheelas representation is based on the
notion of data abstraction [Abelson et al., 1985] and it supports user-controlled hierarchic
nesting of score objects.
Strasheelas representation complements these principles by other principles drawn from
the music representation literature, for example, selectable score parameter (music mag-
nitude) representations [Pope, 1992] (e.g. a pitch can be represented by a key-number,
cent or frequency value), bidirectional links between score objects to facilitate free
traversing in the score hierarchy [Laurson, 1996], temporal containers which organise
their elements sequentially or simultaneously in time [Dannenberg, 1989], organisation of
musical data types in a user-extendable class hierarchy [Pope, 1991; Desain and Honing,
1997], and a highly generic data abstraction interface realised by higher-order functions
[Desain, 1990].
The Strasheela user freely controls which variables in the music representation are con-
strained by which compositional rule. To this eect, Strasheela fully decouples the deni-
tion and application of a rule in order to make the rule application freely programmable.
Strasheela proposes the encapsulation of compositional rules in functions (actually pro-
cedures) as rst-class values [Abelson et al., 1985]. This approach allows the user to
dene rule application mechanisms as higher-order functions expecting rules (i.e. func-
tions) as arguments. Several rule application functions suited to many CSPs have been
dened, which either reproduce rule application mechanisms of existing systems or con-
stitute convenient novel application mechanisms. The user can easily dene further rule
application functions according to their needs (as shown in Sec. 6.2.4).
To be useful in practice, a constraint system must be reasonably ecient. It makes a
big dierence whether a CSP takes seconds or hours to solve. Strasheela is founded on
a constraint programming model based on the notion of computation spaces [Schulte,
2002]. This model makes the search process itself programmable at a high-level. The
programmable constraint model allows the user, for example, to optimise the search
process for CSPs with a particular structure (e.g. harmonic CSPs or polyphonic CSPs)
composition system.
The scarecrows brain consists only of bran, pins and needles. Nevertheless, he is a brilliant logician
and loves to multiply four gure numbers at night. Little is yet known about his interest in music,
but Strasheela is reported to sometimes dance and sing with joy.
7
1. Introduction
by dening what decisions are made during search (the distribution strategy, i.e., the
branching heuristics). For instance, the user can control the order in which variables are
visited in the search process depending on the information available at the time of the
decision (dynamic variable ordering). This decision order inuences immensely the size
of the search space. Most previous systems do not allow the user to customise this. In
Strasheela, such optimisations are independent of the actual problem denition, which
allows the user to easily test a CSP with dierent search strategies or to reuse proven
strategies.
Several novel score distribution strategies have been dened for Strasheela which are
suitable for a large range of musical CSPs. In particular, Strasheela provides a score
distribution strategy which allows the user to eciently solve polyphonic CSPs in which
both the rhythmical structure as well as other parameters (e.g. pitches) are unknown
and constrained in the problem denition [Anders, 2002]. Previous systems discouraged
or even disallowed the denition of such problems for eciency reasons.
1.5. Outline
This thesis consists of four parts: a literature review, a motivation of a generic music
constraint system including a description of its requirements, a presentation of a system
which fullls these requirements, and an evaluation of this system.
Literature Review: The related literature is surveyed in two separate chapters, because
the presented two research elds have been relatively independent so far. Chapter 2
reviews how music can be represented in computer programs. The chapter discusses
both the actual information to express, and fundamental computer science concepts
used to represent this information adequately. Chapter 3 surveys the research into
constraint-based music composition. After introducing the eld in general, the
chapter reviews generic music constraint systems which support a considerable
number of musical CSPs (the present text continues the research which led to
these systems).
Motivation: Chapter 4 analyses the shortcomings of existing generic music constraint
systems in terms of generality (i.e. which musical CSPs cannot be solved by these
systems and why), formulates the research goal based on this analysis (the devel-
opment of a more generic system) and outlines how this research goal is addressed
by the design of Strasheela. The analysis singles out three problematic aspects in
the design of existing systems: their respective music representation, rule formal-
ism and search strategy. Later parts of this text are often organised in accordance
with these three aspects.
Strasheela: Chapters 5 to 9 introduce Strasheela in detail. The rst three chapters ex-
plain Strasheelas design, that is, its music representation (Chap. 5), rule formalism
8
1.5. Outline
(Chap. 6) and search approach (Chap. 7). Chapter 8 demonstrates the use of Stra-
sheela by examples and shows how the dierent parts of Strasheela work together.
Chapter 9 introduces a Strasheela prototype and relates it to the model proposed
in this text.
Discussion: The last part evaluates and discusses the results of this thesis. Chapter 10
compares Strasheelas approach with existing systems. The chapter investigates by
means of general criteria the respects in which Strasheela is more generic than pre-
vious systems. Finally, Chap. 11 summarises the main contributions and suggests
further developments.
Because this thesis presents a highly interdisciplinary subject, the text explains many
terms, even terminology familiar in one of the related areas. In particular, many terms
and concepts common in computer science are briey dened (notational conventions
are specied in App. A).
9
1. Introduction
10
2. Survey I: Issues in Music Representation
for Computer-Aided Composition
For any music system, the format and the features of its underlying music representation
prove to be highly inuential. For instance, a composition system can only output scores
which can be expressed in terms of its music representation.
Various music representations have been proposed, as many systems come with their own
novel representation. Nonetheless, dierent representations also share many concepts and
techniques which have been identied as important and suitable. Dannenberg [1993] and
Wiggins et al. [1993] both survey this research eld.
This chapter reviews how music can be represented in computer programs. It discusses
various issues concerning how information contained in a score is represented in an
appropriate way for computer-aided composition systems. Although this chapter presents
several concepts and techniques proposed by the literature, it covers only a subset of this
research. For example, representations of continuous data as well as representations for
real-time systems are important research subelds, but this text does not even touch on
these subjects. Instead, this chapter focusses on concepts and techniques inuential for
the present research.
Chapter Overview
Section 2.1 introduces what a music representation is. To make the discussion more
specic, Sec. 2.2 briey presents the event-list concept as a simple but fully-edged
representation scheme. Section 2.3 comments on the complexities of representing score
parameters (musical magnitudes). Various hierarchic score topologies are surveyed in
Sec. 2.4.
The remaining sections review the application of fundamental programming concepts to
the denition of a music representation in order to make this denition easy to extend by
making extensions modular and concise. These programming concepts are rst motivated
by underlining the importance of an user-extensible representation (Sec. 2.5). Section
2.6 compares the idea of a textual representation with a representation based on data
abstraction and lists their respective benets and drawbacks. The concept of object-
oriented programming to incrementally dene a data abstraction is introduced in Sec.
2.7. Section 2.8 presents the idea of higher-order programming which can complement
a data abstraction interface by highly generic functions. A brief conclusion summarises
this chapter (Sec. 2.9).
11
2. Survey I: Issues in Music Representation for Computer-Aided Composition
2.1. Introduction
2.1.1. What is a Music Representation?
The present section explains in general terms what a music representation is. The prob-
lem of such an explanation lies in the fact that dierent research areas require very
dierent music representations which are hardly compatible.
This introductory explanation therefore explains primarily what a music representation
is in the context of the present research. It will relate this view to slightly dierent views
from the literature. Section 2.1.2 further widens this perspective and positions the view
on music representation of this research in the overall picture.
A music representation exhibits musical information. For instance, the representation
expresses the information contained a printed score or knowledge how to perform such
a score.
The present research mainly deals with score information. Therefore, the term score
representation (e.g. used by Taube [1993]) could have been used instead in this text.
Nevertheless, the more general term music representation is used throughout this thesis
because this term is more commonly used for denoting a representation of score infor-
mation.
Often, a music representation uses a representation format which is designed for this
purpose. For example, a widespread representation format is common music notation
(sheet music). This format uses specially devised symbols to represent information on
the music such as the pitch and the duration of notes. The present survey studies music
representations for computer programs.
In the context of the present research, the term music representation denotes some
symbolic representation. Other authors also regard a raw waveform or sound analysis
data as a music representation [Wiggins et al., 1993].
More specically, the term music representation denotes some data structure in this
text. Other authors also subsume composition systems under this term as procedural
music representations [Dannenberg, 1993]. The present text would instead speak of a
system which creates or processes a music representation. Still, a representation does not
only consist of explicitly represented information (i.e. static information). Also derived
information (i.e. information deduced from other information) plays a very important
role. For example, common music notation explicitly expresses note pitches. From this
information, the intervals between the pitches can be derived. Derived information is
often obtained procedurally, for example, by a function call.
2.1.2. The Broader Perspective
There exists a substantial amount of literature on music representation and its related
elds. Music representations have been proposed for a wide range of application areas and
12
2.1. Introduction
the music representation literature is often inuenced by specic application areas. The
following paragraphs briey outline these dierent application areas in order to position
the present research in the overall picture using categories proposed in the literature.
Honing [1993] identies several dierent research elds with specic requirements for
an appropriate music representation. Honing distinguishes between two larger research
elds: music analysis and production on the one hand (where representations are of a
technical nature) and cognitive sciences on the other hand (with conceptual or mental
representations). To the rst eld belong research areas such as musicology, computer
music, and music publishing and to the second eld areas such as articial intelligence,
computational psychology, as well as music perception and cognition. The present re-
search clearly belongs into the eld addressing music analysis and production. Therefore
also this survey focusses on the literature from this area.
Wiggins et al. [1993] discuss three general tasks (in music analysis and production) for
which a music representation is used and which inuence its design: recording, analysis,
and composition. Recording requires the representation primarily to be accurate. For
analysis it is important to exploit information which is often only implicit. Finally, com-
position requires a particular exible representation. All three tasks are important for
the present research, but the importance increases from the rst to the third task.
A music representation stores information of dierent musical aspects. The Standard
Music Description Language (SMDL, [Sloan, 1997]) dierentiates between four basic
domains: the logical domain (abstract information), gestural domain (performance in-
formation), visual domain (graphical information), and analytical domain (bibliographic
and interpretative information). For the present research, the logical domain and the
analytical domain are of primary importance.
2.1.3. Terminology
The music representation literature introduces a large number of terms. Unfortunately,
so far there exists no generally accepted terminology. Dierent representations introduce
dierent terms for similar concepts. For example, a score data object which contains
and groups other data objects is either called a constituent (in CHARM, [Harris et al.,
1991]), a structured (in Expresso, [Desain and Honing, 1997]), a collection, a holder
or a container (in dierent versions of Common Music, e.g., [Taube, 1993]).
To allow comparison of dierent representations in a uniform way, this text aims to use
a consistent terminology. For example, this text uses the term container for the above-
mentioned concept. Of course, the constructs of the dierent systems modelling this
concept are not truly equivalent and therefore the text mentions the original terminology
of each representation as well.
13
2. Survey I: Issues in Music Representation for Computer-Aided Composition
2.2. A Basic Representation: Event List
A fairly basic music representation is the event list, where the score consists of a at
list of events. An event is a score object
1
which produces sound. An event features a
set of parameters the basic magnitudes in a music representation such as start time,
duration or pitch.
2
Event lists have been highly successful, especially for sound synthesis. This can be seen in
languages in the Music-N tradition such as Csound [Boulanger, 2000]. Furthermore, also
more advanced systems such as Siren/SmOKe [Pope, 1992] or SuperCollider [McCartney,
2002] still support an event list.
2.3. Parameter Representation
The seemingly simple parameter concept has in fact many facets. Often, parameters
can be specied in several ways, for instance as an acoustic property (e.g. amplitude),
a perceptual property (e.g. loudness), a performance property (MIDI velocity), or a
music notation score symbol (e.g. a dynamic symbol such as f to denote forte). In
practice, however, these clearly distinct properties are often blurred. For example, in a
sound synthesis context the term amplitude is often used to denote loudness. Parameters
are represented symbolically or numerically (e.g. symbolic pitch name or numeric key
number) and are measured in an absolute or relative way (e.g. an absolute duration in
seconds or a relative duration in beats complemented by a tempo specication).
Some systems dene only a single representation form for each score parameter. In the
internal music representation of the visual-programming composition system PatchWork
[Laurson, 1996] and its successor OpenMusic [Assayag et al., 1999], for instance, note
parameters are always specied as integers in the following way: the absolute note dura-
tion is measured in milliseconds, the pitch in midicent (a combination of the MIDI pitch
number and cent values, i.e. middle c is represented by 6000) and the loudness is given
as a MIDI velocity [Agon et al., 2001]. This uniform representation simplies the imple-
mentation of the graphical music editors which are bundled with the representations of
these systems.
Other systems allow the user to choose from a predened set of supported representation
forms. In the composition system Common Music, parameter values can be specied in
several ways. A note pitch, for instance, is either given as a symbolic note name, a MIDI
key-number, or a frequency in Hertz [Taube, 2004].
Still, none of these representation forms does represent the enharmonic spelling (i.e. the
representation does not dier between c and d). To address this issue, some systems
1
The term score object is used to denote anything in a score (e.g. a pitch, a note, or whole score) which
is represented by a single but possibly compound value. Later, in the context of object-oriented
programming (Sec. 2.7) this term is used in a more specic meaning.
2
Following the terminology in composition and sound synthesis, this text uses the term parameter
instead of the terms feature or attribute used elsewhere.
14
2.4. Topologies of Hierarchic Representations
represent the pitch in a composite way. For example, CHARM (Common Hierarchical
Abstract Representation for Music, [Harris et al., 1991]) represents a pitch by three
components: its note name, accidental and octave. The important advantage of such
a representation is the ability to correctly spell accidentals for western tonal harmony.
Unfortunately, such a representation is also limited to western music (i.e. non-western
or microtonal music is excluded).
Other systems express the rhythmic structure in a more elaborated manner. A highly
expressive example is the RTM-notation [Laurson, 1996], featured by PatchWork and by
OpenMusic (in slightly incompatible versions). RTM-notation represents relative dura-
tions in a tree, where each subtree consists of a proportional duration value and optional
children subtrees. The sum of the proportional durations of all children species how the
proportional duration of their parent tree is subdivided and thus what the proportional
duration of each child means. For instance, a triplet is represented by the proportional
duration of the whole triplet and three children of equal duration which optionally
are further subdivided in smaller rhythmic values by further subtrees. RTM-notation
expresses pauses and ties by special data types. Usually, all durations are expressed by
positive integers. However, pauses are expressed by negative integers and a note tied to
its predecessor is expressed by a oat. The RTM-notation reects clearly the hierarchic
structure of rhythmic subdivisions, especially for highly complex embedded rhythms.
On the other hand, this notation is relatively hard to read. ENP-score-notation (Ex-
pressive Notation Package, [Laurson and Kuuskankare, 2003]) supported by PWGL
(PatchWork with OpenGL, another successor of PatchWork [Laurson and Kuuskankare,
2002]) extends the purely rhythmical RTM representation to a full music notation
representation which also contains information such as note pitch, instrumentation and
expressions.
2.4. Topologies of Hierarchic Representations
When musicians talk about music, they rarely talk about single notes. Instead, they talk
about groups of notes like motives, voices, rhythmic patterns, or chords. Furthermore, it
is widely agreed that such structural objects can be hierarchically nested, for instance,
in a hierarchic harmonic structure as in Schenkerian analysis [Forte and Gilbert, 1982]
or in an hierarchic grouping structure [Lerdahl and Jackendo, 1983].
The benets of a structured representation are obvious for the purpose of music composi-
tion and analysis. Even for sound synthesis where event-list-like music representations
are successfully applied since decades (Sec. 2.2) a structured representation can be
benecial. For example, in a Csound score it is tricky to apply an amplitude envelope to
a group of notes [Dannenberg, 1993]. Such an envelope is more easily expressed in the
MIDI protocol using MIDI controller messages. Still, the group of notes and the envelope
are not linked in the MIDI format: when a MIDI le is edited (e.g. in a MIDI sequencer)
the user has to maintain this relation manually (e.g. by rst moving the notes along the
time axis and then the controller data).
15
2. Survey I: Issues in Music Representation for Computer-Aided Composition
Consequently, many researchers studied music representations in which a set of score
objects (e.g. notes) is somehow grouped and this grouping is explicitly represented.
Many music representations allow the user to explicitly group a set of score objects by
collecting all objects of this set in some higher-level object. Several dierent terms have
been proposed for such an higher-level object grouping other score objects (Sec. 2.1.3
already listed several examples when discussing the dierences of music representation
terminology in various systems). In general, this text uses the term score container, but
when discussing a particular system also its respective terminology will be mentioned.
The present section discusses the dierent approaches to hierarchical music represen-
tations found in the literature by sorting them into dierent score topologies. Some
representations dene two-dimensional lattice-like representations (Sec. 2.4.1), others
trees which either mirror the hierarchic layers of traditional music notation (Sec. 2.4.2)
or dene a set of generic containers which can be arbitrarily nested in a tree (Sec. 2.4.3).
Finally, some systems represent music in an acyclic graph (Sec. 2.4.4).
The Score Context Concept
The present research introduces the term score context to denote an arbitrary set of score
objects which are inter-related.
3
A context is dened by specifying how its elements are
inter-related. For example, all objects (e.g. notes and pauses) which belong to a voice
denote a score context. Figure 2.1 depicts this context for the voice myVoice in a formal
way using the common set-builder notation. A container can be used to express such a
score context explicitly.
x : x is part of myVoice
Figure 2.1.: First score context example: the set of all objects belonging to the voice
myVoice
However, in this thesis the term score context is more general. It does not only denote
a set of score objects which are all contained in the same object (e.g. a voice, a measure
etc). Instead, a score context is an arbitrary set of score objects. Thus, another score
context example is the set of notes which all sound at a certain time point t (see Fig.
2.2).
myNote : isNote(myNote)
start
myNote
t end
myNote

Figure 2.2.: Second score context example: the set of all notes which sound at time t
3
The term score context is inspired by Lilypond [Nienhuys and Nieuwenhuizen, 2003] and PWCon-
straints [Laurson, 1996], however the term is redened it here with a much more general meaning.
16
2.4. Topologies of Hierarchic Representations
2.4.1. Two-Dimensional Representation
In **kern the representation of Humdrum [Selfridge-Field, 1997; Huron, 2002] mu-
sic is organised such that the two musical dimensions time and instrument part are
represented in a two-dimensional way. The gure 2.4 shows the topology of the musical
example in Fig. 2.3 using such a two-dimensional representation.

4
2

4
2

Figure 2.3.: Bela Bartok. Mikrokosmos, No. 87, beginning

note
pause
null
token
[...]
[...]
Figure 2.4.: Representation of the Mikrokosmos example (Fig. 2.3) by a two-dimensional
lattice
The **kern representation uses a purely textual (ASCII) format. Notes and other score
objects simultaneous in time are represented in horizontal lines (called records) whereas
time is represented in vertical columns (called spines). Thus, compared with common
music notation or the graphical representation of the **kern topology above, the hori-
zontal and the vertical dimensions are swapped. By this practise, **kern avoids the line
breaks of common music notation.
The two-dimensional representation requires some transformation of the musical data.
Although the duration of records may dier (or have no length at all such as clef decla-
rations), each eld in the two-dimensional representation must be specied. Therefore,
in case a few shorter notes run in parallel with a longer note, it is important that the
persistence of the longer note is explicitly denoted by a null token (see Fig. 2.4).
It can also be seen in in Fig. 2.4, that the data in a single spine is not necessarily a
at sequence: the two notes of the chord in the second measure are placed in a single
eld. Furthermore, the representation may also split the actual note and specic note
parameters (e.g. ngering information or lyrics) in dierent spines.
It should be noted that **kern explicitly represents information necessary to access
two important score contexts: the set of score objects (e.g. notes or pauses) in a part
17
2. Survey I: Issues in Music Representation for Computer-Aided Composition
(including their order in the part) and the set of simultaneous score objects at any
time point in the score (including which object belongs to which part). In case some
simultaneous object is a null-token, then its preceding object is simultaneous.
2.4.2. Representation with Specialised Containers in a Tree
Several representations explicitly embody hierarchic components inspired by music no-
tation. For example, these representations explicitly express notation concepts such as
the whole score, a system, a sta, a voice, a measure, a chord, a pause, or a note by their
own score object. These score objects are often organised in a tree as shown in Fig. 2.5.
score
system
staff
note
staff
pause
chord
Figure 2.5.: Representation of the Mikrokosmos example (Fig. 2.3) by a tree of specialised
containers (the types of this example are inspired by Common Music Notation)
For self-evident reasons, music notation systems in particular favour this representation
form. Examples include textual music notation languages (e.g. Common Music Notation
[Schottstaedt, 1997], Lilypond [Nienhuys and Nieuwenhuizen, 2003], and MusicXML
[Recordare LLC, 2006]), as well as systems which closely integrate music notation (e.g.
PWGL [Laurson and Kuuskankare, 2002] which makes use of ENP-score-notation [Laur-
son and Kuuskankare, 2003]).
The actual structure supported depends on the respective system. For example, Fig. 2.5
depicts the structure of a Common Music Notation score. A score in ENP, on the other
hands, is structured dierently and consists of parts, which in turn consist of voices,
which consists of measures, which consists of beats in RTM-notation (Sec. 2.3) which
nally rhythmically position score events.
Music notation systems often predene a large number of dierent containers, to have a
corresponding container for every concept of nesting in common music notation. Often,
the hierarchic nesting is xed (e.g. all stas are contained in the top-level score object or
in system objects). In other cases, the hierarchic nesting is xed, but levels are optional
if not explicitly specied, the system will automatically create the respective level with
default settings.
The representation of specialised containers in a tree explicitly represents information
required to access a number score contexts. In particular, hierarchic relations are stored
18
2.4. Topologies of Hierarchic Representations
explicitly (e.g. which note belongs to which part). Also the order of score objects (e.g. the
order of notes in a part), and their type (e.g. object x is a note) is explicitly represented.
Some systems allow the user to explicitly encode further groupings, for example, by
marking the start and end of an analysis bracket, a slur, or a crescendo hairpin. Yet, the
context of simultaneous score objects is not explicitly represented in contrast to the
two-dimensional score topology (Sec. 2.4.1).
2.4.3. Representation with Generic Containers in a Tree
The representation of each hierarchical musical concept (e.g. chord, voice, part or sys-
tem) by a special score object type results in a large number of types. For applications
beyond music notation, a large number of types is often not desirable and a more generic
representation scheme is preferred.
Nested Event-Lists
A radically simplied hierarchic representation scheme (when compared with the topol-
ogy of specialised score objects) extends the event list representation (Sec. 2.2) by the
notion of a single container type. This container contains either events or other contain-
ers. Each contained score object features both an explicit start time and a duration to
dene the temporal structure within the container. Examples for such temporal contain-
ers which allow for nested event lists are the EventList in SmOKe [Pope, 1992],
the seq-object in Common Music since version 2 [Taube, 2004], and the Maquette in
OpenMusic [Agon et al., 1998].
Nested Temporal Containers
A related representation scheme complements the notion of events by two dierent
generic temporal container types. The rst container organises its contained score objects
sequentially in time and the second organises them simultaneously in time. Contained
objects can be either events or other temporal containers the resulting structure is
thus again a tree (Fig. 2.6).
simultaneous [score]
sequential [part]
note
sequential [part]
pause
simultaneous [chord]
Figure 2.6.: Representation of the Mikrokosmos example (Fig. 2.3) by tree of generic
temporal containers
19
2. Survey I: Issues in Music Representation for Computer-Aided Composition
This second representation with generic containers can be more concise than the nested
event-list representation, because only the duration of each score object must be specied
explicitly the start time of each object is implicitly specied by its position in a
container (e.g. by the start time and duration of its predecessor in a sequential container).
Several systems apply a representation scheme in which sequential and simultaneous
arrangements of score objects can be nested freely. Examples include the functional
playback-time control language Arctic [Dannenberg, 1984] (and the continuative work
such as the score language Canon [Dannenberg, 1989] and the synthesis language Nyquist
[Dannenberg, 1997]), the visual composition system LOCO [Desain and Honing, 1988],
the representation framework Music Structures [Balaban, 1996], as well as the CAC
systems Haskore [Hudak], and Common Music (the now obsolete version 1) [Taube,
1993]. Also some music notation systems oer this generic representation form as a
complement for specialised containers (e.g. Lilypond).
4
Desain and Honing [1997] point out that using this generic representation, the same
music can be represented in dierent ways to explicitly represent dierent musical in-
formation. For example, the score topology in Fig. 2.6 expresses the sequence of notes
which constitute a part. The topology of Fig. 2.7, on the other hand, expresses which
section of a part belongs to a certain measure. A user-dened nesting of generic temporal
containers thus allows the user to dene which information is represented.
sequential [score]
simultaneous [measure] simultaneous [measure]
sequential [part]
note
sequential [part]
pause
sequential [part]
simultaneous [chord]
sequential [part]
Figure 2.7.: Representation of the Mikrokosmos example (Fig. 2.3) by an alternative tree
of generic temporal containers (cf. Fig. 2.6 using the same temporal containers to express
dierent hierarchic structures and thus dierent score contexts)
2.4.4. Representation with Containers in an Acyclic Graph
The nesting of generic temporal containers in dierent ways in a tree allows for the
representation of dierent information. Some researchers generalised this approach and
allow the user to subsume all this information in a single representation (Fig. 2.8).
4
Some representations (e.g. Arctic, Music Structures and Haskore) introduce operations instead of
special containers to arrange score objects sequentially or simultaneously in time.
20
2.4. Topologies of Hierarchic Representations
container [score]
[measure] [measure]
[part]
note
[part]
pause
[chord]
Figure 2.8.: Representation of the Mikrokosmos example (Fig. 2.3) by a music represen-
tation consisting of multiple orthogonal containers which form an acyclic graph. This
representation subsumes the score contexts expressed in Fig. 2.6 and in Fig. 2.7
For example, CHARM [Wiggins et al., 1993, 1989; Harris et al., 1991; Smaill et al.,
1993] supports such a score topology. As previous systems, CHARM denes events and
containers (called constituents in CHARM). However, each score object in CHARM
features a unique identier. The objects grouped in a container (the particles in CHARM
terminology) are referenced by their identier. This design decision makes it possible to
incorporate an object in multiple containers at the same time. However, no constituent
may be contained in itself (neither directly nor indirectly). The resulting topology thus
forms a directed acyclic graph.
In such a score topology, the multiple dimensions of nested events and containers can
express various information. In the example in Fig. 2.8, one dimension expresses the
sequence of objects in a part, the other the set of objects in a measure. Other examples
include grouping, harmonic information, or motif boundaries.
Each of these dimensions could even be hierarchically nested. For example, the met-
ric dimension could nest notes recursively in beats, measures, and higher-level accent
groups. The harmonic dimension, on the other hand, could represent prolongational
trees created by Schenkerian analysis.
2.4.5. Summary
This survey introduced several score topologies. The following topologies have been re-
viewed.
Event list (Sec. 2.2)
Two-dimensional representation (Sec. 2.4.1)
Tree with specialised containers (Sec. 2.4.2)
21
2. Survey I: Issues in Music Representation for Computer-Aided Composition
Nested event-lists (Sec. 2.4.3)
Tree with two generic temporal containers (Sec. 2.4.3)
Acyclic graph (Sec. 2.4.4)
Some of these topologies generalise other topologies, some topologies stand on their own.
For example, the pure event list is generalised by the nested event list which in turn
is generalised in the acyclic graph representation. The two-dimensional representation,
however, is not related with any other representation.
The topology of a hierarchic music representation inuences which musical information
is represented explicitly. For example, in the two-dimensional representation shown in
Fig. 2.4 the contexts sequence of notes in a system as well as simultaneous notes are
explicitly represented. Much further information can be represented in this topology (as
demonstrated by the **kern representation), but some information can only be expressed
implicitly. In particular, hierarchic nesting cannot be expressed explicitly.
5
Dierent topologies express dierent information explicitly. The tree of temporal con-
tainers depicted in Fig. 2.6 expresses hierarchic nesting (e.g. some note is contained a
chord, which is contained in a part). This information is not expressed explicitly in the
two-dimensional representation. However, the hierarchic representation cannot explicitly
represent the two contexts sequence of notes in a system as well as simultaneous notes
at the same time. Only accidentally, in the second measure of the example every note
of the upper part is simultaneous with the lower part notes. In case of dierent note
start times in the lower part, only a procedural comparison of the start and end times
of all notes would reveal which notes are actually simultaneous and which are not. The
two-dimensional representation, on the other hand, does represent both these contexts
explicitly.
The acyclic graph representation can potentially express any set of related score objects
explicitly. However, for a large number of score contexts to represent, such an explicit
representation can become rather laborious.
Musical information which is only implicitly expressed can still be accessed conveniently
when the representation is designed accordingly. For instance, the data abstraction con-
cept (presented in the following Sec. 2.6) makes this possible. Also, higher-order pro-
gramming (Sec. 2.8) can greatly facilitate accessing various derived information in a
concise way.
2.5. Extensibility
It is impossible for the designer of a music representation to foresee all future uses of the
representation. For instance, composers are constantly developing novel composition and
5
The **kern representation allows for special markings to denote, for example, nested slurs and phrases.
Still, this information is expressed by special symbols and not by explicit nesting.
22
2.5. Extensibility
instrument playing techniques. Furthermore, scores for novel sound synthesis techniques
have to reect the synthesis parameters, and research is developing new insights on music
(e.g. new music theory concepts). For these reasons, a music representation cannot be
xed but requires that it can grow with its use.
Computer music has a tradition of highly extensible music representations. For instance,
important for the success of the musicology research toolkit Humdrum was the fact that
the Humdrum user can easily dene new Humdrum representations according to her/his
very specic needs (e.g. for Bugandan xylophone music, Schenkerian graphs, or dance
steps) [Huron, 2002]. Furthermore, Music-N inspired languages such as Csound allow the
user to extend the number of parameters per event and to interpret the meaning of the
additional parameters.
A counter-example is the MIDI protocol which strictly limits the format and number
of parameters of note events to start time, key-number (integers in [0, 127]), velocity
([0, 127]), channel ([0, 15]) and end time. In case the user wants to apply MIDI events for
a purpose not pre-conceived in the representation design (e.g. to represent microtonal
music with more than 12 pitches per octave) the user cannot adjust or extend the repre-
sentation but has to invent some kludge which works around to problem (e.g. adjusting
the tuning of dierent MIDI channels with the same sound program and distribute note
events on dierent channels accordingly instead of simply extending the pitch resolution).
Particularly exible representations are often integrated in a programming environment
where the composer can extend and adapt the representation by programming. Examples
are the representations of composition systems such as Common Music, OpenMusic,
PWGL, or Haskore.
In such a a programming environment, a representation is more easily extended in case
extensions can be modular and concise. An extension is modular if the user can develop
such an extension without changing the original program which implements the represen-
tation. For example, some user may need to derive information which is not supported so
far (e.g. deduce a symbolic chord description from a set of notes). A modular extension
for such derived information does not need to change the original program itself but only
adds a denition.
An extension is concise if it does not require duplicating functionality or repeating code
which is already part of the original program. For example, some user may need to dene
a new score object type. If such an extension denes the new type as a variant of an
already exiting type where only the dierences to the exiting type require denition,
then this extension is dened concisely.
The following sections discuss how the application of fundamental programming concepts
can make a representation highly extendable by making extensions modular and concise.
Concision and modularity is facilitated by the following concepts: data abstraction (Sec.
2.6), object-oriented programming (Sec. 2.7), and higher-order programming (Sec. 2.8).
23
2. Survey I: Issues in Music Representation for Computer-Aided Composition
2.6. Textual Representation vs. Data Abstraction
Textual Representations
Many music representations proposed in the literature are textual representations (i.e.
use ASCII code). For example, the Csound score is represented textually. Many more
textual representations are discussed by Selfridge-Field [1997].
Raymond [2003] explains the benets of textual representations in general compared with
binary representations: Text streams are a valuable universal format because theyre
easy for human beings to read, write, and edit without specialized tools. Moreover,
text-les are easily read and written by other programs and thus facilitate cooperation
between independent programs. Binary data formats are only suitable for large data
sets which require compression (and even in this case it is important to keep in mind
that textual data compress very well with standard compression tools such as gzip).
Huron [2002] advocates textual music representation formats (e.g. used by Humdrum)
for similar reasons.
Raymond and Huron recommend textual formats primarily for exchanging data between
applications. One could conclude that it would also be a good idea to base the internal
representation of an application on a textual format, that is a format using data which
have a literal representation in a programming language. For instance, numbers, symbols
and lists are literally represented in Lisp.
However, using a textual representation directly inside an application also poses limita-
tions: such a representation only shows its explicitly encoded information. For instance,
it may be necessary to represent note parameters in dierent formats: pitches may be
represented either as MIDI key-numbers or as frequency values (cf. Sec. 2.3). Yet, an
application part (e.g. some user-function) which uses a textual representation directly
and expects pitches encoded as key-numbers would require some changes whenever the
format of the representation changes to frequency values. If multiple parts of an ap-
plication depend on the format of the textual representation, every part would require
changes which can prove extremely hard and error-prone.
Moreover, it is often necessary to represent the same musical information in dierent ways
for dierent purposes. Some application part using the representation may require note
pitches represented by absolute pitches, whereas another requires the intervals between
these pitches, and still another some symbolic specication of the chord formed by these
pitches. It would be convenient, if the music representation could provide such diverse
viewpoints for its musical information consistently even if the internal representation
changes (e.g. regardless whether pitches are represented by key-numbers or frequencies).
Data Abstraction
Computer science suggests the concept of data abstraction [Abelson et al., 1985, Chap. 2]
to address such requirements. The main idea here is to dene an abstract data type (ADT)
which encapsulates a set of values into a single datum and denes a set of operations
24
2.6. Textual Representation vs. Data Abstraction
on this datum. The set of all operations is called the ADT interface. The values are
encapsulated such that only the operations of the interface can process them.
Typically, the interface consists of operations such as a constructor to create a datum, a
destructor
6
, accessors (also known as selectors) and modiers (also known as mutators or
setters) to access and change the encapsulated data, operators for type-checking, equality
checking and arbitrary special purpose operators. For instance, if a system denes a note
as an ADT, it denes a note constructor, accessors and setters to access and change note
parameters (such as duration or pitch), an operator to check whether some datum is a
note et cetera.
When a system uses the data only via its interface, the actual implementation of the data
can change as long as the interface remains the same. For instance, some special note
pitch accessors getKeynumber and getFrequency may implicitly transform the pitch rep-
resentation and consistently return either MIDI key-numbers or frequencies regardless
of the internal format.
Moreover, the interface can abstract whatever information can be derived from the ex-
plicitly stored information. When procedurally derived information is wrapped in an
interface function, it can be accessed as conveniently as explicitly stored information.
For instance, Sec. 2.4.5 showed how dierent score topologies express dierent informa-
tion explicitly whereas other information can only be derived. The representation based
on generic temporal containers (Sec. 2.4.3) can only explicitly represent one of the two
contexts sequence of notes in a system and simultaneous notes at a time. Yet, the
missing context can be derived, for instance, by traversing the whole score hierarchy to
search for the score objects which belong to the missing context (this idea is discussed
further in Sec. 2.8). In most cases, it can even be neglected that such derived information
takes longer to retrieve.
Many systems design their music representations according to data abstraction princi-
ples. For example, CHARM [Harris et al., 1991] denes ADTs for parameters, events and
constituents (i.e. containers). Moreover, CHARM denes special arithmetic operations
for parameters such as time (i.e. a time point) and duration (i.e. an interval between
time points): adding a time and a duration results in a time, whereas adding two du-
rations results in a duration. MusES [Pachet, 1993] introduces a rich algebra of pitch
classes where pitch classes with up to two accidentals are represented as data and
accidentals serve as operations on the pitch data.
Some systems combine the benets of data abstraction with the benets of a textual
representation. For instance, the ENP-score-notation [Laurson and Kuuskankare, 2003]
is a textual input and output format for the internal representation of the graphical score
editor of PWGL.
6
Many computer-aided composition and analysis systems are based on a high-level programming lan-
guage which supports garbage collection and therefore makes a deconstructor superuous.
25
2. Survey I: Issues in Music Representation for Computer-Aided Composition
2.7. Class Hierarchy
A musical score contains data entities of many dierent data types. For instance, a
music representation modelling conventional music notation may require the following
types: notes marking pitch and timing information, articulation signs, and staves to
organise notes in voices. Dierent musical styles may use dierent data type sets. During
the compositional process the composer may even introduce further types (e.g. roman
numbers to sketch a harmonic progression).
Existing music representations often generalise this broad width of possible score infor-
mation. Instead of implementing an enormous set of dierent types in an unrelated way,
many representations dene musical data types as instances of classes organised in a
class hierarchy in the object-oriented programming sense.
In the object-oriented programming (OOP) paradigm [Booch, 1991; Rumbaugh et al.,
1991; Jacobson, 1992], the user denes an abstract data type (Sec. 2.6) by dening a
class where a class instance constitutes the actual data entity (more precisely, a class
instance has a type which the user dened by dening the class). A class instance is
often also called an object. Typically, a class denition denes class attributes (data
encapsulated in a class instance) and class methods (functions or procedures dening
the data abstraction interface).
7
OOP languages dier in their terminology: alternative terms for a class attribute are,
for example, instance variable, member variable, or slot. Another term for methods are
messages to stress the fact that objects communicate with each other.
OOP extends the notion of ADTs by the concept of inheritance. Multiple ADTs have
often much in common. For instance, note classes for dierent output formats may
share attributes and methods. OOP allows incrementally dened ADTs [van Roy and
Haridi, 2004]: the shared attributes and methods are dened only once in a more general
superclass from which a subclass inherits. Inheritance is often used to model an is-
a relation between instances of a subclass and a superclass, whereas attributes often
model an has-a relation between an object and the data at its attributes.
Polymorphism is a general data abstraction concept which is important to structure
programs in a maintainable way. Polymorphic data types are dierent data types which
(partly) dene the same data abstraction interface but respond in dierent ways. For
example, a MIDI note and a Csound note may both understand the interface function
play but cause dierent output formats. In OOP, a subclass inherits all attributes and
methods of its superclass. However, the superclass denitions can be extended or even
overwritten for a subclass to dene polymorphic classes. For example, a subclass MIDI-
note may inherit from a more general class note and add or overwrite the play method.
A well-designed object-oriented music representation is easily extended by the user. For
example, to add new features/attributes or to extend the data abstraction interface the
7
OOP often encapsulates both the actual data and the methods processing these data in a single datum
in contrast to ADTs. Yet, this dierence is irrelevant for the present research.
26
2.7. Class Hierarchy
user denes a new subclass which inherits the full interface and all features of an existing
class without changing its original denition.
Figure 2.9 shows an example of a class hierarchy for a music representation. The diagram
depicts the class hierarchy of Common Music presented by Taube [1993], which was
slightly simplied and translated into the UML [Booch et al., 1998; Fowler, 2003].
8
In
the diagram, class attributes and methods are omitted for brevity.
Figure 2.9.: The class hierarchy of Common Music (cf. Taube [1993])
It can be seen how inheritance is used to derive highly specic concepts (implemented
by classes) from more general concepts. Every Common Music class inherits from the
most general class object. On the next level, Common Music distinguishes between
an event (Sec. 2.2) and a collection (a container, Sec. 2.4). These concepts are fur-
ther sub-divided into specic event classes (e.g. note or even more specic midi-note)
and container classes (e.g. the sequential container called thread and the simultaneous
container called merge, see Sec. 2.4.3). A thus organised class hierarchy allows for user-
extensions (e.g. allows the user to add a new note subclass such as a csound-note with
a specic set of Csound p-elds).
For more than 20 years, OOP has proven a successful programming paradigm to im-
plement a music representation (cf. [Pope, 1991]). This fact is also reected by the
implementation programming languages often used in the eld (e.g. Smalltalk, Common
Lisp with CLOS, Java). The next paragraphs briey introduce further representation
examples.
SmOKe [Pope, 1992], originally designed to provide an exchange format for music pro-
gramming languages, extends the event-list concept by higher level concepts (e.g. nesting
of event-lists in a tree or user-denable middle-level structures such as chord or trill).
8
Taube [1993] refers to Stella, the representation of the now obsolete Common Music version 1.*. When
compare with recent Common Music versions, the class hierarchy of Stella clearly diers in the set
of containers implemented as well as in its terminology.
27
2. Survey I: Issues in Music Representation for Computer-Aided Composition
Desain and Honing [1997] discuss some more advanced OOP techniques in the context
of modelling musical expression knowledge. The authors point out limitations of single
inheritance where each class has at most a single superclass (as in Fig. 2.9). Multiple
inheritance on the other hand, allows for more then a single superclass. Using this
approach, a programmer can model orthogonal representational aspects in in their own
superclass which results in a more modular design.
Figure 2.10 depicts the Expresso class hierarchy of an early design phase (cf. Desain
and Honing [1997]). The denition of the classes note and pause shows similarities to
the Common Music class hierarchy shown above. The denition of containers (called
structured in Expresso), however, is dierent. To model musical expression knowledge,
Expresso complements the common temporal container classes (see Sec. 2.4.3, these
containers are called sequential and parallel in Expresso) by explicit representations of
the ornaments acciaccatura and appogiatura.
Figure 2.10.: Expresso class hierarchy with multiple inheritance resulting in the diamond
problem (cf. Desain and Honing [1997])
Expresso represents two orthogonal representational aspects with four abstract classes
(i.e. classes which will never be instantiated, marked by italic fonts in the UML) which all
inherit from the class structured. Here, the classes simultaneous and successive model a
temporal aspect of the representation whereas the classes multilateral and colateral (sic!)
model a distinction between a container with peer objects and a container representing
an ornament. However, in this class hierarchy some classes have a set of superclasses
which in turn has a common superclass. This situation sometimes referred to as the
diamond problem in the OOP literature [Wikipedia contributors, 2005] causes com-
plications in the inheritance scheme. For example, the class sequential inherits from the
class structured twice: once via the class successive and also via the class multilateral .
9
An alternative multiple inheritance technique is the denition of a mixin class, which
9
In CLOS the implementation language of Expresso such a class hierarchy is legal with a unam-
biguous semantics, well-dened by the precedence list of CLOS.
28
2.7. Class Hierarchy
avoids the diamond problem but still allows the user to dene a set of attributes and
methods in a modular way. A mixin class extends its subclasses without causing the
diamond problem. It can be useful to think of a mixin as an associated class instead of
is-a relation. In Fig. 2.11, the class hierarchy presented before is altered such that the
use of mixin classes avoids the diamond problem.
Figure 2.11.: Expresso class hierarchy excerpt with mixin classes (altered version of a
class hierarchy presented in Desain and Honing [1997])
MusES [Pachet et al., 1996] denes an exhaustive and theoretically founded model to
represent time and temporal relations between time intervals which are dened by a start
time and a duration. The system represents temporal objects (e.g. actual notes) and non-
temporal objects (e.g. pitch classes or intervals between pitches) as classes organised in
a class hierarchy. The particular rich pitch representation of MusES, also modelled by
OOP [Pachet, 1993], has been mentioned above (Sec. 2.3). The denition of MusES
avoids multiple inheritance by using delegation instead: an object references another
object and delegates specic method calls to it. MusES is designed as a generic and re-
usable representation framework to dene other systems. For instance, the application
of MusES in a music constraint system will be discussed later in the text (Sec. 3.3.4).
The visual-programming composition system OpenMusic [Assayag et al., 1999] features
also OOP in a visual way. The OpenMusic user visually denes classes, their attributes
and methods and organise them in a class hierarchy.
OOP languages disagree not only in the OOP terminology (see above), but also in many
details of the paradigm itself. For instance, languages dier in the semantics of multiple
inheritance: languages dier in which method to call in case a method is dened by mul-
tiple superclasses. Other languages disallow multiple inheritance altogether. Only few
languages support OOP generalisations such as a meta-object protocol [Kiczales et al.,
1991] (by which, e.g., the semantics of multiple inheritance can be dened by the user).
These incompatibilities are likely due to the fact that OOP represents a particular com-
plex programming paradigm which misses a solid mathematical foundation in contrast
to other programming paradigms such as functional or logic programming. Neverthe-
less, design patterns [Gamma et al., 1995] a catalogue of higher-order abstractions for
program organisation used by experienced programmers are an example of a unifying
tendency in actual OOP practice.
29
2. Survey I: Issues in Music Representation for Computer-Aided Composition
2.8. Higher-Order Programming for Score Processing
Section 2.6 motivated a data abstraction interface for single score objects, but it is also
important that the user can query and process complex score hierarchies. For instance,
the user may want to transform a hierarchic representation into a at event list (e.g.
to transform the hierarchic representation into a Csound score for output). Often, the
user wants to include only specic events in the output score and exclude others (e.g.
to include only events of a specic instrument or events starting after a specic time
point).
Such a score transformation can be provided by an data abstraction interface function
which recursively traverses the score hierarchy: by visiting all events, the program collects
only the events which meet some specic condition (e.g. collects only events after a
specic start time).
The designer of a music representation does not know which condition the user will need.
The designer may therefore provide many highly similar transformation functions, but
such an approach would violate the concision and modularity design principles (Sec. 2.5).
A more generic interface function allows the user to freely dene this condition by a
test function. Whereas the control structure of the program (i.e. recursively visiting all
events) remains constant, not only some value (i.e. the minimal start time of events) but
some sub-program (i.e. the test function) is given by the user.
The concept of higher-order programming addresses such situations [Abelson et al., 1985;
van Roy and Haridi, 2004]. In functional programming languages which primarily
introduced this concept a function is a rst-class value. A rst-class value can be freely
stored in data structures, can be passed to other functions as arguments, or returned as
results like integers or strings in other programming paradigms. In eect, the concept
higher-order programming introduces subprograms (i.e. functions) as data like anything
else (i.e. a subprogram is a normal data structure like an integer or a string). A function
which expects a function as an argument (or returns a function as result) is called a
higher-order function.
Functional languages usually dene a highly generic and rich interface for the list data
type including several higher-order functions. A typical example is the function lter
which collects only the values of a list for which a predicate a function given as an
argument returns true. For example, calling lter with a list of integers and a function
which checks whether its only argument is even results in a list containing only the even
integers (2.1).
lter([1, 2, 3, 4, 5], isEven) = [2, 4] (2.1)
Further common examples for a generic list interface include the functions map (col-
lecting the results of a unary function applied to each list element), sort (sorting a list
according to a binary predicate used to compare two list elements), nd (returning the
30
2.8. Higher-Order Programming for Score Processing
rst list element for which a predicate returns true), and count (counting for how many
elements a predicate returns true).
In the example at the beginning of the present section, an hierarchic music representation
is transformed into a at event list such that only specic events are included in the
output. The function lter provides the required functionality. Missing is only a function
collectAllObjects which traverses a hierarchically structured score and returns a list with
all score objects contained in the score. Figure 2.12 demonstrates how to extract an event
list from an hierarchic music representation which only contains the events starting after
time point 60 seconds. Whereas the previous example (2.1) used a predened function
(i.e. isEven), this example uses a user-dened function: the function checkEvent checks
whether its argument is an event and whether the start time of this event exceeds a
certain value.
let checkEvent(myEvent)
def
= isEvent(myEvent) getStartTime(myEvent) > 60
in lter( collectAllObjects(myHierarchicScore),
checkEvent)
Figure 2.12.: Extracting an event-list with only specic events (events starting after 60
seconds) from an hierarchically structured score
Section 2.4 introduced the term score context to denote a set of related score objects and
used set-builder notation as a generic device to express such a context. Filtering score
objects is very similar to set-builder notation and a highly suitable means to extract
complex score contexts. For example, the expression in Fig. 2.12 can be transformed
into set-builder notation as shown in Fig. 2.13 (an important dierence between both
expressions is, that a list returned by a higher-order ltering function also expresses the
order of its elements).
x : x belongs to myHierarchicScore
isEvent(x)
getStartTime(x) > 60
Figure 2.13.: Figure 2.12 translated into set-builder notation
The function checkEvent is needed only once in Fig. 2.12. In such a situation, the function
can be dened in place. In the example variant in Fig. 2.14, the function f is dened
in place and given to lter. The colon (:) in the example reads where or such that.
Landin [1966] proposed the where-notation as a substitute for Churchs -notation
[Barendregt and Barendsen, 1991]. When compared with the -notation, the where-
notation has the advantage that it uses the conventional function notation. Landin spells
out where as in
31
2. Survey I: Issues in Music Representation for Computer-Aided Composition
let minTime
def
= 60
in lter( collectAllObjects(myHierarchicScore),
f : f(x)
def
= isEvent(x) getStartTime(x) > minTime)
Figure 2.14.: Extracting an event-list with only specic events (cf. Fig. 2.12) where the
ltering test function is dened in place
g(f where f(x)
def
= function body))
In this thesis, each function is essentially a -expression, that is a rst-class value bound
to a variable. The function f in Fig. 2.14 is even a closure [van Roy and Haridi, 2004]
which consists of the function itself plus the enclosing lexical environment binding the
variable minTime. Such a function denition plays the same role as a lambda expression
in languages such as Lisp and Python or a block in Smalltalk and Ruby.
A function which expects other functions as arguments (i.e. a higher-order function) is
dened like any other function. As a demonstration, Fig. 2.15 denes the function lter
as a recursive function. After an abort condition test (which nishes the recursion at the
end of the list xs), the predicate test is applied to the rst element of xs. In case this
test returns true, this element is added to the accumulated result and lter calls itself
recursively with the rest of xs. Otherwise, this rst element of xs is ignored when lter
calls itself recursively.
lter(xs, test)
def
= if isNil (xs) then nil /* abort condition */
/* Apply test to rst element of xs. Collect only matching elements but recur in any
case with rest of xs */
else if test(head(xs))
then cons (head(xs),
lter(tail (xs), test))
else lter(tail (xs), test)
Figure 2.15.: Denition of the higher-order function lter
Some researchers apply higher-order programming for processing an hierarchic music
representation. For example, Desain [1990] explains this concept in detail in a Lisp pro-
gramming tutorial. Also, some composition systems integrate higher-order programming.
For example, Common Music denes the functions map-subobjects (and its cousin map-
subcontainers) which maps a function f to subobjects (respective subcontainers) of a
given container [Taube, 2005]. Several optional arguments allows the user to control
the behaviour of map-subobjects. For example, with an optional predicate the user
can lter to which subobjects the function f is applied (i.e. all subobjects for which
32
2.9. Conclusion
the predicate returns true). In addition, the user can specify whether map-subobjects
works recursively or processes only the direct subobjects of the given container.
2.9. Conclusion
The representation of music is a complex task. A music representation expresses a large
amount of explicit information which constitutes a score. A representation often provides
means to conveniently access explicitly stored information as well as information which
can be derived from it. The information to represent is inuenced by several factors, for
example, the style of the music or music theory concepts. This can be seen, for instance,
in the complex subeld of representing musical parameters such as pitch and also in the
dierent schemes of hierarchic representations proposed.
This survey proposed the term score context to denote a set of related score objects
and discussed the dierent ways shown in the literature to explicitly store and derive
information required to access such contexts (e.g. by grouping the elements of a context
in a container or by ltering a score for objects which satisfy a test).
A generic music representation requires that it can represent a wide range of dierent
information (e.g. music of dierent styles). Such a representation can be conveniently
extended by the user to express information not supported by predened constructs.
To full these demanding requirements, the eld of music representation calls upon a
range of dierent computer science concepts. The concept of data abstraction is highly
important for any non-trivial music representation. Programming paradigms such as
object-oriented programming and higher-order functional programming allow the ex-
pression of complex musical knowledge in a concise way.
33
2. Survey I: Issues in Music Representation for Computer-Aided Composition
34
3. Survey II: Composing with Constraint
Programming
This chapter introduces the notion of rule-based composition with constraints by re-
viewing musical constraint satisfaction problems and systems which either implement a
specic problem or allow the user to dene such problems.
Chapter Overview
Section 3.1 explains the main idea of constraint programming in an informal way. The
application of constraint programming for music composition is motivated in Sec. 3.2,
rst in general terms and then illustrated by examples from the literature of this research
eld. Section 3.3 surveys generic music constraint systems in which the user denes and
solves own rule-based composition systems. A short conclusion closes this chapter (Sec.
3.4).
3.1. What is Constraint Programming?
Constraint programming is a programming paradigm which introduces techniques to
solve constraint satisfaction problems. A constraint satisfaction problem (CSP) consists
of a set of variables and mathematical relations between these variables which are called
constraints. Usually, a CSP presents a combinatorial problem. A constraint solver nds
one or more solutions for the problem. A solution of a CSP shows for each variable of the
problem a determined value which is consistent with all constraints. A constraint system
(e.g. a programming language supporting constraint programming) allows its user to
dene and solve CSPs.
A simple numeric example may illustrate these concepts. The example introduces the
two variables X and Y and restricts their value by two basic arithmetical operations
(connected by a conjunction).
1
One possible solution for this problem is X = 3, Y = 4,
another solution is X = 1, Y = 6.
X +Y = 7 X < Y
In mainstream programming paradigms or with mainstream programming languages
(e.g. C or Java) this problem is even not expressible at all in the concise way it is given
1
As a convention, constrained variables are written with upper-case letters, see App. A.
35
3. Survey II: Composing with Constraint Programming
here, let alone solve it. To solve a CSP automatically, a special software is needed: a
constraint solver.
The reference to programming languages such a C or Java reveals conicting terminology:
in constraint programming, the term variable has a clearly dierent meaning compared
to its meaning in mainstream programming languages. In mainstream programming
languages, a variable denotes a stateful computational entity: such a variable has always
a specic value and a program can alter the value of a variable with an assignment
statement at any time.
In constraint programming, the notion of a variable is more similar to the notion of a
variable or unknown in mathematics. More specically, a variable in constraint program-
ming is very similar to a logically quantied variable in rst-order logic [Kelly, 1997].
In constraint programming, a variable never changes its value. However, the value of a
variable may only be partially known.
The remainder of this text will always use the term variable in this latter meaning (as
long as not explicitly noted otherwise). Sometimes, the term constrained variable will be
used to explicitly denote a variable in the context of constraint programming.
A constrained variable is a variable which has a domain, that is a set of values it may
take in a solution. Some constraint systems support variables with innite domain (e.g.
the domain of all real numbers in some interval). However, in the remainder of this text,
the domain of a variable is usually a nite set of possible variable values. The values
in the domain are often all of the same type (e.g. boolean domain, integer domain or
domain of nite sets of integers), other systems support domains with mixed types.
As already mentioned, a variable in constraint programming never changes its value
(much like a variable in rst-order logic). The domain of a variable will be reduced
during the search process, but the domain will not change in any other way. That is
to say, a variable is stateless, the information available about a variable will only be
amplied.
2
In principle, constraints can be arbitrary mathematical relations. Examples include nu-
merical relations, set relations, logic relations, and tree or graph relations. Constraint
systems predene a set of constraints and often allow the user to extend this set.
Solving a CSP requires searching. Because the search space that is the set of (partial)
solution candidates of a CSP is often huge, an ecient constraint solver has great
impact on the usability of a constraint system.
There exists a whole library of literature on constraint programming. Only important
introductions are cited here for further references. Apt [2003] provides a general overview
of the eld with many CSP examples. Fr uhwirth and Abdennadher [2003] survey dierent
constraint programming approaches and systems. Dechter [2003] primarily explains how
2
In the actual implementation of some systems discussed below, however, the concept of the variable
and its domain is somewhat decoupled. In such systems, during search the variable is indeed statefully
bound to dierent values of the domain before constraints are applied.
36
3.2. Constraint-Based Composition
constraint solvers nd solutions. A less formal introduction into the eld is given in
the web-tutorial by Bartak [1998] which also links to further resources such as related
journals, conferences, web-links etc.
3.2. Constraint-Based Composition
3.2.1. Motivation
A rich set of strategies for computer-aided composition have been proposed. Section 1.1
listed several examples.
Often, a specic composition strategy can be formalised in dierent ways. For example,
a transformation of existing data can be formalised in the procedural way by imperative
program statements which change this data. Alternatively, this transformation can be
formalised declaratively by a function which expects some data and returns the trans-
formed data without changing the original data. If a strategy can be formalised both
procedurally and declaratively, then the declarative formalisation is often preferable be-
cause declarative programs are compositional and more easy to reason about (cf. [van
Roy and Haridi, 2004]).
A declarative approach is also often used in the eld of music theory. Prominent examples
are the explanations in composition textbooks (e.g. discussing counterpoint, harmony,
musical form or instrumentation). These textbooks primarily specify the properties of a
suitable result without detailing how to procedurally achieve this outcome.
An important device to declaratively express compositional knowledge in a music theory
is the compositional rule. A rule restricts score objects (e.g. sets of notes) and their
parameters (e.g. durations or pitches). Frequently, a rule controls mutual dependencies
between multiple score objects and parameters.
Compositional rules are particularly well suited to describe music, because rules describe
the multi-dimensional nature of music in a modular way. When listening to music, we
perceive various aspects such as rhythm, harmony, voice leading or instrumentation
simultaneously. During the compositional process as well, the composer builds up a
complex network of relationships between all the musical elements, and observes these
elements from various viewpoints more or less simultaneously. Similarly, the formalisation
of a task as complex as composition is greatly simplied when the task is stated in a
modular way. When the task description is broken down into rhythmic rules, melodic
rules, rules on the harmony etc. then the various musical dimensions are formalised one
by one.
It is very dicult to procedurally build up such a network of relations between the
various musical elements of a piece. In addition, changing or adding a single rule can
require redesigning the entire program.
37
3. Survey II: Composing with Constraint Programming
Because rules are well-suited to describe music, rule-based approaches have been pro-
posed since the early days of computer-aided composition. For example, Hiller and Isaac-
son [1958] already formalised counterpoint rules when composing movements (called
experiments) for their famous Illiac suite.
Constraint programming (see Sec. 3.1) has proven a particularly successful programming
paradigm to realise ruled-based systems. Many rule-based composition systems have been
proposed which make use of constraint programming techniques. Several of these systems
will be introduced shortly.
This thesis uses the term music constraint system to denote a system which applies con-
straint programming for musical purposes (e.g. assisting a composer in the composition
process or a scholar in analysing music). A musical constraint satisfaction problem (mu-
sical CSP) implements a formal music theory model (Sec. 1.2) by a set of constraints on
a set of variables.
In contrast to most CAC systems, a music constraint system user creates a musical score
by formally describing a desired result only. Thus, constraints programming frees the
composer to concentrate on what he wants to do in a musical sense; he does not need to
dene how to achieve this outcome.
A musical CSP does not necessarily result in only a single solution. Instead, the restric-
tions and dependencies expressed by a set of rules reduce the set of solution candidates.
A particular advantage of a music constraint system lies in the fact that it allows the
composer to dene compositional rules in a modular fashion. Multiple rules can even
aect the same parameter value. For instance, a musical CSP may restrict the note
pitches of a score by melodic rules on the one hand and by harmonic rules on the other
hand. Each of these separate rules aect the same parameter values, namely the pitches.
However, no rule necessarily determines the parameter values fully. Search nds one or
more solution which fulls all rules.
Many of the examples presented in this and the following chapters mainly constrain note
pitches in some way. In other algorithmic composition strategies, it is often dicult to
adequately address the complexity required to model pitch structures. A rule-based ap-
proach (e.g. constraint programming) is highly suited for this specic task. On the other
hand, other algorithmic composition strategies are well applicable for other composi-
tional tasks, for example, to generate complex trajectories for parameters which control
arbitrary sound synthesis details or the spatialisation of sounds. It therefore makes sense
to complement these dierent strategies and to create dierent aspects of the music by
dierent strategies. Yet, the present text focusses on pure constraint-based composition
for brevity.
3.2.2. A First Example
In the following, a simple (but non-trivial) musical constraint satisfaction example is
described. The example was chosen for its brevity, so it can be discussed in full detail.
38
3.2. Constraint-Based Composition
The example stems from dodecaphonic music composition. In this technique, the com-
poser organises the pitches in a composition with the help of a tone row (also known as
twelve-tone series) and its transformations. Krenek [1952] provides a practical introduc-
tion to dodecaphonic composition (also known as twelve-tone technique).
A tone row is a sequence of twelve tone names of the chromatic scale or twelve pitch
classes, in which each pitch class occurs exactly once. The tone row is used as a device
to achieve music which is coherent in itself even if the music abandons conventional
harmony.
Aiming for a music which is most coherent in itself, several composers also shaped the
tone rows themselves with great care. A prominent example for such special tone-rows is
the set of all-interval series. In an all-interval series, also the eleven intervals between the
twelve pitches are all pairwise distinct (i.e. each interval occurs only once). Therefore,
such a series is sometimes also called an eleven-interval series [Krenek, 1952]. Other
examples for special tone-rows are symmetric rows, in which some part of the row is a
strict transformation of another part.
Figure 3.1 shows an all-interval series example (cited from [Gervink, 1995], but this
example can also be found in [Krenek, 1952]). In this particular case, the series is even
both an all-interval series and a symmetric series. It can be seen that each pitch class
occurs only once in the series. Similarly, also every interval between the pitches (reported
by integers above the sta) is unique. These intervals are computed in such a way that
they are inversional equivalent: complementary intervals such a fth upwards and a
fourth downwards count as the same interval (namely 7).

11

10

6

Figure 3.1.: All-interval series, whose two halves also form transposed retrogrades of each
other (cited from [Gervink, 1995])
The rest of this subsection provides a full all-interval series formalisation as demon-
stration how a musical CSP may look like. An all-interval series obeys only very few
constraints. Therefore, it is often discussed as a CSP example. For instance, [Laurson,
1996, p. 209 f] exhibits an all-interval series implementation.
Figure 3.2 provides the all-interval series CSP denition in mathematical notation. The
example rst creates a music representation (consisting of two lists) and binds it to the
variables pitchClasses and intervals. The constrained variables in this problem are the
elements in these lists. The domains for these constrained variables consist of integers.
In the following, the example applies several constraints to these variables. All these
constraints are connected by conjunctions. This denition has the 3856 solutions known
from the literature [Morris and Starr, 1974].
39
3. Survey II: Composing with Constraint Programming
pitchClasses
def
= a list of 12 undetermined integers, each with the domain 0, . . . , 11
intervals
def
= a list of 11 undetermined integers, each with the domain 1, . . . , 11
/* Constrain the relation between the pitchClasses and the intervals. */
11

i=1
inversionalEquivalentInterval (pitchClasses
i
, pitchClasses
i+1
, intervals
i
)
/* All elements in the pitchClasses as well as the intervals between them are pairwise distinct. */
distinct(pitchClasses)
distinct(intervals)
/* The rst series pitch is determined to 0 (i.e. note name c) to prohibit series transpositions. */
pitchClasses
1
= 0
/* The last pitch is set to 6, because the interval between the rst and the last interval in an all-
interval series must be a tritone [Krenek, 1952, p. 50]. This additional constraint is redundant, but
dependent on the constraint solver it may reduce the search space considerably. */
pitchClasses
12
= 6
Figure 3.2.: All-interval series denition
Compared with the music representations discussed before (see Chap. 2), the music
representation of this example consisting of two lists of parameter values is rather
simple. The list pitchClasses represents the sequence of pitch classes, that is the all-
interval series solution. The list intervals represents the sequence of the intervals between
these pitch classes.
Each pitch class is represented by an integer between 0 and 11 (i.e. between c and b).
Each interval is represented by an integer between 1 and 11 (0 would be unison or an
octave: these intervals do not occur in a twelve-tone row). In the CSP denition, the
values of the pitch classes and the intervals are undetermined they will only be found
during the search by the constraint solver.
The rest of the example consists of a conjunction of constraints on this music represen-
tation. First, the relation between all pairs of consecutive pitch classes and the intervals
between them is restricted by the constraint inversionalEquivalentInterval . This con-
straint is dened in Fig. 3.3. Similar to the wellknown computation of a pitch class from
a key-number, the computation of an inversional equivalent interval between two pitches
makes use of the modulo function.
After the application of inversionalEquivalentInterval in Fig. 3.2, the constraint distinct
demands that all variables in the list pitchClasses are unique. The same constraint is
applied to the list intervals. Finally, two additional constraints determine the rst and
the last pitch class in the series to prohibit series transpositions and to amplify the
40
3.2. Constraint-Based Composition
/* Constraints Interval to be an inversional equivalent interval between the two pitch classes PC1
and PC2 (i.e. a fth upwards and a fourth downwards count as the same interval). */
inversionalEquivalentInterval (PC1, PC2, Interval )
def
=
Interval = (PC2 PC1) mod 12
Figure 3.3.: Denition of the constraint inversionalEquivalentInterval
information available to the constraint solver.
Morris and Starr [1974] present a specially developed algorithm to compute all-interval
series. Nevertheless, music constraint programming has the advantage that the user only
needs to declaratively state the problem without developing a special algorithm, whereas
the latter is often a very time-consuming activity. For example, the constraint system
user may easily add additional constraints which requires the solution to be both an
all-interval series and a symmetrical series (as the example row in Fig. 3.1). Using an
algorithmic approach often requires a laborious algorithm redesign when the problem
specication changes. On the other hand, a specially designed algorithm is often more
eciently executed.
3.2.3. Musical Constraint Satisfaction Problems
Many music theories have been modelled and implemented by constraint programming.
The present subsection lists research projects conducted to demonstrate the variety of
musical CSPs tackled so far.
Counterpoint
There exists a considerable body of literature on counterpoint. Dierent textbooks often
cover dierent musical styles. Today, two style families are taught most frequently. One
approach is oriented at Renaissance music and Palestrina in particular. Fux [1725] wrote
the seminal treatise on this approach; the standard textbook on Palestrina counterpoint
today is [Jeppesen, 1930]. Whereas in the rst approach harmonic considerations are sec-
ondary, the other approach teaches how to compose polyphonic music which expresses
a harmonic progression. Baroque music, for example, usually follows this approach. Im-
portant textbooks on this approach are [Schoenberg, 1964] and [Piston, 1947]. Finally,
some authors teach several dierent contrapuntal styles [Motte, 1981].
The 20th century saw further developments of polyphonic music; important examples
include the dodecaphonic technique of Schoenberg and others, its descendants like seri-
alism or Stockhausens formula composition, the micropolyphony developed by Ligeti,
and Nancarrows rhythmical counterpoint.
There exist several systems creating polyphonic music by means of constraint program-
ming. Scholastic counterpoint (e.g. the strict application of [Fux, 1725]) features a par-
41
3. Survey II: Composing with Constraint Programming
ticular strict set of rules (when compared with other music theory sub-disciplines, e.g.,
rhythm or melody). A strict rule set makes formal modelling more easy, which explains
why counterpoint has been of great interest for designers of rule-based systems.
Ebcioglu [1980] proposes a system for creating two-part orid counterpoint: to a given
cantus rmus the system composes a matching voice which is rhythmically independent.
The author lists almost 50 implemented counterpoint rules which include complex high-
level rules such as the pitches of dierent local maxima (i.e. melodic peaks) within three
measures of the voice are unique. Sources for compositional rules were Joseph Marx and
Charles Koechlin. Because their rules were insucient for automatic composition, the
author added rules of his own. The search strategy embeds heuristics which prefer steps
to skips and note pitches that have not occurred before.
A system for creating species counterpoint was introduced by Schottstaedt [1989], who
aimed to follow the rule set of Fux [1725] as closely as possible. The system implements
all ve species for up to six voices. However, the author modied the original Fuxian
rule set (more then 40 rules are quoted in article) to get closer to Fux actual examples.
In accordance with music theorists (including Fux) which state that rules are merely
guidelines and no absolutes, the system assigns each rule a numeric penalty value to de-
note its relative importance: the system searches for a solution with a small accumulated
penalty.
Polyphonic music in the style of Josquin Desprez is addressed by Laurson [1996], who
implemented several rules from the Josquin chapter of Motte [1981]. The goal of this
research is not so much to simulate a specic historical style, but to study the problem
of polyphonic CSPs in general. The resulting system PWConstraints and its subsystem
score-PMC is further discussed in Sec. 3.3.1.
Also non-European counterpoint-like music theory models have been discussed in the
literature. Chemillier and Truchet [2001] introduce the Nzakara harp problem. This
CSP models a two-voice canon following rules which are typical for the Nzakara harp
repertoire (from the Central African Republic).
Dense musical textures play an important role in the music of the 20th and 21st century.
A musical CSP which resembles Ligeti-like micropolyphony was proposed by Laurson
and Kuuskankare [2001] and another by Chemillier and Truchet [2001].
Harmony
There also exists a huge amount of literature on the subject harmony. Authors dier less
in their focus on a certain style the history of harmony shows a more continuous de-
velopment when compared with counterpoint (where dierent styles are more distinct in
their rule set). However, there exist dierent approaches to explain harmonic phenomena.
Most authors describe harmonic progressions as progressions of chord roots and anal-
yse all chords in terms of their relationship with the tonic (this idea draws on Rameau
[1722]), but dierent conclusions are drawn. One approach is based on the assumption
that chord roots indicate one of the seven (major or minor) scale degrees (notated by
42
3.2. Constraint-Based Composition
roman numerals). Schoenberg [1911] wrote a particular accomplished textbook using
this approach. Another approach (often called functional harmony) only accepts three
dierent main harmonic functions (tonic, dominant and subdominant; notated usually
with their initials) and explains all chords as variants of one of these main functions.
This approach was founded by Hugo Riemann and is used, for example, by Motte [1976].
Finally, some authors including Schenker [1935] and Schoenberg [1969] put particular
emphasis on larger-scale structures in harmonic progressions.
The development of harmony and its study is still ongoing. For example, the inuential
work by Forte [1973] describes the harmonic language of atonal music
3
(i.e. music in
12-tone equal temperament without a tonal centre and often consisting of highly complex
chords) in terms of pitch class sets.
Microtonal music, and in particular music in just intonation is another important ex-
ample for the ongoing development. Music in just intonation is music based on a tuning
of intervals whose frequency ratios can be represented by relatively small integers (e.g.
3
2
constitutes the fth and
5
4
the major third). Whereas common practice uses only in-
tervals consisting of integers whose highest prime factor is 5 (often called 5-limit), 20th
century music greatly extended this set of intervals by new and consonant intervals (e.g.
by the harmonic seventh
7
4
or the harmonic augmented fourth
11
8
), as well as intervals
more dissonant than available in common practice. The term extended just intonation
is often used to label tuning systems which contain intervals beyond the 5-limit. The
seminal work on just intonation is by Partch [1974], Doty [2002] wrote a tutorial on the
subject, and an extensive encyclopedia on just intonation theory is presented by Monzo
[2005].
Much research has been carried out on constraint based harmonisation. Pachet and Roy
[2001] provide a survey on this subject.
Chorale, designed by Ebcioglu [1984, 1992], is a system which creates four-part harmon-
isations of given choral melodies in the style of Johann Sebastian Bach. Chorale received
much attention for the musical quality of its output: according to its author, Chorale
accomplished the competence of a talented music student. Indeed, the music created by
this system can be much more complex than the output of most other harmonisation
systems discussed here (e.g. most authors do without modulations). Ebcioglus detailed
analysis of compositions by Bach resulted in the impressive amount of about 350 rules
implemented by the system. These address two subtasks: the harmonisation (creating
of chord skeleton, style-appropriate modulation and cadencing) and melody generation
(with special care of the outer voices). To meet his purpose in an ecient way, Ebcioglu
rst designed and implemented the constraint programming language BSL (Backtracking
Specication Language).
The often-cited article by Tsang and Aitken [1991] proposes a lucid system with a small
set of 20 rules which creates four-part harmonisations of a choral melody.
3
Schoenberg who was the rst to radically break out of traditional tonic-related harmony detested
the term atonal music; he preferred pantonal music instead.
43
3. Survey II: Composing with Constraint Programming
The system designed by Ramirez and Peralta [1998] also automatically harmonises a
given melody with an appropriate chord sequence. The system nds a sequence of ab-
solute chord names such as [C, Dm, G, C],
4
whereby the system is limited to single key
melodies and only considers diatonic triads in the solution. To increase the musical
quality, the system further constrains solutions to follow standard chord patterns (e.g.
[I, II , V, I]) stored in a database.
Coppelia [Zimmermann, 2001] creates homophonic chord progressions which also feature
a rhythmical structure. The music theory model is split into two layered sub-models
which are implemented by two independent applications. The subsystem Aaron creates
a harmonic plan, represented by harmonic functions in the tradition of Hugo Riemann
(such as [T, S
3
, D
7
, T]) and complements this plan by additional information (e.g. the
duration of each chord and further restrictions on single voices such as the soprano
melody shall move downward). COMPOzE [Henz et al., 1996], the second subsystem,
creates the actual four-voice chord progression from this harmonic plan.
Phon-Amnuaisuk [2001, 2002] presents another system which creates choral harmon-
isations in the style of Johann Sebastian Bach. Phon-Amnuaisuk criticises Chorale
[Ebcioglu, 1992], that this system is hard to modify. To realise a more adaptable con-
straint system design, he proposes a control language which controls the temporal order
of decisions during the composition process. For the four-voice Bach choral example,
the search process may rst create the harmonic skeleton for the given melody, then
outline the bass skeleton, create a draft of the other voices, and eventually create the
nal version of each voice by adding ornamentations such as passing notes.
A CSP based on dodecaphonic pitch class sets is described by Laurson [1996]. In this
CSP, a solution consists of a sequence of (possibly overlapping) pitch class sets. These
results could be used by a composer to organise the harmonic structure of music.
Melody
Melody-writing is highly style-dependent and is not a traditionally established subject
in music education. Nevertheless, the subject is covered, for example, by some textbooks
on composition in general. For instance, Schoenberg [Schoenberg, 1943, 1967, 1995] (the
three textbooks are sorted according to their intended audience from entry-level to ad-
vanced) explains how in classical music a melody expresses the underlying harmony and
how a melody is composed from motifs and their variations. Whereas Schoenberg teaches
melody composition in a more systematic way, Motte [1993] studies various aspects of
melodies from dierent musical styles (ranging from Gegorian chant to Ligeti, including
childrens and political battle songs).
The work of Lothe [1999] constitutes one of the few literature examples on rule-based
melody composition. Lothes system creates minuet melodies (in early classical style)
over a given harmonic progression. The author describes several example rules (based on
4
The authors forbeared to formalise melodic and voice-leading rules as well: the system only creates a
chord name sequence.
44
3.2. Constraint-Based Composition
several sources including music theory literature from the classical period, for example,
[Koch, 1793]) in detail and demonstrates the eect of dierent rule sets with musical
examples.
Rhythm
Over centuries, western music focused on developing the pitch structure instead of
rhythm, which may be the reason why the rhythmic aspect was also largely neglected
by music theory. In one of the rare textbooks on the subject, Cooper and Meyer [1960]
clearly dene rhythmical terms (such as pulse, meter, rhythm, accent, stress, tie, synco-
pation, and suspension) and explain their relations in a theory of rhythm which studies
the hierarchical nature of rhythmical organisation.
Yet, the 20th century saw considerable developments of the rhythmical aspect. Exam-
ples include the additive rhythmical permutations of motifs resulting in changes of
the metric structure introduced by Stravinsky and later further developed by Messiaen
[Messiaen, 1944], Nancarrows already mentioned rhythmical counterpoint, the phasing
technique in the music of Reich, and the astonishingly complex rhythmical structures in
the work of Ferneyhough.
Some musical CSPs in the literature are of a purely rhythmical nature. Truchet et al.
[2001] propose a poly-rhythmic problem in which each voice literally repeats a rhythmical
pattern but common onsets between the voices are avoided.
A system completely devoted to rhythmical CSPs is OMRC [Sandred, 2000a,b, 2003]. For
example, Sandred [2004] proposes a rule-based quantication of the rhythms of every-
day gestures (e.g. extracted from the sound of a passing train) and forces these gestures
into readable music notation. Rules may control what time signatures are allowed and
how often the time signature may change. Additionally, the composer may apply further
constraints. For example, the composer may demand that the quantied result will be
build from pre-composed motifs.
Instrumentation
Berlioz and Strauss [1904] wrote a seminal book on orchestration (orchestration includes
instrumentation and additionally addresses how to balance groups of instruments). In-
strumentation and orchestration are also still developing today. An important example
is the music of Lachenmann, which introduced many new sounds by novel playing tech-
niques. One could even argue, that electro-acoustic music [Ungeheuer, 2002] or computer
music [Roads, 1996] mainly provide means which resemble new musical instruments (e.g.
hard disc recording a software which allows its user to organise sound snippets in time
can be perceived as a new instrument for creating music) which leads to new instru-
mentation techniques.
To accomplish idiomatic instrumental writing, Laurson and Kuuskankare [2000, 2001]
present musical CSPs in which melodic, harmonic and voice leading rules are comple-
mented by instrumentation rules. The authors discuss guitar and brass instrument n-
gering in two case studies. For example, writing music for the guitar in a way which is
45
3. Survey II: Composing with Constraint Programming
well playable requires instrument-typical considerations. Guitar music is performed on
six strings tuned in a particular way on which only four left-hand ngers are placed.
Furthermore, the ngers can only be stretched up to a certain amount and moving the
ngers requires a certain amount of time.
Multi-Media
All musical CSPs discussed so far create scores (or score excerpts) as solutions. The
following CSPs do not fall into this category, but still contribute to music.
Midispace [Pachet and Delerue, 1998] realises sound mixing and spatialisation with con-
straint programming. In real-time, a listener can control the position of sound sources
by a graphical user interface. The system ensures a consistent mix which follows several
constraints.
Ferrand et al. [1999] propose a system which assists an optical music recognition process
(i.e. the automatic recognition of scanned sheet music) in dealing with uncertain and
missing information. Here, constraint programming allows the declaration of musical
knowledge on a high level of abstraction, which can enhance the recognition process to
detect and eventually correct recognition errors of polyphonic music.
RecitalComposer [Pachet et al., 2000] is a system which does not create single pieces of
music but instead creates sequences of pieces forming a music program. To this end, title
attributes are stored in a database (e.g name and author, the duration, describing style,
instrumentation, and tempo). Constraints on a solution sequence of titles aim to balance
the amount of repetition and surprise for the listener. Example constraints are no slow
music, 40 percent of the music is for brass, all authors all dierent, or succeeding
titles are similar in style.
Similarly to image mosaicing, musaicing [Zils and Pachet, 2001] builds a musical se-
quence by specifying global properties of the solution sequence, and letting the system
select and sequence automatically the sound samples. For example, a Beatles song could
be recomposed out of many small fragments from famous rock titles of the sixties.
Zhong and Zheng [2004] propose a melody representation which nds a melody quickly
in constraint-based database when a fraction of the melody is given.
3.3. Generic Music Constraint Systems
Much research has been conducted which employs constraint programming to implement
music theory models as the list of systems in Sec. 3.2.3 demonstrates. Most of these
systems were designed exclusively to solve a single CSP or a small range of problems
(e.g. the automatic harmonisation of a given melody which complies with a xed set of
rules).
Although some authors report that it is not particularly dicult to write such a program
[Schottstaedt, 1989], the design of a system which solves a complex CSP with reasonable
46
3.3. Generic Music Constraint Systems
eciency is demanding. For example, Ebcioglu [1992] deemed it necessary to rst develop
a new programming language for this task (namely BSL).
Moreover, music constraint systems share important requirements. Firstly, any music
constraint system requires some constraint solver. Secondly, domain-specic CSPs share
a considerable amount of domain-specic knowledge: all musical CSPs require modelling
of musical knowledge. For instance, concepts such as note, pitch, or voice are required
in a large number of musical CSPs. Consequently, several more generic systems were
proposed since the early nineties.
This thesis introduces the term generic music constraint system to indicate a system
which is designed to solve a considerable number of musical CSPs in contrast to a
system designed specically to solve a single CSP or a small set of problems. A generic
system is often developed for users such as composers and music theorists who would
hardly consider the design of a new rule-based system from scratch, but can greatly
contribute to the research in this eld.
A generic music constraint system allows these users to dene and solve their own musical
CSPs. In order to be more generic, these system usually aim to be style-neutral (much
like general computer-aided composition environments as OpenMusic or Common Music
aim to be style-neutral).
A pioneering generic music constraint system is Carla [Courtot, 1990]. This system em-
phasises its music representation which features an extensible type system. Constraints
complement this representation. The system features a visual logic programming lan-
guage to make it accessible to non-programmers (such as composers) and is implemented
in a Prolog dialect (Prolog II).
The remainder of this chapter introduces several generic music constraint systems. Three
such systems are presented in detail (Sec. 3.3.1 to 3.3.3) to introduce important concepts
of music constraint programming and to demonstrate how typical requirements on such
a system have been answered in dierent ways. Furthermore, these systems gained par-
ticular importance as they had/have a relatively large number of users. Section 3.3.4
lists further systems.
3.3.1. PWConstraints
PWConstraints [Laurson, 1996; Rueda et al., 1998; Laurson, 1999] is a constraint pro-
gramming language on top of PatchWork [Laurson, 1996; Assayag et al., 1999], a visual
programming language for computer-aided composition.
This present section describes PWConstraints in much detail for two reasons. PWCon-
straints constitutes a ne example for introducing important concepts of constraint-based
computer-aided composition in general. Moreover, PWConstraints served as a role model
for the present research.
PWConstraints consists of two main music constraint systems which share a similar
search strategy. A general constraint programming language oers means to constrain
47
3. Survey II: Composing with Constraint Programming
common data-types, in particular lists of integers (Sec. 3.3.1.1). A special constraint
programming language for polyphonic music oers means to constrain a score repre-
sented in a more complex music representation (Sec. 3.3.1.2). Section 3.3.1.3 discusses
PWConstraints search strategy.
A Side Note: Visual Programming Languages
PatchWork programming constructs present themselves as graphical boxes, which is typ-
ical for many visual programming language (e.g. this is also the case for the widespread
system Max [Puckette, 2002] and its relatives jmax and Pd). Consequently, each main
part of PatchWork comes as such a box. The user interface of the general constraint pro-
gramming language is the box PMC. The box score-PMC is the interface for the special
constraint programming language for musical scores. However, PWConstraints comple-
ments its graphical interface by a textual interface: the main input to the PWCon-
straints boxes the actual constraints are dened by textual Lisp code, using special
functions/macros provided by PWConstraints.
A main purpose of the visual programming paradigm [Boshernitsan and Downes, 2004]
is to make the programming more easy for the user, in particular for programming
beginners. However, not every program becomes more easy to write in a visual way.
Visual programming is highly suited for control ow languages with concurrency (such
as Max). Mathematical relations which are the bread-and-butter of compositional
rules are often more concisely expressed by textual formulae. Similarly, the pattern
matching mechanism provided by PWConstraints (see Sec. 3.3.1.1) can only awkwardly
be expressed visually.
Interestingly, visual programming systems for music are usually not dened in them-
selves but in textual languages. This is in clear contrast to other high-level languages
(e.g. Common Lisp, or Smalltalk) which dene much functionality in themselves. This
dierence clearly hints that the visual programming paradigm does not scale up for
larger projects in general.
On the other hand, visual programming is well suited for the top-level denition of an
application. When compared with a textual interface, a graphical user interface (GUI)
is much more convenient for daily use of an application. Yet, the programming logic
of a GUI is usually xed in contrast to a user interface which consists of a textual
programming language. Visual programming unites the convenience of a GUI with the
exibility of a programming language. PWConstraints follows this pattern: the user
denes important components of a musical CSP (e.g. the rules) in a textual programming
language, but combines them in a visual language. This approach can facilitate large scale
changes of the CSP during the composition process (such as enabling and disabling of
rules).
Further music constraint systems with support for visual programming are Situation
and OMClouds which are both introduced below (Sec. 3.3.2 and Sec. 3.3.3). Also the
visual language Cordial [Rueda et al., 1997] (supporting constraint programming and
48
3.3. Generic Music Constraint Systems
object-oriented programming) has already been applied for music constraint program-
ming [Rueda et al., 2001]. Burnett [2006] provides a bibliography of research on visual
languages, including visual logic and constraint languages.
3.3.1.1. The General Constraint Programming Language: PMC
The Music Representation
The basic concepts of PWConstraints are explained here by sketching a simple example
CSP which will be by and by extended later. In this example, a composer wants to
create a choral melody. The composer decides that the melody consists of 9 notes and
all notes are of equal duration. All melody pitches are situated in the interval between
a3 (an octave below concert pitch) and a4 (concert pitch). In PWConstraints, pitches
are represented by their respective MIDI number. The MIDI pitch representation of
the concert pitch a4 is 69, a3 is represented by 57. Each variable has thus the domain
57, . . . , 69.
The general constraint programming language of PWConstraints (i.e. PMC) is well suited
for constraining a list of integers. Because the said composer is only interested in the
note pitches, a music representation in which the solution consists of a plain sequence of
MIDI-pitches (i.e. integers) is sucient. The composer thus creates a list of 9 constrained
variables with integer domain. In the PWConstraints documentation, the term search
variable is used for constrained variable.
In PMC, all variables are declared together in a list of domain specications (Fig. 3.4). Ac-
cording to PWConstraints terminology, the user denes the search space when declaring
the variables which serve as the music representation. When no further constraints are
applied, each combination of nine pitches in the specied domain is a solution (9 variables
each with a domain of 12 values have 12
9
= 5159780352 solutions in total).
[57, . . . , 69, 57, . . . , 69, 57, . . . , 69, 57, . . . , 69, 57, . . . , 69,
57, . . . , 69, 57, . . . , 69, 57, . . . , 69, 57, . . . , 69]
Figure 3.4.: Declaring the domains for the melody pitches
The user may already determine certain variables in the CSP denition. For example,
the user may specify that any solution begins and ends with a certain pitch by specifying
this pitch as the only domain value for these variables.
In general, the music representation format of PMC consists of a list containing con-
strained variables. PMC supports variables with universal domain. Whereas in the ex-
ample the domain of these variables was a user-specied set of integers, a variable domain
can in fact contain arbitrary values. For example, variable domains often consist of lists
of integers.
49
3. Survey II: Composing with Constraint Programming
The user can interpret variable values in various ways. In the example above, a sequence
of integers were interpreted as the MIDI-pitches of a melody. Alternatively, such a se-
quence can be a sequence of duration values. A list of lists of integers can be interpreted,
for example, as a chord sequence (where a list of integers represents the chord pitches
by MIDI numbers without specifying the chord duration) or as a sequence of rhythmic
motifs (where a list of integers represents a sequence of note durations).
The Rule Formalism
In PWConstraints, a CSP is dened by complementing the declaration of the search
space by compositional rules which pose restrictions on a solution. Figure 3.5 reveals
that the declaration of the variables of the CSP and the rules constraining these vari-
ables are given separately to the constraint solver. Handing multiple rules to the solver
implicitly expresses a conjunction of all these rules. The actual variables are created by
the system automatically. The PWConstraints user accesses variables only within a rule
denition. This design has important consequences (e.g. it is not possible to directly
apply a constraint to variables).
constraint
solver
domain
specifications
rules
solution(s)
Figure 3.5.: In PWConstraints, the variable domain specications and the rules of a CSP
are handed independently to the constraint solver
In PWConstraints, the body of a rule is an expression which returns a boolean value.
Figure 3.6 shows an example of such a rule body which restricts the interval between
two successive pitches (Pitch
predecessor
and Pitch
successor
) to a fth as maximum.
5
Be-
cause the pitches are encoded numerically, the composer can dene numeric relations
between them. The example introduces a new constrained variable Interval and applies
a conjunction of two constraints.
PWConstraints introduces a pattern matching mechanism to control to which variables
in the CSP a rule is applied. This mechanism also binds the free variables in the rule
body. For example, the constraints in Fig. 3.6 can be applied several times to the choral
5
This rule is a simplied version of a Palestrina-style counterpoint rule which is reected by many
counterpoint treatises. In its strict form, the rule prohibits all diminished and augmented melodic
intervals and permits downwards only intervals from minor second to fth plus octave and upwards
all these intervals plus the minor sixth [Jeppesen, 1930].
50
3.3. Generic Music Constraint Systems
let Interval
in Interval = Pitch
predecessor
Pitch
successor
Interval 7, . . . , 7
Figure 3.6.: The body of a rule which restricts melodic intervals
melody notes such that Pitch
predecessor
and Pitch
successor
are bound to every pair of
successive melody pitches.
Pattern matching is well-known, for instances, from the UNIX shell. In the UNIX shell,
a pattern is used to express a character sequence: for instance, a pattern as *test.txt
matches the le names mytest.txt as well as my-other-test.txt.
The pattern matching mechanism of PMC introduces a language to express the position
of elements in the sequential music representation. This language consists of only three
constructs.
6
There are two place-holders and a pattern-matching variable declaration.
The place-holder symbol ? matches exactly one sequence element, and the symbol
matches zero or more elements. The third construct species pattern-matching variables
(which provide bindings for the free variables in a rule body denition). A pattern-
matching variable is bound to the the element at its position in the sequence (all variable
names in a pattern matching expression must be unique).
An example will illustrate this pattern-matching language. In Fig. 3.7, the place-holder ?
matches the rst element in the sequential music representation. The symbol matches
either no element, or the second, or the second and the third and so fourth. Consequently,
the two pattern-matching variables Pitch
predecessor
and Pitch
successor
match any pair of
two successive elements except the rst pair. Note that no nal is required which
would match the rest of the sequence.
[?, , Pitch
predecessor
, Pitch
successor
]
Figure 3.7.: Pattern matching expression matching any pair of two successive elements
except the rst pair
In PWConstraints, a rule denition consists of a pattern-matching part and the rule
body. The pattern-matching part is expressed in the pattern-matching language just
introduced. The rule body is an expression in which the pattern-matching variables of
the pattern-matching part are free variables. The free variable in the rule body with the
same name as some pattern-matching variable is bound to its value (i.e. the sequence
element matching the pattern-matching variable in the pattern).
[pattern matching expression), rule body)]
6
PWConstraints also introduces a fourth construct index variables for convenience. However, all
patterns using index variables can be reproduced with the three constructs introduced in this section.
51
3. Survey II: Composing with Constraint Programming
Figure 3.8 denes a full PWConstraints rule denition consisting of a pattern matching
expression and the rule body expression.
7
For every solution to a CSP, PWConstraints
makes sure that the body of a rule returns true for every match of its corresponding
pattern matching expression.
[[, Pitch
predecessor
, Pitch
successor
],
let Interval
in Interval = [Pitch
predecessor
Pitch
successor
[
Interval 0, . . . , 7]
Figure 3.8.: Full PMC rule denition example which states that the interval between two
consecutive pitches must not exceed a fth
The Rule Scope Concept
This text introduces the term rule scope to simplify the discussion of rules and their
eect. In most cases, the constraints of a rule are applied to multiple variable sets. In
case of the rule in Fig. 3.8, its constraints are applied to any set of variables in the CSP
which consists of two consecutive pitches and the interval between them.
The phrase the scope of rule x denotes the set of variable sets constrained by rule
x. The scope of the rule in Fig. 3.8 is the set of all variable sets which consists of two
consecutive pitches and the interval between them. Figure 3.9 notates this example rule
scope more formally in set-builder notation.
Pitch
1
, Pitch
2
, Interval :
Pitch
1
and Pitch
2
are consecutive pitches, Interval is the interval between them
Figure 3.9.: The rule scope of the rule in Fig. 3.8
It is important to note, that a PWConstraints rule always denes its constraints together
with its rule scope. This scope is primarily expressed by the pattern matching expression
of the rule an integral part of a rule denition which indicates to which sets of score
objects the rule is applied. Other systems (e.g. score-PMC, Sec. 3.3.1.2) support further
techniques to express which variable sets are constrained by a rule (e.g. by providing
means to access further variables from within the rule).
Additional terminology further simplies the discussion of the eect of a compositional
rule. The term rule application denotes the enforcing of the constraints of a rule to
7
In PWConstraints, rules are dened in Lisp this text uses a more abstract mathematical notation
instead. The notation has been further simplied. For example, PWConstraints follows the convention
used by Norvig [1992] to mark pattern matching variables by a preceding ? (e.g. ?Pitch) and to mark
the rule body by a preceding ?if . Also, every PWConstraints rule denition contains a documentation
string.
52
3.3. Generic Music Constraint Systems
a single set of constrained variables. Following this terminology, the pattern matching
expression of a single PWConstraints rule often expresses multiple rule applications.
This text refers to any set of variables constrained by a single rule application as a score
context constrained by this rule (see Sec. 2.4). All score contexts of all applications of a
single rule together form the scope of this rule.
Dependency Between the CSP Denition and the Search Strategy
In PWConstraints, the declarative side of a rule denition and the procedural side of the
search process are somewhat interwoven. The search strategy visits and determines the
variables in the CSP in a specic order.
The validity of any rule application is only checked after the full score context constrained
by this rule is determined.
8
This fact has some consequences for the denition of a rule.
The rst consequence is a benet. Because any variable is already determined when it is
used in the body of a rule, it can be processed by any function. This means that virtually
any Lisp function can be used to dene compositional rules in PWConstraints.
The second consequence restricts the manner how a compositional rule is dened. When
the validity of a rule is checked during the search process, all the variables it constraints
must already be determined.
This restriction is demonstrated by an example. Figure 3.10 denes a rule which con-
strains a melody such that the highest pitch occurs only once. Such a rule must be
formulated by constraining the relation of already determined note pitches. For this pur-
pose, PWConstraints provides the special variable alreadyDeterminedVariables.
9
This
variable is bound to a list of all determined variables (in reverse order, including the
currently visited variable as rst element). Each step during the search process its value
is updated and each rule application sees the current binding. Using this variable, the
rule can be dened as follows: either the value of the currently visited variable is higher
than any previously determined variable, or it is lower than the previous maximum value
(Fig. 3.10).
[[, Pitch],
Pitch ,= max(tail (alreadyDeterminedVariables))]
Figure 3.10.: A PWConstraints rule can only express constraints between already deter-
mined variables: the rule constrains that a melodic peak is unique in the melody
The most important disadvantage is this: checking the validity of a variable only after it
becomes determined severely impairs eciency. This aspect is briey discussed in Sec.
3.3.1.3.
8
Forward-checking rules form a special case (see Sec. 3.3.1.3). The validity of these rules are checked
when a single variable of the constrained score context is still undetermined and the domain of this
variable is possibly reduced.
9
The actual PWConstraints name for this variable is rl.
53
3. Survey II: Composing with Constraint Programming
3.3.1.2. Constraining Polyphonic Music: score-PMC
Because of its music representation a sequence of variables PMC is best suited for
musical CSPs in which some sequence of score objects is constrained. Many musical
CSPs can be expressed that way, particularly music theory models which belong to
some preparatory stage for the actual composition process (e.g. the creation of a purely
rhythmical gure, a dodecaphonic series, or a harmonic progression).
However, most western music is polyphonic by nature. Here, the term polyphony is used
in a more general meaning than usual: most music is organised in multiple layers, for
example, multiple voices, a melody plus accompaniment, or any other musical texture
where multiple events are played simultaneously.
A constraint system must take the polyphonic nature of music into account for most CSPs
outside the category of preparative CSPs. Yet, polyphonic CSPs are hard to express by
only a sequential music representation.
A constraint system with only a sequentially representation can in principle represent
complex polyphonic music after all, the sequential event list representation (Sec. 2.2)
can express highly complex scores but it is still very hard for such a system to express
polyphonic CSPs. Polyphonic CSPs often require constraining complex score contexts.
For example, typical counterpoint rules allow dissonant notes only in very specic cir-
cumstances. Checking whether these circumstances apply for a certain note requires
accessing complex score information such as the pitch of the note and its melodic prede-
cessor or successor notes, the metric position of the note with respect to the measure it
belongs to (e.g. either strong or weak beat) and whether or not the note pitch is conso-
nant with respect the pitches of simultaneous notes (or the chord these pitches form). It
is hard to express and constrain these score contexts in a sequential representation, be-
cause the information required to deduce which score elements belong to theses contexts
is missing.
Additional information to denote score contexts can be added in a hierarchical represen-
tation. For example, the information which note belongs to which voice can be expressed
adequately by grouping all voice notes in a voice container.
PMC supports variables with universal domain (Sec. 3.3.1.1) which can in principle be
used to create hierarchically structured scores with PMC. In this approach, the domain
of a variable does consist of composite values which represent more or less elaborated
musical fragments (e.g. motifs) which can even be created by sub-CSPs. This approach
has been used, for example, for constraint systems build on top of the PMC
10
(e.g. OMRC
[Sandred, 2003], and the system by Jacopo Schilingi
11
). Still, this approach resolves the
restrictions concerning score contexts only partly (e.g. how to access at the same time
the neighbouring notes in the melody, the simultaneous notes, and the metric position
of a note?). Moreover, this approach leads to severe performance problems (which are
discussed in more detail in Sec. 10.1.2).
10
These systems are actually built on the port of PMC to OpenMusic called OMCS.
11
Personal communication at PRISMA meeting, January 2004 at Centro Tempo Reale in Florence.
54
3.3. Generic Music Constraint Systems
PWConstraints proposes an alternative approach to address polyphonic CSPs: PWCon-
straints denes the special subsystem score-PMC. When compared with PMC, the main
change of score-PMC is its more elaborate music representation. Most notably, the music
representation of score-PMC explicitly represents additional score contexts.
12
In contrast
to the one-dimensional music representation of PMC, the music representation of score-
PMC resembles more a two-dimensional musical map. Here, the main dimensions are
notes in sequential order and simultaneous notes. More specically, the music represen-
tation of score-PMC explicitly represents the following contexts:
Melodic Context: For any part of a polyphonic score, the music representation stores the
horizontal order of notes.
Harmonic Context: The representation of simultaneous notes again shows a dependency
between a CSP denition and the search strategy. The harmonic context of a note
x is the set of all notes which are sounding at the attack time of the current note
and are already determined when the note x is visited during the search process.
Harmonic Slice: In contrast to the harmonic context, a harmonic slice is the set of all
notes which are simultaneous to a note x, regardless whether these notes are already
determined or not.
Metric Context: The metric context of a note expresses the rhythmic pattern which this
note belongs to (e.g. a succession of an eighth-triplet) and the position of this note
in the pattern (e.g. second note of the triplet). The rhythmic pattern is represented
by the expressive RTM-notation (Sec. 2.3).
The additionally available score contexts of score-PMC allow the user to complement
melodic rules (e.g. the simple rule restricting melodic intervals which was discussed
above) by rules which constrain other score contexts. The following harmonic rule ex-
ample prevents voice-crossing in polyphonic music. Voice-crossing occurs when an up-
per voice becomes the lower voice and vice versa (see Fig. 3.11). Prohibition of voice-
crossing constitutes a common counterpoint rule, often stated for pedagogical purposes
(cf. [Schoenberg, 1964]).
Figure 3.12 shows an implementation of a rule which prohibits voice-crossing. The rule
constrains that the order of simultaneous note pitches corresponds to the order of parts
(i.e. the note pitch in a lower part is below the simultaneous note pitch in an upper
part). Note that a pattern matching expression in score-PMC matches a note object and
not just a pitch as in PMC. The notes pitch is an attribute of that object.
13
12
Compared with the score context denition given in Sec. 2.4, PWConstraints denes the term context
with a more narrow meaning which only denotes the four contexts listed here.
13
The example transforms an original Lisp implementation of the rule [Laurson, 1996, p. 238] into
mathematical notation. Additionally, the names of the following score-PMC constructs are changed
for better readability. The function getHarmonicContext is originally called hc, getPartNumber is
called partnum and getPitch is called m in PWConstraints.
55
3. Survey II: Composing with Constraint Programming

crossing
voice

Figure 3.11.: Crossing of two voices
[[, note],
let /* sortedNotes is a list containing note and all its already determined simultaneous notes (the
harmonic context of the note). This list is sorted by their part number (the bass has the lowest part
number). */
sortedNotes
def
= sort(cons(note, getHarmonicContext(note)),
f : f(x, y)
def
= getPartNumber(x) < getPartNumber(y))
in /* isIncreasing constraints the pitches of the sortedNotes to be sorted in increasing order. */
isIncreasing(map(sortedNotes, getPitch))]
Figure 3.12.: Denition of a harmonic rule which prohibits voice-crossing (cf. [Laurson,
1996, p. 238])
An important restriction of score-PMC lies in the fact that note pitches are the only
variables in the music representation. All other aspects in particular the rhythmic
structure of the score must be fully determined in the CSP denition. However, the
rhythmic structure can be arbitrarily complex.
In texture-PMC [Laurson and Kuuskankare, 2001], an alternative extension of PWCon-
straints, other note parameters can also be constrained, rhythmic values for instance.
However, the program still requires a rhythmical input. Internally, search variables are
still encoded as pitches. These may be converted to change the rhythmic structure.
Because score-PMCs music representation shows a complex internal structure in which
aspects of the representation and the search are intertwined it would be very hard for
the user to adjust or extend this representation. For example, it would be hard to add
support for additional score contexts which may be required to dene complex CSPs.
3.3.1.3. The Search Strategy
PWConstraints constraint solver performs a chronological backtracking (BT) search
[Dechter, 2003]. This algorithm rst selects a value for a variable from its domain. Then
the algorithm checks whether all constraints applied to this variable (or this and any
other already determined variable) are satised. In case all these constraints are satis-
ed, the algorithm proceeds to determine the next variable. If any constraint fails, the
56
3.3. Generic Music Constraint Systems
algorithm tries sequentially the other domain values for this current variable as soon
as a variable value fulls all constraints, the algorithm proceeds to the next variable.
However, if no domain value satises all constraints, then the algorithm met a dead-end.
In that case, backtracking occurs: the algorithm turns back to the previously visited vari-
able and continues to try out its other domain values. When all variables are determined,
the algorithm found a solution to the CSP.
BT performs a complete search (i.e. if there exists a solution, the algorithm nds it).
The algorithm can nd one, multiple, or all solutions.
BT has several well-known weaknesses:
BT always detects a conict too late: only after all variables related by a constraint
are determined, the constraint may either succeed or fail. This problem is addressed
by constraint propagation (or consistency enforcing) [Dechter, 2003].
Failures with the same cause occur repeatedly (thrashing): BT does not analyse
which variable causes a constraint to fail. This problem is addressed by techniques
like intelligent backtracking (or backjumping) [Dechter, 2003].
When after backtracking the algorithm checks variables which it already checked
before, it it does not remember which value combinations failed before and will
check them again (redundant work). This problem is addressed by techniques like
backmarking [Dechter, 2003].
Finally, BT performs decisions in an order that was xed before the search started
(static variable order and static value order, also known as static variable and value
selection). In particular, the order in which variables are visited during the search
process immensely aects the size of the search space. This problem is addressed
by dynamic variable orderings [Dechter, 2003].
In case of PMC, the order in which variables are visited during the BT search process is
the sequential order of the variables in the music representation.
score-PMC also performs BT. However, the system rst computes an ecient order in
which the notes in the score are visited during the search process. To this end, score-
PMC evaluates the rhythmic structure of the solution. Using this rhythmic structure,
score-PMC calculates a search order for its polyphonic music representation such that
the search process always proceeds from left to right, that is notes with a smaller start
time value are visited rst. This search strategy is the reason why score-PMC requires
that the temporal structure of any CSP is fully determined in the problem denition.
In addition to strict rules (which a solution must always obey), PWConstraints sup-
ports heuristic rules which allow the user to express mere preferences and to avoid over-
constrained situations (where multiple constraints contradict each other). The body of
a heuristic rule returns a number (the heuristic value) instead of a boolean value. For
57
3. Survey II: Composing with Constraint Programming
better solutions the rule returns a higher number. In case of a conict between heuris-
tic rules, the more important heuristic rule (the one with a higher heuristic value) is
preferred to hold.
Anyway, this technique does not necessarily nd any best solution with respect to heuris-
tic rules. Heuristic rules aect the value order [Bartak, 1998], that is in which order the
values of a variables domain are checked. During the search process, a domain value
of the current variable with the highest heuristic value is checked rst. Thus, heuristic
rules guide the search to nd better solutions earlier, but with no guarantee to nd an
optimal solution.
For a better performance, forward-checking [Dechter, 2003] (a limited form of constraint
propagation) can be applied together with BT by dening special forward-checking rules.
Nevertheless, Laurson [1996, p. 209] reports his experience that pure BT performs quite
often surprisingly ecient, and thus forward-checking is rarely used.
3.3.2. Situation
Situation [Rueda et al., 1998; Assayag et al., 1999] was originally conceived in collabora-
tion between the composer Antoine Bonnet and the computer scientist Camilo Rueda as
a constraint system for solving a range of harmonic CSPs. Situation was rst developed
as a PatchWork library, and was later extended and ported as OMSituation [Bonnet and
Rueda, 1999] to the PatchWork successor OpenMusic [Assayag et al., 1999]. Nevertheless,
this text will refer to any version of the system simply as Situation.
Early versions of the system supported quasi ready-made components to dene CSPs. For
example, instead of dening compositional rules from scratch, the user utilised predened
rule templates, that is compositional rules which expected arguments from the user to
detail their eect. Although such an approach limits the CSPs the user can dene, it also
simplies the denition process. Later, the system was extended to support a larger set of
CSPs (e.g. rhythmical CSPs as well). Still, the design of the system is better understood
with its history in mind.
3.3.2.1. Music Representation
The music representation of Situation consists of a sequence of objects. An object has
an internal structure: it consists of one or more points. Furthermore, Situation denes
distances between these points. In many CSPs, a point represents a note pitch, an object
represents a chord containing multiple pitches, and a distance represents an interval
between two note pitches.
Situation distinguishes between internal distances and external distances. An internal
distance represents the distance between two points of a single object (also called vertical
distance: interpreted as the distance between two neighbouring note pitches in a single
chord) and an external distance represents the distance between two points from dierent
58
3.3. Generic Music Constraint Systems
objects (also called horizontal distance: interpreted as the distance between matching
note pitches of two neighbouring chords)
The points and the distances constitute the variables in a CSP. Their domain consists
of numbers.
14
The meaning of points and the distances between them is not specied by Situation.
The user may interpret their meaning at will. For instance, points may represent pitches
which can be interpreted as measured in MIDI key-numbers, or as frequency values
measured in hertz . Points may also represent note start times, for example, measured in
beats. In that case, distances represent temporal intervals.
The music representation of Situation is particularly suited for CSPs on chord sequences:
an explicit representation of pitches and the intervals between them is suitable for many
such problems. The implicit constraints between points and distances conveniently con-
trol the relation between these variables.
Situation is not only convenient to use but also generic. The Situation user can control
which relation actually holds between points and distances. This allows for the range
of interpretations already mentioned. For example, in case points represent MIDI key-
numbers, then the relation between points and distances is governed by addition con-
strains. If, on the other hand, points represent frequencies, then multiplication constraints
govern the relation between points and distances. Situation also oers an alternative
representation mode which consists of a plain sequence of variables. This representa-
tion mode supports variables with universal domains (e.g. OpenMusic chord instances
as domain values).
3.3.2.2. Rule Formalism
Situation provides pre-dened constraints which are not just constraint primitives like
numeric operations (e.g. =, >, or +) but generalised compositional rules. For example,
Situation denes a set of constraints which impose patterns on various viewpoints of
the music (e.g. rules aecting a voice prole by constraining the number of consecutive
upward and downward movements or the number of repeated melodic intervals).
Situation complements these pre-dened constraints by means for user-dened con-
straints. Comparable with the body of a PWConstraints rule, a user-constraint in Situa-
tion is an arbitrary expression (on already determined variables) which returns a boolean
value. In Situation, this expression is encapsulated in a function.
Again similar to PWConstraints, Situation supports a convenient and powerful language
to express to which variables in the music representation a constraint (predened or
user-dened) is applied. The user applies constraints always by the special constraint
application language of the system like in PWConstraints. However, Situation denes
14
Internally, each point is represented by its own variable, whereas a sequence of distances is represented
by a single variable whose domain consists of number sequences.
59
3. Survey II: Composing with Constraint Programming
much more symbols than PWConstraints pattern matching language and Situations
supported symbol set depends on the actual constraint applied. Nevertheless, for many
constraints the application specication has the same structure: an alternating sequence
of range specications and further arguments.
[range specication)
1
, arguments)
1
, range specication)
2
, arguments)
2
, . . . ]
Somewhat simplied, range specications dene the scope of a constraint (the term rule
scope was introduced in Sec. 3.3.1.1). For example, with the following expression the
user may apply a rule to the rst, second and fth object in the music representation
with arguments)
1
and to the seventh to tenth object with arguments)
2
.
15
[0, 1, 4, arguments)
1
, 6 9, arguments)
2
]
This chapter earlier introduced the notion of the set of variables constrained by a sin-
gle constraint application (the score context constrained by a rule or constraint, see Sec.
3.3.1.1). In Situation, this set is specied by several factors including a single range spec-
ication (in the constraint application language), the associated arguments, the number
of arguments expected by a function implementing a user-constraint, and the optional
access of already determined objects (predecessors of current object) within a user-
constraint denition. In summary, the score context constrained by a single constraint
application can be any set of variables in the music representation. Also the set of all
score contexts constrained by multiple applications of a constraint (i.e. the rule scope)
is fully user controllable by range specications of the constraint application language.
Nevertheless, because of the structure of its music representation Situation does only
support score contexts dened by positional relations of objects (and their points plus
distances). Consequently, it is dicult to express rules which constrain complex score
contexts as required, for example, in polyphonic CSPs. Section 4.1.1 explains this limi-
tation further.
3.3.2.3. Search Strategy
Compared with PWConstraints, Situations constraint solver performs a more sophis-
ticated search strategy. The Situation applies a variation of minimal forward checking.
Minimal forward checking [Dent and Mercer, 1994] is a variant of forward checking. By
means of lazy evaluation, minimal forward checking delays the reduction of the domain
of variables not yet visited (i.e. the forward checking) until it is actually required during
the search. That way, the algorithm provides the benets of forward checking (to prune
of domains, i.e. to reduce the search space) by reducing the work the algorithm has to
perform (searching through possibly large domains for consistent values)
Situations rst-found forward checking algorithm [Rueda and Valencia, 1997] adapts
minimal forward checking for hierarchical domains. A variable for distances (a variable
whose domain consists of number sequences, see above) has often a large domain. There-
fore, Situation optionally organises this domain in a hierarchical manner for eciency.
15
Indices are 0-based, index 0 points to the rst object.
60
3.3. Generic Music Constraint Systems
Such an hierarchically structured domain combines domain value subsets which share
some property into a subtree of the domain. That way, a constraint on this common prop-
erty can be propagated for all domain values in the subtree at once. Per default, this
common property is the sum of the sequence elements (i.e. the sum of object distances),
because this property is constrained in many harmonic CSPs. The user can specify a
dierent common property. The data structure of Situations music representation is
optimised for rst-found forward checking (i.e. all distances of an object are represented
by a single variable with a sequence domain, as mentioned already in fn. 14).
In addition to strict CSPs, Situation also supports CSPs with weak constraints. In the
latter, a numeric preference value is specied for each constraint.
3.3.3. OMClouds
The music constraint system OMClouds [Truchet et al., 2001, 2003] extends the compo-
sition system OpenMusic [Assayag et al., 1999].
3.3.3.1. Search Strategy
The search strategy applied by OMClouds diers clearly from the strategies of the sys-
tems discussed so far and therefore it is discussed rst. The system is a based solely
on a heuristic search strategy which iteratively improves an initial random solution to
a CSP. More specically, OMClouds constraint solver performs a local improvement
search strategy [Codognet and Diaz, 2001; Codognet et al., 2002]. This strategy quickly
nds an approximated solution to a constraint problem which fulls many or most of
the constraints imposed.
Internally, a constraint is implemented by a cost function, which returns a numeric value
indicating how much its constraint is violated. In a nutshell, the algorithm starts by
determining all variables of the CSP to some random value from their domain. Then,
the algorithm computes the accumulated cost of all cost functions for every variable to
nd the variable whose current value conforms least a solution to the CSP. This variable
is updated for the next iteration. OMClouds extends this basic idea by a taboo search
method to avoid local minima in the search space.
On the user-level, however, a CSP is expressed by strict constraints (e.g. X = Y ). Only
internally, these constraints are translated into cost functions (e.g. X = Y is transformed
into [X Y [).
The heuristic approach of OMClouds makes the denition of complex CSPs easier be-
cause no over-constrained situation can occur. In a system which only supports strict
constraints, the CSP must explicitly handle special cases in which some rule can be
neglected. In OMClouds, rule denitions can be less carefully formulated: the heuristic
search strategy does not reject a solution because some rules are not always fullled.
61
3. Survey II: Composing with Constraint Programming
Adaptive search does not execute a complete search. If the system is asked multiple
times for a solution to the same problem it usually comes up with dierent solutions,
but the system cannot output all solutions. Moreover, the system does not guarantee to
nd an optimal solution for strict constraints. For examples, constraints like all elements
of a list are pairwise distinct are hard to full and the algorithm may thus never nd
an exact solution to a CSP with this constraint (e.g. OMClouds may never nd a true
all-interval series).
3.3.3.2. Music Representation
In OMClouds, variable domains can consist of any values. However, in every CSP all
variables share the same domain. This restriction limits what CSPs can be dened, but
simplies their denition.
Similar to PMC (see Sec. 3.3.1.1), OMClouds music representation consists of a at
list of variables.
16
Truchet chose this straightforward representation format, because
several composers she contacted were afraid they would be biased by a more expressive
music representation.
17
A purely sequential representation can express several musical
structures. However, such a structure is poorly suited for more complex musical CSPs,
in particular polyphonic problems (see Sec. 3.3.1.2).
3.3.3.3. Rule Formalism
OMClouds supports easy-to-use means to apply constraints homogeneously to all vari-
ables of the CSP. However, when compared with PWConstraints and Situation, OM-
Clouds oers little user-control on the scope of a rule (Sec. 3.3.1.1). Again, this design
results in a system which is clearly limiting but also more easy to use. All constraints of
a CSP are merged in quasi a single rule associated with the music representation. That
is, the rule application of OMClouds applies all constraints together with a uniform rule
scope. The score contexts of which this rule scope consists are based the positions of
variables in the sequential music representation. For example, constraints can be applied
to every single variable or to every variable set consisting of some variable and its next
but one successor. In contrast to the rule application mechanisms of PWConstraints and
Situation, OMClouds does not provide an easy-to-use means to apply a constraint only
to a specic subset of variables (e.g. only on the rst and third variable).
16
In fact, OMClouds feature three representation cases: the solution constitutes either a sequence of
variables, a cycle of variables, or a permutation of variables. In all cases the solution is represented
by a at list.
The second case, however, slightly changes the rule application mechanism while the third case
enforces an additional constraint. In the second case the cycle all elements (including rst and
last) have a predecessor and successor. In the third case the permutation the number of variables
in the music representation equals the number of domain values specied and every domain value
appears only once in a solution. That way, OMClouds can avoid an all-different constraint on the
solution which is hard to full for its heuristic search.
17
Email communication with C. Truchet (23 September 2003).
62
3.3. Generic Music Constraint Systems
3.3.3.4. User Interface
Compared with other generic music constraint systems discussed here, the OMClouds
user can only solve a limited set of musical CSPs. On the other hand, OMClouds is
particularly easy to use. A CSP is fully dened in a visual programming language, which
makes the system easy to learn for composers who have little experience in programming.
Additionally, the user can supervise the search progress: OMClouds can display the
current state of the solution in a graphical score editor.
3.3.4. Other Systems
This sections itemises several further systems. Each of these system allows the user to
solve a considerable range of musical CSPs.
CHARM [Wiggins et al., 1989; Harris et al., 1991; Smaill et al., 1993] is a highly expres-
sive music representation. Although CHARM was not designed as a music constraint
system, the Prolog implementation of CHARM could be extended into such a system.
Pure Prolog itself only supports the equality constraint (unication) and is thus far from
being optimised to solve, for instance, numerical CSPs. Yet, there exist several Prolog
implementations which extend pure Prolog by constraint programming capabilities (cf.
also [Bratko, 2001]). BackTalk [Roy and Pachet, 1997] constitutes a system to solve
CSPs over nite domains of Smalltalk objects. BackTalk implements a substantial num-
ber of constraint solving algorithms (e.g. several consistency enforcing algorithms to lter
variable domains in the rst stage of the search process complemented by solution enu-
meration algorithms including forward-checking, backjumping, and backmarking). The
combination of BackTalk and the music representation MusES [Pachet, 1993; Pachet
et al., 1996] forms a highly expressive and extendable music constraint system which
has been applied to several musical problems (e.g. automatic harmonisation [Pachet and
Roy, 1995, 1998]). However, MusES provides only a limited set of score contexts. MusES
denes an exhaustive set of temporal relations between score objects and thus tempo-
rally related object sets are well accessible. Other relations (e.g. positional relations) are
missing in the representation. For example, information such as the following note in a
melody is not easily accessible [Pachet, 1994]. Also, the pitch representation of MusES
is only suited for tonal harmony and limited to the set of conventional western pitches.
MusES explicitly represents enharmonic spelling (i.e. c and d are distinct pitches). Be-
cause of this important capacity, nevertheless, MusES cannot be easily generalised, for
example, to represent microtonal pitches.
Alvarez et al. [1998] and Rueda et al. [2001] contribute to theoretical computer science
and music constraint programming at the same time. The authors propose PiCO, a
calculus integrating concurrent processes, object-oriented programming and constraint
programming as a foundation for composition systems. This research also developed
Cordial [Rueda et al., 1997] a visual programming language for composers in the spirit
of OpenMusic [Assayag et al., 1999] which shares semantics with PiCO.
63
3. Survey II: Composing with Constraint Programming
Arno [Anders, 2000] supports CSPs on the music representation of Common Music
[Taube, 1997]. The constraint solver Screamer [Siskind, 1991; Siskind and McAllester,
1993] is employed for this purpose. Arno can express a large number of musical CSPs.
In particular, Arno supports polyphonic CSPs as well. However, the system performs an
inecient search except for a specic class of CSPs, namely canons.
OMBT [Truchet et al., 2001; Truchet, a,b] integrates important functionality of the
constraint solver Screamer into the composition system OpenMusic [Assayag et al., 1999].
3.4. Conclusion
A rule-based approach is particularly adequate to state music theories (and has therefore
been applied for centuries). Constraint programming has proven a well-suited program-
ming paradigm to realise rule-based computer-aided composition systems. Many musical
CSPs have already been addressed by means of constraint programming.
Generic music constraint systems pre-dene important building blocks shared by many
musical CSPs and that way greatly simplify their denition. Several generic music con-
straint systems has already been proposed. These systems dier considerably in the
signicant aspects of such a system: they dier in their music representation, the way
they dene and apply rules, as well as the search technique of their constraint solver.
64
4. Motivation and System Overview
The present research aims at creating a generic music constraint system, that is, at
a system which allows its user to dene and solve a large set of musical constraint
satisfaction problems. Chapter 1 described the motivation for this research at large, but
the present chapter describes this motivation more specically. This chapter analyses how
existing systems are limited in terms of generality and outlines how these limitations are
addressed in a more generic system.
Chapter Overview
A number of generic music constraint systems have already been proposed (surveyed
above in Sec. 3.3). Section 4.1 shows that these systems support only a restricted set of
musical CSPs and analyses their limitations. Based on this analysis, Sec. 4.2 formulates
which conditions a more generic system has to meet. Section 4.3 outlines Strasheelas
design by explaining how Strasheela meets these conditions (succeeding chapters will
discuss the design of Strasheela in detail).
4.1. Limited Generality of Existing Systems
A music constraint system can be very useful even if it is specialised for a narrow range of
musical CSPs. For instance, Situation was originally conceived solely for solving a specic
set of harmonic CSPs (Sec. 3.3.2, [Rueda et al., 1998; Bonnet and Rueda, 1999]). In this
form, Situation was successfully employed by Antoine Bonnet for composing his piece

Epitaphe (19921994, for 8 brass instruments, 2 pianos, orchestra and electro-acoustics).

Situation could be used by other composers at this stage of its development if they were
content with Bonnets aesthetic decisions embodied in the set of predened rules Situ-
ation supported. Yet, composers usually prefer to make aesthetic decisions themselves.
Situation was therefore later generalised to cover a larger range of musical CSPs (see
Sec. 3.3.2).
A similar tendency from a more specialised music constraint system towards a more
generic system can be seen in the redesign of the PWConstraints subsystem score-PMC
(Sec. 3.3.1.2, [Laurson, 1996]) into texture-PMC (Sec. 3.3.1.2, [Laurson and Kuuskankare,
2001]).
The present thesis continues this research on how a music constraint system can be made
highly generic. To this end, the present section analyses how existing systems are limited
65
4. Motivation and System Overview
in terms of generality. The key question is: how do existing systems restrict the set of
musical CSPs they support?
Some examples of limitations are quite obvious. For instance, OMRC (Sec. 3.2.3, [San-
dred, 2003]) is solely devoted to rhythmical CSPs, and score-PMC requires the rhythmic
structure to be fully determined in the problem denition. However, there are also many
more subtle limitations which will become manifest only in the course of the present
analysis
The following sections systematically examine three fundamental aspects of any music
constraint system and analyse how existing systems are limited by their design of these
aspects. These aspects are the respective music representation (Sec. 4.1.1), their rule
formalism (Sec. 4.1.2), and the search strategy underlying these systems (Sec. 4.1.3).
4.1.1. Music Representation
Any musical CSP applies some sort of music representation to represent score informa-
tion (i.e. either a full score or some excerpt such as a purely rhythmical structure or a
harmonic progression). The music representation of a music constraint system has great
inuence on its expressivity (i.e. how musical CSPs can be expressed).
Certain aspects in this music representation are variables (unknowns) in the CSP (e.g.
score object parameters such as note pitches or durations). A number of constraints re-
strict these variables. A solution determines all variables without violating any constraint
(Sec. 3.1).
Solution Representation
Section 1.3 introduced the notion of a most generic music constraint system as a system
in which any music theory model conceivable can be implemented. The music represen-
tation of such a system represents in a convenient way any information its user wants to
represent in a solution score. Many existing systems, however, support a music represen-
tation which can make it dicult to express complex musical information. For example,
the following systems support a purely sequential music representation: the PWCon-
straints subsystem PMC (Sec. 3.3.1.1, [Laurson, 1996]), Situation, and OMClouds (Sec.
3.3.3, [Truchet et al., 2003]). A sequential music representation is less suited to repre-
senting music which features multiple layers that run parallel in time (e.g. parallel voices
in polyphonic music).
For example, OMClouds represents music by a sequence of variables where each variable
uniformly stands for either an integer or a list of integers. The user can interpret these
variables in various ways (e.g. variables may represent pitches or durations). Nevertheless,
OMClouds does not support any construct which denotes what dierent variables mean.
In eect, all OMClouds variables in a CSP usually represent the same musical parameter
type. Only a small range of CSPs is expressible in a system where all variables represent
either pitches or all variables represent durations.
66
4.1. Limited Generality of Existing Systems
Compared to OMClouds, the music representation of Situation is more expressive be-
cause the objects in its sequential representation are potentially more structured. The
Situation user denes the structure of the objects by specifying a set of points (i.e. vari-
ables) and distances between these points. The user freely interprets what points and
distances stand for (e.g. pitches and pitch intervals). However, the denition of complex
musical CSPs becomes inconvenient because the Situation music representation does
not provide any constructs to mark what musical concepts are expressed by its objects
and variables (e.g. concepts such as chord, note or pitch). This problem is similar to
the problem pointed out for OMClouds above. In contrast, score-PMC denes a data
abstraction (Sec. 2.6) with non-ambiguously named accessors (e.g. getPitch(myNote)).
CSP Denition Support
Existing systems restrict not only what information is expressed by a solution score,
but also restrict the information that can be constrained. In a musical CSP, constraints
restrict variables which are related to each other in some way: this text introduced the
term score context to denote an arbitrary set of inter-related score objects (see Sec.
2.4). Each application of a compositional rule constrains variables in one or more score
contexts. For example, a melodic rule may constrain the pitches of note pairs that are
direct neighbours in a voice or may pose a restriction on a pair of neighbouring local
pitch maxima (between these maxima there are further notes in the voice).
In a most generic system, users can freely constrain arbitrary score contexts. Existing
systems support only a very limited set of score contexts. For instance, Baroque coun-
terpoint restricts all notes in a voice to pitches of the underlying harmony; exceptions
are allowed for a number of specic circumstances including passing notes. A passing
note (or passing tone) is a non-harmonic note of weak rhythmic quality that is reached
and left by stepwise motion in the same direction.
To check whether a single note forms a passing note requires access to a complex score
context which consists of the following variables: the pitch of the note, the pitches of
its predecessor and successor notes, the metric position of the note with respect to the
measure it belongs to (e.g. either strong or weak beat) and whether or not the note pitch
is consonant with respect to the chord the note belongs to. Few systems support the
access to such a score context and thus few systems are able to express the passing note
concept a decisive concept in many counterpoint rule sets.
Most systems (e.g. PWConstraints and Situation) are limited to a nite set of predened
constrainable score contexts. For instance, all systems discussed in this text support
constraining neighbouring objects in a sequence. However, only a few systems support
the constraining of a context consisting of simultaneous objects which belong to dierent
voices.
A system with particual strong support for polyphonic CSPs is score-PMC (Sec. 3.3.1.2)
and its is likely the only system capable of accessing score contexts as complex as the
context that is required to express the passing note concept. This system explicitly rep-
resents and provides access to a number of score contexts. For example, the neighbouring
67
4. Motivation and System Overview
notes of each voice (melodic context) are collected in a doubly linked list (i.e. a sequen-
tial data structure where each element features pointers to its predecessor and successor
[Cormen et al., 2001]). This data structure supports accessing one or more melodic pre-
decessors (or successors) of any note. Similarly, another doubly linked list contains the
simultaneous notes (harmonic context).
1
Further score context information is accessible
by predicates (e.g. a function to check the metric position of a note in a measure, the
metric context).
The explicitly represented information in the score-PMC music representation is not user-
extendable. For instance, the harmonic context of score-PMC consists only of the actual
note pitches. The representation does not explicitly represent more abstract information
about the underlying harmony such as the chord root nor does it allow the user to add
this information. This restriction complicates the denition of complex harmonic rules
(e.g. rules on the chord progression such as a rule which demands a I-II-V-I progression).
Moreover, the internal representation of score contexts in score-PMC makes it dicult to
extend the set of supported score contexts. The system predenes a set of score contexts
for each note (see Sec. 3.3.1.2) by explicitly storing each context: a score context is not
derived from information stored somewhere in the score but each context for each note
is stored explicitly as a score-PMC note attribute [Laurson, 1996, p. 225]. The approach
is plausible because in the case where these contexts are constrained, score-PMC must
access them for every constraint validation during the search process. However, this
approach complicates user-dened score contexts, because it requires the user to change
the internal details of the system (e.g. it requires replacing the note object by an extended
data structure). Yet, restricting the set of available score contexts restricts what musical
CSPs can be dened.
4.1.2. Rule Formalism
A music constraint system provides a broad range of constraints (e.g. numerical con-
straints or set constraints). A user applies these constraints to variables in the music
representation to express restrictions on these variables. For instance, a composer may
express a melodic restriction which constrains the distance between the pitches of two
consecutive notes so that they do not exceed the interval of a fth (Fig. 4.1). Here, the
interval is measured in semitones, seven denoting a fth.
7 [Pitch
1
Pitch
2
[
Figure 4.1.: Constrain the interval between Pitch
1
and Pitch
2
not to exceed a fth
1
Laurson [1996] calls the set of all simultaneous notes a harmonic slice. In score-PMC parlance, a
harmonic context is the set of simultaneous notes which are already determined at a certain time
during the search process.
68
4.1. Limited Generality of Existing Systems
The constraint in Fig. 4.1 is only dened for the two specic pitches Pitch
1
and Pitch
2
. In
many cases, however, such a restriction should be more general. For instance, a composer
may wish that this restriction holds for every melodic interval. Yet, for complex CSPs
with a large number of rules and variables, it is of paramount importance to dene
the CSP in a modular way. Consequently, existing systems introduced the concept of a
compositional rule.
A compositional rule encapsulates constraints on score objects. As a rule is more general,
its application often constrains multiple sets of variables (i.e. multiple score contexts). For
example, a compositional rule may encapsulate the expression in Fig. 4.1. The application
of this rule may constrain the interval between every pair of consecutive note pitches
in any voice. To this end, the rule is applied to each of these score contexts (e.g. each
consecutive note pair) by some control structure (e.g. iteration). The term rule scope
denotes the set of variable sets that is constrained by a certain rule (see Sec. 3.3.1.1).
In several musical styles, specic compositional rules restrict only specic parts of the
music. For instance, an imitation constraining notes at the same position in dierent
voices often holds only for the rst few notes of a section. Another rule only applied
to the end of a section may cause a cadence.
In a most generic system, the user can freely control the scope of each rule. Existing
systems, however, introduce convenient but proprietary rule formalisms which allow only
limited control on the scope of a rule.
Many existing systems exhibit a principle problem: they do not separate the denition
of a rule from its application. In these systems, the rule denition implicitly denes the
rule application. This approach simplies the rule formalism for the user. Yet, as the rule
application is not fully user-controllable, it is limited to the rule application mechanisms
provided by the respective system.
The rule formalisms of the two most important existing systems, PWConstraints and
Situation, have this problem. These formalisms are now briey discussed with regard to
their generality (see Sec. 3.3 for more details on PWConstraints and Situation at large).
PWConstraints
PWConstraints proposes a pattern matching mechanism to apply rules to the score. This
rule application mechanism plays such an important role for the system that subsystems
were named accordingly. For instance, the name PMC stands for pattern matching con-
straints.
In PWConstraints, a rule is applied to any set of score objects which matches the pattern
matching expression of the rule. This rule application mechanism is highly convenient
for a sequential score context (e.g. for melodic rules) but is less suitable for any non-
sequential context. However, in polyphonic CSPs many contexts are of a non-sequential
nature. Moreover, the pattern matching mechanism cannot be extended by the user and
is restricted to cases considered by the system designer.
69
4. Motivation and System Overview
The system score-PMC (the polyphonic subsystem of PWConstraints) therefore intro-
duces further concepts. In contrast to the more basic PWConstraints subsystem PMC,
the score-PMC music representation introduces a data abstraction (Sec. 2.6) with acces-
sors which can be called within a rule denition (Sec. 3.3.1.2). That way, a few further
score contexts can be accessed within the rule (namely the sequence of melodic notes,
simultaneous notes and the metric position of a note). However, the music representa-
tion of score-PMC is not extendable and only predened contexts are accessible (e.g. no
context is predened for a canon-rule).
To reduce such limitations (which were pointed out by Anders [2003]), Laurson and Ku-
uskankare [2005] propose a novel syntax extension for score-PMC rules which provides
convenient pattern-matching-based access to a number of score contexts such as succes-
sive notes in a voice (melodic context), simultaneous notes (harmonic context), all notes
in a measure or on a single beat. This means that more score contexts can be expressed
by convenient pattern-matching expressions. Still, the user can hardly extend the set of
score contexts supported.
Moreover, a pattern matching mechanism is axiomatically limited to what can be ex-
pressed by a pattern and the PWConstraints pattern matching language cannot express
every possible combination of elements in a sequence (apart from more complex score
contexts). For example, a PWConstraints pattern can easily express any two neigh-
bouring variables in a sequence: for the sequence [a, b, c, d] the pattern [, X, Y ] matches
three times, namely X =a Y =b, X =b Y =c, and X =c Y =d. However, a single
pattern cannot express any pair of neighbouring variables in a sequence such that pairs
do not overlap: for the same sequence [a, b, c, d], there is not a pattern that only matches
X =a Y =b and X =c Y =d.
Situation
The rule application mechanism of Situation is tailored for Situations sequential music
representation: compositional rules are applied to sets of objects which are identied by
their numeric indices in the sequence.
For a sequential music representation, any possible rule scope can be expressed by Sit-
uations index-based rule application mechanism. For a unary rule, the rule scope can
be specied by a list of indices. For an n-ary rule, the rule scope can be specied by a
list of lists of indices. Also, index specications may use short-hand notations for index
ranges (e.g. [1, 3 5, 7] to denote [1, 3, 4, 5, 7]).
Nevertheless, an index-based rule application mechanism is limited to a sequential mu-
sic representation and positionally related score contexts (e.g. neighbouring notes). This
rule application mechanism is insucient for more complex score topologies (e.g. hierar-
chically structured scores, see Sec. 2.4). For instance, it would be highly inconvenient to
express the context required for the passing note concept (see Sec. 4.1.1) only by score
object indices.
70
4.1. Limited Generality of Existing Systems
4.1.3. Search Strategy
Musical constraint programming problems can be extremely complex, that is, the search
space of these problems can be huge. While search strategies such as chronological back-
tracking (Sec. 3.3.1.3) are practical for problems with a relatively small search space,
more complex problems require more sophisticated techniques which can be optimised
for the problem at hand [van Roy and Haridi, 2004].
Dierent musical CSPs require dierent search strategies to perform the search e-
ciently. Such strategies may be inspired by the way a composer works with pen and
paper. For example, in the process of composing a tonal piece, a composer often sketches
the underlying harmonic progression rst before writing the actual note pitches. Other
composers prefer to fully work out the rhythmical structure of the music rst. Polyphonic
music is often written from left to right by jumping between the voices and writing the
rhythmical structure and the pitches at the same time.
The deeper reason for the many limitations of existing systems are their underlying
search strategies. These strategies are optimised for specic problem classes. Indeed,
systems are sometimes purposefully crippled and restrict their users to CSPs which
they can solve eciently. For instance, score-PMC does not allow the user to constrain
the temporal structure of music, because a determined temporal structure is required
by the polyphonic music search approach of score-PMC to compute an ecient static
search order (Sec. 3.3.1.3). In Situation, the search strategy only performs constraint
propagation an optimisation technique to distinctly reduce the search space for its
specic at music representation format (Sec. 3.3.2.3).
Some systems acknowledge the importance of a user-controlled search strategy. For ex-
ample, PWConstraints allows its user to mix forward-checking rules and the standard
backtracking rules in a single CSP [Laurson, 1996]. PWConstraints forward-checking
rules constitute a simple form of constraint propagation. These rules reduce the domain
of variables before these variables are visited and determined in the search process (the
validity of a normal rule in PWConstraints is only checked after all variables constrained
by this rule are determined). However, this approach is not generic. For example, the
user cannot change the order in which variables are visited during the search process.
Also, changing the search strategy requires changing the problem denition.
Surprisingly, most existing systems come with their own specically developed constraint
solver instead of applying an already established solver.
2
Most systems extend existing
general computer-aided composition environments by custom-made constraint program-
ming means. That way, the resulting solver is integrated tightly into the general CAC
environment. Yet, this approach is also a reason for limitations of existing systems: ad-hoc
developed solvers can hardly compete with solvers resulting from specialised research.
A most generic music constraint system is not optimised in a hard-wired way for a
specic class of problems, but allows the user to customise the search strategy for each
2
PWConstraints includes an interface to the Common Lisp constraint programming extensions Screamer
[Siskind and McAllester, 1993], but only as an additional feature [Laurson, 1996].
71
4. Motivation and System Overview
CSP. Such a system employs a constraint solver in which the user can program the search
strategy independently of the CSP denition.
4.2. The Research Goal
The present research aims to develop a generic music constraint system. On one hand,
such a system facilitates the denition of musical CSPs when compared with a general
constraint system (e.g. a programming language with support for constraint program-
ming such as Eclipse or Oz). A generic music constraint system predenes the building
blocks required for these problems. These building blocks fall into three categories: (i)
components for expressing what is known about a score (the music representation), (ii)
mechanisms to express compositional rules and their application to the music (the rule
formalism), and (iii) mechanisms to express how the search is conducted.
3
On the other hand, a generic music constraint system allows the user to dene and
solve a large set of musical CSPs. A most generic music constraint system allows the
implementation of any music theory model (Sec. 1.2) conceivable which is fully formalised
by a set of rules.
A software application must perform with reasonable eciency to be useful in prac-
tice. Therefore, the present research does not aim for maximum generality. Instead, this
research aims to create a system which is highly generic but still reasonably ecient.
The analysis in Sec. 4.1 revealed how existing systems perform with respect to this
research goal:
1. The music representations of existing systems can express complex solution scores
and provide for a large set of score contexts. However, the range of score information
supported by these systems is limited. In particular, complex score contexts
important for implementing many music theories are hard or impossible to access
and constrain.
2. The rule formalisms of existing systems support rule denitions with virtually no
restriction, and concise rule application mechanisms capable of expressing complex
rule scopes. However, existing rule application mechanisms are not generic: rule
scopes important for music constraint programming are not expressible.
3. The search strategies of existing systems allow for the solving of complex musi-
cal CSPs. Yet, these systems implement hardwired optimisations which are only
suitable for specic CSP classes. The search cannot be adapted for dierent CSPs.
Some systems even support only the CSPs for which they are optimised.
The present research aims to create a system which is more generic than existing music
constraint systems but still performs reasonable eciently.
3
In many systems, the search strategy is hidden from the user.
72
4.3. Strasheela Design Overview
In general, a generic music constraint system must be a highly programmable and user-
extendable system. A system designer cannot foresee every possible usage (e.g. every
required score context or rule scope). The new system must therefore allow the user to
adapt and extend the music representation, to dene novel rule application mechanisms
and to control the search process according to their needs.
Firstly, the music representation of this new system allows the user to dene which infor-
mation the score exhibits. For this purpose, fundamental concepts from previous research
on music representation (Chap. 2) are combined into a consistent music representation
conceived for music constraint programming. Specically, the music representation allows
the user to constrain arbitrary score contexts by a rule.
Secondly, the new system introduces a programmable and generic rule formalism. The
user has full control over how a rule is applied to the score.
Finally, the new system is based on a state-of-the-art constraint system, which allows
the user to program how the search is conducted. In particular, it allows for a user-
dened dynamic variable ordering (also known as variable selection, i.e. the order in
which variables are visited during the search process). CSPs for which a proven variable
ordering exist can be solved with this ordering. Moreover, problems which were explicitly
excluded by previous systems for eciency reasons can be solved with suitable novel
orderings.
4.3. Strasheela Design Overview
The main contribution of this research is the design of Strasheela, a generic music con-
straint system. Strasheela constitutes a highly programmable and extendable system.
Strasheela supports musical CSPs which were extremely dicult or even impossible to
solve in previous systems. Despite its more generic design in comparison to existing
music constraint systems, Strasheela still performs reasonably eciently even on highly
complex CSPs.
The present section explains the principle design decisions for the three core aspects of
this contribution. Firstly, Strasheela employs a constraint model based on computational
spaces and customises this approach for musical CSPs (Sec. 4.3.1, the overview starts
by discussing the search strategy, because Strasheelas search approach constitutes its
foundation). Secondly, this research applies a well-understood formalism to dene and
apply compositional rules (Sec. 4.3.2). Thirdly, a rich and generic music representation
has been specically designed for the denition of complex musical CSPs (Sec. 4.3.3).
The design of Strasheela will be explained later in more detail (Chap. 5 to Chap. 7).
4.3.1. Search Strategy
The present research applies a constraint model based on computational spaces. Compu-
tational spaces encapsulate constraint-based computations (which are speculative com-
73
4. Motivation and System Overview
putations) and that way make this constraint model programmable at a high level
[Schulte, 2002]. The present research applies this model for the rst time to a generic mu-
sic constraint system (to the knowledge of the author). Customised to music constraint
programming, this model eliminates many of the performance problems of previous sys-
tems and makes it possible to solve musical CSPs which were too complex for these
systems. Computational spaces are discussed later in more detail (see Sec. 7.2).
Propagate and Search
The space-based constraint model supports an approach to search called propagate-and-
search (also known as propagate-and-distribute). The success of propagate-and-search
is based on three important ideas [van Roy and Haridi, 2004]. Firstly, the approach
maintains partial information. Two pieces of partial information are the pitch of a note
n
1
is above middle c and note n
1
is lower than note n
2
(4.1).
4
pitch(n
1
) > 60 pitch(n
1
) < pitch(n
2
) (4.1)
Secondly, propagate-and-search performs local deductions (i.e. constraint propagation).
For instance, it can be deduced from the previous two partial information pieces that
note n
2
is above c next to middle c (4.2). Constraint propagation considerably reduces
the search space.
pitch(n
2
) > 61 (4.2)
Thirdly, propagate-and-search supports a programmable search process [Schulte, 2002].
Whenever no further local deductions can be done, a decision must be made by the
constraint solver. Such a decision is potentially wrong, it is possible that it must be taken
back. To conduct the search process eciently, it is therefore of crucial importance to
make decisions where a solution is found primarily by local deductions. This approach
reduces the risk of making wrong decisions. The power of propagate-and-search lies in
the fact that this decision-making process is fully programmable (Sec. 7.2 explains the
propagate-and-search approach in more detail).
Score Search
The present research customises the general space-based constraint model for music
constraint programming. Strasheelas music representation design allows the decision-
making process to fully exploit a partial solution score (e.g. all information accessible in
the music representation) at the time a decision a due. Suitable decisions can thus be
automatically computed during the search for a solution score.
For example, the search may progress in temporal order from left to right in the score
even if the temporal structure is undetermined in the problem denition. Alternatively,
4
In this example, pitches are represented by integer keynumbers, 60 represents middle c.
74
4.3. Strasheela Design Overview
the user may have extended the music representation such that it explicitly expresses a
Schenkerian hierarchy [Forte and Gilbert, 1982] of foreground, middleground and back-
ground. The user controls the order in which the search progresses. A user-dened deci-
sion strategy may rst determine the background, then the middleground and foreground
before nally determining the actual musical notes.
Applying the space-based constraint model xes a fundamental search strategy aw of
systems which rely on a xed and static search order pre-computed before the search
starts. The system score-PMC, for instance, does not support constraining the temporal
structure of music because this structure is required to pre-compute an ecient search
order. Applying propagate-and-search makes it possible for the rst time to eciently
solve complex polyphonic CSPs where the rhythmical structure, the pitch structure and
arbitrary further parameters are constrained.
This research proposes a number of novel decision making strategies or distribution
strategies (branching strategies) optimised for various musical CSP classes such as poly-
phonic or harmonic CSPs. These include strategies which rst determine temporal pa-
rameters or which determine parameters from left to right, i.e. in increasing order of
start times of the musical objects they belong to.
Because propagate-and-search decouples the CSP denition and the decision making
strategy, users can easily try out dierent strategies to nd the strategy most suitable
for their CSP. Furthermore, users can also dene their own decision making strategy if
so required.
In addition to this, with the propagate-and-search approach the local deductions (con-
straint propagation) are independent of the data structure. This xes a fundamental
limitation of Situation. Situation the only generic music constraint system supporting
constraint propagation so far (to the authors knowledge) performs propagation only
on a specic sequential music representation format.
The proposed search approach constitutes the foundation of the present research. Stra-
sheela supports very complex CSPs. Solving such complex problems requires ecient
search strategies to make these problems feasible in practice.
4.3.2. Rule Formalism
Previous systems introduced the concept of a compositional rule which encapsulates a
number of constraints (cf. Sec. 3.3.1.1 and Sec. 4.1.2).
A rule is often applied by a dedicated rule application mechanism which is part of the
rule denition. Common rule scopes are shared by many musical CSPs. For example,
in many CSPs a rule is applied to all pairs of neighbouring notes in a melody. Existing
systems dene convenient rule application mechanisms which cover typical cases. Yet,
the rule application mechanisms of existing systems only support specic rule scope cases
and exclude others (see Sec. 4.1.2).
75
4. Motivation and System Overview
Strasheelas rule formalism adopts these two principle ideas from existing systems. Firstly,
compositional knowledge is expressed in a modular way by compositional rules. Secondly,
the scope of a compositional rule is controlled by a rule application mechanism for con-
venience.
In addition, Strasheelas design overcomes a fundamental limitation of existing systems.
In previous systems, only the compositional rules are freely programmable. In Stra-
sheela, the rule application mechanisms are freely programmable as well. As a result,
the Strasheela user can freely control the scope of a compositional rule.
The present research bases its rule formalism on a well-understood formalism. A rule is
encapsulated in a function as a rst-class value with lexical scope [Abelson et al., 1985].
The application of a rule is consequently a function application.
Unlike previous rule formalisms, the formalism proposed here fully decouples the de-
nition and the application of a rule. That way, the user can freely apply each rule to
arbitrary sets of score objects by any control structure.
Strasheela supports the free denition of convenient rule application mechanisms. Such a
mechanism is implemented as a higher-order function which expects a rule (i.e. another
function) as argument. In contrast to previous systems which usually provide a single and
limiting rule application mechanism, the Strasheela user can freely select from a range
of predened rule application mechanisms. The Strasheela prototype already models the
index-based mechanisms of Situation (Sec. 6.2.5), the pattern matching mechanism of
PWConstraints (Sec. 6.2.6), and several mechanisms not supported by previous systems
(e.g. the application of a rule to any score object which meets some user-dened condi-
tion, Sec. 6.2.7). Additionally, the user can easily dene new rule application mechanisms
if so required (this is demonstrated in Sec. 6.2.4).
The rule formalism proposed here forms the basis for several contributions introduced
later. For instance, Strasheelas music representation is highly extendable the pro-
grammable rule application mechanism can be adapted for any extensions. First-class
functions are also essential for some Strasheela extensions (e.g. the motif model outlined
in Sec. 9.1.3.3).
4.3.3. Music Representation
The music representation design for Strasheela is guided by three requirements. The
music representation
Represents the solution score
Facilitates the expression of the musical CSPs
Supports the search process
76
4.3. Strasheela Design Overview
In the present section, the music representation design is motivated by explaining these
three requirements. The actual music representation design is only explained later in
Chap. 5.
4.3.3.1. Representation of the Solution Score
Any solution returned by Strasheela is expressed in the music representation dened by
this system. Consequently, whatever information should be represented in a solution (e.g.
the rhythmical and pitch structure of a polyphonic score, motif boundaries, or instrument
playing technique specications) must be representable in the music representation.
Section 2.5 explained that a music representation designer cannot foresee every possi-
ble use of the representation and that music representations are therefore often highly
programmable and user-extendable. Following this tradition, Strasheelas representation
design aims to (i) predene representation building blocks required for standard mu-
sical CSPs and (ii) support user extensions. This way, Strasheela user controls what
information is expressed by the representation.
4.3.3.2. Facilitating the CSP Denition
In addition to representing the solution, the music representation of a music constraint
system plays also an important role in the actual denition of a musical CSP.
For a generic music constraint system, it is paramount that the user is able to apply a
rule to every score context conceivable. This is potentially every possible set of variables
in the score.
Moreover, it is important to access score contexts in a way which is easy to remember
for the user, because otherwise the denition of complex CSPs becomes impracticable.
For instance, the expression of complex score contexts (e.g. the context for a passing
note denition) is hard to remember if dened by explicitly specifying numerical indices
of score objects. Instead, a score context should be accessed by specifying high-level
information such as the metric position of myNote. In case such a specication become
highly complex it can be encapsulated in a new data abstraction interface function with
a name with is easy to remember.
Strasheela provides highly expressive means to access complex score contexts. For in-
stance, Strasheelas music representation is based on the data abstraction concept (Sec.
2.6) and the data abstraction interface includes higher-order functions which can collect
all objects in the score fullling a user-dened condition.
4.3.3.3. Supporting the Search Process
The third requirement of the Strasheela music representation is related to Strasheelas
search approach. This approach allows the user to select a score decision strategy (distri-
bution strategy) optimised for the specic CSP at hand or even to dene novel decision
77
4. Motivation and System Overview
strategies if so required (see Sec. 4.3.1 and Chap. 7). All information available in the
score must be accessible to make the denition of arbitrary decision strategies possible.
Strasheela supports the denition of complex score decision strategies which can exploit
any information available in the score at the time of the decision. Strasheela compares
score parameter objects (e.g. note pitches or voice durations) which encapsulate con-
strained variables to decide which parameter to distribute. Each parameter object pro-
vides access to the score object (e.g. note or voice object) that the parameter belongs
to. In turn, each note or voice provides access to the whole score as seen from this score
object (e.g. the neighbouring notes in a voice).
78
5. The Strasheela Music Representation
This chapter introduces the Strasheela music representation. This representation com-
plements concepts established in existing music representation research (and presented
in Chap. 2) by the notion of partial information (i.e. constrained variables). The rep-
resentation was designed with the requirements of a generic music constraint system in
mind.
Chapter Overview
After a brief overview of the essential Strasheela score object types (Sec. 5.1), this chap-
ter rst discusses fundamental music representation design decisions. The Strasheela
representation is based on the concept of data abstraction. Its abstract data types are
instances of classes organised in a class hierarchy (Sec. 5.2). The abstract representation
is complemented by a textual representation (Sec. 5.3).
Subsequent sections explain the organisation and usage of a Strasheela score, in particular
how score information is stored and accessed. Section 5.4 details the hierarchical structure
of a Strasheela score and Sec. 5.5 discusses the data abstraction interface. Finally, Sec.
5.6 shows how information required to dene complex musical CSPs can be accessed.
5.1. Overview: Score Object Classes
The Strasheela music representation adopts several established music representation con-
cepts which were discussed in Chap. 2. These concepts are implemented by a set of score
object types (classes) in Strasheela. Similar score object types can be found in existing
music representations (although they possibly dier in terminology).
As an orientation for the remainder of this chapter, Tab. 5.1 provides a brief overview
which links fundamental music representation concepts already discussed before with
the related Strasheela score object types. Each score type listed is introduced in more
detail later in this chapter.
1
The table also references the section of this thesis where the
respective concept or type is discussed.
1
These score object types are documented in detail in the music representation section of
the prototype reference documentation at http://strasheela.sourceforge.net/strasheela/doc/
StrasheelaReference.html.
79
5. The Strasheela Music Representation
Type (Class) Purpose Section
scoreObject The most general score data type.
Musical magnitudes (cf. Sec. 2.3)
parameter A composite data type representing a basic musical
magnitude whose value can be constrained.
5.4.1
timePoint A parameter representing a time point numerically
(e.g. the start time of a note).
5.4.1.1
timeInterval A parameter representing a time interval numerically
(e.g. a duration).
5.4.1.1
pitch A parameter representing a pitch numerically. 5.4.1.1
Score topology representation (cf. Sec. 2.4.3 and 2.4.4)
item A generalisation of the types container and element
(explained below). An item has parameters and
containers.
5.4.2
container An item which contains one or more other score items
(e.g. notes or other containers).
5.4.3
element An item which does not contain other items. 5.4.2
Temporal objects (cf. Sec. 2.2 and 2.4.3)
event An element which produces sound when the score is
played.
5.4.2
note An event which always has a parameter pitch. 5.4.2
sequential A container which expresses that the items contained
in it follow each other in a sequential manner in time.
5.4.4
simultaneous A container which expresses that the items contained
in it occur parallel in time (contained items can spec-
ify a start time oset).
5.4.4
Table 5.1.: Fundamental Strasheela score object types (classes)
80
5.2. Class Hierarchy
5.2. Class Hierarchy
Existing music representations often organise score data types by means of object-
oriented programming (OOP, see Sec. 2.7). Strasheela adopts this proven software design
approach for multiple reasons. This representation
Consists of abstract data types (i.e. data types which are dened by their interface).
Denes a hierarchic typing structure.
Makes the representation highly user-extendable.
The UML diagram in Fig. 5.1 shows an excerpt of the Strasheela music representation
class hierarchy. Depicted are the relations between the classes used to represent temporal
information. Most of these classes were briey explained in Tab. 5.1 and will be explained
in subsequent sections.
Figure 5.1.: Temporal classes of the Strasheela music representation class hierarchy
The following paragraphs explain further the main reasons why Strasheela uses the
object-oriented programming paradigm.
Abstract Data Types
OOP provides a suitable way to dene abstract data types (ADT). Section 2.6 explained
the advantages of ADTs for a music representation. Data abstraction encapsulates the
implementation of data the data is only used via the data abstraction interface.
81
5. The Strasheela Music Representation
An expressive data abstraction interface plays a very important role in the denition of
musical CSPs. For example, many CSP require accessing information derived from other
information stored in the representation (e.g. to access and constrain the interval between
two note pitches). Data abstraction can express such information in a concise and highly
legible way. In addition, complex CSPs require to access arbitrary score contexts (as was
explained in Sec. 4.1.1).
Hierarchic Typing Structure
The Strasheela music representation denes a large number of score types which are
related by is-a relations (e.g. a note is an item is a scoreObject, cf. Tab. 5.1). Many
musical CSPs make use of this hierarchic typing structure. For instance, a CSP may
collect all instances of the class item in the score to constrain them in a certain way
regardless whether these items are instances of the note class or of some container class.
The inheritance scheme of OOP conveniently models such is-a relationships. Subclasses
inherit the interface of their superclasses (e.g. the class pitch inherits the type check
function of its superclass parameter) and may extend (and sometimes even overwrite)
this interface. This results in a much more concise and modular implementation of the
representation than an implementation of separate types would be.
User-Extendable Representation
Furthermore, inheritance allows for convenient user-dened extensions of the Strasheela
music representation. A user class can inherit from a Strasheela class which provides a
generalisation of the intended functionality.
Because of its extensible nature, the Strasheela design aims to keep the representation
core small. The Strasheela representation core provides a number of style-independent
concrete representation classes plus abstract classes which make implementing user ex-
tensions easier. For instance, the representation core provides concrete classes to rep-
resent the temporal structure of music (because music essentially happens in time).
Explicit abstract classes such as the element class are described later in this chapter.
The Strasheela prototype already provides several extensions of this style-independent
representation core which are mentioned in Sec. 9.1.3.
A Side Note: Strasheela is Stateless
Object-oriented programming is often stateful, but the Strasheela music representation
is stateless.
2
Stateless programming allows for declarative programming (including con-
straint programming), which is easier to reason about than stateful (or imperative) pro-
gramming. Strasheelas music representation can incorporate constrained variables, that
is, partial information. During the search process, more information about these vari-
ables becomes known (i.e. the size of each variables domain decreases) and eventually
2
Within a CSP and for the purpose of this thesis, the Strasheela music representation is purely stateless.
Still, the Strasheela prototype (Chap. 9) also denes stateful operations, for example, to edit a
solution.
82
5.3. Textual Representation and Data Abstraction
all variables become determined. Still, the value of a variable is never statefully changed
during search (i.e. it is not bound to any new value) only their domain is narrowed
down. Adding information about a partial value complies with declarative programming,
whereas a stateful binding operation would not.
5.3. Textual Representation and Data Abstraction
Section 2.6 compared the respective benets of a textual music representation and a
representation based on the concept of data abstraction. It was argued that a textual
representation allows easy editing: this is of great importance for a music composition
tool. An abstract representation, on the other hand, can hide the implementations de-
tails: this is crucial when representing something as complex as a score.
The Strasheela design combines the benets of a textual and an abstract representa-
tion. The prime Strasheela music representation is a score ADT (consisting of instances
of Strasheela classes), but Strasheela complements this abstract representation with a
textual representation form.
Strasheela combines these two representational forms by making them transformable
into each other. The textual form is transformed into the ADT by a special constructor
function which is part of the score data abstraction interface. This interface also provides
a function which transforms the abstract score representation into the textual form.
Transforming the textual representation into the ADT allows the Strasheela user to
determine various score settings textually. For instance, the user may textually specify
the number of voices in the score, determine the value of certain note parameters (e.g.
their duration or pitch), or specify a domain for these parameter values.
Transforming the score ADT back into the textual form can be useful, for instance, to
archive a solution score as a text le using the internal Strasheela music representation
or even to edit a solution score such that it can form the input for another CSP.
The Textual Representation Format
This paragraph outlines Strasheelas textual representation format. This textual repre-
sentation is explained fully below, after the main Strasheela classes are introduced (see
Sec. 5.4.3.2).
The textual representation uses records to represent Strasheela objects. A record is a
literate composite data structure consisting of a label, a number of features and feature
values (see App. A.4). For instance, a note can be represented as a record with the three
features startTime, duration and pitch as in Fig. 5.2.
Hierarchically nested scores are textually expressed with nested records, often in com-
bination with lists (see the list of voice notes in Fig. 5.3). Section 5.4 explains the
hierarchical structuring of a score in detail.
83
5. The Strasheela Music Representation
note(startTime: 0, duration: 4, pitch: 60)
Figure 5.2.: A single note, textually represented
Transforming a Textual Form into an Abstract Representational Form
Strasheela proposes a highly exible transformation from the textual to the abstract
representation which is sketched in this paragraph. The function makeScore performs this
transformation from the textual to the abstract representation. In Fig. 5.3, makeScore
performs such a transformation on a textual score consisting of a voice with two notes.
abstract scoreObject) = makeScore(voice(items: [note(pitch: 60),
note(pitch: 62)]))
Figure 5.3.: The function makeScore transforms a textually represented score into a score
ADT (a class instance)
The transformation process conducted by makeScore is user-controllable. A record label
always denotes the meaning of a textually represented score object. For example in
note(duration:1), the label note denotes the meaning of this textual object. Usually,
note(duration:1) means an instance of the class note. Strasheela predenes the meaning
for record labels for all concrete classes in the Strasheela music representation core.
However, the user can optionally extend or overwrite these default meanings by mapping
any class (or constructor function) to a record label.
In Fig. 5.4, the user has overwritten the default meaning of the record labels voice and
note by mapping a class to each label. Classes are rst-class values in the program-
ming language user-interface of Strasheela, as are functions. The freedom to overwrite
the meaning of records with a label allows the user, for instance, to call some subsub-
class of note still simply a note in the textual representation, and to avoid names like
noteWithFeaturesFooAndBar.
makeScore(voice(items: [note, note]),
types (voice: default voice class),
note: my special note class)))
Figure 5.4.: For each type in the textual score, its class in the ADT (or a constructor
function) can be specied as argument to the function makeScore
84
5.4. Hierarchic Representation
5.4. Hierarchic Representation
This section explains the hierarchic structure of a Strasheela score. The section surveys
the most important Strasheela score object classes (which were briey listed in Tab. 5.1)
and their relationships in a score.
The structure of the present section is as follows. Section 5.4.1 introduces the parameter
class. The event class and its textual representation is discussed in Sec. 5.4.2. Section
5.4.3 presents the container class, discusses Strasheelas score topologies and provides
the full textual representation syntax. Finally, Sec. 5.4.4 describes how Strasheela ex-
presses a temporal hierarchy and concludes with a more complex example of a textually
represented score.
5.4.1. The Parameter Class: Variables in the Representation
Section 2.3 introduced the parameter concept as a basic magnitude in the music repre-
sentation. Typical examples include the parameters pitch and amplitude. The literature
also discusses the diculties in representing parameters. For example, dierent music
representations express parameters in clearly dierent ways (e.g. using dierent units of
measurement for the parameter values).
Strasheela denes a composite data structure to represent musical magnitudes. The
class parameter has a number of attributes. The UML diagram in Fig. 5.5 shows these
attributes: info, item, value, and unit.
Figure 5.5.: The parameter class
The parameter attribute value stores the actual magnitude value of a parameter instance.
It is important to note that this attribute points to a constrained variable, that is a vari-
able with a domain (e.g. a nite domain integer). A determined value (e.g. a determined
integer) is a special case for a constrained variable. In fact, constrained variables occur
only as parameter values in the core Strasheela music representation.
The attribute unit explicitly denotes the unit of measurement for the parameter value
attribute. This information is used, for instance, during the transformation of a score
into dierent output formats (see Sec. 9.1.1). For example, a pitch parameter with the
value = 60 and unit = midi denotes middle c.
The attribute item contributes to the hierarchic nesting of a score. This attribute points
to the item object (i.e. an event or container, see below) the parameter belongs to. For
85
5. The Strasheela Music Representation
example, in the case where the parameter instance represents a note pitch, then the
attribute item points to the note instance to which the parameter belongs.
Finally, the attribute info stores arbitrary additional information about the parameter.
Each score object has an info attribute. The info attribute can contain multiple pieces
of information.
An example of parameter instances in the UML is given later together with an item to
which these parameters belong (see Fig. 5.9).
5.4.1.1. Parameter Subclasses
Strasheela predenes several parameter subclasses. Parameter subclasses include pitch,
amplitude, timePoint and timeInterval . These subclasses inherit the four parameter at-
tributes info, item, value, and unit. Each parameter subclass supports specic data
which can be stored by these attributes.
Parameter subclasses dier in the unit of measurement they support. Typical pitch units
are frequency and midi (i.e. key-number). An example of an amplitude unit is (MIDI)
velocity. Temporal units include seconds and beats (i.e. crotchets per second). Beat sub-
divisions are denoted with the temporal unit specication beats(n). This specication
expresses that the parameter value n (an integer) stands for a single beat, 2n for two
beats etc. For instance, with the temporal unit specication beats(3) the duration 1
expresses a quaver triplet.
Strasheela denes a number of parameter unit variants with smaller resolution such as
milliseconds and midicent (in midicent, 6050 expresses a quarter note above middle c).
These units are unnecessary for constraints on reals, but constraints on nite domain
integers are particularly well supported by constraint systems which provide constraint
propagation such as Strasheela (the consequences of constraint propagation is explained
later in Sec. 7.2.3.1 and Sec. 10.1.2). A further pitch unit stands for alternative tunings
used in microtonal music. For example, several composers (e.g. Ivan Wyschnegradsky,
Franz Richter Herf, and Ezra Sims) applied a temperament which subdivides an octave
into 72 equal steps, because such tuning allows for an almost just intonation of 11-limit
intervals [Monzo, 2005, article 72-edo].
Some existing music representations dene specic parameters with multiple values. For
instance, CHARM (Sec. 2.3, [Harris et al., 1991]) represents a single pitch by three val-
ues: the note name, the accidental, and the octave. This pitch representation is more
expressive than a single value representation but is restricted to Western music. The
Strasheela core representation aims to be style-independent and therefore does not pro-
vide such complex parameter representations. However, parameter representations like
this can be dened by the user, due to the extensible nature of Strasheela. In fact, the
Strasheela prototype already denes a constrainable harmony model extension which
includes a multiple-value pitch representation consisting of note degree, accidental, and
octave (see Sec. 9.1.3.2).
86
5.4. Hierarchic Representation
5.4.2. The Event Class
Section 2.2 introduced the notion of an event as a score object which produces sound
when the score is played. The Strasheela class event features two particularly important
attributes: parameters and containers (see Fig 5.6). Both attributes contribute to the
hierarchic nesting of a score. The attribute parameters references to all parameter objects
which belong to an event. The attribute containers, on the other hand, references all
containers the event is contained in (the container class is explained shortly in Sec. 5.4.3).
Figure 5.6.: The event class
An event constitutes a very general notion of a sound-producing score object. Because an
event simply collects a number of parameters, a hand clapping (with temporal parameters
and optionally a specic loudness), a note played on a piano (with a specic pitch,
loudness etc.), or an arbitrary sound synthesis language event (possibly with dozens of
parameters) are all representable as an instance of the event class (or some subclass of
it).
Nevertheless, many Strasheela classes are dened such that their instances are guaranteed
to contain a certain minimal set of parameters. For instance, an event always contains
four temporal parameters: osetTime, startTime, duration, and endTime (the precise
meaning of these parameters and the reasons why this parameter set is not redundant is
detailed in Sec. 5.4.4). In such an object, the attribute parameters contains this minimal
parameter set plus user-specied additional parameters. For convenience, however, such
a class usually denes an explicit attribute for each parameter of the minimal parameter
set as well.
3
For example, the class note an event subclass inherits the explicit
attributes for the four temporal parameters (start time, duration, etc.) and additionally
denes two further explicit parameter attributes pitch and amplitude (see Fig. 5.7). An
example of a note class instance is given below (after its textual representation was
introduced, see Fig. 5.8 and Fig. 5.9).
Important functionality of the class event is inherited (see Fig. 5.1). The abstract class
item is an important superclass of both the event class and the container class. The
attributes parameters and containers are in fact inherited from the item class. Another
event superclass is the abstract class element. Whereas events always produce sound,
other element subclasses can be silent. For example, an initialisation statement for a
sound synthesis language (such as a f-statement for Csound, which places values in a
lookup-table [Boulanger, 2000]) could be represented by a special element subclass.
3
The additional attributes point to the same parameter objects which are also contained in the param-
eter list.
87
5. The Strasheela Music Representation
Figure 5.7.: The note class
5.4.2.1. Textual Representation of Events
Strasheela supports a textual music representation which can be transformed into an
equivalent abstract representation (see Sec. 5.3). For convenience, parameter objects are
not explicitly notated in this textual representation. Instead, the parameter values and
parameter units are given directly to their containing object (e.g. a note). The expression
in Fig. 5.8 creates a note object which corresponds to the note in common music notation
next to it. This example shows how parameter values (e.g. startTime, duration, pitch)
are given directly to their containing note. The meaning of the midicent pitch unit was
explained in Sec. 5.4.1.1.
To express the declarative semantics (or logical semantics) of Strasheela examples, pa-
rameter instances as separate objects are dispensable. Instead, it would be sucient to
store the attributes of any parameter instance as attributes of their containing item in-
stance as suggested by the textual representation. Still, the parameter class abstracts
pieces of related information such as the parameter value and its unit of measurement.
To express the operational semantics of Strasheela, on the other hand, it is essential to
express parameter instances as separate objects. The search process traverses parameter
objects individually and requires accessing arbitrary information about the score as seen
from a separate parameter. The operational semantics of Strasheela are detailed in Chap.
7.
makeScore(note (info: testNote,
startTime: 0,
duration: 2,
pitch: 6050,
timeUnit: seconds,
pitchUnit: midicent))

= 60

Figure 5.8.: The textual representation of a single note and this note in common music
notation
The UML instance diagram in Fig. 5.9 shows the hierarchical structure of a single note:
88
5.4. Hierarchic Representation
the note object itself and multiple parameter objects.
It should be noted that the link between the note and its parameters is bidirectional: a
note references its parameters (via its attribute parameters) and each parameter refer-
ences the item it is contained in (via its attribute item).
Figure 5.9.: The hierarchic structure of a single note and its contained parameters
For convenience, Strasheela denes default values for the attributes of most of its classes.
These attributes are thus optional in the textual representation. For instance, all note
parameter values default to a constrained variable (a nite domain integer). The only
exception is the osetTime attribute, which defaults to 0 (the reason for this exception
becomes clear in Sec. 5.4.4.1). Parameter units also have a default value and are thus
optional. For example, the pitchUnit default is midi (see Sec. 5.4.1.1).
5.4.3. The Container Class
Section 2.4 showed the importance of structuring a score by grouping sets of score objects
with the help of containers. An instance of the Strasheela class container contains one
or more score items (e.g. notes or other containers) and so allows the construction of
hierarchically structured scores.
A container features three attributes which contribute to the hierarchic nesting of a score
(see Fig 5.10). The container class inherits the attributes parameters and containers from
the class item, and these attributes serve the same purpose as for other item subclasses
(e.g. for the event class, see Sec. 5.4.2). The new attribute items references to all objects
directly contained in the container.
An example of a container class instance is shown later (Fig. 5.11 and Fig. 5.12) after
an explanation of the textual representation of containers.
89
5. The Strasheela Music Representation
Figure 5.10.: The container class
5.4.3.1. Textual Representation of a Container with Contained Items
Strasheela supports a textual representation for a container and its contained items,
which allows the convenient creation of a highly nested score ADT from its textual
representation.
The attribute items of the textual container representation expects a list of the con-
tained items. Figure 5.11 depicts a short musical fragment and shows how this nested
example is notated in the textual form. The example creates a container with several
notes (a sequential container expresses that its contained items are arranged sequentially
in time, see Sec. 5.4.4). In the example, the timeUnit specication means that a beat (a
crotchet) has the duration 4 (i.e. the duration 1 denotes a semiquaver) and the pitchUnit
is implicitly set to its default midi .
makeScore(sequential (info: myVoice,
items: [note(duration: 2, pitch: 60),
note(duration: 1, pitch: 62),
note(duration: 1, pitch: 64),
note(duration: 4, pitch: 65)],
startTime: 0,
timeUnit: beats(4)))

Figure 5.11.: The textual representation of a container with several notes and the corre-
sponding music in common music notation
The resulting hierarchic structure is shown in Fig. 5.12 (the parameter objects of the
notes and the container are omitted for clarity). As with the link between an item instance
and its parameters, the link between an item and its containers is bidirectional. The
back-links (i.e. from an item instance to its container and from a parameter instance
to its item) are not specied explicitly in the textual representation but are created
90
5.4. Hierarchic Representation
automatically for convenience.
4
Figure 5.12.: The hierarchic structure of a container with several contained notes
5.4.3.2. The Full Textual Representation Syntax
Section 5.3 explained how the actual meaning of the textual Strasheela music represen-
tation is controlled by the user. The user denes the meaning of a textually represented
item by mapping its label to either a class or a constructor function. Figure 5.13 presents
the syntax of the textual Strasheela music representation more formally.
5
This syntax specication shows that any score is an item and that an item can have
several forms. Each textual item specication (either element or container) starts with
a label (usually a symbol) and surrounds a number of feature-value pairs by parenthesis
(the specication is slightly simplied by omitting the commas between feature-value
pairs). Additionally, a container has a special feature-value pair for its contained items
(feature-value pairs can appear in any order, the feature items is not necessarily the
rst). In general, all features of an item are optional (an item without any feature at all
can also be notated without the parenthesis such as note instead of note()).
The textual music representation of Strasheela is actually a composite value of its pro-
gramming language user-interface (i.e. mathematical notation in this text and the Oz
4
The containers of an item instance can be specied explicitly, for instance, to express that the item is
contained in multiple containers (see Sec. 2.4.4 and Sec. 5.4.3.3).
5
The syntax is specied in the Extended Backus-Naur Form (EBNF) as used by van Roy and Haridi
[2004]. Non-terminal symbols are surrounded by angle brackets as x, literal terminals are surrounded
by single quotes as x, alternatives are separated by a bar as x|y, and optional items are sur-
rounded by brackets as [x]. The ellipsis (. . . ) species an incomplete EBNF rule, e.g., that at the
end of the rule further alternatives are omitted. The sux * (Kleene star) means that the preceding
item is repeated zero or more times whereas the sux + means that the preceding item is repeated
at least once. Multiple symbols can be grouped by round parenthesis to specify a sux for the whole
group.
91
5. The Strasheela Music Representation
score) ::== item)
item) ::== element) [ container) [ expression)
element) ::== label ) ( ( feature) : expression) )* )
container) ::== label ) ( [items: items) ] ( feature) : expression) )* )
items) ::== [ item)+ ] [ expression)
label ) ::== symbol ) [ variable)
feature) ::== symbol ) [ variable) [ . . .
Figure 5.13.: Syntax of the textual Strasheela music representation (tree representation
subset)
programming language for the Strasheela prototype). Consequently, any expression in
this programming language is permitted in the textual representation. For example, sub-
parts of the textual score may be expressed by a variable or generated algorithmically
by a function call.
Furthermore, in the textual Strasheela music representation the textual and the abstract
representation can be mixed. In the textual representation, an item is either the textual
representation of the item or an item instance (i.e. an ADT). The latter option permits
a modular denition of large musical CSPs where sub-CSPs on subparts of the score are
dened separately. Later, these parts are combined and further constraints are possibly
applied to the resulting score.
Such an incremental score construction provides also a convenient means to access the
score context out of an existing score. Using this approach, the Strasheela user constructs
parts of the score (e.g. individual voices) independently and binds these score parts to
identiers. These parts are then combined into the full score. In eect, each of these
identiers references an individual score context (e.g. a specic voice).
5.4.3.3. Score Topologies
The preceding sections showed that a Strasheela score is a highly nested data structure.
Each Strasheela parameter contains a link to its respective item (i.e. an element or a
container), each item links to its parameters and containers (which can be nil), and
each container links to its parameters, containers, and contained items. This nested data
structure allows the user to explore the whole score regardless of where in the score this
exploration starts.
The Strasheela music representation supports dierent score topologies (see Sec. 2.4). If
it is ignored for a moment that all links are bidirectional, the supported score topologies
include the event list (i.e. a at list of events contained in a container), the tree-shaped
92
5.4. Hierarchic Representation
nesting of containers and events, and the nesting of containers and events in an acyclic
graph. In an acyclic graph topology, a container can feature any number of items and
an item can feature any number of containers.
For the purpose of a generic music constraint system, the acyclic graph topology (Sec.
2.4.4) looks particularly promising at rst sight, because it is the most expressive topol-
ogy. This topology explicitly represents an unlimited number of orthogonal score contexts
(i.e. sets of score objects, see Sec. 2.4 for an explanation of this term). Example contexts
include the set of notes in a specic voice, the notes in a specic motif, the notes in
a specic bar, the set of simultaneous notes in a certain time frame, etc. For a generic
music constraint system, it is important to have all these contexts potentially available
for the application of rules. When these contexts are explicitly represented by nesting
its members in a container, these contexts are indeed easily accessible.
The practical use of this graph topology, however, is limited, because this topology often
narrows the set of solutions for a musical CSP too much. For example, the rhythmic
structure of a CSP is already very restricted if the hierarchic nesting already determines
that a certain measure contains one set of notes, a certain time frame contains another
set, and a certain motif contains a further set of notes. Such a situation allows the user
to only apply rhythmic rules which do not conict with the rhythmic structure caused
by the hierarchic nesting.
Instead of the graph topology, Strasheela examples usually apply only a tree-shaped
topology. This topology realises a pragmatic balance between adding useful information
to the score by nesting (e.g. expressing that a certain set of notes belongs to a certain
voice in the score) without predetermining the CSPs solution too much. Additional score
contexts are then expressed in alternative ways which are detailed in Sec. 5.6 and Sec.
6.3.
However, for certain musical CSPs even the tree-shaped topology narrows the set of
solutions too much. For example, the number of notes in a voice is already determined
when a voice is represented as a container containing a certain set of notes. Section 8.3
proposes a less restricting approach which allows the user to constrain, for example, the
number of notes in a voice by still retaining the additional information expressed by
hierarchically nesting these notes in a voice.
5.4.4. Temporal Hierarchy
Research on music representation demonstrated the expressive power of temporal con-
tainers which allow the user to explicitly express the temporal structure of music in
various ways (see Sec. 2.4.3). The Strasheela music representation allows the user to ex-
press and constrain the temporal structure of a score by dening a set of temporal items
with temporal parameters. Every temporal element and temporal container features the
four parameters startTime, osetTime, duration, and endTime. Like any parameter do-
main, the domain of each temporal parameter value can be specied by the user: the
domain of startTime, duration, and endTime defaults to N (i.e. a nite domain integer
93
5. The Strasheela Music Representation
where the maximum domain is platform dependent, for Oz it is 2
27
2) and includes 0,
the default value of osetTime is 0.
This set of parameters seems redundant at rst. For example, if both the start time
and the duration of an item are determined, the end time can be calculated and does
not need an explicit representation. However, in the Strasheela music representation,
parameter values are constrained variables which are not necessarily determined. If these
four parameters are represented explicitly, then each parameter can be freely constrained.
Strasheela denes the two temporal classes sequential and simultaneous as subclasses of
the container class. An instance of the class sequential expresses that its contained items
follow each other in a sequential manner in time. Accordingly, an instance of the class
simultaneous expresses that its contained items run in parallel with each other.
All attributes of the classes sequential and simultaneous are inherited (see Fig. [class
hierarchy]) and have been discussed above. Figure 5.14 shows these attributes for the
sequential class, the simultaneous class features an identical set of attributes (instance
examples with these containers are shown in Fig. 5.11/5.12 and Fig. 5.18/5.19).
Figure 5.14.: The sequential class
The topology of the temporal hierarchy is a tree, but never a graph: an item must not
be directly contained in multiple temporal containers. Otherwise, the temporal structure
expressed by dierent containers would conict.
5.4.4.1. Implicit Temporal Constraints
Strasheela ensures that the temporal structure of a score is always well-formed even
if the temporal structure is undetermined in a CSP denition. To this end, Strasheela
denes a set of implicit constraints on the temporal parameters of temporal items. These
are the only implicit constraints which the Strasheela core representation applies to a
score; further constraints are explicitly applied by the user.
For all temporal items, Strasheela implicitly constrains the start time, duration, and end
time of every temporal item to have the expected relation (Fig. 5.15).
The parameter osetTime expresses a pause before an item (which defaults to 0). The
way in which the oset time of an item aects the start time of this item depends on
94
5.4. Hierarchic Representation
EndTime
item
StartTime
item
= Duration
item
Figure 5.15.: Implicit constraint for every temporal item
the type of its surrounding temporal container (the oset time has no further inuence
for a single note without a surrounding temporal container).
The oset time of an item contained in a simultaneous container denotes how much the
start time of this item is delayed with respect to the start time of the container. Figure
5.16 shows the implicit constraints between the temporal parameters of the simultaneous
container and all its contained items; n denotes the number of items in the container.
n

i=1
StartTime
simItem
i
= StartTime
sim
+ OsetTime
simItem
i
EndTime
sim
= max(EndTime
simItem
1
, . . . , EndTime
simItem
n
)
Figure 5.16.: Implicit constraints for every simultaneous container and its contained
items
The oset time of an item contained in a sequential container species a pause between
the item and its predecessor item. For the rst item in a sequential container, its oset
time species how much the start time of the item is delayed with respect to the container
start time (Fig. 5.17).
StartTime
seqItem
1
= StartTime
seq
+ OsetTime
seqItem
1

n1

i=1
StartTime
seqItem
i+1
= EndTime
seqItem
i
+ OsetTime
seqItem
i+1
EndTime
seq
= EndTime
seqItem
n
Figure 5.17.: Implicit constraints for every sequential container and its contained items
The correctness of these implicit constraints requires that all constrained temporal pa-
rameters have the same unit of measurement. Therefore, all temporal parameter units
are constrained to be equal (i.e. they are unied). For this reason, Strasheela does not
dene a default temporal unit of measurement (otherwise, a user-specied unit of some
temporal parameter and the default unit of other parameters may be inconsistent). This
unit is therefore explicitly specied once in every score.
Strasheela also denes the temporal element pause as an alternative device to express a
temporal gap before any item. Specic musical CSPs are more conveniently dened by
using pauses as separate objects. However, often the explicit use of pauses narrows the
solution set too much: the user may want to constrain whether or not a pause happens
before a certain item. The parameter oset is therefore intended as a more exible
95
5. The Strasheela Music Representation
substitute for the pause object. By constraining the oset of an item, the user can
constrain whether the item is preceded by a pause (namely when the OsetTime > 0)
and the duration of this pause. Alternatively, the user may place explicit pause objects
wherever a pause may occur in the solution and include 0 in the duration domain of
these pauses.
5.4.4.2. Textual Representation of the Temporal Hierarchy
The textual representation of nested temporal items corresponds with the general textual
Strasheela representation introduced before (see Sec. 5.4.3.2). The next example shows
a score which is slightly more nested than previous examples. Firstly, Fig. 5.18 shows
the topology of this nesting.
The root of this hierarchy is a simultaneous container which contains two sequential
containers. In a polyphonic example, sequential containers can represent voices (cf. Sec.
2.4.3). Finally, each sequential container contains several notes. As before, parameters
objects are omitted for brevity.
Figure 5.18.: The hierarchic structure of a polyphonic example (represented by a tree-like
nesting of containers and elements)
Figure 5.19 shows this polyphonic example in common music notation. In all previous
examples, only determined parameter values were specied. In this example, the note
pitches are still undetermined and only denoted as a bar between the minimum and the
maximum of the pitch domain.
Figure 5.20 shows how such a nested score can be created from the textual representation
in a concise way. Previous examples specied the whole score explicitly in textual form.
For the present example, an explicit textual form would be unnecessarily long. Therefore,
the example rst denes the auxiliary function makeNotes which returns a list of notes.
When the actual score is created (with the call to the function makeScore), the function
makeNotes is called in the specication of each sequential container (the example uses
the shortcut labels sim and seq to denote the two temporal containers).
96
5.5. The Data Abstraction Interface

Figure 5.19.: A polyphonic example where the note pitches are undetermined
let /* The function makeNotes returns a list with n textual notes. For each note, the duration
is 1 and the pitch is an undetermined integer with the domain {60,. . . ,72} (created by the
function fdInt). collectN calls a null-ary function n times and returns the collected results in
a list: every note is created by a function call to create a fresh constrained variable for each
note pitch (i.e. multiple notes do not share the same pitch variable). */
makeNotes(n)
def
= collectN(n, f : f()
def
= note(duration: 1,
pitch: fdInt(60, . . . , 72)))
in makeScore(sim (items: [seq (info: alto,
items: makeNotes(3)),
seq (info: tenor,
items: makeNotes(3))],
startTime: 0,
timeUnit: beats))
Figure 5.20.: The textual representation of a polyphonic example (parts of the score are
not specied explicitly but created by function calls)
In this example, the pitch value of each note is a constrained variable with a specied
domain, namely all integers in 60, . . . , 72 (i.e. the pitches in the octave above middle
c the pitchUnit defaults to midi ). These variables are created with the function fdInt.
Section 2.4 showed how a brief Bartok example (Fig. 2.3) can be represented by dierent
score topologies. The topologies based on temporal containers (Fig. 2.6 and Fig. 2.7) can
both be reproduced with Strasheelas temporal containers analogously to the examples
already shown in Fig. 5.11 and Fig. 5.20.
5.5. The Data Abstraction Interface
Strasheela denes its music representation as abstract data types (Sec. 5.3), organised in
a class hierarchy (Sec. 5.2). This section introduces the rich interface of Strasheela score
objects.
The structure of the present section is as follows. Firstly, Sec. 5.5.1 discusses the ba-
sic interface (dening typical ADT functions such as constructors, accessors and type-
checkers). In addition to the basic interface for single objects, Strasheela also denes
97
5. The Strasheela Music Representation
interface functions which process multiple hierarchically-nested score objects together
(Sec. 5.5.2).
5.5.1. Basic Interface
The interface of an abstract data type often denes a standard set of operators. Typically,
an interface subsumes a constructor, a type-checker, an equality-checker, and a number
of accessors to attributes of the ADT (Sec. 2.6). Strasheela denes all these interface
operations for each ADT of the representation (i.e. for each class).
As mentioned above, the Strasheela music representation is purely stateless (Sec. 5.2).
Strasheela denes all interface operations as functions (or polymorphic methods) without
side-eects. As a stateless representation, however, Strasheela does not dene modiers.
6
Similarly, Strasheela does not dene destructors.
7
The basic constructor for each score class is complemented by the previously introduced
score constructor function makeScore, which transforms a nested textual score represen-
tation into the corresponding nested score ADT (see Sec. 5.3). In eect, the user applies
the standard constructors only rarely.
Strasheela denes a hierarchy of types (i.e. a class hierarchy). A type-checker tests
whether a certain object is an instance of a certain class or of a subclass of this class.
For example, both methods isNote and isEvent return true for an instance of the class
note.
Strasheela provides a binary function which tests whether two score objects are equal
(i.e. are the idential value).
8
For each score object, Strasheela denes a set of accessor methods to object attributes.
For instance, the expression getItems(myContainer) returns the list of items directly
contained in myContainer. Similarly, getParameters(myItem) returns all parameters of
myItem in a list.
For convenience, Strasheela denes accessors to single parameters in two versions: one
version directly returns the parameter value and the other returns the parameter object.
6
The actual Strasheela implementation does indeed dene modiers as well: stateful operations allow the
binding of ADT attributes to fresh logic variables. Modiers are important, for instance, to implement
ecient score transformation operations or a graphical score editor. Nevertheless, these examples are
outside the scope of this thesis: for the purposes here, the Strasheela music representation is purely
stateless.
7
A Strasheela implementation language will perform garbage collection (as virtually all languages with
support for functional programming) and destructors are therefore not needed.
8
Such an equality checker will often be provided by the implementation language. In case of the Stra-
sheela prototype, Oz provides this function. The Oz operator == tests whether two values are equal.
For example, the operator returns true when called with two unied variables or two values for
which unication would succeed. For values which cannot be unied (stateful values, including class
instances) the equality checker only returns true when called with two identical values (e.g. two
variables referencing to the same object). However, Strasheela denes the methods == and unify for
score objects. These methods internally transform the possibly nested score object into its textual
representation and then perform an equality check or unication.
98
5.5. The Data Abstraction Interface
The name of the rst accessor simply reects the parameter name, the latter accessor
usually ends with the additional sux Parameter. Thus, getPitch(myNote) returns a
constrained variable representing the note pitch value and getPitchParameter(myNote)
returns the pitch parameter object.
Strasheela denes many more accessor functions, which perform more complex processing
operations instead of simply accessing the direct attributes of an object. Some of these
functions are introduced in the subsequent sections.
A special accessor method is hasThisInfo which returns a boolean value. In Fig. 5.21,
hasThisInfo checks whether one of the information chunks stored in myNote equals
marker. Additional information chunks can be added to a score object with the method
addInfo.
hasThisInfo(myNote, marker)
Figure 5.21.: The method hasThisInfo checks whether the info-attribute of a score object
contains a specic piece of information
5.5.2. Generic Interface for Hierarchic Structures
The previous section surveyed interface functions which process single score objects.
In addition, Strasheela features a number of interface functions which process multiple
objects which are hierarchically nested in a score. Many of these functions are dened
in a highly generic way as higher-order functions (see Sec. 2.8).
5.5.2.1. Processing Lists of Score Objects
Several accessors of the interface return a list of score objects. For instance, the items
contained directly in a container can be accessed by the method getItems which returns
these items in a list. Similar methods exist to access all parameters or direct containers
of an item in a list.
Such a list can be processed by any list function dened in the Strasheela programming
language user-interface (e.g. the Oz language in case of the Strasheela prototype). Many
list functions are higher-order functions, which allow the user to process a list of score
objects in a very concise and powerful way. For instance, the function lter provides a
powerful device to select specic subsets of list elements: lter returns all elements in a
list for which a predicate a function given as an argument (often simply dened inline)
returns true.
Many higher-order list functions are available. Typical examples include map (collecting
the results of a unary function applied to any list element), count (counting the number
of elements for which a predicate returns true), nd (returning the rst list element for
99
5. The Strasheela Music Representation
which a predicate returns true), and sort (sorting a list according to a binary predicate
used to compare two list elements),
Figure 5.22 demonstrates how these functions express a complex selection process in a
concise way. In the example, the function lter processes a list containing all items which
are directly contained in myContainer. This function selects only those items for which
the test function f returns true, that is, those which are note objects and which are
marked with a specic piece of information. The result of lter is further processed by
the function map, which collects the pitches of the selected notes.
map (lter (getItems(myContainer),
f : f(x)
def
= isNote(x) hasThisInfo(x, takeMe)),
getPitch)
Figure 5.22.: A complex selection dened concisely: the expression returns the pitch
values of all objects directly contained in myContainer which are notes and marked
with the information takeMe
Because such selections occur frequently in musical CSPs, Strasheela denes a shorthand
method mapItems which combines getItems with the functions map and lter. The
example in Fig. 5.23 is equivalent to Fig. 5.22. Similar shorthand methods exist for
the combination of getItems with other list functions (e.g. countItems, ndItem, and
lterItems).
mapItems (myContainer,
getPitch,
test: f : f(x)
def
= isNote(x) hasThisInfo(x, takeMe))
Figure 5.23.: A shorthand alternative of the expression in Fig. 5.22
For better readability, mapItems expects the predicate for ltering as a named argument
(also known as keyword argument), called test (for the notation cf. App. A.2). The
argument test is optional (the test defaults to f(x)
def
= true).
5.5.2.2. Interface for Score Hierarchy
Strasheela complements the interface for processing a list of item instances (as dis-
cussed in the preceding section) by a similar interface for processing objects in the whole
hierarchically-nested score.
All Strasheela items understand the higher-order accessor method collect. Similar to ac-
cessors like getItems and mapItems (see above), collect returns a list of score objects.
However, whereas the former accessors only process the items directly contained in a
100
5.6. Score Contexts
container, collect traverses recursively through the score hierarchy and potentially col-
lects all score objects in the whole score. For example, the expression in Fig. 5.24 returns
all pitch parameter objects in myScore, regardless of nesting depth. The optional named
argument test serves the same purpose for collect as it does for mapItems: it expects
a predicate used for ltering the list of score objects. The list collect returns can be
processed by arbitrary list functions (e.g. those discussed in the previous subsection).
collect(myScore, test: isPitch)
Figure 5.24.: Collecting all pitch parameters in the whole score
Strasheela denes a number of similar higher-order methods traversing the whole score
which mirror common list-processing functions. For example, Strasheela denes methods
traversing the whole score which are equivalents of the the list functions map and lter.
9
The example in Fig. 5.25 collects the pitch values of all notes in the whole score which
are marked by a certain information specication (the very similar example in Fig. 5.23
using mapItems instead of map returned only the pitches of the directly contained
notes).
map (myScore,
getPitch,
test: f : f(x)
def
= isNote(x) hasThisInfo(x, takeMe))
Figure 5.25.: Collecting the pitch values of all notes in myScore (regardless of nesting
depth) which are marked with the information label takeMe
5.6. Score Contexts
Section 4.1.1 demonstrated why a generic music constraint system should allow for the
application of constraints on arbitrary score contexts. In order to apply a constraint, a
score context must be somehow accessible. Also, it is important to express score contexts
in a way which is easy to remember for the user, in order to make complex CSP denitions
comprehensible (Sec. 4.3.3.2).
This section rst summarises general principles for expressing score information in Stra-
sheela (Sec. 5.6.1). Section 5.6.2 shows how the Strasheela user denes new score contexts
by utilising this information. Section 5.6.3 concludes with a discussion of the generality
of this formalism and also how this generality is limited (Sec. 6.3 resolves this limitation).
9
Methods such as map and lter are polymorphic functions. The Strasheela methods traversing a score
expect a Strasheela item instance as argument, whereas the list functions expect a list.
101
5. The Strasheela Music Representation
5.6.1. Principles for Expressing Score Information
Strasheela supports several basic principles to store score information. Some informa-
tion is stored explicitly, while other information can be derived from explicitly stored
information.
Explicitly Stored Information: The following principles are supported to explicitly express
information about the score.
The data type (class) of a score object is represented explicitly. Types are
hierarchically organised (class hierarchy). For example, the duration of a note
is a time interval, which is a parameter, which is a score object.
Attributes of a score object are stored explicitly. For example, the value of a
note pitch is stored explicitly.
The value of a score object attribute can be a constrained variable. Such an
attribute value is explicitly represented but can be unknown and can be con-
strained. For example, the pitch value of a note or the duration of a temporal
container can be unknown but constrained.
The nesting of score objects in arbitrary score topologies is stored explic-
itly. For instance, the representation explicitly represents which items are
contained in a container. Moreover, the hierarchic nesting is expressed by bi-
directional links between involved score objects (Sec. 5.4.2.1 and Sec. 5.4.3.1).
The order of score objects on the same hierarchical level is explicitly rep-
resented. For example, a simultaneous container representing an event list
preserves an order of its contained items which is independent of their tem-
poral order (i.e. each item has its own oset time representing the dierence
between its start time and the start time of the container).
Derived Information: Derived information can be deduced by utilising any explicit or
otherwise derived information about score objects. Deducing information which
involves multiple score objects often makes use of the bi-directional links between
score objects. For instance, the interval between the explicitly represented pitches
of two neighbouring notes in a sequential container can be deduced even if only
one of the two notes is given.
These fundamental principles of explicitly stated and derived information are completed
by principles which make the representation more convenient.
Abstract Data Types (ADT): Score objects are dened as abstract data types (classes)
which encapsulate their attributes and provide a rich user interface (Sec. 5.5).
Strasheela denes a number of basic score data types which cover many musi-
cal CSPs and provides a standard interface for these predened types (object
creation, type checker, attribute accessors etc., Sec. 5.5.1).
102
5.6. Score Contexts
In addition, the interface features a number of generic higher-order functions
(methods) to process information in the score (Sec. 5.5.2). For example, this
interface allows the user to concisely dene arbitrary score context accessors
by collecting all score objects for which a test function returns true (cf. Fig.
5.24).
Derived information can be encapsulated into new interface functions. For
instance, the interface provides functions such as getSuccessor which returns
the successor of an item instance in a container.
Incremental ADT Denition (Class Inheritance): Strasheela supports an incremental de-
nition of ADTs. The basic Strasheela types are designed with various extensions
in mind. New types (sub-classes) are dened by inheriting from existing types
(super-classes). Only the required additional attributes and interface functions (or
functions to be overwritten) must be dened by the user. For example, the proto-
type denes classes like chord and scale (to support harmonic CSPs) which inherit
from the class temporalElement.
Textual Representation: Strasheela features a textual representation form which allows
the user to dene various score topologies and to concisely express fully determined
scores, or scores which leaves all variables undetermined, and anything in between
(Sec. 5.4.3.2). This textual representation is a composite datum of the interface
language and can be created by other programs. The textual representation format
and its ADT representation equivalent can be transformed into each other.
The Strasheela user ultimately controls the information that is represented in a score.
The user combines instances of predened score data types (class instances) to build
a music representation. For instance, textual representation makes it easy to assemble
the score from dierent score data types (and to implicitly dene the score topology).
Optionally, the user extends the interface of these types or even denes new types. Class
inheritance greatly simplies the denition of new types.
The fundamental principles used to represent explicit information are not strictly or-
thogonal. Some of these principles are interdependent. For example, dierent data types
often dier also in their set of attributes, the nesting of score objects is essentially stated
in terms of attributes, and the nesting of objects must correspond with the type of these
objects. Morever, the same information can be stored in alternative forms. For example,
the information related to hierarchic typing and the hierarchic nesting could alterna-
tively be expressed by attributes. Hierarchic typing could be expressed by an attribute
which stores a list of type and super-type labels. Hierarchic nesting could be expressed
by an attribute which stores a label denoting a score context such as in voice
1
.
Nevertheless, substituting some representation form by another form can result in a less
concise representation. Hierarchic nesting with bidirectional links allows the container
to access all information within its contained objects and vice versa. For example, a
103
5. The Strasheela Music Representation
contained object can access its own position in the container. Representing all this infor-
mation by additional attributes only (e.g. by additional attributes for every contained
object where these attributes store all the information about their virtual container)
would result in a cumbersome representation. Moreover, representing hierarchic typing
only by an attribute would not provide for the convenience of incremental type denitions
by inheritance
5.6.2. User-Dened Score Contexts
Section 2.4 dened a score context as a set of score objects which are related. The
relation between the elements denes the context. Typical relationship examples are
simultaneous notes or neighbouring chords in a chord progression.
5.6.2.1. Accessing Score Contexts
The denition of a score context utilises the explicitly stored and derived information
provided by Strasheelas music representation which was discussed above. For example,
the music representation can explicitly express which notes belong to a certain voice
by means of hierarchic nesting where the voice is represented by a sequential container
myVoice and all voice notes are items contained in myVoice. The context all notes in
the voice myVoice is then accessed by accessing the items contained in myVoice (see
Fig. 5.26).
getItems(myVoice)
Figure 5.26.: Accessing the context all items in the voice myVoice
As this example shows, a context is accessed by an expression in Strasheelas program-
ming language user interface (i.e. mathematical notation in case of Fig. 5.26). More
complex expressions allow the user to access more complex score contexts. For example,
the voice myVoice may contain both note instances and pause instances. The example
in Fig. 5.27 accesses the context all items in the voice myVoice which are pauses by
ltering the items contained in myVoice. This context makes use of information derived
from information expressed by hierarchic nesting and expressed by typing.
lter( getItems(myVoice),
isPause)
Figure 5.27.: Accessing the context all items in the voice myVoice which are pauses
In both examples a function call returns the full context (i.e. a list of all the items or
pauses in myVoice). There are other cases in which the return value of a function is only
104
5.6. Score Contexts
a subset of all score objects which belong to the full context: the missing part is often
the argument to the function. For example, the context two neighbouring items in a
temporal container can be created by accessing the successor item of a given item (see
Fig. 5.28). This context is crucial for the application of numerous melodic rules (e.g. to
constrain the pitch interval of two neighbouring notes).
myItem, getSuccessorInTemporalContainer(myItem)
Figure 5.28.: The context two neighbouring items in a temporal container
The successor of an item is not represented explicitly (i.e. there exists no explicit successor
attribute), but this information can be derived from the information provided by the
hierarchic nesting due to the bidirectional links between score objects (the function
getSuccessorInTemporalContainer is dened in Fig. 5.29).
5.6.2.2. Abstracting Score Contexts
The expression which returns a score context (or complements a context) can be en-
capsulated in a function. Abstracting a score context denition this way has the same
advantages as function denitions have for programming in general (e.g. the program is
structured into modular subprograms with names which are easy to remember).
It is important for Strasheela as a generic music constraint system, that context accessor
functions can be dened by the user. That way, the user can extend the set of predened
contexts. The following paragraphs introduce two typical techniques used to dene such
functions. The rst technique relies on information expressed by hierarchic nesting to
access context elements directly. The second technique denes a test which checks for
context membership and lters the context elements out of score objects.
Exploiting Hierarchical Information
Strasheela denes several context accessors which evaluate the information provided by
the hierarchic nesting of the score. These functions often make use of the bidirectional
links between score objects. Because of these bidirectional links, the whole score can be
explored as it can be seen from any object.
The function getSuccessorInTemporalContainer expects an item instance as argument
and returns the successor of this item in its temporal container. The function exploits
the information provided by the hierarchic nesting of the score. Figure 5.29 shows how
this function can be dened.
10
10
Like all examples in this text, the denition of getSuccessorInTemporalContainer in Fig. 5.29 aims
for clarity and not eciency. The denition could be optimised such that the list of items in the
container is not traversed three times (by position, length and nth). Possible further improvements
include replacing all lists in the Strasheela music representation by data structures with constant
time access (e.g. arrays or records) [van Roy and Haridi, 2004] and memoizing [Norvig, 1992] the
position of an item in a container.
105
5. The Strasheela Music Representation
getSuccessorInTemporalContainer(x)
def
= let c
def
= nd( getContainers(x),
isTemporalContainer)
items
def
= getItems(c)
nextPos
def
= position(x, items) + 1
in if nextPos > length(items)
then nil
else nth(items, nextPos)
Figure 5.29.: The context accessor getSuccessorInTemporalContainer returns the succes-
sor of an item x in its temporal container
The function rst accesses the temporal container of the item x. Because each item
can potentially be contained in multiple containers (in case the score topology is a
graph), a ltering of all containers of x is performed. It is sucient to nd only the rst
temporal container c (hence the function nd), as an item cannot be contained in multiple
temporal containers (see Sec. 5.4.4).
11
The list of items contained in this container c is
accessed and bound to items. The position of the successor of x in items is computed
and bound to nextPos.
12
After a check whether the list items is long enough to hold a
successor item at position nextPos, this successor item is nally accessed and returned.
13
getSuccessorInTemporalContainer returns nil , in case no successor item exists.
Note that the score object returned by getSuccessorInTemporalContainer is the posi-
tional successor of the given object in its temporal container. However, it is not nec-
essarily the temporal successor. For example, the items contained in a simultaneous
container are not necessarily ordered temporally.
Several score contexts can be expressed by hierarchically nesting score objects in a score.
Examples include all part-of relations (e.g. the notes in a motif, voice, chord etc.), has-a
relations (accessing item attributes and parameters) and positional relations (e.g. neigh-
bouring items). However, other contexts require dierent approaches: the next paragraph
explains another technique.
Filtering Context Members
An alternative technique closely resembles the common set-builder notation. For exam-
ple, the context all temporal items in the score which are simultaneous to some item
item can be notated by set-builder notation as in Fig. 5.30. This context of simultaneous
items is important for many rules (e.g. to apply harmonic rules).
In the following, this set-builder notation example is translated into a Strasheela pro-
gram. The main idea here is to dene a function which returns true for any context
11
The function isTemporalContainer returns true for a temporal container.
12
The function position returns the position of an element in a list.
13
The function nth returns the list element at the specied position.
106
5.6. Score Contexts
x : isTemporalItem(x)
isSimultaneous(x, item)
x ,= item
Figure 5.30.: The context all temporal items in the score which are simultaneous to
some item item in set-builder notation
element and to use this function to lter the context members out of score objects. This
ltering is potentially performed on the set of all objects in the score.
The function getSimultaneousItems (Fig. 5.31) encapsulates the context all temporal
items in the score which are simultaneous to some item item in a function (i.e. this
function is equivalent to the set-builder notation example of Fig. 5.30).
getSimultaneousItems(item)
def
= let /* access temporal top-level container of item */
top
def
= getTemporalTopLevel(item)
/* auxiliary test function */
f (x)
def
= x ,= item
isTemporalItem(x)
isSimultaneous(item, x)
in collect(top, test: f )
Figure 5.31.: The context all temporal items in the score which are simultaneous to
some item item in Strasheela
The denition of getSimultaneousItems accesses rst the top-level temporal container
of the argument item using the function getTemporalTopLevel .
14
Then, getSimultane-
ousItems denes an auxiliary function f which expresses the conjunction of three tests
much like the expression in Fig. 5.30. The function tests whether its argument x is
unequal to item, whether x is a temporal item, and whether x and item are simultaneous
in time. Finally, getSimultaneousItems collects all score objects recursively contained in
top which full the test function f.
The example requires three boolean functions. The type-checking (isTemporalItem) and
the negation of object equality is provided by the basic Strasheela interface. The function
isSimultaneous returns a boolean reecting whether two temporal items are simultaneous
in time or not (see Fig. 5.32). Although isSimultaneous is predened by Strasheela, it is
dened here to demonstrate how such functions can be dened by the user. The function
tests whether the time frames occupied by two temporal items which are denoted by
the start and end time of each item are overlapping.
14
The function getTemporalTopLevel returns the top-level of the temporal hierarchy which contains a
given item. It is dened in Fig. B.1
107
5. The Strasheela Music Representation
isSimultaneous(item
1
, item
2
)
def
= (getStartTime(item
1
) < getEndTime(item
2
))
(getStartTime(item
2
) < getEndTime(item
1
))
Figure 5.32.: The function isSimultaneous tests whether two items are simultaneous
5.6.3. The Generality of the Strasheela Score Context Formalism
The Strasheela music representation is highly generic. The Strasheela user ultimately
controls what explicit information is stored in the score. The user can then freely deduce
derived information from the explicitly stored information (Sec. 5.6.1). In particular, any
score information can be obtained from any object in the score due to the bidirectional
links between score objects.
The generality of the music representation results in a highly generic score context for-
malism. When accessing a score context, the user can freely make use of any information
available in the score. For instance, Sec. 5.6.2.2 introduced two techniques to access
score contexts exemplied by the functions getSuccessorInTemporalContainer (Fig.
5.29) and getSimultaneousItems (Fig. 5.31). Both techniques derive information from
explicitly stored information to access specic score contexts (and make use of the bidi-
rectional links between score objects to access this information).
For accessing a score context, it is essential that the music representation provides enough
information to unambiguously specify the set of score objects which belongs to this
context (to isolate this context). Any score context is accessible in Strasheela as long
as the music representation provides the required information for isolating this context.
There are two cases where the music representation does not provide the required infor-
mation to isolate a score context. In the rst case, the music representation misses some
explicitly stored information. For example, polyphonic music can be represented in a
at event list (e.g. all events are directly contained in a simultaneous container and the
start time of events equals their oset time). However, in such a music representation
the information which note belongs to which voice is missing and the score context all
notes of the rst voice cannot be accessed.
In the case where the music representation misses some explicitly stored information, the
user can restructure or extend the representation. For instance, the user can restructure
or extend an event-list representation of polyphonic music in several ways in order to
make the score context all notes of the rst voice accessible. The user can either add an
information tag to each event (e.g. using the info attribute) to denote the voice it belongs
to, re-organise the hierarchic nesting of score objects to express voice membership, or
even dene a new class which extends the event class by a voice-membership attribute.
In the second case, the music representation explicitly stores the required information,
but this information is not determined in the CSP denition. If so, the necessary infor-
mation will only be determined during the search process. For example, simultaneous
108
5.6. Score Contexts
objects are not accessible in the CSP denition when the rhythmical structure of a poly-
phonic score is undetermined and is constrained by rules as well. In such a case, it is
impossible in the CSP denition to directly access the score context in question. As a
solution to this problem, Sec. 6.3 discusses techniques that apply compositional rules to
score contexts which are inaccessible due to undetermined information in the problem
denition.
109
5. The Strasheela Music Representation
110
6. The Strasheela Rule Formalism
The preceding Chap. 5 discussed how a score is expressed in the Strasheela music rep-
resentation. The present chapter explains how a musical constraint satisfaction problem
is stated in Strasheela: this chapter introduces Strasheelas concept of constraints and
rules and details how constraints and rules are applied to a score.
This introduction only provides the declarative semantics of a musical CSP in Strasheela
in order to make this explanation more easy to comprehend. The declarative semantics is
based on a programming model which subsumes two programming paradigms well-known
in the computer music community: constraint programming (Sec. 3.1) and functional
programming, including higher-order functions (Sec. 2.8). Examples are notated in a
way that has already been used throughout this thesis. This notation uses rst-order
logic notation complemented by the where-notation to notate functions as rst-class
values (App. A).
The operational semantics of Strasheela, however, are based on a dierent program-
ming model. Whereas this section explains the declarative semantics with the notion of
functions, Strasheelas operational semantics is based on concurrent procedures (which
are also rst-class values). Nevertheless, the notation of concurrent procedures would
have required a notation which has been used little so far in computer music research.
1
Instead, the present chapter uses a common mathematical notation, although this no-
tation covers only the declarative semantics. Strasheelas operational semantics will be
explained in Chap. 7.
Chapter Overview
Section 6.1 denes what a constraint is and what a composition rule is in Strasheela.
Strasheelas rule formalism allows for user-dened rule application mechanisms, which is
demonstrated in Sec. 6.2. The last section discusses an issue which arises in many complex
musical CSPs. These problems constrain score contexts which are inaccessible in the
problem denition because the partial music representation misses certain information.
Section 6.3 discusses three dierent approaches to address this issue.
1
Concurrent procedures can be notated, for example, with the -calculus notation [Parrow, 2001] or
the Oz notation [van Roy and Haridi, 2004], and these notations could have been used in the present
chapter. The literature discussing PiCO a calculus which subsumes concurrent objects and con-
straints to provide a foundation for composition systems even proposes its own syntax and expresses
musical examples in this notation [Rueda et al., 2001].
111
6. The Strasheela Rule Formalism
6.1. Constraints and Rules are First-Class Functions
In the declarative semantics of Strasheela, a constraint is a function which returns a
boolean value. All arguments of a constraint are constrained variables (i.e. unknowns as
the variables in rst-order logic for which a domain of possible values is dened, see Sec.
3.1). Also the boolean value returned by a constraint is a constrained variable (i.e. its
value can be unknown).
The variable values given to the constraint as arguments satisfy the constraints re-
striction if and only if the returned value is true. By convention, the truth values are
represented by integers: 1 represents true and 0 represents false (the reason for this
convention will become clear later, see Sec. 7.2.3.4).
Formally, a constraint c(X
1
, X
2
, . . . , X
n
) maps the set of its variable domains (here
D
1
, D
2
, . . . , D
n
) to a truth value as in
c : D
1
D
2
D
n
0, 1
For convenience, a constraint can be a composite expression. The operations in such
a composite expression (notated with the common mathematical symbols) may return
other values instead of truth values. However, the full expression returns a truth value.
For example, the interval between two pitches can be constrained by the composite
expression shown below, where the operations X Y and [X[ return numeric values
instead of truth values.
(Interval = [Pitch
1
Pitch
2
[) 0, 1
A compositional rule is a concept very similar to a constraint. Like a constraint, a rule
is a function which returns a boolean variable. However, a rule is a slightly more general
concept when compared with a constraint. Whereas all arguments of a constraint are
variables, the arguments of a compositional rule can be any score data. This implies that
any constraint is also a compositional rule.
A rule applies constraints to some variables in a CSP. These variables are either directly
given to the rule as arguments (in that case the rule is actually a constraint) or are
accessible from its arguments.
The example in Fig. 6.1 denes two variants of a rule which encapsulates the melodic re-
striction that the interval between two pitches must not to exceed a fth (cf. Fig. 4.1). The
example denes two variants for this rule. The constraint (and rule) limitInterval
pitches
is
a function which expects two pitches which are variables in a CSP. The rule limitInterval
note
,
on the other hand, expects a single note object: the rule accesses the successor note of a
given note, and the pitches of both notes, and then applies the constraint limitInterval
pitches
(i.e. the variant constraining two variables) to these pitches.
2
2
As explained in App. A.1, variables in the CSP start with an upper-case letter (e.g. Pitch
1
) and other
112
6.2. Programmable Rule Application
limitInterval
pitches
(Pitch
1
, Pitch
2
)
def
= 7 [Pitch
1
Pitch
2
[
limitInterval
note
(note)
def
= limitInterval
pitches
(getPitch(note),
getPitch(getSuccessor(note)))
Figure 6.1.: Two rule variants which constrain the interval between two pitches not to
exceed a fth
Rules can be composed from other arbitrary functions of the system including other rules.
For example, in Fig. 6.1 the rule limitInterval
note
calls the interface function getPitch
and the rule limitInterval
pitches
.
In the Strasheela model, rules are applied to a highly expressive music representation.
This representation supports access to score contexts in a generic way (Sec. 5.6). For
example, Fig. 6.1 shows how the successor of a note or the notes pitch can be accessed
within a rule denition. The full Strasheela music representation interface is available in
a rule denition and thus complex score contexts can be accessed and constrained in a
convenient way.
In Strasheela, functions are rst-class values (Sec. 2.8, [Abelson et al., 1985]) and so
constraints and rules are also rst-class values. First-class functions allow for higher-order
functions which expect other functions as arguments. This idea forms the foundation for
Strasheelas programmable rule application.
6.2. Programmable Rule Application
The concept of a function clearly separates its denition and application. Accordingly,
the Strasheela rule formalism fully decouples the denition and the application of a com-
positional rule (or constraint). The user denes a rule and then explicitly applies this rule
to the score. This approach is in clear contrast to previous systems (e.g. PWConstraints
or Situation, see , Sec. 3.3.1 and Sec. 3.3.2) in which the denition and the application
of a rule was amalgamated in a single operation.
In Strasheela, rule application mechanisms are user programmable due to Strasheelas
functional programming foundation. A rule application mechanism is a higher-order
function which expects a rule (i.e. a function) as argument. A rule application mechanism
abstracts an arbitrary control structure which traverses a score to apply a rule to selected
score objects and variables. This approach combines a user-controlled rule scope (i.e. the
set of variable sets the rule constrains, Sec. 3.3.1.1) with a concise notation to express
even complex rule scopes.
identiers start with a lower-case letter (e.g. limitInterval
p
). However, the variables in the CSP are
not always literally represented: often the variables returned by accessor functions (e.g. getPitch) are
constrained directly as in limitInterval
pitches
(getPitch(note
1
), getPitch(note
2
)).
113
6. The Strasheela Rule Formalism
In the remainder of this subsection, the programmable rule application mechanism of
Strasheela is illustrated by a number of examples. These examples are shown with the in-
tention of demonstrating the expressiveness and generality resulting from this approach.
Most examples only show the application of a rule application mechanism to demon-
strate its eect. Nevertheless, all these mechanisms can be dened by the user, which is
demonstrated in Sec. 6.2.4. To be more easily comparable, the rst examples are highly
similar and apply either limitInterval
pitches
or limitInterval
note
(Fig. 6.1) to pitches of
neighbouring notes in a voice.
6.2.1. Direct Rule Application to a Single Object
In the most simple case, a rule is applied directly to a single score object. In Fig. 6.2,
the rule limitInterval
note
(see Fig. 6.1) is directly applied to the rst note in myVoice.
In eect, the interval between this note and its successor is constrained not to exceed a
fth.
let myNote
def
= rst(getNotes(myVoice))
in limitInterval
note
(myNote)
Figure 6.2.: Direct application of a rule to a single note
The whole example returns the boolean variable which is returned by the rule (see Sec.
6.1). This text adopts the convention that a boolean variable returned by a full example
is implicitly constrained to be true.
In most cases, a rule should be more general and not hold for only a single pair of pitches.
Any control structure can be used to apply a rule to multiple sets of score objects. For
example, the rule limitInterval
note
can be applied directly to all notes in a voice which
feature a successor note by iterating through all these notes in a loop.
6.2.2. Applying a Rule to Every Element in a List
A programming technique very similar to iteration is mapping. The higher-order function
map applies a function to every element in a list, collects the results, and returns a list
with all results. In Fig. 6.3, map applies the unary rule limitInterval
note
to all but the
last note in myVoice and so constrains the interval between all pitch pairs in this voice.
Within the rule denition, the successor note is accessed and the interval is constrained
(see Fig. 6.1). The last note of myVoice is omitted (using the function butLast) because
it has no successor and thus the application of limitInterval
note
will fail.
The function map returns a list of boolean variables and the operator

returns the
conjunction of all these values. Again, the boolean returned by the example is implic-
itly constrained to be true, thus all limitInterval
note
applications in the example are
constrained to return true.
114
6.2. Programmable Rule Application
let myNotes
def
= butLast(getNotes(myVoice))
in

map(myNotes, limitInterval
note
)
Figure 6.3.: Constrain the interval between all pitch pairs of myVoice (using the unary
function limitInterval
note
)
6.2.3. Applying a Rule to Neighbours in a List
The previous example (Fig. 6.3) may be somewhat dicult to comprehend: the score
context constrained by the rule is partly given as an argument to the rule and partly
accessed within the rule denition. Because the succeeding note is accessed within the
rule, the list of objects to which the rule is applied must be explicitly reduced with the
function butLast.
The meaning of this example can be expressed more clearly using the higher-order func-
tion map2Neighbours. This function applies a binary function to each pair of neighbour-
ing elements in a list. In Fig. 6.4, map2Neighbours applies the rule limitInterval
pitches
(Fig. 6.1) to each neighbouring pair of note pitches in myVoice.
let myPitches
def
= map(getNotes(myVoice), getPitch)
in

map2Neighbours(myPitches, limitInterval
pitches
)
Figure 6.4.: Constrain the interval between all pitch pairs of myVoice (using the binary
function limitInterval
pitches
)
6.2.4. User-Dened Rule Application Mechanisms
The examples shown so far (Fig. 6.3 and Fig. 6.4) demonstrate how higher-order functions
are used as rule application mechanisms, but these mechanisms can also be dened by the
user. This fact marks an important distinction between Strasheela and existing systems.
The higher-order function map (applied in Fig. 6.3) is a standard means in functional
programming and its denition can be studied in many textbooks on functional program-
ming languages.
3
Figure 6.5 denes the higher-order function map2Neighbours which was
used in the example above (Fig. 6.4). The denition is very brief and consists of only a
call to the function zip which is a relative of the function map.
The function zip itself expects two lists xs and ys and a binary function fn and collects
the results of all calls fn(x
i
, y
i
), where x
i
and y
i
are the i-th element of xs and ys.
For instance zip([1, 6, 3], [4, 5, 6], max) returns [4, 6, 6]. The function zip is dened in the
appendix in Fig. B.2.
3
Abelson et al. [1985] dene map in the Lisp dialect Scheme and [van Roy and Haridi, 2004] dene it
in Oz.
115
6. The Strasheela Rule Formalism
map2Neighbours(xs, fn)
def
= zip(butLast(xs), tail (xs), fn)
Figure 6.5.: Rule application functions can be dened by the user: the denition of the
higher-order function map2Neighbours
The function map2Neighbours calls zip with three arguments, namely two sublists of its
list argument xs (one containing all but the last element and the other all but the rst
element of xs) plus the binary function argument fn.
6.2.5. Index-Based Rule Application
Rule application mechanisms provide a concise and convenient means to express the
scope of a rule. Dierent systems introduce dierent rule application mechanisms. For
example, Situation (Sec. 3.3.2) oers a set of index-based rule application mechanisms
whereas PWConstraints provides a pattern-matching based mechanism (Sec. 3.3.1).
In contrast, this thesis proposes a rule formalism where rules are rst-class functions and
rule application mechanisms are higher-order functions. In this text it is argued that a
rule formalism based on rst-class functions is more generic than the rule formalisms of
existing systems. To substantiate this claim, this section reproduces the rule application
mechanisms of Situation and Sec. 6.2.6 the application mechanism of PWConstraints as
higher-order functions.
Situation oers a set of index-based rule application mechanisms. Situation introduces
a proprietary language (which cannot be extended by the user) to select and control a
mechanism for a rule. The present section describes three fundamental index-based rule
application mechanisms. These mechanisms are reproduced in Strasheela.
The rst Situation mechanism applies a unary rule to every object in an object sequence
whose position (or index) in this sequence is specied. The Strasheela function mapIndex
reproduces this behaviour. This function expects three arguments: a list xs, an indices
declaration decl , and a unary function f. The list xs consists of arbitrary values (e.g.
score objects). The declaration decl is notated as a list containing a mix of single integers
and integer ranges. An integer range is notated here with the # operator as follows.
startIndex#endIndex denotes startIndex, . . . , endIndex
In the example in Fig. 6.6, mapIndex applies the rule limitInterval
note
to the rst three
and the fth note of myVoice. The index specication is the list [1#3, 5] which denotes
the indices [1, 2, 3, 5]. The function mapIndex is dened in Fig. B.3.
The second rule application mechanism applies an n-ary rule to subsequences of objects
in an object sequence. This mechanism can be reproduced by generalising the func-
tion mapIndex to mapIndices. The function mapIndices expects similar arguments to
116
6.2. Programmable Rule Application
let indices
def
= [1#3, 5]
in

mapIndex(getNotes(myVoice), indices, limitInterval
note
)
Figure 6.6.: Reproduced Situation rule application mechanism: apply the rule
limitInterval
note
to all objects which are identied by their numeric index
mapIndex (i.e. a list of arbitrary values xs, an indices specication spec, and a function
fn), but the indices specication uses a dierent format. Whereas mapIndex expects a
at list of specications, mapIndices expects a list of specication lists. Each specication
sublist for mapIndices has the format of the specication expected by mapIndex. For
mapIndices, the function fn expects a list and is applied to the sublist of elements of xs
which match a single sublist specication in spec. For example, the following expression
mapIndices([a, b, c, d, e, f], [[1#3], [2, 5]], fn)
results in these two calls of fn
[fn([a, b, c]), fn([b, e])]
The function mapIndices can be dened analogously to mapIndex (cf. Fig. B.3).
Finally, the third rule application mechanism applies an n-ary rule to every sublist of
n neighbours in a list. This Situation mechanism can be reproduced by generalising the
function map2Neighbours (Fig. 6.5). Whereas map2Neighbours applies a binary rule to
every pair of two neighbouring list elements, the generalised function mapNeighbours
applies an n-ary rule to every sublist of n neighbours in a list (the value n is obtained
by accessing the arity of the rule passed to mapNeighbours). The actual denition of
mapNeighbours is not shown for brevity.
The Situation rule application mechanism reproductions presented here are even more
general then the original. The original mechanisms are restricted to the sequential score
topology of Situation. Instead, the mechanisms of this section can be used on any score
context which can be represented as a sequence. For example, Fig. 6.6 applies a rule to
the notes of a specic voice (which might be only one of several voices from a polyphonic
score). The score context notes of voice myVoice has no equivalent in Situation.
6.2.6. Rule Application with a Pattern Matching Language
This section describes and reproduces the rule application mechanism of PWConstraints
(the reproduction was motivated in Sec. 6.2.5). In PWConstraints, a rule is applied to
all object sets which match the pattern matching expression of the rule. PWConstraints
pattern matching language was introduced in Sec. 3.3.1.1.
117
6. The Strasheela Rule Formalism
The present section denes a very similar pattern matching language in order to repro-
duce this rule application mechanism. This language also denes three symbols. The two
place-holder symbols ? and retain their meaning: the symbol ? exactly matches one
sequence element, and matches zero or more elements.
The pattern matching language introduced here features no pattern-matching variables.
Instead, this language provides the symbol x: every occurrence of a pattern-matching
variable in PWConstraints is substituted by the unvarying symbol x here. The following
example shows an expression which matches any but the rst pair of two successive
elements (cf. the PWConstraints example with the same meaning in Fig. 3.7).
[?, , x, x]
The rule application mechanism of PWConstraints is reproduced by the higher-order
function mapPM. This function expects three arguments: a list xs (of arbitrary values,
e.g., score objects), a pattern matching expression pattern (using the syntax introduced
above), and a unary function f which expects a list. The function mapPM applies f to
every sublist of xs which matches the pattern. For example, the following expression
let pattern
def
= [x, ?, , x, ]
in mapPM([a, b, c, d], pattern, f)
results in two function calls
[f([a, c]), f([a, d])]
Because the symbol matches zero or more elements, the pattern matches multiple
sublists of xs in case an instance of this symbol occurs in the pattern (multiple occurrences
of are possible). Also, the order of elements which match some instance of the symbol
x corresponds with their order in xs.
Figure 6.7 uses mapPM to apply a melodic rule to the pitches of the notes in myVoice.
The example is equivalent to the PMC example presented in Fig. 3.8. The function
mapPM is dened in Fig. B.4.
The function mapPM can be used on any sequential data, like the index-based rule
application mechanisms introduced in Sec. 6.2.5. For example, mapPM can apply rules
to any score context of a highly nested music representation which is represented by a
list.
6.2.7. Rule Application to Selected Objects in a Score Hierarchy
All rule application mechanisms introduced so far apply a rule to specic elements (or
element sets) in a sequence. These mechanisms were implemented by higher-order func-
tions which traverse a sequence in order to apply a rule. Higher-order functions can dene
118
6.2. Programmable Rule Application
let myPattern
def
= [, x, x]
in

mapPM(mapItems(myVoice, getPitch)
myPattern,
f : f([Pitch
predecessor
, Pitch
successor
])
def
=
let Interval
in Interval = [Pitch
predecessor
Pitch
successor
[
Interval 0, . . . , 7)
Figure 6.7.: Reproduced PWConstraints rule application mechanism: constrain the in-
terval between two consecutive pitches of notes in myVoice not to exceed a fth (cf. Fig.
3.8)
control structures which traverse arbitrary data structures. The present section proposes
a rule application mechanism which applies a rule to all score objects in a hierarchical
music representation which meet a certain condition.
Section 5.5.2.2 introduced the higher-order interface for a hierarchically nested score.
That section also introduced a generalised function map which traverses a score hierarchy
(instead of a at list, as the basic version of map). This variant (method) of map expects
an optional argument test to select the score objects for processing.
Functions like this map variant can be used to traverse a score hierarchy and apply a
rule to specic elements in the score. The example in Fig. 6.8 constrains all note objects
in myScore to diatonic pitches (i.e. pitches from a specic scale such as the C-major
scale). In this example, each constrained note can be arbitrarily nested within myScore.

map(myScore, hasDiatonicPitch, test: isNote)

Figure 6.8.: Every note in myScore is constrained to a diatonic pitch
The rule hasDiatonicPitch is dened in Fig. 6.9. The principal part of this function
is expressed by the function isDiatonicPitch, which constrains the pitch class of its
argument MyPitch so that it ts into a diatonic scale. For simplicity, isDiatonicPitch
only constrains whether MyPitch is a member of the pitch classes of the C-major scale.
A more general denition would also allow for transpositions, for example, for the pitches
of the E-major scale. Finally, the function hasDiatonicPitch only applies the constraint
isDiatonicPitch to the pitch of a given note.
Such a mapping-based approach allows the user to apply a rule to arbitrary score ob-
jects. The rst example applied a rule to events (i.e. notes). The next example uses this
template to apply a rule to containers.
Figure 6.10 applies the rule hasUniqueMelodicPeak to every voice in myScore, namely to
every sequential container which is marked with the info-tag voice. The rule hasUnique-
119
6. The Strasheela Rule Formalism
hasDiatonicPitch(myNote)
def
= isDiatonicPitch(getPitch(myNote))
let /* Set of the C-major scale pitch classes */
ScalePCs
def
= 0, 2, 4, 5, 7, 9, 11
in isDiatonicPitch(MyPitch)
def
= (MyPitch mod 12) ScalePCs
Figure 6.9.: The function hasDiatonicPitch constrains whether the pitch of a note ts
into the C-major scale and the function isDiatonicPitch performs this constraint for a
pitch variable
MelodicPeak (Fig. 6.11) constrains that the melodic peak (i.e. the maximum pitch)
occurs exactly once in the list of all note pitches in myContainer.

map(myScore,
hasUniqueMelodicPeak,
test: f : f(x)
def
= isSequential (x) hasThisInfo(x, voice))
Figure 6.10.: The rule hasUniqueMelodicPeak is applied to every sequential container
marked as voice
hasUniqueMelodicPeak(myContainer)
def
=
let pitches
def
= map(myContainer, getPitch, test: isNote)
MaxPitch
in MaxPitch = max(pitches)
/* The constraint once demands that MaxPitch occurs only once in pitches */
once(MaxPitch, pitches)
Figure 6.11.: In the list of all note pitches in myContainer, the maximum pitch occurs
exactly once
6.2.8. Implicit Rule Application
In most cases, a rule is explicitly applied by the user in the CSP denition. However, rules
can also be applied implicitly when the music representation for the CSP is created. For
example, the temporal constraints between temporal items (such as events and temporal
containers, Sec. 5.4.4.1) are applied implicitly (i.e. by the initialisation method of these
classes). Used carefully, implicit constraint applications make the denition of CSPs
more convenient for the user. For instance, by using temporal items the user can rely on
well-formed temporal relationships between these objects.
120
6.2. Programmable Rule Application
6.2.9. Rule Application to Arbitrary Score Contexts
This subsection demonstrates how rules are applied to arbitrary score contexts. However,
there is one requirement to apply the technique shown in this section (and the techniques
shown in the previous sections in this chapter): it must be possible to access a score
context in order to constrain this context. In other words, the music representation must
provide enough information to isolate this context. This restriction will be addressed in
Sec. 6.3.
Figure 6.12 applies a harmonic rule to a polyphonic score. The rule constrains simultane-
ous notes to be consonant. The context of simultaneous notes is not explicitly represented
in the score. Items which are simultaneous to some given item are easily accessed pro-
cedurally as was shown by the denition of getSimultaneousItems (Fig. 5.31) as long
as the score provides enough information to isolate this context (e.g. the rhythmical
structure of the music is fully determined). Whereas getSimultaneousItems returns ar-
bitrary items (e.g. including containers), simultaneous notes are accessed similarly (e.g.
the output of getSimultaneousItems is ltered, cf. Fig. 2.15).
let bassNotes
def
= getNotes(getBass(myScore))
in

map(bassNotes,
harmonicRule : harmonicRule(bassNote)
def
=
let simNotes
def
= getSimultaneousNotes(bassNote)
in

mapPairwise(simNotes,
g : g(note
1
, note
2
)
def
=
if getVoice(note
1
) ,= getVoice(note
2
)
then isConsonant
lessStrict
(note
1
, note
2
)
else true)
map(simNotes, h : h(simNote)
def
=
isConsonant
strict
(bassNote, simNote)))
Figure 6.12.: Apply harmonic rules which constrain simultaneous notes to form consonant
intervals. The rule between an upper-voice note and a bass note is more strict than
between two upper-voice notes
In Fig. 6.12 (second line), the function map processes all bass notes in myScore. For any
bass note, the pairwise intervals between all its simultaneous notes (from higher voices)
as well as the interval between the bass note and any higher note are constrained. For
these two cases (interval between two higher notes vs. interval between a bass note
and a higher note), slightly dierent rules are applied according to common practice
[Jeppesen, 1930]. The less strict rule isConsonant
lessStrict
allows for perfect and imperfect
consonances including the fourth. This interval is prohibited in the strict rule version to
avoid
6
4
-chord inversions.
121
6. The Strasheela Rule Formalism
Both rules are again applied by a higher-order function (mapPairwise and map). The
function mapPairwise applies a binary function to all pairwise combinations of a given list
(here the list of all simultaneous notes). In function g, the constraint isConsonant
lessStrict
is only enforced for note pairs where the notes belong to dierent voices. This precondi-
tion assures that the rule application even works for non-homophonic music.
The rule isConsonant
lessStrict
(Fig. 6.13) denes that two notes are consonant, in case
the absolute distance between the pitches of these notes (expressed in key-numbers) is
either a prime, a minor third, a major third, or a fourth, etc.
isConsonant
lessStrict
(note
1
, note
2
)
def
=
let Interval
in Interval 0, 3, 4, 5, 7, 8, 9, 12, . . .
Interval = [getPitch(note
1
) getPitch(note
2
)[
Figure 6.13.: Constrains the pitches of a note pair to form a consonant interval (less
strict version which allows for prime and fourth)
The denition for isConsonant
strict
is very similar to isConsonant
lessStrict
, only the in-
tervals prime and fourth were removed from the set of allowed intervals.
isConsonant
strict
(note
1
, note
2
)
def
=
let Interval
in Interval 3, 4, 7, 8, 9, 12, . . .
Interval = [getPitch(note
1
) getPitch(note
2
)[
Figure 6.14.: Constrains the pitches of a note pair to form a consonant interval (strict
version which does not allow for prime and fourth)
This example accessed and constrained simultaneous notes, but any score context can be
accessed and constrained in this manner as long as the music representation provides
enough information to isolate this context.
6.3. Constraining Inaccessible Score Contexts
6.3.1. The Inaccessible Score Context Problem
The Strasheela rule application techniques introduced in the preceding Sec. 6.2 allow for
freely constraining the variables in any score context. For example, Sec. 6.2.3 demon-
strated how melodic rules are applied to neighbouring notes in a voice and Sec. 6.2.9
showed how harmonic rules are applied to simultaneous notes. A Strasheela CSP can
utilise any information available in the score to access score contexts.
122
6.3. Constraining Inaccessible Score Contexts
However, the only score contexts that have been constrained so far are those which are
accessible in the CSP denition. The music representation must contain sucient infor-
mation in order to allow access to the required score context. For instance, neighbouring
notes in a voice can be accessed when the position of these notes in the voice is known.
Similarly, sets of simultaneous notes can be accessed, in case the temporal structure of
the music is already determined in the problem denition.
There are cases which require constraining score contexts which are inaccessible in the
problem denition, because sucient information is missing in the denition. These
contexts only become known during the search process. An example is a melodic rule
which constrains the intervals between neighbouring local pitch maxima (i.e. melodic
peaks) in voice. Such local maxima stand out and are treated with special care in many
musical styles (e.g. only a small interval might be permitted between melodic peaks).
With the techniques introduced so far, a rule concerning these maxima can only be
applied in cases where it is already known in the problem denition which notes will
become the melodic peaks in the solution.
A second example which exhibits the restriction caused by missing information is a
harmonic rule which constrains the intervals between simultaneous note pitches. In the
case where the rhythmic structure is undetermined in the problem denition, it is also
not known which notes occur simultaneously and it is thus not known which note sets
the rule should be applied to.
This inaccessible score context problem is in some form shared by all music constraint
systems. The only exceptions are systems which perform a nave generate-and-test search:
the generate-and-test algorithm rst generates a fully determined solution candidate and
then tests whether this candidate fulls all constraints. This approach results in a vastly
inecient search process. Other systems and in particular systems which perform
constraint propagation to increase eciency such as Strasheela have to deal with
missing or partial information in the problem denition.
6.3.2. Delayed Rule Application
The inaccessible score context problem can be addressed in a number of ways. A rst
approach is to delay the constraint application. A delayed rule application waits until
enough information is available before anything is done. For instance, a delayed rule ap-
plication may wait until some context which was inaccessible in the problem denition
becomes accessible, and then it applies its constraint to it. For example, a harmonic
rule may wait until the search process decided on the rhythmical structure and the con-
text simultaneous notes can be accessed. Then the harmonic rule applies its constraints
to variables (e.g. note pitches) in this context.
This approach shows similarity to the way a composer works. A composer might also
delay certain decisions until enough information is available. For example, when creating
a rhythmically complex texture a composer might rst decide on the rhythmical structure
before writing the actual note pitches and considering the voice leading (which depend
123
6. The Strasheela Rule Formalism
on information about the rhythmical structure). Therefore, delaying decisions appears a
natural solution to the inaccessible score context problem.
However, delaying the application of rules requires the introduction of new concepts to
the underlying constraint formalism which make use of the notion of time (i.e. the time
spent searching). For example, a delayed rule application could run concurrently in its
own thread and could be applied to a dataow variable [van Roy and Haridi, 2004]. Ini-
tially, the value of this variable is undetermined, but it becomes bound to the respective
score context once enough information is available. The rule application mechanism waits
until this happens. It then applies the rule to this score context. The notation of such
CSPs would require new constructs, for example a notation for concurrent processes (as
in the -calculus [Parrow, 2001] or the Oz notation [van Roy and Haridi, 2004]) More-
over, such CSPs introduce a procedural element (the delaying of the rule application)
to the otherwise declarative problem statement. This procedural element can make the
problem harder to understand, because the CSP eectively changes during the search
process.
6.3.3. Reformulating the Problem Denition
A second approach addresses the inaccessible score context problem simply by reformu-
lating the CSP such that any undetermined context is avoided. For example, a typical
problematic case is a polyphonic CSP which constrains both the rhythmic structure and
the pitch structure and which applies rules to simultaneous notes as well as to notes
which are neighbours in a voice. In fact, polyphonic CSPs which allow users to constrain
the rhythmical structure as well as the pitch structure have proven to be so problematic
that very few existing systems support such problems.
All polyphonic Strasheela score examples shown so far applied a tree-like score topology
established by nesting simultaneous and sequential containers (Sec. 5.4.4). In this repre-
sentation, the information required to access neighbouring notes in a voice is provided.
On the other hand, the information required to access simultaneous notes is missing in
cases where the rhythmical structure is not determined.
An alternative CSP denition could instead apply a lattice-like score topology similar to
the one used by Humdrum which was discussed in Sec. 2.4.1. A container of a lattice-
like topology represents its contained elements in a two-dimensional array in contrast to
the one-dimensional item sequence of the Strasheela containers. In the two-dimensional
topology, one dimension represents simultaneous score objects of the same duration in
time slices (corresponding to the Humdrum records) and the other dimension represents a
temporal sequence of score objects (corresponding to the Humdrum spines). Homophonic
musical CSPs in which all voices share the same rhythmic structure, but which can
still feature dierent rhythmic values can be expressed very well with such a container
for a lattice-like topology. Even if the rhythmic structure of the score is undetermined
in the problem denition, such a topology always provides the information required to
access simultaneous notes as well as neighbouring notes.
124
6.3. Constraining Inaccessible Score Contexts
This approach can even be generalised for non-homophonic CSPs. To this end, the music
representation introduces an additional parameter for each score object in the lattice-
container whose value is a boolean variable. This parameter indicates whether its score
object is tied to its predecessor in the voice. A score object tied to its predecessor serves
a similar purpose as the null token in the Humdrum representation.
Additional constraints are applied to an object with a tie-parameter. For example, the
pitch of a note tied to its predecessor note is the same as the pitch of this predecessor
note. The application of these additional constraints again may require the user to access
a context which is undetermined in the CSP denition: in the problem denition, it
is not necessarily known which notes are tied and which are not: the variable of the
tie parameter may be undetermined. Therefore, reformulating the CSP is not always
sucient to avoid inaccessible score contexts. Another approach is required to address
this problem in a generic way.
6.3.4. Using Logical Connectives
The inaccessible score context problem can also be addressed by introducing logical
connectives such as disjunction, implication and equivalence as constraints. These logical
connectives constrain the validity of logical expressions and can thus constrain other
constraints. For example, implication can express that a constraint must hold in cases
where certain conditions are met. The information required to decide either way can be
missing from the problem denition (i.e. this information becomes available only during
the search process). The expressive power is thus similar to delayed rule application.
Still, rst-order logic (the formalism which denes logical connectives, [Kelly, 1997])
constitutes a well-established foundation to dene complex systems.
Section 6.1 dened constraints and rules as functions which always return a boolean
variable. This variable reects the validity of the constraint/rule. The rule formalism
was designed with logical connectives in mind. So far, the returned boolean variables
were only used to explicitly connect multiple rule applications by conjunctions and con-
straints/rules were always constrained to hold.
The following example demonstrates how the inaccessible score context problem is
addressed by logical implication. This example consists of a polyphonic CSP which con-
strains both the rhythmic structure and the pitch structure. If the rhythmical structure
is undetermined, the context simultaneous objects is inaccessible (depending on the
score topology used in the problem as was discussed above).
Figure 6.15 denes the harmonic rule simultaneousNotesAreConsonant which constrains
the interval between two notes to be consonant in case these notes are simultaneous.
Instead of directly accessing the context of simultaneous notes, this context is expressed
by a rule. The implication expresses that if the rule isSimultaneous (dened in Fig. 5.32)
is fullled (i.e. returns 1) then the rule isConsonant (dened in Fig. 6.14) must hold as
well.
125
6. The Strasheela Rule Formalism
simultaneousNotesAreConsonant(note
1
, note
2
)
def
=
isSimultaneous(note
1
, note
2
) isConsonant(note
1
, note
2
)
Figure 6.15.: A harmonic rule which makes use of an implication constraint: note
1
and
note
2
being simultaneous implies that their pitches form a consonant interval
Instead of directly accessing the score context of simultaneous notes to apply a rule, the
rule simultaneousNotesAreConsonant is applied to all object sets which potentially form
this score context, that is, all note pairs which are possibly simultaneous in a solution.
In an extreme case (where note duration domains contain extremely dierent values)
any note of one voice can occur simultaneously with any note of another voice in a
solution. To guard against this extreme case, it is safer to apply the rule to any possible
combination of two notes from dierent voices, that is, to each pair in the Cartesian
product of all notes of a rst voice and all notes of a second voice.
As before, a rule which is applied to multiple score object sets can be applied using a
suitable higher-order function. The higher-order function mapCartesianProduct expects
two lists xs and ys and a binary function fn and applies this function fn to all possible
combinations of xs and ys. For example, the call
mapCartesianProduct(xs, ys, fn)
results in
[fn(xs
1
, ys
1
), fn(xs
1
, ys
2
), . . . , fn(xs
1
, ys
n
), fn(xs
2
, ys
1
), . . . , fn(xs
n
, ys
n
)]
Using mapCartesianProduct, the rule simultaneousNotesAreConsonant is applied con-
veniently to all note pairs which are potentially simultaneous in a solution (Fig. 6.16).

mapCartesianProduct(getNotes(voice
1
),
getNotes(voice
2
),
simultaneousNotesAreConsonant)
Figure 6.16.: Application of simultaneousNotesAreConsonant to all potentially simulta-
neous note pairs
6.3.5. Comparison
Comparing the dierent approaches used to address the inaccessible score context prob-
lem is dicult, because these approaches are very dierent in nature.
126
6.3. Constraining Inaccessible Score Contexts
It is always a good solution to reformulate the CSP denition such that inaccessible
score contexts are avoided. However, this approach might not be generic enough: some
musical CSPs cannot be expressed that way. For example, the lattice-like score topology
is suited to solving homophonic CSPs where both the rhythmic as well as the pitch
structure is undetermined in the denition. However, non-homophonic CSPs again lead
to inaccessible score contexts: it may be unknown in the problem denition whether a
note is tied to its predecessor note.
The approach based on delayed rule application, as well as the one based on logical
connectives (e.g. the implication constraint), are both more generic. For example, both
approaches allow for polyphonic CSPs with rhythmically independent voices where both
the rhythmic as well as the pitch structure is undetermined. A logical connective is more
declarative whereas a delayed application is more procedural. Logical connectives consti-
tute a well-established formalism with a clear semantics. CSPs using delayed application,
on the other hand, can be harder to comprehend because of their procedural nature.
Furthermore, a logical connective such as implication is more expressive than a delayed
application because it works both ways whereas delayed rule application only works one
way. For example, the implication in the rule simultaneousNotesAreConsonant aects
both its arguments. If the two notes are simultaneous, then it follows that they must
also be consonant. At the same time, if the two notes are not consonant, then it follows
that they must not be simultaneous.
A delayed rule, on the other hand, only reacts to the availability of a score context.
If during the search process two notes turn out to form a dissonant interval, then the
delayed rule application cannot propagate this information (i.e. it does not deduce that
these notes cannot be simultaneous; constraint propagation was informally introduced
in Sec. 4.3.1 and will be explained in more detail in Sec. 7.2.1).
For constraining inaccessible score contexts, logical connectives are preferred in this
research to the other approaches because of their clear semantics and their generality.
Nevertheless, it should be noted that Strasheela also supports delayed rule applications
because Strasheelas approach to search is based on concurrent constraint programming
(see Sec. 7.1).
127
6. The Strasheela Rule Formalism
128
7. How Strasheela Finds a Solution
This chapter explains how Strasheela nds a solution for a musical constraint satisfaction
problem.
Chapter Overview
The rst two sections of this chapter provide requisite background information. Section
7.1 very briey outlines a declarative concurrent programming model which forms the
foundation for the constraint model introduced in Sec. 7.2. Section 7.3 explains how this
constraint model is customised to solve musical CSPs.
7.1. The Underlying Programming Model
The previous Chap. 5 and Chap. 6 explained the declarative semantics of Strasheela
in terms of three essential programming paradigms: constraint programming, object-
oriented programming, and functional programming. Constraint programming forms the
core of any music constraint system. Object-oriented programming is the foundation
of Strasheelas music representation. Finally, Strasheelas rule application mechanisms
utilises functional programming techniques.
This multi-paradigm computational model which subsumes these three paradigms was
chosen in the chapters above to explain what Strasheela does, because these programming
paradigms are well-known in the computer music community. Furthermore, a CSP based
on constraint programming plus functional programming can be conveniently notated by
a familiar mathematical notation. However, Strasheelas actual foundation is a dierent
computational model, namely a model which has the following fundamental features
[Schulte, 2002]:
Partial Values: Logic variables provide partial values: a variable can be free (nothing is
known about its value), partially determined (e.g. it is known that it is a list without
knowing all list elements) or fully determined. Constraints such as unication add
information about variable values.
Concurrency: Computations can be executed in multiple concurrent threads (threads are
created explicitly).
Synchronisation of Threads on Variables: The computation in a thread blocks if a logic
variable used in a statement of the thread lacks required information. Another
129
7. How Strasheela Finds a Solution
thread might provide this information. That way, threads communicate with each
other via variables (dataow variables). This form of concurrency is stateless and
thus declarative (no conicts of shared resources can occur).
First-Class Procedures: Computations can be abstracted in procedures which are rst-
class values and support lexical scoping.
This basic model introduces some notion of variables and constraints. However, this
model still lacks support for searching. Section 7.2 extends this model to a constraint
model which performs a complete search.
1
The basic model itself is explained in more
detail in [Schulte, 2002].
2
The functional programming and object-oriented programming capabilities of Strasheela
are based on this basic model. First-class functions are a special case of rst-class pro-
cedures: a function is a procedure which returns a single value. The object-oriented
programming model can be dened in terms of procedures (see [Abelson et al., 1985; van
Roy and Haridi, 2004]).
7.2. The Constraint Model Based on Computational Spaces
Strasheela is founded on a constraint programming model which introduces the notion of
computational spaces [Schulte, 2002], a programming construct for encapsulating specu-
lative computations such as constraint-based computations. This model makes the search
process programmable at a high-level. Computational spaces are the main extension of
this model when compared with the basic model outlined above.
For a new user, two aspects of this constraint model are apparent: (i) the model per-
forms local deductions (constraint propagation) automatically, whereas (ii) speculative
decisions during the search process (branching, distribution) are programmed by the
user. This observation led to the slogan propagate-and-search [van Roy and Haridi, 2004]
(also known as propagate-and-distribute [Schulte and Smolka, 2004]). This labelling fo-
cuses on two important features of this model (propagation and distribution) but masks
other features. The following Sec. 7.2.1 will therefore rst introduce the propagate-and-
search approach (the main ideas of this approach were already introduced in Sec. 4.3.1,
Section 7.2.2 explains propagate-and-search by an example). Section 7.2.3 will discuss
other essential features of the space-based constraint model.
Propagate-and-search constitutes a general search approach, but for brevity the expla-
nation here will focus on CSPs over variables whose domains consist of natural numbers
(nite domain integers). For the same reason, this explanation is restricted to binary
1
Additionally, this basic model can be extended to support committed choice constraint programming
(dont care non-determinism) by introducing a parallel conditional [Schulte, 2002].
2
Schulte [2002] calls this model Oz Light, and this model is basically the same as the Message-Passing
Concurrent Model in [van Roy and Haridi, 2004].
130
7.2. The Constraint Model Based on Computational Spaces
choice points (alternatives) in the search tree, although the approach supports n-ary
choice points.
The space-based constraint programming model is only outlined in the present text.
Schulte [2002] describes in detail the notion of computation spaces and the consequences
of their introduction. For example, the author discusses the features of a constraint model
based on spaces (such as user-dened search strategies), but also presents a comparison
of search approaches based on spaces (which perform copying and can perform recom-
putation, explained below) and approaches based on trailing (which undoes previous
decisions if required) which are used by most constraint systems. The programming
textbook by van Roy and Haridi [2004] explains how this model is dened on top of more
fundamental programming models such as declarative programming (i.e. functional and
logic programming without the notion of search) and concurrent programming. Schulte
and Smolka [2004] provide a tutorial on constraint programming with nite domain in-
tegers using this model, and M uller [2004] a tutorial on nite set constraints. Duchier
et al. [1998] apply this model to natural language processing and explain many aspects
of the constraint model as required for the purpose of the book.
7.2.1. Propagate and Search
The propagate-and-search approach represents explicitly what is known about a CSP.
There are two types of information on variables: explicit information about the values
of variables and explicit information about the constraints applied to these variables.
A computation space encapsulates the information available at a certain stage during the
search process. The information is expressed by (i) a constraint store, and (ii) a number
of constraint propagators which are contained in a space (see Fig. 7.1). The constraint
store stores partial information about the values of variables. A propagator represents a
constraint on these variables. A computation space also contains a distributor (explained
below).
3
propagator propagator
constraint store
distributor
computation space
...
Figure 7.1.: Propagate and search: a computation space encapsulates the information
available on a CSP at a certain stage during the search process
3
A computation space contains further entities which are not discussed in this introduction (e.g. a
mutable store, cf. [van Roy and Haridi, 2004]).
131
7. How Strasheela Finds a Solution
A constrained variable is a logic variable about which additional information is known: it
has a domain of possible values. The constraint store expresses this domain information
as well as which variables are equal. The store has the form of a conjunction of basic
constraints. A basic constraint is a piece of information which is restricted to very few
simple forms.
In the case of nite domain constraints, there are two forms. X D denotes that D is
the domain of the constrained variable X. In case the domain contains only a single value
as in X n (or X = n), then X is determined,. The second form X = Y expresses
that two variables are equal (unied). Neither X nor Y must be determined here. An
example of the information contained in a constraint store is the following conjunction.
X 1, . . . , 5 Y = 7 Z = X
All constraints which dont t into this form of basis constraints are non-basic constraints.
Most constraints are non-basic constraints. Examples include X < Y or all values
in [X
1
, . . . , X
n
] are distinct. Non-basic constraints are represented by propagators. A
propagator strives to amplify the information in the constraint store by adding new
basic constraints to the store which are consistent with the constraint store and follow
from the constraint expressed by the propagator itself. For example, for a constraint
store
X 1, . . . , 5 Y 1, . . . , 5
the propagator X < Y narrows the variable domains such that the constraint store is
updated to
X 1, . . . , 4 Y 2, . . . , 5
Each propagator is a software agent, that is a procedure running in its own thread.
Therefore, multiple propagators can reduce the domain of variables in turn. Constraint
propagation performs local deduction which reduces the search space (by narrowing the
domains of variables) without actually searching (i.e. without performing any decision
which may be wrong and requires undoing) and without excluding any solution. A prop-
agator disappears when it is entailed by the constraint store (i.e. all its variables are
determined). A space is solved when it contains no propagator (usually, all its variables
are determined then). A space is failed when it contains a failed propagator, that is a
propagator which is inconsistent with the constraint store.
However, constraint propagation does not necessarily lead to a solution (or a fail). For
example, for the store
X 1, 2 Y 1, 2 Z 1, 2
132
7.2. The Constraint Model Based on Computational Spaces
the propagators X ,= Y , X ,= Z, and Y ,= Z cannot reduce the problem further. Each
propagator only sees the variables it constrains but does not directly communicate with
other propagators. When no further propagation is possible, the hosting computation
space is said to be stable.
In this situation, constraint distribution comes to a decision which results in two new
computation spaces which are easier to solve (see Fig. 7.2). A distributor is also a software
agent which waits until its hosting computation space becomes stable. The distributor
then yields two child spaces of the stable parent space. Both child spaces inherit all
information available in the parent space (e.g. the constraint store and the propagators).
Additionally, the distributor applies an arbitrary constraint C to any variable in one
child space and its complement C to the corresponding variable in the other child
space. Thus, the distributor executes a nondeterministic choice operation where C and
C denote alternatives. For example, the constraint C may determine some variable to
a certain value (e.g. to the rst value in its domain), in which case C excludes this
value from the variable domain.
parentcomputationspace
constraintstore
propagator propagator
[...]
distributor
childcomputationspace
constraintstore
propagator propagator [...]
distributor
childcomputationspace
constraintstore
propagator propagator [...]
distributor
constraintC constraintC
Figure 7.2.: Constraint distribution creates two child spaces which are the result of two
complementary decisions (expressed by the two added constraints C and C)
Adding a constraint and its complement (i.e. two equivalent alternatives) at the same
time does not change the set of solutions. It will, however, often restart propagation.
A distribution strategy denes the decision that is performed by the distributor. A com-
mon strategy is rst fail : this strategy determines some variable with the smallest domain
to its leftmost domain value (and removes this value from the variable domain in the
second child space). The following section 7.2.2 explains this distribution strategy with
an example.
The branching caused by constraint distribution results in a search tree. The example in
the following section also depicts the corresponding search tree in Fig. 7.4. This search
tree is investigated by an exploration strategy for solutions, for example by the depth
rst search algorithm (the exploration strategy concept is explained later in more detail
in Sec. 7.2.3.3).
133
7. How Strasheela Finds a Solution
The combination of constraint propagation and constraint distribution together with
an exploration strategy yields a complete method to solve a CSP.
7.2.2. An Example
The following paragraphs illustrate constraint propagation and distribution by comput-
ing an all-distance series. An all-distance series is a musical CSP very similar to an
all-interval series (Sec. 3.2.2). Both CSPs dene a series which consists of unique pitch
classes and unique intervals between these pitch classes. The CSPs only dier in the
way the pitch classes and the intervals are related. In an all-distance series, the intervals
between the pitch classes are absolute distances (instead of inversional equivalent inter-
vals as in an all-interval series). That is a fourth upwards and a fourth downwards are
regarded as the same interval (instead of a fourth upwards and a fth downwards in the
case of an all-interval series).
Figure 7.3 shows the denition of the all-distance CSP of length 4 (i.e. the solution series
consists of only 4 pitch classes). Figure 7.4 shows the full search tree for all solutions of
this CSP. Each tree node represents a computation space. Solved spaces are drawn as
diamonds , failed spaces as boxes and distributable spaces as circles .
xs
def
= a list of 4 undetermined integers, each with the domain 0, . . . , 3
dxs
def
= a list of 3 undetermined integers, each with the domain 1, . . . , 3
/* Constraints relation between xs and dxs. */
3

i=1
dxs
i
= [xs
i
xs
i+1
[
/* All elements in the solution xs as well as the intervals dxs between them are pairwise distinct. */
distinct(xs)
distinct(dxs)
Figure 7.3.: All-distance series denition of length 4
Only 17 nodes are necessary to nd all 4 solutions. The table below the tree shows the
basic constraints on the elements of the solution series xs and the intervals dxs before
their respective spaces get distributed.
The constraints added by the rst fail distribution strategy are shown next to the tree
arcs. The strategy always aects some variable with minimal domain size (in the case
where multiple variables share the same minimal domain size, the leftmost variable is
chosen). In the rst space it creates a choice point which either binds the rst element
of dxs to 1 (in space 2) or removes this value from the domain (in space 3).
134
7.2. The Constraint Model Based on Computational Spaces
5 6
4
2 3
1
dxs =1
1
2
dxs =2
xs =1
1
xs =1
1
dxs =1
1
Space xs dxs
1 [{0, . . . , 3}, . . . , {0, . . . , 3}] [{1, . . . , 3}, {1, . . . , 3}, {1, . . . , 3}]
2 [{0, . . . , 3}, . . . , {0, . . . , 3}] [1, {2, 3}, {2, 3}]
3 [{0, . . . , 3}, . . . , {0, . . . , 3}] [{2, 3}, {1, 2, 3}, {1, 2, 3}]
4 [{1, 2}, {1, 2}, {0, 3}, {0, 3}] [1, 2, 3]
5 [1, 2, 0, 3] [1, 2, 3]
6 [2, 1, 3, 0] [1, 2, 3]
Figure 7.4.: All-distance series: search tree for all solutions of length 4
After each distribution step propagators adjust the domains of all variables accordingly.
In space 2 the propagator constraining all intervals to be distinct removes 1 from the
other two interval domains in dxs. In space 4 the propagator determines the third interval
to be 3, which awakens the distance propagator to reduce the domain of last two series
elements in xs to 0, 3. This again reduces the domain of the rst two series elements
as well, because all elements are constrained to be dierent.
7.2.3. Features of the Space-Based Constraint Model
The constraint programming model based on the notion of computational spaces has
a number of benecial features. The following paragraphs list those features which are
particularly inuential for music constraint programming. These features are partly the
result of the programmable nature of this constraint programming model and there-
fore characteristic of this model (in particular recomputation, explained in Sec. 7.2.3.5).
Other features do not necessarily depend on spaces and are in some form shared by other
popular constraint systems (e.g. SICStus Prolog and ECLiPSe both support reied con-
straints, explained in Sec. 7.2.3.4). Nevertheless, all these features are listed here because
existing music constraint systems often do not support any of these features.
4
4
The literature on music constraint programming already introduced the term constraint reication
in the discussion of the BackTalk system (cf. Roy and Pachet [1997]), but with a clearly dierent
meaning. The Smalltalk programming language in which BackTalk is implemented allows a
program to reason about itself: Smalltalk is a highly reective language. A reied Smalltalk object
can be processed as data. For example, the Smalltalk stack is reied which allows a program to
inuence the order in which statements on the stack are executed.
135
7. How Strasheela Finds a Solution
7.2.3.1. Constraint Propagation Between Domain Specic Variables
A constrained variable features a nite domain which constitutes a nite set of possible
values for this variable. This domain is uniform; all possible values for a variable belong to
the same mathematical domain (i.e. constrained variables are quasi typed). Particular
common domains include nite domain integers (i.e. natural numbers, often simply called
nite domain) [Schulte and Smolka, 2004] and nite sets (of natural numbers) [M uller,
2004]. Further domain examples are real intervals [Daz et al., 2005], graphs [Dooms
et al., 2005b], records [Duchier et al., 2004], and maps (domain of functions) [Deville
et al., 2005].
5
Propagators maintain constraints between variables of specic domains. For example,
an addition constraint X + Y = Z is dened for three numbers (e.g. nite domain
integers or real intervals). Customarily, constraints are named accordingly (e.g. nite
domain constrains or nite set constraints). Many constraints express a relation between
variables of the same domain, while some constraints dene a relation between variables
of dierent domains. For example, the membership constraint X S often expresses a
relation between a nite domain integer and a nite set of integers.
Constraint propagation is important, because it reduces the domains of variables by
making local deduction. In this way, propagation signicantly reduces the search space
without making any decision which is potentially wrong and requires undoing.
Constraint domains allow the combination of natural modelling with ecient solving.
Domains are based on mathematical concepts which allow for universal application.
Constraints on domain specic variables usually implement highly-optimised domain
specic propagation algorithms.
Many existing music constraint systems, on the other hand, support variable domains
which consist of arbitrary values. However, these systems do not support constraint
propagation which considerably aects their performance.
7.2.3.2. Distribution Strategy
The propagate and search approach allows the user to program how the search process is
conducted. Two independent aspects of the search process are controlled separately: how
the search tree looks like and how this tree is explored. The former aspect is controlled
by the distribution strategy, the latter by the exploration strategy (explained below).
The distribution strategy denes the shape and content of the search tree. For example,
it controls how many nodes exist and what constraints are added at each branching of
the search tree. The user can dene simple distribution strategies from scratch with a
few lines of code (cf. [Schulte, 2002, p. 37]). However, a high-level interface can make the
5
The references given in this paragraph point to Oz-related literature, that is literature of immediate
relevance for the Strasheela prototype. Schulte [2002, p. 1] provides more general citations on variable
domains.
136
7.2. The Constraint Model Based on Computational Spaces
denition of distribution strategies more convenient. This section presents an interface
which is a distinctly simplied version of the Oz procedure FD.distribute [Duchier
et al., 2004], which is the standard means for solving nite domain integer CSPs in Oz.
Essentially, distribution strategies dier in two aspects: (i) in which order are variables
visited and (ii) which domain value is selected by the distribution strategy. The high-level
interface presented here only expects a formal specication for these two aspects.
Order: Which variable is distributed? This aspect is specied by a binary boolean func-
tion. Internally, the distributor uses this function to compare pairs of variables:
the variable which performs best with respect to this order relation is distributed
(if multiple variables perform equally well, then the rst is chosen). For example,
the rst-fail strategy distributes some variable with the smallest domain size. This
variable ordering is specied as follows:
myOrder(X, Y )
def
= getDomainSize(X) getDomainSize(Y )
Value: How does the distribution strategy aect the domain of the selected variable?
This aspect is specied by a unary function expecting a variable and returning
a reduced domain specication for this variable (most often a single value). For
example, the rst-fail strategy decides for the minimal domain value:
myValue(X)
def
= getMinimalDomainValue(X)
The high-level interface for creating distributors cannot be dened in the notation used
so far, because it requires a notation with support for procedures and concurrency.
6
It depends on the actual CSP as to which distribution is suitable (e.g. ecient) for this
problem. A variable ordering which follows the well-proven rst-fail principle rst deter-
mines those variables which are particularly hard to solve [Bartak, 1998]. For instance,
6
The high-level interface described in the present section for dening distribution strategies is dened
below in Oz notation [van Roy and Haridi, 2004]. This denition is best understood together with
[van Roy and Haridi, 2004, Chap. 12], which explains in a step-by-step manner how a distribution
strategy is executed in terms of space operations.
The procedure MyDistributor expects as arguments the order specication Ord and value specica-
tion Val as discussed above, together with a list Xs containing all variables in the CSP. MyDistributor
waits until its hosting space is stable. It then lters out already determined variables. In case there
are no undetermined variables left (i.e.Vars is nil and the CSP is solved) the distributor does noth-
ing. Otherwise, the variable to distribute is selected (Order is a higher-order function a relative of
Sort which returns the list element tting best the given ordering function), bound to Var, and
a domain specication Dom is computed for this variable. The binary choice states that the domain
of Var is either Dom or its complement. These two alternative constraints are added to the two child
spaces (which are created by the choice statement). The recursive call of MyDistributor adapts the
distribution for these child spaces to the reduced list of variables (once ltered out, variables will
never be considered for distribution again).
Again, the example aims for clarity and not eciency (the variables are traversed twice: once when
ltered and then again when selecting the variable to distribute).
137
7. How Strasheela Finds a Solution
the previously discussed rst-fail strategy rst distributes some variable with the small-
est domain. An alternative approach distributes some variable to which most constraints
are applied.
Common decisions for a particular domain value in addition to the domain minimum
include the median or the maximum. Alternatively, the domain of a variable can be
reduced to its lower half instead of deciding on a specic domain value (this technique
is sometimes called domain splitting). A well-proven rule of thumb for value orderings
is the succeed-rst principle which decides on a promising domain value rst [Bartak,
1998].
A distribution strategy can dene heuristics to place a better solution earlier in the
search tree. Such a best-rst principle can be interpreted as a variation of the succeed-
rst principle. A typical heuristic for a musical CSP avoids repetition and uniformness.
For many musical CSPs it is appropriate to randomise the decision for a domain value.
Still, it is important that a distribution strategy is deterministic, that is, if executed
multiple times a distribution strategy must make the same decision in each computation
space. This is particularly important for large CSPs which require recomputation (Sec.
7.2.3.5). A distribution strategy can be seemingly random but still deterministic when
based on pseudo-random values.
Dynamic Variable Ordering
It is important to note that a distribution strategy decides which constraint to add only
when it is actually required and thus can make use of all the information available at the
current state of the search (dynamic variable and value order). In contrast, most existing
music constraint systems apply search strategies in which the order of decisions is deter-
mined before the search starts (static order). Section 7.3 will discuss the consequences
of static versus dynamic variable orderings for musical CSPs.
proc {MyDistributor unit(order:Ord value:Val) Xs}
{Space.waitStable}
local
Vars={Filter Xs fun {$ X} {FD.reflect.size X} > 1 end}
in
if Vars \= nil
then
Var = {Order Vars Ord}
Dom = {Val Var}
in
choice {FD.int Dom Var} [] {FD.int compl(Dom) Var} end
{MyDistributor unit(order:Ord value:Val) Vars}
end
end
end
138
7.2. The Constraint Model Based on Computational Spaces
7.2.3.3. Exploration Strategy
While a distribution strategy denes the shape of the search tree, an exploration strategy
traverses this search tree to nd solutions. The denitions of a distribution strategy and
an exploration strategy are independent of each other. A search engine (which imple-
ments an exploration strategy) expects a CSP together with a distributor encapsulated
in a script (a procedure).
A basic exploration strategy such as depth-rst search can be used to nd a single
solution, as well as multiple or all solutions. Depth-rst search is particularly adequate
for nding a single solution. In this case, the most critical aspect is a suitable distribution
strategy.
Several exploration strategies have been proposed which do not require exploring the
full search tree for nding a good or an optimal solution according to a user-dened
formal criterion. Limited discrepancy search (LDS) [Harvey and Ginsberg, 1995] trusts
that the heuristics which dene the search tree (i.e. the distribution strategy in our case)
is appropriate. This heuristic denes a preference for the rst alternative of two child
spaces. LDS limits the number of discrepancies, that is, how often the path in the search
tree takes the wrong turn and follows the second alternative. Schulte [2002] adapts LDS
to the space-based constraint model.
Informed strategies exploit information contained in partial solutions to decide where
to continue the exploration. Best-rst search or A*-search (Russell and Norvig [2002])
has been adapted for the space-based constraint model [Schulte, 2002; Tack and Botlan,
2005].
Best-solution search (branch-and-bound, BAB) [Schulte, 2002] searches for a series of
solutions. Whenever the search nds a solution, the next solution is constrained to be
better with respect to a user-dened comparison constraint. This additional comparison
constraint performs propagation (i.e. excludes too poor solutions). When no further solu-
tion can be found, the last solution is optimal with respect to the comparison constraint.
The exploration strategies above expect a formal criterion for a best solution. For LDS,
this criterion is the heuristics which creates the search tree (i.e. the distribution strategy),
best-rst search expects a cost function, and BAB a comparison constraint.
For some CSPs, however, it is extremely hard to formalise what a good solution con-
stitutes. For example, for a musical CSP it is often impossible to state in a formal way
what distinguishes a good solution from a poor one. Still, musicians are easily able to
make this distinction.
The formalised musical knowledge expressed in a CSP can be combined with the non-
formalised expertise of the user when the user interactively guides the exploration of
the search tree. Schulte [1997] proposes the Oz Explorer to conduct a visual search.
The Explorer features a graphical user interface which shows the search tree and allows
for interactive exploration. For example, the user can examine the partial solutions of
nodes in the search tree and decide which alternative to explore further. In addition, the
139
7. How Strasheela Finds a Solution
Explorer is an excellent device to analyse the search process (e.g. to improve or debug a
distribution strategy).
Exploration strategies can be dened from scratch (cf. [Schulte, 2002]). However, in
doing so the user must take many subtleties into account. Therefore, Tack and Botlan
[2005] propose high-level abstractions to make the denition of exploration strategies
more convenient for the user. These abstractions automatically incorporate well-known
optimisations (e.g. last alternative optimisation [Schulte, 2002]) and recomputation (see
Sec. 7.2.3.5) and also provide a visual search interface.
An exploration strategy is often called a search strategy instead. The present text prefers
the more explicit term exploration strategy. This term cannot be confused with the
broader notion of the term search strategy which covers all aspects of the search process
(i.e. constraint propagation, distribution, as well as the exploration of the search tree).
7.2.3.4. Reied Constraints
Complex CSPs often call for constraining under which circumstances some constraints
hold. For example, logical connectives such as disjunction, implication, and negation
constrain the validity of other constraints. Section 6.3.4 discussed an example which
applies an implication constraint to express that two notes are consonant in cases where
they are simultaneous. A similar technique allows the user to solve over-constrained
problems where some constraints cannot be satised.
A reied constraint (sometimes called meta-constraint) is a constraint C which reects
its validity in a boolean variable B. A boolean variable is expressed by a nite domain
variable with the domain 0, 1 and is often called a 0/1-variable (0 denotes false and 1
denotes true).
(C B) = 1 B 0, 1
The discussion of reied constraints clearly shows the dierence between the declarative
semantics of constraints discussed above (e.g. in Chap. 6) and the operational semantics
discussed in the present chapter. In the declarative semantics, constraints are expressed
by functions which expect variables as arguments and always return a boolean variable.
Thus, all constraints are quasi-reied constraints. These declarative semantics were only
introduced to simplify the discussion in this text (e.g. to avoid introducing a notation
which expresses concurrent procedures).
In the operational semantics, constraints are expressed by propagators (i.e. concurrent
procedures) which inuence a set of variables but do not return any value. Such a con-
straint must always be true. Nevertheless, a propagator may constrain a 0/1-variable
which reects whether the values of the other variables full the constraint expressed
by the propagator. A reied constraint is thus a special case which is used when this
additional expressive power is required. For example, reied constraints are required to
140
7.2. The Constraint Model Based on Computational Spaces
express a disjunction or an implication between two constraints in Strasheelas opera-
tional semantics. On the other hand, a conjunction does not require reied constraints
(see below).
Many propagators come in two variants: a reied version and a non-reied version.
The constraint model based on computation spaces also supports a generic reication
operation for arbitrary constraints which also provides propagation [Schulte, 2002].
Like any variable, a 0/1-variable can be freely constrained. Popular constraints for
0/1-variables are logical connectives (i.e. conjunction, disjunction, exclusive disjunction,
negation, implication, and equivalence).
The dierence between the declarative semantics expressed by functions and the op-
erational semantics expressed by propagators has an additional eect. Because every
constraint returns a boolean variable in the declarative semantics, it is necessary to
explicitly express a conjunction of all these constraints. In the operational semantics,
a constraint (which is not a reied constraint) must always hold. Consequently, multi-
ple independent constraints must all hold and explicit conjunctions are not required. In
eect, less constraints are required in the operational semantics.
Modelling Soft Constraints
Composers often want to formulate a preference rather than a strict rule. For instance, a
composer might prefer small melodic intervals but still wishes to allow for larger intervals.
There are cases where composers merely want to relax a constraint and other cases where
they want an optimal solution.
Soft constraints can be modelled by reied constraints. Besides, reied constraints also
allow for solving over-constrained problems where only some of the stated constraints
can be satised.
A 0/1-variable is a nite domain integer and all integer constraints can be applied to
0/1-variables. For instance, multiple 0/1-variables can be added. In Fig. 7.5, the con-
straint howManyTrue constrains how many 0/1-variables in the list bs equal 1 (i.e.
true) by adding these variables (the examples still use the declarative semantics for con-
venience). Related examples are the common constraints exactly, atLeast and atMost
(exactly/atLeast/atMost N 0/1-variables equal 1).
howManyTrue(bs, N)
def
= N =

bs
Figure 7.5.: The constraint howManyTrue constraints that N (a nite domain integer)
variables in bs (a list of 0/1-variables) are true (i.e. 1)
The constraint probability (Fig. 7.6) is a variant of howManyTrue and constrains that the
probability for a variable in bs (a list of 0/1-variables) to equal 1 lies between the oating
point numbers min and max (a small error is introduced by the rounding operation)
141
7. How Strasheela Finds a Solution
probability(bs, min, max)
def
=
let l
def
= length(bs)
minDomain
def
= round(l min)
maxDomain
def
= round(l max)
in howManyTrue(bs, fdInt(minDomain, . . . , maxDomain))
Figure 7.6.: The constraint probability constrains the probability of a 0/1-variable in bs
equaling 1
The compositional rule preferSteps expresses skips in the melody are allowed, but there
should be more steps than skips (Fig. 7.7). This rule can be expressed by howManyTrue
or probability. Although the rule preferSteps is not a truly soft rule, it models the eect
of a soft rule on each individual interval, by applying a strict rule on the sequence of all
intervals.
preferSteps(intervals)
def
=
probability(map(intervals, f : f(Interval )
def
= Interval < 3), 0.7, 0.9)
Figure 7.7.: Prefer intervals which are smaller than a minor third: the probability of
intervals which are less than 3 is between 0.7 and 0.9
The rule preferSteps expects intervals as an argument, which is a list of the absolute
distances between the pitches in a melody (which are expressed by key-numbers). In the
following rule application example, the intervals between the melodyNotes are created
by processing this list of notes with the help of map2Neighbours (map2Neighbours is
introduced in Sec. 6.2.3).
let melodyNotes
def
= getItems(myMelody)
intervals
def
=
map2Neighbours(melodyNotes,
f : f(note
1
, note
2
)
def
=
/* Create and return the variable Interval , which is constrained
to the absolute distance between the pitches of two neighbouring
notes. */
let Interval
in Interval where
Interval = [getPitch(note
1
) getPitch(note
2
)[)
in preferSteps(intervals)
The rule preferSteps constrains the probability of steps in a melody, but it does not
optimise their number in any way. Sometimes, however, a composer is not only interested
142
7.2. The Constraint Model Based on Computational Spaces
in relaxing a constraint, but in nding a solution which optimally fullls a soft constraint.
For example, a composer may be interested in nding a solution which is guaranteed
to have a minimal number of skips in the melody. However, strictly constraining all
intervals to steps may not be possible, because some rules (e.g. harmonic rules) force
some melodic skips.
In such a case, a rule like preferSteps can be redened such that it constrains a good-
ness measure which expresses how well the rule is fullled (Fig. 7.8). An optimisation
technique like best-solution search (Sec. 7.2.3.3) can then be used to nd an optimal
solution: this best-solution search would constrain any next solution to have a higher
goodness measure than the previous one until no better solution can be found.
preferSteps2(intervals, goodness)
def
=
howManyTrue(map(intervals, f : f(Interval )
def
= Interval < 3), goodness)
Figure 7.8.: A variant of preferSteps (Fig. 7.7) which measures the goodness of the
solution: a higher goodness value means more intervals are smaller than a minor third
This approach works well to optimise a single soft rule. Moreover, it can be extended
for several soft constraints. Best-solution search optimises according to only a single
comparison constraint. However, multiple goodness measures can be composed into a
single parent goodness measures (e.g. by weighting individual goodness measures with
individual factors and summing all weighted goodness measures). Nevertheless, this ap-
proach is somewhat cumbersome, and the future work Sec. 11.2 points out a dierent
approach.
7.2.3.5. Recomputation
Recomputation is a technique which saves memory (RAM), often at the expense of in-
creased run time [Schulte, 2002]. Musical CSPs are often highly complex: these problems
involve much information (memory-demanding data) and generate deep search trees. Re-
computation is an essential technique for solving large problems which otherwise would
require an excessive amount of memory.
Search performs decisions which can be wrong and possibly must be reversed (i.e. the
path through the search tree must take the other turn to nd a solution). To this end,
most constraint systems are based on trailing. Trailing records how the search process
changes state so that it can go back to an earlier node of the search tree (backtrack) and
explore a dierent branch.
The space-based constraint model replaces trailing by the two techniques: copying and
recomputation. In this model, a node in the search tree and a copy of a computation
space are independent concepts. Pure copying (also called pure cloning) leaves a clone
of a parent space (with all the information it contains) at every branching node of the
search tree. This approach allows the search process to move back upwards to an earier
143
7. How Strasheela Finds a Solution
space copy in the search tree and that way to reverse decisions. This approach makes
distribution and exploration strategies fully programmable. However, it also requires a
large amount of memory for large search trees. Full recomputation, in contrast, leaves
only a single space copy in the root node of the search tree. Whenever a decision must be
undone, the required computation space is reconstructed by repeating all decisions which
led to it. Recomputing a node in the search tree only requires to know the path leading
from a (direct or indirect) parent-node which has a space copy to the node to recompute.
The original decisions are repeated by repeating the distribution (Sec. 7.2.3.2) for the
nodes in this path. At each node on the way, those distribution constraints are added
to the space copy which correspond with the continuation of the path to the node to
recompute. The distribution constraints possibly cause constraint propagation before the
next distribution on the way is conducted.
7
However, pure copying and full recomputation are two extreme cases. Fixed recomputa-
tion leaves a space copy at every n-th branch of the search tree, where n (the maximal
recomputation distance) is dened by the user. Adaptive recomputation always leaves
an additional space clone between a newly recomputed node and the parent node it
is recomputed from. Adaptive recomputation results in more densely positioned space
clones where failed spaces occur and so takes clustered failures into account (when search
encounters a failed node, often the full subtree fails [Schulte, 2002]). Finally, batch re-
computation [Choi et al., 2001] records all decisions of the distribution strategy and
reconstructs a space by applying all recorded constraints in a single step which reduces
the required runtime (e.g. omits the intermediate propagation steps). Recording all con-
straints applied by the distribution is only a very small overhead when compared with
leaving a space copy at every search tree branch.
8
In summary, recomputation allows the user to trade memory for run time and that
way allows for the solving of complex CSPs. Whats more, Schulte [2002] showed that
recomputation can even require less run time than pure copying.
7.3. Specialising the Constraint Model for Music
The present section explains how Strasheela customises the space-based constraint model
for a generic music constraint system.
7.3.1. The Basic Idea
7.3.1.1. Motivation
Experience in constraint programming in general has shown that the order in which
variables are visited during the search process has an immense impact on the size of
7
Recomputation requires a deterministic distribution strategy as was already pointed out in Sec. 7.2.3.2.
8
Batch recomputation is presently not supported by Strasheelas implementation platform Mozart and
thus not by the Strasheela prototype (Sec. 9).
144
7.3. Specialising the Constraint Model for Music
the search space [Kumar, 1992]. This general experience has been conrmed for music
constraint programming. For example, score-PMC (Sec. 3.3.1.2) applies a sophisticated
variable ordering which is optimised for polyphonic music and is well-suited for arbitrarily
complex rhythmical structures [Laurson, 1996]. The poor performance of Arno (Sec.
3.3.4), on the other hand, stems primarily from its nave variable ordering which causes
much redundant work [Anders, 2000].
Most existing music constraint systems apply a static variable ordering: the order in
which variables are visited during the search process is settled before the search actually
starts. However, musical CSPs can be extremely complex and it is often impossible to
nd an ecient static variable order. Consequently, only specic CSP classes are solved
eciently by static search orderings and some systems even make the denition of other
CSPs impossible. For instance, score-PMC is limited to CSPs which fully determine the
rhythmical structure in the problem denition: score-PMC depends on this rhythmical
structure to compute its ecient static variable ordering [Laurson, 1996]. The search of
score-PMC proceeds from left to right in the polyphonic score: basically, notes with
a smaller start time are visited earlier. To compute such a variable ordering before the
search starts, it is of course essential that the temporal structure of the score is xed.
In contrast to score-PMC, Arno allows the user to constrain the rhythmical structure
and the pitch structure. Still, its static search order is only suited to a small class of
musical CSPs. Arno determines the score voice-wise: after all notes of the rst voice are
determined, the search proceeds to the notes of the next voice. This approach detects
conicts between voices very late and only performs well in the case of canons.
A dynamic variable ordering, on the other hand, is not xed before the search begins.
Instead, a dynamic variable ordering can make use of all information available in a
partial solution to decide which variable to visit next. For example, a dynamic variable
ordering allows a system to apply the variable ordering of score-PMC even if the temporal
structure is not known in the problem denition: the search always continues with the
most-left note in the score which is still undetermined.
A suitable variable ordering is highly problem-dependent. For example, a contrapuntal
CSP might be solved best by processing from left to right in the score and completing
all voices more-or-less in parallel. In contrast, a harmonic CSP might be solved best by
rst deciding upon an abstract harmonic progression (e.g. I ii V I ), then for the pitches
of the bass voice and the soprano voice and nally for the other voices.
A generic music constraint system therefore requires two features: it should allow for
dynamic variable orderings and it should allow for dening variable orderings suitable
for the problem at hand (and for selecting predened variable orderings for convenience).
As well as solving musical CSPs which could be solved by existing systems, such features
allow for solving many CSPs which were intractable before. The importance of these two
features (together with constraint propagation) can hardly be over-estimated for a system
which aims to solve diverse musical CSPs reasonably eciently.
145
7. How Strasheela Finds a Solution
7.3.1.2. Approach
The requirements detailed above led to the application of the space-based constraint
model (Sec. 7.2) as a foundation for the generic music constraint system Strasheela.
This model allows for a dynamic variable (and value) ordering and allows for the free
denition of such orderings. A variable ordering is dened by dening a distribution
strategy (Sec. 7.2.3.2) independently of the problem denition.
The denition of a distribution strategy is orthogonal to the other features of the space-
based constraint model. By dening special score distribution strategies, Strasheela pre-
serves all other features of this constraint model. For example, Strasheela supports e-
cient constraint propagation, user-dened exploration strategies, reied constraints and
recomputation (Sec. 7.2.3).
A distribution strategy bases its decisions on the information presently available in the
partial solution. For example, Sec. 7.2.3.2 discussed the rst-fail distribution strategy
(which distributes some variable which at the present stage of the search process has
the smallest domain). A generic music constraint system must allow for the denition
of arbitrary score distribution strategies. Such a system requires that a score distribu-
tion strategy has access to any information contained in the partial solution score. The
design of Strasheela fullls these requirements. Instead of processing variables (as the
distribution strategies discussed before), a score distribution strategy processes param-
eter objects (e.g. pitch instances, see Sec. 5.4.1). A parameter instance provides a link
to its hosting item instance and so to the full music representation a score distributor
can thus make use of any information available for its decisions.
A distribution strategy reduces the domain of the distributed variable. A score distributor
aects the value attribute of a parameter object which is a variable. For example, a
suitable technique for many musical CSPs is to determine this variable setting it to a
(pseudo) randomly selected value of its domain.
A suitable distribution strategy results in a relatively small search tree. This requires
careful design, because any distribution step makes a decision which possibly leads to a
fail. Constraint propagation, on the other hand, is never wrong when it reduces variable
domains. This observation leads to an important rule-of-thumb for designing distribution
strategies: a good strategy keeps the number of required decisions at a minimum and
distributes then in such a way that propagation does most of the work.
Section 6.3 discussed the problem that musical CSPs often constrain score contexts which
are inaccessible in the problem denition. As long as these contexts are inaccessible, con-
straints on these contexts propagate very weakly (if at all). For example, if simultaneous
notes are constrained by a harmonic rule then the constraints of this rule rule can only
propagate for notes which are known to be simultaneous. If the rhythmical structure is
undetermined in the problem denition and hence the context simultaneous notes
is inaccessible propagation blocks until this context becomes inaccessible. A distribu-
tion strategy for a CSP which constrains inaccessible score contexts should therefore
take special care to resolve these contexts at an early stage so as to make propagation
146
7.3. Specialising the Constraint Model for Music
possible. In analogy to the rst-fail principle for variable orderings (Sec. 7.2.3.2), the
present research calls this guideline the resolve-inaccessible-contexts principle. For in-
stance, in many musical CSPs the temporal structure should be determined relatively
early in case-dependent contexts (for example where simultaneous notes are inaccessible
but constrained).
7.3.2. Score Distribution Strategies
The present section proposes a number of score distribution strategies which cover a
large number of musical CSPs. Further strategies can be dened by the user according
to needs.
To make the discussion of score distribution strategies more concise, the high-level in-
terface for the denition of distribution strategies introduced in Sec. 7.2.3.2 has been
adapted for score distribution strategies. The original interface expects two functions:
a binary order function which denes the variable ordering and a unary value function
which denes how the domain of the distributed variable is reduced. In the adapted
interface, the order function expects two parameter instances, but the value function
remains the same.
9
7.3.2.1. Adapting the First-Fail Principle
Section 7.2.3.2 discussed the rst-fail principle as a rule-of-thumb used to formulate
distribution strategies. This guideline is also well-suited to many musical CSPs. Typical
strategy examples either distribute a variable with the smallest domain or alternatively
a variable on which most constraints are applied.
9
This adapted interface is very similar to the interface dened in footnote 6. The new denition sub-
stitutes occurrences of a single variable in the former denition by accessing this variable from a
parameter object using the method getValue.
proc {MyScoreDistributor unit(order:Ord value:Val) Xs}
{Space.waitStable}
local
Params={Filter Xs fun {$ X} {FD.reflect.size {X getValue($)}} > 1 end}
in
if Params \= nil
then
Var = {{Order Params Ord} getValue($)}
Dom = {Val Var}
in
choice {FD.int Dom Var} [] {FD.int compl(Dom) Var} end
{MyDistributor unit(order:Ord value:Val) Params}
end
end
end
147
7. How Strasheela Finds a Solution
The denition of the rst-fail variable ordering function for parameter objects diers
only slightly from the ordering function for plain variables. The order expressed by
the function isParamWithSmallerDomainSize (Fig. 7.9) results in a variable ordering
which distributes the value of some parameter with the smallest domain size. The only
dierence to the rst-fail strategy for plain variables presented above is that the version
for parameter instances has to access the parameter values by getValue.
isParamWithSmallerDomainSize(param
1
, param
2
)
def
=
getDomainSize(getValue(param
1
)) getDomainSize(getValue(param
2
))
Figure 7.9.: Variable ordering specication of rst-fail strategy for parameters
A full distribution strategy specication complements the variable ordering with a deci-
sion as to how the variable domain is reduced (the term value ordering denotes a slightly
less general concept but will still be used here for brevity). Commonly, the rst-fail
strategy decides on the smallest domain value. For musical CSPs, a heuristics which
randomly selects a domain value is often preferable. Still, such a randomised decision
must be deterministic such that recomputed decisions would decide the same way (cf.
Sec. 7.2.3.2).
Order: isParamWithSmallerDomainSize
Value: For example getMinimalDomainValue or getRandomDomainValue
In the remainder of this chapter, a number of variable orderings suitable for various
musical CSPs are presented. All these variable orderings can be complemented by any
value ordering.
7.3.2.2. Resolving Inaccessible Score Contexts
Resolving a Single Context
The general rst-fail principle is primarily suitable for CSPs in which all constrained score
contexts are already accessible in the problem denition. Otherwise, inaccessible contexts
which are constrained (Sec. 6.3) should be resolved early to make the propagation of these
constraints possible (resolve-inaccessible-contexts principle, Sec. 7.3.1.2).
A typical example of an inaccessible but constrained score context is the context of
simultaneous score objects in case the temporal structure of the music is undetermined
in the problem denition. One variable ordering which addresses this inaccessible context
determines all temporal parameters rst and other parameters only later.
It is sucient to test whether the rst of two parameter objects is a temporal parameter
in order to express a variable ordering which determines temporal parameters rst (Fig.
148
7.3. Specialising the Constraint Model for Music
7.10). In the case where isTemporalParameter (which tests class membership) returns
true for the rst parameter, then the ordering function preferTemporalParameter returns
true and this parameter will be distributed. Alternatively, if preferTemporalParameter
returns false then this expresses a preference for param
2
whether it actually is a
temporal parameter or not. In the case where param
2
is not a temporal parameter, then
this preference will be overruled as long as there is still any undetermined temporal
parameter. In the case where there is no undetermined temporal parameter left, it is
adequate to distribute a parameter which is not a temporal parameter.
10
preferTemporalParameter(param
1
, param
2
)
def
= isTemporalParameter(param
1
)
Figure 7.10.: Variable ordering specication which determines all temporal parameters
rst in order to make score contexts based on the temporal structure (e.g. simultaneous
notes) accessible
A variable ordering which rst determines the temporal structure of the score is only a
special case for variable orderings which rst resolve some inaccessible but constrained
score context. The resolve-inaccessible-contexts principle is applicable to any inaccessible
score context. For instance, a harmonic CSP may explicitly represent analytical harmonic
information (e.g. chords denoted by roman numerals) and constrain this harmonic infor-
mation as well as the actual note pitches (which depend on the harmonic information).
For such CSPs, it is usually suitable to distribute the harmonic information rst before
distributing the actual note pitches.
Resolving Multiple Contexts in Order
The preceding subsection addressed how a constrained but inaccessible score context can
be resolved at an early stage of the search process. Doing so facilitates constraint prop-
agation. Such a variable ordering approach can be generalised for multiple constrained
but inaccessible score contexts. During the search process, these contexts are then deter-
mined in a certain order. For example, such a variable ordering can be applied to solve a
harmonic CSP which explicitly represents analytic harmonic information and constrains
this harmonic structure, the actual note pitches, and the rhythmical structure at the
same time. A distribution strategy for a such a CSP may rst determine the temporal
parameters of the chords and notes to resolve contexts which depend on the rhythmi-
cal structure. Then it may determine all chord parameters and nally the actual note
pitches.
10
The eciency of such a distribution strategy can be improved. The class membership of parameter
objects does not change during the search process. Therefore, it is sucient to compute a static
variable ordering which the distribution strategy only executes without further checks during the
search process. Such an improvement requires a minor change of the high-level interface denition
MyScoreDistributor (fn. 9) for the denition of static variable orderings. The implementation of this
adapted denition is omitted here for brevity: basically, the line calling the function Order is omitted
and the distributor expects an ordered list of parameter objects.
149
7. How Strasheela Finds a Solution
While this variable ordering can be suitable for harmonic CSPs, other CSPs require their
own ordering. This depends on the CSP. Therefore, a generic means is desirable to easily
dene variable orderings suitable for a given CSP.
The function determineInOrder denes such a generic means. It expects a specication
stating the parameters aected and the order in which they are determined. This spec-
ication is given as a list of unary boolean functions the order of the test functions
species the order in which the corresponding parameters are determined. For example,
Fig. 7.11 denes the variable ordering for the harmonic CSP proposed above. This or-
dering rst determines all temporal parameters, then all chord parameters, and nally
all note pitches.
determineInOrder([isTemporalParameter,
isChordParameter,
isPitch])
Figure 7.11.: A variable ordering for harmonic CSPs which rst determines the temporal
structure of the score, then the harmonic structure and nally the actual pitches
The function determineInOrder is dened in Fig. 7.12. The function expects a list of
test functions and returns a variable ordering function which compares two parameter
objects and decides accordingly. For its decision, determineInOrder makes use of the
function getTestIndex. This function expects a list and a boolean function test and
returns the position of the rst element for which this function test returns true.
11
The
variable ordering function created by determineInOrder calls getTestIndex with both its
parameters and decides on the parameter for which the smaller index is returned (in
case getTestIndex returns equal values for two parameters, the strategy opts for the rst
parameter).
12
determineInOrder(tests)
def
=
let /* Append a default test function which always returns true. */
allTests
def
= append(tests, [f : f(x)
def
= true])
in g : g(param
1
, param
2
)
def
=
getTestIndex(param
1
, allTests) getTestIndex(param
2
, allTests)
Figure 7.12.: The function determineInOrder returns a variable ordering function which
determines parameters in the order specied by a list of test functions
11
The function function getTestIndex is dened in Fig. B.5.
12
Again, distribution strategies which only exploit score information which does not change during the
search process are implemented more eciently by a static variable ordering which only performs the
ordering once before the search starts.
150
7.3. Specialising the Constraint Model for Music
Combining the Principles Resolve-Inaccessible-Contexts and First-Fail
The variable orderings discussed above distribute selected parameters (i.e. parameters
for which some test function such as isTemporalParameter returns true) in an arbitrary
order. For example, the variable ordering which rst determines the temporal structure
(Fig. 7.10) does not express any preference as to which temporal parameter to distribute
in case there are multiple undetermined temporal parameters (or which non-temporal
parameter to distribute in case all remaining undetermined parameters are non-temporal
parameters).
For many CSPs, a more suitable strategy combines the resolve-inaccessible-contexts prin-
ciple with the general rst-fail principle. For example, instead of determining all temporal
parameters (or non-temporal parameters) in no particular order, some temporal param-
eter with smallest domain size is distributed rst. Alternatively, some parameter on
which most constraints are applied is chosen. Figure 7.13 expresses a variable ordering
which rst tests whether one of two parameters is a temporal parameter, and then de-
cides for the temporal parameter. If either both or none of the parameters is a temporal
parameter, then the parameter with smaller domain size is distributed.
13
The function
isParamWithSmallerDomainSize is dened in Fig. 7.9.
preferTemporalParamOrParamWithSmallerDomainSize(param
1
, param
2
)
def
=
let b
def
= isTemporalParameter(param
1
)
in if xor(b, isTemporalParameter(param
2
))
then b
else isParamWithSmallerDomainSize(param
1
, param
2
)
Figure 7.13.: Variable ordering specication which in general determines all temporal
parameters before the non-temporal parameters, but determines a temporal (or non-
temporal) parameters with smallest domain size rst.
The rst-fail principle can be applied in a similar way when multiple contexts are resolved
in a certain order. In case multiple parameter objects are of the same rank, some
parameter with the smallest domain (or most applied constraints) is distributed rst.
7.3.2.3. The Left-to-Right Variable Ordering
For many musical CSPs, a suitable distribution strategy determines parameter objects
from left to right. Such an approach distributes parameters in order of the start times
of the items (events or temporal containers) these parameters belong to. The resulting
13
Again, the eciency of such a distribution strategy can be improved. The parameters could rst be
sorted into temporal parameters and non-temporal parameters before the search starts. During search,
the distribution rst processes temporal parameters to nd the parameter with the smallest domain
size. After all temporal parameters are determined, then the search process processes non-temporal
parameters.
151
7. How Strasheela Finds a Solution
variable ordering is very similar to the variable ordering applied by score-PMC (Sec.
7.3.1.1, [Laurson, 1996]).
14
However, score-PMC estimates a static variable ordering
before the search starts and therefore requires that the rhythmical structure of the music
is fully determined in the problem denition. A distribution strategy, on the other hand,
allows for a dynamic variable ordering. A distribution strategy can be dened in such a
way that it proceeds from left to right whether the rhythmical structure of the music
is determined or not.
Figure 7.14 denes a left-to-right dynamic variable ordering. The main criteria for the
variable ordering are the start times of the temporal items to which the parameter
objects in question belong. The function isLeftmostParam rst accesses the parameters
associated start times and checks whether these start times are already determined (i.e
whether getDomainSize returns 1) and then bases its decision mainly on the values of
these start times. In case only one of these two start times is already determined, the
respective parameter is preferred (else-clause of outer if-expression).
15
In case both start
times are determined, isLeftmostParam opts for the parameter to which the smaller
start time belongs (else-clause of inner if-expression). Finally, in case both start times
are equal then isLeftmostParam prefers a temporal parameter. This condition causes a
relatively early determination of temporal parameters.
isLeftmostParam(param
1
, param
2
)
def
=
let start
1
def
= getStartTime(getItem(param
1
))
start
2
def
= getStartTime(getItem(param
2
))
isStart
1
Bound
def
= (getDomainSize(start
1
) = 1)
in if isStart
1
Bound (getDomainSize(start
2
) = 1)
then if start
1
= start
2
then /* If the items of both parameters start at the same time, then prefer
a temporal parameter (e.g. determining a duration may propagate and
other item start times get determined). */
isTemporalParameter(param
1
)
else /* Prefer the parameter whose item has a smaller start time */
start
1
start
2
else /* If only one parameter has a determine start time, then prefer that parameter.
*/
isStart
1
Bound
Figure 7.14.: A left-to-right dynamic variable ordering
14
The variable ordering of score-PMC takes not only the start time but also the duration of notes into
account. In case two notes share the same start time, then the longer note is visited rst.
The distribution strategy presented here ignores the note durations. Still, this strategy can be
extended accordingly.
15
In case no start time is determined, isLeftmostParam decides for param
2
. The distribution strategy
assumes that the start time of the leftmost item in the score is always determined before the search
starts. Otherwise, the search would start with an arbitrary parameter.
152
7.3. Specialising the Constraint Model for Music
The dynamic left-to-right variable ordering presented in this section turns out to be
highly suitable for a wide range of musical CSPs. In particular, it allows for solving
polyphonic problems which are very hard or even impossible to solve using existing
systems. In addition, this variable ordering is applicable for arbitrarily nested score
topologies and therefore is a good candidate for a default variable ordering for musical
CSPs in general. Section 8.2.2 will compare the eciency of this strategy with other
strategies.
153
7. How Strasheela Finds a Solution
154
8. Strasheela Examples
This chapter shows Strasheela in action with concrete musical examples. In these exam-
ples, the three main aspects of this research the music representation (Chap. 5), the
rule formalism (Chap. 6), and the search approach (Chap. 7) work together.
The purpose of this chapter is mainly to demonstrate Strasheelas usage on full examples.
These examples model relatively simple and highly conventional music theories. The
explanation of a conventional theory can be brief and refer to the literature for further
details. The implementation of this theory as a computational model, on the other hand,
can be discussed in full detail for a relatively simple theory. Nevertheless, this chapter is
not intended to appraise the expressive power of Strasheela by a few examples. Instead,
chapter 10 provides a more formal evaluation by comparing Strasheela and existing
systems using a few general criteria.
Strasheelas capabilities go far beyond what is shown in this chapter. In fact, Strasheela
users are free to implement their own rule-based music theory. Several further exam-
ples are provided with the Strasheela prototype. They can be found online at http:
//strasheela.sourceforge.net/strasheela/doc/StrasheelaExamples.html. These
examples are given in Oz code and can thus be executed (and also edited at will). For
example, Strasheela has been used to create music in extended just intonation where com-
positional rules control aspects of a microtonal harmonic progression such as a smooth
transition between chords (e.g. by constraining the consonance/dissonance-degree and
the root relations between neighbouring chords as well as their degree of familiarity with
respect to chords in common practice).
Strasheela also allows its user to combine the constraint-based approach with other
conventional techniques of algorithmic composition. For instance, Lindenmayer systems
(L-system) [Prusinkiewicz and Lindenmayer, 1990] are an established device in algorith-
mic composition used to create highly self-similar musical structures [Supper, 2001]. An
L-system has been applied to create the global musical form where dierent L-system
symbols are interpreted as dierent motifs. The local details of the score however in
particular the pitch structure and their resulting harmony are constrained by compo-
sitional rules.
Chapter Overview
Section 8.1 explains how Fuxian rst species counterpoint for two voices can be de-
ned and solved in Strasheela. The subsequent orid counterpoint example (Sec. 8.2)
constrains both the rhythmic and the pitch structure. This calls for a suitable score
155
8. Strasheela Examples
distribution strategy. Finally, Section 8.3 demonstrates how an important aspect of the
musical form the score topology can be constrained.
8.1. Fuxian First Species Counterpoint
The example presented in this section formalises the rst-species counterpoint for two
voices as described by Fux [1725, Chap. 1]. This example will be presented in full so as
to demonstrate how the dierent parts of Strasheela work together. At rst, the example
is stated in musical terms (Sec. 8.1.1). Section 8.1.2 formalises the example. Section
8.1.3 discusses the search process and presents a musical result. A conclusion (Sec. 8.1.4)
summarises this example.
8.1.1. The Music Theory
In rst species counterpoint for two voices, the task is writing a tting counter-melody
(the counterpoint) for a given melody (the cantus rmus).
First species counterpoint is homophonic. Note durations are irrelevant in the rst
species: notes of parallel voices always start and end together (i.e. all notes are of equal
length, usually all notes are semibreves). Also, both voices start and end together (i.e.
the cantus rmus and the counterpoint have the same number of notes).
Some rules restrict the melodic aspect of the counterpoint writing. Only specic melodic
intervals are allowed. These are the intervals up to the fourth, the fth, and the octave
(no note repetition is permitted here). All notes must be diatonic pitches (i.e. there can
be no augmented, diminished, or chromatic melodic intervals). The counterpoint remains
in a narrow pitch range. Melodic steps are preferred (this rule is not mentioned by Fux
[1725, Chap. 1], but it is very important for acceptable results and the Fuxian examples
follow this rule as well).
Furthermore, some rules restrict the relation between both voices. Open and hidden
parallels are forbidden, that is, direct motion in a perfect consonance is not allowed. Only
consonances are permitted as intervals between simultaneous notes and there should be
more imperfect than perfect consonances. The rst and last notes, however, must form
a perfect consonance. Finally, the counterpoint must be in the same mode as the cantus
rmus.
1
1
Some Fuxian rules are omitted here for brevity. The omitted rules are the following:
No melodic skips follow each other in the same direction.
Skips must be compensated for.
The last-but-one note pitch of the counterpoint must form a cadence where depending on the
mode the counterpoint is raised by a semitone. The last-but-one pitch is always the II degree for
the cantus rmus and the VII degree for the counterpoint. For example, in Dorian mode without
any accidentals the last counterpoint pitch is always c.
156
8.1. Fuxian First Species Counterpoint
8.1.2. The Formal Model
A musical CSP consists of a music representation containing the variables and the rules
which constrain these variables. Firstly, the music representation is shown (Sec. 8.1.2.1).
Section 8.1.2.2 then denes the rules one by one.
8.1.2.1. The Music Representation
The music representation models important aspects of the music theory described in Sec.
8.1.1. Only the actual rules are not modelled by the music representation itself.
The solution score contains two voices: the cantus rmus and the counterpoint. A voice
can be represented by a sequential container (Sec. 5.4.4) which contains the notes of the
voice. Two parallel voices can be represented by nesting their two sequential containers
in a simultaneous container. The resulting full score topology is thus as shown below.
makeScore(sim(items: [seq(items: [note, . . . ])
seq(items: [note, . . . ])]))
The cantus rmus is given, that is, this melody is fully determined. This example uses
the rst cantus rmus introduced by Fux which is shown in Fig. 8.1. Each note duration
is a semibreve, that is, each duration has the value 4 if the timeUnit is set to beats
(Sec. 5.4.1.1). When the pitches of this melody are measured in key-numbers (i.e. the
pitchUnit is set to midi ), these pitches are represented by the following sequence.
[62, 65, 64, 62, 67, 65, 69, 67, 65, 64, 62]

Figure 8.1.: The cantus rmus

Much information is available about the counterpoint. The number of notes in this
voice is the same as in the cantus rmus (i.e. 11). The duration of each note is also
determined to be a semibreve (i.e. 4). The only variables in this CSP are the pitches of
the counterpoint.
A tone can be repeated once at maximum (instead, the example shown below completely prohibits
repetitions).
The melody must not expose any tritone, even when this interval is reached stepwise (in the
example shown below, only the tritone between two neighbouring notes is prohibited).
From an interval larger than an octave contrary motion into an octave is not allowed.
157
8. Strasheela Examples
These pitches are restricted to a small range (here, an octave and a third). The actual
domain of each note pitch depends on whether the counterpoint is situated above or
below the cantus rmus. In the present example, the counterpoint is the upper part and
the pitch domain for each of its voices is set to 60, . . . , 76 (i.e. c4e5)
Figure 8.2 shows the top-level denition of the example. The function rstSpeciesCounterpoint
creates the music representation, applies all rules to this representation and returns the
solution myScore. The rules are dened in the following section.
let /* Auxiliary function to create a single voice, represented by a sequential container. makeVoice
expects a list of pitches (i.e. constrained variables) which are incorporated into note objects. */
makeVoice(pitches)
def
=
makeScore(seq(items: map(pitches,
f : f(Pitch)
def
=
note(duration: 4, /* semibreve (four beats) */
pitch: Pitch,
pitchUnit: midi ))))
in rstSpeciesCounterpoint()
def
=
let cantusFirmus
def
= makeVoice([62, 65, 64, 62, 67, 65, 69, 67, 65, 64, 62])
/* fdList returns a list of nite domain integers. The list length is set to 11 (the length
of the cantus rmus) where each integer has the domain {60, . . . , 76} */
counterpoint
def
= makeVoice(fdList(11, 60, . . . , 76))
myScore
def
= makeScore(sim(items: [counterpoint, cantusFirmus],
startTime: 0,
timeUnit: beats))
in /* Return myScore and apply a number of compositional rules to the counterpoint */
myScore where
restrictMelodicIntervals(counterpoint)
onlyDiatonicPitches(counterpoint)
noDirectMotionIntoPerfectConsonance(counterpoint)
onlyConsonances(counterpoint)
preferImperfectConsonances(counterpoint)
startAndEndWithPerfectConsonance(counterpoint)
Figure 8.2.: Top-level denition: rstSpeciesCounterpoint creates the music representa-
tion and applies all rules to it (the rules are dened in Sec. 8.1.2.2)
8.1.2.2. The Compositional Rules
The top-level denition rstSpeciesCounterpoint (Fig. 8.2) applies six rules to the coun-
terpoint. The rst two rules (restrictMelodicIntervals and onlyDiatonicPitches) imple-
ment the melodic restrictions specied in Sec. 8.1.1, the remaining rules (noDirect-
158
8.1. Fuxian First Species Counterpoint
MotionIntoPerfectConsonance, onlyConsonances, preferImperfectConsonances, and star-
tAndEndWithPerfectConsonance) implement the relation between both voices. The for-
malisation of these rules is explained in this subsection.
The rule restrictMelodicIntervals (Fig. 8.3) constrains the melodic intervals between all
neighbouring note pitch pairs of a given voice. The actual constraints are expressed by the
two auxiliary constraints restrictInterval and preferSteps. The rule restrictMelodicIntervals
merely applies these constraints to the intervals between the neighbouring note pitches
of a given voice (the auxiliary context accessor getInterval is dened later in Fig. 8.10
and map2Neighbours was introduced in Fig. 6.4).
let restrictInterval (Interval )
def
= Interval 1, . . . , 5, 7, 12
/* preferSteps constrains the average of intervals to be in {1.5, . . . , 3.0}, that is the average
interval is between 1
1
2
semitones and a minor third. */
preferSteps(intervals)
def
= let /* Encoded oating point arithmetic mean value (see below)
*/
Mean
def
= fdInt(15, . . . , 30)
in /* To represent a oating point value by a constrained vari-
able with integer domain, the arithmetic mean is multiplied
by 10. */
Mean = 10

intervals
length(intervals)
in restrictMelodicIntervals(myVoice)
def
=
let intervals
def
= map2Neighbours(getItems(myVoice), getInterval )
in

map(intervals, restrictInterval )
preferSteps(intervals)
Figure 8.3.: Only certain melodic intervals are allowed and small intervals are preferred
The constraint restrictInterval expresses that only specic melodic intervals are permit-
ted, namely every interval between a minor second and a fourth, as well as a fth and
an octave. Implicitly, restrictInterval forbids repetitions as the unison is not permitted.
The constraint preferSteps expresses a preference for smaller intervals: preferSteps has
the eect that a solution exhibits more steps than larger intervals. This function con-
strains the average interval (i.e. the arithmetic mean) to an interval between 1
1
2
semitones
and a minor third.
Figure 8.4 denes the second melodic rule which constrains all notes contained its ar-
gument myScore (i.e. the counterpoint in this example) to diatonic pitches. These are
the pitches of the Dorian mode starting at d (i.e. the white keys on the piano). This
rule is very brief, because it makes use of the constraint isDiatonicPitch, already dened
in Fig. 6.9. As a demonstration, the rule is dened more generally than required for
this specic example. The rule collects all notes contained either directly in the con-
tainer myScore or in some sub-containers. This allows the user to apply this rule to a
159
8. Strasheela Examples
voice which also contains explicit pause objects or to a polyphonic score consisting of a
hierarchy of containers (the function collect was introduced in Fig. 5.24).
onlyDiatonicPitches(myScore)
def
=
let pitches
def
= map(collect(myScore, test: isNote), getPitch)
in

map(pitches, isDiatonicPitch)
Figure 8.4.: All notes must be diatonic pitches (i.e. there can be no augmented, dimin-
ished, or chromatic melodic intervals)
The remaining rules constrain the relationship between both voices. The voice-leading
rule noDirectMotionIntoPerfectConsonance (Fig. 8.5) forbids open and hidden parallels.
This rule subsumes the four fundamental rules of Fux [1725] into a single rule: perfect
consonances must not be reached in direct motion. The last line of the rule expresses
this statement: precisely if the interval (Interval ) between two simultaneous notes is a
perfect consonance then the directions of the voices leading into this interval (Dir
1
and
Dir
2
) must dier. The lines before this statement dene these variables Interval , Dir
1
and Dir
2
. The function f is applied to every pair notePre and noteSucc of neighbouring
notes in the counterPoint. The simultaneous notes of these two notes (i.e. the notes
of the cantus rmus) are accessed by the function getSimNote (an auxiliary function
dened below in Fig. 8.9). The directions (Dir
1
and Dir
2
) of the melodic intervals are
constrained: the constraint direction encodes this direction by a nite domain integer
(the actual encoding is irrelevant here as the directions must dier only; the function
direction is dened in Fig. B.6). Finally, Interval is constrained to the absolute distance
between the pitches of noteSucc and its simultaneous note by the auxiliary function
getInterval .
The harmonic rule onlyConsonances (Fig. 8.6) constrains each interval between simulta-
neous notes to be consonant. The constraint isConsonance is dened in Fig. 8.11 below.
The rule preferImperfectConsonances expresses a preference for imperfect consonances.
More specically, the rule constrains the number of perfect consonances between si-
multaneous notes to less than half of the total number of voice notes. The constraint
isPerfectConsonance is dened in Fig. 8.12.
The rule startAndEndWithPerfectConsonance (Fig. 8.8) constrains the rst and the last
note of the counterpoint to form a perfect consonance with the simultaneous cantus
rmus note (i.e. either a prime, a fth or an octave). In case the counterpoint is below
the cantus rmus, however, their interval must form an octave. A fth is not possible as
it would change the mode of the cantus rmus [see Fux, 1725, p. 31].
The remainder of this section denes the auxiliary functions which have been used in
the rules. The auxiliary context accessor getSimNote (Fig. 8.9) returns the (single) note
which is simultaneous to myNote.
The function getInterval (Fig. 8.10) returns the constrained variable Interval and con-
160
8.1. Fuxian First Species Counterpoint
noDirectMotionIntoPerfectConsonance(counterPoint)
def
=

map2Neighbours(getItems(counterPoint),
f : f(notePre, noteSucc)
def
=
let Dir
1
Dir
2
Interval
in Dir
1
= direction(getPitch(notePre),
getPitch(noteSucc))
Dir
2
= direction(getPitch(getSimNote(notePre)),
getPitch(getSimNote(noteSucc)))
Interval = getInterval (noteSucc,
getSimNote(noteSucc))
isPerfectConsonance(Interval ) Dir
1
,= Dir
2
)
Figure 8.5.: Open and hidden parallels are forbidden: perfect consonances must not be
reached by both voices in the same direction
onlyConsonances(counterPoint)
def
=

mapItems(counterPoint,
f : f(note)
def
= isConsonance(getInterval (note, getSimNote(note))))
Figure 8.6.: The interval between every pair of simultaneous note pitches is consonant
preferImperfectConsonances(counterpoint)
def
=
let notes
def
= getItems(counterpoint)
simIntervals
def
= map(notes, f : f(note)
def
=
getInterval (note, getSimNote(note)))
NumberPerfectConsonances
in /* Summation of the truth values (0/1-variables) whether the intervals are perfect
consonances. */
NumberPerfectConsonances =

map(simIntervals, isPerfectConsonance)
NumberPerfectConsonances <
length(notes)
2
Figure 8.7.: Imperfect consonances are preferred over perfect consonances
161
8. Strasheela Examples
let isSuitableInterval (CounterpointPitch, CantusPitch)
def
=
(CounterpointPitch CantusPitch) 12, 0, 7, 12
in startAndEndWithPerfectConsonance(counterpoint)
def
=
let rstNote
def
= first(getItems(counterpoint))
lastNote
def
= last(getItems(counterpoint))
in isSuitableInterval (getPitch(rstNote),
getPitch(getSimNote(rstNote)))
isSuitableInterval (getPitch(lastNote),
getPitch(getSimNote(lastNote)))
Figure 8.8.: The rst and the last counterpoint note forms a perfect consonance with the
cantus rmus
getSimNote(myNote)
def
= rst(getSimultaneousNotes(myNote))
Figure 8.9.: Access the simultaneous note of a given note
strains Interval to the absolute distance between the pitches of two given notes.
2
getInterval (note
1
note
2
)
def
= let Interval
in Interval where
Interval = [getPitch(note
1
) getPitch(note
2
)[
Figure 8.10.: Return the interval between the pitches of two notes (a constrained variable)
The last two auxiliary functions dene the constraints isConsonance (Fig. 8.11) and
isPerfectConsonance (Fig. 8.12). Both denitions are very similar and constrain a given
Interval so that it is a member of a set of consonances (minor and major third, fth, . . . )
respectively perfect consonances (prime, fth, octave). Note that isConsonance omits the
prime, which is not allowed here as an interval between simultaneous notes.
2
This auxiliary function makes several rule denitions more clear but impairs eciency. This function
is called by several rules on the same note pairs and thus creates redundant constrained variables
and propagators.
A possible optimisation of this rule which retains its modularity uses memoization [Norvig, 1992].
Such an optimised version of getInterval maintains an internal table for two note combinations. In
case the interval between two notes was constrained before, the rule returns that interval. Otherwise
a new constrained variable is created, stored in the table and returned.
162
8.1. Fuxian First Species Counterpoint
isConsonance(Interval )
def
= Interval 3, 4, 7, 8, 9, 12, 15, 16
Figure 8.11.: Constrain Interval to a consonance
isPerfectConsonance(Interval )
def
= Interval 0, 7, 12
Figure 8.12.: Constrain Interval to a perfect consonance
8.1.3. Search Process and Results
This section discusses the search process for the example. Two dierent score distribution
strategies are applied and their performance is compared. Finally, an example result is
presented.
A distribution strategy consists of a variable ordering and a value ordering (see Sec.
7.3.2). In this particular CSP, all constraints can be applied directly: the music rep-
resentation contains enough information in the problem denition such that there are
no inaccessible constrained score contexts (Sec. 6.3).
3
Therefore, an established general
variable ordering such as rst-fail which distributes some variable with smallest do-
main is suitable (Sec. 7.3.2.1). The rst-fail variable ordering has been complemented
with the value ordering which selects the domain value that is closest to the arithmetical
means of the domain boundaries.
It turns out that constraint propagation carries out most of the work to solve this CSP
(i.e. to successfully determine all 11 pitches of the counterpoint). Figure 8.13 shows the
resulting search tree for nding the rst solution which consists of only 22 distributable
spaces and 16 failed spaces besides the solution (see Sec. 7.2.2 for the meaning of the
tree node shapes). The tree has a depth of 23. The search process requires about 50
milliseconds.
4
For comparison, the left-to-right variable ordering (Sec. 7.3.2.3) which is particularly
suited to solving polyphonic CSPs has been applied as well (with the same value
ordering, that is, using getMediumDomainValue). For this CSP, the rst-fail and the
left-to-right variable orderings are comparable in eciency (in great contrast to the next
example). The left-to-right distribution also makes 22 decisions until it nds the rst
solution (the shape of the search tree is dierent) and also requires about 50 milliseconds.
One solution of this CSP is presented in Fig. 8.14. For this solution, a randomised value
3
In fact, the rule noDirectMotionIntoPerfectConsonance (Fig. 8.5) does constrain a context inaccessible
in the problem denition (only during search does it become known which simultaneous notes form
a perfect consonance). Still, this rule does not call for a special distribution strategy.
4
All measurements are the rounded average of 10 runs and were conducted on a Pentium 4, 3.2 GHz
machine with 512 MB RAM (Fujitsu Siemens Scenic P320 i915G), running Linux with kernel 2.6.12
(Fedora Core 3) and Mozart 1.3.1.
163
8. Strasheela Examples
Figure 8.13.: The rst-fail distribution strategy results in a relatively small search tree to
nd the rst solution for this example (i.e. to determine all 11 pitches of the counterpoint)
164
8.1. Fuxian First Species Counterpoint
ordering has been used. Still, the search process was comparable in eciency to the
gures reported above.

Figure 8.14.: One solution for the CSP (using random value ordering)
8.1.4. Conclusion
This section showed Strasheela in action on a full example. The example demonstrated
how the dierent aspects of Strasheelas design (its music representation, rule denition,
rule application and search process) work together.
The top-level denition rstSpeciesCounterpoint (Fig. 8.2) creates the music represen-
tation used in this example. This representation consists of a hierarchy of score objects
(containers and notes). The whole score is largely determined in the problem denition
only the counterpoint pitches are undetermined.
These pitches are constrained by six rules. The rules are dened in a modular way as
functions (Fig. 8.3 to 8.8). In this particular example, every rule is applied directly to
the counterpoint within the top-level denition rstSpeciesCounterpoint.
These rules constrain diverse score contexts (Sec. 2.4). Every score context constrained
in this example is listed here to show their diversity (together with the corresponding
rule).
A single note pitch (onlyDiatonicPitches, Fig. 8.4)
A pair of neighbouring note pitches (restrictMelodicIntervals, Fig. 8.3)
A pair of simultaneous note pitches (onlyConsonances, Fig. 8.6)
The set of all sets of simultaneous note pitches (preferImperfectConsonances, Fig.
8.7)
A more complex note pitch set consisting of neighbouring and simultaneous note
pitches (noDirectMotionIntoPerfectConsonance, Fig. 8.5)
In this example, the scope of each rule (Sec. 3.3.1.1) is dened within the rule (i.e. each
rule was only applied once and accessed the set of constrained score contexts within the
rule). Some rules apply their constraints uniformly to all instances of a particular score
context. For example, onlyConsonances applies a constraint to every pair of simultane-
ous note pitches. Other rules constrain only specic instances of a certain context. For
165
8. Strasheela Examples
instance, startAndEndWithPerfectConsonance constrains only the rst and last pair of
simultaneous note pitches.
A number of higher-order functions are also used to conveniently access score contexts
or to apply a rule to the score. As a clarication, these functions are listed here.
Rules are applied to every element of a list with map (e.g. in onlyDiatonicPitches,
Fig. 8.4)
Rules are applied to neighbouring elements in a list with map2Neighbours (e.g. in
noDirectMotionIntoPerfectConsonance, Fig. 8.5)
Complex score contexts are expressed by traversing the whole score hierarchy to
collect all objects which meet a certain condition: getSimNote (Fig. 8.9) is imple-
mented by getSimultaneousNotes, a variant of getSimultaneousItems (Fig. 5.31)
Rules are applied to every element of a list which meets a condition (implicit
ltering) with mapItems (e.g. in onlyDiatonicPitches, Fig. 8.4)
This section also discussed the search process for this example. It was shown that much
of the search process is conducted by constraint propagation alone. Two distribution
strategies were compared and revealed little dierence in performance for this example.
The subsequent example, however, will constrain the rhythmical and the pitch structure
of the music and thus constrain score contexts which are not accessible in the problem
denition. This subsequent example will demonstrate the enormous inuence of a suitable
distribution strategy on the size of the search space and thus the eciency of the search
process.
Comparison with Existing Systems
Strasheela has been designed for solving complex musical CSPs. Fuxian rst-species
counterpoint, however, was chosen as a rst example because it is familiar and relatively
simple. Although this section only reproduced a standard example, the Strasheela im-
plementation still demonstrates some of Strasheelas features which are important for
addressing more complex CSPs.
The Strasheela user creates the music representation explicitly (cf. Fig. 8.2) which results
in a high degree of exibility. For example, the user may reshape the score topology
(i.e. further structure the score by introducing additional containers) or replace the
predened note objects by user-dened objects. Existing music constraint systems such
as PWConstraints (Sec. 3.3.1) and Situation (Sec. 3.3.2) do not support such exibility.
In Strasheela, the rule application is transparent (i.e. the user can understand how the
rule application is working [Raymond, 2003]) and user-programmable. To simplify the
rule denitions for the present example, the rules are directly applied to the counterpoint
container and not to the whole score (cf. Fig. 8.2). PWConstraints and Situation support
166
8.1. Fuxian First Species Counterpoint
neither accessing parts of the music representation outside a rule nor direct rule applica-
tion. In the example above, each rule encapsulates its own rule application mechanism.
For example, the rule noDirectMotionIntoPerfectConsonance employs the higher order
function map2Neighbours (cf. Fig. 8.5). Each of these mechanisms can be dened and
thus freely changed by the user (e.g. Sec. 6.2.4 denes map2Neighbours). PWConstraints
and Situation, on the other hand, provide proprietary rule application mechanisms which
are limited with respect to transparency and are not user-programmable. Although these
systems support all score contexts required for this example (in Situation, it will be hard
to express the context constrained by the rule noDirectMotionIntoPerfectConsonance),
they will fail for score contexts which were not explicitly arranged for by their designers.
Chapter 10 will conduct a systematic and detailed comparison of Strasheela with existing
music constraint systems.
167
8. Strasheela Examples
8.2. Florid Counterpoint
This example demonstrates Strasheelas capabilities for polyphonic CSPs where both
the pitch structure as well as the rhythmical structure is constrained by rules. Previ-
ous systems either hardly support polyphonic CSPs at all or require that the temporal
structure is determined in the problem denition (cf. score-PMC, Sec. 3.3.1.2).
This example was designed to be relatively simple. Therefore, it compiles rules from var-
ious sources instead of following a specic author closely (as did the previous example).
For example, some rules are variants of Fuxian rules introduced before, but rhythmi-
cal rules were inspired by Motte [1981]. Accordingly, the result does also not imitate a
particular historical style (but neither does Fux, cf. Jeppesen [1930]).
8.2.1. The Music Theory
This example creates a two-voice counterpoint like the previous example in Sec. 8.1.
The music representations of both examples are hence very similar. The representation
consists of two parallel voices (voice
1
and voice
2
). These voices are represented by two
sequential containers nested in a simultaneous container as before. In this specic
example, voice
1
contains 17 notes and voice
2
15 notes.
The start time and end time of each voice is further restricted. voice
1
begins one bar
before voice
2
. This is expressed by setting the oset time of these two sequential con-
tainers (contained in a simultaneous container) to dierent values (the oset of voice
1
is
0, and the oset of voice
2
is a semibreve, i.e. 16 as the temporal unit is beats(4), see Sec.
5.4.1.1). Both voices end at the same time (the end time of both sequential containers
is unied).
In contrast to the previous example, all pitches and also all durations are searched for.
In this specic example, each note duration has the domain quaver, crotchet, minim.
The pitch domain for each note in voice
1
is set to f3, . . . , g4, the domain for the note
pitches of voice
2
is slightly greater (f3, . . . , c4).
The example denes rules for various aspects of the music. The example applies rhythmic
rules, melodic rules, harmonic rules, voice-leading rules and rules concerning the formal
structure.
Rhythmical rules
Each voice starts and ends with a minim note value.
Note durations may only change slowly across a voice: neighbouring note
values are either of equal length or dier by 50 percent at maximum (e.g. a
quaver can be followed by a crotchet, but not by a minim).
The last note of each voice must start with a full bar.
Melodic rules
168
8.2. Florid Counterpoint
Each note pitch is restricted to the diatonic pitches of the C-major scale.
The rst and last note of voice
1
must start and end with the root c.
The melodic intervals between neighbouring pitches in a voice are limited to
a minor third at maximum (i.e. the melodic intervals are more restricted here
than in the Fuxian example).
An important rule constrains melodic peaks: the maximum and minimum
pitch in a phrase occurs exactly once and it is not the rst or last note of the
phrase. In this example, a phrase is dened simply as half a melody. Finally,
the pitch maxima and minima of phrases must dier. This rule concerning the
melodic contour inspired by Schoenberg has great inuence on the musical
quality of the result (subjectively evaluated) but also on the combinatorial
complexity of the CSP.
Harmonic rules
Simultaneous notes must be consonant.
The only exception permitted here are passing tones, where note
1
is a passing
tone (i.e. the intervals between the note and its predecessor as well as between
the note and its successor are steps and both steps occur in the same direction)
where a simultaneous note
2
started before note
1
, and this note
2
is consonant
with the predecessor of note
1
. Because the rhythmical structure of the result
is undetermined in the problem denition, the context of simultaneous notes
cannot be accessed directly and is therefore constrained by logical connectives
(see Sec. 6.3.4).
Voice-leading rules
Open parallel fths and octaves are not allowed. However, hidden parallels
are unaected here in contrast to the Fuxian example.
Formal rules
Both voices form a canon at the fth: the rst n notes of both voices form
(transposed) equivalents. In the case here, n = 10.
8.2.2. Search Process and Results
Like the corresponding section for the Fuxian example, this section compares the perfor-
mance of dierent distribution strategies for the example and nally presents a musical
example of the output.
In contrast to the previous example, this example constrains a score context which cannot
be accessed in the problem denition. The problem denition does not provide enough
information to access simultaneous notes in the music representation. Nevertheless, the
169
8. Strasheela Examples
relation between simultaneous notes is constrained by the harmonic and voice-leading
rules.
A distribution strategy is therefore required which determines this score context at an
early stage to support propagation of the constraints of rules applied to this context
(Sec. 7.3.1.1). The left-to-right score distribution strategy (Sec. 7.3.2.3) was applied.
This distribution strategy found the rst solution in about 4 seconds (189 distributable
spaces, 175 failed spaces, search tree depth 47).
For comparison, a distribution strategy which rst fully determines the temporal struc-
ture and then searches for the pitches (Sec. 7.3.2.2) was applied as well. To determine the
temporal structure, the distribution only needs to determine the note duration values.
The note oset time values are already determined in the problem denition. Start time
and end time values are determined by propagation once the durations are known. This
distribution strategy did not nd any solution within an hour (i.e. not even within about
900 times more time than needed by the left-to-right strategy)! After that time period,
the search process was interrupted.
5
This distribution strategy nds solutions for the rhythmical structure which indeed fulls
all rhythmic rules but are in conict with rules concerning the pitch structure. However,
these conicts are detected very late which causes much redundant work.
An analysis of the CSP revealed that the complexity of the problem was greatly reduced
when the rule which demands unique melodic minima and maxima was left out. With
such a reduced CSP, a solution can be found with both distribution strategies in a
reasonable amount of time.
The left-to-right strategy clearly outperforms the other strategy on this reduced CSP as
well. To nd the rst solution for the reduced CSP, the left-to-right strategy required
about 1.7 seconds (92 distributable spaces, 70 failed spaces, search tree depth 53) and
the strategy which rst fully determines the temporal structure required about 14 sec-
onds (630 distributable spaces, 601 failed spaces, search tree depth 62). The left-to-right
strategy is thus almost ten times faster than the other strategy for this simplied exam-
ple.
These gures demonstrate the great importance of a suitable search strategy for a musical
CSP. This requirement is particularly pressing for complex problems which constrain
inaccessible score contexts.
The gures indicate the suitability of the left-to-right distribution strategy for polyphonic
CSPs. The importance of user-denable distribution strategies is underlined by these
gures: dierent CSPs require dierent distribution strategies and a system designer
cannot foresee every CSP dened by a user.
5
For this CSP, the rst-fail distribution strategy (Sec. 7.3.2.1) is equivalent to a distribution strategy
which rst fully determines temporal structure. In this CSP, the duration parameters have the smallest
domain (only three duration domain values vs. at least 14 pitch domain values) and are therefore
determined rst by rst fail.
170
8.3. Constraining the Shape of the Temporal Hierarchy
Figure 8.15 shows one solution for this CSP. voice
1
is depicted as the lower voice. Again, a
randomised value ordering has been used (whose search process was comparably ecient
to the mid-domain-value ordering used for the measurements).

Figure 8.15.: One solution of the orid counterpoint CSP (using random value ordering)
In conclusion, it should be mentioned again that previous systems such as PWConstraints
and Situation do not support musical CSPs like the one presented in this section due to
their computational complexity (cf. Sec. 4.1.3). The left-to-right distribution strategy,
proposed by the present research, solves problems like this in a few seconds and thus in
a reasonable amount of time for practical use.
8.3. Constraining the Shape of the Temporal Hierarchy
8.3.1. Motivation
The temporal hierarchy (Sec. 5.4.4) expresses fundamental aspects of the musical form.
For instance, in the counterpoint example of Sec. 8.1 the temporal hierarchy species
the number of notes per voice and the number of voices in the score.
The user may want to somehow constrain this hierarchic structure as well. For instance,
it may be required that the number of notes in a voice depends on other factors in the
CSP such that it is only found during search. For example, a CSP may constrain so that
the rst and the last note of a voice starts with a new measure (cf. Sec. 8.2). The CSP
may apply additional rhythmic rules which are in conict with this condition and the
xed number of notes given to the voice (i.e. the CSP would require that either a few
notes are added or removed to t the melody into the measure boundaries).
In non-deterministic logic programming (i.e. the programming model of pure Prolog
[Bratko, 2001]), a variable can reference a composite data structure which can contain
any other data including other variables. For instance, a non-deterministic version of
the list predicate append(Xs, Ys, Zs) constrains three list variables (which can contain
any data) and implicitly constraints the length of each list. Using this programming
model makes it easy to constrain the shape of an hierarchic data structure. However,
this model also easily leads to highly inecient programs because this model does not
support constraint propagation.
Constraint propagation greatly improves eciency, but gives rise to domain specic
variables (Sec. 7.2.3.1). These variables cannot reference a composite data structure with
arbitrary data. Instead, the relation between such variables is controlled by constraints
171
8. Strasheela Examples
which are dened for specic domains (e.g. a propagator expressing that some nite
domain integer is a member of some nite set). The shape of a score consisting of nested
class instances (i.e. score objects) cannot be constrained directly using propagation and
domain specic variables: the constraint system (variables plus propagators) for the
domain sequence of class instances would be missing.
8.3.2. Approach
The present example proposes an alternative approach to constrain the shape of a score
hierarchy which is based on domain specic variables and propagation. This approach
is consistent with the Strasheela model as introduced in the Chap. 5 to Chap. 7. For
this purpose, the example labels score objects with an additional existence attribute
represented by a variable. When the variable has a certain value (e.g. 0), the whole score
object is regarded as non-existent. Such a non-existent score object is only contained
in the internal score, but omitted when the score is transformed to any output format.
For temporal items which form the temporal hierarchy an extra existence attribute
is not required. Instead, the duration parameter can also be interpreted as an existence
attribute. This example proposes that any temporal item is non-existent if its duration
equals 0 (regardless of the temporal unit of measurement). The constraint isExisting (Fig.
8.16) encapsulates this concept.
isExisting(myItem)
def
= getDuration(myItem) ,= 0
Figure 8.16.: Constraint to express whether a temporal item exists
The notion of a non-existent item is not limited to single events with the duration 0. A
temporal container with duration 0 is also considered non-existent. If the duration of a
temporal container equals 0, all its contained items are also constrained to this duration
(Sec. 5.4.4.1), which means the contained items are considered non-existent as well. In
an extreme case, even the top-level container can be of duration 0, meaning that the
whole score tree is non-existent or empty.
8.3.3. Elimination of Symmetries
By introducing the notion of non-existent items, multiple solutions become equivalent.
For example, the only relevant parameter of a non-existent item is its duration. All
other parameters of such an item (such as a notes pitch) are irrelevant, and multiple
solutions which dier only in the pitch of a non-existent note are equivalent. Similarly,
in a container with multiple items only the number of non-existent items matters, but
their position is irrelevant.
172
8.3. Constraining the Shape of the Temporal Hierarchy
Multiple equivalent solutions are called symmetries [Schulte and Smolka, 2004]. Elim-
inating such symmetries can drastically reduce the search space. In the following, two
constraints are dened to eliminate the two symmetries mentioned.
In case the constraint isExisting (Fig. 8.16) returns false for some item, then all param-
eters of this item must be determined to some xed value (e.g. to their median domain
value). That way, there is only a single solution for this item. The only exceptions are
the parameters startTime and endTime which do not introduce any symmetries: once
the osetTime of an item is determined and its duration equals 0, then propagation
deduces startTime
myItem
= endTime
myItem
and both parameters only depend on the
temporal position of other items in the score. For instance, if the end time of some item
in a sequential container becomes determined and the oset time of the following item
is determined as well, then the start time of this following item becomes determined
by propagation alone. Figure 8.17 denes this rst symmetry-elimination constraint:
eliminateSymmetries
1
must be applied to all temporal items in the CSP. The constraint
determineVariables determines all parameter values of myItem (except for its start time
and end time).
eliminateSymmetries
1
(myItem)
def
=
isExisting(myItem) determineVariables(myItem)
Figure 8.17.: Eliminate symmetries I: determine all parameters of a non-existent tem-
poral item
To eliminate the symmetries caused by the position of non-existent items in a container,
the next constraint restricts all non-existent items to the end of the item sequence
stored in a container. That way, there is only a single solution for non-existent items in
a container. Figure 8.18 denes this constraint. eliminateSymmetries
2
must be applied
to all temporal containers in the CSP (the function map2Neighbours was dened in Fig.
6.4, mapItems was introduced in Fig. 5.23).
let zerosOnlyAtEnd(xs)
def
= map2Neighbours(xs, f : f(X, Y )
def
= X = 0 Y = 0)
in eliminateSymmetries
2
(myContainer)
def
=
zerosOnlyAtEnd(mapItems(myContainer, getDuration))
Figure 8.18.: Eliminate symmetries II: non-existent temporal items occur only at the
end in a container
The eect of the constraint eliminateSymmetries
2
also allows for the denition of a rule
relevantLength which constrains the number of items in a temporal container (Fig. 8.19).
The number of existing items in a container is the position of the last existing item (i.e.
the item before an item with duration value 0). The denition uses the function mapInd,
173
8. Strasheela Examples
which is a variant of the function map: madInd applies a binary function with the index
of a list element as its rst argument.
let aux(xs, N)
def
=

mapInd(xs, f : f(i, X)
def
= X > 0 N i)
N length(xs)
in relevantLength(myContainer, N)
def
=
aux(mapItems(myContainer, getDuration), N)
Figure 8.19.: The rule relevantLength constrains the number of existing items in a
myContainer to N
8.3.4. Discussion
The approach proposed here the interpretation of the duration parameter as an ex-
istence attribute allows the user to constrain the shape of the temporal hierarchy.
This approach makes the denition of new musical CSPs possible. For example, the user
can constrain the rhythmical structure in a contrapuntal CSP as detailed above. The
user can also encapsulate a musical segment such as a motif or a transitional sequence
without specifying its exact length (e.g. encapsulated by a function which creates the
segment and applies some constraints to it). An example of such a segment is a sequen-
tial container with notes where the intervals between the notes are constrained to form
a sequence (cf. Fig. 8.20) without exactly specifying the actual number or the actual
pitches of the notes. This container could then be inserted in a score wherever such a
phrase is required (e.g. as a transition) and additional constraints could further restrict
the number of notes, for example, to t into some metric structure and the pitches, e.g.,
to t into some harmony.

Figure 8.20.: The required length of a sequence (here falling thirds) may depend on other
constraints (e.g. the metrical structure)
Nevertheless, the generality of the proposed approach to constrain the temporal hierarchy
is restricted. The actual shape of the temporal hierarchy cannot dramatically change
when comparing dierent solutions, subtrees can only either exist or not exist. Because
the actual score topology formed by temporal items can only change within certain
bounds, also the musical texture (e.g. whether a solution is homophonic or polyphonic)
cannot change much. In other words, constraints on the musical texture are limited.
A simultaneous container (Sec. 5.4.4) can be used like an event-list where contained items
can freely overlap temporally thanks to the oset parameter of all its contained items.
This allows the user to express various complex temporal structures (e.g. piano music,
174
8.3. Constraining the Shape of the Temporal Hierarchy
where the textures is continuously changing) within a single simultaneous container. In
this case however, the access of score contexts (e.g. melody vs. accompaniment) requires
additional work (e.g. by marking of all members of a context by special info-tags).
Constraining the temporal hierarchy makes the whole CSP more complex. The de-
nition of every rule must take non-existing events into account. For example, a rule
constraining the last note of a voice (e.g. the rule constraining the last interval to a per-
fect consonance in counterpoint example in Sec. 8.1) cannot be applied directly, because
the score context last note of the voice is inaccessible in the problem denition (Sec.
6.3).
175
8. Strasheela Examples
176
9. The Strasheela Prototype
This chapter briey introduces a software prototype of the Strasheela model proposed by
this text and states the dierences between this prototype and the model. The prototype
can be downloaded at [Anders, 2006].
Section 9.1 surveys how the prototype extends the model described in this thesis both
in general (e.g. by providing a number of output formats including MIDI, Csound, and
Lilypond) as well as for specic CSPs (e.g. for harmonic or motivic CSPs). Section 9.2
discusses the user interface programming language of the prototype, how this language
relates to the mathematical notation used in this thesis, and the requirements of alter-
native languages for later implementations. Section 9.3 explains the limitations of the
prototype.
9.1. Relation to the Model Presented
The present research provides the software Strasheela, a full implementation of the
generic music constraint system presented in this text. The prototype implementation
supports every concept detailed in the preceding chapters (e.g. the full music represen-
tation, the rule formalism, the space-based constrained model and its customisation for
musical CSPs). In addition, the prototype exceeds the model described in this text in a
number of ways.
9.1.1. Input and Output
The prototype features the ability to export its internal music representation to other
representation formats by score interface functions. Particularly interesting exchange
options include music notation and sound synthesis software formats as well as the
format of other computer-aided composition environments. Strasheela presently exports
its internal music representation to the music notation software Lilypond [Nienhuys and
Nieuwenhuizen, 2003], MIDI les, score les of the sound synthesis languages Csound
[Boulanger, 2000], and Common Lisp Music (CLM) [Schottstaedt], as well as to the
music representation of the composition system Common Music [Taube, 2004]. Also, the
textual Strasheela music representation (Sec. 5.4.3.2) can be exported for archival and
editing purposes and imported again. Further formats, as well as support for the import
from various formats into Strasheelas internal format, can be added.
177
9. The Strasheela Prototype
The prototype usually runs on its own, but it can also be integrated into existing com-
position systems. The prototype runs either in an Emacs-based interactive development
environment (IDE) [Kornstaedt and Duchier, 2004] or as a server in the background
which is fed code via Internet sockets. For example, the Strasheela server can be started
and fully remote controlled by a Common Lisp system presently the lingua franca for
computer-aided composition such as Common Music [Taube, 1997], PWGL [Laurson
and Kuuskankare, 2002], or OpenMusic [Assayag et al., 1997].
9.1.2. Extended Strasheela Core Music Representation
The prototype extends the music representation discussed in this text. For instance,
the prototype extends the interface of score objects. Some methods introduced in this
text are more general in the prototype, and many additional methods are provided.
For example, in the prototype the interface for an instance of the class note consists of
more than 100 methods. The present text introduced only the essence of this interface.
Several methods in the prototype are more expressive than what was presented above.
For instance, several higher-order methods such as map, collect or lter (Sec. 5.5.2.2)
support additional arguments which allow the user to specify to which depth the score
hierarchy is traversed and whether a subtree or the whole graph is traversed.
9.1.3. Extensions for Specic CSP Classes
The design of the Strasheela music representation core aims to be style-neutral. For
instance, virtually any music requires the representation of time and of temporal relations
between objects in the score. Therefore, the Strasheela representation core supports these
aspects (the Strasheela model introduced so far is called Strasheela core in this chapter).
Independent extensions, on the other hand, augment the representation core by vari-
ous style-specic representation details which facilitate the denition of specic musical
CSPs. For example, the representation core features only a style-neutral pitch represen-
tation which consists of a single value. In the harmony-model extension (Sec. 9.1.3.2),
this pitch representation is augmented into a composite representation consisting of pitch
class, octave and accidental.
Strasheela has been designed for solving complex musical CSPs. Complex CSPs are de-
ned more easily in a modular way, for example, by using relatively generic extensions
such as those introduced in this section. Existing music constraint systems such as PW-
Constraints (Sec. 3.3.1) and Situation (Sec. 3.3.2), on the other hand, do not support
user-extensions of their music representation (such as the harmony model or the motif
model presented below). The following paragraphs outline some extensions provided by
the Strasheela prototype.
178
9.1. Relation to the Model Presented
9.1.3.1. Pattern Rules
Patterns play an important role in music composition. Therefore, composition systems
such as Common Music [Taube, 2004] dene a rich set of patterns. Also, patterns play an
important role in music analysis. For instance, Conklin and Witten [1995] propose a for-
mal model for patterns in various explicit and derived musical features (e.g. the sequence
of note pitches in a melody, the intervals between these pitches, and the directions of
these intervals).
Situation introduces the notion of pattern rules (constraints). Example pattern rules
include a voice prole constraining the number of consecutive upward and downward
movements, or a rule constraining the melodic intervals of a voice (not) to use a specied
set of intervals. As these patterns are enforced by constraints, they can be combined
with other constraints. Also PWConstraints proposes a technique which shows some
similarities with pattern rules. PWConstraints users can dene a contour by an envelope
(break point function) and constrain a pitch succession to approximately follow this
envelope.
The Strasheela prototype proposes a simple and generic pattern formalism which allows
users to easily dene pattern constraints. A pattern rule is formally a procedure (a
function according to the rule formalism of this thesis, Chap. 6) which constrains a
sequence of variables. A pattern rule is applied by accessing any sequence of variables
from the CSP and applying the constraint to this sequence. For example, a pattern rule
can be applied to the list of the note pitches of some voice, the intervals between these
pitches, or the roots of a harmonic progression.
About fty pattern rules are already predened in the prototype. Examples include
generalisations of common binary or ternary constraints for sequences (e.g. min, max,
increasing, decreasing), constraints inspired by Common Music patterns (e.g. cycle,
palindrome, markovChain), constraints on the contour, and constraints on sequences of
boolean variables constrained by reied constraints (e.g. oneTrue, someTrue, allTrue).
9.1.3.2. Constrainable Harmony Model
The Strasheela prototype provides a harmony model. This model renes the pitch rep-
resentation of Strasheelas core representation. The rened pitch representation intro-
duces notions such a pitch class, octave, degree, and accidental and denes their relations
by constraints (e.g. the constraint pitchClassToPitch(PitchClass, Octave, Pitch)). This
pitch representation is highly generic. It supports the pitch class concept of set theory
[Forte, 1973], the enharmonic spelling of western tonal music, and also microtonal music
(e.g. pitch classes and accidentals can be specied in cents instead of semitones).
The model implements the harmony concepts interval, scale, and chord by music rep-
resentation classes, and extends Strasheelas standard note class. These classes employ
a rened pitch representation. For example, a note features the additional parameters
pitch class, octave, degree, and accidental (besides the pitch parameter) and implicitly
179
9. The Strasheela Prototype
constrains the relation between these parameters. A chord instance features parameters
such as root (whose value is a pitch class) and pitchClasses (whose value is a pitch class
set). The music representation of a harmonic CSP contains instances of these new classes
(often together with other score objects such as temporal containers) and restricts their
relation by a set of constraints.
The model predenes constructs to greatly simplify the denition of typical harmonic
CSPs. The model supports the specication of implicit constraints on the intervals,
scales and chords by databases. For example, the user may specify that all chords must
be major or minor triads, but that these triads can be transposed freely. Usually, the
pitch class of each note must be a member of the pitch class set of its simultaneous chord
except the note meets special conditions (e.g. the note is a passing note). The model
predenes this rule the user only passes the exceptions (e.g. passing note, auxiliary
note) as reied rules on a single note.
Naturally, arbitrary further constraints can be applied by the user. For example, the
user may demand that neighbouring chords must have common pitch classes, or that the
dissonance degree across a progression of several chords should gradually increase.
9.1.3.3. Constrainable Motif Model
Motifs and gestures play an important role in music composition. In many musical styles,
motif relations are a vital device for constructing compositional form. So far, there has
been little research into how to represent and constrain motifs for music composition.
Sandred [2003] introduces the idea of domains of fully pre-composed rhythmic motifs
for OMRC and allows the user to apply further constraints on the music (e.g. a motif
must t into a bar structure without syncopation). OMRC itself is limited to purely
rhythmical CSPs, however, similar techniques can be used in PMC for other parameters.
The Strasheela prototype proposes a highly generic model to represent and constrain
motifs. This model distinguishes unrelated motifs (in music analysis often denoted by
letters such as a vs. b, [Schoenberg, 1979]) as well as variations of a single motif (e.g. a
1
vs. a
2
). The user freely constrains both dimensions.
A motif is represented by a new container subclass (containing arbitrary items, e.g., a
sequence of notes) with two additional parameters motifIndex and motifConstraint
whose values are nite domain integers. The rst parameter controls the motif identity
whereas the second parameter controls the variation.
The user denes the set of possible motif identities and variations in two databases.
The parameter motifIndex is an index into the motif identity database and the param-
eter motifConstraint is an index into the motif variation database. The motif model
applies implicit constraints to each motif instance which are based on its two parameters
(motifIndex and motifConstraint) as well as the content of the two databases (motif
identity database and motif variation database).
180
9.1. Relation to the Model Presented
On the one hand, each motif identity database entry denes arbitrary motif features.
For example, the motif database entry in Fig. 9.1 denes absolute note durations and
the pitch contour of the motif (i.e. a sequence of pitch interval directions between notes
of the motif). A motif database with this entry will usually dene further entries which
specify dierent note duration sequences and pitch contours. Specifying the two aspects
note durations and pitch contour is only an example, other databases will specify other
motif features.
motif (durations : [2, 2, 2, 8] pitchContour : [=, =, ])
Figure 9.1.: A possible declaration of the rst motif from Beethovens 5th Symphony
On the other hand, each motif variation database entry (a motif constraint) is a rst-
class procedure which arbitrarily constrains the motif instance using its motif identity
features. For instance, in case the motif identity database species note durations and
the pitch contour (see above), then the entries of the motif variation database use these
specications when applying constraints to a motif instance. For instance, the rst motif
variation database entry may constrain the motifs note durations and note pitches to
follow the sequence of durations and the pitch contour specied in the motif identity
database entry which will be chosen by the search process. The second entry of the
motif variation database may instead constrain the motif pitches to follow the inverse
of the pitch contour dened by this database entry. The motif identity database entry
and the motif variation database entry that is nally used for a specic motif instance
is only decided during the search process.
The set of solutions of a single motif thus depends on three user-controlled dimensions:
(a) the set of entries in the motif identity database, (b) the set of entries in the motif
variation database, and (c) the ambiguity implicit in the motif constraints in the variation
database. For example, constraining only the pitch contour allows many motif variants
with the same motif index and the same motif constraint.
Each of these dimensions can be further constrained. For instance, in a succession of
several motifs the motif index and motif constraint may be constrained to follow some
pattern. The pitches of the motif notes constrained to follows some motif contour may
additionally be constrained to follow some harmonic progression (Sec. 9.1.3.2).
However, this motif model is even more general. Entries in the motif identity database
may contain undetermined variables. For instance, the user may constrain a set of motif
score instances to be the same motif (e.g. all motif indices are equal), but the actual
shape of the motif may be undetermined in the CSP denition and may depend, for
instance, on contrapuntal motif combinations (e.g. the actual shape of a fugue subject
and counter-subject may only be found during search).
Furthermore, the number of items in a score motif instance may be constrained (Sec.
8.3). For instance, dierent entries in the motif database maybe of dierent length.
181
9. The Strasheela Prototype
To model free, non-motivic sections between motivic sections, a specic motif constraint
may not apply any constraints to the motif instance at all. A decision for this motif
constraint then means a decision for a non-motivic section.
1
The formalism proposed here may also serve to dene higher-level formal relations.
Whereas a plain motif contains notes, a higher-level motif contains motifs. A higher-
level motif constrains the sub-motifs in the same way a basic motif constrains notes by
deciding upon the parameters motif index and motif constraint and thus deciding for an
entry in the motif database and an entry in the motif constraint database. For example,
the user may want to constrain the sequence of motifs contained in a higher-level motif
by applying a pattern rule (Sec. 9.1.3.1) to sub-motif indices, or by applying a pattern
rule to the sequence of the maximum pitches of each motif.
9.2. Programming Language Issues
Strasheela makes use of a programming language as its user interface. Whereas the
present text uses mathematical notation for this purpose, the prototype employs the
Oz programming language [Smolka, 1995; van Roy and Haridi, 2004] as user interface
language (and as implementation language as well).
Oz has been chosen because it meets the requirements of Strasheela very well. Stra-
sheela applies a multi-paradigm programming approach which is founded on three pro-
gramming paradigms: the space-based constraint model, functional programming, and
object-oriented programming (Chap. 5 7). Only a few languages oer this combination
of programming paradigms. The Oz language natively supports all these paradigms in a
coherent whole.
There are a number of dierences between the mathematical notation used in this text
and Oz notation. Firstly, mathematical notation has solely declarative semantics which
serves its purpose as a means to communicate the main ideas of the Strasheela design. Oz,
on the other hand, also has clear operational semantics, which is essential for an actual
software implementation. Therefore, Oz notation is often more specic. For example, the
equality operator of mathematical notation (=) has a number of dierent counterparts
in Oz notation: the boolean predicate (==), unication (=), the equality constraint (=:),
and stateful binding (:=).
In addition, the Oz programming language supports a number of concepts for which no
counterpart was introduced in the mathematical notation of this thesis. For example, this
text introduced no mathematical notation for rst-class procedures, concurrent threads,
nor classes or methods. Still, all these concepts play an important role in the implemen-
tation and usage of Strasheela. For example, Chap. 6 introduced compositional rules as
(sequential) rst-class functions. Instead, rules in the prototype are actually expressed
by (possibly concurrent) rst-class procedures (cf. Sec. 7.2.3.4).
1
To eliminate symmetries (i.e. dierent solutions which are equivalent), this non-motivic motif con-
straint should determine the motif index as well.
182
9.3. Limitations of the Prototype
Oz notation has its own syntax which clearly diers from mathematical notation. For
example, a function denition in Oz diers clearly from the notation used in this text.
Also, similar syntactic constructs often dier in small details. For instance, the notation
of a list is similar in both notations, but Oz omits the commas (e.g. [1, 2, 3] vs. [1 2 3]).
Although Oz constitutes a highly suitable language for implementing the Strasheela
model, another language could have been chosen. In principle, the Strasheela model can
be reproduced in any language which supports the programming paradigms required by
its design (i.e. support for the space-based constraint model, functional programming
and object-oriented programming). For instance, Strasheela can be implemented in
a language which supports functional programming and object-oriented programming
plus an interface to C++. This interface would then allow the use of the Gecode library
[Gecode authors, 2006], which implements the space-based constraint model (interfaces
to Gecode are in progress for a number of programming languages including Oz [Avispa
Research Group, 2006] and Common Lisp
2
). For example, such an approach makes it
possible to implement Strasheela in Common Lisp.
9.3. Limitations of the Prototype
This section discusses limitations of the prototype with respect to its generality. These
shortcomings are discussed in the present chapter, because they are shortcomings of the
current implementation and not of Strasheelas actual design.
Strasheela also has one principle limitation which restricts its generality compared with
existing music constraint systems: Strasheela features domain specic variables (Sec.
7.2.3.1) instead of variables with universal domain. The advantages and disadvantages
of this approach are explained in Sec. 10.1.2
Hard-Wired Variable Domains in Implicit Constraints
Section 6.2.8 explained that specic Strasheela objects apply constraints implicitly for
convenience when an object is created. The only implicitly applied constraints of the
Strasheela core are the temporal constraints aecting the temporal items (Sec. 5.4.4.1).
The Strasheela extensions presented in the present chapter the harmony model and
the motif model apply further constraints implicitly.
In the present prototype implementation, implicit constraints are hard-wired to specic
variable domains. For instance, all implicit temporal constraints are hard-wired to nite
domain integers.
In principle, the Strasheela user can freely control the domain for every variable in the
music representation. For instance, a numeric variable may be either a nite domain in-
teger or a real number from the XRI constraint system [Avispa Research Group, 2005].
2
Personal communication with Camilo Rueda, 26 July 2006 at IRCAM, Paris.
183
9. The Strasheela Prototype
Variables to which constraints are applied implicitly, however, are restricted to the do-
main of their implicit constraints. This means that temporal parameters are limited to
nite domain integers.
The consequences of this limitation depend on the parameters. In case of the tempo-
ral parameters, for example, this limitation makes complex rhythms (e.g. nested tu-
plets) harder to realise (although setting the temporal parameter attribute timeUnit to
beats(N) where N is reasonably large can still make complex nested tuplets possible,
Sec. 5.4.1.1). Moreover, the osetTime of temporal items cannot be negative (e.g. items
in a sequential container cannot overlap).
The limitation could be addressed by refactoring the implicit constraints. The refactoring
would introduce an abstraction layer which substitutes the domain specic constraints
by more general constraints. Which domain specic constraint is called by such a more
general constraint would be depend either on a global user setting or on the domain of
the variable arguments supplied to the more general constraint.
Random Variable Ordering without Recomputation
Strasheela predenes a number of distribution strategy (Sec. 7.2.3.2) building blocks,
including a random value ordering. For musical applications, such a value ordering is
highly important.
The present implementation of this value ordering, however, is indeterministic (i.e. re-
peating a distribution during the search process does not necessarily result in the same
decision). This implementation violates a fundamental requirement of a distribution
strategy [Schulte, 2002]. Searching using pure copying does work, but recomputation (a
technique for saving memory which is important for solving complex CSPs, Sec. 7.2.3.5)
is not possible with this value ordering.
The limitation could be addressed by making the value ordering deterministic even if it
appears to be random. For example, random numbers could be stored inside the search
script with an unambiguous mapping between a node in the search tree and a specic
random number. Alternatively, the limitation could be addressed by using the constraint
library Gecode (see above). Gecode supports batch recomputation which records all
decisions of the distribution strategy (Sec. 7.2.3.5).
Memory Consumption
Strasheelas music representation can store a considerable amount of information. How-
ever, this data requires space (i.e. RAM). Only a small part of this data actually changes
during the search process. The search process only determines the values of the variables
in the representation (the value of parameters such as note durations and pitches) all
other data is static.
The space-based constraint model uses copying (cloning of spaces) to make the search
process programmable. In the current implementation, the whole music representation is
copied during the search process, although only a small part of this data only changes. For
184
9.3. Limitations of the Prototype
complex CSPs with deep search trees, this results in considerable memory requirements.
Recomputation makes it still possible to solve arbitrarily complex problems.
A better solution would be to avoid wasting memory in the rst place. In an improved
version of the Strasheela prototype, the music representation would be stored outside
the computational space and only the constrained variables would be stored within the
space and would be copied. An abstraction layer would provide for an unambiguous and
bidirectional mapping between the parameters in the music representation outside and
the variables inside the computational space. This mapping between the music represen-
tation and the variables would make it possible to utilise all information provided by the
representation and the variables together in order (i) to apply constraints to the vari-
ables and (ii) to conduct the score distribution strategies presented in Sec. 7.3.2. Such
an approach would drastically reduce the required amount of memory and also slightly
improve runtime eciency by reducing the amount of copying and garbage collection.
185
9. The Strasheela Prototype
186
10. Comparison and Evaluation
Section 4.2 stated the goal of the present research: this research aims at creating a
system which facilitates the denition of musical CSPs (when compared with a general
constraint system) and at the same time is highly generic (i.e. allows for the denition of
a large set of musical CSPs). A most generic music constraint system would support the
implementation of any music theory model conceivable, but such a system would possibly
be highly inecient. For usability, the present research aims at creating a system which
is highly generic but still reasonably ecient.
This chapter evaluates the present research. The chapter asks (i) whether Strasheela in-
deed facilitates the denition of musical CSPs, (ii) whether it is generic and (iii) whether
is it reasonably ecient.
The evaluation of the rst aspect of this goal can be brief. Experience with existing
generic music constraint systems shows that such systems indeed considerably facilitate
the denition of musical CSPs. This experience is conrmed by the present research. For
instance, the denition of the presented counterpoint examples (Chap. 8) in a general
constraint system without any special support for musical CSPs would amount to repro-
ducing considerable parts of Strasheela in such systems. These examples require a music
representation which expresses the temporal structure as well as the pitch structure of
polyphonic music and which allows the application of compositional rules to a large set
of score contexts.
The generality of Strasheela is evaluated by comparing its degree of generality with the
generality of existing systems. Only the most important generic music constraint systems
are taken into account in this comparison: this chapter compares PMC, score-PMC, Sit-
uation, and the combination of MusES and BackTalk with Strasheela in terms of their
generality. All these systems have been discussed before, but from a dierent perspective.
Section 3.3 introduced the fundamental idea of each system. Section 4.1 analysed fun-
damental limitations of these systems with respect to generality before Strasheela was
introduced. In contrast, this chapter evaluates the design of Strasheela by comparing it
with existing systems. Further systems are left out of the discussion for brevity. Section
3.3.3 already showed that OMClouds is less generic than PWConstraints and Situa-
tion. Also the other systems presented in Sec. 3.3.4 are either by order of magnitudes
less ecient (CHARM and Arno), mainly theoretical (PiCO), or much less developed
(OMBT).
The eciency of Strasheela is evaluated by examining Strasheelas eciency for the
examples shown previously and by comparing how dierent systems allow the user to
control the search process. Nevertheless, this text does not compare the performance of
187
10. Comparison and Evaluation
existing music constraint systems with Strasheela. The main reason is the claim of the
present research that Strasheela allows its user to solve CSPs which are hard or even
impossible to solve with existing systems hence, a comparison is not possible for such
CSPs. In addition, there exist no standard musical benchmark CSPs so far.
Chapter Overview
This chapter discusses the three important design aspects of a music constraint system
in turn. Section 10.1 compares the music representation of the dierent systems and
examines what information can be represented and constrained in each system. The
rule formalism and in particular the rule application mechanisms of dierent systems
are investigated in Sec. 10.2. Section 10.3 compares how the search can be adopted to
specic musical CSPs in these systems. Section 10.4 then asks how Strasheela could be
further generalised. A brief conclusion summarises this chapter (Sec. 10.5).
10.1. Music Representation
It can be argued that research into music representation and research into music con-
straint systems have only marginally inuenced each other so far. On the one hand,
few genuine music representations incorporate the notion of variables to represent par-
tial information and to express relations between unknowns. Examples for such repre-
sentations include Music Structures [Balaban, 1996] and the Prolog implementation of
CHARM [Harris et al., 1991]. These representations, however, introduce variables mainly
as a theoretical device, for example, to express that some information can be missing
[Balaban, 1996]. The authors do not state actual CSPs which require search for solving.
Music constraint systems, on the other hand, have made little use so far of the rich
means developed by music representation research to represent information about mu-
sic. For example, music representation research identied hierarchic representations as an
important means to express score information (cf. Sec. 2.4). However, the following sys-
tems represent music only by a sequence of objects: PMC (Sec. 3.3.1.1, [Laurson, 1996]),
Situation (Sec. 3.3.2, [Rueda et al., 1998]), and OMClouds (Sec. 3.3.3, [Truchet et al.,
2003]). Similarly, music representation research identied the data abstraction concept
as an expressive means, for instance, of allowing for complex derived score information
(cf. Sec. 2.6). Still, only a few music constraint systems provide any data abstraction
layer.
The present research proposes an integration of the notion of constrained variables
(solved by search) and several established music representation principles. Doing so re-
sults in both a more expressive general music representation as well as a more expressive
music constraint system (which makes use of this representation).
188
10.1. Music Representation
10.1.1. Representation Format
The denition of a musical CSP makes use of various pieces of musical information. There
is the information which expresses the actual solution (e.g. a number of notes with their
parameters start time, duration and pitch), and there is the information required to
express the score contexts aected by a compositional rule (cf. Chap. 4). The designer of
a generic music constraint system cannot foresee the CSPs to which the system will be
applied. It is therefore not possible to foresee which information and in particular which
score contexts will be required from the music representation. For a generic system, it
is therefore crucial that the user is able to dene which information is represented and
which score contexts can be accessed. This section compares Strasheela to existing music
constraint systems concerning these requirements.
10.1.1.1. Strasheela
Strasheela supports several basic principles to store score information. These principles
were explained in Sec. 5.6.1 and are only briey summarised here. Explicitly stored
pieces of information are the type of each score object, its attributes, the nesting of score
objects, and the order of score objects at the same nesting level. Derived information can
be deduced by utilising any piece of explicit information or of other derived information
about score objects. Any score object provides access to all information in the score due
to bidirectional links between all objects. Additional principles make the representation
more convenient to use and extend: Strasheela score objects are abstract data types which
are dened incrementally by class inheritance. The textual representation facilitates the
specication of score topologies.
10.1.1.2. Comparison
The fundamental principles of explicitly stated and derived musical information (see
above) are also supported by other music constraint systems, but these systems support
only a subset of these principles or support them in a less general way. For example,
existing systems store less explicit information and limit access to derived information.
As a rst overview, Tab. 10.1 compares how dierent systems support the principles
of explicitly stated information. Explicitly represented nesting and order are subsumed
in the score topology column. The table distinguishes between user dened and user
denable, where user dened means that there is no default setting for this principle
(e.g. Situation does not predene any score object attributes) and user denable means
that the user can extend what a system predenes (e.g. the MusES user can extend the
predened attributes of a note). Each system is discussed in more detail below.
Table 10.2 continues this overview for derived information. The table shows which score
contexts dierent systems can access in order to deduce information from these contexts.
Please note that this table implicitly shows which score contexts can be constrained. The
189
10. Comparison and Evaluation
Attributes Score Topology Type
PMC none at sequence n/a
score-PMC xed tree (xed nesting) xed
Situation user dened at sequence n/a
MusES and BackTalk user denable one or more temporal
sequences
a
user denable,
hierarchical
Strasheela user denable user dened
b
user denable,
hierarchical
a
Each container (TemporalCollection) stores temporal objects (e.g. notes, chords) sorted by their start
time and duration, i.e. no explicit non-temporal order is represented.
b
Multiple topologies supported (e.g. nested event lists, tree of generic temporal containers, acyclic graph
of arbitrary containers, see Sec. 5.4.3.3).
Table 10.1.: The explicitly represented information supported by the music representa-
tion of various music constraint systems. The information represented by
variables is not taken into account. See Sec. 10.1.2 below.
supported derived information is also explained in more detail below in the discussion
of each system.
Derived Information Accessible About
PMC positionally related variables (only via rule application
mechanism)
score-PMC xed set of predened contexts
Situation (i) attributes of a single object,
a
(ii) positionally related
objects (only via rule application mechanism)
MusES and BackTalk any set of temporally related score objects
Strasheela any set of score objects
a
The attributes of Situation objects include explicitly represented distances between attributes (see
Sec. 3.3.2.1).
Table 10.2.: The derived information supported by the music representation of various
music constraint systems
PMC
The PWConstraints subsystem PMC constitutes primarily a general constraint program-
ming language and does not provide much support for the special needs of musical CSPs.
The music representation of PMC consists of a at sequence of variables and supports
neither score objects, attributes, nor hierarchic nesting of objects. However, the domain
of a variable can consist of arbitrary values including composite data. This feature
allows for a hierarchically nested score section as a solution for a single variable (Sec.
10.1.2 discusses this aspect in more detail).
190
10.1. Music Representation
An object in the music representation (i.e. a variable in the sequence of variables) only
knows its own value and has no further information about the partial solution. For
example, a variable does not know its position in the sequence nor the values of its
neighbours. Only the rule application mechanism has access to the list of all already-
determined variables and allows the user to access the position of the current variable
and its predecessors.
Consequently, derived information can only deduce information from positionally related
variables which are already determined. For example, it is possible to access the current
variable and its predecessor (e.g. in order to constrain the distance between these vari-
ables). However, no information about variables after the current variable (i.e. about
undetermined variables) is accessible.
score-PMC
In contrast to the more general PMC, the seminal PWConstraints subsystem score-
PMC (Sec. 3.3.1.2, [Laurson, 1996]) specically addresses musical CSPs, in particular
polyphonic CSPs. The system introduces special compound score data types (e.g. note,
beat, harmonic slice) which form a hierarchically nested score. Thus, score-PMC supports
all four principles of explicitly represented information (object attributes, hierarchic
nesting, order, and type information).
However, the user can hardly adapt this representation. Score data types have a xed
set of attributes, the user cannot introduce new types, and the score topology is xed.
The user is limited to minor adaptations such as specifying the size of the score (e.g. the
number of notes in a part). The system score-PMC supports more derived information
than most other music constraint systems, yet the set of supported score contexts is also
xed and cannot be extended by the user.
Situation
Situation constitutes a highly generic music constraint system. When compared with
score-PMC, the representation of Situation (Sec. 3.3.2.1) is kept more abstract for gen-
erality. Whereas score-PMC xes its music representation by modelling distinct musical
concepts (e.g. note, beat, measure, part), the Situation user denes the attributes for a
Situation object and freely interprets what an object and its attributes stand for (e.g.
an event with a set of parameters or a chord with a number of pitches).
Situation facilitates the denition of a music representation by a template-like speci-
cation. The solver (which implicitly creates the music representation) features several
arguments by which the user controls the music representation creation: for example,
the user species the total number of objects, the number of attributes for each object
(specied as points and distances, see Sec. 3.3.2.1), the domain for each attribute, etc.
However, Situations music representation limits what information can be stored in the
score. For example, the score topology of Situation is limited to a sequence of objects
which feature a number of attributes. However, Situation does not support hierarchic
191
10. Comparison and Evaluation
nesting (e.g. note x is contained voice alto). Also, a Situation object lacks explicit type
information. It is therefore hard to dene CSPs which relate objects of dierent types
where each type features its own set of attributes and possibly even implicit constraints
(e.g. a note object and a voice object type, where the note type features the attributes
start time, duration and pitch, and a voice object type features the attributes start time,
duration, no attribute pitch, but implicitly constrains its contained notes to follow each
other sequentially in time).
The Situation user can interpret the meaning of objects and their attributes at will, but
Situation does not assist any interpretation with specic data abstractions (i.e. there are
no accessors such as getPitch or getDuration). Moreover, Situation does not arrange for
user-dened data abstractions to ease the denition of complex CSPs (e.g. predened
Situation constraints cannot be used within user-dened data abstractions).
Moreover, Situation objects only know their attributes. For instance, positional infor-
mation is not accessible in music representation. This information is only provided for
Situations rule application mechanism (cf. PMC above).
Finally, Situation provides only limited means to access derived information. For exam-
ple, Situations rule application mechanism for user-dened constraints provides means
to access already determined score objects which allows the user to freely derive informa-
tion from their attribute values. Like in PMC, however, no information on undetermined
objects is accessible. Also, Situation does not predene any accessors for specic score
contexts. For example, Situation lacks the polyphonic contexts accessible in score-PMC.
In Situation, only positionally related score contexts (e.g. neighbouring object pairs in
the sequence) are supported by Situations rule application mechanism (Sec. 3.3.2.2 and
Sec. 6.2.5).
MusES and BackTalk
The music representation MusES together with the constraint solver BackTalk forms
a highly expressive music constraint system (Sec. 3.3.4). MusES supports most of the
previously mentioned four principles of explicitly represented information. MusES score
objects feature a type, a set of attributes, and allow objects to be nested (e.g. temporal
objects such as notes can be contained in a temporal collection). Moreover, MusES is also
highly extendable. MusES consists of a set of Smalltalk classes and that way implements
a set of incrementally dened abstract data types which allows for easy extension by the
user.
Some representational aspects are more convenient in MusES than in Strasheela: MusES
inherited the reective capabilities of its implementation language Smalltalk. For exam-
ple, for any given object instance the system provides access to the set of its attributes or
to its superclass (for security reasons, Oz requires the programmer to implement access
to such information explicitly). MusES also features a particularly convenient notation
of some concepts: MusES represents many musical concepts by special classes in place
of instances of more generic classes. For example, MusES features a special class for
each pitch class and for each interval. For instance, MusES represents the pitch class c
192
10.1. Music Representation
as well as the interval augmented fourth by their own classes with clear class names.
Together with the Smalltalk syntax, this design results in a highly legible pitch class
algebra notation.
Yet, MusES is less expressive than Strasheelas music representation in two respects.
MusES supports hierarchic nesting only in a limited way and its pitch representation is
restricted to western tonal music.
The limitation concerning hierarchic nesting is related to MusES excellent support for
temporal information. Every temporal object in MusES (e.g. a playable note or chord)
stores its start time and duration. Temporal objects can be grouped in a temporal
collection (a container). A temporal collection ensures that its contained objects are
always sorted according to their start time and duration [Pachet et al., 1996]. This
design allows for an easy and ecient access to, for example, all objects in a temporal
collection within a certain time span. Based on this ability, MusES denes an exhaustive
set of temporal relations between temporal objects.
However, the fact that temporal collections implicitly sort their contained objects re-
stricts the expressiveness of these collections. Firstly, temporal collections cannot be
nested in MusES (it would disturb the automatic temporal ordering). Secondly, tempo-
ral collections always order their contained objects temporally. However, not all ordering
information required in a score is of a temporal nature (e.g. the order of parts in a score
is non-temporal).
Pachet [1994] motivates the design of MusES by comparing it with MODE (i.e. with
MODEs music representation, called SmOKe [Pope, 1992]). SmOKe supports nested
event lists (a SmOKe event list is also an event, see Sec. 2.4.3) and the events in an
event list can be arranged in a non-temporal order. However, a SmOKe event does
not provide access to its start time: this information is only known by its event list.
This representation scheme is sucient for the sound synthesis purposes of SmOKe, but
proved unsatisfactory for the analytical purposes of MusES which required access to
more information. This led to the dierent design of MusES.
Strasheelas music representation combines the benets of both SmOKe and MusES.
Like SmOKe, Strasheela supports hierarchic nesting and any ordering of score objects
(including a non-temporal order). Beyond this, Strasheela provides access to any infor-
mation available within a score object (e.g. the start time of an event). This augmented
expressivity is obtained by two design principles: bidirectional links and constraint prop-
agation. Strasheela allows for hierarchic nesting of containers (SmOKes score topology
is only a special case of the topologies supported by Strasheela). However, the hierar-
chically nested score objects in Strasheela are bidirectionally linked: every container can
access all information stored in its contained items and vice versa. Furthermore, pa-
rameters exchange knowledge about their values (which are constrained variables), due
to constraint propagation. For instance, information about temporal parameters such as
start-time, duration and end-time is propagated between temporal containers and events
(Sec. 5.4.4).
193
10. Comparison and Evaluation
Based on this design, MusES exhaustive set of temporal relations can be reproduced in
Strasheela (either as plain accessors or as constraints), as was demonstrated for the con-
straint isSimultaneous (Fig. 5.32) and the constraint getSimultaneousItems (Fig. 5.31).
Their implementation can be less ecient in Strasheela than in MusES, but it is still
suciently fast. For example, accessing specic score contexts may require a traversal of
the full score, but usually a score context must be accessed only once (e.g. objects are
accessed once by ltering to apply a constraint, but the search process does not need to
lter these objects again).
An important contribution of MusES is its support of enharmonic spelling [Pachet, 1993].
MusES denes an algebra of pitch classes (e.g. the pitch class c raised by an augmented
fourth results in f which diers from g). However, this spelling is hard-coded into
system and alternative pitch representations are not supported. For example, MusES
cannot represent music which does not t into western enharmonic spelling such as
music in the gamelan pelog scale or music using Partchs 43-tone scale [Partch, 1974].
The Strasheela user, however, can choose a pitch representation according to their needs.
The core music representation of Strasheela introduces a single-value pitch representation
which can be measured in various units such as key-number, cent, or Hz (Sec. 5.4.1.1). It
can thus express conventional western music as well as music using alternative tunings,
although this core representation does not support enharmonic spelling.
This single-valued representation can be extended by the user. For example, the harmony
model dened by the Strasheela prototype complements this representation by composite
pitch representations (see Sec. 9.1.3.2). This model denes a pitch representation which
consists of a degree, an accidental and an octave and which thus allows for the expression
of enharmonic spelling. Moreover, this extended representation is not limited to the
western set of pitch classes. For instance, in this representation a degree can denote a
pitch class measured in cents. The multiple components of the pitch representations (e.g.
the single-valued pitch of Strasheelas core representation as well as the degree, accidental
and octave) are represented by individual constrained variables, the relationship between
them is maintained by constraints, and the user can apply further constraints to each of
the components independently.
10.1.2. Constrainable Aspects
System Comparison
Music constraint systems dier not only in their music representation format, but also in
which aspects of this representation can be constrained. In this regard, two factors must
be taken into account. Systems dier in (i) what values a variable domain can consist
of and (ii) where a variable can occur in the music representation. For example, each
score-PMC variable is (i) an integer and (ii) denotes a pitch (a key-number).
The systems PMC and MusES plus Backtalk support variables with universal domains,
that is, their domain can consist of arbitrary values. The domain of a PMC variable
194
10.1. Music Representation
consists of arbitrary Lisp values and likewise the domain of a Backtalk variable consists
of arbitrary Smalltalk objects.
Situation and Strasheela, on the other hand, support constraint propagation and there-
fore CSPs in these systems usually only make use of variable domains which are sup-
ported by their constraint propagation algorithms. The actual domains supported de-
pend on the system. Situation supports numbers (e.g. a mix of integers, oats and ratios)
whereas Strasheela supports, for instance, nite domain integers and nite sets of integers
(cf. Sec. 7.2.3.1).
Situation and Strasheela both support variables with universal domains as well. Situa-
tion comes with an example where variable domains consist of chord objects (i.e. CLOS
instances). Similarly, the Strasheela prototype provides a proof-of-concept implementa-
tion of a constraint system with universal domain variables.
1
This example is dened in
terms of the space-based constraint model and supports the features of this model (e.g.
a user-dened distribution strategy is supported) except propagation. This exception
highlights the fundamental drawback of the universal domain. Constraint propagation
greatly reduces the size of the search space and thus improves eciency enormously
(Sec. 7.2.3.1). Universal domain variables, however, cannot benet from constraint prop-
agation, because the propagation algorithms have been optimised for specic domains.
The respective advantages and disadvantages of universal domains and specic variable
domains are discussed further below.
The variable domains supported by a system and its music representation both inuence
where a variable can occur in this representation. Naturally, a variable can only substitute
a value which corresponds to its domain and the occurrence of a variable is restricted by
the format of the music representation. For instance, a Situation variable is a number
(when constraint propagation is used) and numbers only occur as an attribute (a point or
a distance) of an object within Situations sequence of objects. Consequently, in Situation
a variable can only occur as an attribute of such an object. Table 10.3 lists the respective
variable domains supported by the dierent systems and where variables can occur in
the music representation.
Consequences of Specic Variable Domains
Strasheela (i.e. Oz) is restricted to specic variable domains in order to support con-
straint propagation (see above). This restriction to specic variable domains is the only
aspect in which Strasheela is less general than any other generic music constraint system.
The restriction is caused by a decision for performance instead of generality. Nowadays,
virtually every major general constraint system (e.g. SICStus [SICS], ECLiPSe [Cheadle
et al., 2003], GNU Prolog [Diaz, 2003], ILOG Solver [ILOG], Gecode [Gecode authors,
2006]) supports constraint propagation for an ecient search, and consequently supports
constraints on variables with specic domains. In the space-based constraint model, the
set of natively supported variable domains is not xed. A number of domains have been
1
The example can be found in the test directory of the Strasheela release available from https://
sourceforge.net/projects/strasheela/.
195
10. Comparison and Evaluation
Variable Domain Variable Occurrence
PMC any value element in sequence
score-PMC integer note pitch (key-number)
Situation number
a
attribute of object in sequence
MusES and BackTalk any value anywhere
Strasheela specic domains
b
any value suiting the variables
domain
c
a
A domain consists, for example, of integers, oats or ratios (possibly mixed). Alternatively, Situation
supports variables of universal domains. However, choosing this option simplies the music represen-
tation into a at list and disables constraint propagation.
b
In principle, universal domains are possible. However, constraint propagation is only supported for
specic domains (e.g. nite domain integers, and nite sets). The set of supported domains can be
extended.
c
For example, constrained variables can occur as attributes of score objects or locally in rules.
Table 10.3.: Constrainable music representation aspects in various music constraint sys-
tems
developed by the Oz community (see Sec. 7.2.3.1) and development is still ongoing. All
domains available in Oz are also available in the Strasheela prototype.
It turns out that the restriction to specic variable domains narrows expressivity less
than it might rst appear. The consequences of this restriction are best understood by
distinguishing between two cases. Firstly, a single variable whose domain consists of
composite values can often be translated into a composite value which contains multiple
variables. For example, a single list variable expressing a list of n integers can be sub-
stituted by n integer variables grouped in a list. Secondly, there exist genuine variable
domains which cannot be directly substituted. Such a genuine domain often corresponds
to some common mathematical domain such as the integer, real, or set domain. For ex-
ample, a constraint on real numbers (e.g. a logarithm constraint) cannot be adequately
substituted by a constraint on integer variables. Likewise, a variable for a set (or list) of
integers whose cardinality (or length) is undetermined and constrained in the problem
denition cannot be substituted by a set (or list) containing multiple integer variables.
There are variable domains which can be substituted by multiple variables of other
domains. This case occurs relatively often in music constraint programming. For exam-
ple, OMRC [Sandred, 2000a] expresses domains of rhythmic motifs by variables whose
domain consist of lists of integers. Likewise, Pachet and Roy [1995] advocate the use vari-
ables whose domains consist of composite Smalltalk objects (e.g. note and chord objects)
and criticise the harmonisation system by Ballesta [1994] for using multiple variables to
express the multiple features of a single note or interval.
Substituting a composite variable with multiple variables not only preserves the same
semantics, but can result in a manifestly more ecient search. Finding a value for a
variable whose domain consists of instances of composite data boils down to a nave
196
10.2. Rule Formalism
generate-and-test search for this variable. In contrast, splitting this composite variable
into multiple variables allows the search process to proceed incrementally and that way
inconsistent solution candidates are excluded earlier (this advantage is similar to the
advantage of backtracking over generate-and-test). Furthermore, multiple variables allow
for constraint propagation between these variables. For example, the all-interval series
CSP is solved more eciently when the series is represented by a sequence of multiple
variables instead of a single variable whose domain consists of fully determined sequence
solution candidates. Also the poor performance of Ballestas system is not necessarily
caused by too many variables and constrains (as criticised by Pachet, see above), but
primarily by an inadequate variable ordering (e.g. harmonic decisions on the chord level
should be made rst, after constraint propagation has reduced the domain for chord
variables, cf. Sec. 7.3.1).
A genuine variable domain cannot be directly substituted by variables of other domains.
Still, even such genuine domains can often be encoded by variables of other domains.
For instance, the example presented in Sec. 8.3 eectively constrains the lengths of lists
of items in a container without resorting to a variable with a list domain. Instead, this
example introduced the notion of an existence attribute (represented by the duration
of a temporal item). Similarly, Dooms et al. [2005a] propose the implementation of a
graph domain by boolean variables (0/1-variables) plus logical connective and summation
constraints.
10.2. Rule Formalism
A musical CSP consists of variables and constraints on these variables like any CSP. The
meaning of the variables is dened by the music representation (e.g. a certain variable
represents the pitch of the rst note of a certain voice). Constraints on variables express
compositional rules applied to the music representation.
Constraints are applied to variable sets which constitute various score contexts. Section
3.3.1.2 discussed this notion for score-PMC. A constraint (or rule) is either applied
directly to a score context (cf. Fig. 6.2) or by some convenient application mechanism
which implicitly accesses the score contexts and applies the constraint to it (e.g. the
pattern-matching based mechanism of PWConstraints, Sec. 6.2.6).
Direct application requires direct access to the music representation and its variables.
This is provided, for example, by the combination of MusES and BackTalk. For instance,
with MusES and BackTalk a rule may be applied by an iteration whose running index
is used to access neighbouring notes in a melody as follows (cf. [Roy and Pachet, 1997])
myConstraint(nth(myMelody, i ), nth(myMelody, i + 1))
Systems which do not allow direct access to the music representation and its variables
(e.g. PWConstraints, Situation, OMClouds) require special rule application mechanisms.
197
10. Comparison and Evaluation
Dierent systems oer dierent application mechanisms. For example, Situation oers an
index-based mechanism (Sec. 3.3.2.2) and PWConstraints a matching-based mechanism
(Sec. 3.3.1.1).
Strasheela provides both direct constraint application as well as convenient rule appli-
cation mechanisms. The fact that a Strasheela rule is a rst-class function
2
forms the
foundation for its rule formalism. This allows the user to program complex rule applica-
tion mechanisms as higher-order functions (Sec. 6.2).
Arbitrary control structures can be dened in terms of higher-order functions, including
the rule application mechanisms of previous systems. As a demonstration, the index-
based rule application mechanisms of Situation and the pattern-matching based mecha-
nism of PWConstraints have been reproduced in Strasheela (Sec. 6.2.5 and Sec. 6.2.6).
On the other hand, rst-class functions can also express rule application mechanisms
which cannot be adequately reproduced by the mechanisms of Situation and PWCon-
straints. The mechanisms of both systems are highly suited to applying a rule to el-
ements in a sequence, because these mechanisms rely on positional relations (such as
neighbouring elements). First-class functions, on the other hand, can process arbitrary
data structures (e.g. trees or graphs, besides sequences). For example, a rule can be ap-
plied to all score objects in a hierarchic score representation for which some test function
returns true (Sec. 6.2.7).
Hence, a rule application mechanism based on the notion of higher-order functions is
more generic than the mechanisms of Situation and PWConstraints.
Experience with Strasheela shows that a CSP which makes use of suitable higher-order
functions is often distinctly more concise than an equivalent CSP which applies con-
straints directly (e.g. by explicit nested loops). Higher-order functions can abstract away
complex control structures which implement mathematical concepts (e.g. the Cartesian
product, Fig. 6.16) or knowledge about the music representation (e.g. implicit traversing
of the score hierarchy, Sec. 6.2.7).
Because Strasheelas rule application mechanisms allow for a concise and expressive CSP
denition, other music constraint systems may be interested in adopting these mecha-
nisms. These rule application mechanisms can be easily reproduced in the combination
of MusES and BackTalk. This system is implemented in Smalltalk and Smalltalk pro-
vides code blocks, which are essentially rst-class functions. Realising Strasheelas rule
application mechanisms in systems which dont support direct access to their music rep-
resentation and its variables (i.e. Situation, PWConstraints and OMClouds) requires
internal changes to these systems.
2
A Strasheela rule is actually a rst-class procedure (Sec. 6.1 and Sec. 7.2.3.4).
198
10.3. Search Strategy
10.3. Search Strategy
The operational foundation of Strasheela is the space-based constraint programming
model (Sec. 7.2) which provides important features for dening and solving CSPs includ-
ing constraint propagation, user-dened distribution strategies, user-dened exploration
strategies, reied constraints and recomputation.
Particularly apparent features are propagation and distribution. Constraint propagation
is an essential constraint programming technique used to reduce the size of the search
space (cf. Dechter [2003]) and nowadays is supported by virtually all major general
constraint programming systems. Constraint distribution allows for user-dened and dy-
namic variable and value ordering. The choice of a variable ordering has an immense
inuence on the size of the search space, in particular, if the ordering can exploit knowl-
edge about the structure of the CSP (Sec. 7.3).
The space-based constraint model and its implementation in the Mozart system is com-
petitive in performance with other state-of-the-art constraint systems and superior for
large problems. M uller [2001] compared the performance of constraint propagation in
Mozart with ILOG Solver, ECLiPSe, SICStus, and GNU Prolog. Similarly, [Schulte,
2002] compared the performance of space-based copying and recomputation with the
trailing-based search of these other systems.
Strasheela makes the space-based constraint programming model and its features avail-
able for the rst time in a generic music constraint system (to the knowledge of the
author). The space-based constraint model has been applied before for music compo-
sition [Henz et al., 1996], but this research only implemented a very specic CSP (see
Sec. 3.2.3). Instead, the present research analysed the particular complexities of musical
CSPs in general (such as inaccessible score contexts, Sec. 6.3.1) and identied the need
for user-denable dynamic variable orderings. To address this need, the design of Stra-
sheela allows a score distribution strategy to exploit any information available in the
score at the time the distribution happens. Because constraint distribution is orthogonal
in the space-based constraint model, Strasheela preserves all other features of this model.
The present research proposes special score distribution strategies tailored for musical
CSPs which dene dynamic variable orderings for typical musical CSPs. The performance
gures provided in Sec. 8.2.2 clearly indicate the importance of a well-chosen distribution
strategy in solving musical CSPs. For example, the left-to-right strategy (Sec. 7.3.2.3)
allows the user to eciently solve polyphonic CSPs which existing systems (e.g. score-
PMC) explicitly made impossible to dene due to their computational complexity.
Consequently, the features of Strasheelas constraint solver are only partially (if at all)
supported by existing music constraint systems. Constraint propagation is supported
in some form by both Situation and BackTalk (but not by PWConstraints). BackTalk
supports a number of consistency enforcing techniques (see Sec. 3.3.4), but these are
performed only once before the actual search process starts. Situation implements a
variant of minimal forward checking for hierarchical domains (see Sec. 3.3.2.3), but its
199
10. Comparison and Evaluation
propagation algorithm is only suitable for its specic data structure (where all distances
of an object are represented by a single variable, cf. Sec. 3.3.2.3).
No existing music constraint system allows for user-dened dynamic variable and value
ordering as supported by Strasheela (to the knowledge of the author). BackTalk supports
dynamic variable orderings. Yet, MusES does not provide the required information to
dene score distribution strategies as proposed in Sec. 7.3.2.
Some existing systems allow the user to choose a search strategy (i.e. an exploration strat-
egy). For example, BackTalk supports a number of enumeration algorithms. BackTalk
even supports lookback schemes (e.g. backmarking, [Dechter, 2003]). In contrast, look-
back schemes have not yet been proposed for the space-based constraint programming
model. Due to the programmable nature of this model, this feature could be provided in
principle [Schulte, 2002]. However, BackTalk does not distinguish between the creation
(distribution) and the exploration of the search tree, and special exploration strategies
such as a best-solution search (which does not need to explore the full search tree to nd
an optimal solution, Sec. 7.2.3.3) are not provided.
10.4. Can Strasheela be Further Generalised?
Section 5.6.1 discussed the basic principles that Strasheela uses to represent explicit in-
formation: the type of a score object, its attributes, the hierarchic nesting of objects,
and the order of objects on the same hierarchical level. The value of object attributes
is the only explicitly represented information that can be unknown and constrained in
Strasheela. The explicit information represented by the other principles must be deter-
mined in the CSP denition. An interesting research question is how the other principles
can support unknown and constrained information.
Section 8.3 proposed an approach which eectively constrains the hierarchic nesting of
objects, although only in a limited form. When comparing dierent solutions, the actual
shape of the hierarchy cannot dramatically change, subtrees either exist or do not exist
(see Sec. 8.3.4).
A generalisation of this approach allows the user to freely constrain the score topol-
ogy. Such a generalisation makes it possible to freely constrain the musical texture (e.g.
whether a solution is homophonic, polyphonic, or a complex piano-like texture which is
constantly changing). The generalisation can be based on available research into graph
domain constraint programming such as Dooms et al. [2005b]. Graph domains have
already been applied in elds such as bio-informatics [Dooms et al., 2005a] and compu-
tational linguistics [Debusmann et al., 2005]. Tree domains have also been proposed for
musical applications. Curry et al. [2000] show how score trees are represented by nite
domain constraints.
Nevertheless, a constrainable representation of the hierarchic nesting based on graph
domains is still not fully generic, as the nesting of a solution must also correspond
with the type of the nested objects (e.g. a note object cannot contain another item).
200
10.5. Summary
A further generalisation of Strasheela would make the type information of score objects
constrainable. Constraining the type of an object implicitly constrains the set of its
attributes and its ADT interface. However, such a generalisation is not compatible with
the object-oriented programming paradigm. This means that such a generalisation would
not only require a fundamental redesign of Strasheela, but that the redesigned music
representation would have to do without the benets of object-oriented programming
such as concision due to incremental ADT denitions. Moreover, it can be dicult to
realise such a generalisation with specic domain variables which support constraint
propagation. In order to make the object type constrainable, the set of all its attributes
and its whole interface must be represented by variables as well. The values for these
data can have diverse formats and are therefore dicult to represent with specic domain
variables. This data is more easily represented by universal domain variables, but such
variables do not support constraint propagation and thus such a representation would
slow down the search process considerably.
In general, CSPs become more complex (to dene and to solve) when more principles
of explicit information support unknown and constrained information. Increasing the
amount of unknown and constrained information increases the number of inaccessible
score contexts. Inaccessible score contexts, however, complicate the application of com-
positional rules (cf. Sec. 6.3 and Sec. 8.3.4). For example, in cases where the hierarchic
structure is unknown, information such aswhich note belongs to which voiceandwhich
note follows which other note is unknown and consequently, all rules constraining these
contexts must be applied by techniques suitable for inaccessible score contexts. In cases
where even the type of an object is unknown, the interface of this object is also unknown
and no type-specic function can be applied to the object before enough information
about its type becomes available. In addition, inaccessible score contexts also result in
eciency problems. Constraints applied to inaccessible score contexts can only weakly
propagate. If the constraint application is delayed (e.g. because type information is miss-
ing) no propagation is possible at all. In such a situation, the search process detects a
conict too late (when a delayed rule is applied) which results in severe performance
problems.
In summary, it is in principle possible to further generalise Strasheela. However, such a
generalisation would complicate the denition of musical CSPs. Moreover, such a gener-
alisation would considerably impair the eciency of the search process.
10.5. Summary
This chapter compared Strasheela with existing systems in terms of generality. The
chapter showed that Strasheelas basic principles for representing score information are
only partially supported by existing systems. Consequently, all information stored in the
music representation of any existing system can be reproduced in Strasheela, but these
201
10. Comparison and Evaluation
systems can only partly reproduce the information supported by Strasheela.
3
In practice,
this has the eect that musical CSPs in existing systems usually express less information
about the music because information for which the adequate principle is missing is often
omitted. Moreover, the limited support of existing systems for derived information also
restricts which score contexts can be constrained. Strasheelas support for modular and
concise user extensions of the music representation are also missing in most existing
systems. All these factors restrict the denition of musical CSPs in existing systems and
in particular these factors make the denition of complex CSPs more dicult in existing
systems than in Strasheela.
On the other hand, Strasheela also has a drawback in terms of generality when compared
with most existing systems (the exception is Situation). Most of these systems support
variables with a universal domain whereas Strasheela supports variables with specic do-
mains. However, specic variable domains allow for constraint propagation which greatly
improves eciency. Moreover, this chapter showed that specic variable domains limit
the generality only in certain cases. CSPs where universal variable domains have been
most successful in music constraint programming in the past can often be reformulated
into a CSP on specic domain variables. In addition, Strasheela is not limited to specic
variable domains in principle.
The chapter also compared the rule formalisms of these systems. It was shown that the
existing rule application mechanisms (which are featured by the systems Situation and
PWConstraints) can be reproduced in Strasheela but not the other way round. Strasheela
facilitates the denition of complex CSPs because it allows the user to program rule
application mechanisms according to their needs.
Moreover, this chapter compared the search strategy of the systems. It was shown that
Strasheelas search process is more exible than other systems as it can be customised
for the needs of the musical CSP at hand. In this matter, the support of user-dened
dynamic variable and value orderings are particularly important. Existing systems, on
the other hand, are optimised for specic CSP classes and their customisation capabilities
are limited. Consequently, these systems are less suited for CSPs for which they are not
optimised whereas Strasheela has no such bias towards specic CSP classes.
After showing that Strasheela is in many respects more generic than existing music
constraint systems, the chapter asked how Strasheela could be further generalised. Ap-
proaches which would make the hierarchic nesting and the typing of score objects con-
strainable were presented. In addition, the consequences of such generalisations regard-
ing the ease of CSP denitions and the eciency of the search process were discussed.
3
This claim is valid only when implementation language dierences are ignored. For example, Common
Lisp supports ratios and complex numbers as built-in data types and thus PWConstraints and Situ-
ation can incorporate such data types in their representation. Such data types are missing in Oz and
thus Strasheela cannot easily reproduce a representation containing such types. Nevertheless, Oz can
be extended by libraries for these types. Alternatively, Strasheela can be re-implemented in Lisp.
202
11. Conclusion
In this chapter, the main contributions of the present research are summarised (Sec.
11.1) and an outline of future work is presented (Sec. 11.2).
11.1. Main Contributions
This thesis presents a highly generic music constraint system, called Strasheela. Stra-
sheela allows its user to dene a large set of musical CSPs and solves them in a reasonably
ecient way. When compared with general constraint systems (e.g. general programming
languages supporting constraint programming), Strasheela greatly simplies the deni-
tion of musical CSPs by predening the building blocks required for this class of CSPs.
When compared with existing generic music constraint systems (presented in Sec. 3.3),
Strasheela allows its user to dene and solve a considerably larger set of musical CSPs or
makes their denition more modular and concise and that way allows for more complex
CSPs to be implemented (see Chap. 10).
Strasheela is more generic than existing music constraint systems primarily because
Strasheela is more programmable and more extendable. The Strasheela user controls the
information expressed by the music representation, programs how rules are applied to
the music, and denes the decision-making of the search process.
Music Representation
The present research proposes a highly expressive music representation, well-suited to
dening complex musical CSPs (Chap. 5). When compared with general music represen-
tations (presented in Chap. 2), Strasheela adds constrained variables to the representa-
tion together with ecient means for solving. When compared with the representations
of existing generic music constraint systems, Strasheelas basic principles for represent-
ing information (see Sec. 5.6.1) are only partly supported by existing systems (see Sec.
10.1). As a result, Strasheelas representation can contain more information and provides
full user control over the contained information. Moreover, Strasheela allows its user to
access any score context which can be isolated with the information provided by the
score from any object in the score (Sec. 5.6).
Rule Formalism
Strasheelas rule formalism allows the user to constrain complex variable combinations
in a convenient way (Chap. 6). The user controls the scope of compositional rules with
203
11. Conclusion
rule application mechanisms which can be dened by the user according to needs. In
Strasheela, a compositional rule is encapsulated by a rst-class function (Sec. 6.1). A
rule application mechanism is a higher-order function which expects a rule and a score
(or some part thereof) and applies the rule to sets of variables in the score. This the-
sis presents several rule application mechanisms, including the mechanisms of PWCon-
straints and Situation (Sec. 6.2). Moreover, Strasheela supports constraining inaccessible
score contexts, for example, by implication constraints (Sec. 6.3).
Search Approach
Strasheela solves a wide range of musical CSPs in an ecient way by making the search
process adaptable to these problems (Chap. 7). Strasheela employs the space-based con-
straint model, because this model makes the search process programmable at a high-level
(Sec. 7.2). For example, by dening a distribution strategy, the Strasheela user denes a
dynamic variable and value ordering which is independent of the CSP denition. Stra-
sheela customises this model for music constraint programming: Strasheela allows a score
distribution strategy to exploit any information available in the score whenever it per-
forms a distribution step (Sec. 7.3). Strasheela preserves all other features of the model
such as constraint propagation, user-dened exploration strategies, reied constraints,
and recomputation (Sec. 7.2.3). This thesis presents special score distribution strategies
suitable for a large range of musical CSPs. For example, the text revises the left-to-right
search strategy of score-PMC such that the revised strategy eciently solves polyphonic
CSPs including problems which score-PMC explicitly made impossible to dene due to
their computational complexity (Sec. 7.3.2.3 and Sec. 8.2.2).
Prototype
The present research provides a prototype implementation of the presented model (Chap.
9). In addition, the prototype features enhancements such as style-specic extensions
of the music representation (e.g. a harmony model and a motif model) and output
into several music representation formats. The prototype comes with a set of example
applications and documentation.
11.2. Future Work
User-Guided Search Process
Strasheela solves a CSP autonomously, that is, without any user input besides the CSP
itself. This approach requires a full formalisation of all compositional knowledge relevant
to solve the CSP. However, a music theory is often presented partly in an informal way.
For instance, composition textbooks teach a signicant amount of professional skill by
musical examples instead of explicit rules.
It would therefore be interesting to investigate how a music constraint system can sup-
port both forms of user knowledge: explicit compositional knowledge (formalised in a set
204
11.2. Future Work
of rules) and knowledge which is learnt from examples but has not been explicated. For
example, Ovans and Davison [1992] propose a system which closely integrates formalised
compositional rules followed automatically by the systems with a manual composition
process. The rules only perform constraint propagation, but every decision during the
search process is made by the user. The system implements rst species Fuxian coun-
terpoint, and features a graphical user interface which shows the pitch domain of each
note (this domain is shown like a chord). The user determines the pitch of a note by
clicking on one of its domain pitches. Constraint propagation then reduces the domains
of the other notes according to the explicit rules. It would be interesting to investigate
how such an approach can be generalised for arbitrary musical CSPs. A generalisation
requires a very dierent user interface, because the approach of Ovans and Davison is
limited to CSPs which only constrain the pitches of monophonic melodies. Moreover,
it is desirable to allow for an optional automatic completion of a hand-edited partial
solution, as well as for manual changes to a solution. For example, the user may enter
important parts by hand and then ask the system to ll in the details. Also, the user
may change details of an unsatisfying solution and ask for a new solution which ts the
changes.
Integrating Soft Constraints
Music theories are usually full of exceptions which are very hard to fully formalise.
Moreover, most compositional rules only express preferences instead of strict rules. Also,
composers wish to grade the importance of compositional rules such that less important
rules may be neglected in an over-constraint situation.
Preferences can be expressed by two techniques in Strasheela: reied constraints and
best-solution search. For example, reied constraints can express the probability of how
often a rule must be fullled (Sec. 7.2.3.4). Best solution search (Sec. 7.2.3.3) nds
multiple solutions and always constrains the next solution to be better according to a
user-dened criterion. This criterion can express a conjunction of preferences, possibly
graded by importance (this technique was applied by Henz et al. [1996] to express soft
compositional rules). Still, both techniques express preferences only indirectly by a
combination of strict constraints. In eect, these techniques are relatively inexible and
inecient.
Therefore, it would be interesting to study alternative approaches which introduce soft
constraints as single propagators which can be relaxed. For example, [Bistarelli et al.,
2002] propose a highly generic way to express concurrent soft constraints based on the
notion of semirings (i.e. an algebraic structure similar to a ring which denes a set
of values and the two binary operators + and ). These constraints can express well-
known soft CSP classes including fuzzy CSPs, probabilistic CSPs and weighted CSPs
(these cases dier in the semiring of truth values they are based on). Delgado et al. [2005]
present an implementation of semiring-based constraints for Oz which can combine hard
and soft constraints in a single CSP and which could be applied directly in Strasheela.
205
11. Conclusion
User Interface
The current Strasheela prototype employs the Oz language as user interface because of
its unique features as a multi-paradigm programming language. However, Oz has hardly
been used in the eld of computer music so far and consequently a new user needs some
eort to learn Strasheela. In particular composers and music theorists without much
programming experience are likely to shy away from a system which requires learning a
new programming language.
Porting Strasheela to a language which is more widely-used in the computer-aided com-
position community or which oers special support for unexperienced programmers
makes the system more easily accessible. An example of a widely-used language is Com-
mon Lisp, because there exist a considerable number of Lisp-based composition systems
(e.g. Common Music [Taube, 2004], PatchWork [Laurson, 1996] with its successors Open-
Music [Assayag et al., 1997] and PWGL [Laurson and Kuuskankare, 2002], ACToolbox
[Berg]). Visual languages enjoy much popularity among musicians (especially data-ow
language such as Max and its relatives [Puckette, 2002]). Section 3.3.1 discussed the
benets and disadvantages of visual programming. It would be interesting to investi-
gate how visual programming can facilitate the use of a generic music constraint system
like Strasheela in a way that scales well even for complex CSPs. For instance, it may
be suitable that the user denes in a visual way the top-level of a CSP (which usually
only instantiates the score and applies rules to it), whereas the constructs used by this
top-level (e.g. the actual rules) are dened textually. An alternative approach oering
special support for unexperienced programmers is the use of a programming environment
for a purely textual language, which is especially targeting programming beginners as
exemplied by DrScheme [Findler et al., 2002].
Learning-Based Rule Generation
The Strasheela user denes a music theory model completely manually. In the context of
computer-aided composition, this approach has the advantage that the user has full con-
trol and can even dene a theory for a musical style which does not exist yet. Composers
are often primarily interested in developing their own distinct musical language instead
of replicating an existing musical style. Nevertheless, this freedom is coupled with the
eort of creating such a music theory denition.
Research on computational models of music composition often also applies a learning-
based approach where compositional knowledge is deduced from existing pieces. For
example, Hild et al. [1992] propose HARMONET, which uses a connectionist approach
in order to learn choral harmonisation from Bach choral examples. Cope [1991, 1996]
presents EMI, a system which analyses given pieces of music and uses the result of this
analysis in order to create similar pieces. For example, EMI extracts reappearing phrases
typical for a certain style (called signatures) and analyses the function of musical frag-
ments (e.g. whether they raise an expectation or express a conclusion, using a grammar
called SPEAC).
It would be interesting to integrate learning techniques into Strasheela in such a way
206
11.2. Future Work
that the user would mix manually-dened compositional knowledge with explicitly rep-
resented knowledge deduced from existing pieces. If the knowledge learnt is represented
in the form of rules, then this knowledge can be freely mixed with handwritten rules.
For example, [Morales and Morales, 1995] propose a system which automatically creates
rules in rst-order logic (horn clauses) given a musical example and rule templates. The
textbook [Mitchell, 1997] introduces learning techniques including the learning of rules.
Real-Time Constraint Programming
Many computer musicians are interested in creating interactive music performance soft-
ware, as exemplied by the large user community of systems like Max/MSP and Super-
Collider. Programs created by these systems can show a very complex behaviour (e.g.
demonstrated by the interactive real-time composition Lexikon-Sonate [Essl, 2004]). Nev-
ertheless, due to the procedural nature of these systems it is hard to create software whose
output complies with a complex music theory (e.g. some theory of harmony conceived
by the user).
It would be interesting to explore how constraint programming can facilitate the creation
of interactive music software which implements complex music theories. SuperCollider
already supports constraint programming, but only in a very limited form: constraints
can lter undesirable values out of a SuperCollider pattern.
1
Oz, on the other hand,
provides several features which makes it a promising platform for real-time constraint
programming. For instance, soft real-time programming capabilities are provided by its
scheduling facilities. Also, Oz can emulate Max dataow programming approach: Oz
data (e.g. numbers or score fragments) can ow through streams between concurrent
processes which synchronise on the availability of data via logic variables (cf. Sec. 7.1).
A process in such a dataow program can conduct any computation, including solving
a CSP. For example, the full Fuxian example (Sec. 8.1) takes only about 50 msec to
solve, which may be fast enough for a real-time context. Oz supports techniques which
can ensure that a process takes only a limited amount of time in an interactive pro-
gram. Oz supports both dont know nondeterminism (where a CSP is solved by a
complete search; this was the only approach discussed so far) and also dont care non-
determinism (where a decision may require fullling complex constraints but is never
taken back once made; this is also known as committed choice constraint programming).
While dont know nondeterminism is well-suited for creating composition software,
dont care nondeterminism is well-suited for creating interactive improvisation soft-
ware. Moreover, search is encapsulated in Oz: a search process runs concurrently in its
own thread (i.e. it does not block the rest of the program) and it can be cancelled in cases
where it does not nd a solution within a limited amount of time. A complete search
for a sub-CSP can even be encapsulated within a committed choice CSP. Finally, Oz
networking support (e.g. support for sockets) allows for a coupling with other systems
such as Max or SuperCollider.
1
This feature is implemented and documented by the so-called crucial library which is part of Super-
Collider.
207
11. Conclusion
208
Appendix A.
Notational Conventions
Throughout this thesis, mathematical notation is used as a kind of pseudo-code to com-
municate examples. For this purpose, decisions for notational variants aimed for legibility
instead of rigour.
The notation complements rst-order logic notation by functions as rst-class values
plus a number of data types with a literal representation.
Great care has been taken to choose a notation which follows common conventions. Still,
the present appendix provides a brief summary of this notation.
The notation discussed here is the notation used in this thesis. The user-interface of the
Strasheela prototype, on the other hand, uses the Oz programming language (see Sec.
9). The mathematical notation introduced here has solely been chosen to simplify the
discussion in this text: it is not required to know any particular programming language
for reading the examples given here.
A.1. Variables
The notation of a variable reects the nature of its value to make examples more readable
(see Tab. A.1
1
). In general, a variable is notated starting with a lower case letter. Such
a variable can be bound to, for instance, a number, a function, or an ADT.
A constrained variable (i.e. a partial value with a domain) is usually notated starting
with an upper case letter to stand out more clearly. Such a variable can be understood as
a logically quantied variable. In case it is not decided whether a variable is constrained
(e.g. if a variable is an argument in a function denition) the lower case notation may
be used.
It should be noted that a variable which binds a composite value or an ADT is always
notated with a lower case letter (e.g. myNote) even if it contains a constrained variable.
For example, the pitch value of myNote may be a constrained variable (e.g. MyPitch =
getPitch(myNote)), but myNote itself is written starting with a lower case letter.
1
For comprehensibility, instead of an abstract syntax specication (e.g. in BNF notation) an informal
mix of simple examples and BNF constructs are used.
209
Appendix A. Notational Conventions
Concept Notation
variable x
constrained variable (partial value) X
set variable S
(variable) denition x
def
= expression)
introducing a local variable (rst form) let denitions)
in expression)
introducing a local variable (second form) expression) where denitions)
notation short hand (second form) expression) : denitions)
Table A.1.: Variable notation
A set is notated in bold face.
For convenience, the notation does not necessarily require to explicitly declare a variable.
The scope of a variable is its surrounding block, for instance, a function denition. Many
examples explicitly dene a local scope for variables with either the let or the where
form. The where-notation can also be used to express a number of conditions on a value
as in expression) : conditions).
A.2. Functional Abstraction
In this thesis, a function is a rst-class value bound to a variable (i.e. a -expression,
see discussion in Sec. 2.8). Nevertheless, the common notation for a function denition
and function application is used (see Tab. A.2), making examples most easy to read by
avoiding the less widespread lambda calculus notation. Instead, the present text uses the
where-notation [Landin, 1966] as syntactic sugar for an anonymous function. For exam-
ple, the function f serves the purpose of an anonymous function in g(f wheref(x)
def
= x
2
)
or shorter g(f : f(x)
def
= x
2
).
For convenience, the notation allows for functions with keyword arguments (named ar-
guments), which are also optional arguments.
A.3. Control Structures
The if-then-else conditional is written in a common way. This expression always returns
a value (see Tab. A.3).
210
A.4. Values and Data Structures
Concept Notation Example
(function) denition square(x)
def
= x
2
function application square(3) = 9
inline function denition (serves purpose
of anonymous function)
map([1, 2, 3], f : f(x)
def
= x
2
) = [1, 4, 9]
function with optional keyword arguments
(denition and applications without and
with optional argument)
addN(x, n: 1)
def
= x +n
addN(3) = 4
addN(3, n: 2) = 5
Table A.2.: Functional abstraction
Concept Notation Example
conditional if boolean expression)
then expression)
else expression)
Table A.3.: Control structures
A.4. Values and Data Structures
A number of data types is used in the present text. Not all these data types are used in
common mathematical notation, therefore the notation is also inspired by the syntax of
various programming languages and Backus-Naur form (see Tab. A.4).
A symbol (which evaluates to itself) is notated starting with a lower case letter, much
like a variable (which evaluates to its value). Usually, the context will tell what is meant.
In mathematical notation, a list is often notated simple as 1, 2, 3. Instead, the notation
used here is slightly more explicit and encloses a list in brackets. The empty list is denoted
with the symbol nil. A set is written in the common way (i.e. enclosed by curly braces). A
record is a composite data structure with a label (in Tab. A.4, the label is myLabel) and
values (x
1
to x
3
) at named features (a to c). Although the record notation is similar to
the notation of a function call, the context will tell what is meant. Following conventions
of mathematical notation, the elements in a list, set, and record are separated by colons.
A.5. Operations
The notation of operations on values (e.g. numerical operations, set operations, and
logical connectives) follows the usual conventions (see Tab. A.5).
The operational semantics of these operations is overloaded for simplicity. For example,
the notation of this thesis only uses a single equality operator (x = y). This operator
can have a number of operational meanings. These meanings are clearly distinguished in
211
Appendix A. Notational Conventions
Concept Notation Example
two truth values true, false or 1, 0
number 1
symbol mySymbol
list (sequence) and empty list [1, 2, 3] and nil
set 1, 2, 3
record with a number of features and their values myLabel(a: x
1
, b: x
2
, c: x
3
)
ADT (i.e. a value without literal representation) myValue)
Table A.4.: Values and data structures (which evaluate to themselves)
the Oz notation (and thus the Strasheela prototype examples), for instance, unication
(X = Y), the equality test returning a boolean (X == Y), and the equality constraint (X
=: Y).
Similarly, the examples in Tab. A.5 show the overloading of numerical and set operators.
These operators can be applied as functions with determined arguments returning a
determined value but also as constraints to constrained variables.
Concept Notation Example
numerical operation example (deterministic addition
and addition constraint)
x = y +z
X = Y +Z
set operation example (boolean membership predi-
cate and membership constraint)
x Set
X Set
logical connectives (conjunction, disjunction, nega-
tion, implication and equivalence)
B
1
B
2
B
1
B
2
B
B
1
B
2
B
1
B
2
cumulative conjunction

[B
1
, B
2
, B
3
] or

n
i=1
B
i
Table A.5.: Operations
212
Appendix B.
Additional Denitions
The present appendix provides denitions which were released from the body of this
thesis for brevity. The main text motives these functions, explains their behaviour and
points here into the appendix for their denition. In turn, each denition shown here
references the section in which it was discussed.
getTemporalTopLevel (x)
def
= let c
def
= nd (getContainers(x),
isTemporalAspect)
in if c = nil
then x
else getTemporalTopLevel (c)
Figure B.1.: The function getTemporalTopLevel is discussed in Sec. 5.6.2.1
zip(xs, ys, fn)
def
= if xs = nil
then nil
else cons (fn(head(xs), head(ys)),
zip(tail (xs), tail (ys), fn))
Figure B.2.: The function zip is discussed in Sec. 6.2.4
213
Appendix B. Additional Denitions
let /* The function ranges returns a list of elements in xs which match decls. decls is a list consisting
of single indices and startIndex#endIndex pairs. */
ranges(xs, decls)
def
=
/* mappend(xs, fn) is a variant of map(xs, fn). The mappend argument fn must return a list.
mappend appends all collected sublists (hence the name: map-append or mappend). */
mappend(decls,
f : f(decl )
def
= if isPair(decl )
/* decl is a start#end pair.
The expression sublist(xs, start, end) accesses the sublist of xs
that consists of the start-th to end-th elements (including). */
then let start#end
def
= decl
in sublist(xs, start, end)
/* decl is a single index. */
else [nth(xs, decl )])
in mapIndex(xs, decls, fn)
def
= let matchingXs
def
= ranges(xs, decls)
in map(matchingXs, fn)
Figure B.3.: The function mapIndex is discussed in Sec. 6.2.5
214
mapPM(xs, patternExpr, fn)
def
=
let /* collectPM is a highly recursive auxiliary function which returns in a list any list of elements from
xs (a list) which match the patternExpr (a list of pattern symbols). The argument matchingXs
(initially always nil ) is used to pass matching elements accumulated so far to the next recursive
call of collectPM. */
collectPM(xs, patternExpr, matchingXs)
def
=
if patternExpr = nil xs = nil
then [reverse(matchingXs)]
else /* Recursively call collectPM depending on the rst symbol in patternExpr. */
let symbol
def
= head(patternExpr)
xsTail
def
= tail (xs)
patternTail
def
= tail (patternExpr)
in result where
result =
8
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
<
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
:
collectPM(xsTail , patternTail , matchingXs) if symbol = ?
collectPM(xsTail ,
patternTail ,
cons(head(xs), matchingXs))
if symbol = x
let /* minimal length of remaining pattern is length of
patternTail without any occurence of * (Kleene star)
*/
minLength
def
= length( lter(patternTail ,
f : f(x)
def
= x = ))
in /* mappendTail recursively applies f to every non-nil
tail of xs and appends the resulting lists. */
mappendTail (xs,
f : f(sublist)
def
=
if length(sublist) minLength
then collectPM(sublist,
patternTail ,
matchingXs)
else nil )
if symbol =
in map(collectPM(xs, patternExpr, nil ), fn)
Figure B.4.: The function mapPM is discussed in Sec. 6.2.6
getTestIndex(x, tests)
def
=
/* The function ndPosition expects a list and a unary boolean function and returns the position
of the rst list element for which the function returns true. In getTestIndex, this list consists of
boolean functions. */
ndPosition(tests, f : f(test)
def
= test(x))
Figure B.5.: The function getTestIndex is discussed in Sec. 7.3.2.2
215
Appendix B. Additional Denitions
direction(X
1
, X
2
, Dir)
def
=
/* Dir is constrained to the direction of the interval between X 1 and X 2. An interval upwards (the
predecessor is smaller than the successor) is represented by 2, an horizontal interval (the predecessor
and the successor are equal) is represented by 1, and an interval downwards by 0. X 1, X 2, and
Dir are all nite domain integers. */
let IsUp
def
= fdInt(0, 1)
IsEq
def
= fdInt(0, 1)
in 0 Dir 2
IsUp = (X
1
< X
2
)
IsEq = (X
1
= X
2
)
Dir = 2IsUp + IsEq
Figure B.6.: The function direction is discussed in Sec. 8.1.2.2
216
Appendix C.
The Strasheela Website
Chapter 9 introduced the Strasheela prototype which implements the Strasheela model
presented in this text and also provides several extensions of this model. The prototype
be obtained from Sourceforge at https://sourceforge.net/projects/strasheela/.
The Strasheela website (http://strasheela.sourceforge.net) provides documenta-
tion for this prototype which includes installation instructions, links to relevant Oz doc-
umentation, as well as reference documentation for the Strasheela core and its exten-
sions. In addition, the website presents and explains several Strasheela application exam-
ples (http://strasheela.sourceforge.net/strasheela/doc/StrasheelaExamples.
html). These include not only the examples presented in chapter 8, but also other ex-
amples as well.
217
Appendix C. The Strasheela Website
218
Appendix D.
Source Material
The material presented in this text has been partly published before in the following
articles.
Anders, T. (2000). Arno: Constraints Programming in Common Music. In Proceed-
ings of the 2000 International Computer Music Conference [Anders, 2000].
Anders, T. (2002). A wizards aid: ecient music constraint programming with
Oz. In Proceedings of the 2002 International Computer Music Conference [Anders,
2002].
Anders, T., C. Anagnostopoulou, and M. Alcorn (2005). Strasheela: Design and Us-
age of a Music Composition Environment Based on the Oz Programming Model. In
P. v. Roy (Ed.), Multiparadigm Programming in Mozart/OZ: Second International
Conference, MOZ 2004, LNCS 3389. Springer-Verlag [Anders et al., 2005].
In addition, material was presented in the following talks.
Anders, T. (2004). The Score Description Language: A Tutorial. PRISMA Meet-
ing,
1
Centro Tempo Reale, Florence, January 2004.
Anders, T. (2004). Comparing Constraint Based Composition Systems: A Survey.
PRISMA meeting, IRCAM, Paris, June 2004.
Anders, T. (2005). Composing Music by Composing Rules. Invited lecture, Music
Informatics Research Group, Edinburgh University, March 2005.
Anders, T. (2006). Why is Strasheela a Generic Music Constraint System? PRIS-
MA meeting, IRCAM, Paris, June 2006.
1
PRISMA (Pedagogia e Ricerca Internazionale sui Sistemi Musicali Assistiti) is an international group
of about twenty composers and researchers, where many members have expertise in music constraint
programming.
219
Appendix D. Source Material
220
Bibliography
Abelson, H., G. J. Sussman, and J. Sussman (1985). Structure and Interpretation of
Computer Programs. MIT Press.
Agon, C., G. Assayag, J. Baboni, and K.Haddad (2001). OpenMusic (OM) 4.0. Users
Manual Reference & Tutorial. Paris: IRCAM.
Agon, C., G. Assayag, O. Delerue, and C. Rueda (1998). Objects, Time and Constraints
in OpenMusic. In Proceedings of the 1998 International Computer Music Conference,
Ann Arbor, Michigan.
Alvarez, G., J. F. Diaz, L. O. Quesada, F. D. Valencia, G. Assayag, and C. Rueda
(1998). PiCO: A Calculus of Concurrent Constraint Objects for Musical Applications.
In ECAI 98 Workshop on Constraints for Artistic Applications, Brighton.
Anders, T. (2000). Arno: Constraints Programming in Common Music. In Proceedings
of the 2000 International Computer Music Conference, Berlin, Germany.
Anders, T. (2002). A wizards aid: ecient music constraint programming with Oz. In
Proceedings of the 2002 International Computer Music Conference, Goteborg, Sweden.
Anders, T. (2003). Composing Music by Composing Rules: Computer Aided Composi-
tion employing Constraint Logic Programming. Technical report, Sonic Arts Research
Centre, Queens University Belfast.
Anders, T. (2006). Strasheela. http://strasheela.sourceforge.net/ (accessed 29
May 2006).
Anders, T., C. Anagnostopoulou, and M. Alcorn (2005). Strasheela: Design and Usage of
a Music Composition Environment Based on the Oz Programming Model. In P. V. Roy
(Ed.), Multiparadigm Programming in Mozart/OZ: Second International Conference,
MOZ 2004, Number 3389 in Lecture Notes in Computer Science. Springer-Verlag.
Apt, K. R. (2003). Principles of Constraint Programming. Cambridge University Press.
Ariza, C. (2005). Navigating the Landscape of Computer-Aided Algorithmic Composi-
tion Systems: A Denition, Seven Descriptors, and a Lexicon of Systems and Research.
In Proceedings of the International Computer Music Conference.
Ariza, C. (2006). algorithmic.net: A Lexicon of Systems and Research in Computer-Aided
Algorithmic Music Composition. http://www.flexatone.net/algoNet/index.html
(accessed 15 May 2006).
221
Bibliography
Assayag, G. (1998). Computer Assisted Composition Today. In 1st Symposium on Music
and Computers, Corfu.
Assayag, G. and C. Agon (1996). OpenMusic Architecture. In Proceedings of ICMC 96,
Hong Kong, China.
Assayag, G., C. Agon, J. Fineberg, and P. Hanappe (1997). An Object Oriented Visual
Environment For Musical Composition. In Proceedings of the International Computer
Music Conference, Thessaloniki.
Assayag, G., C. Rueda, M. Laurson, C. Agon, and O. Delerue (1999). Computer As-
sisted Composition at IRCAM: From PatchWork to Open Music. Computer Music
Journal 23(3).
Avispa Research Group (2005). XRI and LP for Mozart. https://gna.org/projects/
xrilpoz/ (accessed 4 August 2006).
Avispa Research Group (2006). GEOZ - Improving Mozart Constraints Serv. http:
//home.gna.org/xrilpoz/index.html (accessed 4 August 2006).
Balaban, M. (1996). The Music Structures Approach to Knowledge Representation for
Music Processing. Computer Music Journal 20(2).
Ballesta, P. (1994). Contraintes et objets: clefs de vo ute dun outil daide a la composition.
Ph. D. thesis, Universite du Maine.
Barendregt, H. and E. Barendsen (1991). Introduction to Lambda Calculus. Technical
report, Department of Computer Science, Catholic University of Nijmegen.
Bartak, R. (1998). On-Line Guide to Constraints Programming. http://kti.mff.cuni.
cz/~bartak/constraints/ (accessed 25 May 2006).
Baum, L. F. (1993, orig. 1900). The Wonderful Wizard of Oz. Wordsworth.
Berg, P. AC Toolbox. http://www.koncon.nl/ACToolbox/ (accessed 24 May 2006).
Berlioz, H. and R. Strauss (1904). Instrumentationslehre. Edition Peters.
Bistarelli, S., U. Montanari, and F. Rossi (2002). Soft concurrent constraint program-
ming. In European Symposium on Programming, Grenoble, France.
Bonnet, A. and C. Rueda (1999). OpenMusic. Situation. version 3 (3rd ed.). Paris:
IRCAM.
Booch, G. (1991). Object Oriented Design with Applications. Benjamin-Cummings.
Booch, G., J. Rumbaugh, and I. Jacobson (1998). The Unied Modeling Language User
Guide. Addison Wesley.
222
Bibliography
Boshernitsan, M. and M. Downes (2004). Visual Programming Languages: A Survey.
Technical Report UCB/CSD-04-1368, Computer Science Division (EECS), University
of California, Berkeley.
Boulanger, R. (Ed.) (2000). The Csound Book. Perspectives in Software Synthesis, Sound
Desing, Signal Processing, and Programming. The MIT Press.
Bratko, I. (2001). Prolog. Programming for Articial Intelligence (3rd ed.). Addison-
Wesley.
Bresson, J., C. Agon, and G. Assayag (2005). OpenMusic 5: A Cross-Platform Release
of the Computer-Assisted Composition Environment. In 10th Brazilian Symposium
on Computer Music, Belo Horizonte, Brazil.
Burnett, M. (2006). Visual Language Research Bibliography. http://web.engr.
oregonstate.edu/~burnett/vpl.html, (accessed 27 April 2006).
Cheadle, A. M., W. Harvey, A. J. Sadler, J. Schimpf, K. Shen, and M. G. Wallace (2003).
Eclipse: An introduction. Technical Report IC-PARC-03-1, IC-Parc, Imperial College
London.
Chemillier, M. and C. Truchet (2001). Two Musical CSPs. In Seventh International Con-
ference on Principles and Practice of Constraint Programming, Musical Constraints
Workshop, Paphos, Cyprus.
Choi, C. W., M. Henz, and K. B. Ng (2001). Components for State Restoration in
Tree Search. In T. Walsh (Ed.), Principles and Practice of Constraint Programming
CP 2001 : 7th International Conference, Volume 2239 of Lecture Notes in Computer
Science. Springer-Verlag.
Codognet, P. and D. Diaz (2001). Yet Another Local Search Method for Constraint
Solving. In T. Walsh and C. Gomes (Eds.), Proceedings of the AAAI Symposium
Using Uncertainty in Computation. AAAI Press.
Codognet, P., D. Diaz, and C. Truchet (2002). The Adaptive Search Method for Con-
straint Solving and its Application to musical CSPs. In IWH02, International Work-
shop on Heuristics, Beijing, China.
Conklin, D. and I. H. Witten (1995). Multiple Viewpoint Systems for Music Prediction.
Journal of New Music Research 24(1).
Cooper, G. and L. B. Meyer (1960). The Rhythmic Structure of Music. The University
of Chicago Press.
Cope, D. (1991). Computers and Musical Style. Madison, WI: A-R Editions.
Cope, D. (1996). Experiments in Musical Intelligence. Madison, WI: A-R Editions.
223
Bibliography
Cormen, T. H., C. E. Leiserson, and R. L. Rivest (2001). Introduction to Algorithms
(Second ed.). MIT Press.
Courtot, F. (1990). A Constraint Based Logic Program for Generating Polyphonies. In
Proceedings of the International Computer Music Conference, Glasgow.
Curry, B., G. A. Wiggins, and G. Hayes (2000). Representing Trees with Constraints.
In Proceedings of the First International Conference on Computational Logic, Lecture
Notes in Computer Science 1861. Springer-Verlag.
Dannenberg, R. B. (1984). Arctic: A Functional Language for Real-Time Control. In
Proceedings of the 1984 ACM Symposium on LISP and functional programming. ACM
Press New York, NY, USA.
Dannenberg, R. B. (1989). The Canon Score Language. Computer Music Journal 13(1).
Dannenberg, R. B. (1993). Music Representation Issues, Techniques, and Systems. Com-
puter Music Journal 17(3).
Dannenberg, R. B. (1997). Machine Tongues XIX: Nyquist, a Language for Composition
and Sound Synthesis. Computer Music Journal 21(3).
Debusmann, R., D. Duchier, and J. Niehren (2005). The XDG Grammar Development
Kit. In P. V. Roy (Ed.), Multiparadigm Programming in Mozart/OZ: Second Inter-
national Conference, MOZ 2004, Lecture Notes in Computer Science 3389. Springer-
Verlag.
Dechter, R. (2003). Constraint Processing. Morgan Kaufmann.
Delgado, A., C. A. Olarte, J. A. Perez, and C. Rueda (2005). Implementing Semiring-
Based Constraints Using Mozart. In P. V. Roy (Ed.), Multiparadigm Programming in
Mozart/OZ: Second International Conference, MOZ 2004, Lecture Notes in Computer
Science 3389. Springer-Verlag.
Dent, M. J. and R. E. Mercer (1994). Minimal Forward Checking. In 6th IEEE Inter-
national Conference on Tools with Artifcial Intelligence, New Orleans, Louisiana.
Desain, P. (1990). LISP as a Second Language: Functional Aspects. Perspectives of New
Music 28(1).
Desain, P. and H. Honing (1988). LOCO: A Composition Microworld in Logo. Computer
Music Journal 12(3).
Desain, P. and H. Honing (1997). CLOSe to the edge? Advanced object oriented
techniques in the representation of musical knowledge. Journal of New Music Re-
search 26(1).
224
Bibliography
Deville, Y., G. Dooms, S. Zampelli, and P. Dupont (2005). CP(Graph+Map) for Ap-
proximate Graph Matching. In F. Azevedo, C. Gervet, and E. Pontelli (Eds.), 1st
International Workshop on Constraint Programming Beyond Finite Integer Domains,
Sitges, Barcelona, Spain.
Diaz, D. (2003). The GNU Prolog web site. http://gnu-prolog.inria.fr (accessed
25 May 2006).
Daz, J. F., G. Gutierrez, C. A. Olarte, and C. Rueda (2005). Using Constraint Pro-
gramming for Reconguration of Electrical Power Distribution Networks. In P. V. Roy
(Ed.), Multiparadigm Programming in Mozart/OZ: Second International Conference,
MOZ 2004, Lecture Notes in Computer Science 3389. Springer-Verlag.
Dooms, G., Y. Deville, and P. Dupont (2005a). A Mozart Implementation of CP(BioNet).
In P. V. Roy (Ed.), Multiparadigm Programming in Mozart/OZ: Second International
Conference, MOZ 2004, Lecture Notes in Computer Science 3389. Springer-Verlag.
Dooms, G., Y. Deville, and P. Dupont (2005b). CP(Graph): Introducing a Graph Com-
putation Domain in Constraint Programming. In 11th International Conference on
Principles and Practice of Constraint Programming, Number 3709 in Lecture Notes
in Computer Science, Sitges, Barcelona, Spain. Springer-Verlag.
Doty, D. B. (2002). The Just Intonation Primer. An Introduction to the Theory and
Practice of Just Intonation (Third ed.). San Francisco, CA: Just Intonation Network.
Duchier, D., C. Gardent, and J. Niehren (1998). Concurrent Constraint Programming
in Oz for Natural Language Processing. http://www.ps.uni-sb.de/~niehren/Web/
Vorlesungen/Oz-NL-SS01/BookHomePage.html (accessed 25 May 2006).
Duchier, D., L. Kornstaedt, M. Homik, T. M uller, C. Schulte, and P. V. Roy (2004).
Mozart Documentation. System Modules (1.3.1 ed.). Mozart Consortium. http://
www.mozart-oz.org/documentation/system/index.html (accessed 25 May 2006).
Ebcioglu, K. (1980). Computer Counterpoint. In Proceedings of the International Com-
puter Music Conference 1980, Queens College, New York City, USA.
Ebcioglu, K. (1984). An expert system for schenkerian synthesis of chorales in the style
of J.S.Bach. In Proceedings of the 1984 International Computer Music Conference,
Paris.
Ebcioglu, K. (1992). An Expert System for Harmonizing Chorales in the Style of J.S.
Bach. In M. Balaban, K. Ebcioglu, and O. Laske (Eds.), Understanding Music with
AI: Perspectives on Music Cognition, Chapter 12. MIT Press.
Essl, K. (2004). Lexikon-Sonate. http://www.essl.at/works/Lexikon-Sonate.html
(accessed 8 August 2006).
225
Bibliography
Ferrand, M., J. A. Leite, and A. Cardoso (1999). Improving Optical Music Recognition
by means of Abductive Constraint Logic Programming. In Portuguese Conference
on Articial Intelligence, Number 1695 in Lecture Notes in Articial Intelligence.
Springer.
Findler, R. B., J. Clements, C. Flanagan, M. Flatt, S. Krishnamurthi, P. Steckler, and
M. Felleisen (2002). DrScheme: A Programming Environment for Scheme. Journal of
Functional Programming 12(2).
Forte, A. (1973). The Structure of Atonal Music. Yale University Press.
Forte, A. and S. E. Gilbert (1982). Introduction to Schenkerian Analysis. W. W. Norton
& Company.
Fowler, M. (2003). UML Distilled: A Brief Guide to the Standard Object Modeling
Language (3rd ed.). Addison Wesley.
Fr uhwirth, T. and S. Abdennadher (2003). Essentials of Constraint Programming.
Springer.
Fux, J. J. (1965, orig. 1725). The Study of Counterpoint. from Johann Joseph Fuxs
Gradus ad Parnassum. W.W. Norton & Company. translated and edited by Alfred
Mann.
Gamma, E., R. Helm, R. Johnson, and J. Vlissides (1995). Design Patterns. Addison-
Wesley.
Gecode authors (2006). Gecode. generic constraint development environment. http:
//www.gecode.org/ (accessed 22 April 2006).
Gervink, M. (2003, orig. 1995). Die Strukturierung des Tonraums. Versuche einer System-
atisierung von Zwolftonreihen in den 1920er bis 1970er Jahren. In W. Auhagen, B. Gat-
jen, and K. W. Niemoller (Eds.), Systemische Musikwissenschaft. Festschrift Jobst P.
Fricke zum 65. Geburtstag. available at http://www.uni-koeln.de/phil-fak/muwi/
fricke/ (accessed 29 May 2006).
Harris, M., A. Smaill, and G. Wiggins (1991). Representing Music Symbolically. In IX
Colloquio di Informatica Musicale, Genoa, Italy.
Harvey, W. D. and M. L. Ginsberg (1995). Limited Discrepancy Search. In C. S. Mellish
(Ed.), Fourteenth International Joint Conference on Articial Intelligence, Monteal,
Quebec, Canada. Morgan Kaufmann Publishers.
Henz, M., S. Lauer, and D. Zimmermann (1996). COMPOzE intention-based music
composition through constraint programming. In Proceedings of the 8th IEEE In-
ternational Conference on Tools with Articial Intelligence, Toulouse, France. IEEE
Computer Society Press.
226
Bibliography
Hild, H., J. Feulner, and W. Menzel (1992). HARMONET: A Neural Net for Harmonizing
Chorales in the Style of J.S.Bach. In R. Lippmann, J. Moody, and D. Touretzky
(Eds.), Advances in Neural Information Processing Systems 4 (NIPS 4), San Mateo,
CA. Morgan Kaufmann Publishers.
Hiller, L. and L. Isaacson (1993, orig. 1958). Musical Composition with a High-Speed
Digital Computer. In S. M. Schwanauer and D. A. Lewitt (Eds.), Machine Models of
Music. MIT press. reprint of original articel in Journal of Audio Engineering Society,
1958.
Honing, H. (1993). Issues in the representation of time and structure in music. Contem-
porary Music Review 9.
Hudak, P. The Haskore Computer Music System. http://haskell.org/haskore/ (ac-
cessed 24 May 2006).
Huron, D. (2002). Music Information Processing Using the Humdrum Toolkit: Concepts,
Examples, and Lessons. Computer Music Journal 26(2).
ILOG. ILOG Solver. http://www.ilog.com/products/solver/ (accessed 24 May
2006).
Jacobson, I. (1992). Object-Oriented Software Engineering A Use Case Driven Ap-
proach. Addison Wesley,.
Jeppesen, K. (1971, orig. 1930). Kontrapunkt (4th ed.). Leipzig: Breitkopf & Hartel.
Kelly, J. (1997). The Essence of Logic. Prentice Hall.
Kiczales, G., J. des Rivieres, and D. G. Bobrow (1991). The Art of the Metaobject
Protocol. MIT Press.
Koch, H. C. (2000, orig. 17821793). Versuch einer Anleitung zur Kompositions. in
Rudolstadt. reprint: Georg Olms Verlag.
Kornstaedt, L. and D. Duchier (2004). The Oz Programming Interface (1.3.1 ed.). Mozart
Consortium. http://www.mozart-oz.org/documentation/system/index.html (ac-
cessed 25 May 2006).
Kretz, J. (2003). Continuous Gestures of Structured Material. Experiences in Computer
Aided Composition. In PRISMA 01. Milano: EuresisEdizioni.
Kumar, V. (1992). Algorithms for constraints satiscation problems: A survey. AI
Magazine 13(1).
Krenek, E. (1952). Zwolfton-Kontrapunkt-Studien. Mainz: B. Schotts Sohne.
Landin, P. J. (1966). The Next 700 Programming Languages. Communications of the
ACM 9(3).
227
Bibliography
Laske, O. (1981). Composition theory in Koenigs Project One and Project Two. Com-
puter Music Journal 5(4).
Laurson, M. (1996). PATCHWORK: A Visual Programming Language and some Musical
Applications. Ph. D. thesis, Sibelius Academy, Helsinki.
Laurson, M. (1999). Recent Developments in PatchWork: PWConstraints a Rule
Based to Complex Musical Problems. In Symposium on Systems Research in the Arts,
Baden-Baden. IIAS.
Laurson, M. and J. Duthen (1989). PatchWork, a graphical language in PreForm. In
Proceedings of the International Computer Music Conference, San Francisco, pp. 172
175.
Laurson, M. and M. Kuuskankare (2000). Towards Idiomatic Instrumental Writing: A
Constraint Based Approach. In Symposium on Systems Research in the Arts, Baden-
Baden.
Laurson, M. and M. Kuuskankare (2001). A Constraint Based Approach to Musical
Textures and Instrumental Writing. In Seventh International Conference on Princi-
ples and Practice of Constraint Programming, Musical Constraints Workshop, Paphos,
Cyprus.
Laurson, M. and M. Kuuskankare (2002). PWGL: A Novel Visual Language based on
Common Lisp, CLOS and OpenGL. In Proceedings of International Computer Music
Conference, Goteborg, Sweden.
Laurson, M. and M. Kuuskankare (2003). From RTM-notation to ENP-score-notation.
In Journees dInformatique Musicale (JIM), Montbeliard, France.
Laurson, M. and M. Kuuskankare (2005). Extensible Constraint Syntax Through Score
Accessors. In Journees dInformatique Musicale, Paris.
Lerdahl, F. and R. Jackendo (1983). A Generative Theory of Tonal Music. MIT Press.
Lothe, M. (1999). Knowledge Based Automatic Composition and Variation of Melodies
for Minuets in Early Classical Style. In W. Burgard, T. Christaller, and A. B. Cremers
(Eds.), KI-99: Advances in Artical Intelligence: 23rd Annual German Conference on
Articial Intelligence, Volume 1701 of Lecture Notes in Computer Science. Springer.
McCartney, J. (1996). SuperCollider, a New Real Time Synthesis Language. In Proceed-
ings of the 1996 International Computer Music Conference, San Francisco.
McCartney, J. (2002). Rethinking the Computer Music Language: SuperCollider. Com-
puter Music Journal 26(4).
Messiaen, O. (1944). The Technique of my Musical Language. Paris: Alphonse Leduc.
Trans. John Sattereld.
228
Bibliography
Miranda, E. R. (2001). Composing Music with Computers. Focal Press.
Mitchell, T. M. (1997). Machine Learning. McGraw-Hill.
Monzo, J. (2005). Encyclopedia of Microtonal Music Theory. http://tonalsoft.com/
enc/encyclopedia.aspx (accessed 24 May 2006).
Morales, E. and R. Morales (1995). Learning Musical Rules. In Proceedings of the IJCAI-
95 International Workshop on Articial Intelligence and Music, 14th International
Joint Conference on Articial Intelligence (IJCAI-95), Montreal, Canada.
Morris, R. and D. Starr (1974). The Structure of All-Interval Series. Journal of Music
Theory 18(2).
Motte, D. d. l. (1976). Harmonielehre. Barenreiter-Verlag.
Motte, D. d. l. (1981). Kontrapunkt. Barenreiter-Verlag.
Motte, D. d. l. (1993). Melodie. dtv/Barenreiter.
M uller, T. (2001). Constraint Propagation in Mozart. Ph. D. thesis,
Naturwissenschaftlich-Technische Fakultat I der Universitat des Saarlandes.
M uller, T. (2004). Problem Solving with Finite Set Constraints in Oz. A Tutorial
(1.3.1 ed.). Mozart Consortium. http://www.mozart-oz.org/documentation/fst
(accessed 25 May 2006).
Nienhuys, H.-W. and J. Nieuwenhuizen (2003). Lilypond, a System for Automated Music
Engraving. In Proceedings of the XIV Colloquium on Musical Informatics (XIV CIM
2003), Firenze, Italy.
Norvig, P. (1992). Paradigms of Articial Intelligence Programming: Case Studies in
Common Lisp. Morgen Kaufmann Publishers.
Ovans, R. and R. Davison (1992). An Interactive Constraint-Based Expert Assistant for
Music Composition. In Ninth Canadian Conference on Articial Intelligence, Univer-
sity of British Columbia, Vancouver.
Pachet, F. (1993). An Object-Oriented Representation of Pitch-Classes, Intervals, Scales
and Chords: The basic MusES. Technical report, LAFORIA-IBP-CNRS, Universite
Paris VI.
Pachet, F. (1994). The MusES system: an environment for experimenting with knowl-
edge representation techniques in tonal harmony. In First Brazilian Symposium on
Computer Music, SBC&M 94, Caxambu, Minas Gerais, Brazil.
Pachet, F. and O. Delerue (1998). MidiSpace: a Temporal Constraint-Based Music
Spatializer. In ECAI 98 Workshop on Constraints for Artistic Applications, Brighton.
229
Bibliography
Pachet, F., G. Ramalho, J. Carrive, and G. Cornic (1996). Representing Temporal
Musical Objects and Reasoning in the MusES System. Journal of New Music Re-
search 5(3).
Pachet, F. and P. Roy (1995). Mixing Constraints and Objects: a Case Study in Au-
tomatic Harmonization. In I. Graham, B. Magnusson, and J.-M. Nerson (Eds.), Pro-
ceedings of TOOLS-Europe95, Versailles, France. Prentice-Hall.
Pachet, F. and P. Roy (1998). Formulating Constraint Satisfaction Problems on Part-
Whole Relations: The Case of Automatic Musical Harmonization. In ECAI 98 Work-
shop on Constraints for Artistic Applications, Brighton.
Pachet, F. and P. Roy (2001). Musical Harmonization with Constraints: A Survey.
Constraints Journal 6(1).
Pachet, F., P. Roy, and D. Cazaly (2000). A Combinatorial Approach to Content-Based
Music Selection. IEEE MultiMedia 7(1).
Papadopoulos, G. and G. Wiggins (1999). AI Methods for Algorithmic Composition: A
Survey, a Critical View and Future Prospects. In Proceedings of the AISB99 Sympo-
sium on Musical Creativity.
Parrow, J. (2001). An Introduction to the -Calculus. In J. A. Bergstra, A. Ponse, and
S. A. Smolka (Eds.), Handbook of Process Algebra. Elsevier.
Partch, H. (1974). Genesis of a Music: An Account of a Creative Work, Its Roots and
Its Fulllments (2nd ed.). DaCapo Press.
Phon-Amnuaisuk, S. (2001). An Explicitly Structured Control Model for Exploring Search
Space: Chorale Harmonisation in the Style of J.S. Bach. Ph. D. thesis, Centre for Intel-
ligent Systems and their Application, Division of Informatics, University of Edinburgh.
Phon-Amnuaisuk, S. (2002). Control language for harmonisation process. In C. Anagnos-
topoulou, M. Ferrand, and A. Smaill (Eds.), Music and Articial Intelligence: Second
International Conference, ICMAI 2002, Volume 2445 of Lecture Notes in Computer
Science. Springer-Verlag.
Piston, W. (1947). Counterpoint. Norton.
Pope, S. T. (Ed.) (1991). The Well-Tempered Object: Musical Applications of Object-
Oriented Software Technology. MIT Press.
Pope, S. T. (1992). The Smoke Music Representation, Description Language, and Inter-
change Format. In Proceedings of the International Computer Music Conference, San
Jose.
Prusinkiewicz, P. and A. Lindenmayer (1990). The Algorithmic Beauty of Plants.
Springer-Verlag.
230
Bibliography
Puckette, M. (1991). Combining Event and Signal Processing in the MAX Graphical
Programming Environment. Computer Music Journal 15(3).
Puckette, M. (2002). Max at Seventeen. Computer Music Journal 26(4).
Rameau, J. P. (1984, orig. 1722). Treatise on Harmony (Traite de lharmonie reduite `a
ses principes naturels). Dover. translated, with and introduction and notes, by Philip
Gossett.
Ramirez, R. and J. Peralta (1998). A constraint-based melody harmonizer. In ECAI 98
Workshop on Constraints for Artistic Applications, Brighton.
Raymond, E. S. (2003). The Art of Unix Programming. Addison-Wesley Professional.
Recordare LLC (2006). MusicXML Denition. http://www.recordare.com/xml.html
(accessed 24 May 2006).
Roads, C. (1996). The Computer Music Tutorial, Chapter Chapter 18 Algorithmic
Composition Systemsand Chapter 19 Representation and Strategies for Algorithmic
Composition. MIT press.
Roy, P. and F. Pachet (1997). Reifying Constraint Satisfaction in Smalltalk. Journal of
Object-Oriented Programming 10(4).
Rueda, C., G. Alvarez, L. O. Quesada, G. Tamura, F. D. Valencia, J. F. Daz, and
G. Assayag (2001). Integrating Constraints and Concurrent Objects in Musical Ap-
plications: A Calculus and its Visual Language. Constraints 6(1).
Rueda, C., M. Lindberg, M. Laurson, G. Block, and G. Assayag (1998). Integrating
Constraint Programming in Visual Musical Composition Languages. In ECAI 98
Workshop on Constraints for Artistic Applications, Brighton.
Rueda, C., G. Tamura, and L. O. Quesada (1997). The Visual Model of Cordial. In
XXIII Conferencia Latinoamericana de Informatica (CLEI97), Valparaso, Chile.
Rueda, C. and F. D. Valencia (1997). Improving Forward Checking with Delayed Evalu-
ation. In XXIII Conferencia Latinoamericana de Informatica (CLEI97), Valparaso,
Chile.
Rumbaugh, J., M. Blaha, W. Premerlani, F. Eddy, and W. Lorensen (1991). Object-
oriented Modelling and Design. Prentice-Hall.
Russell, S. J. and P. Norvig (2002). Articial Intelligence: A Modern Approach (2nd
ed.). Prentice Hall.
Sandred, O. (2000a). OMRC 1.1. A library for controlling rhythm by constraints (2nd
ed.). Paris: IRCAM.
Sandred, O. (2000b). OMRC library Tutorial. version 1.1 (2nd ed.). Paris: IRCAM.
231
Bibliography
Sandred, O. (2003). Searching for a Rhythmical Language. In PRISMA 01. Milano:
EuresisEdizioni.
Sandred, O. (2004). Interpretation of everyday gestures composing with rules. In
Proceedings of the 2004 Music and Music Science Conference, Stockholm.
Schenker, H. (1956, orig. 1935). Der freie Satz. Universal Edition.
Schoenberg, A. (1943). Models for Beginners in Composition. Los Angeles: Belmont.
Ed. Leonhard Stein.
Schoenberg, A. (1964). Preliminary Exercises in Counterpoint. New York: St. Martins
Press. Ed. Leonard Stein.
Schoenberg, A. (1967). Fundamentals of Musical Composition. London: Faber and Faber.
Ed. Gerald Strang and Leonard Stein.
Schoenberg, A. (1979). Grundlagen der musikalischen Komposition. Universal Edition.
Schoenberg, A. (1986, orig. 1911). Harmonielehre (7th ed.). Wien: Universal Edition.
Rev. ed. 1922. Trans. by Roy Carter as Theory of Harmony. Berkeley and Los Angeles:
University of California Press. 1978.
Schoenberg, A. (1995). The Musical Idea and the Logic, Technique, and Art of Its Pre-
sentation. New York: Columbia University Press. Ed., trans., and with a commentary
by Patricia Carpenter and Severine Ne.
Schoenberg, A. (1999, orig. 1969). Structural Functions of Harmony (Second revised
ed.). Faber and Faber. Ed. Leonard Stein.
Schottstaedt, B. CLM. http://ccrma.stanford.edu/software/snd/snd/clm.html
(accessed 25 May 2006).
Schottstaedt, B. (1997). Common Music Notation. In E. Selfridge-Field (Ed.), Beyond
MIDI. The Handbook of Musical Codes. MIT press.
Schottstaedt, W. (1989). Automatic Counterpoint. In M. V. Mathews and J. R. Pierce
(Eds.), Current Directions in Computer Music Research. The MIT Press.
Schulte, C. (1997). Oz Explorer: A Visual Constraint Programming Tool. In L. Naish
(Ed.), Proceedings of the Fourteenth International Conference on Logic Programming,
Leuven, Belgium. The MIT Press.
Schulte, C. (2002). Programming Constraint Services, Volume 2302 of Lecture Notes in
Articial Intelligence. Springer-Verlag.
Schulte, C. and G. Smolka (2004). Finite Domain Constraint Programming in Oz. A Tu-
torial (1.3.1 ed.). Mozart Consortium. http://www.mozart-oz.org/documentation/
fdt (accessed 25 May 2006).
232
Bibliography
Selfridge-Field, E. (Ed.) (1997). Beyond MIDI. The Handbook of Musical Codes. MIT
press.
SICS. SICStus Prolog. http://www.sics.se/isl/sicstuswww/site/index.html (ac-
cessed 24 May 2006).
Siskind, J. M. (1991). Screaming yellow zonkers. Technical report. Draft of 29 September
1991.
Siskind, J. M. and D. A. McAllester (1993). Screamer: A Portable Ecient Implemen-
tation of Nondeterministic Common Lisp. Technical Report IRCS-93-03, University
of Pennsylvania Insitute for Research in Cognitive Science.
Sloan, D. (1997). HyTime and Standard Music Description Language: A Document-
Description Approach. In E. Selfridge-Field (Ed.), Beyond MIDI. The Handbook of
Musical Codes. MIT press.
Smaill, A., G. Wiggins, and M. Harris (1993). Hierarchical Music Representation for
Composition and Analysis. Computing and the Humanities Journal 27(1).
Smolka, G. (1995). The Oz Programming Model. In J. van Leeuwen (Ed.), Computer
Science Today: Recent Trends and Developments, Volume 1000 of Lecture Notes in
Computer Science. Springer-Verlag.
Supper, M. (2001). A Few Remarks on Algorithmic Composition. Computer Music
Journal 25(1).
Tack, G. and D. L. Botlan (2005). Compositional Abstractions for Search Factories.
In P. V. Roy (Ed.), Multiparadigm Programming in Mozart/OZ: Second International
Conference, MOZ 2004, Lecture Notes in Computer Science 3389. Springer-Verlag.
Taube, H. (1991). Common Music: A Music Composition Language in Common Lisp
and CLOS. Computer Music Journal 15(2).
Taube, H. (1993). Stella: Persistent Score Representation and Score Editing in Common
Music. Computer Music Journal 17(4).
Taube, H. (1997). An Introduction to Common Music. Computer Music Journal 21(1).
Taube, H. (2004). Notes from the Metalevel. Swets & Zeitlinger Publishing.
Taube, H. (2005). Common Music Dictionary. http://commonmusic.sourceforge.
net/doc/dict/ (accessed 24 May 2006).
Truchet, C. Backtrack Modules. http://www.ircam.fr/equipes/repmus/OpenMusic/
Documentation/OMUserDocumentation/DocFiles/Reference/backtrack/ (ac-
cessed 25 May 2006).
233
Bibliography
Truchet, C. OMBacktrack Tutorial. http://www.ircam.fr/equipes/repmus/
OpenMusic/Documentation/OMUserDocumentation/DocFiles/Reference/
backtracktutorial/ (accessed 25 May 2006).
Truchet, C., C. Agon, and P. Codognet (2001). A Constraint Programming System
for Music Composition, Preliminary Results. In Seventh International Conference on
Principles and Practice of Constraint Programming, Musical Constraints Workshop,
Paphos, Cyprus.
Truchet, C., G. Assayag, and P. Codognet (2001). Visual and Adaptive Constraint
Programming in Music. In Proceedings of International Computer Music Conference
2001, Havana, Cuba.
Truchet, C., G. Assayag, and P. Codognet (2003). OMClouds, a heuristic solver for
musical constraints. In MIC2003: The Fifth Metaheuristics International Conference,
Kyoto, Japan.
Tsang, C. P. and M. Aitken (1991). Harmonizing music as a discipline of constraint
logic programming. In Proceedings of then International Computer Music Conference,
Montreal.
Ungeheuer, E. (Ed.) (2002). Elektroakustische Musik. Handbuch der Musik im 20.
Jahrhundert, Volume 5. Laaber.
van Roy, P. and S. Haridi (2004). Concepts, Techniques, and Models of Computer Pro-
gramming. MIT Press.
Wiggins, G., M. Harris, and A. Smaill (1989). Representing Music for Analysis and
Composition. In EWAIM89, Genova.
Wiggins, G., E. Miranda, A. Smaill, and M. Harris (1993). A Framework for the Evalu-
ation of Music Representation Systems. Computer Music Journal 17(3).
Wikipedia contributors (2005). Diamond problem. In Wikipedia, the Free Encyclopedia.
http://en.wikipedia.org/wiki/Diamond_problem (accessed 12 December 2005).
Wolkow, A. (2005, Russian orig. 1939). Der Zauberer der Smaragdenstadt. Leiv
Buchhandels- und Verlagsanstalt.
Xenakis, I. (1992). Formalized Music: Thought and Mathematics in Composition (Re-
vised ed.). Pendragon Press.
Zhong, N. and Y. Zheng (2004). Constraint-Based Melody Representation. In U. K.
Wiil (Ed.), Computer Music Modeling and Retrieval: Second International Symposium,
CMMR 2004, Volume 3310 of Lecture Notes in Computer Science. Springer-Verlag.
Zils, A. and F. Pachet (2001). Musical Mosaicing. In Proceedings of the COST G-6
Conference on Digital Audio Eects (DAFX-01), Limerick, Ireland.
Zimmermann, D. (2001). Modelling Musical Structures. Constraints 6(1).
234