On Turtles & Dragons

and the dangerous quest for a media art notation system

ed. Version 1

Copyright: The Contributors icense: CC-BY-SA

II

Contents

III

IV

FOREWORD

ON TURTLES AND DRAGONS

Welcome to the record of our mission. We set off, brave knights in somewhat polished armour upon steeds fleet of foot, out to survey the lands of the realm and come back with int eresting stories, with special int erests in a few less explored regions off in one corner of the lands. We liken our foray to the construct ion of a map. As you know, at the edges of a map where none fear to tread, we believe there be dangers and things of which we cannot speak, and therefore prefer to remain silent. As we approach such areas of invest igat ion, we realise that our discussions have the habit of becoming self referent ial and complex, or begin to recognize that we paltry few are no match for the matters of concern. Dragons appear on a map where nothing can be found. And thus we will speak of dragons as those areas before which we ret reated, fearful of getting lost or losing too much time and being dist ract ed from our main area of int erest. It has been five days of Book Sprint. Six invitees came together in order to map out some parts of the territory of int erest. We have found several phrases for what int erests us and seem to find the problematic areas of notat ion for int eract ive syst ems to be suitably general. And so it was that we came together armed only with a handful of ideas each and fingers ready to type. As Paul Erdos liked to say, our "brains are open!" The five days were a cont inual cycle of discussion, collect ion, sorting, writing, revising and back to it. It was good to have help because the invest igat ion deep into the forest of int eract ivity turned out to be harder and more plagued with difficulties than we ever could have imagined. This was our goal, and the later chapters reflect our invest igat ions and att empts to communicate to one another the ideas and problems with which we worked and fought to look at the problems of notating int eract ivity. A phenomenon encount ered early on in the discussion of notat ion is the "bott omless stack of turtles" effect: the infinite regress of informat ion layers, all of which could be expressed through some kind of notat ion. Peel off one successfully notated strata and beneath it lies another, and beyond that still another. Yet not all these levels of focus are equally useful for the expression, discussion, or transmission of a part icular piece of work to a specific audience. In contrast to dragons, these reptiles are friendly and slow and immensely patient. As we outline below, we came to see the stack of turtles, a metaphor for recursion and layering, as a way of navigating the complex hierarchies of notational techniques and points of view. For us the turtle has become a symbol of sympathy and subject ivity, the walls are anoin ted with cart oons of them and our imaginat ions are filled with turtles as ways of thinking things. This book is a manifest at ion of what happened over these days. We hope it is of int erest, that the things we saw and the conversat ion we had will be relevant to more than a few other souls. We also hope that some other brave knights will mount their steeds and, perhaps armed bett er to conquer cer tain dragons, can illuminate some corners of the realm. The map will lose some of its white patches and give up some of its secrets. And reveal a few more dragons themselves.

WHAT IS NOTATION?

WHAT ARE WE EXACTLY TALKING ABOUT?

"Everyt hing is brown"1 ". ...- . .-. -.-- - .... .. -. --. .. ... -... .-. --- .-- -. " As with so many things, it is best to start where we feel most comfortable. And that might as well be breakfast. "Take 2 chicken eggs per person and break them into a bowl. Heat a pan to medium temperat ure. Add salt, pepper, and one teaspoon of milk per egg to the bowl. Mix for 2 minutes. Put one teaspoon of butter per egg into the heated pan. Add the mixt ure from the bowl. When the mixt ure starts to solidify scrape and mix with wooden spoon until all of the mixt ure is firm. Serve with toast." Completely obvious as to what it will produce, this highly formalised language of a recipe is an easily grasped notat ion. It is readily apparent what will come out, but it is of int erest to see how the proport ions are decided. The text form de scribes the process quite exactly, allowing litt le room for error or int erpretat ion. This might be seen as a good notat ion for a very concrete process.

The Crux of Complexity

This book arose as an att empt to collect and collate various ideas and problems around notat ion that many of us have been dealing with on several levels. We have processes of varying complexity, and we would like to work out ways to dis cuss them that move beyond handwaving to looking at the deepest level of code. For example: How do we remember the struct ures of the facilitat ion syst ems that we have developed and pass them on to others? How to discuss the possibilit ies for int eract ion, and the way a media should be playing, given the act ions of possible visitors? How to invest igate the inner workings of an experience to see whether it makes sense, without building it completely in advance? We want to think of notat ion as an abstract ion, a simplificat ion, and int uitive or studied way of writing somet hing down that succinctly summarises the import ant points of a given situat ion, process, object or syst em. And then we woke up. The borders of the phenomenon of "notat ion" behave like Tant alos' fruit, hanging just in reach until we at tempt to reach for them. As soon as a limiting feat ure seems to be within reach, an example or somet imes an argument will betray that limiting power from the just-discovered feat ure, and lift those borders to the sky. For the purpose of meaningful exchange on the subject of the phenomenon of "notat ion," a common terminology is as indispensable as a discretionary border to the field of discussion. We do not want to bite off more than we can chew.

Some Definitions of a Sort

The set of attributes used here is put together for practical reasons only. It is not chosen completely arbitrarily but in search of a "least common denominator" which may appear incorrect or unjust ified in some (we hope rare) cases. Some dragons will have to be slain at another time. For the sake of clarity in expression, "notat ion" will, unless specifically stated otherwise (or obvious from the cont ext), be used to refer to a concrete example, a concrete use of a "notat ion syst em." A notat ion syst em consists of all the pos sible forms of represent at ion, rules and dependencies that make sense within that notat ion syst em. Within a given nota tion syst em, all possible forms of represent at ion will be referred to as "vocabulary" of that syst em. The sum of all rules and relat ions possible within a notat ion syst em will be called "inner logic".2 Many names exist for a given piece of notat ion in the respect ive notational syst em. A recipe, a score, some pseudo code, a script. New possibilities require that we ext end a piece of notat ion. When the microwave oven arrived, the terminology for its use had to be created, and before then the electric beater. As recipes are written in a slightly for malised way using natural language, such addit ions were easy to make, and the notational syst em of recipes became a lit tle larger and more complex. When tape players as a new musical instrument or ext ended playing techniques were added to concert music performances, an ext ension of musical notat ion of staves had to be found. The semantic content of a given piece of vocabulary is of import ance. The same phrase could mean similar yet very significantly different things when imagined as dance, acoust ic or mathematical notat ion. For instance the sym bolic statement A-> B, where we say "from A to B" might refer to a movement of a dancer across the stage, the change 4

of an actor between emotional states or a mathematical funct ion between two sets. It is vital that we agree upon such semant ic cont ents - fundamental misunderstandings can arise if not. This is perhaps one of the critical parts in the crea tion of a notat ion syst em, or the decision to use a part icular notat ion syst em. Construct ivists never tire of claiming that meaning is negotiated, and it is by no means less true in this case. This will be seen later as we examine several examples, where the semant ics of each symbol need to be explained or discussed. When thinking about notat ion as a phenomenon of human behaviour, some examples pop up right away. Mus ical scores, geographical maps and mathematical diagrams might be amongst the most common and widespread forms. These and all other more or less exotic, ancient and helpful uses of notat ion syst ems all have their own int ernal logic set of rules, their own set of "vocabulary" of possible different signs and symbols, and their own purpose. Each notat ion sys tem is arbitrary when viewed from the outside and a complete universe, a "reality" with its own rules and logic from within. Each of these discretionary human-created realities is focused on very specific aspects considered relevant and deliberately ignores or at least leaves open others. We can look at a notat ion syst em not as an answer to a specific quest ion but as tool to produce answers to a specific type of quest ion/problem and to preserve, communicate, or reproduce those relevant aspects. This highlights their essence, the indispensable core attributes that a syst em must have to be a notat ion syst em. A syst em becomes a notat ion syst em when it has a working set of inner logic (rules) using a set of abstract re present at ions (vocabulary) of aspects of potent ially universal experience deemed relevant to be different iated between, preserved or communicated about. This definit ion includes all communicat ion, all language, and especially all forms of written language (here is a dragon!). So the only reasonable thing to do is to add restrict ions to the list of necessary attributes: the purpose of com municating about somet hing specific while int ent ionally not communicating about somet hing connect ed. This leaves out communicat ion per se (as an end in itself) as too general to be useful to be discussed here. The restrict ive prerequisite of being ext ernalised - of a syst em existing independently from the person who uses it - has to be seen in the same light: it keeps our subject specific enough to be useful.

The "Is it a Notation System?" Test

We somet imes found that we were speaking of things that, upon closer inspect ion, were not really notational syst ems. We refined our idea and thought that, in order to keep our heads clear, we would try to find a process to help identify the quality of somet hing as a notat ion syst em. The following test is int ended to be applied in a sequent ial fashion to a given syst em to see how well it conforms to our idea of a notational syst em. Testing syst ems for their notation-system-ability (expanded below): 1. 2. 3. 4. 5. Is there a working set of inner logic? Is there a "vocabulary"? Are the aspects potent ially accessible to at least one ent ity/person? Are other aspects int ent ionally left out? Is it an overlooked dragon?

In det ail: 1. Is there a working set of inner logic? Each notat ion syst em constitutes only a sect ion of a larger "reality" and creates its own litt le world made only of what is relevant from the perspect ive of the inner logic. Even if the described piece of reality is lacking logic, the notat ion syst em describing it doesnt. A syst em to keep track of completely random and unrelated events (for in stance) is consist ent and logical in itself, and has struct ural and synt actical logic. 2. Is there a "vocabulary" There are several criteria that come together here. Does it speak about anything? Are there things that we can notate in some way in this syst em? If there is nothing in the vocabulary, then there is nothing to say with it. Then the next test is whether it gets easier to talk about the area that is being notated, that is, that the syst em uses an abstract ion of what it describes. 5

A descript ion of an object or a circumstance by using the object or circumstance itself is of course not an abstrac tion. If the vocabulary notating the syst em is not simpler than the syst em being notated, then the syst em is not in a meaningful way a notat ion syst em, it is not part of a focused "vocabulary syst em" as the descript ion and the described are identical. Such notat ion syst ems can be found for example in 1:1 maps where the map is of the exact size and nat ure as the territory. These are not only useless, but also represent the limit of the smallest possible level of abstract ion (none) and the highest possible level of det ail and complexity.3 3. Are the aspects potent ially accessible to at least one ent ity/person?4 The use of a notat ion syst em which cannot be int erpreted by anyone at all makes absolutely no sense. While the IChing sticks might reflect and thus notate my emotional state, we doubt that anyone can act ually read that nota tion. The minimum requirement of at least one potent ial addressee can be seen in the example of personal shorthand notes. Usually the circle of potent ial "readers" is greater, and maximisat ion of the number of potent ial read ers is a core motivat ion for the use of standardised notat ion syst ems (e.g. mathematical, musical, geographical orient at ion notat ion syst ems). We will look at this in more det ail below in the chapter Beethoven's Deathbed. 4. Are other aspects int ent ionally left out? This is very closely related to the condit ion of abstract ion and the depth of representable level of det ail. In the process of abstract ion, a decision must be made on the relevance of pieces of informat ion, deciding whether to leave them out and concentrate on others. For example, a piece of musical notat ion (a score) may cont ain instruc tions for the handling of a specific instrument (a pianofort e) but not the manufact urer of the instrument (Bsen dorfer), the size or form of the room in which it is to be played (chamber music hall) or the individual person to play the work (Rubinstein). All these fact ors can and probably will influence the produced outcome, but are not considered relevant from within that specific musical notat ion syst em. The element of int ent ionality is import ant for a reader as it necessary to know that what was omitt ed was int en tional, rather than being left out accident ally. 5. Is it an overlooked dragon? Does the inclusion of a syst em in the notat ion syst ems under discussion have potent ial to increase insight into the phenomenon or pract ice of notat ion, or does it foreseeably open a door to an unanswerable argument, which at best can only lead to frustrat ion, or worst to a metaphysical whirlpool? If the latter, it is a dragon, and we will not slay it here. 1. Universal "answer" to everyt hing, perceived in the Oliver Wendell Holmes, "Mechanism in Thought and Morals," Phi Beta appa address, Harvard Univerist y, June 29, 1870 (Bost on: J.R. Osgood and Company, 1871) ^ 2. For those who are aware of formal language theory, the vocabulary is the set of well-formed formulas that can be formed in a given formal language, the inner logic is the synt ax of the language. 3. In the sense of olmogorov Complexity 4. Wittgenstein's Private anguage Argument is considered a dragon in this cont ext. And left to fut ure adventurers and slayers.

FORMS OF NOTATION
Si (como afirma el griego en el Cratilo) el nombre es arquetipo de la cosa en las letras de 'rosa' est la rosa y todo el Nilo en la palabra 'Nilo'. The Golem (Borges 1967) Every notat ion syst em is preoccupied with the most effective ways of communicating informat ion, so it's rules and shapes depend greatly on the nat ure of the Int erpreter. The first technical quest ion then should be: who is this notat ion for? Who will get it? If we att end to formal considerat ions, the main dist inct ion is obvious and self-explanatory: humans and machines are very different int erpreters. Then there is the quest ion of precision. For the human int erpreter, the relationship between score and perfor mance is directly proport ional to the value of the performer or int erpreter of a given notat ion: we are necessarily hung up on the exact proport ions of a medical prescript ion but indulge to improvisat ion when following a cooking recipe. In terest ingly enough, in the cont ext of art istic performance we demand a precise execut ion of the score, but we dedicate our senses to everyt hing in the performance that is not in the code. On the other hand, ext reme levels of precision in a given field can result in absurdity, as the Argentinian writer Jorge uis Borges illustrates in his famous short story On Exact itude in Science. ike in the book that inspired it, ewis Carrol's Sylvie and Bruno Concluded, the science of cart ography becomes so ambitious that "only a map on the same scale as the empire itself will suffice", rendering the whole ent erprise useless and the era of Cart ography terminated. While we can say we have produced an almost 1:1 scale map that is indeed very useful -Google's satellite photos of the earth are literal, though its use of the Mercator project ion inherits its blind spot around the poles- we can handle it thanks to another kind of notat ion, JavaScript and XM. We see here a form of 1:1 map that has become possible through the virt ualisat ion of informat ion and the ease of navigat ion that does not require the map to be physically pre sent to get in the way. The Era of Cart ography might be terminated but the tool does not yet defy the purpose. The following examples of notat ion are described here to stress the differences between the main universal sys tems and learn how to make the letters of rose into the rose, and cont ain every drop of the Nile between its four lett ers.

Gestural Notation

Gestural Notation

Ancient Egyptian cheironomy: Wind instrument players are being guided in the music by hand signs (2563 B.C.E.) Apparently the beginning of everyt hing notation-wise, cheironomy is the art of using hand signals to direct a music performance. Whereas in modern conducting the notes are already specified in a written score, in cheironomy the hand signs indicate melodic curves and ornaments. Cheironomers fell out of grace when modern conducting techniques developed in the XVII cent ury. Before arajan, the role was fulfilled by any member of the band that happened to carry a stick -sometimes the violinist with his bow, others a lutenist moving the neck of his instrument. Today it is mainly used as inspirat ion for those less metrically struct ured composit ions which require individualized direct ion to specific players. It could be int eresting to see what we could do of it today, maybe a funny exercise that clarifies one of the main or core purposes for using notat ion and notat ion syst ems: to transport informat ion in an understandable, agreed manner, even if we only have ourselves to agree with at the end of the process. This will come up further down when talking about notat ion for the purpose of clarificat ion (simplificat ion) and reflect ion in the next chapter. Hand and other nonverbal body signals are also well known in various subcult ures including the Italian criminal cult ures, where they have the ex pressive power akin to speech.

Scientific Notation
The scient ific community is fond of notat ion, and for very good reasons. Mathematicians, physicists, ast ronomers and chemists not only need to navigate impossibly huge numbers and struct ures but also handle slippery concepts like "in finite" with great precision. Also, the scient ific tradit ion of open development requires that the abstract ions innate to these fellows' thinking patt erns are cont ained within reach of their colleagues and ext ended to everyone involved in the same line of research, independently of their proprietary tools or incompatible non-Roman scripts. They scient ific pursuit is very often a cross-gender affair, due to the fluid communicat ion in between those subjects that used to be called Natural Science until the industrial revolut ion and a Venn diagram of its notat ion syst ems would reveal a high degree of promiscuity, with mathematics at the center of it. Quant um mechanics theory uses a kind of specific notat ion called Bra-ket (also known by his creator's name, -Dirac) but it can also be used to denote abstract vect ors and linear funct ionals in pure mathematics. Probability theory and statist ics has its own convent ions but as added to the standard mathematical notat ion syst ems. All scient ific syst ems resolve a need for clarity, concision and reduct ion (though, surprisingly enough, ambigu ous expressions do appear in physical and mathematical texts). This abstract origami cont ains massive volumes of data in a few characters but, as it is meant to survive the evolving fashions and technologies of its time, it must be open en ough, and very expandable. As a cautionary tale for those that are not, there is the ungratefully ret ired (and named) zen zizenzizenzic, an early notat ion representing the eighth power (as in the zenzizenzizenzic of 7 would be the power 7 8) which, according to 16th cent ury welsh mathmatician Robert Recorde doeth represent the square of squares squared ly. Such notat ion has now fallen away to the role of a hist orical oddity, and probably rightly so. 8

The decimal system and the derived metric system of notation has achieved widespread acceptance. There are of course other disciplines that exist in the int erstices of science research that do not share the exact ing nat ure of physics or mathematics, and their ever changing notat ion syst ems reflect the illusive nat ure of their fields. In psychology, for example, the need for a standardized evaluat ion syst em has produced efforts like the Brief Psychiatric Rating Scale (BPRS), the Scale for the Assessment of Positive Symptoms (SAPS) or the Scale for the Assessment of Negative Symptoms (SANS) along with other measures of psychopathology, without any of them being completely satis factory. It would be claimed by many in the mathematical or "exact" sciences that such pieces of notational terminology are insufficiently useful to be called scient ific at all.

Musical Notation
The late Canadian pianist Glenn Gould produced two mast erpieces of musical genius performing the same piece with al most three decades in between. His 1955 debut recording of Bach's Goldberg Variat ions made him a legend at 22, quickly becoming one of the most famous piano recordings of all times. When the second one was released in 1981, it did so a few weeks shy of his death of a stroke at age 50. The two recordings are separated by a life of int ense creative power and self-destructive addict ions and couldn't be more different, both in tone and expression. In a new album that includes both versions, Washington Post music critic Tim Page says "Almost anything you could say about Glenn Gould you could say the opposite and have it be somewhat true." And yet the notat ion behind it remains exactly the same. Art istic notat ion is tricky, and it requires a very special kind of int erpreter: an artist. This is to say, not only one with a previous specialised knowledge -just like with scient ific notat ion, one that has learned the language- and can execute the score with impeccable precision but will also embed the performance with his own int uit ion, imaginat ion, charisma, ex perience and insight. Those variables, that we casually call talent or even genius, cannot be notated or prescribed; they belong to the flexible and ever ambiguous field of art istic int erpretat ion and expression. We can say that the most im port ant part of the art istic performance is un-notateable, and any effort to include has been exhaust ed without success. That doesn't mean that such efforts haven't produced many int eresting outcomes; the room for art istic ex pression seems to be as wide as the number of the notators dedicated to the task. An ext reme example would be the in famous Faeries Aire and Death Waltz, a comic score by John Stump, peppered with congenial but impossible instruct ions like "Release the penguins", "Remove valve" or a "Go real fast - sleepage may occurr."

More int erest ingly, John Cage, who famously struggled with traditional notat ion all his life, produced different syst ems for very unorthodox orchestras, somet imes with hilarious and game-changing consequences. There was HPSCHD, a fivehour craziness involving seven harpsichords, 52 tapes of computer-generated sounds and 64 slide project ors, and there was the Etudes: Eleven or twelve years ago I began the Freeman Etudes for violin solo. As with the Etudes Australes for piano solo I want ed to make the music as difficult as possible so that a performance would show that the impos sible is not impossible and to write thirt y two of them. The notes written so far for the Etudes 17 32 show, 10

however, that there are too many notes to play. I have for years thought they would have to be synt hesized, which I did not want to do. Therefore the work remains unfinished. Early last summer ('88) Irvine Arditti played the first sixt een in fifty six minutes and then late in November the same pieces in forty six minutes. I asked why he played so fast. He said, "That's what you say in the preface: play as fast as possible." As a result I now know how to finish the Freeman Etudes, a work that I hope to accomplish this year or next. Where there are too many notes I will write the direct ion, "Play as many as possible."

Autobiographical Statement (John Cage, 1990)

Dance Notation
The most disciplined of performing arts, ballet, has produced many notat ion syst ems of its own, but the most fascinat ing and instruct ive from a technical point of view is without a doubt Vladimir Stepanov's work for the Imperial Ballet of St. Petersburg, 'Alphabet des Mouvements du Corps Humain. Rejecting the pict ograph methodology that was the choreographic standard since the XVIII cent ury, he chose to emulate the more precise musical score, and deconstruct ed every step into the most element ary movements a single part of the body can produce, encoding each movement into a "note".

11

The first computerized notat ion syst em, designed a cent ury later by Eddie Dombrower for the Apple II, displayed an an imated figure on the screen, following the choreographer instruct ions. Unsurprisingly, it didn't go very far. Today Stepanov's archives are displayed as a museum relic and his art has been replaced by a real time notat ion syst em of 1:1 scale: video.

Painting Notation
In the fine arts, many art istic notat ion syst ems have curiously betrayed the ambit ion of eliminating the artist from the art process through mechanical means, like Peter Benjamin Graham's New Epoch Art Notat ion. Apparently the world's first high level visual notat ion syst em for painting, NEA "separates the act of conceiving an image from the act of paint ing": NEA composit ions are known as Sets. Sets use a unique 'thematic' struct ure called thematic orchestrat ion which is closely related to chaos theory in physics. This method of drawing utilizes a process apart from con vent ional abstract ion. The raw subject matter is synt hesized into a theme. A theme is a configurat ion of lines which embodies what the composer feels is the essence of the raw subject. The paintings are then 'grown' by sensitively repeating and overlapping the themes in a rhyt hmic manner always with slight differences building up a complex latt ice of enclosed organic and asymmetrical shapes.' (see tessellat ion) 'The theme is the 'visual title' of the work. iterary titles are taken from the raw subject or from int uitive literary associat ions that may occur during the act of composit ion. Every line and every shape put where it is on purpose, no happy accidents, no random use of ges ture, and no reliance on drips or splatters. Every shape asymmetrical, and unique in form; its nat ure and posit ion related to every other; and its posit ion, the overall struct ure, never repeating the ent ire evolut ion of the image during its making, also premeditated and in fact, cont aining much of its meaning; a composed image that alt hough subject to det erminism, will never repeat itself even if the ent ire process of making be gins with identical working condit ions. The child of relatively simple rules that can be applied almost ef 12

fortlessly be people with reasonable sensibility and craft skill but who NEED NOT BE ARTISTS; the part icipa tion of professional artists only serving to increase further the diversity of invent ion. New Epoch Art, Peter Benjamin Graham (Int erACTA No 4 1990)

New Epoch score of Grainger Country by Peter Graham 1979

Spatial (as in, non-textual) Notation

In notat ion, different formats will use the same data for very different purposes. When the goal is to highlight specific relationships between elements (like dist ance, size, connect ion or differences) or stressing a point of view, nothing beats the graphical or spatial syst ems, a favorite of the general public for its approachable and colorful results, but also of ad vertisers, preachers and politicians for it allows for a great deal of manipulat ion. The most common and widely used example of spatial notat ion is of course the map, whether physical or abstract. Thanks to the fashionable field of data visualizat ion and the amazing tools developed in the last few years, we even see these days we see mixt ures of both, like in this expressionist map of human povert y created with World Mapper.

This map is quite emphatically not the territory, but is still truthful and still a map. The natural borders are repurposed as a cont ainer for an unexpected rating value; the proport ion of the world populat ion living in povert y. Precision, 13

though, is here sacrificed for a higher impact. If we want ed to use the data for purposes other than the denunciat ion of a humanitarian crisis, we would be bett er off with a numeric notat ion. Maps are double-edged tools, for on top of manipulat ion, they also leave room for endless misint erpretat ions. In They Rule, a project that illuminates the invisible networks of corporate power, the human eye inevitably sees connec tions that don't exist. This is not the int ent ion of the artists but, as it usually happens with maps of the underground where the layout sacrifices realism for the sake of clarity, we derive false informat ion under the apparent but false prem ise of proximity.

They Rule Despite the high att ent ion paid to data visualizat ion projects nowadays, not all spatial notat ion serves sociocult ural agen das; somewhere in between artist and spatial syst ems lie architect ure and engineer diagrams. We see them every day, from the fire exit maps that welcome us in malls to eonardo da Vinci's studies of the human body, though it was only after the Industrial Revolut ion that such kinds of notat ion became standarized. We will look at these forms more deeply in the Purposes Chapter.

Computer Notation
A successful notat ion syst em begins with formal models of that which it wants to notate. This is an int ellectual exercise we've been performing from punching cards to HTM5, with varying levels of abstract ion and it is the task that Vladimir Stepanov put himself to when trying to register the movements the Imperial ballerinas with absolute precision. It is hard not to wonder what such a visionary would have done with a lapt op on his hand but wouldn't be far away from the dancing robot that awaits the reader a few chapters away. In cert ain ways, most code is not so different from formal scient ific notat ion, though the binary number syst em uses powers of two instead of powers of 10, and the computer doesn't understand what it int erprets. Some people are capable of writing straight into computer code without running it, the way Beethoven wrote music without hearing it but the opposite will not occur. An image file might as well be the notat ion of the image it generates but, while only a machine can render it visible to us, the machine itself remains blind to what is in it. That is, until the Singularity. It is construct ive to remember that computers don't read between the lines but they do fill in the blank. A mac hine is literal, not metaphorical, and is always going to int erpret the code exactly the way it was written, whether it ser ves our purpose or not. There are specific semant ic int erpretat ions of what cert ain things mean, and there may be in 14

determinism as a hidden part. This int eresting problem -and its consequences- are illustrated in a subt le difference between the almostidentical visual programming languages1 Pure Data and Max/MSP.

Two simple Pure Data or Max patches. The left example has indeterminacy as a Pure Data patch, the standard solution is shown on the right with a t b b ("trigger bang bang") object. A very simple patch is created with a single bang that is connect ed to two print objects labelled A and B. This is a de finite computer program and will be run by the computer without any further int ervent ion by a person to implement the finer det ails. In Pure Data, two outcomes are possible. They are
A: bang B: bang

and
B: bang A: bang

In the technical explanat ion of the semant ics of Pure Data patches, it is stated that the choice that is made between these two options is purely nondeterminist ic, there is no decision. In the current implement at ion, there is a specific de terminat ion; the connect ion that was created first is used first. But this is not part of the program specificat ion and later implement at ions might change this. This is an int eresting way for the syst em to fill in the blanks. As indicated above, the trigger object can be used to det ermine which event should happen in what order. In Max/MSP, the filling in of the blanks is more literal. The blank space and the posit ion of objects are seman tically significant. The object to the right is fed data first. So in that syst em, the result will always be
B: bang A: bang

Stay put: the trickiness of computer notat ion will be described in more det ail further within these pages. 1. See the discussion: http://en.wikipedia.org/wiki/Visual_programming_language

15

PURPOSES FOR NOTATION

But suicides have a special language. ike carpenters they want to know which tools. They never ask why build... Ann Sexton After looking at some basic terminology and clarifying some rules, forms and phenomena of notat ion syst ems, we finally get to the point where we can discuss why build. Or, in other words, for what human ent erprises is notat ion useful and int eresting? This we have discovered so far: the purposes can be numerous.

To Understand
In essence all forms of notat ion transport data from one medium to another; and at the core of every transit ion there is the mind being both actor and audience. For that reason, our first example of notat ion has an audience of one, with the purpose of creating a personal overview through ordering, clarifying and reflecting on a complex idea, not ion or topic: the infamous note to self. We are all familiar enough with this concept, and it is fair to say it is not learned but int uitive: the universal instinct to write down a list of represent ative values and look at it, in search for somet hing that we haven't seen yet. Since the primary purpose is not to communicate an idea to a larger group but to create a broadened understanding for ones own self, we can get away with freest yle, as long as we can read it later. Int erest ingly enough, our freest yle has a penchant for lists and diagrams.

One of the stages of this chapter being developed. When the process ext ends beyond one's own eyebrows to involve other people, the notat ion must be legible. Used effectively, doodles can work as well as very det ailed maps, if they are sufficient in the cont ext to organise the 16

group and coordinate their act ivities. A group of various specialists needing to be coordinated will have few pieces of ter minology in common, so some struct ures that can be individually annotated for each specialist need to be found.

This storyboard cont ains instruct ions for moving camera and objects to coherently help a film team est ablish what is going to happen, and what they need to do. This notat ion syst em aims at giving (technical) instruct ions as well as an overview. The various specialists will however use the same syst em in various different ways: while the camera crew will use it to group shots by angles to minimise camera changes, the costume team will use it to check that all pieces are ready to go and will coordinate with cont inuity to ensure that the right stains are in the right places.

To Navigate
Often data becomes so complex that we need a process of compression to get an overview of the whole syst em. The most literal notat ion syst ems are probably maps. Taking the whole world and reducing it to a globe to be held in one's hands, or projecting it flat to be kept in a book, spread on a table, or hung on a wall allows us to identify the relationships between the different elements that configure our planet. It is valuable in a way could not be gathered through lists of meters, kilometres and other terrestrial conven tions of land.

17

In order to make one's way through a territory, a mud map scratched in the dirt showing the trails and trees as navigat ion may suffice. We have a way with maps; even confront ed with a path to climb a rock wall with the safety bolts indicated, or the posit ion of waypoints in an orient eering race, maps are universally understood or at least easily learned. et's not forget, the use of maps is learnt and their standard elements are soaked in cult ural and political cont ext, such as which way is up, or what region is at the center. More conveniently, maps also allow for multiple layers of data to overlay and juxt apose in engaging ways with out redundancy, as in collecting traffic networks, noise levels and rubbish collecting patt erns in the same city, or study ing and monitoring human movement in public spaces such as malls or airports. The more complex the ideas, act ions or processes are, the stronger the need for notat ion (and/or the higher the level of abstract ion). Somet imes the creat ion of a cert ain mapping syst em requires a lot of effort to develop an idea that looks simple in retrospect, such as the multit ude of globe project ions onto two dimensional maps where the precise dist ort ion of the Mercator project ion allows angles to be read from the map and used immediately for navigat ion. Anyone int erest ed in this matter can dive into the extraor dinary How to Lie with Maps, by Mark Monmonier. We'll call it dragons for now. When the lands of unmanageable proport ion are symbolic values, we tend toward the kind of compression that charact erizes scient ific notat ion syst ems. Much of science consists in digging and exploiting patt erns. For augmenting precision in evaluat ion, a very large amount of data must be collect ed, requiring a high level of compression. For instan ce, Euler's second law of mot ion states that the rate of change of angular moment um (also denoted H) about an axis is equal to the sum of the ext ernal moments M about that point but, for calculating purposes, it is bett er used in this for mula:

Universality facilitates the cross-pollination of scientific ideas: as long as we can show that the axioms are true and stay within the range of applicability (physics, economics, electrical circuits), we can use this formula for many other purposes without even having to understand what's in it. The process of abstraction from specific problems to a notation allows us to apply abstract and general results and techniques that have been proven to be true without having to be proven in each specific field individually. We can use the same tech niques to solve this differential equation whether it be describing momentum, population growth or the amplitude of an oscillating circuit.

For calculat ions, one often uses Scient ific Notat ion which is based on powers of the base number 10 (as opposite to the binary syst em, which is based on the powers of 2 and only uses the numbers 0 and 1) and researchers use it to navigate through very large (or very small) numbers. Instead or writing 1,230,000,000 and 0.00000000123 we write 1.23e9 and 1.23e-9 and have a more concise and easily comprehensible way to talk about these values. More import antly, notat ions need to convert concepts with reluct ant cont ours and unstable shadows into man agable units, what eibniz liked to call "ideal ent ities," and then proceed to pin them down. The examples range from Stephen Wolfram's notat ion for two state cellular automata to Georg Cant or's transfinite numbers, of which (omega) is the lowest and (equivalently) (Aleph-null) the first with a series of Alephs above that. In both cases, symbolic values 18

are used as a variable to find out more about what it is when it is too messy, too big or too dist ant for us to see, and are subject to calculat ions and logical processes that reveal the edges of those dark spots. Indeed, there is still a lack of clar ity about somet hing called the Cont inuum Hypothesis exploring the way that infinities int errelate, a dark spot that may be more delineable in the fut ure.

To Share and Archive

Any kind of archiving project offers a very specific kind of problem: preserving informat ion in a way that could be meaningful for fut ure generat ions those unknown ent ities with crazy devices and impossible slang. For a cult ure fond of tradit ion, there is a lot to remember: processes, ideas, concepts, designs. While the mechanics of archiving material should be simple enough - reduce the work to its atomic indivisible parts without renouncing any of its cont ent - institu tions are plagued with quest ions of authenticity, readability and universality. Mainly: How much can you reduce a work before turning it into another work? What quotat ion syst em will remain stable enough so that we can int erpret it in the years ahead? How do we ensure the code for int erpreting will remain in our cult ure? This has become a significant problem in the realm of electronic art over the past few decades as devices become ob solete or corrode, replacement parts become unavailable and data formats unreadable. Scient ific and everyday data is not immune. The data capt ured on the first Viking missions to Mars in the 1970s is largely lost to unstoppable bit rot 1, piles of self-burnt CDs and DVDs are succumbing to ultraviolet radiat ion as we speak.
Whether or not a form of notation lends itself to archiving depends on a large variety of factors, often dependent upon the length of time that one would like to think of as being culturally relevant to keep the information. Notations can easily become as complex as the thing they are trying to simplify.

19

Gerhard Dirmoser's "Erzhl uns Linz" diagramm.

Gerhard Dirmoser maps collect ions by using diagrammatic represent at ions of exhibit ions. The diagram above shows material from an exhibit ion at the Nordico2 in inz which exhibits 600 pieces that have personal stories att ached to them. He reduces it to a diagram, abstracting the struct ures in the show by using verbs that appear in the texts of the personal stories and created co-relation between them in 4 sect ions; collecting, exhibiting, remembering and (story) tell ing. He also int egrates a timeline around the periphery of the diagram. This combinat ion of several diagrammatic tech niques to give a meaningful whole is one of his charact erist ics.

20

To Engineer

To Engineer
When designing a device to be built, the engineer or designer aims to include enough informat ion to allow the device to be built exactly as it has been planned and analysed. A high level of det ail is required in order to let the engineer know that the calculat ions relating to strength, movements, volt age, etcetera will hold. The Engineering notat ion also uses aggregat ion, inheritance and funct ional hierarchies in order to most carefully and clearly communicate the design. Such precision in design often calls for a mixt ure of spatial and scient ific syst ems at a number of levels of det ail. Electrical cir cuit boards, for instance, are int eresting kind of maps where every element describes its role in a symbolic but also ex plicitly physical fashion. Similarly, some computer software such as Pure Data use notat ion that both describes and generates the output; here the map and the territory are in some way int erchangeable. An engineer will use a variety of notat ions to describe the object being planned and built. Two-dimensional project ions of the thing being built are common. These will correspond to some agreed-upon standards, whether they use the nota tion of technical drawing for steel parts, or the standards for multiple cross sect ions in boatbuilding. Mathematical nota tion will be used to take the dimensions of the object and calculate import ant properties such as the righting moment, strength of beams, the damping effects of an induct or. Every syst em will be chosen so as to have a sufficient level of de tail for all concerned.

Paradoxically, engineers and invent ors also archive their ideas to prot ect them from being used or copied, de scribing a process, product or design so it may not be exploited without permission in the fut ure. Patents, for instance, are int ended to describe an innovat ion so as to allow people to use it, but always licensing it from the patent owner, whilst exposing the det ails of funct ionality for educational purposes. Because of its economical (or simply antisocial) am bit ion, the language is often as imprecise as the law allows for, trying to embrace as many uses or variat ions of a given process or struct ure as there could be, such as Apples infamous rounded corners.

21

Gillette's patent drawing of the Razor, 1904

22

To Economise

To Economise
A common motivat ion that appeals to all disciplines is efficiency in time and space. It this case, compression is not only the process but also the goal, whether to archive material in the smallest possible space, or to transmit or record it in the smallest amount of time. Tachygraphy naturally comes to mind, a symbolic writing method used by notators along the cent uries in a variety of machines. The hist ory of algebraic notat ion from the first rhetorical face where all calcula tions were described verbally, to the last symbolic one where every element is a symbolic replacement of the rhetoric pre mises could potent ially be studied as a process of language reduct ion, where the syst em of notat ion itself has been boiled down to its most concentrated form, always without losing any of its informat ion. This is also the challenge we witness while reading the algebraic-like proposals of Cart esian logics, or in udwig Wittgenstein's Tractatus Logico-Philosophicus , whose original name was Der Satz (commonly proposit ion, sent ence, phrase or set, but also leap). In computational theory, the business of stuffing as much data in the smallest amount of time/space can be dis cussed as different degrees of olmogorov complexity, a measure of the computational resources that are needed to spec ify the original object. In any case, readability is reduced not transformed; the notat ion requires to be transformed back into its original syst em state by the reader through reading. In many such syst ems, the cont ext of the notat ion is very rich, enabling a small movement or subt le difference in sign to transport a very significant difference in meaning.

23

Eclectic shorthands: The Lord's Prayer in Gregg and a variety of 19th-century systems

24

To Analyse

To Analyse
As Spaniards say, "who cuts the cake gets the cherry," so here we will conveniently choose Reality Shift from Times Up as an illustrating motive. In order to analyse the options of the syst em, a diagram that displays the ways in which 16 cylinders that rotate around their own axis in groups of 4 using handle devices. The diagrams below show how the cylinders work with the possible mot ions through the cylinders in various posit ions being shown in red, the rotat ions in blue. Such a notat ion becomes a necessity when starting to think about more than four cylinders working at a time. The second diagram shows the path to take in order to navigate through all 16 cylinders arranged in a grid.

Motions and arrangements in a 2x2 block of rotating cylinders.

25

Possible paths through the labyrinth "Reality Shift." This notat ion had the purpose to predict and communicate movement sequences for an installat ion piece. What can be seen is the view above of the blocks of cylinders. The cylinders are grouped together in a group of 4 and rotate against one another. One import ant design fact or was that when someone leaves the block of cylinders, they should be in the same state as when ent ered, otherwise no one would be able to get into them aft erwards. This means that there is only one posit ion possible where people can get into and out of the blocks. They also didn't want it to be too obvious how to get through it, leading them to look into the idea of the cylinders moving against each other. Safety was a concern; if the doors moved in contrary direct ions to one another, this would create a chopping act ion when the doors closed. The diagrams helped est ablishing how to move them around and the path to take in order to navigate through it.

26

"Reality Shift" in use in Maubeuge, France

To Interpret
An int eresting example of int erpretat ion can be seen in the pract ice of rebuilding or re-implementing old electronic art pieces, especially in various code forms. The process, to take a concrete hypothetical example, involves taking a carefully engineered piece of video hardware and analysing the process, developing an abstract ed notat ion of the import ant pro perties and processes of the original electronic piece and working out what is not import ant about it; hand soldered and wirewrapped circuitry, for instance. Then the abstract ed version of the piece, notated in whichever form seems most appropriate, is used to implement the piece once again in a chosen programming environment. The new piece is in some sense a copy of the original, yet it is very dist inct, being built on a very different substrate. One int erpretat ion is that the code is an abstract ion of the electronics, one turtle standing upon another, a different point of view has the two turtles standing next to one another, both being a level of det ail upon which the same turtle, the act u al piece of art itself, can stand. The purpose of the notat ion is to derive the core funct ionality and aesthetics of the original piece in order to allow a new int erpretat ion. A similar process happens when musicians decide to int erpret one another's work. They de cide what is import ant, whether it is a melody or other "hook" line, the rhyt hm, the chord progression or the text. Devo's int erpretat ion of the Rolling Stones "Satisfact ion" took very litt le of the original to produce a vitally new in terpretat ion; other int erpretat ions are closer to the original to the point of near int erchangeability. A similar process takes place when music is arranged for different instrument at ion and the sust ained notes of a viola are left to fade or re peatedly struck on a piano. The need to notate and obt ain the core elements of a given piece in order to int erpret it can be seen, on some level, to be based upon the nonexistence of another form of notat ion from which the original piece was created. In general, notat ion for int erpretat ion serves the main purpose of preparing a communicating a set of rules, commands, steps, etc. with the core int ent ion of product ion rather than reproduct ion.

To Disguise

27

To Disguise
Both the most photogenic and elusive use of notat ion, this is the art of obscuring or concealing data in order to transmit it without revealing its cont ents or simply, without being seen. Many today, from int ernational men of myst ery to liber tarian crypt opart iers, use encrypt ion to prot ect their transmissions from eavesdroppers, or steganography to bury mes sages in inconspicuous carriers and create necessary paths of private communicat ion in our hypersurveilled society. Crypt ographys most notorious icon in the west ern world, the Enigma Cipher Machine, was designed perform one al phabetic substitut ion cipher after another, making it nigh upon impossible for the enemy to decrypt the message, or so time-consuming that even the correct resolut ion of the message would be useless. In the original model, letters are scrambled by a set of consecutive rotatable wheels that change posit ion all the time.

Simplified circuit diagram of a 3-wheel Service Enigma Not so different from the Enigma, Tor -the onion router- allows for anonymous browsing and packages exchan ge between people. In this case what is concealed is not the message but the sender, creating a maze of ever rotating IPs that make it difficult for organisms like the RIAA or fascistic governments to know for sure who is doing what and where. The rules of this syst em are obvious: both receiver and producer of the notat ion must share the key to the code, otherwise the material will be lost like tears in the rain. About the complexity of the key, it depends mostly on the dang er itself: how smart or well-equipped is the enemy. How much do we care? The most secure method, a one-time pad, can not be reused (as its name suggests) and is therefore complex to transport. A simple method might be easily transport ed, but is also easily broken.

To Notate
We come to the end of our necessarily finite list of possible purposes for notat ion. In the next few chapters we will look at some of the properties of notat ion, about the levels of abstract ion, readability and int erpretability. Armed with the places notat ion can be used and the forms it can take, let us vent ure into some ideas about the very struct ure of nota tion. 1. The ong Now Foundat ion invest igates many of the responses to this problem, dealing with quest ions of informa tion det eriorat ion and semant ics that reach across cult ural tradit ions. See http://longnow.org/ 2. Website: http://www.nordico.at/

28

TURTLES ALL THE WAY DOWN1

"We don't live on a ball revolving around the Sun," she said, "we live on a crust of earth on the back of a giant turtle." Wishing to humor the woman Russell asked, "And what does this turtle stand on?" "On the back of a second, still larger turtle," was her confident answer. "But what holds up the second turtle?" he persist ed, now in a slightly exasperated tone. "It's no use, young man," the old woman replied, "it's turtles all the way down." Reprinted in Stephen Hawking "A brief history of time" 1988. In the Foreword we ment ioned the "bott omless stack of turtles" effect. evels of increasing abstract ion, notational ac curacy, arrayed one on top of another. Yet not all these levels of focus are equally useful for the expression, discussion, or transmission of a part icular piece of work to a specific audience. Some levels of abstract ion are insufficiently det ailed, consisting more of an evocative "descript ion" than a funct ional notat ion. Other levels may be so needlessly det ailed that they no longer allow a human reader to understand and meaningfully int erpret a work. Yet those layers may be much more appropriate for machine readers (computers), which can int erpret much greater amounts of input, faster, and with more accuracy. This of course depends upon the format of the notat ion: not all deep levels of descript ion are necessarily formal and are thus possibly not readable by a computer. Finally the turtle-descent begins to beg the quest ion: at which layer of det ail does a notat ion becomes the work itself? Put philosophically: when, especially in considerat ion of int erac tive and digital cont ent, does the map become equal to the territory?2 Or at least struct urally indist inguishable from it. And which level is most useful in any given cont ext for getting somet hing done? For every domain - indeed for each work - this quest ion may be answered differently. The ext remes of detail-level are what we consider Dragons. They are notat ion levels too abstract or too specific to be useful as communicat ion of the desired expression or informat ion. Of part icular fire-breathing danger is natural language as a represent ational syst em: insidiously slippery, adaptable to all levels of det ail, shaping reality and its re lationships, yet demanding creative and imaginative effort to int erpret. A more paranoid approach might spot the pos sibilities of Goedelian inconsistency in any syst em capable of notating arithmetic. But in between the dragons of too lit tle and too much is a rich landscape of space and time based notational syst ems to explore. Not too hard, not too soft, it's just right. Different types of notat ion syst ems will be most appropriate and most useful to different groups of readers. At the most simplist ic level, a division between human beings (with our limitat ions of percept ual speed and accuracy) and machines (with their limited powers of inference, adapt at ion, or change) becomes relevant. Within the human group there are many useful dist inct ions such as age, training, role (such as creator, int erpreter, or consumer) and cult ural tradit ion. Within the machine readers of notat ion, one might need to consider practical det ails such as operating sys tem, installed software, syst em architect ure, and performance. At the int erface between machines and humans there is an ent ire specialized category of notat ion syst ems - computer programming languages - varying depending on fact ors such as purpose, relative readability to one side or the other (aka high or low level languages), informat ion philosophy, and aesthetics. Especially informationally dense time-based media benefit from using different notat ion syst ems at part icular timescales. A song might be depict ed at the highest level by broad lines, arcs, and markings equating to the emotional and dramatic shape of its sect ions, on the level of seconds by a chord chart, at a second-by-second level by a musical score det ailing every (or at least most every) note, and at an even finer granularity (fract ions of milliseconds) by a con tinuous line expressing the waveform that is created by a performance of the music, a line precise enough to reproduce the sounds themselves, with computer software and hardware.

Sameness

29

Sameness
At some level we feel that there is a cert ain level of struct ural isomorphism ("iso" meaning same, and "morphism" mean ing shape) and below this level there are further degrees of refinement that do not get closer to the original, only ex amine it in further det ail. In software we see this quite explicitly with the level of source code for a given program being struct urally isomorphic to the program being run. Deeper below we get the compiled binary, the microcode in the CPU and deeper into electronics - this gives us more det ail, but does not define the program more explicitly. Above this level the notat ion leaves aspects open that are defined more closely as we move down the turtles. It is also worth noting that there are cert ain types of descript ion that are equivalent. As we will see below, we can regard a suitably dense series of values and the mathematical equat ion of the curve passing through them is equivalent, as we can pass from the values to the equat ion by int erpolat ion or curve fitting and back by calculating values explicitly. As these levels of descript ion are equivalent, we can talk about equivalent turtles and raise the quest ion of translat ion. It may well be that the idea of translating across only makes sense below this level of struct ural isomorphism, but the answer remains open. Below that level there are definite ideas of being the same. However, once again looking ahead to the example of a robot choreography below, a sequence of key frames can be represent ed as numbers indicating the posit ion of the various parts of the robot, 3D models of the robot, numbers indicating the value of the various robot act uators or det ailed photographs. Then the informat ion cont ained in these descript ions, these various notat ions, can be used to obt ain the informat ion in the others, making these descript ions somehow equivalent. The terms code and program cont inually raised problems as we conversed and discussed. At some point we be lieved that we had to ret ire the problem to the dragon department, but the following seems to be somet hing that works. The code, the source code written in any computer language, can be compiled to give a working program. This program, when run, has cert ain behaviour. We regard the behaviour of the program to be the thing that we are int erest ed in. The behaviour is the semant ics of the code. The code is the notat ion. Two pieces of code are equivalent if they produce the same behaviour, that is, they have the same semant ics. Because the code produces the program, we will talk of these as equivalent, we have int roduced the term struct ural isomorphism to describe this. Programs also raise other issues around uniqueness and things. The same code compiled to a program can be run simul taneously on two computers, given two things that are obviously not the same thing but are essentially the same. Two Firefox browsers on the same architect ure are essentially the same. A technical way of different iating these is to talk of instances of a program, or instant iat ions. This might be ext ended to speak of two games of chess as being instances (they are not played the same but they do start from the same state and have the same rules) or two performances of a score as being instances of that score.

Program == Score?
et us take this opport unity to raise another difficult quest ion: Is the code or the patch the score? In many computer music, the composer (who may well be the performer) creates a piece of software as a patch or as code. It can be argued that this code is the score. However it is import ant, if not vital that the symbols in a score should have the potent ial to be executed by any software/program with any hardware, and/or any human being able to connect to the cont ext. Chosen symbols for a score should go beyond a specific soft-or hardware creating a metalanguage for int erpretat ion. Ot herwise it is not, in some sense, a score, rather it is an encoding of a specific piece and performance of music. It is a nota tion of it, perhaps too specific to be a score. If we take this as a given a code or a patch is not a score any more than an efficient compression of a piece of music using advanced adapt ive compression techniques is a score. Note however that a patch that implements an instrument, where the performer uses notes on paper next to the computer, is not using the patch as a score. This also shows the import ance of being aware of the point of perspect ive when judging the level of abstract ion and/or precision of a notat ion syst em. Differences can exist inside and outside the notat ion syst em, but only those that are de scribed from within (following the inner logic) exist on the inside of that specific "reality". Whatever is not part of the vocabulary of a notat ion syst em because it is not possible to produce a meaning in the syst em's reality has to be ignored by the judging spect ator in the same way that it is ignored by the vocabulary of the syst em at hand. The idea of sameness, of struct ural isomorphism, is somewhat difficult in the case of music, as it has to do with 30

how we measure it. As the piece of music goes deeper in levels of descript ion from "a piano fugue" to "the second record ing of the Goldberg Variat ions by Glenn Gould" and then down to the various media with which that could be re produced we come to ask whether there is struct ural isomorphism between the P, CD and MP3 versions, or whether the version I hear on my stereo is different from that heard on yours when we play the same CD. These are difficult ques tions once again and we believe we hear the sound of a dragon breathing down our necks, so let's leave this part icular white area on the map for later explorers to more deeply det ermine. Below we will cont emplate the idea of a layer of abstract ion so low that a composer is able to compose a piece without hearing it ever, knowing that the instance of it when performed will match his expect at ions due to the cult ural cont ext and the specificity of his writing. On a more abstract level, at a higher turtle, at the Data Ecologies symposium the composer and musician from Toxic Dreams, Michael Strohmann, posed the problem of finding ways to notate electronic music such that he could plan what he was going to do before he sat down in front of the computer. He would thus avoid the danger of slipping into the miasma of getting lost in acoust ic det ails and losing track of what he was initially trying to do. Finding the approp riate level of det ail is an ongoing issue, especially given the capacity of a computer to allow arbitrarily deep and fine meddling with acoust ic det ails. Taking the example of music and scores further, we can see that the top level of descript ion - the overall struc ture of a musical work - could benefit most from a "timeline" style of notat ion: a timescale reading (typically, at least in the west) from left to right, with height and shape of a line or form equating roughly to dramatic effect, or volume, or tension, or business. The line or shape might have text ures or shapes representing the instrument at ion, feeling, or dens ity of the music in a sect ion. This style and granularity of notat ion seems most appropriate for roughly describing and analyzing a dramatic linear work such as music, theater, film, or book, though it's conceivable that as music, the timeline itself could be play able if we assume quite a lot of int erpretat ion by the performer. How then we might look at a similarly-scaled notat ion syst em for int eract ion? How do we show broad dramatic changes when a work can change over time based on user input? These problems will concern us later in the book, let's concentrate upon a non-interactive example now in order to more closely examine the ideas.

A Robot Dance Gedankenexperiment

Example diagram of levels of notat ion for specific types of work. We recently saw a robot arm performance created by Matt Gardener, an Australian media artist, where the mot ions of the industrial robot arm were fluid and animist ic, very much like the mot ions of an animated character in a cart oon. We learnt some elements about what Matt had done and will att empt to int egrate them with our other understandings of how a robot notat ion can work in this example looking at the various levels of descript ion. At the top level we have somet hing that we might call the high level "descript ion," often in natural language. This is perhaps not quite a notat ion, but we cannot see the strong line between high level descript ion and lower level notat ions that might also use a lot of natural language elements. The descript ion might be as short as "The robots dance symmetrically and gracefully before the water cannons are turned on, whereupon they fan the water into the air." This would then be refined to more det ail, for instance a small sect ion might be "The robot's arm lowers to the left, swooping like a swallow down and parallel to the ground until it comes to an abrupt stop, pulsing slightly as though breathing." From this verbal descript ion, that is perhaps equivalent to the written instruct ions in a stage play, undoubt edably a form of notat ion, we can begin to create somet h ing more formally written down. We come to the mid level: "human readable notat ion" syst ems. We imagine the various posit ions of the robot being like key frames in an animat ion, with various annotat ions that show the type of mot ion between these frames; arrows, indicat ions of rotat ions, accelerat ions, etcetera. Talking with Matt, it seems that the programming occurs in a similar fashion, with a number of key frames being used as fixed posit ions and the robot moving in a linear fashion between them. In order to create this, an operator can posit ion the robot manually to these key frames or the key frames can be generated in a scripting language that might also include such struct ures as repeating loops and offsets. In order to move from the key frames with annotat ions and curves to the linear programming mode, a suitable large number of key frames need to be construct ed so that the series of straight 31

lines approximate the desired curve. As an example, the robot operator has most programs around 30 lines long, Matt's program for the performance had around 280 lines.

The problem of creating key posit ions is a difficult one. Industrial robots are often manipulated by a cont roller directly and the desired posit ions are created on the shop floor and stored. An animat ion program that simulates the mot ion of the robot arm allows a designer to find cert ain posit ions using the simulated robot arm. The posit ions can also be found by calculat ion or other ways. The linear int erpolat ion has its loops dismantled to create the lower level movement of the robot arm, with a Gcode program that exactly states the types of movements that the robot arm should make. G-code is the industrial stan dard for Computer Numerical Cont rol, CNC, and is used by everyt hing from a home made 3D printer to a multiaxis re dundant robot arm syst em for milling.

On the other hand, we could look at the movements between the key frames as a collect ion of parametric equat ions, in terpolating the movement in ways that have a cert ain amount of smoothness using funct ions that the machine can im plement easily. One version might use a combinat ion of linear elements and circle segments, another might use bezier int erpolat ion, sine waves, or polynomials. We imagine that the mot ion could be created using a mathematical funct ion that uses the parameter t corresponding to time to move along and give the resulting servo motor posit ions.

Such a piecewise funct ion uses a collect ion of funct ions and given the appropriate tools, the mathematics involved can be analysed to enforce cert ain constraints on the robot servo movement. The funct ions can then be drawn to give a curve.

32

This curve shows us what movement should be created by the robot servos. One implement at ion of this would be to quantize the movement, to sample the mathematical funct ions at regular values of t and use these values to create Gcode instruct ions. The curve would be approximately the same as the mathematical curve, with small errors due to finite exactness and round-off errors. Similarly we could take the G-code values and do some curve fitting process to find a mathematical expression that most closely approximates the series of values. One might say that the G-code and the mathematical curve are levels of det ails that are equivalent because we can translate from one to the other by quant isa tion and curve fitting - they are adjacent turtles, so to speak. As we climb down our stack of turtles, we get to what we will think of as a low level, the "machine readable code" that is perhaps too messy for humans to read on the level of complete movements of the robot. If we imagine the cont rol of the robot coming from a computer syst em, the output is some kind of data that goes to the robot cont rol syst ems. A data bus is one example, sending commands to various Digital Analogue Converters (DACs) that supply values to the servo electronics, Rotary encoders (Renc) on the robot arms tell the electronics where the arms are so that the cont rols are accurately implement ed. Feedback cont rol syst ems compare the desired value and the act ual value and enforce a correct ion to keep the syst em on track. If we were to have a volt age probe on the output of the DAC, we would see a curve over time that should closely approximate the curve shown above as a result of the mathematical formula notat ion.

At this lowest level of explanat ion we might find a whole spectrum of possibilities for notat ion utilised by engineers, mechanists and mathematicians to describe what is going on, how the instruct ions from the computer are translated into cont rol values, fed to motors, the way that the masses of the robot parts slow mot ions or enhance moment um to make cert ain cont rol mechanisms impossible. The forward dynamics that tell us the mot ion of the robot parts in respon se to cert ain robot servo act ions are a notat ion that summarises the mechanical device to a collect ion of matrices. An other mathematical model can describe the way that feedback loops lead to cert ain errors in the posit ion of servos and the resulting robot arm configurat ion. These notat ions tell us about the errors and allow us to analyse the way that cer tain planned mot ions will not work the way they are planned, or that cert ain desired mot ions can be easily created using an apparently different planned mot ion that, with the nonlinear dynamics of the robot arm, lead us to a mot ion that is not as planned but perfectly as desired. For those who want to dive deeper into the turtles, we get descript ion for steel flexing and cryst alline struct ure, servo motor models with magnetic effects and frict ion problems, sensor syst em noisiness and error correct ion, electronics design, chip and transist or design, then down further to quant um effects and the deepest layers of reality 33

that would explode this slim volume if we were to att empt to notate them any further. There be very small dragons, but they are not ours to slay today. 1. http://en.wikipedia.org/wiki/Turtles_ all_the_way_down ^ 2. http://en.wikipedia.org/wiki/Map%E2%80%93territory_relat ion ^

34

BEETHOVEN'S DEATHBED
Pict ure a deaf old man, those classically piercing eyes, scribbling frant ically as ideas pour from his imaginat ion through his fingers and his quill onto sheets of paper arrayed around his small room. The pages are collect ed, copied, studied and at some dist ant place, the premier of this symphony takes place, unheard by this man yet heard so perfectly within his head. Rapt urous applause is also as dist ant from his experience as the notes and harmonies of the symphony, acclaim for another great work might percolate through the written record of the day. This story, fanciful as it may be and loaded with the delivered imagery of Beethoven as some mad genius, con tains several elements of truth and illuminates several problems and possibilities that a complete and useful notational syst em can offer. In this chapter we aim to invest igate some of these things and see the ways in which these ideas int er relate.

Reading, Writing and Reception

The first way of looking at this fanciful anecdote is seeing the processes taking place. Music, symphonic and complex, emerges in the mind of a composer. How exactly this happens is probably one of the big quest ions of life and creativity, but we will have more to imagine about this later. For now let us suppose that it appears fully formed in his imaginative genius centre, problematic as that idea is. The first thing that happens is that a process of writing takes place. Using an existing notational syst em with possible side notes and other marks of meaning, the writer transfers the imaginat ion of sound into a series of notes played by a number of instruments at various times, with speeds changing and dynamics building from melody to effect. This act of translat ion is perhaps one of the most myst erious. The next process to take place is one of reading -the conduct or, musicians and other associated people work through the score, imagining parts, pract icing lines, deciding which instruments and how many thereof will be needed to turn this score into a performance. Maybe someone will change the page breaks to make the playing more straightforward in a performance scenario, another person will re-write on another staff as they come from a slightly different tradit ion. They will read, they will understand, they will adapt and enlarge, adding nuances or arguing over emphasis, the violins will rehearse and discuss the difficulties and the conduct or will work out how to balance all the pieces into a whole. As the night of the premiere approaches, the det ails of coordinat ion will come together, scribbled marginal notes will be correct ed and the form will become apparent. On the night of the performance, a process of recept ion1 takes place. The audience made of a huge spectrum of people, from fawning court iers through to highly proficient musicians, will hear the piece as delivered by the orchestra. They will listen and enjoy, they will be transport ed by melodies or ast ounded by timpani, some will close their eyes and dream, others will focus on the prancing conduct or and the sawing violins, full of enraged and act ive energy. Perhaps the conduct or escapes his lonely deathbed room to sit and enjoy the spect acle, watching his music affect people so deep ly, perhaps he will watch the musicians and follow his imagined sound of the symphony in his mind as it matches and differs from that which is delivered. A non-deaf composer might be able to annoy the conduct or at rehearsals, our deaf old man can only gaze upon what he sees and att empt to bring imaginat ion and its visual effects into alignment.

egibility

35

egibility
The core act ion in this process is, to our minds, that of reading. One of the core properties of a notational syst em is that of readability. The Voynich Manuscript has been created by an apparently int elligent person or persons, it meets all the requirements of a text made of symbols to be formally nont rivial, yet no one knows of what it speaks. This text is un readable, yet we believe that the notat ion it uses is that of a language that is meant to be telling us somet hing. The rant ings of seers and the insane, the scribbled pages of paranoid notes written in code, are also unintelligible. Thus we can not speak of a notational in any useful sense of the word, unless there is, at least, the possibility of reading. There is a story of a scientist, high on ether in the process of experiments in the 18th cent ury, who reached an epiphany of total understanding late one night. Unable to recall the cont ent of his epiphany the next morning, he resolved to undert ake the same experiment but with a notebook in his hand, ready to document, notate or otherwise capt ure the essence that he had discovered. The experiment succeeded, contrary to what we would expect, epiphany was reached, and the next morning he read, in large letters, "Everyt hing is brown."2 Two immediate explanat ions arise. The first is that the epi phany was not as good as it seemed, as the understanding reached was somewhat less than enlightening. Another in terpretat ion is that the way that his shamanic mind was working enabled a cert ain very compact summary of his epi phany and, if only he and we could understand the notat ion, the words, their positioning, the twists of their lines, all these things would convey the cont ent of his experience. A third explanat ion might be that this att empt at a universal notat ion, able to explain the secrets of the universe and proffer a theory of everyt hing, is a notational impossibility, and such dragons ought only be avoided unless one is well equipped, brave and in good company. For now let us assume that notational syst ems in which we are int erest ed are defined by a minimum level of readability. What does such readability require, what can help it, what might be a simple nice to have? On the most essential of levels, we need shared knowledge. An Australian Aboriginal meeting a well educated enlightenment scholar in Botany Bay in 1788 would find very litt le in common other than the shared experience of what they saw and that of being humans. Upon such commonalities much can be built, in part icular language. In a notational syst em we want to assume much more, as a notational syst em should enable a short ening of descript ion and the transmission of ideas and experi ences. For this the writer and reader, or let us call them the users, of a given notated thing, have to agree on what they are talking about and how they will divide up the world. Skirting these issues of epistemology and all that jazz, a dragon lair of the most difficult sort, we talk blithely of a shared terminology as the ability to be speaking about the same things and to agree what words or other symbols to use to refer to these things. When an instrument builder and a composer talk of music, they will have difficulties as the instrument maker talks of timbre and resonances while the composer thinks in melodies and count erpoint. When discussing game experience, a player speaks of flow and being lost, a graphic designer of gritt iness and dist racting Moire effects. Not only technical issues, but also cult ural issues, the cult ural cont ext, plays a huge role. Writing three violin parts in the cont ext of an orchestra means that there will be several violinists playing each of those parts, the existence of orchestras themselves are dependent upon so many element including the lack of amplificat ion, the accumulat ion of wealth, management struct ures of orchestral discipline and the display of social import ance.

Reproducibility
From a score, a notated piece of music, one also expects that the resulting readings give the same result (to the required level of "same") each time the piece is read. This begs the quest ion of sameness, which is closely related to our levels of descript ion, which turtle we are looking at. A Fluxus piece consisting of a short instruct ion "draw a straight line and fol low it"3 lives primarily from the openness of int erpretat ion. We expect to recognise the reflect ed melodies in a perfor mance of the Goldberg Variat ions, whether played on piano, harpsichord or ukelele. Every implement at ion of a schematic diagram should give a circuit that works in exactly the same way. The descript ive form of a folk music piece performed in a remote alpine village should cont ain enough informat ion that an ethnomusicologist can identify similarities to related hist ories and see the changes that have happened in the ensuing years when she re-visits the vil lage and hears the same piece passed on through generat ions since the original notating. Whether the descript ion is a sound recording, a video, a descript ion of the movements or an annotated score, all these might have some use. And they should hopefully enable if not the reproduct ion of the piece, then at least a comparison of the piece with another similar piece. 36

The idea that a composer should, based upon their previous experience with the ensembles for whom they write, be able to imagine the music very closely while writing a score, is a very strong idea of the role and capacity of a composer. This idea requires that the composer is working on a level that is deep enough that the resulting instance of that score being played is very close to their int ended. Many of the examples we saw above allow a lot freedom in the int erpretat ion of the piece. These are written at a higher turtle than Beethoven did. The det erminat ion of the audience plays a vital role in the select ion of a notat ion. Who are the implementers of the descript ion, what are their concepts, cult ural cont exts, levels of det ail. We come here very quickly to quest ions of which turtles we need to be talking about. A stage direct or discussing a piece might discuss the music in terms of em otional changes and rhyt hmic strength when speaking to the head of music, but will talk about the length of a held note when talking directly to the musician during a rehearsal. Here we can get lost in our discussion of turtles and the det er minism that our symbol syst ems enforce upon us and we refer to those chapters for those and related det ails. For rea dability quest ions we are concerned with the commonalities. The commonalities can be most quickly built up, explored and the missing commonalities repaired in a one to one dialogue. The audience of one scenario, where two people sit together to discuss a situat ion int ensely, allows close read-write loops, discussing a situat ion with each person holding a pen in their hand, agreeing upon symbols and their use, the scope of the page its what it is staging changing as they dis cuss, reflect and reflexively build a notational syst em. Two people, tight reactive loops, a syst em that can react and grow quickly. Taking this idea one step further, the audience of zero scenario leaves the reader, writer and creator of the notational syst em in one person, scribbling, drawing and trying to explain to themselves what it is that is going on in a given situat ion. Through the ext ernality of a score one is confront ed with one's own imaginat ion and through att empts to understand what it means one can discover the faults in a struct ure, the strange curves of an argument, the open quest ion at the end of a story. Not only is a dialogue with oneself through the ext ernalised possible, but argument, bant er and perhaps even heckling become part of the ext ernalised int ernal monologue. ooking at a scribbled curve, drawing a circle around it and writing "HUH!?" to let yourself know that you do not know what you are talking about. Then starting to defend one's posit ion or develop a refinement, building the piece through ext ernalised int rospect ion. We could go further and claim that the ext ernalised nat ure of notat ion, where a creator gets the ideas out of their head and drawn onto paper, chalked onto a blackboard, scratched in the dirt or summarised in some other notational form, is the creat ion of a new other. This other is a colleague with whom the creator can converse on equal terms, constructing the notat ion themselves, and thus get a grip on what they are trying to do and develop the piece iteratively. Whether sketches, doodles, snippets of text or music, diagrams of lights and their movements, all these forms allow a single creator to explore the possibilities of their ideas and thus create. Thus the idea int roduced above, that the symphony appears fully-fledged in the mind of the genius, is most likely false. Snippets and ideas appear, are sketched, assembled, collect ed and re-worked until the notated piece, the score, can be looked at and the whole thing can be apprehended and thought about as a single thing. There is an argument that notat ion is one tool on the path to acts of so-called genius.

1. Recept ion as a generalised form of listening. this might be a wobbly concept. But it is probably a very wobbly con cept and has been thought about by much cleverer people than us. ^ 2. Other versions of the story have various other more or less meaningless statements including a smell of pet roleum. a, b 3. One of the "Composit ions 1960" by a Monte Young (*1935-) ^

37

STEAL THIS NOTATION!

What do we need to think about in order to create notat ion? How might we consider the role of symbols and tools in building a notat ion syst em? For artists who use notat ion in their everyday work - electronic or otherwise - it is clear that every notat ion syst em has a specific aesthetics, whether we are aware of that or not. When we build scores in part icular, from combined notat ion syst ems, it helps to be explicit about how tools and select ions of symbols cont ribute to a specific aesthetic. This chapter is about creating new notat ion syst ems using the example of art istic pract ice. It is about how existing tools are influencing us in the process of creating new notat ion syst ems, and out of notat ion syst ems, scores. For every notat ion syst em to become a meaningful syst em, there is a precondit ion of agreement on a common terminology in a specific cult ural cont ext. e.g. in the cont ext and pract ice of electronic art, there would be terms like filter, envelope, curve, digit, line, FFT (Fast Fourier transform) ..... , and this specific community would have an un derstanding of what is being referred to, including the complexity of the term itself. In building a new notat ion syst em, you could use different symbols for these, but usually you are det ermined by already existing syst ems and tools. Artists often find themselves in the posit ion of creating new notat ion syst ems. The prospect of creating 'new' syst ems happens when you are dissatisfied with the utility of the syst ems you are using. But of course you are influenced by the syst ems you know, and out of that literacy, you are creating somet hing new, from literacy in the old syst ems. You need to think: what do I want to describe and who do I want to address? We want to think about meaningful or use ful symbols for this. These are key quest ions.

Symbols
Most used symbols in notat ion syst ems are visual. Examples of audible or gest ural symbol do exist. Gest ural symbols are used for example the person at the airport who helps the planes to come in with lit signs in his hands, or a realtime im provisational conduct or, use gest ural symbols. Morse code (as well as being visual) is an audible example. Any kind of sonic warning sign exists in an int elligible social syst em of sounds (such as a shark alarm in a beach town, versus a fire alarm). Audible notat ion can also take place in a musical score in which players get audio instruct ions via headphones, for example.

Parameters
The term parameter, connect ed with disciplines like maths, logic, linguist ics, and environmental science came into play in art in the 1950s, with the emergence and art istic use of tools allowing to act ually measure funct ions like frequency, amplitude, and so on. Any funct ion's properties and their transit ion from one state to another will be expressed with the values of a specific parameter space. In digital tools the parameter space for amplitude is commonly defined to be bet ween 0.0 and 1.0. All values in between are possible measurements of this parameter. So 0.0 is silence and 1.0 is the maximum value without dist ort ion. The next examples show the select ion of different symbols for envelope. The basic parameters of an envelope are time and amplitude. The notat ion symbols chosen for an envelope can vary. In the first example, we choose a graph and a curve as a graphic expression showing the values of the parameters: amplitude and time. Example A: The symbol is a graphic curve

In the second example of the same envelope, we can choose digits to represent the same values within the same para 38

meters. At time 0.0, I have an amplitured of 0.0. At time 0.23, I have an amplitude of 0.3 and so on. Example B: The Symbols are Digits 0.0 0.0, 0.23 0.3, ........

Tools Determinism
The tools we are working with influence the select ion and aesthetics of symbols that we choose to create. At the same time, a tool could already have a set of symbols embedded in it, for example a program like PD (Pure Data) has a symbol for objects, messages, number boxes and so on. Furthermore, these symbols already have quite a complex meaning. So there is a difference between kinds of tools - a tool like Pure Data having existing symbols with complex meaning, or a tool like a pencil. In programs used to create electronic art, we are used to having on our screen a number of different boxes connect ed to each other. This is not just a funct ion of Pure Data or Max/MSP but it also has a long pre-history of cabling and devices, or more generally processes and data flow, impacting upon the thinking and aesthetics of notat ion syst ems in general. Such images, coming from the tool, will influence our approach to create notat ion syst ems and scores. And with that, the aesthetics of the syst em and the work being produced. In this specific example, a notat ion for electronic instruments written on paper is influenced by the aesthetics of MAX/MSP or PD. Here we are influences to think in terms of streams of data, whether it be parameter values, sound or video, flowing between boxes that manipulate, store and pass on those streams.

If your tool is a guitar, you might be drawing on a notat ion syst em such as the following, where the immediate influence is to think of chords and harmony.

Notating chord based guitar music can be done by using these symbols arranged appropriately over the page, for instance at appropriate points adjacent to the text of a song, or replacing them with the abbreviat ions describing the chord. Rather than the more explicit descript ion of the posit ion of the fingers, we could simply write Am to mean the A minor chord. These two notat ions would be equivalent in terms of their informational cont ent, where one reminds the reader of the hand shape of the chord, while the second relies on a level of expertise.

The Connection between Symbols and evels of Abstraction

A specific symbol is not necessarily connect ed to a specific level of abstract ion (or "turtle") within a notat ion syst em. In fact there is no necessity to connect a specific symbol to a specific layer of abstract ion. But in many cases it turns out that specific symbols are more relevant than others, for a specific level of abstract ion. 1.) On a higher level, one might choose a spoken language to describe what one wants. For example, "Play a Melody." In the notat ion "Play a Melody", everyt hing is open except that its a melody. There are many refinements pos sible without leaving the realm of natural language. "Play a joyous melody" or "Play a familar melody slightly wrongly" 39

lead somewhere, "Play a melody and repeat it until it breaks" leads somewhere else ent irely. "Play a melody and follow it" references a Fluxus tradit ion which brings in a whole swathe of other possibly import ant implicat ions, not ment ioned in the notat ion itself. 2.) This is a different level of abstract ion: on the one hand it's less abstract but its not getting rid of a large degree of openness.

In this notat ion much more is defined - the rhyt hm, tonality - but what is totally open is who or what is playing it, so it also is ext remely open. The rhyt hm is there but not the meter, etc. However the use of modern West ern staff notat ion would indicate a cert ain cult ural milieu, the non-use of print ed parallel lines adds a cert ain casual approach which might lead to a more effusive and frivolous playing style. In any one notat ion syst em you can have different layers of abstract ion, and combine them. It is the combina tion that is specific to a part icular outcome. For example, you might want to be really det ailed about the timing of a note, but you also might just want to say, regarding the volume, "play it really loud". A high level of abstract ion for one symbol might be leaving things open for int erpretat ion, but this is totally cont ext dependent (very abstract symbols can mean very specific things to specialist communities of pract ice). It is also the case that the same notat ion, int erpreted by different int erpreter communities or pract itioners, can change the level of abstract ion without changing the symbols. For example, lets say I have a video in which a person is moving from A to B. et's give this video as a score to a musician for musical int erpretat ion. In this case, you have a high level of abstract ion in the score. BUT if you use the same video score and say "this is a score for a human perform er", who you want to do a very similar traversal from a point A to point B in a similar space of performance as the original score, then in this second instance and use of this score, the notat ion is so precise and laborious it becomes fair ly useless as a notat ion... indeed its debatable whether this is even notat ion. It is an abstract ion, losing informat ion about the smell of the room, perhaps it is in black and white and the colours of the clothing are lost, or it is silent so any connect ion to the accompanying sound is lost. Formally it may be said to meet the requirements of the notat ion test, perhaps the more relevant quest ion is whether it is useful or int eresting as a piece of notat ion. The dance hist orian might appreciate it strongly in 20 years time as a notat ion with which to compare the movement of the performer over their int ervening development. Many others would be less int erest ed.

Scores
With tools, symbols and parameters, artists create a score. There are at least two different kind of scores. There are scores meant for int erpretat ion, which are always an algorithm, such as a recipe, which aims to be put into act ion by an int erpreter. The second kind of score's purpose is to make a transcript ion or document at ion of an act ion that has taken place already. In this part we are talking only about scores for int erpretat ion. The symbols in a score should have the potent ial to be executed by any software/program with any hardware, and/or any human being able to connect to the cont ext. Chosen symbols for a score should go beyond a specific soft-or hardware creating a metalanguage for int erpretat ion. The main purpose is to leave a struct ure which has the potent ial to be transferred to other syst ems. This means that our score should lie above the level of struct ural isomorphism with the resulting music. The following is a piece of code out of an Arduino sketch, relevant only for this specific environment. It is not fulfilling the above requirements of a score, as it isomorphic to the process that it encodes/notates. s->selectBank ( BAN_A ); s->setPatch ( OSC_1_TO_MIXER | OSC_2_TO_MIXER | OSC_3_TO_MIXER); s->setWaveform ( OSC_A, SINE); s->setFrequency ( OSC_1 , 40.0f); s->setFrequency ( OSC_2, 77.0f); s->setFrequency ( OSC_3, 1.9f); 40

s->setAmplitude ( OSC_1, 0.2f ); s->setAmplitude ( OSC_2, 0.3f ); s->setAmplitude ( OSC_3, 0.4f ); This next example of a notat ion, with the same meaning as the example above, is open to any syst em and in terpreter and therefore could be executed by any machine and any human being. If you know the terminology, the diag ram is fulfilling the requirements of a score. In this case its up to the int erpreter which kind of oscillating syst em being used: an electronic oscillator, a string, a voice, motorised devices, three tract ors, ....

Mixing Systems and Symbols to create Scores

When we make a score we can express the same instruct ions in different ways, choosing symbols out of different nota tion syst ems and combining them. We can combine different notat ion syst ems and use different levels of abstract ions. There is no rule and no limit. In opposit ion, a notat ion syst em/language has specific rules (grammar) and a limited set of symbols (alphabet). We might regard the notat ion syst em of a score as the union or collect ion of the used notational syst ems, if we wish to be formal. Consider the same algorithm/score expressed in different ways: 1.) move from A to B This notat ion syst em is a subset of the English language (sent ence). The symbols are atin letters A and B, and the words "move", "from", "to". The grammar or inner logic of the notational syst em seems to allow sent ences of the form "move x to y" where x and y can be a capital letter. 2.) A B The notat ion syst em is a subset of the English language, but it is combined with symbols of arrows. The symbols are atin letters and arrows. The grammar might be "x y" where x and y can be any capital lett er. 3.) AB The notat ion syst ems are subsets of the English and Greek language. The symbols are atin and Greek letters and the syst em is otherwise the same as the version with arrows. 4.)

The notat ion syst em is graphics and a subset of the English language. The symbols are straight lines, and atin letters. The grammar might be that the symbol must be a rectangle, 41

two capital letters are within the rectangle and the two capital letters are joined by a straight line. Some of these notational syst ems are isomorphic, some are more specific. We would claim that the first three syst ems are equivalent, while the fourth example cont ains posit ion informat ion and is thus more specific. This is an example of a score with mixed symbols from different notat ion syst ems:

dyn= dynamic, Act=actor, Vc=violoncello, S=sinewave, Voc=vocals, D=dancer, t=time Is this mix a new notat ion syst em or is it a score using mixed symbols from different existing notat ion syst ems? It de pends. If it was used just for one score, then arguably it is not a new notat ion syst em, but it could be a starting point for the creat ion of a new notat ion syst em with rules, a limited vocabulary and set of symbols, and a developing inner logic. Here we are at a point, where we may indeed recognize the creat ion of a new notat ion syst em. With a score like this, very common in cont emporary art pract ice, the int erpreters are forced to use their im aginat ion based in their specific cult ural cont ext. Quest ions will arise like: How long, with which pitch and timbre should the whole note be played? What kind of movement will the dancer choose? How will all the int erpreters coordinate themselves?

42

IT'S THE CULTURAL CONTEXT, SCHTOOPID!

It is a banality but ever worth saying, that the cont ext of any utt erance is needed to comprehend that utt erance. Some statements require less immediate cont ext, such as Pythagoras' Theorem, whereas the statement "I completely agree ex cept about the birds" is unintelligible. And nothing less can be said about notat ions. Every notat ion lies in its cont ext, with an int ent ion, a purpose, assumpt ions and presumpt ions. When a young person says "wimp" they are probably referring to some weakness of the object of derision. When a physicist says wimp she says it with capital letters and a WIMP is a Weakly Int eracting Massive Part icle. This in turn means that they are speaking of a subatomic part icle with mass (so not e.g. a photon) that experiences the weak force but not the strong or electromagnetic forces as well as the gravitational force and are a possible explanat ion for so called "dark matter." Jargon is a form of notat ion, the use if the word WIMP precludes the necessity to explain a whole broad swathe of concepts. So jargon is a shorthand. Jargon also creates communities, people who can use that jargon clump together and become a self selecting group, a localised cult ural cont ext. With this existing shorthand, verbal communicat ion is ac celerated. A similar role is played by the creat ion of notat ions - finding a symbol for a concept enables that concept to be easily manipulated. This brings us to our turtles once again - choosing the right level of expression has to do with the cult ural con text into which one is creating a score. The level of det ail should not be mind numbingly high, with precise movements sending all to sleep but the technicians, nor too low, when it will be greeted as a form of handwaving. A given community also chooses the types of things they want to do with a notational syst em, formulating types of quest ions and ways of asking. This thus informs the types of things that the notational syst em will be asked to do. A group of improvising electronic musicians have litt le use for a notat ion syst em for melodies, but might have a strong desire to form a notat ion to define the way that each musician can take and modify the performance of another. An improvising theatre group cares not about the words of a piece but the struct ure of how they build a piece from the three words given by the audience - a well composed piece allows them and even forces them to develop an ent rancing story by forcing them into cert ain struct ures within which they can find the needed dialogue and movement for their characters as they develop them. The type of explanat ions that we are cont ent with depend upon the cult ure within which we live. In a modern urban scientific-technological society, a storm is explained and written with a synopt ic chart and a summary of air movements, we see it coming with a fall in barometric pressure. An animist society will foresee the storm by observing ants' movements and where birds are flying and might explain it with the moods of gods and spirits unhappy with the way the tribe is acting. A community does act ivities and undert akes projects. By taking things that exist and working out ways of sum marising them, we work out what our pract ice is and that form of compact ificat ion leads to simplificat ions. Those simplificat ions are a form of notat ion, inasmuch we have ways of speaking about them that other people understand. De pending upon the act ivities that we undert ake, cert ain forms of notating become more amenable and usable. Through the spread of computers, the business technique of using spreadsheets has spread as a cult ural technique so that people are more happy about creating things in a spreadsheet environment and can understand that type of informat ion dis play. The idea of using boxes and arrows to describe processes or hierarchical relationships has also become a widespread technique and forms of notat ion that use this, such as abView for designing laboratory data acquisit ion and manipula tion or Pure Data for sound and data manipulat ion, income inequalities or complex arrays of company ownership as diag rammed in TheyRule, have become relatively easily understood. The audience remains an import ant issue in int eract ive art pieces. Who are they? In many cult ural product ions there are two audiences that are apparent. One is the audience who will use the end product of the product ion. The re quirements for a serious game and those for a children's game are widely different and will greatly effect what is in the product ion. However the audience that is perhaps most relevant for notat ion is not the user audience but the people who are helping create the piece - programmers, designers, etc. The notat ion used to communicate within the team pro ducing a piece will be dependent upon all the things that that team have done and the way that they can work together, experiences with various forms of communicat ion and their understanding of what their role can be. Beside that this team has to notate instruct ions for the user audience and even more, to make a record of the users' int eract ion be 43

haviour that will be relevant for further strategies and might lead into considerat ions for specific cult ural cont ext adap tions.

44

INTERACTIVITY

45

WHAT IS INTERACTIVITY?
So far we have been talking about act ions and the notat ion for act ions. What happens, in what order, when in relat ion to what else. We see act ions, people, syst ems, machines acting in the world. Taking Maturana's ideas of autopoietic sys tems, one is confront ed with the idea of borders of syst ems and the allowable effects across those borders. This surface, we could call an int erface. The idea of int eract ive syst ems, as opposed to act ive and reactive syst ems, has to to with a looping series of act ions and react ions between 2 or more act ors. Ah, this sounds so easy!

A Note to Academics
This discussion could, has and will cont inue to fill volumes. As doct orates are written, as artworks and business models are planned and made, as people act, react and int eract, as all this happens people will cont inue to think about the fund amental nat ure of int eract ivity, what it means, what it does, why we do it, what the point might be. In this miasma we dare not tread or dive any deeper than absolutely necessary, thus we would like to suggest a simple, a naive yet hopefully not too wrong concept of int erfaces, int eract ivity and all the conundrums associated with it. One of the silliest yet most telling typographical errors as we write such words is the int roduct ion of the term interreac tive. While unintended, this expression seems fort uitous. When we talk of int eract ion, we really are talking about interreaction. At the first level, we have act ion. I walk to the corner. The next level is react ion. I insert some money in a mac hine and press a butt on, some chocolate comes out. The machine has reacted to me, but I have not, in any useful sense, reacted to the machine. I see a friend, say hello, they reply, I offer some chocolate, we converse about the day, we decide to for a drink, a long night ensues. This is int eract ion of the finest sort. If my friend did not react to my act ions by rea lising I needed a good talk, if I did not react to my friend by realising she was hungry and needed dinner, then our friendship would be less deep. We want to claim that the typo has given us a new idea, that we are int erest ed in int erreact ion, but will suc cumb to popular usage and stick with int eract ion.

How to Recognise it in the Wild

So what are some of the properties of this int eract ivity thing? While the feedback loops must exist, they need not be realtime. A game of chess is int eract ive, the board acting as an int erface between the players. The game can be so slow as to be played by mail, with days passing between moves. Much mail art might also fall ino this category, with musicians exchanging tapes that they use to build upon and create new versions, remixes and other modificat ions before sending on a tape of the updated piece. We would not hesitate to call this an int eract ive process. However in many cases simul taneity is necessary, whether the int eract ion is arm wrestling or improvised music. Arm wrestling with even a small delay could prove tragically impossible or even dangerous. Improvised music most often lives from the realtime liveness of the experience, as the musicians bounce off one another, come together and create somet hing through rhyt hmic, tonal and physical proximity. Here we see an example of where multiple act ors in a network have different experiences of an int eract ive moment. A good conduct or responds to the orchestra and the nuances of their playing as they respond to her movements and guides, an int eract ive moment of int erest for a number of reasons. However the third actor in this network, the audi ence member, has no significant effect upon the orchestra or the conduct or, only able to listen. The act of listening can be act ive, with the listener act ively part icipating, but there is no clear effect back to the orchestra-conductor unit to let a truly int eract ive moment occur. On the other hand in a small venue, two musicians playing together will be int eract ing strongly as they play together, the audience will respond to their music and the react ions of the audience, il luminated by the lights spilling offstage, will effect the musicians once again forming a multisided network of int eract iv ity. With all likelihood there are a swathe of connect ions to sender-receiver theory, actor-network theory and a whole bunch of other well-thought-out philosophical, sociological and psychoneurological theories and studies to under mine, support, contradict or twist these ideas in a number of ways. Inasmuch as we are not too wrong, we will leave these things by the roadside, comment upon them as int eresting but not for now, and carry on with our plan to have a 46

working idea of int eract ion.

Experthood and Nesting

We think it is worthwhile to int roduce one more idea about int eract ion that we have found useful, that of nest ed in teract ivity. This has been ment ioned above in the discussion about orchestras, conduct ors and audiences, but we think there is a bett er descript ion worth thinking about. When we first start to move, according to many theories of very early childhood development, our percept ual syst ems and motor syst ems are pret ty messy, unstruct ured and chaotic, even random. As the flush of sensory informat ion falls upon our skin, eyes, ears and other sensory syst ems, we begin to organise, arrange and understand how this all fits together. This seems to be some deep fundamental human propert y, whether or not we are a blank slate at birth. Our eyes develop the ability to recognise shapes, then we begin to register the act ions of our hands and feet. Anecdotal evid ence of the surprise and joy on a child's face as they work out the connect ion between cert ain muscle act ions and the re sulting hand act ions indicate that there is a part of our lives when we are learning to even use our hands, taking ad vantage of feedback. Tension a muscle, watch the movement of a limb, recognise a gest ure. We see this in the more tragic case of people who have lost the use of cert ain nerves, brain regions or body parts and who are painfully relearning to use their bodies using all sorts of feedback loops. But at some point, with luck, it becomes sublimated and the act of grasping is natural, reaching out, holding th ings, moving and placing are all second nat ure act ions that no longer consciously use the feedback loops involved. Our mind is free to do more int eresting things like learn to talk, throw toys around to experiment with gravity and other such basic science. When we receive an instrument the first time, say a violin, the scratchy sound that is produced is a horror to the ears. As we play a bit, trying things out, we tense muscles, changing pressure upon the strings, the angle of the bow, the speed of movement. A tight feedback loop is created between the mot ions of our hands, the effects upon the fiddle, the react ions as sound and our react ions to that sound as we try to correct our act ions to make the sound less horrify ing. This learning process goes on and as we move forward, we begin to sublimate the feedback loops so that the sound of the bow across the strings is not modulated by the errors in our presssure correct ion loop but by our desire to have a sound that modulates the int ensity of its sound, the bite of the notes, the abruptness of the staccato att acks. Our fingers on the fingerboard wobble not to find the note but because we are modulating it ever so slightly to add a tragic vibrato. At this point, our conscious mind has to focus less on the feedback int eract ion loop of the violin and our hands and we are freed up to pay att ent ion to the conduct or of the ensemble, to wink at handsome men walking past on the street as they toss a donat ion or listen to our colleagues and improvise freely without too much technical dist ract ion.

Reacting Reactors - hmo

Rocket scientists might at times be helpful but are not of essential import ance to having a working int eract ive syst em. In fact the minimum requirements to be met by an ent ity are very very basic. Being able to react to a stimulus is all that is needed to start playing the game of int eract ivity. When analising some simple int eract ive syst ems the ent ities acting and re-acting within these syst ems with one and another can be divided into three major groups: Humans (h): this group is rather self explanatory Machines (m): For the purpose at hand, "machine" will describe any human-made device able to react to a stimulus, thus being able to be a reacting ent ity in an int eract ive syst em. Other (o): "Other" is an umbrella term for any ent ity which is neiter human nor machine and able to react to a stimulus. This group cont ains also but is not limited to animals, plants, and aliens (once they visit). The possible one-on-one scenarios for int eract ion as all int eract ion is happening via a channel or media (think telephone/computer int erface and the like):

hh

47

hh

This being the most common form of int eract ion and probably also the richest in terms of complexity and occurence is left out without being a dragon in itself but simply because of the enormous scope of the field which is covered to great ext end elsewhere. It will however be on the radar if it occurs within the boundaries of a formal int eract ive syst em.

hm

Many hours of potent ially product ive human lifetime are invest ed in this kind of int eract ivity (think Pong, think DOOM). Human-machine int eract ion happens via an int erface and (in most cases) beholds the necessity of a translat ion process between the ent ities. HM is often a cover for the more complex HMH where machines int erleave and make HH int eract ions possible. Whether this is the social software of Web 2.0 or the social hardware of shared int eract ive installa tions makes no real difference.

mm

Machines "talking" to one and another is also a rather common, but mostly ignored event. One just has to think of mail servers communicating and act ually having a dialogue via a prot ocol with a connect ed computer running an email pro gram. The prot ocol for such in int eract ion is well defined with RFCs (Requests For Comments, the standards of the in ternet) describing the various parts of the prot ocol. These RFCs set up a notat ion to describe the act ions and react ions of each agent in such a MM int eract ion, and this formal descript ion of the prot ocol allows an analysis of the process in order to confirm that the prot ocol is error free. It is also possible to verify any given server implement at ion against the prot ocol, in essence to confirm that the part icular implement at ion, the code, does no more and no less than that which is required by the notat ion in the RFC. 48

ho

Everyone ever having thrown a stick to be ret urned by a playful dog has experienced the joy lying in the repetitive in teract ion with somet hing not being a machine or a human. As the above example shows so well "if" and "then" are es sential to int eract ivity.

oo

Dogs chasing cats and cats ignoring dogs shows that int eract ivity is in no way dependent on the inclusion of humans or machines.

mo

Thanks to the behavioural science and its scientists also animals (even farm animals) get to take IQ and other tests via touchscreens and other animal-computer int erfaces. Int eract ive syst ems usually are composed of more than that minimum of two ent ities, giving social dynamics room to take effect and giving the part icipating ent ities the option to switch between being part of an int eract ion to just being an audience with limited or no influence on the path of events unfolding.

Wanting to be an Actor
The main focus will be laid on human-machine int eract ivity within a formal syst em, but the findings usually are valid for other variants of hmo-interaction. The driving motivat ions for an ent ity to int eract with another one within an int erac tive syst em are manifold, but some core motivat ions can be found far more frequently than others. At a very general level the most potent motivator is joy. The methods of achieving the experience of joy may vary, but they all have in common that they make the individual want to cont inue or repeat the act ion that has led to this desir able condit ion. In (int eract ive) games it is often some form of success, a sense of achievement which keeps the actor "hooked" and wanting more. This search for joy via success in solving problems seems fundamental to (human) nat ure 49

and even if initially it was curiosity or simple obedience to orders which put the individual in that problematic situat ion, the joy coming with success (even if received bit by bit in small doses) soon becomes the main drive for cont inuat ion and repetit ion. At a first glance, the Stockholm Syndrome seems irrelevant for HM int eract ion, but perhaps it is not. The main quest ion for motivat ions is one of access, the first usage. What is the motivat ion to start an int erac tive experience? In the arts this is often less import ant as the piece will be present ed in a space that self-selects people int erest ed in exploring the possibility for int eract ive art. The quest ion as to the welcoming nat ure of the int erface and whether it is open and clear enough can be raised. I may be int erest ed in int eracting, but if I cannot see what the sensor might be or recognise any effects of my act ions (i.e. the react ion is missing) then the initial action-reaction cycle has al ready gone wrong.

Hidden ayers
There is a large and int eresting issue of visible, hidden and deeper levels of int eract ion. As we begin int eracting with a syst em, it is often import ant to have clear react ions. As we begin to understand these int eract ions, our cont rol of the in teract ion leads us to att empt finer cont rol and, with a syst em of int erest, we begin to dig deeper into the possibilities of the syst em. Initially these deeper levels might have been invisible because the sensors were not act ive or, more likely, the det ails of our act ions were not subt le enough to evoke the deeper levels, somewhat like the first stages of learning the violin. The det ails of our act ions were being observed by the syst em but the nuances were not comprehensible to us. The somewhat malicious version of this is the hidden int eract ion, where our int eract ions work according to our expect a tions, clicking a link and moving to a new page, but the extra int eract ions with a Google databank and thereon to the advertising world are hidden from view yet subt ley apparent in the advertisements we are confront ed with. In the next two chapters we will be invest igating cert ain syst ems and the possibilities to notate them in different ways. Since the path is dark and full of dangers, we won't att ack any of the many complex and int eresting platforms that most excite our imaginat ion. Instead, we will follow the strategy of the common neuroscientist, who prefers to work with the humblest of creat ures -the slug-in order to advance without missteps in a simpler researching enviroment. We will start from the earliest and simplest stars of the game world: the most beloved puzzle in hist ory and the first computer game that ever was. Enter slug n.1 and slug n.2.

50

SLUG N. 1: NOTATING THE RUBIK'S CUBE

Everybody knows the Rubik's Cube, a simult aneously frustrating and delightful 3D puzzle that is as iconic of an 80's childhood as arate id and giant shoulder pads. His creator, the Hungarian architect and designer Ern Rubik, called it Bvs Kocka (the Magic Cube) and it became so massively successful it even caused an impact on Hungary's economy (for the bett er). "The device is also somet imes called the Hungarian Horror -informed Time magazine on their March issue for 1981- since it can induce temporary dement ia in otherwise balanced citizens". uckily for all of them, a bearded pro fessor of mathematics created a notat ion syst em for it. So they could cheat. The game was in fact an accident. Ern was trying to resolve a problem for the class he was teaching at the Academy of Applied arts and Design: how blocks could move independently without falling apart. After several failed at tempts involving elast ic bands, he carved a wooden model where blocks were holding themselves together through a complex combinat ion of posit ion and shape and paint ed each side with a different color, as a breadcrumbs guide. It was when he tried to ret urn the blocks to their original posit ions that The Magic was born. It was wonderful to see how, after only a few turns, the colors became mixed, apparently in random fashion. It was tremendously satisfying to watch this color parade. Like after a nice walk when you have seen many lovely sites you de cide to go home, after a while I decided it was time to go home, let us put the cubes back in order. And it was at that mo ment that I came face to face with the Big Challenge: What is the way home? It took him several weeks and numerous calculat ions to regroup the colors, but very litt le to understand the potent ial of his discovery, and applied for his Hungarian patent in January 1975. The Rubik's craze that followed result ed in many cheat books and newspaper pages of "easy" methods for resolving the puzzle, all using idiosyncratic notat ions with piti ful levels of usability. Notes on Rubik's Magic Cube", published in 1980, int roduced a method so simple and int uitive en ough that would rapidly become the standard: the Singmast er's Notat ion. David Singmaster was int erest ed in Group Theory, a branch of abstract algebra where the focus of research is on struct ures more than numbers. His notat ion relies heavily on the posit ion of the player, who must be looking at the cube from slightly above and to the right of it, from where he can see the three faces at the top, front and right. In the Singmaster notat ion, those faces are named "Up", "Front" and "Right" ( U, F and R). The faces the player don't see -the bott om, back and left of the cube- are "Down", "Back" and "eft" (D, B and ).

U for the Upper face F for the Front face D for the Down face 51

 B for the Back face for the eft face R for the Right face

The Singmaster in Detail

A score in this notat ion is a series of moves which are defined as follows. A letter by itself indicates a quarter-turn clockwise. A prime symbol ['] after a letter means to turn the face counter-clockwise a quarter-turn. A letter followed by the numeral 2 (occasionally superscript) means to turn the face 180 (and it doesn't indicate direct ion because both clockwise and anticlockwise lead to the same place). Changing to lowercase (u, f, d, b, l and r) indicates that only the first two layers of that face must be twist ed, leaving the third layer behind.This could of course be done in two movements but such is the ambit ion of the notat ion: to reduce the score to the minimum number of lines. The inner layer of the cube has Middle, Equatorial, and Side, where M is turning the layer between L and R downward (clockwise if looking from the left side); E means turning the layer between U and D to the right (counterclockwise if looking from the top) and S means turning the layer between F and B clockwise. When the whole cube must be turned about one of its axes, the notat ion prescribes a convent ional axis in dicators X, Y, and Z, where X is from left to right, Y from up to down and Z, from the front face to the back, but the rule is scarcely followed, since most players wisely prefer to say: turn the cube upside down.

A sample of the popular but controversial Fridrich Method

Notating is Engineering, not Designing

Singmast er's solut ion allows for players to use "algorithms", which in the Rubik's universe is a sequence of moves that leads to the desired posit ion. The CFOP method -more popularly but somehow cont roversially known as the Fridrich Method-, requires that we first solve the first two layers. This is how the early cheatsheets or algorithms look like: FR: (R U' R') Dw (R' U2) (R U'2) (R' U R) F: (' U Dw') ( U'2) (' Dw2) (R U' R') BR: (R' U R Dw') (R U2) (R' U2) (R U' R') B: ( U'' Dw) (' U'2) ( U2) (' U ) What we learn is that a notat ion for a game must derive from its mechanics and not from its parts. The cube has 54 colored squares but, while that is precisely correct, it is not relevant, because not all of them can be rotated or rearranged. There are only 26 "cubies" that rotate on a central axis, of which 8 are corners, 12 edges, and 6 centers and only 20 of them move, because the centers are fixed. The colors are, in this sense, also dist racting for they do not group similar objects. 52

The "aws of a Rubik's Cube" matter greatly to the synt ax. Following these coordinates, the cube can be orien ted 24 ways: the upper face (U) can be twist ed in 6 different ways and, for each upper face, the front face can be twist ed in 4 different ways (6 4 = 24). The player can only flip an even amount of edges and never flip 2,4,5,7 or 8 corners in the same direct ion. We can only do an even amount of swaps or cycle an odd amount of pieces, and cannot flip a single corner. If we found ourselves with only two pieces to swap, a single edge to flip or a single corner to twist, our cube is simply broken -or badly assembled- and cannot be resolved.

A Curiosity: God's Number

Every serious player ambit ion is perfect ion. In Go, a match between masters where every movement is the best and most inspired movement is called divine (Kami-no-Itte); in chess, a mistake-free match is casually known as The Gold. For speed cubers, there is God's number, the furthest dist ance any posit ion can be from solved. Since July 2010 that number is known to be 20, thanks to Google's employees 20% time program and 35 CPU years of computer power.

53

SLUG N. 2: PONG
At each level of det ail each "turtle" we can similarly describe more complex int eract ive cont ent, with some special precisions that enable us to deal with the complexities of an int eract ive syst em part icularly its dependence on user, player, or audience input. To elaborate each of these levels, well take as example the classic video game (and in fact the classic "example of a video game" video game): Pong. ets walk through the various levels of notational det ail available to us, and look at the ways that an int eract ive notat ion might have some special qualities to att end to and benefit from.

First Turtle: Sometimes a Sentence or two is enough

At the most basic level of notational abstract ion, we have a very brief high-level descript ion, invoking familiar concepts and using litt le jargon, which could be either written or verbal - somet hing like the "Fluxus notat ion", of sentence-asscore. Quite similarly, a game design can be as simple as a sent ence of instruct ions - the game's rules. Recently and notably, gameplay experimenters Hide&Seek created a series of what they call "Tiny Games", int roducing the concept with the tagline "Somet imes a sent ence or two is enough." Tiny Games are gameplay composit ions consisting of a few simple rules designed (or at least adapt ed) for a very specific locat ion and cont ext. For example, one Tiny Game called Eye Contact is meant to be played in a crowd on a terrace in a public place: "A game for two or more players. Race from one end of the terrace to the other. Youre only allowed to move while youre making eye cont act with someone else."

Hide&Seek, Eye Contact, 2012. Photo: Paul Ben nun

For more complex, int eract ive, and digital pieces like our Pong example, the short verbal descript ion is not en ough to encapsulate all the rules, and instead might be considered an "elevator pitch" for the game. Unlike a Fluxus score, which itself is considered sufficient informat ion to perform the piece it represents, a short text notat ion of a video game is not sufficient to bring the game of Pong into existence. It is simply sufficient to give an adult with sufficient command of language and knowledge of cult ure a basic idea of the game and its funct ions. So what makes it a notat ion at all? And what's special about an int eract ive high-level notat ion? For an int eract ive piece, at this ext remely high level it is crucial to speak specifically of what the int eract ant (usually, but not always, a human) must act ively do in order to int eract with the work. As act ions, these descript ions center around descript ive verbs. In the case of Pong, consider the following: "Pong is a simple two-player competitive game resembling Table Tennis (Ping-Pong), with each player oc cupying either the left or right side of a rectangular screen. You play by moving a short bar called a "paddle" up and down along the far edge of your side, in order to hit a bouncing ball and keep it from passing off your side of the screen. Whenever you block the ball, you also send it bouncing back across the screen, and try to cause the other player to miss it." The short verbal notat ion of an int eract ive work should include the Who (is int eracting), the What (the int erac tants do) and the Where (the act ion takes place) of the work. By comparison, here is a text about pong which is tech 54

nically correct, but is far less useful as a notat ion: "Pong is an arrangement of pixels moving on a screen. A ball represent ed by four pixels aligned in a square moves around the screen and somet ime makes numbers increment."

Next Turtle: Visual Overview

At a cert ain level quite early on in a det ailed notat ion or descript ion of an int eract ive work, it becomes necessary to use represent ational visuals. In screen-based (or at least screen-including) works such as a video game, that represent at ion will often be an analog of the screen itself. In a spacialized work (for instance, David Rokeby's Very Nervous System), the visual notat ion will more likely take the form of an architect ural rendering - a diagram with reference to the physical world. And of course, many works will have (and have notated) elements of both. et's look at our Pong example again. In the notat ion below, a rectangle represents the screen of a television or comput er, which is the player's visual int erface - the only means through which int eract ants get feedback to the game state. ines, arrows, dots, and text in the notat ion represent both the on-screen informat ion, and the meta informat ion about what is seen. Color and line style are used to dist inguish the markings which are represent ative of the screen itself, and those which are notat ions referring to the behavior of the syst em. For instance, black markings in this Pong diagram in dicate the on-screen elements themselves: the solid black rectangle represents the boundaries of the playing field, the dott ed center line is the symbolic boundary of each player's side of the court, the small black bars are the two player "paddles," the black square is the ball, and the numbers are the score. Other informat ion about the behavior of the sys tem is described using a syst em of colors. The "walls" are shown and notated in green, the "goals" in pink, and the mo tion of the player paddles in brown. Of course these color choices are somewhat arbitrary - the import ant point is that some sort of meta-information (color, font, line style, etc) is used to dist inguish different types of informat ion and help visually organize (separate or connect) elements of the notat ion.

This notat ion type is very good at explaining elements of the syst em which are cont inually present in the experience. In a work which often changes its means of feedback or has different int eract ion "states," a notat ion will need multiple re nderings in order to describe each of them. If the int eract ion feedback is in constant flux with litt le or no consist ency from moment to moment, then the screen diagram may not be a very useful form of notat ion for that work.

Next Turtle: Storyboard

55

Next Turtle: Storyboard

Another type of visual notat ion is the storyboard or keyframe animat ion. The int eract ive work is represent ed in multiple consecutive "frames" representing the same experience capt ured at different moments in time. Notat ions of this kind are more useful in a non-interactive linear work, to describe in a few snapshots the course of act ion over a longer time. In int eract ive work, a storyboard such as this one from our pong example below, might be most useful for explaining what did or could happen during the course of one playthrough of the game. As in the annotated screenshot style, a visual (colorful) dist inct ion is made between the elements of the screen itself (here shown in thick black line) and the meta-information, which here represents the behaviors of elements such as the mot ion of the ball, and the change of the score number.

In storyboard/keyframes of this type, it's import ant to think about the proper granularity of the sequence. What mo ments in the experience are import ant to capt ure in the portrayal of change over time? Are there ext remes of mot ion which would be meaningful for explaining the ent ire gest ure, such as with animat ion keyframes? Or is it necessary to break down mot ions into smaller increments to show the process of changes taking place, and allowing the human mind to re-create the mot ions without much int erpolat ion? All in all, the storyboard descript ion of Pong shown here is at a rather fine level of det ail. This level might be useful, for instance, to notate the desired or expect ed physics behavior of the ball bouncing around the court, or it might be a useful visual way to describe the typical progress of a game to someone who has not played it.

Next Turtle: Control/interface Diagram

The aim of this level of notat ion is showing the det ails of how the int eract ant will express themselves. What is their range of possible act ion? This will vary based on platform, even within the same work. Missing in the screen-based visual represent at ion of a work is one very crucial element: the means of int eracting with it the language of human gest ures or act ions that makes the work act ually int eract ive. A notat ion of an int eract ive syst em without some discussion of the int erface would be grossly incomplete. Of course just about any type of human mot ion could be recognized by an int eract ive syst em and turned into a react ion within the syst em, whether that act ion is "freest anding" or in relat ion to a physical object. Conscious int eract ion ranges from the tiniest muscle movement, such as a subt le touch on a sensitive touchpad or the twitch of an iris or eyelid, as in the EyeWriter and other more conven tional assist ance technologies, to full-body swinging and jumping as in many sensor-based artwork, or pushing and pull ing the massive levers inside the cont rol booth of a construct ion crane.

In the case of Pong, these diagrams notate two of the earliest and best-recognized physical int erfaces to the game: the paddle and the joyst ick. The "paddle" cont roller is in fact is so closely associated with the game Pong because it was created and sold as a cont roller specifically for the earliest versions of the television game. This explains the unusual 56

naming of the cont roller, which bears no visual similarity to the traditional analog table tennis paddle after which it is named. In these two part icular Pong cont rol illustrat ions, the cont roller itself is depict ed in a clean-lined represent at ion, while the specific act ions available to the player are annotated in brown. We can see from the two side-by-side represen tat ions that the user's act ions are very simple, and that the two cont rollers, though using different direct ions of move ment (rotating around a central axis, or bi-directional in a line) are accomplishing the same act ions within the game in terface (moving the player's paddle up and down along the edge of the court.) This abstract ion between specific physical int erfaces, and syst em behaviors, is one of the most import ant reasons to specifically include an int erface diagram in the notat ion of any int eract ive syst em.

Next Turtle: Flow Charts and State Charts

Flow charts and state diagrams are perhaps the notat ion methods most closely associated with the design of int eract ive work, in that they are the simplest and most comprehensible notat ion syst ems which att empts to capt ure the syst em's logic: the cause and effect of specific int eract ions themselves. They show how the choices that the int eract ant make over the course of the int eract ion session will change the work, and how it will respond to the int eract ant with new in format ion that allows them to make further choices. Within these categories of int eract ion charts there are multiple standards used in various professions to improve legibil ity and cont inuity of meaning. These standards usually include specific shapes such as rectangles, diamonds, and ovals to communicate cert ain types of meaning such as decisions and outcomes. Depending on the need of communicating through the diagram with different groups of people (and on their familiarity with the standard), it may make sense to use these standardized notat ion syst ems, though its not necessary. This flow chart for Pong is an example of an att empt to describe its "game loop." Each round or match of the game is described from the standpoint of the ball's behavior, and the game's react ion to its state. The end of the flowchart is reached, the current match is done, and if the current match led to the winning point scored, then the en tire game session is complete. Notably, lower levels of the gameplay, such as the physics cont rolling the speed and direc tion of the ball, are not dealt with at this level of det ail. It's noted at cert ain moments that the ball will change direct ion and velocity, but the calculat ions behind these changes are not explained.

57

Next Turtle: Rules

In an analog int eract ive piece which depends more on player/ interact ant behavior for the syst em to run (for instance, Pict ionary, or Tennis, or our Tiny Games example above), the collect ion of step-by-step rules and various "if-then" cases could be considered synonymous with the algorithm of the int eract ive work itself. Rule descript ions can be text only, or a combinat ion of text and images. In general they are written as commands to the int eract ant, instructing them what to do from moment to moment and in part icular cases (which are states of the syst em). For instance in the large-scale analog game Interference by Nathalie Pozzi and Eric Zimmerman, the rules were described to players with text and images on a sheet of paper that was available near the games playing area. The notat ion cons ists of close-up images of the playing field and pieces, along with expect ed steps of act ion, and even small pict ograms showing moments in an example play session.

58

59

Pozzi, Nath alie and Eric Zimmerman. "Rules for Interference." 2012.

In digital int eract ive cont ent, it can also useful to notate the work using the sequence of rules, and this method is par ticularly used to teach the behavior of the syst em to a new int eract ant. A special variat ion on listing the rules is a tutori al, in which each step of the rules is accompanied by an int eract ive opport unity for the int eract ant to try the specific rule. Since digital int eract ive work can sense the int eract ants input, it can det ermine whether the int eract ant has suc cessfully understood the rule and move on to the next. In our Pong example, a notat ion of the rules (written or spoken outside the game) would simply say "Cont rol the paddle by pushing your joyst ick up and down," while a tutorial inside the game might, after delivering the same informa tion, wait to receive input from the game that the player had successfully moved their paddle all the way to the top and the bott om of the navigable area before cont inuing to the next step in the instruct ions. In this way, int eract ion itself becomes part of the notat ion.

Next Turtle: Game Code

The software code underlying any digital work is of course a type of notat ion. However for the purposes of our discuss ion, we feel that the int eresting examinat ion is of the less precise, more int erpretable and generally readable (not requir ing specialist knowledge of part icular synt ax) levels of notat ion. For that reason, we have set aside discussions of Pseudocode, scripting languages, high level computer languages, machine languages, and the like. However all of these types of det ailed notat ion could be fruitful areas of inquiry in fut ure work.

60

Next Turtle: Notating the Play of a Game

Next Turtle: Notating the Play of a Game

Finally it's worth looking at codified ways of describing the moment-to-moment play within a single instance of a single int eract ive work. ike other media (music, dance), notat ion can serve to record the act ual process of the work in act ion. Moment-to-moment notat ions are very useful for games that are heavy in strategy and do not place significant value on the quality of physical movement. In cert ain strategic games this type of notat ion act ually can become quite simple, as all of the act ions are easily convert ed into discrete symbols. nowledge of the game's rules, combined with a list of the act ions, can render a precise impression of the session. This is especially notable with classic games like Chess and Bridge, and very likely cont ributes to their communicability and popularity through time. Other games with complex physical struct ures are somet imes notated through a series of images or diagrams of movement. For instance the game Go can be portrayed as a sequence showing each move by the two alt ernating players. The sport of American Football is often notated (especially for training purposes) with a book of strategic illustrat ions called a Playbook. By comparison - as a thought experiment - we could imagine that it is possible to real-time-notate an int eract ive work such as Pong, which uses simple physical cont rols in real time. For instance, what if a joyst ick cont roller being used in a game of pong were also connect ed to an old-fashioned mechanical plotter (such as those used in seismographs), with the paper roll moving at a constant speed? Through this method, you could track a player's vertical path over the course of an ent ire game, with fast and sudden moves creating sharp inclines, and slow, moves creating gradual slopes. Indeed, it would be int eresting to use the output as a musical score!

61

EXAMPES

62

INTRODUCTION TO THE EXAMPLES

In cont emporary art istic practise most of the time artists invent a score for just one specific purpose. This goes so far that, even in music, few pieces are played more than once. As ment ioned in the Steal this Notation chapter, artists feel rightly free to grab whatever they need from any existing notat ion syst ems to express themselves. In many ways during these process new symbols appear and are fed back into existing syst ems, thereby changing and expanding them. A good image for that process is the cathedral and the bazar (as stolen from the famous essay by free software advocate Eric S. Raymond). et us say the cathedral is the classical music notat ion developed over thousands of years and the bazar are all the individual approaches to creating a score. The bazar is a teeming mass of independent act ors, all fol lowing their own ideas and desires, taking on projects and abandoning them when finished or losing int erest, recycling the parts left lying around in ways unintended by the original construct ors and att aching whatever they like to whatev er they can as they see fit. The cathedral is the syst em that "just works," where one is allowed to enter and do what one must, but there is no way to att ach a rope to the wall or assemble a small stall in an unused corner. The cathedral does what it should and new funct ionality is a rare thing. But the cathedral and the bazar are cont inuously influencing each other. The cathedral absorbs parts of the bazar and curiosities inside the cathedral start to get new meanings in the bazar. There are no radical changes in either arena, att empts at denoting innovat ion are more like setting markers in a time cont inuum. The cathedral reacts slowly and ponderously with due regard for tradit ion and backwards compatibility. The bazar is a teeming mass of tiny incre ments where every development can be seen to derive from a number of other developments, remixing them in int erest ing ways. At some point the quant itative difference is large enough to feel like a qualitative one, and someone gets the credit for innovat ion. The following examples show art istic approaches towards notating their work for different purposes. Many of these examples borrow notational techniques from their own and other fields or misuse techniques that they find in teresting. These sd-hoc techniques are very carefully rooted in the bazar, and some might also say bizarre, with explana tions needed. This is an import ant aspects of any non cathedral, and perhaps many cathedral, notat ions. The semant ics of the symbols are unclear and require negotiat ion. The negotiat ion may be one sided as when the artist makes state ments about what exactly each symbol and connect ion means, which will be the case with more developed works. Other negotiat ions might be more explorative, being developed by colleagues working together to discuss a problem and want ing to be talking about cert ain aspects as they arise in the discussion. A further category might be taken from some of the less well defined musical scores, where the score might appear to be litt le more than a postcard of a painting or mess of lines and colour swatches, where the performers have to negotiate how they will go about int erpreting the score without being able to int eract with the creator of the score. The following examples will unpack some of these issues and hopefully illuminate them in int eresting ways.

63

BLACK BOX SESSIONS (2008)

Alex Davies2 is a sound & video artist. His work spans a diverse range of media including film, network, realtime audiovisual manipulat ions and responsive installat ions. Times Up2 is an art connect ive based in inz, Austria. Founded in 1996, Time's Up has its principal locus in the inz harbour of Austria. The mission of the connect ive [sic!] mission is to invest igate the ways in which people int eract with and explore their physical surroundings as a complete cont ext, discovering, learning and communicating as they do.They have collaborated ext ensively since 1999.

Working Out the Possibilities

A mediated installat ion environment which offers the public a dist inct perspect ive on live performances. Times Up and Alex Davies have curated a series of performances present ed by 11 national and int ernational performers who will be present throughout the course of the exhibit ion. The performances take place is a pitch black room and are viewed by individual audience members via an infra-red camera and monitor syst em. This unique environment shifts the relationship between performer and spect ator and challenges dominant visual percept ion.

http://timesup.org/content/black-box-sessions-10-2009 Only one audience member at a time goes into a completely black environment where the only light source is a hole the size of a pin prick. ooking through the hole a screen can be seen showing an infrared image of them in the space. The audience sees themselves from behind as well as other things around them, including the performers ap parently walking in and performing in the dark next to them. The audience member can experience the performance only through the camera-screen channel. The image below shows the artist Patrick Huber performing with an audience member:

64

http://timesup.org/content/black-box-sessions-10-2009 The int eresting thing was that Time's Up and Alex Davies had created a 'green box' scenario where they did re cordings of the performers acting as if they were in the dark. These green box recordings were then treated so that they had no background (green removal) and made to look like they had been filmed with the same camera as the infrared surveillance camera that was watching the audience members from behind. Then the audience member could see them selves in the pitch black space and could only see the performers through the technology of the camera and the screen. If the audience member left the screen, they were alone ina pitch dark room which, after having the bright screen in one eye, was even darker than before. Att empts to find the performer were destined to fail, as they were not there, but the confusion of darkness and the multichannel sound syst em helped create a convincing illusion. The performance recordings were planned to start after the audience member had arrived at the peephole. Sen sors in the ent rance tunnel regist ered the movement and direct ion of a visitor, a second sensor regist ered whether they were standing in front of the peephole. It was assumed that the audience member was alone, this was stated at th en trance and a red-green light combinat ion was used to indicate whether a new audience member could enter. They want ed to be able to synchronize it all so it worked properly, leading the crew to sit down and work out everyt hing which could happen in the space, including playing the video and merging it with the live footage from the camera. To help realise this, they created a diagram showing the ideal things which would happen and the not so ideal:

65

Some of the terminology used on the page might help understanding. The bott om right shows the Black Box from above. A visitor (V in the rest of the diagram) passes the 1 sensor then the 2 sensor as they come into the Black Box and then stand in front of the screen. On the left side of the diagram there is the comment "nothing" indicating that nothing is going on. In this state, the green lamp is on. The correct next act ion is for a visitor to enter, triggering the 1 sensor and moving to state "V in " where the lamp turns to red. If they turn around and leave, not liking the dark, the state changes back to "nothing" and the lamp goes green. The lamp remains red in every state other than "nothing" as only in the "nothing" state does the syst em believe there is nobody in the Black Box. Normally the visitor then triggers the 2 sensor, so the state is "V in room" before the screen sensor is tri ggered indicating that the visitor is looking into the screen hole. Then the state becomes "V at screen" and the video is played. When the video finishes, "playend" triggers, the visitor leaves the screen and then triggers the 2 and then 1 as they leave and the syst em state ret urns to "nothing" so the lamp is green and the next audience member can enter. The rest of the diagram is filled with lines and states indicating the ways that the visitor and the syst em are doing things that have not been planned for. Perhaps the visitor leaves the screen before the video finishes and wanders around in the dark. Or even leaves the Black Box completely. Perhaps some sensor triggers unexpectedly. The discussion amongst the crew tried to work out what to do in all these cases. This diagram was developed in order to enabl the Max MSP patch to do everyt hing at the correct time. They also created a numbered diagram to assist with working out this process, as the diagram above is filled with all sorts of human readable informat ion.

66

This piece is a more simple piece, with few enough states that it was analysable on an A4 page. The process of discussing the piece and its dynamics on a page with human readable comments and annotated arcs between named states made analysing the syst em feasible and helped work out what the appropriate dynamics should be when the unexpected oc cured. Translating this to the machine readable situat ion of a series of rows of numbers, one row per state, made the dynamics readable and implementable by the Max/MSP patch. This series of translat ions, from the human readable state diagram to the numbered version to the array of numbers to the code that would implement the state machine dynamics, is error prone but given that the implement at ion is correct, the three notat ions are struct urally isomorphic. 1. 2. Bio and other det ails at http://schizophonia.com/ ^ Website: http://timesup.org ^

67

20 SECONDS INTO THE FUTURE (2010)

Founded in 1996, Time's Up has its principal locus in the inz harbour of Austria. The mission of the connect ive [sic!] mission is to invest igate the ways in which people int eract with and explore their physical surroundings as a complete cont ext, discovering, learning and communicating as they do.1

Semantic Networks for Ideas

The Physical Narrat ion installat ion 20 Seconds into the Future2 was shown at the "ong Night of Research" in November 2010 at the Johannes epler University in inz, Austria. The piece is a room, apparently the room of a scientist, inves tigating the ways in which some mathematical ideas and some physical properties int eract. The piece frames itself as a science and research communicat ion piece. Visitors are welcomed into the room as if the scientist, Henry adigan, has just left and will be back shortly. They are asked to make themselves at home and look around the room. In the room there is not only the present at ion of the work the character would have talked about, but it is also his working space.

As a result the visitor not only receives a present at ion about science, but also about the life of a scientist, about the motivat ions to undert ake science, to carry out experiments and research in general, to work the way he chose to. In the window a clock radio is playing a radio talkback show with the int erviewee talking about their life as an independent scientist. Callers ask about non institutional science, raising issues with cont emporary research pract ices, and a wide ranging conversat ion takes place. The radio show is formed in an endless loop, the listener always has the feeling to be in the middle of the show. Faxes arrive and messages are left on the answering machine about his life as an independent scientist, the lack of respect one receives and the freedom one has. In the corner the inact ive machine that is seen in a video stands, in the video there is evidence present ed of time travel into the fut ure, where a watch undergoes the experi ence of travelling 20 seconds into the fut ure. The sound effects match those from the next room - the visitor is lead to believe that adigan is in the next room and is currently travelling into the fut ure - at least he will have the experience of spending a few minutes in the machine, while a much longer time will have passed. Feasible time travel is a lot more boring than the classical causal loop stuff of science fict ion. The experience of visiting the space is one of following links from one object to another. A visitor finds somet h ing to be int erest ed in, whether it be video, an int eract ive visualisat ion, a computer game, a mathematical genealogical 68

tree or any number of other art efacts of the scient ist's life. istening to the radio show, the speakers chat about subjects that appear elsewhere in the room. The fax messages and answering machine offer suggest ions, a letter next to the com puter screen from a friend in Namibia adds some colour that explains a photo perched on the shelving and an art efact on the window sill. The form of the room is a network of objects, semant ically linked by reference and similarity. Thus for an analysis of the piece, we want ed to look at the way that various objects were semant ically related. We collect ed many of the objects and joined them. The automatic graph visualisat ion package GraphViz was used to lay out the graph.

The resulting graph, which is by no means a complete mapping of all the objects in the space, shows the way that a visitor can move from a given object in the room to an idea, story element or concept and then onto other objects. From the fragment shown above, one can see that many of the concepts are related and reachable from some of the eas ily played with items such as the 3 sided Hyperbolic Pong game. One element that is not att ached to anything else, however, is the photograph of a woman who could be his mother. There is no indicat ion on the back or elsewhere who this person might be, so the visitor is left to use their int ui tion based upon the clothing and hair style, the tint of the photo and its posit ion in the room. However this photo is not connect ed to any other objects in the room, and only vaguely connect ed to the fact of his childlessness (this connect ion is only vague so has not been indicated in the diagram). Thus we could say that the photo is removable - we will not lose anything of import ance by omitting it and may help clarify the installat ion by removing narrational clutter. Or it sug gests that the story element needs to be bett er int egrated, with a telephone call ment ioning her or a letter from her tuc ked into the back of the photo frame. This notational effort was of value in order to analyse the piece. A more comprehensive piece of notat ion would enhance the ability to evaluate the usefulness of each element of the installat ion. The purpose when creating these net works was similar, to see whether the visitor was passed on from element to element. This was not clear from the frag ment, as it would only work when completed, so that all installat ion elements were present and could be looked at. The notat ion would also help work out what smaller version of the piece could be present ed in a way that would still work effectively. A reduced version should not have any isolated elements, for instance. 1. http://timesup.org/ ^ 2. http://timesup.org/content/20-seconds-future ^

69

HACKING CHOREOGRAPHY BETA V.02 (2012)

ate Sicchio is a choreographer, media artist, researcher and performer whose work spans from dance performances, in stallat ions web and video projects. In her research she looks into parallels between code and choreography as well as to find implicat ions in areas such as live notat ion and live coding. Most of ate's work involves live realtime project ion in live performance, which she programs, choreographs and performs. She normally creates all the elements of the pieces. In her head she doesn't separate these things - they're all choreography to her, including sitting and codi ng. Her project developed out of the idea of how does code become choreography. abans notat ion syst em focused on "the body" not the general space of the performance." She thinks in terms of relationships as choreography and is int erest ed in finding new ways to hack choreographic piece.

Notation in ive Compositional Processes

Hacking choreography beta v.02 was an experiment at University of incoln. There she developed a series of experiments to test out some theories. A choreographic object is not a substitute for the body, but rather an alt ernative site for the understanding of potent ial instigat ion and organizat ion of act ion to reside. Ideally, choreographic ideas in this form would draw an att entive, diver se readership that would event ually understand and, hopefully, champion the innumerable manifest at ions, old and new, of choreographic thinking a quote by William Forsythe (2009) So, we're not substituting the body, there is still a relationship between the body and choreography, it's just not centred around the body anymore like aben's work and the focus has moved a litt le away from it. Forsythe pro poses that choreography is a way of thinking. One thing which ate doesn't quite agree with is his use of the word 'or ganisat ion'. She likes to think of choreography as relationships rather than this. Within her own work she want ed to find somet hing similar which she could do with scores, and start to change them in realtime. With her video tracking syst ems she jacked a inect and was int erest ed in how she could hack the movement syst em. She began with existing scores, taking this as the code and then changed it to make the perfor mance. Below is the first hack she start ed with, taken from the Fluxus Performance Workbook:

70

And here it is being performed by ate:1 In this score she labeled objects and created hers with paper and stickers. The first part was her following the score, int erpreting it rather than hacking it and she used audience members within the piece as the score said you could. She did this with very litt le preparat ion, the idea being that the composit ion would emerge in realtime. This is because she want ed the hack to be in realtime. There isn't a lot of choice within the score of what to do, so she want ed to make her decisions in a realtime, pressured situat ion. The initial way she decided to hack it was to rip up the paper and catego ries, and making new ones out of the language which exist ed. She ended the piece when she got frustrated with not knowing what she had done, fed up with making the decisions in realtime. The next choreographic hack was when she made her own code. It's made to look like java script, but also made so a dancer could read it:

71

Finally she had a setup with two dancers, defining the movement she was int erest ed in. The dancers had to int erpret this. The choreography sect ion is the relationships created. She then gave the dancers an order to run it in, giving the 72

code to them the day before the performance, without any other instruct ions and so they had to work them out them selves. Halfway through the performance ate put up new instruct ions, this was the hack, and the dancer's had to chan ge their relationship depending on this. ate did act ually give them the hack an hour before the performance, she they knew it was coming but they had to figure it out in the moment.2 The order they performed it in was the movements straight through, then they performed it with the re lationships and then they performed the hack when it appeared on the screen. One of ate's colleagues in the computer science department got very excited about the piece and wrote the code up act ually in Java script as opposed to ate's pseudo script, but it is very similar:

The last one ate made was about the dancer being a hacker, where she gave verbal instruct ions:3 What was int eresting about this one was, that the score start ed with what ate was saying, but then the dancer had to slowly change what she was doing so she wasn't doing what ate was instructing. The score isn't what ate was saying at all, just at the beginning, making the dancer the hacker. This experience led ate to a community of live cod ers. What she has in common with them is live compositional processes, in that there is some kind of syst em or score that is set up and within it there is a frame from change. There is also a transparency about it, with the project ion of code. This led her to work with Alex Mcean on a piece called Prism 11.

Prism 11: Real Time Notation

73

Prism 11: Real Time Notation

In this piece they tried to build up a feedback loop between them. ate would do set movements which she had precoded and were project ed behind her and he was using his syst em (unfort unately he had brought the wrong one for the performance), which was project ed too. ate would change dynamics and qualities of the movements whilst performing, and he would change the dynamics and qualities of the sound he was producing, which would then in turn make ate change what she was doing; this created a feedback loop. Both the codes were project ed side by side. This was completely un-rehearsed beforehand and most of ate's choreography doesn't use sound so this was a big challenge. They want to perform it again in a rehearsed manner. There is a score but this live compositional process is the priority. ate is taking part in other choreographic hacks. One at the Arnolfini in Brist ol, with a live notat ion group who look at where live code and live art meet. ate is one of the live coders, and the artists are going to draw ate a score, which she follows live as it gets created. 1. http://vimeo.com/36369236 ^ 2. https://vimeo.com/36369338 ^ 3. https://vimeo.com/36369416 ^

74

ISORHYTHMIC VARIATIONS (2006)

Michael J. Schumacher1 is a composer, performer and installat ion artist based in New York City. Schumacher works pre dominantly with electronic and digital media, specializing in computer generated sound environments that evolve con tinuously for long time periods. ISORHYTHMIC VARIATIONS came about as an att empt to convert my installat ion "Noema" into a performance piece. Originally it was a commission from Ne(x)tworks Ensemble, who performed it at the Stone in NYC. Isorhythm is a technique used in the middle ages where a rhyt hmic patt ern and melodic patt ern of different lengths were repeated together. In this case, the rhyt hmic patt ern is fixed, and is the same length for all players. Instead of a repeating melodic figure players choose from a group of motives, pitches, ext ended techniques, etc. Melodies emerge from the int eract ion of the ensemble. ISORHYTHMIC VARIATIONS may be performed with or without electronic accompaniment. A new version of ISORHYTHMIC VARIATIONS was just composed in collaborat ion with Sabrina Schroeder as part of an exquisite corpse composit ion. It will be performed by Cont act Music of Toront o Canada.

Score

75

76

1. http://www.michaeljschumacher.com/ ^

77

SITTING IN MY CHAIR (2001)

Elisabeth Schimana1 has been working as a composer, performer and radio artist since 1983. She has ongoing coopera tions with the Austrian unstradio and the Theremin Center Moscow. She also focusses on research in the field of women, art and technology. 2005 she founded IMA (Institut for Media Archeology)2 Elisabeth Schimana: composit ion, sound design, vocals, live performer, Andre Smirnov: instrument design and nato programming, Yuri Spitsin: sound design and kyma programming, sitting in my chair was produced by Musikprotokoll in cooperat ion with the Theremin Center Moscow

Notation for an interactive Performance

The armchair addressed in the title is a very special chair, not a synonym for chairs in general, but exactly this chair whose charact erist ic screech was the beginning point for the musical design of 'sitting in my chair'. The acoust ic materi al consists of these screeches, which were recorded as audio data and then (re)played during the performance. In this very chair the sounds are manipulated simply through the movement of the hands. It's not a coincidence that Elisabeth Schimana developed this piece in Moscow at the electronic Institute of the Tschaikovski Conservatory, an Institute with the proud name Theremin Center. As Schimana discovered the Theremin years ago, it wasn't so much the sounds that int erest ed her, but the body and stage presence of this antique instrument that is played without touching. What is the body doing on stage in the age of electronic music? This quest ion with its area of tension between raftwerks Puppets and today's expressively malt reated lapt ops is uncoiled again and again by Elisabeth Schimana in her work. The Theremin surfaced as the perfect challenge: for what reason the thereminic ant enna technology with its transparent magnetic field as int erface are used - as direct sound cont rol or as trigger for completely different mechanisms - it pro vides the electronic musician a necessity for a physical presence. The physical mot ion on stage as a category is again scrutinised if in 'sitting in my chair' the artist sits almost motionless in her chair. With minimal finger, hand, and head movements, the dense snarl of chair screeches are erased while the stringent lighting is changed one field after the other with every erasing procedure. (nina ross )

The Score

ch=output channel, lr=soundlayer, r.h=right hand, l.h=left hand, c.p=center posit ion, v=velocity of the hands move ments Performance time: Is depending on the performers act ions. miminum time would be 160 seconds. 78

Chair sound matrix: Is the heart of the piece. The target is to erase via the thereminsensor int erface each of the 48 cells. Each cell is connect ed to the light matrix and any state of the matrix is generating informat ion for the panning and amplitude progress. Live processed voice: The voice is free, not connect ed to the matrix and processed by granular synt hesis, which generates three slightly different soundstreams out of one vocal input. Light: has to be considered as two overlapping layers of sqares each of them divided in the raster of 8*6. The 48 patt erns are generated depending on the informat ion generated by the chairsoundmatrix. The focus is on the changing light rays inside the light cone, rather than on the project ion on the floor. The only possibility to make this cone visible is to produce a lot of smoke. In the final state the performer is disappearing in light. Panning and amplitude: Both parameters depend on the relative time informat ion (how many cells are erased) generated by the chairsoundmatrix. You can consider the circle as the space for the audience, chairsounds are moving to the center mixed into one stream while the granularstream is moving outside and splitting up into three streams. At the same time the amplitude is decreasing to zero. Chairsoundlayers: All layers cont ain recorded chairsounds from the performance chair without any additional sound processing. ayer 1 cont ains cont inous chairsounds and layer 2 peak fragments of layer 1. All three output chan nels differ from each other. The looped soundstreams durat ion is free in time not depending on the matrix time, which results connect ed to the matrix time in a permanent shifting of the material.

Sensor stuff and engines

Velocity of the hands movements : After experimenting it turned out that it makes sense to divide the hands movements into 4 different velocities, from v1 slow movement to v4 fast movement to cont rol the syst em. velocities 1 to 3 are re lated to the soundlayers, with each act ion one cell will be erased - 20 seconds of sound disappear - the light patt ern is changing. Velocity of the hand movements and center position: R.h/v4 is switching voice on and off, l.h/v4 is switching between the granular synt hesis, parameters grain durat ion and grain dencity. This two parameters are cont rolled by the movements of the cent erposit ion, very tiny movements with the head and the upper part of the body. The body has to be tuned: In the ant ennas' sphere of act ion the zero point has to be found to tune the body to the syst em. All data generated by the performer's body movements are derived by two identical theremin-sensors located to the performers left and right. Sensorantennas: The construct ion of both sensors is based on the theremin principle: to change the frequency of the ultrasonic oscillator by the capacitance of the performers body, hand or whatever goes close to the ant enna. The audio frequency sound is produced then by heterodyning the outputs of two ultrasonic oscillators. The fixed oscillator operates in the region of 350 Hz with the above ment ioned variable oscillator being above this frequency, the differ ence equaling the frequency of the sound being produced. Allthough that would only make a minute difference, the theremin sensor cleverly has two very high frequency oscillators. That way, even a 0.05% change in the variable os cillator can be substant ial at audio frequency -enough, with good design, to give a range of several oct aves. Analysing the sensor signals : Two audio range signals from the sensors are producing the stereo input for the computer. a special developed MAX/MSP patch analyses the input signal's pitches, converts them into float point values and produces filt ering and scaling to achieve a stable and linear mapping of the measured data over the whole range of possible dist ances. In "sitting in my chair" the patch is int erpreting the derived floatingpoint data as velocity (increasing and decreasing values over time) of the hands' movements. Center position: It means, that if the performer is sitting in the point of the equal dist ance from both sensors, they produce equal signals and taking the difference between the values of the analysed signals and scaling them we can get values, reflecting even the slightest deviat ions of the body from the central posit ion. Generating midi data: Several treshhold- presets are set in the MAX/MSP patch to produce the proper MIDI data to connect with other programs and machines. The kyma engine: Is responsible for the chairsoundmatrix, live voice processing and sound streams mixing. based on the received MIDI informat ion and the circular time pointer posit ion (0seconds to 160seconds) the targeted chair soundmatrix cell index is calculated. This cell index is deact ivated which results in immediate sound cancellat ion. At the same time the cell index is passed down the pipeline to the NATO component. 79

NATO: Is an ext ent ion of the MAX/MSP program. It produces the appropriate video images to be project ed creating the light cone.

Initial state, photo from the performance at Musikprotokoll / Graz 2001

Inbetween states, photo from the performance at Musikprotokoll / Graz 2001 1. http://elise.at ^ 2. http://ima.or.at/ ^

80

FORMOCRACY (2012)
ev edit and Andreas Dekrout both work at Game Gest alt 1, a game design company based in Vienna. ev edit teaches game design at several institut ions and universities, he also is the CEO of Game Gest alt. Andreas Dekrout has a background in legal academic work, social ent repreneurship, and act ivism. He is the project manager of Formocracy.

When the Map became the andscape

The project Formocracy is about the creat ion of real-world political pressure for desired change by members of the online community whith int egrated mechanics producing precise action-oriented orders which are backed by the legitimacy of democratic processes. All this is powered by the psychological mechanics taken from gamedesign, to motivate users to "play" for a higher purpose while simply satisfying their urge for joy. This example follows the path of development from the initial "lets do that thing", existing only as an exciting goal in our minds. Via a trial and error phase of trying to develop a working struct ure capable of achieving that goal we came to a notat ion syst em. The evolut ion of that notat ion syst em and the feedback effects upon the form of notat ion syst ems used at several stages had an enabling effect on the further development of the project as a whole. As the struct ure be came clearer and clearer in our thoughts so did also the way we would draw sketches and charts.

The image shows one of the first drafts for the decision-making part of Formocracy. During the first few (full) days of discussion and turning the idea which was later to become formocracy around in our heads the mechanics and logical sequences were draft ed out on a piece of paper (in a restaurant during dinner). Without prior agreement on any specific notat ion syst em, we used what felt natural to us. anguage (German), lines, arrows, wave-lines, boxes, and so forth. Using this int uitive notat ion we were able to work on what turned out to be less clear than expect ed and could edit and change the concept at the same time.

81

Flowchart for the logic sequence for a proposal within the Formocracy syst em in the virt ual and real world. After the most essential quest ions and challenges were solved the need to communicate the planned flow of act ion arose. The above flowchart is to be read within the cont ext of the design document, it is embeded in it. It does make some sense, but is not completely understandable without additional informat ion.

82

Pen on paper draft for an aspect of the user-interface in Formocracy (the user-profile page) plus some logic and ex planatory comments. Making user-user and user-system int aract ion possible, the int erface must be as self explanatory as possible, while being capable of transferring all necessary act ion and informat ion. This makes the user-interface a nota tion syst em in its own right.

83

Mock-up of a user-profile page for formocracy with explanatory comments in boxes. Putting less of a demand of abstrac tion tasks on the viewer, we produced the above mock-up for demonstrat ion purposes only. The development work is also, at this stage of realisat ion, still done with pen amd paper.

84

Flowchart representation of out and ingoing information and user flow and loops of the formocracy platform with examples and minimal explanations done in textform for presentation purposes.

Conclusion
Especially when working on a project of such high complexity and with int erdependencies between all kinds of aspects (from security-challenges to ethic standards to the fun fact or) you learn very quickly that things have a very very strong tendency of getting more complex and complicated the more you work on them. Initially we used mindmap-like re present at ions to accompany our conversat ions, just to make sure we were talking about the same thing. Very soon these "maps" became our trust ed method of navigat ion in the fast-growing landscape of thoughts that we created on our way. Unintent ionally these original notat ions of our thoughts formed and influenced the way we struct ured and thought about their cont ent. This feedback loop between our imaginat ion of struct ures and their represent at ion turned out to be ext remely useful. In a way thinking about our own ext ernalised thoughts created a creative syst em of "int eract ivity with self".

In our last transformational step all our notes, mindmaps, and flowcharts were collect ed and made publicly available in the form of a design document.2 While such a design document consists of struct ured text and explanatory and exemplary graphics, it must not be overseen that it is a "cooking-recipe" of its very own kind. When starting to act ually program the software this is the one point of reference by which success or failure of the programmers is judged. 1. http://gamegest alt.com/ ^ 2. at: http://fc.yurp.at:8888/display/FORM/ ^

85

LONDON IMPROVISERS ORCHESTRA

Elisabeth Schimana1 has been working as a composer, performer and radio artist since 1983. She has ongoing coopera tions with the Austrian unstradio and the Theremincenter Moscow. She also focus on research in the field of woman, art and technology. She is also a founding member of IMA (Institut for Media Archeology).2

Gestural realtime notation for an improvising orchestra

In improvising orchestras, the notat ion usually consists of live generated gest ural hand signs. Some are repeatedly applied and used in multiple composit ions, but some are created for very specific purposes. The set of symbols is in troduced before a performance, and then the process of int eract ion between the conduct or and the orchestra starts.

"One may ask why an Improvisers Orchestra has composers and/or conduct ors. The two free improvisat ions go to show that they do not act ually need them. However, various people have come up with different ways to shape and direct the music, without using any convent ional musical scores. Thus Dave Tucker's "conduct ion" is an example of the conduct or det ermining who should be playing with what int ensity at any given time. But he in turn is influenced by the feedback of what is act ually played. Steve Beresford and Evan Parker work the same way, except that in both the examples heard here, one musician is free to play in a sort of concert o situat ion. Chris Burn achieves somet hing similar, but uses a predetermined sequence of who should be playing with whom to explore some of the myriad of small combinat ions that exist in such an orchestra. Rhodri Davies invest igates the potent ial quietness of a large ensemble, while Caroline raabel explores the organic processes of such a sensitive body. Simon H Fell's composit ion is perhaps the most cont rolled piece heard here, but even this leaves a considerable amount of freedom. Finally, there are two delightfully subversive at tempts to produce random chance music - the antit hesis of an improvising orchestra - by dividing the musicians into un relating individuals (Adam Bohman) or independent sect ions (Philipp Wachsmann). All of the conduct ors and/or com posers are members of the Orchestra, except for Dave Tucker who has performed with several of the musicians in other cont exts as a guitarist." 3 1. http://elise.at/ ^ 2. http://ima.or.at/ ^ 3. iner notes of Proceedings by the ondon Improvisers Orchestra, http://www.emanemdisc.com/ E4201.html a, b

86

WE TELL STORIES (2008)

Six To Start 1 is an Alt ernate Reality Game and Next Generat ion Narrative company based in ondon, U. They create games, apps, and transmedia experiences for clients in the U and overseas.

Mapping Reading - Navigating a Story Space

We Tell Stories was a collect ion of stories that Six To Start developed and present ed in collaborat ion with the publishing house Penguin weekly for a period in 2008. The narratives were designed to use, in a non-trivial way, the possibilities of cross media narrat ion. The opening story, "21 Steps," based upon The 39 Steps, plays out in a Google Maps environment. The rest of the stories use the online environment in different ways including blogs and Twitter. The last story, "The (former) General in his abyrinth"2 is a network of story elements that can be traversed by the reader as they move around in the General's labyrinth, the story dependent upon the reader's movements. It is described by Six To Start as "a rich and complex story by Mohsin Hamid, uses an ent irely new form of branching storytelling to allow readers to explore the memories of an ageing, and not quite fict ional, general."

The notat ion summarises the possible mot ions of the reader in the General's labyrinth. As the reader moves, cert ain story elements are played out. Passing back, the reader is confront ed by an alt ernate hist ory of the General's life. This path dependency makes the explorat ion of the General's world subject ive in a simple yet non-trivial way. In his blog2, Adrian Hon, who acted as the "Story architect" for the whole project and worked closely with all the people who implement ed the story, said: Before Moshin began writing, I called him up to discuss the capabilities of our story architect ure. I thought it was going to be a quick five minute call, but we ended up talking for about an hour, trying to work out what would be the most int eresting and achievable style of story. Ult imately Mohsin decided not to do a traditional branching narrative, and settled upon doing somet hing else. Maybe a still life. Still life is a term I came up with earlier on, to describe one possibility in which readers could navigate around an essentially frozen world. There would be no branching narrative, but there would be 87

branching paths, and readers would need and want to read all the cells. Imagine if you froze time you could walk around and look through rooms in a building. Collect ively those rooms would tell a single story not a dozen different stories and there would be no end. [...] I was genuinely impressed with the struct ure. To be honest, I think he understand the possibilities bett er than I do, because this is not a struct ure that I would have come up with myself. In a comparatively small number of cells, Mohsin managed to demonstrate three different styles of int eract ive storytelling, and link them together into a single overarching still life. "The (Former) General" is not as visually impressive as some of the other stories, but Im immensely proud of it. The int erface, art design and story all meld together beautifully, and I believe its the most in novative and original piece of storytelling in the six weeks. Its not quite a game, and while it does have branching, it doesnt allow the reader to affect the outcome of story only their own experience of it. It truly is somet hing that you couldnt do in a book, and here, it tells a powerful tale as affecting as any novel.

1. http://www.sixt ostart.com/ 2. Website: http://www.wetellstories.co.uk/stories/week6/ ^ 3. http://mssv.net/2008/04/22/creating-the-former-general/ ^

88

APPENDICES

89

A CLOSING CAKE
We have reached the end of this sprint and the end of this book. This chapter wishes to thank you by sharing the ab stract ion of a delicious cake that isa, our wonderful cook, shared with us.

Versions
This is version 1.0 of our point of view on Notat ion. We have slept litt le and written too much, discussed more and learnt a wide variety of things. This version can stand on its own feet, but it is also growing. Thus we would like to wel come you, the reader, to cont ribute to it. We see several things that would have a place in the next version: Bob Rotenburg's present at ion paper at the Data Ecologies Symposium offers a far more int ricately worked out set of thoughts about the cont exts of notat ion that we have managed. We hope to develop a version that can grow in here as a new chapter. Examples of other works, from a variety of sources, would help round out techniques and ideas. We are a very fin ite group of people with a very specific set of experiences. So a broad range of new examples would help inform the further development of the book Several themes have been moved to the Hold chapter to be dealt with properly. Topics such as treating legal and other social contract syst ems as notational syst ems have been thrown around but we have agreed that they re quire more time and effort then we can give them in this cont ext.

History
This version 1.0 was created between Sept ember 5 and 10, 2012, by Elisabeth Schimana, Andreas Dekrout, Simone Boria, Marta Pierano, Heather elley, Rachel O'Reilly, Tim Boykett and Adam Hyde in a book sprint at the unstraum Goet hestrasse in inz Austria. The sprint was part of the Physical and Alt ernate Reality Narratives (PARN) project with the support of the Cult ure Programme 2007 - 2013 of the European Union, the City of inz, the State of Upper Austria and the Federal BMU. PARN is a project from Time's Up, FoAM, Blast Theory and ighthouse invest igating new forms of storytelling in physical and alt ernate reality spaces. This Book Sprint was the second phase of an invest igat ion that start ed at the Data Ecologies 2012 event with the subt it le "The Map and The Territory." Three of the part icipants, Simone Boria, Elisabeth Schimana and Tim Boykett were pre sent at that event. The document at ion1 of the event, prepared by Emilie Giles and Monique Alvarez, had been made re cently available and was of value, part icularly to those who had not been present. These were Andreas Dekrout, Marta Pierano and Heather elley, who skimmed and scanned the document at ion but did not get too deep within. We had been warned by our Book Sprint facilitator, Adam Hyde, to avoid having too many fixed points in the planned book, that the symposium document at ion might be more of a hindrance than a help. The process of a Book Sprint is getting increasing ly well defined but not well understood outside the mind of Adam, so the last person present, Rachel O'Reilly, was here to document the process and observe the techniques used. We eagerly await her results, so as to bett er understand what was happening to us. Rachel also brought with her a critical eye and fast fingers to help get some of the wonderfully spoken but hard to write parts of the process down into the book. ast but not least, Grenzfurthner/monochrom jumped in with some cover art and we are ready to rock and roll.

Recipe
This Poppy Seed Cake arrived on the Thursday. Unfort unately we cannot pack a slice in a PDF, so we send on the most practical of all notat ions, a recipe. Poppy Seed Cake Ingredients: 250g butter, 250g iced sugar, 6 eggs, 1/2 tsp. salt, 250g grounded poppy seeds, 1/2 tsp. natron, 500g apples, vanilla sugar, lemon zest How to: Beat butter, sugar, yolk until fluffy; mix seeds, zests, salt, natron, sliced apples; mix it with a spoon; whisk egg whites until firm - then fold in; into the oven with 60 for 1 hour; cool it down, then iced sugar on top! 90

Sponsors

Disclaimer
"This project has been funded with support from the European Commission. This publicat ion reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the informat ion con tained therein." 1. Wiki cont aining document at ion: http://wiki.physicalnarration.org/wiki/index.php/DE12TheMapAndTheTerritory
^

91

92

93

94

95

96

97

98

99

100