Erik Conrad Thesis Committee

Master’s Project Design Document Sha Xin Wei

Information Design and Technology Ken Knoespel
04.17.02 Stephanie Stricklan
 æther is an experiment in how new technologies of representation affect

human perception. It acts as a haptic surface for computer mediated visual

information to enable tangible experiences common to painting and sculpture,

but rare in digital media. The painter sees with tactile vision and translates that

experience into a physical movement, after which it exists as a three-dimensional

surface, becoming a union sensory of experience. By physically navigating a

textured visual space, æther allows for a physical experience of textual

information analogous to the visual experience of white-space in a poem. By

immersing the experience of reading into a haptic visual space, different ways of

seeing are allowed to dynamically combine and separate, giving rise to new

forms of visual expression and reception.

The philosopher Marx Wartofsky has argued for a radically culturalist

reading of all visual experience. He concludes that “human vision is in itself an

artifact, produced by other artifacts, namely pictures.” All perception, he

contends, is the result of historical changes in representation. (Jay, 5) The visual

experience offered by the computer is based on older methodologies (analytic

perspective, photography) that have been tacitly learned and internalized with

repeated exposure. As Western culture embraces the representational

conventions of digital media, one must account for the virtues of traditional

media, lest they be lost.

The vision of the painter is described by the words of Cézanne: “Art is a

personal apperception, which I embody in sensations and which I ask the

understanding to organize into a painting.” (Merleau-Ponty, 238) For Cézanne,

2
to paint was to have "... the need to no longer see ... to be too close ... to lose

oneself without landmarks in smooth space." (Deleuze, 493) In order to do this,

he had to first forget all that he had learned about vision both from science and

through science. Visual experience is composed of more than mere

perception–cognition plays an integral part. Visually communicating experience

requires conscious control over the mixing of what one sees with what one

knows. In order to recapture the scene, “all the partial views one catches sight of

must be welded together; all that the eye’s versatility disperses must be united;”

one must, “join the wandering hands of nature.” (Merleau-Ponty, 242)

Cézanne wanted to paint the experience of a scene, which is more than

placing a frame around the light that it reflects. Cézanne’s reality was not

photographic reality. Today, the transparency of the photograph’s influence on

visual perception can be alarming. Susan Sontag warns of a “mentality which

looks at the world as a set of potential photographs” and argues that “reality has

come to seem more and more what we are shown by camera.” (Pallasmaa, 20)

This mentality can be seen in action in Ralph Weber’s, On the Aesthetics of

Architecture. In it, he states that people tend to overestimate the height of

vertical objects by an average of 30 percent. Weber then explains that this

phenomena is why mountains depicted in paintings are “super-elevated

compared to their actual condition.” To illustrate this, he selects one of

Cézanne’s landscapes of Mount Saint Victoire and pits it against a photograph of

the same mountain. (Weber, 134)

3
 This example clearly demonstrates how technologies of representation

can alter the way that people believe that they see. In this case, the

representation interpreted by a mechanistic visual system is believed to be ‘more

real’ than the human interpretation of the experience. When the vision of cultural

producers becomes mechanically influenced, those visual systems are then

reified through their work. If we value actual embodied vision, then it is the

responsibility of cultural producers to acknowledge the difference between

artificial visual systems and embodied vision, and design accordingly.

Photographic vision and its biases, are an extension of the visual

paradigm created by Albertian perspective. Alberti’s statement “Painting is

nothing but the intersection of the visual pyramid following a given distance, a

fixed center and certain lighting” (Pallasmaa, 17) implies an equality between the

image in perspective and visual experience that survives to this day.

“The convention of perspective, which is unique to European art

and which was first established in the early Renaissance, centers
everything on the eye of the beholder. It is like a beam from a
lighthouse–only instead of light traveling outward, appearances
travel in. The conventions called those appearances reality.
Perspective makes the single eye the center of the visible world.”
(Berger, 111)

Perspectival vision does not equate to the experience of two active, stereoscopic

eyes of embodied vision, it is instead vision from a monocular, fixed abstract

point. The danger with perspectival vision is that it claims to represent reality.

However, the reality that it represents is that of the disembodied viewer

“separated from the seen (the scene) by Alberti’s shatterproof window.” (Jay, 55)

4
 The invention of artificial perspective serves as an example of how visual

thinking has changed. At its conception, it was not universally accepted as an

accurate depiction of reality. To many of those schooled in vision of the

fourteenth century, its acceptance came only gradually, and sometimes with

misgivings. Brunelleschi himself is said to have even felt the contradiction

between the system and the evidence of his eyes. (White, 206) Leonardo da

Vinci invented a method of perspective that entailed foreshortening not only into

the picture plane, but horizontally and vertically across it as well. Leonardo’s

synthetic perspective was concerned with the transference of the subjective

appearances of the real world, both physical and psychological, onto a flat

surface. His method was based on curves rather than straight lines. Instead of

Alberti’s notion of a plane that intersects the pyramid of vision, Leonardo favored

that of a sphere, concave to the eye, intersecting the visual cone. (White, 207-

208)

Leonardo’s synthetic perspective from The Birth and Rebirth of Pictorial Space by John White

5
Interestingly, despite the obscurity of Leonardo’s synthetic perspective, it seems

much closer to biological perception than does that of Alberti. Gibson notes:

… if we could combine all these two-dimensional projections of

a three dimensional visual world into a single scene, we would
obtain a two-dimensional space, in the geometrical sense, which is
non-Euclidian.
It would have the properties of the theoretical space defined by
the surface of a sphere considered as a two-dimensional surface,
i.e. it would be boundless and yet finite, and it would return upon
itself. The space composes of one’s combined visual fields may be
said to be a curved space in the sense that a point which traces a
straight line will eventually come back to the position from which it
started instead of traveling off endlessly in the same direction.
(Gibson, 122)

The directions of deformations in the visual field during forward locomotion, as

projected on a spherical surface around the head (from Gibson)

Visual experience in Western culture is not conditioned by images alone.

Similar to some of the visual methodologies mentioned above, writing takes that

which exists spatially and temporally, in this case the spoken word, and

translates it to two-dimensional optical space. Language and writing are

6
commonly equated with thought, and learning to read is often the only visual

training that one undergoes in their lifetime. Therefore, if one wishes to fully

understand visual culture’s influence on visual experience, one must consider

technologies of writing as well.

“Technologies are not mere exterior aids, but also interior transformations

of consciousness, and never more than when they affect the word.” (Ong) We

are at a time when the technologies of writing and representation are changing,

and with it, the way in which human beings think. As new technologies

increasingly influence the way that people interact with written word, it is

important to consider the effects that their underlying ideologies impose on

language. Already, Western culture’s bias towards written language has

devalued, as unknowing, sensory forms of communication. The computer only

reinforces this ideology.

The current state of the interface to the digitized word is greatly

impoverished. ‘Point and click’ essentially amounts to poking one’s way through

a document. Imagine a conversation working this way! In this interaction, the

body is reduced to less than an eye and a finger. Although most information is

presented visually, the computer does not even take advantage of the full

capabilities of vision. Focus is directed at a flat two- dimensional surface,

ignoring bifocal depth perception as well as peripheral vision. “Learning to see is

the basis for all of the arts except music,” writes Flannery O’Connor, “I know a

good many fiction writers who paint, not because they’re any good at painting,

but because it helps their writing. It forces them to look at things.” The vision

7
described here by Flannery O’Connor cannot be accomplished through the

screen. It is a vision that is analogous to what Delueze and Guattari describe as

“close-range” vision. “Close-range” vision takes place in “tactile” or “haptic”

space, rather than optical space. Haptic space does not establish an opposition

between two sense organs but rather invites the assumption that the eye itself

may fulfill this nonoptical function.” (Deleuze, 492-493) An example of close

vision can be seen in the painter, who is able to assign tactile values to visual

perceptions. (Gombrich, 16) It is an immersive, embodied experience that

engages the peripheral and the proprioceptive, and can be achieved by pushing

that which is believed to be intangible (vision, thought) into a sensory experience.

With the screen serving as a purely optical space, and physical interaction limited

to the fingertips, the computer neither fosters nor evokes this way of seeing that

is so important to creative expression.

The influence of the computer’s predisposition to optical space is evident

in the World Wide Web. Despite the fact that the web has popularized the notion

of navigating through information spaces, the design of most digital texts still

clings to legacy media in form of the page, which in a digital environment comes

even closer to its printed cousin in achieving two-dimensionality. Navigation

implies a third dimension, yet digital texts rarely use multidimensional conceptual

forms as models for their design. Instead, information design and ‘architecture’

operates almost exclusively in abstract terms, ignoring the potential aspects of

embodied navigation in space. Limited by the screen, they do not take

8
advantage of the construction of the spaces created by navigation, limiting

themselves to two dimensions, once again ignoring other ways of knowing.

The computer embodies a way of thought based on the separation of mind

from body and thus, thought from experience. Although most today would

discount Cartesian duality, few are able to escape its grasp. The mind/body split

is so fully installed in our language, that it is difficult to even discuss this issue

without resorting to concatenated neologisms such as the “mind/body,” “lived

body,” “think/know,” or “being in the world.” Current arguments for an embodied

mind fall short because they are often launched from traditions that are

predicated on dualism and privilege the abstract and transcendent over the

embodied and concrete. (Penny, 34) Disregarding body knowledge is antithetical

to human experience, and is dangerous when it becomes the primary way of

thinking. The computer, as both tool and medium, imposes and propagates this

ideology.

Since body knowledge exists outside of the realm of words, then there is a

need for a communicative medium that extends beyond words; a medium that

exists between the immaterial world of abstract thought and the material world of

physical experience. æther suggests a change in the nature of people’s

interactions with digital technologies from an abstract, disembodied activity to an

embodied experience by allowing interaction with the intangible through tactile

vision in haptic space.

9
Functional Specifications/Description

Materials/Specifications:

• 6’ (width) x 6’ (height) x 4’ (deep)

• frame constructed out of 58 ft. of 1/2” copper pipe
• gauze fabric for projection surface
• data projector, mounted on ceiling @ 10 ft, and 45˚angle
• overhead track lights (2) - 30W bulbs
• photocells (32)
• BASIC Stamp BS2p 40 Module
• PowerMac G4 (933 MHz, 512 MB RAM, Mac OS 9.2) graphics
• Powerbook G3 (333 MHz 384 MB RAM, Mac OS 8.6) input + sound
• QuickTime 5 pro
• Cycling ’74 Max4/MSP2 + third party externals
o 0f0003.MASCHIN3NKUNST nato.0+55+3d modular
http://www.eusocial.com/
o Peter Elsea’s Lobjects
http://arts.ucsc.edu/ems/music/research/Lobjects.readme.html
o PeRColate v.0.9b3 Max 4 – [munge~] & [242.imgmatrix] objects
http://music.columbia.edu/PeRColate/
o OpenSoundControl (Berkeley)
http://cnmat.cnmat.berkeley.edu/OSC/
o Akaobjects’ [speech] object
http://www.iamas.ac.jp/~aka/tme/gallery/
• 60 ft. of 4 twisted pair network cable
• Mac serial cable (9-pin)
• Keyspan USB-to-Serial Adapter
• Crossover cable
• 1/8” minijack to 1/8” minijack audio cable

The physical interface is a table-like structure with a curved translucent

fabric surface that begins parallel to the tabletop closest to the reader and ends

parallel to the vertical surface at the rear. The frame is constructed of half-inch

copper pipe, six feet high, six feet wide and four feet deep. Layers of fabric

provide multiple surfaces for the text projected from above through. Beneath the

fabric lie a grid of photocells which act as the source of input, measuring the

intensity of the shadows cast by the reader’s gestures from the light mounted

10
above. The photocells are arranged in rows of ten, ten, eight, and four. They are

wired to a BASIC Stamp BS2p 40 Module that rests underneath and

communicates the output of the resistor-capacitor circuits via RS-232 to Max

running on a PowerBook G3.

The 32 individual photocells are connected to the BASIC stamp by way of

resistor/capacitor circuits. The resistance of the photocells varies according to

the amount of light to which they are exposed. Thus by measuring the amount of

time that has elapsed since the time the pin on the stamp goes ‘high’ until the

time that the capacitor discharges, one can tell the resistance of the photocell

and thus the intensity of the light to which it has been exposed. The stamp reads

these values as 16-bit numbers, which due to its memory constraints must be

read and sent in four groups of eight values in order to transmit data from all

thirty-two inputs. The data is sent from the stamp via serial communication to a

Macintosh PowerBook G3 where the data is read by an abstraction (or patch)

written in the Max visual programming environment.

When Max gets the data several things happen. First, data from each

sensor is smoothed with a moving average to cancel noise. Second, the amount

of change (delta) since the last sample was taken is measured and summed with

the values from all thirty-two sensors. This value is used to control the system’s

audio response. The sensor data is also grouped into one of five areas,

corresponding to both physical and narrative locations. Here, the data from each

sensor is weighted, within its group, in order to give shape to the different areas.

For example, in a square of nine sensors the center might be the most heavily

11
weighted, and most influential to the overall regions value, while the outer

sensors may carry less influence.

The sums of the weighted values for each of the five areas are then

transmitted to the graphics machine. The relative values of the various regions

determines the strength of the visual feedback, i.e. if an area, in general, has a

higher value than the others it will be the most visible. The visual output for each

of the five groups is drawn separately with variables determining the start and

end points of the text selection, font, font-size, color, rag, linespacing, and x,y

position. The five images are then collaged together with variable opacity and z-

position (layering).

12
Sketch (process)

Before construction began, I made a three dimensional conceptual sketch.

I divided the ‘Daughters of Minyas’ section of Book IV of the Metamorphoses into

roughly five sections. Each section represented an individual narrative thread,

with the exception that two of the sections (beginning and end) were part of the

frame story. Binding elements were also given their own thread. Each thread

was cut at points where the person or the verb tense changed, devices Ovid

used to draw in the reader/listener and advance the action. Each patch of each

thread was woven together with clear filament and suspended from the ceiling.

The threads were allowed to spin and overlap. The purpose of this sketch was to

get a physical sense of some of the structural qualities of the selection.

13
Sketch (process)

14
Max Code

Main patch for receiving, sorting, and smoothing serial data from the BASIC stamp

Subpatch to sort out data for individual sensors from lists generated by the STAMP

15
Subpatch that converts a series of five ASCII characters into one floating point number

Patch that monitors raw sensor data, also shown (not implemented) subpatches that calculate the
sum and delta of all sensor data

16
Sensor map patch where data is gathered and weighted in five regions and sent to graphics
machine via OpenSoundControl (OSC) This is also where the data is initialized, or ‘zeroed out’
so that all changes are relative to the initial condition.

17
The initialization patch sets background color, resolution, reads text file, sets initial text line
numbers, sets font, rag, and linespacing

The text control patch sets text position, size, color, movement and drawing

18
The collage text patch receives the images from the text control patch, controls the layering and
opacity based on sensor values and outputs the final image to the ekran patch

The ekran patch displays the final output and hides the mouse and menubar

19
Screenshot, font–Arial

Screenshot 2, font–Arial

20
Screenshot, font–Times

Screenshot 2, font–Times

21
(not implemented) patch that manipulates sound of text ‘spoken’ by Macintosh Speech Manager
to reduce voice to a texture

22
Physical interface

Plexi-glass surface of frame

Sensor layout

23
Breadboard and stamp (side)

Breadboard and stamp (top)

24
Frame without fabric

Frame with fabric surfaces

25
Physical interface (performance)

Physical interface (performance)

26
Bibliography

Berger, John. Ways of Seeing. London: British Broadcasting Corporation;

Harmondsworth, Penguin, 1972.
Blakney, R.B. The Way of Life - Lao Tzu. New York: Penguin Books, 1983.
Bolter, Jay David. Writing Space The Computer, Hypertext, and the History
of Writing. Hillsdale, New Jersey: Lawrence Erlbaum Associates,
Publishers, 1991.
Calvino, Italo. Six Memos for the Next Millennium. Trans. Patrick Creah
New York: Vintage Books, 1993
Calvino, Italo. Why Read the Classics? Trans. Jonathan Cape. New York:
Vintage Books, 2000.
Campbell, Jeremy. Grammatical Man: Information, Entropy, Language, and Life.
New York: Simon and Schuster, 1982.
Capra, Fritjof. The Tao of Physics. Boston: Shambala, 1991.
Carrol, Lewis. Alice's Adventures in Wonderland. New York: Puffin, 1992.
Damasio, Antonio R. Descartes’ Error : Emotion, Reason, and the Human Brain.
New York: Avon Books, 1995.
de Certeau, Michel. The Practice of Everyday Life. Trans. Steven Rendell.
Berkeley, California: University of California Press, 1984.
Deleuze, Gilles. The Fold : Leibniz and the Baroque. Trans. Tom Conley.
Minneapolis, Minnesota: University of Minnesota Press, 1992.
Deleuze, Gilles. The Logic of Sense. Trans. Mark LesterNew York: Columbia
University Press, 1990.
Deleuze, Gilles, and Guattari, Felix. A Thousand Plateaus: Capitalism and
Schizophrenia. Trans. Brian Massumi. Minneapolis, Minnesota:
University of Minnesota Press, 1988.
Gibson, J.J. The Perception of the Visual World. Boston: Houghton Mifflin, 1950.
Gleick, James. Chaos Making of a New Science. New York: Penguin Books,
1987.
Gombrich, E.H. Art and Illusion. Princeton, New Jersey: Princeton University
Press, 1960.
Goodman, Nelson. Languages of Art: An Approach to a Theory of Symbols.
Indianapolis, Indiana: Hackett Publishing, Inc., 1976.
Goodman, Nelson. Ways of Worldmaking. Indianapolis, Indiana: Hackett
Publishing, Inc., 1978.
Goranzon, Bo. (Ed.) Skill, Technology and Enlightenment: on Practical
Philosophy. London: Springer–Verlag, 1995.
Harris, Roy. The Origin of Writing. LaSalle, Illinois: Open Court, 1986.
Hofstadter, Douglas R. Gödel, Escher, Bach: an Eternal Golden Braid. New York:
Basic Books, 1999.
Holstein, Alexander. Pointing at the Moon. Rutland, Vermont: Charles E. Tuttle
Company, Inc., 1993.
Huggett, Nick, ed. Space from Zeno to Einstein. Cambridge, Massachussetts:
The MIT Press, 1999.

27
Jay, Martin. Downcast Eyes: the Denigration of Vision in Twentieth–Century
French Thought. Berkeley, California: University of California Press,1993.
Lakoff, George, and Mark Johnson. Philosophy in the Flesh: The Embodied
Mind and Its Challenge to Western Thought. New York: Basic Books,
1999.
Landow, George P. Hypertext 2.0. Baltimore: The Johns Hopkins University
Press, 1997.
Maturana, Humberto R. and Francisco J. Varela. The Tree of Knowledge : The
Biological Roots of Human Understanding. Boston: Shambhala, 1998.
Mereau-Ponty, Maurice. Alden Fisher ed. The Essential Writings of Merleau-
Ponty. New York: Harcourt, Brace & World, 1969
McLuhan, Herbert Marshall. The Gutenberg Galaxy: The Making of Typographic
Man. Toronto: University of Toronto Press, 1962.
Meggs, Philip B. A History of Graphic Design. New York: John Wiley and Sons,
Inc., 1983.
Merton, Thomas. The Way of Chuang Tzu. New York: New Directions, 1969.
Murray, Janet H., Hamlet on the Holodeck : The Future of the Narrative in
Cyberspace. Cambridge, Massachussetts: MIT Press, 1997.
Ong, Walter J. Orality and Literacy. London: Routledge, 1982.
Ovid. The Metamorphoses. trans. Allen Mandelbaum. New York: Harvest
Books, 1995.
Pallasmaa, J. The Eyes of the Skin: Architecture and the Senses. Lanham, MD:
National Book Network Academy Editions, 1996.
Parmenter, Ross. The Awakened Eye. Middletown, Connecticut: Wesleyan
University Press, 1968.
Peitgen, Heinz-Otto, Hartmut Jürgens, and Dietmar Saupe. Chaos and Fractals.
New York: Springer-Verlag, 1992.
Penny, Simon. “The Virtualisation of Art Practice: Body Knowledge and the
Engineering World View.” CAA Art Journal Fall (1997): 30-38.
Porter, Tom. The Architect's Eye: Visualization and Depiction of Space in
Architecture. New York: E. & F. N. Spon., 1997
Sacks, Oliver W. The Man Who Mistook His Wife for a Hat. New York: Simon
and Schuster, 1998.
Shlain, Leonard. Art and Physics: Parallel Visions in Space Time and Light. New
York: Quill William Morrow, 1991.
Schwenk, Theodor. Sensitive Chaos: The Creation of Flowing Forms in Water
and Air. London: Rudolf Steiner Press, 1965.
Spencer-Brown, G. Laws of Form. Ashland, Ohio: Bookmasters, 1994.
Stafford, Barbara Maria. "Presuming images and consuming words: the
visualization of knowledge from the Enlightenment to post-modernism."
Consumption and the World of Goods. Ed. John Brewer and Roy Porter.
London: Routledge, 1993. 462-477.
Varela, Francisco and Evan J. Thompson. The Embodied Mind.. Cambridge,
Massachusettes: MIT Press, 1992.
Virilio, Paul. Open Sky. London: Verso, 1997.

28
Weber, R. On the Aesthetics of Architecture: A Psychological Approach to the
Structure and the Order of Perceived Architectural Space. Aldershot:
Brookfield, 1995.
White, John. The Birth and Rebirth of Pictorial Space. London: Faber and
Faber, 1967.
Wittgenstein, Ludwig. (trans.) Philosophical Investigations : the English Text of
the Third Edition. Trans. G. E. M. Anscombe, New York, New York:
Pretence Hall, 1999.

29