Visual
Visual
Kevin G. Barnhurst
University of Illinois at Chicago
[email protected]
Abstract
Formal theories reveal a visual dimension underlying better-known picture
theories. Patterns of elements such as lines, shapes, and spaces, along with
their properties, generate emotional responses and follow visual styles in
society. The elements combine into systems that create perspectives on the
world. Formal awareness may generate an understanding of visual
philosophies and their inherent values and consequences. Picture theories
start from the tension between scientific invention and artistic expression.
From linguistics and philosophy, semiotics provides terms for analyzing
pictures as signs that mediate among mind, eye, and reality, operate within
codes, and reproduce mythology. From film and literary aesthetics,
narrative theory offers analytical structures that reproduce realism through
supposed objectivity, rationality, and autonomy in dialogue with
conventions and genres. Critical, cultural, and poststructural theories assert
the inauthenticity of pictures, social construction of representation, and
instability of meaning. Visual aesthetics, analysis, criticism, and ethics have
entered flux in digital times.
1
Barnhurst / IEofComTheory&Phil / 2
Formal theories
Most attention to visual phenomena in communication studies has
focused on images, from photography, television, and film to digital
screens. Another dimension of visual studies deals with the form underlying
pictures, elements such as the lines, shapes, and colors in media with or
without images. To find how form—the arrangement or presentation
aspects of communication—produces meaning, visual studies researchers
assume that viewers employ a theory of elemental form. Formal theory
began with the notion, proposed by British philosophers in the seventeenth
century, that the source of knowledge of the external world is the
experience recorded by sensory organs. This seemingly commonsense idea
encompasses a particular set of propositions about how the mind and the
eye work together, about what raw material the eye finds in the visible
world, and about when the mind encounters beauty or aesthetic experience.
Formal theory proposes that the mind actively interprets what the eye
records. In the middle of the nineteenth century, the lab of German
psychologist Wilhelm Wundt demonstrated that vision is a mental act, not a
simple, instant response to stimuli. In one experiment, they fitted subjects
with distorting lenses, such as prisms, that turned their vision upside-down,
and found that subjects soon adjusted, performing everyday activities
without difficulty. In studies with subjects cured of congenital blindness,
they demonstrated that visual perception occurs in the mind rather than in
the eye alone.
Formal theory also proposes that the eye and the mind record elements,
the fundamental parts and qualities of visual form. In the eighteenth century
the German dramatist and critic G. E. Lessing (1984) defined space as a
fundamental element of painting. A similar approach found its way into
psychology in the late nineteenth century, when Wundt employed the
empirical method of introspection to identify the elements of visual
perception. Psychologists early in the twentieth century then investigated
specific meanings of forms such as vertical and horizontal lines, following
the Gestalt argument that the whole is greater than the sum of the parts,
with elements implicit as the parts that formed Gestalt structures.
Formal theory further proposes that the mind interprets all visuals
similarly, whether they are from the various arts, nature, or dreams.
Aesthetics traditionally was the exclusive province of the arts, assigning
painting the highest rank but setting standards for observing beauty in the
natural world. Early in the twentieth century, critics such as Roger Fry and
Clive Bell insisted on viewing all arts in equal terms. Art historians and
artists such as Wassily Kandinsky used three formal elements—point, line,
and plane—to explain twentieth-century movements that made art out of
everything from unrecognizable shapes to ordinary household objects. An
Barnhurst / IEofComTheory&Phil / 3
turns or follow a train of thought, lines do not exist in nature, where they
may seem to mark boundaries but dissolve upon closer inspection, as
observed in chaos theory. Lines are a simplification to manage the visible
world and an invention to organize experience. The property that makes the
line elemental is extent, its length or measure.
Humans employ shape to recognize and search for objects and can
hardly imagine a seamless world, where discrete objects lack a discernible
shape. The physiology of human vision enhances the contours, or edges of
things, and a “lateral inhibition”—wherever the eye records sudden changes
in the visual field—makes a shape appear distinct. Shape “makes” an
object, giving it an existence as a “thing.” The property that makes shape
elemental is called figure. A shape depends on the line of its border, just as
lines are defined by points, but imagining any of them would be impossible
without a surrounding space. Position is nonsensical without a field, and
figure meaningless without a background. In plane geometry, points and
lines exist on the Cartesian grid, which defines position and length from the
center intersection of two perpendicular axes. Art defines space not from
the center but from the exterior edges or frame.
Humans usually respond to undifferentiated space—the Gansfeld
described in psychology—with visual and motor disorientation
(experienced by covering the eyes with white plastic spoons and staring
toward a light). The idea of space provides comfort by setting boundaries
between nations, neighborhoods, and individuals. Space need not be
rectangular, but the artist’s frame or Cartesian axis completes the network
of visual elements. It also produces a sort of anti-matter universe, so that
every dot is surrounded by a shape with a hole in it. Lines and shapes define
inside and outside, ink-black or screen-white—but the surroundings have
just as much claim to forming points, lines, and shapes. This property,
which makes space elemental, is called ground. Space occupies a special
rank in the list of elements. Its use has been central to controversies over
media designs and practices such as photography and typography, which
depend on an understanding of space.
Point, line, shape, and space appear most frequently on the lists of
elements and seem to form an irreducible core. All four share a definition as
two-dimensional objects, figures on either a negative or a positive ground.
Their properties—position, extent, figure, and ground—are fundamental to
other terms in the many lists of elements cited.
Most authors list several elements that have neither a figural property
nor any fundamental position or extent. Instead of elements, they are
descriptions of the elements, visual attributes that describe elements by
identifying their relationship to other elements or to their surroundings:
direction, scale, tonal value, and color. A position may be high or low, to
Barnhurst / IEofComTheory&Phil / 5
the left or the right, only in relation to other elements or to the frame. That
relation is direction. The cardinal directions—vertical, horizontal, and
diagonal—describe some elements inside the frame, such as a line or
oblong shape but not a single dot or square. Two common analogies—maps
and clocks—describe relations among elements. With the map analogy,
direction relates to the points of the compass. With the clock analogy,
direction relates to the quarter hours, as in the expression, “twelve o’clock
high.” Both methods work well for the traveler in a given position, but in
visual aesthetics they require a designated starting point.
In isolation, no element is either large or small in absolute. The term
size refers to fixed units of measurement. A ton overwhelms a gram, and the
baby is tiny compared to the adult, but in visual images, size is always
relative to the surrounding space and to other objects. A photograph is large
only in proportion to the page and to the typography and other objects
around it. The term scale encompasses that flexibility and suggests a pattern
of relationships. For example, a photograph occupying much of the screen
will appear even larger if something tiny such as a logo sits nearby.
Elements in an image also exist on a continuum ranging from black to its
opposite. The range differs on a screen that glows whiter than paper but not
blacker than ink. The tonal value of an element is not an “absolute” but a
relationship that depends on the surroundings. A mid-gray on a light
background appears darker to the eye than the same gray on a dark ground,
an effect called simultaneous contrast.
Finally, one can describe visual elements by their colors. Like the
super-element space, color is more elaborate than its fellow attributes. It
masks and enhances tonal value, scale, and direction, and it shares the
phenomenon of simultaneous contrast. Color itself is a complex of
fundamental qualities in visual perception and conventional practices
among artists and societies. Physics, psychology, sociology, and the studio
arts contribute to the study of color. Each attribute works across most or all
the elements and properties.
All the elements with their defining properties and attributes (see the
Table One) come under the rubric form (a term sometimes used as a
confusing synonym for shape). The formal terms may underlie any
additional terminology needed for the arts involving live motion, including
theater and dance, and also suggest a likely basis for motion media
produced by the rapid succession of still images. To demonstrate the value
of elements and attributes requires exploring two questions: How does form
convey meanings? And what role does formal theory play in larger systems
of spatial organization?
Barnhurst / IEofComTheory&Phil / 6
Table One
Elements of Visual Form
earlier. The first two generate stasis. Unity governs the grouping of the
elements, either on a central point, along a line, or within a common
silhouette. Unity provides a base of order, and three other principles govern
the employment of formal attributes. Balance assigns the attributes visual
weight, making larger, darker, and higher elements “heavier,” and then
equalizes elements from side to side (in symmetry), around another point or
axis (in asymmetry), or along a spiral (in radiality).
The next two generate visual action. Contrast requires differentiating
attributes (small and large, dark and light, and so forth) to create variety.
Rhythm sets up a pace or sequence by repeating or gradually changing the
attributes and then (sometimes) interrupting the expected flow. Proportion,
the crowning principle, governs the division of space and accomplishes the
other four in either “classical” (that is, obscure) or “Oriental” (apparent)
ratios. “Classical” proportions start from a rectangle with an esoteric ratio
(such as one from the Fibonacci series), but “Oriental” proportions start
from a square (1:1), an immediately clear rectangle. (Although the latter
term employs the sense used to describe the rugs and not the racism Edward
Said condemned, it may still contain a scent of racism.) Like any system of
values, the principles operate in continual tension with each other.
The principles relate closely with Gestalt laws. The laws of grouping
repeat and elaborate the means for achieving the principle of unity, and
their opposite achieves the variety that achieves the principle of contrast.
The Law of Prägnanz restates the principle of balance, expanding it to
include a preference for regular and simple forms. The two theories differ
because one describes perception where the other directs creation of “good”
or attractive images. Gestaltists remain silent on the principles of rhythm
and proportion, which guide the functional processes of design, and instead
explore the viewer’s experiences with learning, teaching, and solving visual
problems, about which design principles fall silent.
In a sense the Gestalt laws and design principles were attacks on
inductive formal theory. Both were modern, envisioning a holistic, top-
down approach to images and the immediate recognition of the beautiful,
effective design or image. Both argued against the so-called dumb method
of manipulating elements by trial and error, insisting on a higher, creative
sense. But both movements also depended on those same elements. Gestalt
psychology described the process of perception and suggested how
elements combine to produce new and greater structures. Principles of
design describe the properties of beauty and explained how particular arrays
of elements produce an aesthetic effect. Both are also functionalist. They
extend rather than replace the atomic or molecular view found in formal
theory.
The two theories do more than provide structure for the elements: they
Barnhurst / IEofComTheory&Phil / 10
ignore or may close off critical avenues of inquiry that formalism would
later embrace. Each is also a creed proposing a theory of “goodness.”
Principles of design in general application have consequences for society,
so that a product or female body, to cite two examples, becomes the center
of design and desire, while the costs of industrial production and for women
remain hidden along with their (male) masters. Because of their similarity,
Gestalt laws would have a kindred effect. As a visual creed, Gestalt laws
favor order, simplicity, and stability, qualities that—however appropriate
for such things as philosophy books, banking websites, and conservative
editorial pages—hew to the elitism of modern thought. They also contradict
the complexity, ferment, and urgency that historically have characterized
realism in literature and, for media studies, in news of everyday life.
Systems of perspective
Other systems of visual organization ingrained in communication are
called perspective. They combine the elements and attributes of design into
larger relationships that serve the purposes of power. One is to establish
relative importance. All perspective systems work to create hierarchies, just
as the Mercator map projection assigns greater importance to northern
countries. To place some powers above others, the systems create an
illusion. Maps color the oceans and lands and add illustrations of mountains
or ships to give the projection a natural feel. Systems of perspective assign
power and privilege but disguise their purposes as natural. The systems
most influential in visual representations are spatial hierarchy, atmospheric
perspective, chiaroscuro, and linear perspective.
Spatial hierarchy imposes a discipline of scale, tonal value, direction,
and sometimes color to assign greater importance to some objects in space.
Egyptian art used scale alone to suggest rank. The ruler appears larger than
any subjects, and in some drawings slaves are smaller still. In works of
Byzantine art, direction (position within the frame) appears to suggest
moral stature. The venerated personage occupies a position higher than
underlings and adorers. The works often combine scale with direction, and
special colors are markers of important persons. In communication,
representations employ spatial hierarchy to show and organize contents.
Important persons and information go on top, in a larger, bolder
typographic setting, or occupy more space.
Atmospheric perspective uses tonal value, direction, and in some cases
scale to create illusory depth on a flat surface. In Egyptian art, one object
overlaps another to imply a receding background. Roman frescoes and vase
paintings use gradations in tonal value to suggest receding space. In
atmospheric perspective, objects resting closer to the viewer are positioned
lower in the frame. Although primarily used for illusion, atmospheric
Barnhurst / IEofComTheory&Phil / 11
viewer at the center of the arrayed world, exalting the individual and
turning the world into a human domain. The media, like other cultural
artifacts, reiterate this philosophical position every day. Placing the self at
the center of attention is widely accepted especially in the West.
Linear perspective has adapted and absorbed the other systems.
Chiaroscuro and atmospheric perspective usually accompany linear
perspective. Spatial hierarchy plays a role because the important and
admirable objects usually appear nearer, that is, larger, darker, and lower in
the frame, when they do not appear at another focal point resulting from the
arrangement of objects. The illusionism of chiaroscuro and atmospheric and
linear perspective disguises and strengthens the hierarchy by hiding its
constructedness behind references to the natural world.
No perspective reproduces precisely what humans see or what is out
there physically, as Erwin Panofsky (1991) observes. Perspective systems
are like precision instruments; their measurements provide sharp, clear
insights, but not without limitations. The careful examination of objects
such as a ball and a shoebox under a light will reveal the limits of
chiaroscuro. If they really do occupy opposite sides of the object, the
highlight and deep shadow would hardly be visible at the same time.
Atmospheric perspective also yields to observation. Any cross-country
traveler can note how often the more-distant hills or mountains seem darker
and larger rather than fainter. Limitations of linear perspective are easy to
demonstrate. On flat terrain one walking down a sidewalk or railroad bed
fixing the eyes on the horizon can observe how the rails (or edges) appear to
converge toward a vanishing point, in classic linear perspective, but the
pavement or rails nearest the eyes appear parallel to each other when one
looks down. Focusing one’s eyes on the middle distance using peripheral
vision can reveal the rails parallel nearby and converging in the distance.
For parallel lines to converge requires that they curve the whole way.
Identifying the limitations demonstrates palpably how any perspective
system clarifies some aspects of human experience while obscuring others.
Besides experience, perspective systems also spring from custom, the
conventions that cultures devise to structure the external world. When
artists draw, they must filter what they see through some system, much as a
camera filters the landscape through a lens. Hierarchy in media images
makes editorial priorities and political leanings immediately clear, but that
strength is also a weakness, persistently pushing some to the bottom of the
screen, to the back of pictures, or out of sight entirely. Readers who
disagree with media priorities may subvert the system by looking
elsewhere, express their discomfort by ignoring mainstream media, or
create alternative media. One cannot neatly prioritize the world without
removing conflicting points of view, but an absence of hierarchy presents
Barnhurst / IEofComTheory&Phil / 14
Picture theories
Visual theory has tended to focus not on form but on the picture. Where
form seems merely to contain content, pictures seem to be the content, the
substance of visual experience. In communication, visual theory drew
originally on European art history to explore the relationships between
image and reality (Summers, 1989). In media studies, the focus further
narrowed to the study of lens-based media, although expansive in ranging
across film/cinema studies, perceptual psychology, and visual anthropology
and sociology (Griffin, 2008). Technics of mediation have been the main
current in visual communication studies and the basis for institutional and
organizational connections.
Theory in media studies begins with photography in the nineteenth
century, and its main tensions arise between science and art. Unlike earlier
plastic arts, photography is a product of science relying directly on an
implement that records traces of the physical world. The camera seems
capable of documentation, making pictures that exist apart from their
makers and feeding out meanings in ways not unlike direct experience. Yet
mechanization could not exist without the human hand in designing if not in
operating the camera directly, making camera arts also a means of
expression. Choices taken in creating the machine and then employing it to
make pictures do more than merely “write” the world from artificial light or
sunlight. Cameras “see” the world through social processes that transform
the resulting image no less than other arts do (Crary, 1990). Because
photography is a creative process and a tool that draws in reflections of
material objects, its products require close analysis.
Semiotics
A central theory for analyzing pictures whether moving or still has been
semiotics, an approach from philosophy. John Locke coined the term
semiotic to refer to the doctrine of signs. A sign is any utterance or sound,
gesture or image, deemed meaningful. Semiotic terms have immediate use
to communicators and audiences for analysis and also critique.
The Swiss linguist Ferdinand de Saussure proposed that signs are a
fusion of a signifier (such as a word in language) with a signified (its
corresponding concept). Although Saussure refers primarily to linguistic
semiology, breaking apart the internal structure of signs has value for visual
studies. The signifier is a phenomenon outside the mind, such as a picture,
Barnhurst / IEofComTheory&Phil / 16
and the signified is inside the mind, the particular sense or meaning one
assigns to the signifier. Much confusion in interpretation is merely the
mistaken fusing of the signifier with the signified, the assumption that one’s
mental idea is in fact the same as whatever in the external world prompted
one to think about it. From that fusion, an easy step is to assume that others’
minds hold the same signified in response to the signifier, the root of
cultural misunderstanding.
The signifier and signified together form the sign, a kind of conceptual
triangle that repeats throughout the theory. The sign has a similar, complex
relationship with the referent, whatever in the world of things or ideas the
sign refers to. Confusion of the sign with its referent is an even greater
source of perplexity and strife in human affairs. In photography the picture
(the sign) is not the object pictured (the concrete referent), just as in
typography the shape or design of a particular letter (the sign) is separate
from the letter (the abstract referent). Saying so may seem obvious, but
much of the fuzzy thinking about objectivity in modern science results from
mistaking one’s picture for nature or the visible story for a natural process.
The Belgian surrealist Rene Magritte illustrated the mistake when his
painting The Treachery of Images showed a brown pipe with the legend,
“Ceci n’est pas une pipe” (“This is not a pipe”).
The three-in-one quality of semiotics is most present in the work of the
American philosopher Charles Sanders Peirce, who proposed three general
classes of signs. A first class of signs is symbolic, individual symbols or
groupings such as the alphabet, which society has agreed upon but which
have no necessary connection to the thing they represent. The letter a, for
example, is an arbitrary shape. Its history through Phoenician and Greek
lost the ties to its origin in the Egyptian ox-head hieroglyph. It became a
free-floating form, useful but independent of objects seen. Even its
relationship to sound is arbitrary, especially in English where it stands for
seven phonemes. Its symbolism can also inspire emotions, as in the
symbolic significance of capital A as a mark of virtue in school or of
adultery in Nathaniel Hawthorne’s The Scarlet Letter.
A second class includes icons, which in some ways resemble the things
they represent. A photograph of a trophy, for example, coincides with
almost every detail from the trophy itself so that ignoring their differences
is easy. Illustrations and cartoons may resemble seen objects only slightly
but are icons because of the similarity between sign and referent. Like
symbols, icons may seem neutral, such as the pictograms used to guide
users of digital interfaces or airports, or they may seem charged, like the
icons found in religious spaces. But all icons express power, as the focus of
visual studies on iconic pictures underlines.
Indexes comprise a third class of signs, which represent things
Barnhurst / IEofComTheory&Phil / 17
Narrative
Visual theory for pictures also emerged out of aesthetics, principally
from film but also literary studies. Where still pictures could draw on long
traditions of “proper” practice in the visual arts, cinema seemed to shift the
ground underlying earlier theater performances and religious rituals.
Thinkers such as Hugo Münsterberg and Rudolf Arnheim developed film
Barnhurst / IEofComTheory&Phil / 19
Author Bio
Kevin G. Barnhurst is Professor Emeritus of Communication at the
University of Illinois at Chicago, USA, and also retired professor and Chair
of Communication in the Digital Era in the School of Media and
Communication, University of Leeds, UK. His research areas include
critical issues in visual media, digital culture, media history, political
communication, and audience sociology. He is the editor of Media
Queered: Visibility and its Discontents (New York: Peter Lang) and other
works on visual history and theory.