See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/228584191

User Interface, Navigation Design and Content Representation: Three

Perspectives on World Wide Web Navigation

Article · January 2010

CITATIONS READS

3 9,070

1 author:

Stefan Leuthold
University of Basel
2 PUBLICATIONS 3 CITATIONS

SEE PROFILE

All content following this page was uploaded by Stefan Leuthold on 11 December 2014.

The user has requested enhancement of the downloaded file.

 User Interface, Navigation Design and Content
Representation: Three Perspectives on World Wide Web
Navigation

Stefan Leuthold
Department of Psychology
University of Basel
Missionsstrasse 62a
4055 Basel
Switzerland
[email protected]

Doctoral Thesis, submitted to the

Department of Psychology, University of Basel, Switzerland

Doctoral committee:
Prof. Dr. Alexander Grob (Chairman)
Prof. Dr. Klaus Opwis (First Reviewer)
Prof. Dr. Lothar Müller (Second Reviewer)
Prof. Dr. Andreas Monsch (Neutral Member)
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 2

Contents

Summary 3

Introduction 5
Emergence of Human-Computer Interaction from the Cognitive Revolution . . . 5
Human-Computer Interaction as Improving Human Information Processing . . . 8
Navigation as a Topic within Human-Computer Interaction . . . . . . . . . . . . 11

Three Approaches to Navigation on the World Wide Web 13

The Theoretical Approach: Solving the Navigation Problem . . . . . . . . . . . . 13
The Empirical Research Approach: Pieces of a Neglected Puzzle . . . . . . . . . 14
The Economic Approach: Consumers and Information Workers of the 21st Century 15
Organization and Commonalities of the Three Manuscripts in the Appendix . . . 16

Summary of Manuscripts 17
Manuscript Summary 1: Beyond Web Content Accessibility Guidelines: Design of
Enhanced Text User Interfaces for Blind Internet Users . . . . . . . . . . . 17
Manuscript Summary 2: Influence of Task Complexity and Navigation Design
on User Behavior on the World Wide Web: Eye-Tracking Study Measuring
Navigation Performance and Strategy . . . . . . . . . . . . . . . . . . . . . 19
Manuscript Summary 3: Error Prevention in Online Forms: Use Color Instead of
Asterisks to Mark Required-Fields . . . . . . . . . . . . . . . . . . . . . . . 21

Conclusions 23
Questioning the Value and Cost of Guideline Application . . . . . . . . . . . . . 23
Methodological Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Better Models of Navigation Behavior . . . . . . . . . . . . . . . . . . . . . . . . 25

References 27

Acknowledgements 31

Appendix 32
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 3

Summary

Human-computer interaction (HCI) came into existence in the 1960s as a specialized

continuation of developments which spurred the cognitive revolution and led to the
emergence of cognitive science. Its elementary topic is human information processing,
enabling human beings to work with computers as efficiently and flawlessly as possible.

This manuscript-based doctoral thesis is concerned with a topic within HCI: Naviga-
tion on the World Wide Web. As with navigation in the real world, users must be able to
determine their position, chart their course, and find out how far they have already come.
In order to improve online navigation in this regard, certain navigation mechanisms and
corresponding user interface elements were adjusted for human information processing. In
an attempt to exhaust cognitive and motor capabilities in order to help computer users
execute their navigation tasks better, user behavior was analyzed for navigation tasks in
the three stages of 1) input perception, 2) integration of input with respect to user goals,
and 3) articulation of an appropriate response in the user interface.
Three manuscripts that challenge existing navigation standards or guidelines are pre-
sented. The first manuscript (Leuthold, Bargas-Avila, & Opwis, 2008) considers navigation
tasks from the user interface perspective: For users to be able to navigate, they must be able
to interact efficiently with the user interface elements. However, the standard for accessibil-
ity of web pages is not enough for users who are blind – user interfaces are not usable, just
because they are accessible: For a study with 39 blind participants, an enhanced text user
interface (ETI) was constructed and was shown to be an improvement for blind internet
users regarding navigation of a search form. They needed less time, made fewer errors,
and were more satisfied with the ETI. No improvements could be measured for a browse
task, which, in this specific context, was interpreted to indicate that users had problems
understanding the navigation labels. If users do not understand navigation labels, it is hard
to improve usability by adjusting user interface elements.
The second manuscript (Leuthold, Schmutz, Bargas-Avila, Tuch, & Opwis) stud-
ies navigation tasks from the navigation design perspective, researching conflicting guide-
lines as to whether large numbers of navigation items should be presented in menus or in
groups. The study, with 120 participants, shows that, first, navigation mechanisms should
be extended with different kinds of navigation items adapted to the complexity of users’
navigation tasks, and secondly, that displaying many grouped navigation items at once is
better fitted to perception and cognition than dynamic menu navigation: Users needed less
time and fewer eye fixations for their first clicks and were more successful with grouped
navigation items than with menus.
The third manuscript (Pauwels, Hübscher, Leuthold, Bargas-Avila, & Opwis, 2009)
analyzed a form fill-in task and researched navigation from the content representation
perspective. Studying 24 participants’ navigation between input fields on a web form, it
was found that users made fewer errors, were faster and more satisfied with colored required
fields, compared to the de-facto standard for indicating required fields, the asterisk (*).
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 4

The manuscripts in this thesis illustrate why guidelines must be validated empirically.
They demonstrate the value of task analysis to study user behavior and improve navigation,
they illustrate how user-centered design helps create user interfaces that outweigh existing
guidelines and standards, and they show how qualitatively gained user interface solutions
benefit from quantitative examination.

The original manuscripts are included in the appendix.

 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 5

Introduction

Emergence of Human-Computer Interaction from the Cognitive Revolution

Human-computer interaction (HCI) today is studied by a wide variety of scholars:

Most of the more than 130 contributors to Sears and Jacko’s (2008) recent handbook of HCI
are university researchers, specializing in psychology, computer sciences, information tech-
nology, industrial engineering, and other more specialized departments such as ergonomics
or user interface design. A minority of authors works at software companies such as IBM
or Microsoft, or user research and user interface design companies. This section describes
the origins of this wide variety, starting from the historical developments that in the 1950s
led researchers from diverse faculties to the conclusion that the human mind should and
could be studied, thereby laying the foundation for the field of cognitive science. Funding
for computer sciences then led to the emergence of HCI in the 1960s. Thus, HCI evolved
as a branch of applied cognitive science.

The Cognitive Revolution. From the 1950s to the 1970s, the intellectual movement of
the cognitive revolution reclaimed psychology’s position as the science of the mind, after be-
haviorists had claimed for about 30 years that psychology should not concern itself with the
mind, but only with directly observable behavior (Watson, 1924). Behaviorists explained
human behavior as a series of stimulus-response reactions, denying the importance or even
the existence of concepts like “consciousness” (Watson, 1924), because the method of in-
trospection, which was the dominant method used in the beginnings of psychology (James,
1890), failed to gain verifiable insights into the workings of the human mind, but relied on
many mind-related concepts to explain psychology’s central theories.
Gardner (1985) locates the first major attack on established behaviorist thinking in
1948, at the California Institute of Technology, where talks by leading scientists of the
time covered almost all developments that contributed to the birth of cognitive science.
John von Neumann compared the computer to the brain. Warren McCulloch believed one
should exploit parallels between the functioning of the nervous system and logical devices
in order to understand human perception. Karl Lashley claimed that behavior such as
playing a rapid piece on the piano could not be explained by behaviorist theory, because
there is no time for feedback to become stimuli to trigger the necessary responses that
– according to behaviorists – constitute human behavior. He postulated the existence of
central brain processes that controlled the kind of behavior which could obviously not stem
from stimulus-response-reactions.
Five mostly unrelated developments led to the birth of cognitive science, according
to Gardner (1985). The first development stemmed from mathematics and computation:
Based on previous work by logicians, Alan Turing showed in 1936 that if a calculation could
be described precisely enough in terms of problem-solving, then his Turing machine could
be programmed to generate the result. This led other scientists to believe that if human
behavior or thought processes could be described precisely enough after the same fashion,
it should be possible to design a computer that operates like the human brain. The second
development was the neuronal model: Warren McCulloch and Walter Pitts showed in the
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 6

early 1940s that a nerve cell and its connections with other nerve cells could be modeled
in terms of logic. Norbert Wiener was responsible for the third development, which he
called “cybernetics”. His wartime investigations of servomechanisms to control anti-aircraft
artillery or airplanes led him to believe, in the early 1940s, that mechanical machines and
human behavior had much in common in terms of feedback and self-regulation: They could
both be said to show goal-directed behavior if they determined the differences between
their actual state and their desired state and then integrated this feedback in order to work
towards reducing the differences. Anderson (2007) adds that wartime research in human
performance also led to the first application of information processing theory to human
beings, for example examining how much information could be processed per time unit.
The fourth development goes back to Claude Shannon’s description of the two states of
electromechanical relay switches (on/off) in the principles of logic (true/false) in the late
1930s. This served as the existence proof of a representation of information that is entirely
independent of its specific content (and was the invention of today’s digital representation
of information in binary digits – or “bits”). The final development was the study of brain
damage in wartime victims, especially language deficits or difficulties in recognition.
If human beings showed goal-directed behavior like machines (Wiener), and if the
corresponding operations of thought could be precisely described in terms of logic (Turing),
and logic could be embodied electronically (Shannon), then one could say that computers
consisting of this electronic embodiment think like human beings – not least since human
beings’ neural hardware can be modeled in terms of logic (McCulloch and Pitts).

Emergence of Cognitive Science. This was the situation that led researchers to in-
vestigate mind-related matters and exchange their findings and ideas at conferences such
as the one that became the beginning of cognitive science: Miller (2003) dates the birth of
cognitive science back to September 11, 1956, at a Massachusetts Institute of Technology
(MIT) symposium on information theory.
On this day, according to Gardner (1985), Allen Newell and Herbert Simon presented
their Logic Theory Machine, which produced the first complete proof of a theorem ever
carried out on a computer, and Noam Chomsky presented his approach to grammar, which
was able to produce natural language. For Karl Lashley. the “problems raised by the
organization of language seem [...] to be characteristic of almost all other cerebral activity”
(Gardner, 1985, p. 12). Indeed, contemporary cognitive psychologists such as Anderson
(2007) believe Chomsky’s work to be the crucial contribution made by cognitive scientists
to refute behaviorists’ theories. A brief illustration, following Gardner’s (1985) account,
shows why: Chomsky’s transformational grammar took certain basic sentences as a starting
point and applied certain transformations on their structure. The ambiguity of sentences
like “The teacher’s marks are very low.” was thus explained as stemming from the same
transformation (explaining to whom something belongs, using the genitive) but starting
from two fundamentally different basis sentences (Gardner, 1985, p. 186). In one sentence,
a teacher is graded and receives low marks: “The teacher has very low marks.”, in the other,
a teacher grades his pupils and gives them low marks: “The teacher gives very low marks.”.
Every computer or human being capable of carrying out such transformations can be said
to understand the ambiguity of the above sentence and how to resolve it – if representations
exist on which to carry out the transformation. For human beings, this meant the existence
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 7

of unobservable representations of phrases in the mind. In his famous reply to one leading
behaviorist’s explanation of language production, Chomsky used arguments like this to
prove that the complexity of language could not be explained by behaviorism (Chomsky,
1959).
Gardner (1985), referring to conversations with Miller, also mentions September 11,
1956, as the date of birth for cognitive science, but adds another pivotal event in the summer
of 1956: John McCarthy, Marvin Minsky, Allen Newell, Herbert Simon and others gathered
at Dartmouth College to discuss what was later termed artificial intelligence. Inspired by
works such as Ashby’s (1952), they also wanted to build mechanistic models of the human
brain that arrived at the same conclusions and succeeded or failed at the same problems.
Reflecting on this history of the cognitive revolution, Miller (2003) actually calls it a
counter-revolution to behaviorism: Mind-related concepts have been central to psychology,
their existence was then denied, and subsequently reinstated. Members of the cognitive rev-
olution believed that the human mind should and could be studied, albeit indirectly: Using
new methods, cognitive scientists derived propositions about the mind and its workings that
resulted in behavior, and they conceived experiments for empirical validation. According
to Anderson (2007), this is still the most important aim of psychology today.

Emergence of HCI as a Separate Field. At the MIT symposium in 1956, everything

needed for the emergence of HCI was already in place – except for a clear research program
and the funding needed to institutionalize it in universities. These two crucial contributions
– research program and funding – can be argued to come from a single person (Kent, 2008;
Taylor, 1990): J. C. R. Licklider. Licklider published two landmark papers (Licklider, 1960
and Licklider & Taylor, 1968) that spurred the birth of human-computer interaction as
a separate field (Kent, 2008). In them, he envisioned using computers as communication
devices to connect human beings all over the planet for information exchange and collabo-
ration, forming man-computer symbioses in which each organism does what it does better –
human beings think and reason, whereas computers process data. Taylor (1990), Licklider’s
coauthor of the aforementioned 1968 paper, recounts that when Licklider then became direc-
tor of the Information Processing Techniques Office of the Pentagon’s Advanced Research
Projects Agency (ARPA) in 1962, it was only logical for him to fund research programs
based on or in accordance with his ideas. He allocated a budget that was bigger than the
total amount of money spent on computer research by all government-supported agencies
at that time, which led to the establishment of a research base with the first PhD programs
in computer science at four universities: Berkeley, Carnegie Mellon, the MIT, and Stan-
ford (Taylor, 1990). The related projects led to the Internet, the mouse, the concept of
“windows” in a computer program, and hypertext (Kent, 2008).
Even before the birth of HCI as a discipline, progress in computer sciences led to the
development of cognitive architectures. Examples are Allen Newell and Herbert Simon’s
Logic Theorist, the General Problem Solver or the State Operator and Result cognitive
architecture (SOAR). These cognitive architectures model and thus predict predict, to a
certain degree, human thinking and behavior (Card, Moran, & Newell, 1983; Newell, 1990).
The first HCI textbook, The Psychology of Human-Computer Interaction by Card et al.
(1983), is built around the Model Human Processor (MHP) cognitive architecture and
contains so many derivations of laws still valid and impressively accurate predictions of user
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 8

behavior that it can still be used to teach HCI courses. It introduces models like Goals,
Operators, Methods, and Selection Rules (GOMS) or the Keystroke-Level Model (KLM),
which are still widely used within HCI to simulate user behavior with given user interfaces
(Card et al. 1983; Kent, 2008; Pew, 2003; Sears & Jacko, 2008; Shneiderman & Plaisant,
2004).
Today, human-computer interaction plays an important role in psychology: Ander-
son, in his introductory chapters to his treatise of cognitive psychology, explicitly mentions
human-computer interaction as a reason to practice cognitive psychology in the first place
(Anderson, 2007). As examples, he refers to two classic texts in HCI. The first is Card
et al.’s (1983) aforementioned HCI textbook. The second explains information foraging
theory, which is able to predict human behavior by treating human beings as organisms
maximizing their information consumption relative to their goals (Pirolli & Card, 1999),
simulating their behavior with the SNIF-ACT cognitive architecture (see next subsection).

In conclusion, HCI evolved from the same developments that led to the cognitive
revolution, the emergence of cognitive science, and, in particular, ARPA’s research programs
in computer science, originating in the 1960s.

Human-Computer Interaction as Improving Human Information Processing

Work on cognitive architectures relied heavily on the information processing approach
to analyze human performance, which originated from work in artificial intelligence (Miller,
2003) and from the warfare research that led to the birth of cybernetics (Miller, 2003;
Mindell, 2002).
As Gardner (1985) documents, cognitive scientists in general, and artificial intelligence
researchers like Newell and Simon in particular, are strongly convinced that the computer
serves as a viable model of how the human mind works. Since computers are composed of
memories containing information and processors operating on those memories, it is not far-
fetched to assume that the mind can be viewed as information processor as well. Warfare
researchers like Wiener (see previous subsection) arrived at human information processing
from a different path: They tried to optimize human performance (Mindell, 2002). One
prominent post-war example of such a study yielding results still used in HCI today is Fitts’
law (Fitts, 1992). Originally published in 1954, Fitts analyzed the ability of human beings
to control the amplitude of movement when trying to hit two targets in turns as fast as
possible with a pointer.
Card et al.’s (1983) textbook is a good example of how human information
processing is, at a rudimentary level, studied in HCI in three stages (cf. Figure 1):
Stage 1 is the perception of new information in the environment. Stage 2 is concerned
with the integration of information, i.e. mental processing of information in order to
generate a solution for the problem to be solved. Stage 3 is the articulation of the
response generated in the second stage. Card et al. (1983) include Fitts’ law (although
they derive it differently) to approximate mouse movement times when users try to
click on a target on the computer screen. This model of human information processing
is used in Mindell’s (2002) history of HCI and is mirrored by all cognitive architec-
tures used in HCI, for example the Model Human Processor (MHP) by Card et al., 1983
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 9

(cf. Figure 1), or Adaptive Control of Thought-Rational (ACT-R) by Anderson et al., 2004.

Figure 1. Human information processing at a rudimentary level. The figure shows Mindell’s (2002)
three phases (perception, integration, and articulation) in relation to the cognitive architecture of
Card et al.’s (1983) Model Human Processor, consisting of three memories and three corresponding
processors.

Perception Integration Articulation

Long-term memory
Working memory
!! Visual image store
!! Auditory image store

Cognitive
processor

Perceptual Motor
processor processor

External world

HCI’s mission can thus be stated as follows: Human-computer interaction must im-
prove the three stages of information processing (cf. Figure 2). In HCI, computers and
computer systems are adjusted to the perception, integration, and articulation of human
beings, not vice versa. The remainder of this section contains a brief summary of three
essential ingredients of HCI research: First, improvements must be measured. Secondly,
information processing can only be assessed once the user tasks are established. Lastly,
theories describing human information processing allow for (numerical) simulation of user
behavior.

Figure 2. Mission of HCI: Improve human information processing.

Perception Integration Articulation

HCI’s Improve perception of Improve integration of Improve articulation

mission information transmitted the perceived information of the users’
from computers to users with goals and previously responses: Apply the
(by means of screen acquired knowledge in chosen operators with
displays as well as audio order to choose the right the least amount of
and haptic feedback). operators. cognitive and physical
effort.

Measuring User Interface Improvements. The term usability is used to summarize

several measures by which computer systems should be improved (Nielsen, 1993; Shakel,
1991): Learnability (novices become efficient quickly), memorability (users do not have
to learn the same interaction twice), efficiency (users can work fast), errors (users do
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 10

not commit errors), and user satisfaction (users are subjectively satisfied). As explained
at the end of the next section, three of those measures have been applied to assess
improvements in all manuscripts in this thesis, and it was hypothesized that certain
changes to a user interface’s navigation mechanism would improve usability and therefore
human performance.

Task Analysis as Starting Point of User Interface Design. User interface design
should start with an analysis of the tasks users execute in order to achieve their goals:
Users try to achieve goals with tasks that are carried out using certain operators, equipped
with input acquired from the environment, certain previous knowledge, having to observe
certain process-limits. These six factors are summarized as the Rationality Principle by
Card et al. (1983). According to Newell (1990), this is the simplest extension over mere
control systems whose feedback loops enable them to react to input from the environment:
Going from control system to knowledge-level system, “the system takes action to attain
its goals, using all the knowledge that it has” (Newell, 1990, p. 50). This corresponds to
Wiener’s notion of goal-directedness of systems. Task analysis lies at the center of HCI: If
tasks are not known, there is no starting point for deriving user interfaces, because they
must not be derived from what an arbitrary person in an arbitrary software company
happens to think when he looks at the database model of the system – this is the main
point of the user-centered design methodology contained in every modern HCI textbook
such as Sears and Jacko (2008). Also, without knowledge of the user tasks, no reasonable
evaluation can be conducted – because every result of the evaluation could be explained
away stating that users hardly ever encounter this specific case.

Simulation of User Behavior. Exact numerical simulations of user behavior are very
hard to achieve, because the number of reasonable combinations of basic operators (eye
movement, mental processing, keystroke, mouse movement, mouse click, etc.) grows very
quickly with the number of steps that a task consists of. The comparison of predictions of
models with different levels of granularity by Card et al. (1983, p. 172) shows a decline
in the accuracy of predicted behavioral sequences with every level of granularity. Thus,
predictions of user behavior related to hypotheses in HCI experiments have to be restricted
to only a few user actions in order to be accurate enough to be tested.
The case study reported by John (2003) gives an impression of the quality of numerical
simulations. In this study, 30 GOMS models were built to predict user behavior for the
15 most frequent tasks that a telephone company’s operators had to execute with their
workstations. Whereas the manufacturer of new workstations that were evaluated in order
to replace the current ones believed that operators were several seconds faster, the GOMS
models predicted they would be an average of 0.63 seconds slower – a stunning fit with the
empirical data, which showed that operators were 0.65 seconds slower.
Extending Anderson’s Adaptive Control of Thought (ACT) cognitive architecture into
SNIF-ACT (Scent-based Navigation and Information Foraging in the ACT architecture),
Pirolli and other researchers have been able to predict with high accuracy which links
users follow for certain tasks (e.g. using the Yahoo! website) or how many pages users
visited before leaving the website (Pirolli, 2007), using rational task analyses to model
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 11

users’ information foraging behavior.

Independent of whether user interface improvements are tested with extensive
numerical simulations or directly with experiments, thinking through them in terms of
cognitive architectures, for example writing a small GOMS or KLM model for an individual
user interaction, is a valuable help to compare alternatives on a conceptual level and make
decisions about which user interface improvements to test or simulate thoroughly (Kent,
2008; Shneiderman & Plaisant, 2004).

In conclusion, improving human information processing in HCI starts with task anal-
ysis and models based on cognitive architectures. The simplest model consists of three
stages: Perception of input, integration of input regarding user goals, and articulation of
appropriate motor action.

Navigation as a Topic within Human-Computer Interaction

Most modern textbooks on human-computer interaction do not treat navigation as a
separate topic (e.g. Kent, 2008; Sears & Jacko, 2008; Shneiderman & Plaisant, 2004), and
many GUI design texts treat the topic rather lightly (e.g. Fowler & Stanwick, 2004), Cooper
(2007) being an exception. As for world-wide-web-specific textbooks, Kalbach (2007), Koy-
ani et al. (2004), and Levene (2006) back up their guidelines and recommendations with
references to research, whereas Rosenfeld and Moreville (2006) and Tidwell (2006) gather
practical advice and thorough descriptions of user interface elements used for navigation
with only occasional references to research, and they seem to contain more unresearched

Figure 3. Norman’s Human Action Cycle: Seven stages of action serve as a process model to
analyze perception, integration, and articulation in human information processing.

Goal

Intention Evaluation

Sequence Interpretation

Integration

Articulation Perception

Execution Perception
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 12

than researched guidelines.

Following the argument of the preceding subsection, if navigation as topic within HCI
is to improve human information processing, then navigation should be treated as an opti-
mization problem. Navigation tasks should be approached in terms of the Rationality Prin-
ciple, simulating the users’ perception, integration, and articulation with respect to their
goals, in order to design navigation mechanisms that improve measures such as Nielsen’s
aforementioned five usability factors (Nielsen, 1993). This approach can be viewed as an
application of Donald Norman’s Human Action Cycle, a generic process model that can be
used to analyze user tasks in terms of perception, integration, and articulation (Norman,
1988), cf. Figure 3.
The user interface does not have to be a graphical user interface (GUI) – it can also
be a command-line or voice-controlled user interface. Improved navigation should result in
improved usability measures: Users learn navigation mechanisms more quickly, remember
navigation options and mechanisms better, execute navigation tasks faster, commit fewer
errors, and are subjectively more satisfied (Nielsen, 1993).
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 13

Three Approaches to Navigation on the World Wide Web

This thesis focuses on a special kind of navigation: Navigation on the World Wide
Web. As with navigation in the real world, users need to be able to determine their position,
chart their course, and find out how far they have already come (Merriam-Webster, 2009).
Correspondingly, when navigating a website, users must be able to determine at any given
time which parts of the website they perceive, what navigation options they have to view
other parts of the website in order to get closer to their goal, and what parts of the website
they have already visited, so they do not try to access these parts again if that does not
serve their goals.
In all manuscripts contained in this thesis, hypotheses were derived from applied
cognitive science theories describing human cognition and behavior. In one manuscript,
it is argued for example that navigation should be conceived such that working memory
load is reduced, because users are not able to mentally deal with more than Miller’s seven
plus/minus two chunks of information (Miller, 1956). Another manuscript uses the Ra-
tionality Principle (Card et al., 1983) as its starting point for building hypotheses. All
theoretical arguments follow the HCI tradition of viewing computer users as human infor-
mation processors, considering certain aspects of perception, integration, and articulation
for different tasks, or in different contexts, respectively. Besides theory, current empirical re-
search has been reviewed, and in one case, qualitative empirical research has been conducted
to inform the design of a user interface that was then researched quantitatively. In order
for the research to have practical relevance, every manuscript contains a brief consideration
of the specific benefits for companies improving navigation on their websites according to
the research outcomes. It can be argued, of course, that HCI is always relevant, because
its main goal is to improve human performance, and every performance improvement in
a company should, once paid for, directly contribute to its profitability. This argument
follows Michael Porter’s assertion that companies have higher profitability than their com-
petitors for only two reasons: lower cost or better product/service differentiation from the
customer’s point of view (Porter, 1980). An HCI performance improvement can contribute
to lowering costs in a company’s value chain or to improving product/service differentiation,
if the improvement relates to a customer-oriented computer system.
Thus, for this thesis, navigation has been approached from three different angles,
explained in the following subsections: First, what can be derived from applied cognitive
science theories about human cognition and behavior, secondly, what has been researched
empirically, and finally, what economic benefits improved navigation systems can deliver.

The Theoretical Approach: Solving the Navigation Problem

Existing literature describes navigation mechanisms (e.g. Spence, 1999; Whitaker,

1998), or mechanisms of cognition that are relevant for human information processing (like
the activation of concepts, see e.g. Anderson, 2007). Spence (1999) proposes a navigation
framework that consists of four user activities: Browse (in order to retrieve new content),
model (build an internal model of what a user encounters during browsing), interpret (deter-
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 14

mine how close one is to the desired goal), and formulate a browsing strategy (determining
whether and how to move on). It is easily seen that this framework fits nicely into the model
of perception, integration, and articulation: Browsing corresponds to articulation; model-
ing, interpreting and formulating a browsing strategy are a more fine-grained description
of integration; the perception phase is suppressed in the model, but its existence is obvious
because users browse in order to retrieve content.
Such frameworks are useful to predict user behavior using simulations (implementing these
frameworks), detailing the aspects of perception, integration, and articulation that are of
interest. Indeed, more sophisticated simulations using cognitive architectures such as the
Model Human Processor (Card et al., 1983), SOAR (Newell, 1990), or EPIC (Executive-
Process/Interactive Control; Kieras, Meyer, Mueller, & Seymour, 1999), which for example
predict task execution times remarkably well, can all be viewed as detailed versions of the
simple perception, integration, articulation model (see Figure 1). In such simulations, user
behavior can be described in minute detail at the keystroke-level, if needs be, and thus nav-
igation behavior can be studied and evaluated without examining every single interaction
empirically.
If user goals and initial system state are given, two variants of the navigation problem
can be formulated. The first variant is the static navigation problem, considered as problem-
solving behavior in three steps that correspond to the rudimentary human information
processing model of perception, integration, and articulation: 1) Detect system state and
acquire task-relevant input. 2) Determine whether input already satisfies the goal. If so,
stop information processing. If not, determine navigation mechanisms or other user interface
options to get closer to the goal. Evaluate alternatives and choose corresponding action to
be articulated. 3) Execute action chosen in the preceding step. Actions can be as diverse
as executing an eye movement, acquiring new input for perception, or setting a new goal,
for example clicking a certain button with the mouse. The system state changes only upon
articulation. The second variant is the dynamic navigation problem. The system state
changes over time, even without user action. If users are not able to dedicate a portion of
their attention to detect changes in the system state which are relevant to their tasks and
integrate the new inputs thus acquired, they probably fail to achieve the task.
Ware (2008) argues that users behave such as to optimize the cost of (cognitive) be-
havior, trying to use mental comparisons instead of eye movements, eye movements instead
of mouse movements, and mouse movements instead of moving the whole body. Applied to
the two variants of the navigation problem, navigation behavior is optimal if users have to
articulate actions only after perception and integration are exhausted, and if no unnecessary
acts of perception and integration occur. Each of the three manuscripts contained in this
thesis uses this theoretical idea to substantiate or motivate hypotheses.

The Empirical Research Approach: Pieces of a Neglected Puzzle

The empirical research found and consulted during the elaboration of the three
manuscripts that make up this thesis can be classified into three types. The first type
is concerned with resolving design dichotomies. This research is structured as “we have
compared A to B and our experiment showed that A is better”. The second type is re-
search about factors influencing user behavior, looking at different factors and determining
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 15

which dominate user behavior under certain circumstances and which do not. The last type
of research evaluates navigation aids, i.e. additions to existing website navigation mecha-
nisms to improve usability. In the following, examples for all three categories are mentioned
so far as they have been found to be typical for a certain category. Detailed accounts of
relevant research are contained in the manuscripts in the appendix.

Design dichotomies. For example: Larson and Czerwinski (1998) found that users
are faster with navigation systems that present more options per level than with navigation
systems that show fewer options but need more levels to make the whole content accessible.
Tullis and Pons (1997) compared different ways of marking required fields on web input
forms and found some to be more efficient than others. Pearson and van Schaik (2003)
found that blue links are better than red ones for visual search tasks, and that menus
should be positioned on the left or on the right, but not on the top.

Factors influencing user behavior. For example: Katz and Byrne (2003) reported that
users – contrary to many beliefs that formed guidelines – do not show dominant search or
navigation behavior based on their personal preferences, but adapt their behavior to the
task, switching their strategy based on task success.

Navigation aids. For example: Olston and Chi (2003) proposed the integration of
searching and browsing behavior, visually highlighting terms that users are assumed to look
for while browsing. Most research in the area of accessibility also falls into this category:
Hardwick, Furner, and Rush (1998), for example, proposed enhancing regular content,
whereas Savidis and Stephanidis (1998) proposed introducing additional possibilities of
interaction.

Improving navigation for a certain user task is time-consuming if the relevance of

dozens of empirical research findings has to be assessed – and reassessed for every other user
task – compared to simulating navigation behavior using a model of perception, integration,
and articulation, at the necessary level of detail. After all, a theoretically sound and valid
simulation of user behavior should not contradict, but rather replicate, most empirical
findings. For this reason, Newell (1990) expressed his dissatisfaction with many research
papers that resolve dichotomies as their only contribution to psychological theory. He
mentions Fitts’ law as an example of a theory that allows the prediction of actual behavior
instead of just resolving a dichotomy. The only cosmetic defect he finds with Fitts’ law
is that it is merely concerned with articulation, leaving out perception and integration –
which is why the Model Human Computer (Card et al. 1983) integrates Fitt’s law as only
one of many principles predicting human behavior.

The Economic Approach: Consumers and Information Workers of the 21st Century
Anderson (2007) mentions the vast growth of information as one of the reasons why
psychologists should concern themselves with human-computer interaction: Human beings
have to be able to guide their attention to relevant information without using too much
time or cognitive effort to do so. The amount of information is growing at stunning rates:
Gantz et al. (2008) estimate that 281 billion gigabytes of information were available at the
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 16

end of 2007, and that by 2011, the amount of information will be ten times the amount of
2006. Improving navigation through this vast amount of information can thus be assumed
to yield economic benefits, as it affects both information workers within companies as well
as customers using these companies’ electronic channels.
The relevance of navigation through information systems is emphasized by McKinsey
research (Johnson, Manyika, & Yee, 2005) showing that jobs in the US are shifting away
from the primary and secondary sector to the tertiary sector at an impressive rate, creating
new jobs where complex interactions between customers and organizations become the
norm – and because complex interactions can not be handled with standardized processes,
information workers need to find and use information quickly and reliably.

Organization and Commonalities of the Three Manuscripts in the Appendix

The manuscripts presented in this thesis assume three different perspectives on nav-
igation on the World Wide Web: The user interface perspective investigates the elements
of which a navigation user interface should consist in the first place, and how they have to
be assembled in order to fit human capabilities best. The navigation design perspective is
concerned with the improvements gained from both the arrangement of the user interface
elements on the screen as well as the different possibilities of displaying the same navigation
options. The content representation perspective looks beyond finding the right navigation
option on the screen to finding the right content or functionality within the web page on
which a user has landed.
The studies cover different aspects of perception, integration, and articulation: The
first study is concerned with blind internet users. The conception of a text user interface
enhanced for their perception and integration is documented and the interface is tested
empirically and compared to an interface that is compliant with current accessibility guide-
lines. In the second study, it is shown that, articulation being equal, users adjust perception
and integration to the complexity of the navigation task they attempt, and that some navi-
gation mechanisms make better use of perception and integration than others, saving users
behavioral cost. The third study is concerned with users who are already on the right web
page but need to navigate within this page. A difference as small as replacing the asterisks
with colored backgrounds to mark required fields on web forms is shown to cause differences
in user behavior and satisfaction.
Among other measures, task execution time, number of errors, and subjective sat-
isfaction were used in all three manuscripts to assess the usability of the user interface.
These are the three of Nielsen’s five usability measures (Nielsen, 1993) that can be studied
in single experiments. To study the other two factors, learnability and memorability, users
needed to be retested several times to measure learning curves and memory retention rates,
which were judged to be less important for this thesis.
Another commonality of the three manuscripts is that they are concerned with costs
of user behavior. The first manuscript deals with the cognitive costs of blind internet users
dealing with input from graphical user interfaces and how those costs could be reduced
with an alternative user interface, the enhanced text user interface. The second manuscript
deals with the costs of acquiring, processing and acting on navigation options. The third
manuscript deals with the costs of detecting visual elements on the screen.
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 17

Summary of Manuscripts

The following manuscripts constitute this thesis. The first and third manuscripts
have already been published and the second has been submitted to an established human-
computer interaction journal and is currently under peer review:

1. Leuthold, S., Bargas-Avila, J. A. & Opwis, K. (2008). Beyond web content acces-
sibility guidelines: Design of enhanced text user interfaces for blind internet users. Interna-
tional Journal of Human Computer Studies, 66(4), 257-270. doi:10.1016/j.ijhcs.2007.10.006
2. Leuthold, S., Schmutz, P., Bargas-Avila, J. A., Tuch, A. & Opwis, K. Influence of
task complexity and navigation design on user behavior on the world wide web: Eye-tracking
study measuring navigation performance and strategy. [submitted to the International
Journal of Human Computer Studies]
3. Pauwels, S. L., Hübscher, C., Leuthold, S., Bargas-Avila, J. A., & Opwis, K.
(2008). Error prevention in online forms: Use color instead of asterisks to mark required
fields. Interacting with Computers, 21(4), 257-262. doi:10.1016/j.intcom.2009.05.007

Manuscript Summary 1: Beyond Web Content Accessibility Guidelines: Design of Enhanced

Text User Interfaces for Blind Internet Users
The first manuscript deals with how navigation on the World Wide Web can be im-
proved from the user interface perspective: What user interface elements do users interact
with in the first place, and how do they have to be assembled in order to fit human ca-
pabilities best? The user interface presented here was improved for blind internet users’
working memory usage and correct derivation of mental models. This manuscript serves as
an example of how navigation can be improved – using user-centered design methods like
ISO 13407.
Motivation and Study Design. Several countries have passed laws to enforce accessi-
bility of the internet for handicapped users. But websites do not become usable just because
their content is accessible. This manuscript contributes to the discussion of how important
usability is from the users’ perspective: Even world-wide standards like the World Wide
Web Consortium’s Web Content Accessibility Guidelines (WCAG) might not even make a
difference in terms of usability measures like efficiency, number of errors, or satisfaction in
website usage. Current research shows that the WCAG are rarely used, and if they are used,
they are not enough – in one study, almost half of the blind participants did not succeed
with their task, and about half of the observed errors have to be attributed to problems
beyond accessibility.
Based on theory, as well as qualitative interviews and observations of blind internet
users (following the design process described in ISO 13407), nine guidelines have been devel-
oped for construction of an enhanced text user interface (ETI). Three main considerations
are documented as the largest influences on guideline development:
1. Consistency. The same user interface elements should always be used in the same
manner and never behave differently.
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 18

2. Working memory load reduction. At any given moment, users should not be re-
quired to handle more than the famous seven plus/minus two chunks of information.
3. Building correct mental models. Users should be able to build correct mental
models from what they perceive of the user interface and the website.
The nine ETI guidelines are an extension of the existing WCAG guidelines, and the
ETI satisfies all accessibility measures; but it is adjusted to the capabilities of the blind
in order to improve usability, as measured by efficiency, number of errors, and subjective
satisfaction.
An experimental design with 39 blind participants executing a search and a navigation
task was chosen. They used the JAWS 5.1 screenreader software and, if required, a braille
terminal, to execute the tasks on a website in three different versions: a) The original
unaltered website. b) The same website with alterations that made it compliant to all
WCAG guidelines. c) The same website with the enhanced text user interface. Task
performance in seconds, average number of errors, as well as subjective evaluation of the user
interface were measured after each task as operationalization of three of Nielsen’s usability
measures. Subjective evaluation was measured with three items: How pleasant it was to
surf on the website (pleasant), how the website compared to other websites users normally
visited (comparison), and how fast they worked on the task (speed).

Findings and Conclusions. No significant differences were found between the original
and the WCAG-compliant websites for all measures. Therefore, data for the original and the
WCAG-compliant website have been merged to calculate differences between the graphical
user interface (GUI) and the ETI.
It is shown that blind internet users completed the search task significantly faster
with the ETI than with the GUI, committing fewer errors, subjectively preferring the ETI
regarding the pleasant and comparison measure. For the navigation task, however, no
significant differences exist.
The results demonstrate that the WCAG are not enough for blind internet users.
Extending them with at least the six of the nine guidelines for the ETI that cover specific
needs of the blind would lead to significant usability improvements. Since significant differ-
ences were found only for the search task and not for the navigation task, we concluded –
in line with our personal impressions from experimental observations – that the problems
users had with the information architecture outweighed the differences between the ETI
and the GUI: If users do not understand their navigation options, they can not navigate
– regardless of whether they are blind or sighted. However, the majority (55%) of the 39
users still preferred the ETI in direct comparison to the GUI, 30% found the interfaces to
be equal and 15% preferred the GUI to the ETI.

Implications for Future Research. Future research should aim at gaining insight into
navigation strategies for people with different handicaps. For blind internet users, there is
at least one crucial difference in navigation strategies that new navigations should overcome:
Trial and error is a very efficient navigation strategy for sighted users, but it is of no use
to the blind. For sighted users, an ambiguous label in a navigation system is simply clicked
with the maximum loss of a few seconds – however, for blind users, following the wrong
link can mean several minutes lost. Future research might show that websites for the blind
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 19

should be split into navigation pages and content pages, with each page type improved for
its specific purpose.
Of course, future research should also study Nielsen’s two remaining usability factors,
learnability and memorability, and there may be additional factors that should be included
to improve usability for the blind.
Finally, each of our nine guidelines should be varied and researched empirically, and
additional guidelines (for example how to conceive navigation labels) should be constructed
and tested.

Manuscript Summary 2: Influence of Task Complexity and Navigation Design on User

Behavior on the World Wide Web: Eye-Tracking Study Measuring Navigation Performance
and Strategy

In the second manuscript, navigation on the World Wide Web is looked at from the
perspective of the actual design of the navigation system: Once the user interface elements
and corresponding guidelines are defined, what can be improved in the graphical design
and layout of the navigation items? In the corresponding experiment, in terms of the basic
information processing model pictured in Figure 1, input for perception was varied and
the effect on integration and articulation was studied. Thus, task performance, navigation
strategy, and subjective preference was researched for different navigation designs with
different levels of task complexity. This manuscript serves as an example of what can be
improved regarding navigation mechanisms – there are many design options to choose from.

Motivation and Study Design. The World Wide Web is growing. In August 2008, a
website consisted of on average 5’600 web pages. Current literature on navigation as well
as HCI textbooks contain many guidelines for navigation that lack empirical validation,
and no empirical research has been found to date that treats navigation as problem-solving
behavior, studying whether predictions from theoretical frameworks describing navigation
behavior hold true.
As a theoretical framework, this study uses the Rationality Principle included in the
Model Human Processor cognitive architecture. The Rationality Principle postulates that
user behavior is determined by three factors describing the objective situation (goals, task,
and operators), and three factors describing individual users’ limitations (perceivable inputs,
user knowledge, and processing ability). Because the resulting behavior can be characterized
as problem-solving behavior, it should be possible to improve navigation systems by looking
at the navigation problem in these terms. Thus, two of the Rationality Principle’s factors
were varied: a) simple tasks versus complex tasks were studied for factor task, and b) simple
navigation versus extended visible navigation versus extended menu navigation were studied
for factor inputs. Extended navigation is navigation with additional navigation elements
that have been added to help users navigate; in visible navigation, all navigation elements
are displayed at once; in menu navigation, all navigation items are hidden behind a menu.
Improving navigation through variation of perceivable input sought to improve ef-
ficiency, errors, and satisfaction – the three usability measures used in all manuscripts
contained in this thesis. To minimize the effects of other factors (such as individual partic-
ipants’ processing ability) that affect quantitative measures, efficiency was operationalized
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 20

as time to first click and as number of eye fixations needed to arrive at that first click.
For the short timespan needed for the first click, it was assumed that predictions of user
behavior were sufficiently accurate and that improvements derived from theory manifested
empirically.
The manuscript hypothesized that navigation is improved in several ways:
1. If users attempt complex tasks, efficiency, errors and satisfaction are improved
using extended navigation instead of simple navigation – if the additional navigation items
are constructed in a way as to reduce the number of mental operations needed to compare
the goal state to the navigation item.
2. Within extended navigation, users adjust their navigation strategy to task com-
plexity – preferring the additional navigation items that reduce their cognitive effort for
complex tasks and ignoring those items for simple tasks.
3. Based on the notion of cognitive cost, efficiency and satisfaction are improved
using visible navigation instead of menu navigation – because users perceive and integrate
more items per eye movement and thus need fewer eye and mouse movements to decide
which navigation item to choose, and a subjective notion of cognitive cost results in higher
satisfaction with the more efficient navigation.
To test these hypotheses, 120 participants were divided in six groups and used the
three navigation designs in all possible sequences to attempt a total of 36 tasks. The tasks
were presented in randomized order and divided into twelve tasks per navigation design, six
of which were simple tasks and six complex ones. User performance was operationalized as
number of fixations before first click, time before first click, and correctness of first page.
To measure navigation strategy, it was determined from log files whether users chose a
navigation item from extended navigation. Subjective preference was measured using four
items, asking users to assess ease of use, helpfulness, effort needed, and frustration with a
certain navigation design.

Findings and Conclusions. Both task complexity and navigation design influence effi-
ciency: Users needed less time and fewer eye fixations for simple tasks (compared to complex
tasks), and they were more efficient with extended visible navigation (compared to menu
navigation). They also subjectively preferred extended visible navigation on three of four
scales, rating it to be easier to use, to be more helpful, and to cause less frustration. In-
terestingly, users committed fewer errors with simple tasks than with complex tasks using
simple navigation and extended visible navigation, but not using menu navigation. Visible
navigation is more efficient and causes fewer errors than menu navigation.
Task complexity influenced navigation strategy: Although participants did not ab-
solutely prefer items from extended navigation for complex tasks, they chose significantly
more such items for complex tasks than for simple tasks, adapting their behavior depen-
dent on task complexity. Although only a small payoff, adding extended navigation item
to simple navigation pays for complex tasks. Task complexity also influenced the number
of eye fixations needed: Users needed more fixations and were slower for complex than for
simple tasks.
That task complexity influences user performance as well as navigation strategy is
insofar astonishing as users not only needed more time to mentally process the perceived
items longer – they also needed more eye fixations for complex tasks, increasing the amount
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 21

of visual information processed for complex tasks. This could be explained with the top-
down process of perception triggering more eye fixations for complex tasks because this
is less costly for cognition than conducting more mental comparisons. This cost-based
explanation would be supported by participants’ showing another cost-based effect, adapting
their navigation strategy in an interesting way: For complex tasks, 24% of users, after
clicking the correct navigation item and landing on the correct product page, clicked another
navigation item and navigated away from the correct page before they had perceived enough
information to determine that their navigation task was solved. Was the expected cost of
mental comparisons and eye fixations greater than the cost of navigating to another web
page? This behavior was observed for only 1% of users executing simple tasks.
Implications for Future Research. Because filters used in search forms are imple-
mented to eliminate the amount of mental operations needed to process search result lists,
it should be studied under what circumstances users prefer navigation items that eliminate
mental operations and under what circumstances they prefer filters. Google introduced a
left-side navigation items to filter their results list in May 2009, in addition to real drop-
down filters that they still use on their extended search page (Mayer, 2009).
If the result that users need more eye fixations for complex tasks is interpreted as
the top-down visual processing leading them to trade off eye fixations against mental com-
parisons, the turning point should be empirically documented and compared to theory:
Do users start to trade off mental comparisons against eye movements at around 15 men-
tal comparisons? And are there user-specific or contextual factors influencing this turning
point? Another trade off worth studying is the point at which users start to trade off mouse
movements against eye movements. For this reason, for very large numbers of navigation
items it should be possible to observe a preference reversal where navigation menus are
being preferred to visible navigation items. Research into these questions could lead the
way to improving navigation systems for users’ cognitive costs.

Manuscript Summary 3: Error Prevention in Online Forms: Use Color Instead of Asterisks
to Mark Required-Fields
The third perspective of improving navigation on the World Wide Web is the content
representation perspective: Even if users are able to use the user interface and the navigation
design to get to a page with content/functionalities important to them, their attention still
needs to be allocated correctly within the page. This third study addressed the problem
of guiding attention to required fields when working with online forms. Like the second
manuscript, this manuscript also serves as an example of what can be improved regarding
navigation – guiding perception with user interface elements.
Motivation and Study Design. Information workers enter data into input forms just
as factory workers place items on a conveyor belt. If productivity for both is defined in
terms of transformations that can be carried out on the input (data or item) per time unit,
companies should improve online forms in order to enable users to enter the right data into
the corresponding input fields. Above all, if users are not paid for entering data, as is the
case with online shoppers, for example, it seems reasonable to advise companies to design
their online forms in a way as to maximize input speed and accuracy. Experience, however,
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 22

indicates that completion rates of online forms in e-commerce applications can be increased
by 10% to 40%, thus leading to huge economic gains.
Today, the asterisk is mentioned in or recommended by many guidelines as the de-
facto standard to indicate required input fields in web forms – although only one study
researching several possibilities for marking required input fields could be found. Thus, the
aim of this third study was to test a simple hypothesis: Users are faster, more satisfied,
and commit fewer errors when required input fields in online forms are colored, compared
to using an asterisk placed next to the input field label.
To test the hypothesis, 24 employees of a financial institute were asked to attempt
two tasks, one with an online form using the asterisk to mark required fields, the other with
an online form with colored input fields to indicate required fields. Completion time and
error rate, i.e. the total number of required fields left blank for all attempts to submit the
online form until successful submission, were measured. After each task, users filled out
the Questionnaire for User Interaction Satisfaction (QUIS), which was used as satisfaction
measure.

Findings and Conclusions. Participants committed significantly fewer errors with on-
line forms using colored fields to mark required input, and they were significantly faster
and more satisfied. It took users 41% less time to complete the online form successfully if
required fields were marked using background color, and they committed 60% fewer errors.
They were also significantly more satisfied with colored required fields.
It is concluded that even small details matter in user interface design. Although for
the sake of accessibility, designers should be advised against using colored fields as sole
identifier of required fields, coloring contributes much to guiding users’ attention to the
right locations on the screen – and users are aware of that, as shown by their satisfaction
ratings.
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 23

Conclusions

Three different kinds of conclusions can be drawn from the manuscripts in this the-
sis and the respective literature and theories studied to produce them. First, this thesis
raises certain concerns about guideline application: Guidelines must be empirically vali-
dated (many current guidelines are not). And independent of whether they were validated
or not, it might cost a company more to test and apply existing guidelines than to con-
ceive their own. Design solutions derived with user-centered design are superior to purely
guideline-conformant solutions. The second kind of conclusions consists of methodological
issues every researcher deals with and how they manifested in the course of this thesis.
Lastly, the case is made for an extension of cognitive architectures to include emotions in
order to predict navigation behavior based on a more complete model.

Questioning the Value and Cost of Guideline Application

Guidelines Should be Validated Empirically. Every study conducted for this thesis
challenged a prevalent guideline for navigation on the World Wide Web by empirical vali-
dation: The enhanced text user interface study researched the Web Content Accessibility
Guidelines (WCAG, 1999). The navigation design study shed light on the conflicting guide-
lines of Nielsen (1999), who recommends using menus to display a large number of navigation
items, and Fowler and Stanwick (2004), who advise designers to display as many navigation
items on a single web page as possible. The required fields study researched the guideline
recommending an asterisk to mark required fields on online forms.
The summaries of the respective theoretical backgrounds show that, for these guide-
lines, almost no empirical validation is available, and not much is cited in the works that
contain them. It can be argued, of course, that the World Wide Web Consortium’s accessi-
bility guidelines, the de-facto standard that is even referenced in laws regarding handicapped
users (see manuscript 1), have more weight than an acclaimed usability guru’s preference
for menus, and should consequently have been validated more thoroughly. But in all three
cases, user interface designers and web technology practitioners around the globe might rely
on guidelines of inferior usability – as measured with Nielsen’s (1993) usability factors –
because every manuscript documents usability improvements over and above the existing
guidelines by using rather simple measures.
In this regard, this thesis is making a case for more work like that of Koyani et
al. (2004) or Levene (2006), and perhaps for a reworked format of guideline textbooks to
include empirical results as well as the guidelines’ fancy illustrations.
User-Centered Design, Not Guidelines, Generates Good Usability. Guidelines for web-
site design such as those drawn up by Lynch and Horton (2009), Fowler and Stanwick (2004),
or Nielsen (1999) contain a lot of practical advice about how to set up, staff, and run website
projects, and – above all – how web pages should be designed. These guidelines are valu-
able and this thesis does not argue against using them. But consideration of the principles
obeyed in this thesis, which led to user interfaces with improved usability, demonstrates the
value of user-centered design (UCD), which demands much more than guideline application.
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 24

Guideline application does not guarantee usability with respect to Nielsen’s (1993) –
or anybody’s – criteria, as the first manuscript illustrates. Consistently conceived guide-
lines only ensure user interface consistency (which is good, but not good enough). Interfaces
become usable if they are derived from user requirements, take into consideration what hu-
man information processors are able to do and what they are not, and take the Rationality
Principle’s six factors of user behavior into account. Considering this approach, govern-
ments and companies might want to reconsider passing laws that enforce accessibility only,
spending millions to make their websites accessible when in fact bad usability prevented
customers from performing their tasks.
User-centered design, as proposed by ISO 13407 (1999), starts with a specification
of the users’ context and as well their as organizational requirements. It then proposes
to produce design solutions derived from those requirements and, lastly, to evaluate the
design solutions against the requirements. Guidelines are considered in the production of
design solutions, because if they have sufficient reach (as have many guidelines that became
standards), every internet user around the world will sooner or later encounter and get used
to them – and building on habituation is in most cases better than having users learn new
forms of interaction.

Balancing Cost and Value of Guideline Application. Even more important than the
application of guidelines directly to user interface problems might be to use them to generate
evaluation criteria – if their rationale is sufficiently known. User interface designers derive
design solutions that can and should be tested with real users, but testing them in an expert
review against a set of readily available guidelines and thinking through the reasons why
a certain guideline can or must not be violated is a quick and inexpensive way to improve
the usability of the design solution. Shneiderman and Plaisant (2004, p. 142) explicitly
mention guidelines review as an expert review method in addition to heuristic evaluation,
consistency inspection, cognitive walkthrough, or formal usability inspection.
But checking user interfaces against arbitrary – and probably inconsistent or even con-
tradictory – guidelines might even cost more than the economic value it creates: Validating
every explicit and implicit guideline introduced by any party of a website project (busi-
ness stakeholders, requirements engineers, information architects, user interface designers,
graphical designers, programmers, etc.) involves a lot of human effort and cost, because
many people’s inputs have to be consolidated in a consistent and transparent fashion.
Gathering and testing user interface design guidelines for one’s own company is cer-
tainly an economically reasonable way of rendering guidelines valuable: The cost and effort
of guideline conception, validation, distribution and updates are spread over many websites
and other corporate applications. Design patterns are a way of documenting guidelines
such that conceptual designers, who can only generate design solutions by permutations of
these patterns, cannot even conceive of user interfaces that are not guideline-compliant –
Tidwell’s (2006) design pattern library can serve as an inspiration in this case, although the
status of empirical validation of most patterns remains unclear.
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 25

Methodological Conclusions

The three manuscripts in this thesis correspond to the three types of empirical research
summarized above: The study about required fields (manuscript 3) resolved a dichotomy,
finding that marking required fields with background color is superior to marking them with
an asterisk. The navigation design study (manuscript 2) researched two factors influencing
user behavior : Task complexity and navigation design influence not only user performance
and subjective preference, but also the navigation strategy that they use to solve naviga-
tion tasks. The enhanced text user interface study (manuscript 1) can be classified as a
conception of new navigation aids for blind internet users.
Every type of research had its own methodological issues. The dichotomy was easiest:
Generate two alternative design solutions and test them with the same measures – the only
problem being whether the differences really show, i.e. are neither too small nor covered by
stronger factors influencing the same dependent variables. In the study with blind internet
users, we assume that navigation labels had such a strong effect on performance that we
were unable to measure significant differences between the two interfaces for the browse task.
Labels could have been researched beforehand to make sure that users understood them,
but this would have threatened ecolocial validity. The main problem researching factors
influencing user behavior is, of course, to isolate the factors of interest, rule out all other
influences and still have an ecologically valid experiment. In the navigation design study,
we traded off ecological validity against statistical significance and reliability. Conceiving
new navigation aids is arguably associated with the biggest conceptual effort, because one
has to start at the very beginning of user-centered design and understand and specify the
context of use and the user and organizational requirements.

Better Models of Navigation Behavior

Four distinct navigation strategies were found in the three manuscripts contained
in this thesis. The first is trial and error : Users are not sure whether a link leads to a
desired page. They follow it if they believe an existing navigation mechanism brings them
back quickly should the click turn out to be wrong. In the study with blind internet users
(manuscript 1), this navigation strategy was observed to be very costly and user interface
designers have to be very careful not to rely on it (for example designing web pages optimized
for impulse buying, distracting users from their original goals). The second is matching a
known item to navigation items. The navigation study (manuscript 2) documents this
strategy: Users have a certain known item in mind and consider category navigation items
until they find one (or the best, respectively) that they believe takes them closer to the
known item. The third strategy is matching a task-implicit attribute to navigation items.
This was also observed in the navigation study: Users look for a product on sale and follow
a link called “discounts”. The last strategy is paying attention to what sticks out most. The
additional saliency provided by the background color caused users to direct their attention
to the required fields in the respective study (manuscript 3). To date, no textbook or
research article is to be found documenting navigation strategies like these systematically
and from an applied cognitive science perspective. Future researchers and authors of HCI
textbooks should consider gathering all observed and all theoretically feasible navigation
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 26

strategies and work towards an overarching model of navigation behavior that explains
strategy selection.
Such an endeavor could address an important theoretical gap: In this thesis’
manuscripts, hypotheses were derived from purely cognitive theories, excluding emotion.
Notions such as Ware’s (2008) about minimizing cognitive cost led to assumptions about
observable user behavior that were tested empirically. In the tradition of cognitive architec-
tures, behavior was assumed to be composed of certain basic operators, each taking place
in a matter of hundreds of milliseconds. Of the Rationality Principle’s six factors, only
task (in all studies), operators (navigation design study), inputs (navigation design study),
and limits in processing ability (seven plus/minus two rule in first study) were taken into
account. Regarding the other two factors, a product’s brand or previous experiences with
similar websites, for example, needed to be factored in as previous knowledge and input, and
emotion as evaluation of a goal’s relevance. Whereas models like Spence’s (1999) are not
fine-grained enough to account for such factors, GOMS-extensions, which include previous
knowledge (Card et al., 1983) predict behavioral sequences correctly only for the timespan
of a few seconds, and to date exclude emotion altogether.
Pirolli (2007) documents interesting applications of the SNIF-ACT cognitive archi-
tecture that describes user behavior on a level in between Spence’s (1999) and Card et al.’s
(1983) models. Although SNIF-ACT also composes user behavior of certain basic operators,
Pirolli’s (2007) information foraging theory manages to describe link-following navigation
behavior on a level of tens of seconds and minutes instead of hundreds of milliseconds.
Laird (2008) describes an extension of the Soar cognitive architecture that includes
emotion. Langley, Laird, & Rogers (2009), in their recent exposition of research issues and
challenges regarding cognitive architectures, explicitly add emotion to their open issue list
regarding cognitive architectures.
If a cognitive architecture could be extended in such a way, not only would a more
complete theory of navigation emerge (simulating goal-directed behavior, including previous
knowledge as well as emotions constantly updating goals), it would also prepare the way
to test navigation designs automatically. With such a cognitive architecture, one would be
able not only to predict which links users follow (optimizing cost of behavior and expected
utility), but also, for example, when users would be distracted from their original goals by
certain products from known brands, whether the distraction is pleasant and able to change
their current goal, and how their preferences would be updated in the process.
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 27

References

Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., Qin, Y. (2004).
An integrated theory of the mind. Psychological Review, 111 (4), 1036 –1060. doi:
10.1037/0033-295X.111.4.1036

Anderson, J. R. (2007). Kognitive Psychologie (6th ed.). Heidelberg, Germany: Spektrum

Akademischer Verlag.

Ashby, W. R. (1952). Design for a brain. Oxford, United Kingdom: Wiley.

Card, S. K., Moran, T. P., Newell, A. (1983). The psychology of human-computer inter-
action. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Publishers

Chomsky, N. (1959). A review of B. F. Skinner’s verbal behavior. Language, 35 (1), 26-58.

Cooper, A., Reimann, R., Cronin, D. (2007). About face 3: The essentials of interaction
design. Indianapolis, IN: Wiley Publishing, Inc.

Fitts, P. (1992). The information capacity of the human motor system in controlling the
amplitude of movement. Journal of Experimental Psychology, 47, 381-391.

Fowler, S., Stanwick, V. (2004). Web application design handbook: Best practices for
web-based software. San Francisco, CA: Morgan Kaufmann Publishers.

Gantz, J. F., Chute, C., Manfrediz, A., Minton, S., Reinsel, D., Schlicht-
ing, W., Toncheva, A. (2008). The diverse and exploding digital uni-
verse: An updated forecast of worldwide information growth through 2011.
Retrieved from http://www.emc.com/collateral/analyst-reports/diverse-exploding-
digital-universe.pdf

Gardner, H. (1985). The mind’s new science: A history of the cognitive revolution. New
York, NY: Basic Books.

Hardwick, A., Furner, S., Rush, J. (1998). Tactile display of virtual reality from the world
wide web: A potential access method for blind people. Displays, 18, 153-161.

ISO 13407. (1999). Human centred design processes for interactive systems. Geneva,
Switzerland: International Standards Organization.

James, W. (1890). The principles of psychology. New York, NY: Henry Holt & Co.

John, B. E. (2003). Information processing and skilled behavior. In Carroll, J. M. (Ed.),

HCI models, theories, and frameworks: Toward a multidisciplinary science (pp. 55-
101). San Francisco, CA: Morgan Kaufmann Publishers.

Johnson, B. C., Manyika, J. M., Yee, L. A. (2005). The next revolution in in-
teractions. Retrieved May 24, 2009, from The McKinsey Quarterly website:
http://www.mckinseyquarterly.com/The next revolution in interactions 1690
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 28

Kalbach, J. (2007). Designing web navigation. Sebastopol, CA: O’Reilly Media, Inc.

Katz, M. A., Byrne, M. D. (2003). Effects of scent and breadth on use of site-specific search
on e-commerce web sites. ACM Transactions on Computer-Human Interaction, 10 (3),
198-202.

Kent, N. L. (2008). Cyberpsychology: An introduction to human-computer interaction.

New York, NY: Cambridge University Press.

Kieras, D. E., Meyer, D. E., Mueller, S., Seymour, T. (1999). Insights into working memory
from the perspective of the EPIC architecture for modeling skilled perceptual-motor
and cognitive human performance. In Miyake, A., Shah, P. (Eds.), Models of working
memory: Mechanisms of active maintenance and executive control (pp. 183-223).
New York, NY: Cambridge University Press.

Koyani, S. J., Bailey, R. W., Barnum, C., Bosley, J., Chaparro, B., Dumas, J., ... Zim-
merman, D. (2004). Research-based web design and usability guidelines. Retrieved
May 24, 2009, from the U.S. Department of Health & Human Services Usability.gov
website: http://www.usability.gov/pdfs/guidelines.html

Laird, J. E. (2008). Extending the Soar cognitive architecture. In Wang, P., Franklin, S.
(Eds.), Artificial General Intelligence, 2008: Proceedings of the First AGI Conference
(pp. 224-247). Amsterdam, Netherlands: IOS Press.

Langley, P., Laird, J. E., Rogers, S. (2009). Cognitive architectures: Re-

search issues and challenges. Cognitive Systems Research, 10 (2), 141-160.
doi:10.1016/j.cogsys.2006.07.004

Larson, K., Czerwinski, M. (1998). Web page design: Implications of memory, struc-
ture, and scent for information retrieval. CHI ’98: Proceedings of the SIGCHI Con-
ference on Human Factors in Computing Systems, 25-32. New York, NY: ACM
Press/Addison-Wesley Publishing Co.

Leuthold, S., Bargas-Avila, J. A. & Opwis, K. (2008). Beyond web content ac-
cessibility guidelines: Design of enhanced text user interfaces for blind inter-
net users. International Journal of Human Computer Studies, 66 (4), 257-270.
doi:10.1016/j.ijhcs.2007.10.006

Leuthold, S., Schmutz, P., Bargas-Avila, J. A., Tuch, A. & Opwis, K. Influence of task
complexity and navigation design on user behavior on the world wide web: Eye-
tracking study measuring navigation performance and strategy. [submitted to the
International Journal of Human Computer Studies]

Levene, M. (2006). An introduction to search engines and web navigation. Harlow, United
Kingdom: Addison Wesley Publishing Company.

Licklider, J. C. R. (1960). Man-Computer Symbiosis. IRE Transactions on Human Factors

in Electronics, 4–10.
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 29

Licklider, J. C. R., Taylor, R. W. (1968). The computer as a communication device.

Science and Technology: For the Technical Men in Management, 21-31.

Lynch, P. J., Horton, S. (2009). Web style guide: Basic design principles for creating web
sites. New Haven, CT: Yale University Press.

Mayer, M. (2009, May 12). More Search Options and other updates from our Searchol-
ogy event. Retrieved November 22, 2009, from The official Google Blog website:
http:googleblog.blogspot.com200905more-search-options-and-other-updates.html

Merriam-Webster. (2009). Merriam-Webster online dictionary entry on navigation. Re-

trieved May 1, 2009, from http://www.merriam-webster.com/dictionary/navigation

Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our
capacity for processing information. The Psychological Review, 63, 81-97.

Miller, G. A. (2003). The cognitive revolution: A historical perspective. Trends in Cogni-

tive Sciences, 7, 141-144.

Mindell, D. A. (2002). Between human and machine: Feedback, control, and computing
before cybernetics. Baltimore, Maryland: The John Hopkins University Press.

Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University

Press.

Nielsen, J. (1993). Usability engineering. San Francisco, CA: Morgan Kaufmann Publish-
ers.

Nielsen, J. (1999). Designing web usability. Berkeley, CA: New Riders.

Norman, D. A. (1988). The psychology of everyday things. New York, NY. Basic Books.

Olston, C., Chi, E. H. (2003). ScentTrails: Integrating browsing and searching on the web.
ACM Transactions on Computer-Human Interaction, 10 (3), 177–197.

Pauwels, S. L., Hübscher, C., Leuthold, S., Bargas-Avila, J. A., & Opwis, K. (2009). Error
prevention in online forms: Use color instead of asterisks to mark required fields.
Interacting with Computers, 21(4), 257-262. doi:10.1016/j.intcom.2009.05.007

Pearson, R., van Schaik, P. (2003). The effect of spatial layout of and link colour in
web pages on performance in a visual search task and an interactive search task.
International Journal of Human-Computer Studies, 59 )(3), 327-353.

Pew, R. W. (2002). Evolution of human-computer interaction: From Memex to bluetooth

and beyond. In Sears, A., Jacko, J. A. (Eds.). The human-computer interaction hand-
book: fundamentals, evolving technologies, and emerging applications (1-17). New
York, NY: Lawrence Erlbaum Associates.

Pirolli, P. L. & Card, S. K. (1999). Information foraging. Psychological Review, 106,

643-675.
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 30

Pirolli, P. L. (2007). Information foraging theory: Adaptive interaction with information.

New York, NY: Oxford University Press.

Porter, M. (1980). Competitive strategy. New York, NY: The Free Press.

Rosenfeld, L., Moreville, P. (2006). Information architecture for the world wide web:
Designing large-scale web sites (3rd ed.). Sebastopol, CA: O’Reilly Media, Inc.

Savidis, A., Stephanidis, C. (1998). The Homer UIMS for dual user interface development:
Fusing visual and non-visual interactions. Interacting with Computers, 11, 173-209.

Sears, A., Jacko, J.A., 2008. The human-computer interaction handbook: Fundamentals,
evolving technologies, and emerging applications (2nd ed.). New York, NY: Lawrence
Erlbaum Associates.

Shakel, B. (1991). Usability – context, framework, definition, design and evaluation. In

Shackel, B.; Richardson, S. J. (Eds.). Human factors for informatics usability. Cam-
bridge, United Kingdom: Cambridge University Press

Shneiderman, B., Plaisant, C. (2004). Designing the user interface (4th ed.). Reading,
MA: Addison-Wesley.

Spence, R. (1999). A framework for navigation. International Journal of Human-

Computer Studies, 51 (5), 919-945.

Taylor, R. W. (1990). In memoriam J. C. R. Licklider: 1915-1990. Palo Alto, CA:

Digital-Systems Research Center.

Tidwell, J. (2006). Designing interfaces: Patterns for effective interaction design. Se-
bastopol, CA: O’Reilly Media, Inc.

Tullis, T.S ., Pons, A. (1997). Designating required vs. optional input fields. CHI ’97
Extended Abstracts on Human Factors in Computing Systems, 259-260. New York,
NY: ACM.

Ware, C. (2008). Visual thinking for design. San Francisco, CA: Morgan Kaufmann
Publishers.

Watson, J. B. (1924). Behaviorism. New York: Peoples Institute.

WCAG. (1999). Web content accessibility guidelines 1.0. Retrieved May 24, 2009, from
the W3C recommendation website: http://www.w3.org/TR/WAI-WEBCONTENT/

Whitaker, L. A. (1998). Human Navigation. In Forsythe. C., Grose, E., Ratner, J. (Eds.).
Human factors and web development (pp. 63-71). Mahwah NJ: Lawrence Erlbaum
Associates.
 THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 31

Acknowledgements

First and foremost, I am greatly indebted to Dr. Javier Bargas-Avila, for without his
continuing support, occasional (and not always flattering) remarks about how he believed
my doctoral thesis was doing, and his understanding of what it meant to me to be able to
work full-time in a consultancy and still write a thesis, I would not even have finished my
first psychological experiment a few years ago.
Prof. Dr. Klaus Opwis contributed valuable perspectives on how to focus on inter-
esting and important questions and cautioned when I wanted to make it too complicated.
My coauthors at the University of Basel with whom I had the pleasure – and it really
was a pleasure – of collaborating helped out whenever I was in need and provided not only
a cheerful work environment, but also shared much of their iTunes libraries to make sure I
was listening to music instead of thinking aloud all the time: Thank you very much Peter
Schmutz, Stefan Pauwels, Alexandre Tuch, and Christian Hübscher.
Dr. Gudrun Moeller has read and commented on the whole text and helped me shape
it into an acceptable form, assuring me that it was reasonable to assume that an end existed
even for this enterprise.
My late mother, Erika Leuthold, who, as a single mother, did without much for
herself so I could study theoretical physics at the university, and my late grandmother,
Emma Leuthold, always had an open door (and lots of food) when I needed a place to get
back in touch with what really counts. I am sad beyond words, and I regret finishing this
thesis only after both of them passed away this summer.
My grandfather, Hans Ruedi Leuthold, who was the best role model a child could
wish for, taught me most of the few things I know to do by hand instead of head, and made
me never miss my father.
Finally, the partners at my company Stimmt AG, Lukas Karrer, Glenn Oberholzer,
and Marc Sturzenegger, who – besides being glad that I was away from office at least
sometimes – gave me some paid time off to conduct research and write my papers, because
they believe in the value of conducting solid research and trying to find business applications.
THREE PERSPECTIVES ON WORLD WIDE WEB NAVIGATION 32

Appendix
 ARTICLE IN PRESS

Int. J. Human-Computer Studies 66 (2008) 257–270

www.elsevier.com/locate/ijhcs

Beyond web content accessibility guidelines: Design of enhanced text

user interfaces for blind internet users
Stefan Leuthold, Javier A. Bargas-Avila!, Klaus Opwis
Department of Psychology, University of Basel, Missionsstrasse 60/62a, CH-4055 Basel, Switzerland
Received 26 January 2007; received in revised form 17 October 2007; accepted 17 October 2007
Communicated by P. Pendharkar
Available online 22 October 2007

Abstract

Websites do not become usable just because their content is accessible. For people who are blind, the application of the W3C’s Web
Content Accessibility Guidelines (WCAG) often might not even make a significant difference in terms of efficiency, errors or satisfaction
in website usage. This paper documents the development of nine guidelines to construct an enhanced text user interface (ETI) as an
alternative to the graphical user interface (GUI). An experimental design with 39 blind participants executing a search and a navigation
task on a website showed that with the ETI, blind users executed the search task significantly faster, committing fewer mistakes, rating it
significantly better on subjective scales as well as when compared to the GUIs from other websites they had visited. However,
performance did not improve with the ETI on the navigation task, the main reason presumed to be labeling problems. We conclude that
the ETI is an improvement over the GUI, but that it cannot help in overcoming one major weakness of most websites: If users do not
understand navigation labels, even the best user interface cannot help them navigate.
r 2007 Elsevier Ltd. All rights reserved.

Keywords: Accessibility; Blind users; Graphical user interface; Interface for the blind; Screenreader; Text interface; Usability for the blind; User interface;
WAI; WCAG

1. Introduction tion and transaction services are to be made accessible for

visually impaired people. In this context, websites com-
The graphical user interface (GUI) is the most wide- pliant to a certain standard of accessibility are referred to
spread user frontend for applications today and the as barrier-free-websites (BehiG, 2002; BehiV, 2003).
dominant user interface for websites on the internet. But To bolster laws like Section 508, the World Wide Web
graphical elements like windows and buttons are designed Consortium’s Web Accessibility Initiative (WAI) devel-
for sighted users; blind people can neither perceive nor use oped guidelines known as the Web Content Accessibility
them (Kieninger, 1996). To compensate for this disadvan- Guidelines (WCAG, 1999). Application of these guidelines
tage, several countries have passed laws to enforce ensures that HTML code is readable by screenreader
accessibility of websites for handicapped users. In the software like JAWS, supporting handicapped users to
United States, these laws are being published as Section access the website. Nowadays, the WCAG are the de-facto
508 of the Rehabilitation Act (Section 508, 1998), which standard when it comes to accessibility of websites. The
prohibits discrimination against people with disabilities in above-mentioned Swiss act explicitly refers to them.
all aspects of daily life, including education, work and Currently, the WCAG are mainly a result of ideas and
access to public buildings. Since 2004, a Swiss act has discussions in the WAI working group, since by nature,
required the government to provide access to all internet working toward standards is different from empirical
services for people with disabilities. Namely, communica- research. An overview of guidelines, standards and style
guides for human–computer interaction (HCI) is provided
!Corresponding author. Tel.: +41 61 2673522; fax: +41 61 2670632. by Stewart and Travis (2003). No empirical studies are
E-mail address: [email protected] (J.A. Bargas-Avila). referenced on the WAI website that would support the

1071-5819/$ - see front matter r 2007 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ijhcs.2007.10.006
 ARTICLE IN PRESS
258 S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270

WCAG’s normative character, demonstrating positive because it relies only on text and not on graphical
impact on user behavior. Whereas HCI research has representations of content and navigation items. ETI
yielded many sets of user interface design guidelines can be regarded as an extension of the WCAG insofar
(Nielsen, 1994; Shneiderman and Plaisant, 2004), to date, as the guidelines can be associated to the last 3 of the
it has solely focused on accessibility and technological aids 14 WCAG, explicating them for blind internet users.
for perceptually impaired people (cf. Section 2.1). Thus, it could be a first step toward an exhaustive set of
The vast majority of the WCAG contain checkpoints empirically researched guidelines that may be used to ensure
and core techniques to improve the programming of usability, and not only accessibility, of user interfaces for
websites to render them accessible for all users and all blind users.
devices, and only the three guidelines stated in Table 1
contain advice for user interface design in the sense of 2. Theoretical background
conceptual design, comparable to Shneiderman’s and
Nielsen’s work cited above. This illustrates that, regarding This section starts with a brief overview of relevant
websites, the situation for blind and visually impaired users research in HCI regarding accessibility and the conception
resembles the situation for sighted users in the mid-1990s: of guidelines for user interface design. It then summarizes
Content on websites is often accessible, but not conse- current research which shows that, as of today, the WCAG
quently usable. are rarely used, and if they are used, their impact is not as
According to Nielsen (1993), a website is usable if it originally intended. The end of the section contains a short
satisfies the five criteria shown in Table 2. This definition is overview of approaches that go beyond the WCAG to
context-independent. It needs to be conceptualized into support blind or visually impaired people.
concrete, applicable guidelines on the user interface level to
be of practical value in real design contexts. This has been 2.1. Relevant HCI research regarding accessibility and
demonstrated for sighted users by Agarwal and Venkatesh guidelines for user interface design
(2002) or Venkatesh and Agarwal (2006), as they were able
to determine the success of e-commerce websites from their Both Jacko’s and Brewster’s summary of recent HCI
empirically researched set of the Microsoft Usability research make it clear that most researchers dealing with
Guidelines (Keeker, 1997). perceptually impaired people have devoted their time to
This research sheds light on what influences usability investigate the accessibility of information and/or func-
from a blind person’s point of view. It also shows how a tionality via computers in order to enable or enhance usage
new set of guidelines leads to a new kind of user interface, for diverse user groups (Brewster, 2003; Jacko et al., 2003).
that will be called enhanced text user interface (ETI), However, they have not focused on usability or user
interface guidelines, leaving a theoretical gap to address.
Edwards’ exemplary work on the Soundtrack user interface
illustrates this point: The aim of the research was to adapt
Table 1 a mouse-based interface into an auditory form, i.e. make it
The three non-technical WCAG
accessible for blind users, and not to conceive user interface
No. Guideline guidelines for the blind (Edwards, 1989).
Many researchers have contributed to the development
12 Provide context and orientation information of user interface guidelines for sighted users (see e.g.
13 Provide clear navigation mechanisms
14 Ensure that documents are clear and simple
Nielsen, 1994; Shneiderman and Plaisant, 2004), and the
International Standards Organization (ISO) has published
Numbering corresponds to numbering of WCAG (WCAG, 1999). a vast body of work on them as well (as described in
Stewart and Travis, 2003).

Table 2 2.2. WCAG are rarely used

Nielsen’s five usability criteria
The WCAG certainly do lead in the right direction.
Factor Explanation
However, Sullivan and Matson (2000) found that if content
Efficiency Users accomplish their tasks quickly and without accessibility is defined in a continuous, rather than
much cognitive effort dichotomous manner, 29 of 50 of the Web’s most popular
Errors Users commit only few errors and are able to
sites can still be classified as inaccessible. Klein et al. (2003)
recover quickly
Satisfaction Users are satisfied with how the website works examined 157 websites of public high schools in Iowa with
Memorability Returning users do not have to relearn the use of different methods, including Bobby, an automated engine
navigation items and functionalities for checking WCAG compliance. They found that 94.3%
Learnability Novice users of a website get productive quickly, of these pages did not pass the Bobby priority 1 check (e.g.
finding information and using the website’s
provide alternative text for all images) and 98.1% did not
functionality
pass priority 2 check (e.g. do not use fixed font size). These
 ARTICLE IN PRESS
S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270 259

studies indicate that although the WCAG have existed 2.4. Beyond WCAG
since 1999 and there are corresponding laws demanding
their application, there have been few consequences or Why do blind people still have many problems even if a
advancements in this field. Hackett et al. (2004) even page is fully WCAG-compliant? We believe that this is
found that the accessibility problems seem to have partially due to the GUI itself. This interface, even if
worsened in the last few years. They used the Internet tweaked, still remains an interface for sighted users. The
Archive’s Wayback Machine to analyze the accessibility of graphical interface elements are designed to serve as visual
random websites from 1997 to 2002 and found that input.
websites became progressively inaccessible, whereas
their complexity increased over the years. Lazar et al. 2.4.1. Crossmodal output
(2004) found, in a survey conducted with 175 webmasters, One way of solving this problem is enhancing the visual
that 73.7% of respondents were familiar with the Section elements of the existing interface with crossmodal output
508 laws and that 56% thought that their website aimed at the blind (audio or haptic cues). Hardwick et al.
was accessible. Most respondents (78.9%) were also (1998) developed a browser capable of generating force-
aware that there are software tools to check accessibility. feedback from a VRML file. Using this browser it is
These results suggest that missing knowledge is not the possible to use haptic displays to communicate three-
main reason for the lack of development of accessible dimensional images to blind users. Yu et al. (2002)
websites. developed web-based multimodal graphs designed for
One problem area could be financial aspects. Developing visually impaired and blind people. The information in
an accessible website requires a substantial amount of the graph is conveyed through haptic and audio channels.
work. In consideration of the relatively small number Unfortunately, these ways of enhancing the interface are
of handicapped users, decision makers could often opt often tied to special hardware and require a considerable
against implementation of WCAG. Richards and Hanson amount of work on the website. Additionally the main bulk
(2004) addressed this problem. They claim that most of the visual interface is not altered, so many problems
approaches to the implementation of accessibility stan- remain.
dards are based on unrealistic economic models in which
web content developers spend too much and receive too 2.4.2. Dual user-interface paradigm
little. This is due to the narrow definitions given to both Another way of addressing the problems beyond the
those who benefit from enhanced access to websites and WCAG goes back to Savidis and Stephanidis (1998),
what is meant by ‘‘enhanced access’’. To the knowledge of proposing that blind and sighted users need different
the authors there is currently no further research in this interfaces (dual user-interface paradigm). We believe that
direction. this idea deserves further consideration. If blind people
navigate differently, they need another kind of interface.
2.3. WCAG are not enough GUI elements have no value for them. They only generate
audio-clutter on the screenreader output. This leads to the
Compared to sighted users, navigation behavior of blind idea of altering the nature of the user interface instead of
people is completely different. Anyone who has ever just tweaking the visual elements. This would provide the
witnessed a blind person surfing a web page with a blind with a stripped-down and clean interface containing
screenreader and a braille terminal has noticed instantly the same information, but structured in a different way.
that the blind person’s strategies of accessing an informa-
tion space cannot be compared to visual navigation 2.4.3. Extension of screenreader capabilities
(Savidis and Stephanidis, 1998). Web page content is being described with HTML code
However, the WCAG show how to tweak the visual which is rendered into a user interface. All user interface
interface to make web page content accessible for elements correspond to HTML elements. Both sighted as
handicapped users. For users with visual impairments this well as blind users must be able to perceive the same
may be sufficient, but for blind people it is not enough. navigation and content elements. But, whereas a sighted
Petrie et al. (2004) showed that of all impairment groups, person scans the layout of a website, using visual cues to
surfing the web is most difficult for the blind. Only 53% of focus on interesting elements, blind and visually impaired
the blind succeeded in their tasks, while 75% of partially users have to deal with a screenreader’s linear narration of
sighted, 83% of dyslexic and 85% of physically impaired all HTML elements: A screenreader cannot know whether
and hearing impaired succeeded. In their test with 51 an element is simply used for layout purposes and could be
disabled users, they collected 585 problems of which only left out, it just reads aloud all objects encountered in the
55% were related to WCAG checkpoints; 45% could be exact sequence they are contained in the HTML code. Thus
present on any WCAG-conforming site. Due to this kind it often happens that blind users need to extract structure,
of phenomenon, Powlik and Karshmer (2002) noted that navigation and content from a quite chaotic audio babble,
making a resource accessible does not necessarily make it even if the page is fully compliant with the WCAG. This
usable. problem was recognized by Fukuda et al. (2005). They
 ARTICLE IN PRESS
260 S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270

claimed that the traditional way of evaluating accessibility navigation options on the left. Blind users often could
by automated tools does not lead to usable websites for the not make a connection between the highlighted ‘‘News’’-
blind. To solve this problem they proposed that important tab and the corresponding second-level-navigation on
elements of navigation and structure as well as factors the left.
contributing to readability must be included in the ! Blind users often only learn by chance which navigation
accessibility measurement. Their evaluation measures are options are recurrent on every page. Again, consider the
navigability (how well structured the web content is) and tab navigation mentioned in the first example. The
listenability (how appropriate the texts are). ‘‘News’’-tab was always shown on top of every page,
This structural problem was addressed in several studies although users could not make sense of it. In the
(Donker et al., 2002; Pontelli et al., 2004). Ramakrishnan worst case, it took a user almost 30 min to execute a
et al. (2004) developed HearSay. This screenreader is based task, and when he was near completion, he mistook the
on a novel approach: It automatically partitions the web ‘‘News’’-tab for a navigation option in the subnaviga-
document through tightly coupled structural and semantic tion which brought him back to the homepage where he
analysis, which transforms raw HTML documents into had to start over.
semantic structures to facilitate audio browsing. In fact this ! It takes blind users a lot of time to explore navigation
method tries to mimic the screening behavior of sighted options: Whereas trial and error is a valid navigation
users, so the structure of the website is communicated to strategy for sighted users, we never observed blind users
the blind in a more efficient way. The work of Ramak- applying this strategy. It seems to be too much effort to
rishnan et al. seems very promising. Unfortunately the navigate back and forth between different web pages
evaluation they present is still on a very rudimentary level, just for the sake of exploration.
tested by only five blind users.
Most approaches of developing a new user interface for Thus, theories of navigation and qualitative insights
blind users pursue promising ideas, but lack quantitative from our exploratory studies demonstrate the necessity of a
empirical evaluation of the benefits for the blind. In this user interface that is enhanced regarding navigation
study we will provide a direct quantitative comparison through and interpretation of the linear representation of
between two interface types. content, using user interface elements especially designed
for the blind. The next section shows how such an interface
3. Necessity of an enhanced user interface could be conceived.

In order to successfully use a navigation, users first have 4. Goals, guidelines and requirements for the ETI
to form a mental model of the underlying navigation space.
In order to form a mental model, users have to make sense 4.1. Introduction and construction of the ETI
of the grouping and labeling of navigation items. This
sense-making is based on cognitive processes coupled to The GUI has been introduced to make use of human
sensory input. Many models follow this train of thought perceptual abilities in order to reduce demands on working
(e.g. Spence’s framework for navigation: Spence, 1999). memory: Instead of learning hundreds of commands by
The sensory input leading to mental model formation is heart, users could see all available commands for a selected
completely different for sighted and for blind users: For object and reserve their cognitive resources mainly for
sighted users, visual grouping and non-audible attributes decision making. The only reason sighted users can
(such as text size, color and formatting) yield a great deal instantly use a computer program or website without
of insight into the intended grouping of navigation items training is their ability to see data objects represented on
and hence communicate the intended structure of the the screen together with the corresponding commands.
navigation space. Blind users have to form their mental Blind users need representations that enable them to
model of the navigation space based solely on the linear hear or feel data objects and available commands.
representation of navigation items and audible cues added Preferably, an enhanced user interface for the blind would
to visually represented content. make use of auditive cues to communicate structure, since
Some qualitative insights and examples from exploratory Lee (2004) showed that only 10% of the blind population
studies we conducted explain blind users’ problems with a uses Braille.
GUI: Our proposal therefore, is to use a text-only interface in
order to free blind users from having to listen to auditive
! Blind users cannot guess relationships between primary clutter from visual user interface elements, and impose a
and secondary navigation items if they are expressed structure consisting of auditive cues. For the construction
visually. The website used in the first exploratory studies of this text-only user interface, we surveyed existing HCI
had a tab navigation on top with navigation items research about guidelines (cf. Section 2.1) as well as user
‘‘News’’, ‘‘Weather’’, ‘‘Stock Exchange’’, ‘‘Money’’, etc. interface approaches beyond the WCAG (cf. Section 2.4).
The ‘‘News’’-tab was highlighted, and there was a We then decided to follow the dual user interface paradigm
subnavigation consisting of five groups with a total of 26 (cf. Section 2.4.2), focusing on the application of two of
 ARTICLE IN PRESS
S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270 261

Shneiderman and Plaisant’s Eight Golden Rules of Interface insights from exploratory studies and interviews with blind
Design (Shneiderman and Plaisant, 2004): ‘‘power users’’ (referred to in Section 3). In building these
guidelines we followed the approach for human-centered
! Strive for consistency. This guideline applies to labeling,
design laid out in ISO 13407 (as explained in Stewart and
order and effects of user interface elements.
Travis, 2003): Users were actively involved in interface
! Reduce short-term memory load. At any given moment,
conception; the two guidelines mentioned above were used
users should not be required to handle more than
to allocate function according to proper use of human skill;
Miller’s famous seven plus/minus two chunks of
and the design solutions were iterated before the experi-
information (Miller, 1956) in order to successfully use
ments started.
the interface.
An overview of all nine guidelines is provided in Table 4.
The first group of guidelines deals with context and
We focused on these two guidelines because we believed
orientation information (WCAG no. 12 in Table 1). The
them to be a reasonable starting point considering the
second group addresses navigation (WCAG no. 13 in
effort of programming experimental test settings and
Table 1), and the last guideline is included to keep the user
conducting experiments with dozens of blind internet
interface as clear and simple as possible (WCAG no. 14 in
users. Furthermore, Norman’s (Norman, 2001) emphasis
Table 1).
of mental models as the source of errors has also led us to
pay special attention to consistency, as only the consistent (1) Communicate menu structure through numbering
use of user interface elements enables users to make Scope: Every menu item is numbered, and the total
successful predictions of interface behavior, based on the number of menu items is announced before the menu.
inspection of their mental models. Rationale: We assume that blind users navigate
We call the resulting text-only user interface the ETI with higher levels of certainty and efficiency if they
because it contains enhancements which should make it can use working memory for either of two possible
superior to a regular text-only user interface that conforms strategies:
to the WCAG. ! In the case of navigation menus with many items, they
know they must focus their attention and remember
4.2. Goals of the ETI only a few items to come back to after they have
aurally scanned the whole menu. This navigation
If the ETI supports the usability factors in Table 2, our strategy should lead to a one-by-one evaluation of
goals with the ETI can be directly related to those factors: navigation items: Users are likely to follow one of the
It should take blind users less time to complete tasks, they first navigation items that seems to fit their goal. Of
should commit fewer mistakes and show greater satisfac-
tion surfing the website. These goals will be the criteria for
the empirical evaluation of the interface, stated in our Table 4
ETI guidelines overview
hypotheses in Section 6. Table 3 shows which guidelines are
expected to affect which of Nielsen’s usability factors. The No. Guideline Basis (cf. Section 4.1)
reason for omitting memorability and learnability is
explained in Section 6. Guidelines concerning context and orientation information, WCAG no. 12
1 Communicate menu structure through Working memory
numbering
4.3. ETI guidelines 2 Label all user interface elements Mental models
3 Place buttons after options in forms Qualitativea
This set of nine guidelines was constructed based on the 4 Do not use unnecessary words to create Qualitativea, working
context memory
two guidelines stated in Section 4.1, as well as qualitative
Guidelines concerning navigation, WCAG no. 13
Table 3 5 Frame every page with the same Consistency, mental
Usability factors and ETI guidelines elements models
6 Add navigation menu on all pages, Consistency, mental
Efficiency Errors Satisfaction Memorability Learnability except pages at the end of the page models
hierarchy
a
Guideline 1 Yes Yes Yes Yes Yes 7 Place generic navigation and Consistency
Guideline 2 No Yes No Yes Yes continuative links at the bottom of the
Guideline 3 No Yes No No Yes page
Guideline 4 Yes No Yes No No 8 Place search on top of the homepage to Qualitativea,
Guideline 5 Yes Yes Yes Yes Yes facilitate task initiation consistency
Guideline 6 Yes Yes No No No
Guideline 7 Yes No Yes No No Guidelines concerning clarity and simplicity, WCAG no. 14
Guideline 8 Yes Yes Yes Yes Yes 9 Eliminate all visual elements used Qualitativea, working
Guideline 9 Yes Yes Yes Yes Yes solely for layout and branding memory
a a
Yes: respective guideline expected to affect the corresponding usability Based on qualitative insights from our exploratory studies with blind
factor, No: expected not to. users.
 ARTICLE IN PRESS
262 S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270

course, there is only need to remember more than one continuing after the button in order to find them.
navigation option because labels are ambiguous. If no This guideline is based on empirical observations in
navigation item could be mistaken to be an option our exploratory studies and not derived from the
except for the navigation item really leading to the guidelines in Section 4.1.
desired content, users could always rely on a simple (4) Do not use unnecessary words to create context
one-by-one evaluation of navigation options until the Scope: Context must not explain everything expli-
right navigation item is found. citly, but contain hints to form a mental model of the
! In the case of navigation menus with few items, they navigation space.
can listen to the whole menu without restricting their Rationale: As a web page loads in the browser
attention to single items, because they can remember window, sighted users determine whether they have
all navigation items. The second strategy should navigated to the right page by looking at cues like
result in users’ taking the time to get an overview of the page title. Our first set of ETI guidelines for the
all navigation options first, which would be desirable exploratory studies added auditive clutter with the
according to many guidelines (see e.g. Shneiderman intention of creating context: The user interface read
and Plaisant, 2004) because users then feel more in location labels in plain German and announced as well
control over the user interface. as explained all page elements. For example, at the
beginning of each page, the screenreader read: ‘‘You
This guideline is based on the body of research around are on page [path].’’ Although users welcomed the idea
human information processing (Card et al., 1983; of labeling every object to create context (cf. Guideline
Byrne, 2003), and uses the short-term memory guide- 2), they wanted it to be shorter: Enough context to
line referred to in Section 4.1: Users have to compare decide what to do and form a mental model, but not
their current navigation goal to navigation items in more.
their working memory and decide how to behave. This This guideline can be viewed as an explication of
is a tedious task if the number of possible navigation WCAG no. 14 (cf. Table 1) and can thus be associated
items exceeds the working memory span. with Shneiderman and Plaisant’s working memory
(2) Label all user interface elements guideline (cf. Section 4.1).
Scope: Examples for user interface elements include (5) Frame every page with the same elements
grouping, menus, links and images. Scope: Always clearly indicate the beginning and end
Rationale: For blind users to be able to form a mental of a web page. Order and format menus similarly on all
model, they need auditive cues to understand what the pages, so that they sound alike. Include page title, but
interface consists of and what relationships exist between do not display path to page.
interface elements. Similarly, users with sight use visual Rationale: As visual structures of different pages
hierarchy, nearness and other principles from gestalt within a website should look alike, auditory structures
theory as a starting point to form relationships between of different pages should sound alike so that blind users
elements (see e.g. Palmer, 2002). Of course, for trivial may rely on a single overarching structure to direct
cases like link labels, the label itself becomes even more their attention to appropriate areas of a page. In the
important: Blind users often use a function which exploratory studies, users indicated that they do not
displays all links contained on the page they are on. If need to hear the whole path to the page they are on
link labels are not self-explanatory, they cannot know since a good page title, in most cases, renders that
where a link leads (e.g. any link labeled ‘‘more’’). information useless (cf. Guideline 4). Instead, they
This guideline is based on research about mental wanted an indication of where exactly a page started
models (Norman, 2001; Yoshikawa, 2003). and ended so they could verify that they began certain
(3) Place buttons after options in forms tasks at the beginning of a page and ended their search
Scope: Place all form options after the label of the for certain information or functionality at the end of
form but before the submit button, and place the link the page.
to extended search before the submit button. This guideline is supported by the consistency
Rationale: In many GUIs, designers use an input principle stated in Section 4.1 and based on research
field for text entry, place the submit button next to it about mental models (Norman, 2001).
and put additional options below the button. This (6) Add navigation menu on all pages, except pages at the
serves the need of most sighted users, who do not end of the page hierarchy
require options like language or document type Scope: Make a website accessible like a file naviga-
restrictions to be included in their search. As long as tion system, stating all options to move further down
these options are visible somewhere near the locus of the hierarchy and provide a back function to go one
attention (Raskin, 2000), users are able to find and use level up in the hierarchy.
them. Blind users, on the other hand, can only guess Rationale: Some blind users have been observed to
whether there are further options after the submit load the appropriate web page for their task, but
button, and would have to explore the area aurally, instead of looking for the page’s content, they followed
 ARTICLE IN PRESS
S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270 263

a link in the page’s navigation menu and never even (9) Eliminate all visual elements used solely for layout and
noticed there was content. The proposed guideline branding
prevents users from following links if they are at an end Scope: Blind users’ working memory should not be
node of the page hierarchy. overloaded with audio clutter.
This guideline is based on the consistency principle Rationale: Considering that blind users are not able
stated in Section 4.1 and research about mental models to process visual cues, the enhanced user interface for
(Norman, 2001). the blind should not contain any elements that pollute
(7) Place generic navigation and continuative links at the the screenreader’s linear audio reproduction of the
bottom of the page content. Audio clutter stems either from structural
Scope: Place all generic navigation items as well as layout elements (such as those used to enforce exact
continuative links that are believed to help users spacing between layout elements) or from elements that
continue browsing after they are done with a page, at are used for branding purposes. Attention should be
the bottom of the respective page. paid to not leaving away too many elements since, in
Rationale: On most websites, generic navigation the exploratory studies, blind users indicated they did
links (e.g. help, contact) are placed somewhere in the not want to be patronized and feared getting incom-
top corners. For sighted users, scanning the page and plete information (cf. Section 4.4.1).
learning that they just have to look in the corners to This guideline is based on the fact that graphical
find those items is enough to work with it. However, for elements for branding are designed for sighted users;
blind users a screenreader reads these elements aloud blind people can neither perceive nor use them
every time they are being encountered. Therefore, by (Kieninger, 1996). Therefore, the audio output should
placing them somewhere else than at the bottom of the not be polluted with useless information stemming
page means there will always be content on a web page from these elements, thus supporting the working
that is separated by these links. Users expect generic memory guideline (cf. Section 4.1).
navigation items to exist and they will most certainly
search for them if they need them, e.g. blind users can
use the find-functionality and type in ‘‘help’’ to find the 4.4. Requirements for reasons of feasibility
help-link.
This guideline is based on the consistency principle Advocating the need for an alternative interface like the
(cf. Section 4.1) and on research about design and ETI is a sensitive issue. Blind users could fear they are not
layout of different types of navigation as reported in getting complete information because some HTML ele-
Rosenfeld and Morville (2006). ments are no longer included in the ETI. Decision makers
(8) Place search on top of the homepage to facilitate task could fear high implementation and maintenance costs in
initiation building not only a GUI, but also an ETI in the future.
Scope: Search should be easily accessible and there- Hence the ETI has to fulfil two requirements to be feasible,
fore be placed at the top of the homepage, but can be as explained in the next two sections.
left out on every subsequent page.
Rationale: Because tasks with a website can be 4.4.1. Completeness of information
divided into navigation and search tasks, it seems In the exploratory studies we conducted to develop the
reasonable to design the ETI in a way to facilitate the ETI, blind users expressed severe concerns: They were
initiation of either task type. Search should be placed worried about being discriminated against with a
on top of the homepage to be perceived as being of ‘‘crippled’’ interface and that they would be deprived of
equal value to main navigation. To reduce audio vital information. One user reported asking sighted
output to a minimum, however, search should not be colleagues to describe images discovered in the content,
included in subsequent pages: If it were placed on top advising us not to leave images away simply because he
of every page, it would be rather annoying, if it was at could not perceive them. If we provide blind users with an
the bottom of every page, it might as well only be on alternative interface that suits their navigation behavior
the homepage. Users in the exploratory studies did not better, it is crucial that this interface contains the same
indicate problems with this. information as the visual interface.
This guideline is based on the consistency principle
(cf. Section 4.1), although we recommend not to include 4.4.2. Implementation with a content management system
search on every page: Simply repeating user interface The population of blind internet users is very small and
elements on every page of a website is not, per se, of little economic importance for most companies. Thus,
helping users achieve their tasks. Making the very available funding to make websites accessible is often
prominent search functionality accessible only on the limited. If we want an ETI for blind users to have any
homepage is consistent and avoids search functionality chance of success, the implementation has to be simple and
in locations where users are focused on navigation must not require a huge amount of manual work. Most
tasks. websites of bigger companies use powerful content
 ARTICLE IN PRESS
264 S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270

management systems (CMS) that allow usage of different 5.3. Procedure

standard templates. These templates define the location of
navigation and content as well as layout aspects like font The experiments were conducted in the Department of
types and colors. To make the implementation of the ETI Psychology’s usability laboratory in Basel, Switzerland.
feasible, a CMS must be able to generate it automatically. The websites were stored on a local server and browsed
The authors believe this to be possible if the guidelines with Internet Explorer 6.0. Blind participants used the
from Section 4.3 are integrated in the CMS. screenreader JAWS 5.1 and if required a braille terminal
(ALVA 544 Satellite). The braille terminal was not
5. Experimental design restricted because only few people really use it, and
those who do use it mainly for verification purposes
5.1. Objects of evaluation and use the screenreader output all the same. All audio and
video data from the sessions were digitally recorded for
The experiment was conducted with the current website later analysis.
of the University of Basel (http://www.unibas.ch). This The blind participants were first introduced to the test
website contains the usual information about the Uni- setting by the supervisor. They had the opportunity to get
versity, its departments and personnel. At the time of accustomed to the test setting and adjust the screenreader
evaluation, the website was partly compliant with the to their needs with a short practice task. After a short pre-
WCAG. To avoid changes to the website during the test questionnaire (with some questions regarding their
experiment, a copy of the relevant eight pages necessary to blindness to ensure that the necessary criteria were met, see
conduct the tasks was used. Section 5.2), half the users started with two main tasks
Three interfaces were examined as explained in Table 5 using a GUI (Unibas or WCAG): First, they had to find a
(see Figs. 1 and 2). specific person working in the library (search task). For
their second task, they had to locate the address of the
5.2. Participants University’s social counsel (navigation task). They then
repeated the same two tasks with the ETI. After each task
This study was conducted with 39 blind participants. execution, they completed a subjective rating of the
They were recruited with the help of organizations for the interface. The other half of the users started with the
blind and corresponding newsletters. To participate in this ETI, using a GUI afterwards. To avoid learning artefacts
study, they had to be considered totally blind (less than 2% the interface order was randomized (cf. Fig. 3). The
remaining vision), and made use of Freedom Scientific’s experimental session ended by having them fill out the
screenreader JAWS to surf the Internet for at least 1 year. post-test questionnaire (some demographic questions).
The average age was 36 years (SD ¼ 14:72; Range: 18–64)
and the gender distribution was 72% male and 28% 5.4. Tasks and data collected
female.
We focused on three areas:
Task performance: Concerning task performance, certain
users were not able to complete some of the tasks within
Table 5
User interfaces examined in this study
15 min. Their performance on uncompleted tasks was
coded with 900 s, because hypotheses were not helped by
Name Explanation cutting times short and findings would still be valid.
Average number of errors: As to the errors committed, 13
Interface 1 Unibas The current website’s original interface. The
code was not fully WCAG compliant, but it different errors were observed during task execution. The
worked flawlessly with the screenreader eight errors in Table 6 occurred more than once and have
Interface 2 WCAG This interface looked exactly like Unibas, but been included in the statistical analysis.
it contained all necessary changes to reach full Subjective evaluation of the user interface: Three satisfac-
compliance with the WCAG:
tion measures, ‘‘pleasant’’, ‘‘comparison’’ and ‘‘speed’’,
(1) Extension of TITLE-tag to provide a
unique page title for every unique page were collected. Due to time constraints we decided against
(2) Language attribute ‘‘LANG ¼ de’’ complex satisfaction scales (Brooke, 1996) and used a
added simple question for each measure:
(3) Re-labeled ‘‘more’’-links with
distinguishable names
(1) Pleasant: ‘‘How pleasant was it to surf on the
(4) Abbreviations like ‘‘Eng’’ replaced with
whole sentence like ‘‘This page in English’’ University of Basel website?’’ (Six-point Likert scale;
(5) LABEL-tags added within forms 1 ¼ very unpleasant, 6 ¼ very pleasant).
Interface 3 ETI The alternative interface, designed according (2) Comparison: ‘‘How do you rate the handling of this
to the ETI guidelines stated in Section 4.3. website in comparison to other sites you normally
The HTML code for the eight pages used was
visit?’’ (Six-point Likert scale; 1 ¼ a lot worse,
written from scratch
6 ¼ a lot better).
 ARTICLE IN PRESS
S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270 265

Fig. 1. Homepage of WCAG-compliant website used in this experiment.

6. Hypotheses

The aim of this research is to develop and validate

guidelines to enable construction of ETIs. Our guidelines
(cf. Section 4.3) are based on the assumption that
accessibility is not enough for the blind. They should
lead to significant improvements in all of Nielsen’s
usability factors (cf. Table 2). This leads to the following
hypotheses:
H1 : Efficiency. Tasks are completed faster with the ETI
than with the GUIs.
H2 : Errors. Fewer errors are committed using the ETI
than using the GUIs.
H3 : Satisfaction. Users are more satisfied with the ETI
than with the GUIs.
There are no hypotheses for Nielsen’s factors four and
five, memorability and learnability, because long-term
memory tests were not conducted, and participants took
part in only one test. Those factors will have to be
addressed in future research projects.

7. Results

Fig. 2. Homepage of ETI website used in this experiment. 7.1. No difference between Unibas und WCAG

We could not detect a difference between the interfaces

(3) Speed: ‘‘How would you describe the speed with which Unibas and WCAG (cf. Section 5.1) with regard to task
you worked on this task?’’ (Six-point Likert scale; completion time, number of errors committed and sub-
1 ¼ very slow, 6 ¼ very fast). jective evaluations, as shown in Table 7.
This is mainly caused through our object of evaluation:
These questions were asked after each interface used. Six- The differences between the GUIs Unibas and WCAG, as
point Likert scales were chosen to ensure familiarity: In described in Table 5, stem from only a few modifications.
Switzerland the school grades use a six-point scale. Because no differences were found, the data for Unibas and
 ARTICLE IN PRESS
266 S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270

Unibas ETI
n=10

n=10 WCAG ETI

Blind users
Pre-Test Post-Test
(n=39)
n=9 ETI Unibas

n=10

ETI WCAG

Fig. 3. Procedure.

Table 6 7.2.2. Errors

Eight most frequently observed errors Analysis of the eight most frequent errors (cf. Table 6)
No. Name Description
yields the same pattern as the efficiency analysis: Regarding
the search task, users committed significantly fewer errors
1 Search button Users do not find or click the search button, with the ETI in comparison to the GUI, but for the
respectively. They click a wrong user navigation task, there was no difference (cf. Table 9).
interface element (e.g. question mark
standing for ‘‘help’’)
2 Labeling Users follow wrong links, either because 7.2.3. Satisfaction
they do not understand the link label, or Users prefer the ETI significantly more than the GUI
because they explore one link after another and rate the ETI to be significantly better than other
3 Extended Extended search has three options: website websites they visit (cf. Table 10). However, users did not
search search, Google search and people search.
rate their working on the tasks to be significantly faster
Because people search is third, many users
mistakenly searched the website or Google with the ETI.
4 Search option Users are not able to adjust the search
option (from website search to people 8. Discussion
search)
5 Search results Users hear an empty search input box first
8.1. Summary and interpretation of findings
page and start a new search (assuming their
search terms did not yield results), instead of
looking further down the page where search This experiment demonstrates that the WCAG are not
results were listed enough for blind internet users. From the nine guidelines,
6 Time Task could not be completed within 15 min only numbers 4, 5, and 7 are applicable also to sighted
7 Zero results Users unsuccessfully try to find the desired
web page with the search engine (wrong
users. The other six guidelines cover specific needs of
keywords) blind users. Thus, the ETI guidelines must be regarded as
8 Page structure Users find the same main navigation on an extension of the WCAG, producing a user interface
every page and are not sure whether they are more usable than the WCAG-compliant GUI, as our
on a new page or on the same page, because results show for three of Nielsen’s five usability criteria
there is no indication of where they are
in Table 2.

! Efficiency: The search task was completed significantly

WCAG were merged for all following calculations, faster with the ETI. That this is not the case for the
comparing the GUI to the ETI. navigation task is a strong indicator for the importance
As expected, no sequence effects were found in any of a solid information architecture (Rosenfeld and
variables (i.e. neither usage of Unibas nor usage of WCAG Morville, 2006) as a necessary precondition for any
caused users to perform differently on the ETI). navigation space: If people do not understand the labels
that are assigned to navigation menus and items, if titles
7.2. Usability factors and subtitles do not convey information meaningfully,
enabling users to form a mental model, the user interface
The subsequent sections state the results for the three itself cannot compensate for the loss of the resulting
examined usability factors. inefficiency.
! Errors: In the search task, blind users committed
7.2.1. Efficiency significantly fewer errors using the ETI. As with
It took blind users significantly less time to complete the efficiency, there was no difference for the navigation
search task with the ETI compared to the GUI, but there task—the main problem was presumed to be based on
was no difference for the navigation task (Table 8). incorrect mental models.
 ARTICLE IN PRESS
S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270 267

Table 7
Comparison of usability factors for blind users using Unibas and WCAG

Percentiles Ua pa

Unibas ðn ¼ 19Þ WCAG ðn ¼ 20Þ

P25 P50 P75 P25 P50 P75

Efficiency (s)
Search 314 518 900 394.5 808 900 140.5 p ¼ :166
Navigation 150 365 611 174 385 662 183.5 p ¼ :857

Errors (number of errors)

Search 1 2 3 1 2 3 175.5 p ¼ :687
Navigation 1 2 2 1 1 2 185.5 p ¼ :901
Satisfaction (rating on six point Likert-scale)
Pleasant 3 4 4 3 4 4.75 175.5 p ¼ :687
Comparison 3 4 5 3 4 4 172.0 p ¼ :627
Speed 2 3 4 2.25 3 4 157.5 p ¼ :365
a
Mann-Whitney test (data not normally distributed), p: two-tailed.

Table 8
Efficiency comparison between ETI and GUI

Percentiles Ta pa

ETI ðn ¼ 39Þ GUI ðn ¼ 39Þ

P25 P50 P75 P25 P50 P75

Efficiency (s)
Search 165 280 774 358 615 900 69 ðn ¼ 32Þ po:000
Navigation 101 323 876 150 365 611 289.5 ðn ¼ 34Þ p ¼ :891
a
Wilcoxon test (data not normally distributed), p: two-tailed.

Table 9
Error comparison between ETI and GUI

Percentiles Ta pa

ETI ðn ¼ 39Þ GUI ðn ¼ 39Þ

P25 P50 P75 P25 P50 P75

Errors
Search 0 1 2 1 2 3 79.0 ðn ¼ 31Þ po:001
Navigation 1 2 2 1 1 2 181 ðn ¼ 27Þ p ¼ :843
a
Wilcoxon test (data not normally distributed), p: two-tailed.

Table 10
Satisfaction measures with ETI and GUI

Percentiles Ta pa

ETI ðn ¼ 39Þ GUI ðn ¼ 39Þ

P25 P50 P75 P25 P50 P75

Satisfactionb
Pleasant 4 5 6 3 4 4 95.0 ðn ¼ 29Þ po:007
Comparison 4 5 6 3 4 5 93.5 ðn ¼ 28Þ po:011
Speed 3 4 5 3 4 4 183 ðn ¼ 33Þ p ¼ :075
a
Wilcoxon test (data not normally distributed), p: two-tailed.
b
Internal consistency: Cronbach a ¼ :8027.
 ARTICLE IN PRESS
268 S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270

! Satisfaction: Although one might assume that the navigation strategies: For blind users, trial and error is not
efficiency and error analysis would lead to users being a viable navigation strategy, because it takes them too
comparably satisfied with the ETI, because they could much time to return to the same point once they have taken
not navigate faster and committed the same amount of the wrong direction. However, sighted users can commit
errors in the navigation task, subjective measures show a errors in order to save cognitive resources. For them, it
huge difference between the ETI and the GUI. In fact, does not consume a large amount of time if they explore
users are more satisfied with the ETI. Aside from the the navigation by trial and error, instead of properly
significant difference in scores on an absolute scale, a evaluating all navigation options and choosing the one they
direct comparison shows that the majority of blind users think is suited best for the task. Further studies might show
assesses the ETI to be better than the GUI: 55% of 39 that due to this difference in navigation strategies, the ETI
blind users rate the ETI better than the GUI, 30% say needs to discern between navigation pages and content
that there is no difference and only 15% say the ETI is pages containing exclusively either navigation menus or
worse than the GUI. content but never both.
The information architecture of a website is even
These results also demonstrate that user interfaces for more important to blind than to sighted users, and this
blind users could be conceived using the same methods of is a crucial factor for successful navigation. Further
user centered engineering applied to developing user research needs to determine the size of the effect of
interfaces for sighted users, see e.g. Mayhew (1999). appropriate labeling of navigation menus and items.
Companies and institutions would be able to rely on There was no experimental condition with changed labels
worldwide standards for the design of user interfaces for for the GUI as well as for the ETI, but we suppose that
blind users that have been elaborated and tested using labeling was one of the main reasons that the navigation
proven methods of user interface design. task made no significant difference between the ETI and
the GUI.
8.2. Strengths and limitations of our research Our guidelines contribute to three of Nielsen’s usability
factors (cf. Table 2), leaving at least two open issues for
The obvious strength of our research is the application of further research to address:
the user centered design method according to the ISO
13407 standard: The final user interface used in this ! The two factors our hypotheses did not cover: Learn-
experiment was fashioned in several iterations, integrating ability and memorability. It would be important to
knowledge about HCI theory and guidelines with insights improve the ETI in order to study and improve blind
from exploratory studies (qualitative interviews, observa- users’ learning and usage of website functionality,
tions and usability lab tests with blind users). But the especially regarding novice users.
resulting ETI’s usability was not only researched qualita- ! Identification and exploration of all factors contributing
tively—unlike most usability lab studies that are conducted to usability for blind internet users. Our research
with only around 10 participants and thus cannot be used focused on guidelines for user interface elements only,
for statistical analysis—it was researched quantitatively and in the future, one could follow the approach of
with 39 blind users. Venkatesh and Agarwal (2006) and investigate which
The most serious limitation of this study arose from factors make blind users successful in terms of task
time constraints: Welcoming users to the usability lab, completion, which factors contribute to user satisfaction
letting them get acquainted with the test setting and and to what extent.
recording task execution of two tasks already took
well over an hour. We did not want users to work longer
Finally, variations and improvements to each guideline
during a single test session. We would have liked to
should be considered and researched empirically.
conduct the experiment with more than one search task
and more than one navigation task, and, ideally, we would
have tested the ETI guidelines with different user 8.4. Conclusion
interfaces, in order to understand the impact of completely
different content. Also, rather short tasks were used, We conclude that, for blind users, the ETI is more usable
because we wanted to make sure participants were able than the GUI, and our results indicate great potential
to complete them. beyond the WCAG for improving the Internet experience
of blind users. Effects from navigation labels should be
8.3. Implications for further research researched with high priority along with variations of the
nine guidelines for different website types. As long as
In the future, empirical studies should be conducted to governments and companies are unable to guarantee the
gain insights into navigation strategies and corresponding usability of their websites, they should think carefully
navigation guidelines to build new navigations for the before investing in the accessibility of the websites’ content
blind, because there is at least one crucial difference in and functionality.
 ARTICLE IN PRESS
S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270 269

Acknowledgments Human–Computer Interaction Handbook. Lawrence Erlbaum Associ-

ates, London, pp. 504–525.
Above all, we thank Ruedi Ruchti, who, himself blind, Keeker, K., 1997. Improving Web-site usability and appeal: Guidelines
compiled by MSN usability research. http://msdn.microsoft.com/
recruited all participants for the exploratory studies, library/default.asp?url=/library/en-us/dnsiteplan/html/improvingsiteusa.
demonstrating the necessity of this research. Tanja Schuler, asp.
in her function as head of the University of Basel’s Web Kieninger, T.G., 1996. The ‘‘growing up’’ of hyperbraille: an office
Office, funded the experiment, granting incentives and workspace for blind people. In: 9th Annual Symposium for User
transportation costs. Interface Software and Technology, ACM Association for Computing
Machinery, Seattle, Washington, USA.
We are very grateful for the support of ‘‘access for all’’, a Klein, D., Hansen, L., Michaelson, S., Blanck, P., 2003. Electronic doors
foundation in Zurich, Switzerland, which provided us with to education: study of high school website accessibility in iowa.
the hard- and software needed to conduct this experiment. Behavioral Sciences and the Law 21, 27–49.
We thank Milena Gassmann and Pascal Saladin for their Lazar, J., Dudley-Sponaugle, A., Greenidge, K.-D., 2004. Improving web
effort in planning and conducting the experiments and the accessibility: a study of webmaster perceptions. Computers in Human
Behavior 20, 269–288.
endless hours of video analysis and transcription needed Lee, S., 2004. Universal access to mobile computing by modular i/o
for our analysis of the data. Martin Rohner organized and design. In: Technology and Persons with Disabilities Conference 2004,
conducted the first series of exploratory studies and Los Angeles, USA.
contributed a lot to the development of the guidelines. Mayhew, D.J., 1999. The Usability Engineering Lifecycle: a Practitioner’s
Laura Wiles and Ardalan Tajalli helped us proofread this Handbook for User Interface Design. Academic Press, San Diego,
USA.
article. Miller, G.A., 1956. The magical number seven, plus or minus two: some
Web programming company ‘‘aseantic’’ programmed all limits on our capacity for processing information. The Psychological
needed variants of the website used in the exploratory Review 63, 81–97.
studies. Stefan Pauwels programmed all needed variants of Nielsen, J., 1993. Usability Engineering. Morgan Kaufmann, San Diego,
the website used in the actual experiment. USA.
Nielsen, J., 1994. Enhancing the explanatory power of usability heuristics.
In: Conference on Human Factors in Computing Systems, Boston,
References MA, USA, pp. 152–158.
Norman, D.A., 2001. The Design of Everyday Things, fourth ed. MIT
Agarwal, R., Venkatesh, V., 2002. Assessing a firm’s web presence: a Press, London.
heuristic evaluation procedure for the measurement of usability. Palmer, S.E., 2002. Vision Science Photons to Phenomenology, third ed.
Information Systems Research 13 (2), 168–186. MIT Press, Cambridge, MA, USA.
BehiG, 2002. Bundesgesetz über die Beseitigung von Benachteiligungen Petrie, H., Fraser, H., Neil, K., 2004. Tension, what tension? website
von Menschen mit Behinderungen (Behindertengleichstellungsgesetz). accessibility and visual design. In: International World Wide Web
hhttp://www.admin.ch/ch/d/sr/151_3/index.htmli. Conference, New York, USA.
BehiV, 2003. Verordnung vom 19. 2003 über die Beseitigung von Pontelli, E., Son, T.C., Kottapally, K., Ngo, C., Reddy, R., Gillan, D.,
Benachteiligungen von Menschen mit Behinderungen (Behindertengle- 2004. A system for automatic structure discovery and reasoning-
ichstellungsverordnung). hhttp://www.admin.ch/ch/d/sr/c151_31.htmli. based navigation of the web. Interacting with Computers 16,
Brewster, S., 2003. Nonspeech auditory output. In: Jacko, J.A., Sears, A. 451–475.
(Eds.), The Human–Computer Interaction Handbook. Lawrence Powlik, J.J., Karshmer, A.I., 2002. When accessibility meets usability.
Erlbaum Associates, London, pp. 221–239. Universal Access in the Information Society 1, 217–222.
Brooke, J., 1996. Sus: a ‘quick and dirty’ usability scale. In: Jordan, P.W., Ramakrishnan, I.V., Stent, A., Yang, G., 2004. Hearsay: enabling audio
Thomas, B., Weerdmeester, B.A., McClelland, I.L. (Eds.), browsing on hypertext content. In: International World Wide Web
Usability Evaluation in Industry. Taylor & Francis, London, UK, Conference, New York, USA.
pp. 189–194. Raskin, J., 2000. The Humane Interface New Directions for Designing
Byrne, M.D., 2003. Cognitive architecture. In: Jacko, J.A., Sears, A. Interactive Systems. Addison-Wesley, Boston, USA.
(Eds.), The Human–Computer Interaction Handbook. Lawrence Richards, J.T., Hanson, V.L., 2004. Web accessibility: a broader view. In:
Erlbaum Associates, London, pp. 97–117. International World Wide Web Conference. New York, USA.
Card, S.K., Moran, T.P., Newell, A., 1983. The Psychology of Rosenfeld, L., Morville, P., 2006. Information Architecture for the
Human–Computer Interaction. Erlbaum, Hillsdale, NJ. World Wide Web: Designing Large-Scale Web Sites, third ed. O’Reilly
Donker, H., Klante, P., Gorny, P., 2002. The design of auditory user Media.
interfaces for blind users. In: NordiCHI, Aarhus, DK. Savidis, A., Stephanidis, C., 1998. The homer uims for dual user interface
Edwards, A.D.N., 1989. An auditory interface for blind users. Human– development: fusing visual and non-visual interactions. Interacting
Computer Interaction 4, 45–66. with Computers 11, 173–209.
Fukuda, K., Saito, S., Takagi, H., Asakawa, C., 2005. Proposing new Section 508, 1998. Section 508 of the rehabilitation act. hhttp://
metrics to evaluate web usability for the blind. In: CHI 2005 www.section508.gov/i.
Conference on Human Factors in Computing Systems, Portland, Shneiderman, B., Plaisant, C., 2004. Designing the User Interface, fourth
USA. ed. Addison-Wesley, Reading, MA.
Hackett, S., Parmanto, B., Zeng, X., 2004. Accessibility of internet Spence, R., 1999. A framework for navigation. International Journal of
websites through time. In: ASSETS 2004. ACM Association for Human–Computer Studies 51 (5), 919–945.
Computing Machinery, Atlanta, USA. Stewart, T., Travis, D., 2003. Guidelines, standards, and style guides.
Hardwick, A., Furner, S., Rush, J., 1998. Tactile display of virtual reality In: Jacko, J.A., Sears, A. (Eds.), The Human–Computer Inter-
from the world wide web: a potential access method for blind people. action Handbook. Lawrence Erlbaum Associates, London,
Displays 18, 153–161. pp. 991–1005.
Jacko, J.A., Vitense, H.S., Scott, I.U., 2003. Perceptual impairments and Sullivan, T., Matson, R., 2000. Barriers to use: usability and content
computing technologies. In: Jacko, J.A., Sears, A. (Eds.), The accessibility on the web’s most popular sites. In: Conference on
 ARTICLE IN PRESS
270 S. Leuthold et al. / Int. J. Human-Computer Studies 66 (2008) 257–270

Universal Usability, Association for Computing Machinery, Arlington, Yoshikawa, H., 2003. Modeling humans in human–computer inter-
VA, USA. action. In: Jacko, J.A., Sears, A. (Eds.), The Human–Computer
Venkatesh, V., Agarwal, R., 2006. Turning visitors into customers: a Interaction Handbook. Lawrence Erlbaum Associates, London,
usability-centric perspective on purchase behavior in electronic pp. 119–149.
channels. Management Science 52 (3), 168–186. Yu, W., Reid, D., Brewster, S., 2002. Web-based multimodal graphs for
WCAG, 1999. Web content accessibility guidelines 1.0. hhttp://www. visually impaired people. In: Cambridge Workshop on Universal Access
w3.org/TR/WAI-WEBCONTENT/i. and Assistive Technology (CWUAAT), Cambridge, UK, pp. 97–108.
 Influence of Task Complexity and Navigation
Design on User Behavior on the World Wide
Web: Eye-Tracking Study Measuring
Navigation Performance and Strategy

Stefan Leuthold, Peter Schmutz, Javier A. Bargas-Avila ∗ ,

Alexandre N. Tuch, Klaus Opwis

Department of Psychology, University of Basel, Missionsstrasse 60/62a, CH-4055

Basel, Switzerland

Abstract

Web site navigation systems must help users determine their position, plan their
actions, and find their way back. To fit users’ tasks, navigation must organize con-
tent into meaningful groups, and its graphical design must fit users’ perceptual
and cognitive abilities. An eye-tracking laboratory study with 120 participants was
conducted to study the influence of task complexity and navigation design on user
performance, navigation strategy, and subjective preference. Two starting points
are uncovered for better supporting navigation behavior on the World Wide Web:
Firstly, navigations should be extended with different kinds of navigation items
adapted to the complexity of users’ navigation tasks – but only if these navigation
items are not hidden behind dynamic menus. Secondly, displaying many grouped
navigation items at once is better fitted to perception and cognition than dynamic
menu navigation: Regarding the first click to load a new page, users need less eye
fixations, are faster and more successful.

Key words: Navigation, Menus, Grouping, Links, Browsing, Task complexity,

Information architecture

∗ Corresponding author. Phone: +41-61-2673522, Fax: +41-61-2670632

E-mail address: [email protected] (J. A. Bargas-Avila).

Preprint submitted to Elsevier Science 23 February 2009

1 Introduction

The Internet is growing tremendously: Alpert and Hajaj (2008) report that
in 1998 Google search was based on an index that contained 26 million web
pages. It grew to over 1 billion pages in 2000, and surpassed the 1 trillion mark
in mid-2008. If these numbers are used as an approximation for the number of
existing web pages, they can be compared to the number of active web sites
Netcraft (2009) reports every month. This comparison shows that, on average,
a web site in August 2008 hosted about 5’600 pages. This number grew more
then tenfold between 2000 and 2008.
It can be assumed that a few web sites have a very large number of pages, and
most web sites only a few. Since large web sites only become large if many
users visit them repeatedly, there is an undisputable need to conceive a suit-
able navigation providing fast and effective access. Navigation guides users
through a web site, helping them to plan their actions, determine where they
are, and find their way back if they follow an inappropriate navigation item
(Rosenfeld and Morville, 2006).
Most web design or user interface design textbooks distinguish between two se-
quential conceptual tasks (e.g. Lynch and Horton, 1999; Garrett, 2003; Shnei-
derman and Plaisant, 2004): The task of organizing items into meaningful
groups, and the task of navigation design, dealing with the question of how
to graphically represent links and other navigation mechanisms, in order to
make user navigation as efficient and effective as possible.
The present work focuses on navigation design. It starts with cognitive psy-
chology’s notion of navigation being a problem that is solved by users whose
behavior can be understood using the Rationality Principle. Afterwards, two
of the Rationality Principle’s factors are examined: Firstly, considering factor
task, we are interested in how task complexity influences navigation behavior
and whether different kinds of navigation items to access the same content are
beneficial. Secondly, considering factor inputs, we are interested in how user
performance and satisfaction depend on different navigation designs.

2 Theoretical background

2.1 Navigation as problem solving

In their landmark book on the psychology of human-computer interaction,

Card, Moran, and Newell (1983) introduced the Rationality Principle. It pos-
tulates that six factors determine user behavior. Three of those factors describe
the objective situation: 1) goals, 2) task, and 3) operators. They do not depend
on the individual user. The three remaining factors describe individual users’

2
limitations: 4) perceivable inputs, 5) user knowledge, and 6) processing ability.
Based on previous works such as that of Newell and Simon (1972), Card et al.
add that rational behavior can often be described as problem-solving behav-
ior: Users behave according to the Rationality Principle, using its operators
to get from some initial state to a goal state.
In the following, an online store is used to illustrate the application of the
Rationality Principle’s six factors to web navigation tasks. The initial state is
assumed to be a browser window showing a certain company’s home page, and
the goal state (factor 1) is the user’s confidence that some desired shopping
item has been ordered successfully and is going to be delivered to their home.
Leaving out search facilities, because they are of no interest for this study, the
task (factor 2) is to find out which navigation items to click in the right order
to finally arrive at the page with the desired shopping item. The only navi-
gation operator (factor 3) available is the mouse click on a navigation item.
Users decide which navigation item to click (i.e. on which item to apply the
navigation operator) based on which navigation items they perceive as inputs
(factor 4), what domain knowledge is accessible from long-term memory (fac-
tor 5), and their ability to process perceived inputs and domain knowledge in
order to arrive at a decision about what to do (factor 6).
This work focuses on two of these six factors, which can be argued to have
obvious relevance for navigation design:

(1) Task (factor 2). In a navigation task, users have to maintain a goal state
(factor 1) in working memory and need to check it against the input
(factor 4) retrieved for comparison. If task complexity is defined as the
number of criteria of the goal state, user behavior should depend on task
complexity, because the more complex a task, the harder it is to acquire
input for working memory and check against the goal state.
(2) Input (factor 4). User behavior should depend on what users perceive as
input, since operators can be chosen and knowledge in long-term memory
can be activated only according to what is selected from perception.

In the next two sections, relevant theory and previous work regarding these
two factors are summarized and three hypotheses for our own work are derived.

2.2 Task complexity

User performance depends on task complexity

Sticking with the example of the online store: When users visit the web site,
according to the Rationality Principle their behavior depends on a goal that
they want to achieve. In some cases, this goal has no inherent criteria to be
met, e.g. users are just looking around to see whether something catches their

3
attention. This translates into the very simple task of browsing through the
site, and using the mouse-click operator on any object that surmounts the
attention threshold and is sufficiently appealing. In other cases, the goal is
achieved by matching a single criterion, e.g. users know exactly what they
want to buy (say, the ‘The Devil Wears Prada” DVD). The task is to identify
the navigation item that promises the shortest path to the desired shopping
item. Users expect to find only one item that matches, and for every item
they encounter, they can easily determine whether it is the desired item or
not. In a third type of cases, the goal has several criteria and can be achieved
with many items, e.g. users just like a specific actor, say Matt Damon, and
would like to buy some bargain DVD that they do not already own. A search
for “Matt Damon” on Amazon.com returned 224 movie results at the time of
writing this article – if they were shopping on Amazon, how would users go
about finding out which of the 224 DVDs were a bargain? The corresponding
navigation task is harder now: Users need to evaluate for each navigation item
whether the target page could carry items satisfying all criteria (i.e. do not
own it yet, bargain, and Matt Damon). In the three kinds of cases described,
the goal’s number of criteria varies, and it has been argued that navigation
tasks become harder for users with every additional criterion.
For the sake of readability, we will refer to navigation tasks carried out to meet
goals with one criterion as simple tasks. Navigation tasks to meet goals with
several criteria will be referred to as complex tasks. The previous paragraph
can thus be summarized in our first hypothesis: Users perform better with
simple tasks than with complex tasks.

Users adopt their navigation strategy to task complexity

Simple tasks are defined as being simpler because they require fewer mental
comparisons. Human-computer interaction textbooks (e.g. Fowler and Stan-
wick, 2004; Shneiderman and Plaisant, 2004) show several ways to alter navi-
gation designs in order to reduce the amount of mental comparisons required,
e.g. using search forms allowing users to enter several criteria to narrow down
their search result lists. This way, users can be sure that every item in the
search result list already matches their goal criteria. The drawback of this
solution, in problem-solving terms, is the introduction of new operators: In
order to use input forms and interpret search result lists, users need to cor-
rectly apply several motor and mental operators other than mouse click and
comparison.
The only way to reduce the amount of mental comparisons needed without
introducing new operators is the extension of the existing navigation: New
navigation items are added in such a way that every new item corresponds to
a criterion that could appear in complex tasks. Navigation items like “Bar-
gain” or “New” would already satisfy one of the criteria required, if users are
looking for a bargain Matt Damon DVD they do not already own.

4
Henceforth, navigation designs consisting of only one kind of navigation item
will be called simple navigations; those consisting of more than one kind ex-
tended navigations. Navigation items representing store categories like “Ac-
tion” or “Children’s books” are called category navigation items; those elimi-
nating a mental comparison are called service navigation items, cf. section 3.3.
We expect users to perform complex tasks better with extended navigations,
if the added service navigation items eliminate mental comparisons and help
them solve their tasks.
In problem-solving terms, navigation behavior could be explained as users’
trying to eliminate one criterion at a time to get closer to the goal (difference
reduction). For simple tasks, this elimination can be achieved with a single
mouse click on a category navigation item: “The Devil Wears Prada” is a com-
edy, and a click on the respective “Comedy” category navigation item reduces
the difference. For complex tasks, this elimination often cannot be achieved
with a category navigation item (bargain Matt Damon DVDs could be found
in “Adventure”, as well as in “Drama” or “War” – or in none of those cat-
egories). In these cases, users have a higher chance of getting closer to their
goal if they chose a service navigation item: If they are looking for bargains,
new items, or bestsellers, they can just click on the respective item.
Thus, we assume that, for complex tasks, it is often easier to find fitting
service navigation items than category navigation items. Consequently, our
second hypothesis is that users adapt their navigation strategy to task com-
plexity for some tasks and will, on average, choose more service navigation
items for complex tasks than for simple tasks.

2.3 Navigation design

Extending navigations with different kinds of navigation items like the service
navigation items in the previous paragraphs could pose a problem: If user be-
havior depends on perceivable input (Rationality principle’s factor 4, cf. 2.1),
for navigation tasks this means that it depends on how many navigation items
are presented as well as how they are presented.

How many navigation items to present

Several studies summarized in Larson and Czerwinski (1998) suggest restrict-

ing navigation to a maximum of about nine items with reference to Miller’s
(1956) famous 7±2 rule for short-term memory storage. Rosenfeld and Morville
(2006) state general consensus to be that users’ abilities to visually scan the
page constrain the number of links that can safely be displayed together, not
their short-term memory capacities. LeCompte (2000) agrees: He collected

5
and explained the most common false applications of the 7±2 rule in human
computer interaction.
Spool et al. (1999) report a negative correlation between the number of links
displayed on a page and users’ task success, but add that they were not sure
about how to interpret the finding. Nielsen (1999) states he does “find it ex-
cessive to dedicate 20% of an interior page to a listing of top-level options, all
of which can be accessed from the home page at the cost of a single additional
click” (p. 203). He would restrict the number of navigation items displayed,
recommending dynamic navigation, showing more options only if users select
a menu item – although he also mentions that showing the breadth of navi-
gation could be beneficial to users.
Grouping navigation items and using group labels as new navigation items al-
lows users to access a vast number of items with a few clicks over a few levels
(e.g. three levels with five items each give access to 125 items), as mentioned
by Shneiderman and Plaisant (2004). They cite several non-web studies find-
ing that users became lost when dealing with more than four or five levels of
depth. Response time as well as number of errors increased as menu depth in-
creased. Larson and Czerwinski (1998) found those findings also confirmed for
web applications: In their study, users had to deal with either 16 navigation
items on the first level and 32 on the second level, or 8 navigation items on
each of three levels. They performed better with only two levels of depth.
In contrast to GUI standards in desktop applications, where the majority of
menus are placed at the top of the window, Fowler and Stanwick (2004) recom-
mend displaying as many navigation items as possible on a single screen. They
argue that large lists of navigation items are easy to scan and – if they are
organized in a comprehensible way – much faster than nested menus, where
users have to deal with navigation items level by level.

How to display navigation items

Considering extended navigations introduced in section 2.2: Should the differ-

ent kinds of navigation items be grouped and displayed all at once – henceforth
referred to as extended visible navigation, favored by Fowler and Stanwick
(2004) – or should only the group titles be displayed, showing more items if
users click on a group title – extended menu navigation, favored by Nielsen
(1999)?
Ware (2008) describes human beings behaving as cost-efficiently as possible
in terms of “cognitive cost”, preferring internal pattern comparisons to eye
movements, eye movements to head or hand movements, and using head or
hand movements rather than moving the whole body, see Table 1.

For extended visible navigation, users only have to process the number of vis-
ible navigation items. For extended menu navigation, users have to process
the number of visible menus, open a menu, and process the respective menu

6
Table 1
Cognitive cost of user behavior (Ware, 2008, p. 104)
User behavior Cognitive cost (typical empirical values)
Internal pattern comparison 40 msec
Eye movement 100 msec
Mouse select 1’500 msec

entries. Based on Table 1, users should, for maximum efficiency, prefer several
internal pattern comparisons to an eye movement, and several eye movements
to a mouse select.
Considering eye fixations, Ware (2008) states that, besides the bottom-up pro-
cess synthesizing detected visual features into patterns, a top-down process
exists, taking care of content acquisition, guiding attention to task-relevant
features, and optimizing for “cognitive cost”. It should take users fewer fixa-
tions to select a certain navigation item from extended visible navigation for
two reasons: On the one hand, it displays more items at a time than extended
menu navigation. Thus, more internal pattern comparisons can be conducted
before additional eye movements are initiated. On the other hand, eye move-
ments protect users from unnecessary costly mouse movements, and users have
no reason to avoid mouse clicks in extended visible navigation.
Considering execution times, once users have found the correct group in ex-
tended visible navigation or have opened the correct menu in extended menu
navigation, the times they need to process the items are the same, if both navi-
gation types contain the same number of navigation items. The only difference
in execution times stems from finding the correct group with eye movements
in extended visible navigation versus finding the correct menu with eye move-
ments and opening it with a mouse click in extended menu navigation. Users
will be faster with extended visible navigation if only a small number of eye
movements are needed to find the correct group of navigation items on the
screen (considerably smaller than the approximately 15 eye movements to
make up for a single mouse select, see Table 1). If users have a subjective
notion of “cognitive cost”, they should also subjectively prefer the navigation
type that “costs less”.
Thus, we arrive at our third hypothesis: For two equally organized hierarchies
of navigation items, extended visible navigation outperforms extended menu
navigation in terms of the number of eye fixations needed for the first click to
load a new page, as well as the time needed for that click. Extended visible
navigation is also subjectively preferred.

7
3 Method

3.1 Design

In order to study the effects of task complexity and navigation design on user
performance and subjective preference, a 2x3 related-samples design was used
with the factors task complexity (simple task or complex task, cf. section
2.2) and navigation design (simple navigation, extended visible navigation or
extended menu navigation, cf. sections 2.2 and 2.3). The primary dependent
variable was user performance operationalized by the number of eye fixations
before the first click to load a new page, the time needed to perform the first
click and its correctness. Other dependent variables were navigation strategy
and subjective preference measured by ratings of ease of use, helpfulness, effort
and frustration (10-point scales).

3.2 Participants

A total of 120 participants, mainly undergraduate students of the University

of Basel, participated in the experiment in return for course credits. The ma-
jority of the participants were recruited via an online announcement on the
Department of Psychology’s web site. The average age was 25 years (SD =
8.17; range: 18-57), the gender distribution was 29% female and 71% male.

3.3 Apparatus & materials

Parts of an online store were implemented: Users could browse through three
main categories (books, DVDs, music) represented with tabs at the top of the
page and use a navigation on the left side to explore lists containing the store
items, in accordance with the guidelines from current research and textbooks
(for sample screenshots see Fig. 1). The following pages were implemented:
Home page. Empty except for top-level navigation containing the home tab
and the main category tabs for books, DVDs and music.
Category pages. A total of 60 category pages were implemented, 20 for each
main category. Each category page displayed a list of 15 books, DVDs or music
CDs, respectively, with the following information for each item: Title, author
or director or artist, release year, and price. For each shopping item, there was
a link in the list to put it into the shopping cart.
Service pages. For the extended navigation designs (see below), four service
pages were implemented in addition to the category pages for every main cat-
egory. The four pages contained the same types of lists as the category pages,

8
but corresponding to the service navigation items “New”, “Preview”, “Best-
sellers”, or “Bargains”, respectively.
Detail pages. For all navigation items that could be used to fulfill a task, a
detail page was programmed displaying the shopping item title, author or di-
rector or artist, price, release date, and edition. Every detail page contained
a link to put the shopping item into the shopping cart and a link to return to
the previous page.

Throughout the store, the main navigation with tabs for home, the main
categories, as well as a generic navigation containing a shopping cart icon and
links to “My account” and “Help” were visible. Users could click a button
“Task finished” if they had finished a task and wanted to return to the list of
tasks in order to begin the next one (see 3.4 for the procedure).

Fig. 1. Main category pages used for this study (books, DVDs, music; shown in
extended visible navigation condition)

Navigationsexperiment, Stefan Leuthold, Peter Schmutz, Olivia Brenzikofer, Klaus Opwis

Konzept: Es gibt immer vier Groupn

1.! Categoryn (20 Items)
2.! Interpreten bzw. Namen (2 Items)
3.! Erweitertes Angebot (3 Items)
4.! Service (5 Items)

9
Navigation design manipulation

Three versions of the online store were built (see Fig. 2). There was a con-
trol condition (simple navigation) and two different implementations of the
experimental level (extended navigations):

(1) Simple navigation. Control condition, showing 20 alphabetically sorted

category navigation items for books, DVDs or music, respectively.
(2) Extended navigation. This condition showed the same navigation items as
the simple navigation, and ten additional task-specific navigation items.
The navigation items were arranged in four distinct groups: the first group
showing the 20 category navigation items, the second containing two links
to names-based navigation (pages were not implemented), the third con-
taining three links to additional offerings (like “DVD players” or “News-
papers”), and the fourth containing five service navigation items. Four
of these service navigation items (“New”, “Preview”, “Bestsellers”, and
“Bargains”) were the same for all main categories, because we wanted
users to learn that alternative navigation items existed and that they
could rely on these, if they wanted, to solve complex tasks.
For the extended navigations two different designs were implemented:
(a) Extended visible navigation. All group titles and navigation items
were always visible.
(b) Extended menu navigation. Users had to click on a group title in order
to open the group and display the navigation items it contained.

All participants were presented with exactly the same navigation items and
pages. The procedure described in section 3.4 shows that effects of users’
knowledge about shopping items and processing ability (two of the Rationality
Principle’s six factors, cf. section 2.1) are averaged out, leaving only factors
task and input that could show effects, because goals and operators were
controlled (giving all users the same tasks to solve in a randomized order,
providing only links and no search mechanism).

User tasks

To study user performance navigation strategy regarding task complexity, two

different kind of tasks were created:

(1) Simple task. Finding an item that was described explicitly (e.g. “Put the
DVD ‘Liar, Liar’ with Jim Carey in the shopping cart.”). Users knew
that one and only one such item existed.
(2) Complex task. Finding an item that fitted the requested requirements
(e.g. “Find a DVD in Swiss German which was a box office success and
that is suitable for your 5-year-old cousin. If there is such a DVD, put
it into the shopping cart.”). Users did not know whether such an item

10
Fig. 2. Navigation designs used in this study (from left to right: simple navigation,
extended visible navigation, extended menu navigation)

Navigationsexperiment, Stefan Leuthold, Peter Schmutz, Olivia Brenzikofer, Klaus Opwis

existed at all. Every complex task consisted of at least two criteria, one of
which matched with one of the four service navigation items mentioned
in the explanation of the navigation designs above: Users were asked
to find an item that either was new on the market, would be released
soon, was a bestseller, or a bargain. In order to prevent users from being
able to simply match the visual pattern and force them to compare the
decoded content of the visual pattern to the criteria, complex tasks did
not use words that also appeared in the label of the navigation item.
In the example mentioned above, one criterion was to find a DVD that
was a “box office success”. Users had to match this requirement with the
service navigation item “Bestsellers”.

Measurements

User performance and behavior variables were measured using eye-tracking

and click data:
Fixations before first click. How many eye fixations were logged by the eye-
tracker before users clicked on an item to load a new page (the condition “load

11
a new page” is important, because in the extended menu navigation, the first
user click is always used to open the menu and not to select a navigation item
to load a new page). A fixation was recorded when a user fixated an area with
a radius of 30 pixels for at least 100 ms.
Time before first click. After users started the task: How much time it took
them to click a navigation item in order to load the new page.
Correctness of first page. It was logged whether the navigation task could be
completed on the first page users loaded. In that case, the click was coded to
be correct. Navigation strategy was measured as follows: It was logged whether
the first click was on a service navigation item or on the service navigation
menu.

Subjective preference was measured using short questions that could be an-
swered on 10-point Likert scales. The scales were: 1 = “completely disagree”
and 10 = “completely agree” (scale 1), 1 = “low” and 10 = “high” (scale 2)
or 1 = “not at all” and 10 = “very much” (scale 3). Measurements for the
following statements were collected:
Ease of use. “In this version, it was easy for me to complete the tasks.” – scale
1
Helpfulness. “The navigation helped me in completing my tasks.” – scale 1
Effort. “What level of effort was needed in order to fulfill these tasks?” —
scale 2
Frustration. “How much did using the web site upset you?” – scale 3

3.4 Procedure

Participants were first introduced to the experiment: They were shown the
task screen (which displayed a list of the navigation tasks for the respective
navigation type) and instructed on how to begin and end tasks. If they com-
pleted 30 of the total of 36 tasks within 30 minutes, they received a chance to
win one of two iPod Shuffles, and thus were offered an incentive to perform
the tasks as fast as possible.
Before each experiment, the experimenter ran a program that generated three
randomized lists of 12 tasks each, six of which were simple tasks and six of
which were complex tasks. The three lists covered all 36 tasks and each task
was posed to a participant only once. This way, no participant attempted
the same sequence of tasks, and tasks were randomized with respect to the
navigation design conditions. Order effects were eliminated through random
assignment of participants to one of six groups: Participants in different groups
used the three different navigation designs in different sequences (Fig. 3). Af-
ter each navigation design, participants filled out a questionnaire rating the
navigation on the subjective measures outlined in section 3.3.
At the end of the experiment, sex and age of the participants were recorded.

12
Fig. 3. Procedure: 120 participants were split up into six groups, and attempted six
simple and six complex tasks with each of the navigation designs
Independent variable 1: Independent variable 2:
Navigation design Task complexity
(within-subjects) (within-subjects)

Simple Extended visible Extended menu

!!six simple tasks !!six simple tasks !!six simple tasks

!!six complex tasks !!six complex tasks !!six complex tasks
Simple Extended menu Extended visible

!!six simple tasks !!six simple tasks !!six simple tasks

!!six complex tasks !!six complex tasks !!six complex tasks
Extended visible Simple Extended menu
Intro-
duction !!six simple tasks !!six simple tasks !!six simple tasks
and !!six complex tasks Subjective !!six complex tasks Subjective !!six complex tasks Subjective
assignment
Extended visible measures Extended menu measures Simple measures
to one of
the six
!!six simple tasks !!six simple tasks !!six simple tasks
sequences
!!six complex tasks !!six complex tasks !!six complex tasks
Extended menu Simple Extended visible

!!six simple tasks !!six simple tasks !!six simple tasks

!!six complex tasks !!six complex tasks !!six complex tasks
Extended menu Extended visible Simple

!!six simple tasks !!six simple tasks !!six simple tasks

!!six complex tasks !!six complex tasks !!six complex tasks

3.5 Data gathering and statistical analysis

Eye-tracking data were recorded with a Tobii 1750 eye-tracker, using Tobii’s
ClearView software. Afterwards, raw data were exported and post-processed
using tailored visual basic (VBA) programming. Number of eye fixations and
milliseconds to first click were aggregated over all 36 tasks in such a way that
values were averaged over all simple tasks and over all complex tasks of an
individual user.
For statistical analysis an alpha level of .05 was used. Fixation data were
log-transformed before statistical analysis, because they were not normally
distributed, but were skewed to the left. Statistical analyses were thus only
conducted on variables with normally distributed values according to the
Kolmogorov-Smirnov test. All post hoc tests were adjusted according to Bon-
ferroni, and where Mauchly’s test indicated that the assumption of sphericity
had been violated, the respective test statistics have been reported according
to Greenhouse-Geisser.

13
4 Results

4.1 User performance

Fixations before first click

A two-way within-subject analysis of variance (ANOVA) was conducted to

evaluate the effects of task complexity and navigation design on the number
of fixations before the first click. There was a significant main effect of the
independent variable task complexity, F (1, 105) = 37.85, ηp2 = 0.265, p <
.001. The main effect of the independent variable navigation design was also
significant, F (1.98, 208.27) = 4.15, ηp2 = 0.038, p < .017. However, there was
no significant interaction between the two factors, F (1.97, 207.14) = 1.01, ηp2
= 0.009, p = .367. The results of these tests, as well as the means and standard
deviations for each of the groups, are reported in Table 2 and Fig. 4.
For further analysis of the main effects, single comparisons were conduced
using one-tailed t tests. They showed that there was only one significant dif-
ference for simple tasks: Participants needed fewer eye fixations for extended
visible navigation than for extended menu navigation. For complex tasks, how-
ever, study participants needed significantly fewer eye fixations using extended
visible navigation compared to both extended menu navigation and simple
navigation. The results of these tests are reported in Table 3, descriptive data
gathered in Table 2.

Fig. 4. Average number of eye fixations needed for the first click, for simple and
complex tasks, with the three different navigation designs

1.9
Fixations before first click (log)

1.8

Complex tasks
1.7
Simple tasks

1.6

1.5
Simple Menu Visible

14
Time before first click

Another two-way within-subject ANOVA was conducted to evaluate the ef-

fects of task complexity and navigation design on the time before the first click.
There was a significant main effect of the independent variable task complex-
ity, F (1, 119) = 67.34, ηp2 = 0.361, p < .001. The effect of the independent
variable navigation design was also significant, F (1.97, 234.81) = 8.57, ηp2 =
0.067, p < .001. There was also an interaction found between the two inde-
pendent variables task complexity and navigation design, F (1.83, 218.02) =
4.57, ηp2 = 0.037, p < .014. The results of these tests, as well as the means and
standard deviations for each of the groups, are reported in Table 2 and Fig.
5.
For further analysis of the main effects, single comparisons were conduced us-
ing one-tailed t tests. They showed that, for simple tasks, participants were
significantly faster with simple navigation than with extended menu navi-
gation, and significantly faster with extended visible navigation than with
extended menu navigation. For complex tasks, participants were significantly
faster with extended visible navigation than simple navigation and than ex-
tended menu navigation. The results of these tests are reported in Table 3,
descriptive data gathered in Table 2.

Fig. 5. Average time (msec) needed for the first click, for simple and complex tasks,
with the three different navigation types
8,000

7,500
Time before first click (msec)

7,000

Complex tasks
6,500
Simple tasks

6,000

5,500

5,000
Simple Menu Visible

Correctness of first page

Analysis using chi-square revealed that participants’ first clicks were more suc-
cessful with simple tasks than complex tasks for extended visible navigation,

15
χ2 (1, n=1’433) = 4.21, p < .040. The same was found for simple navigation,
χ2 (1, n=1’424) = 30.79, p < .001. But there was no significant difference for
extended menu navigation, χ2 (1, n=1’436) = 2.35, p = .125. Data are reported
in Table 2.
Participants were more successful with their first clicks when using extended
visible navigation, compared to extended menu navigation: For simple tasks,
χ2 (1, n=1’436) = 6.858, p < .009, and for complex tasks, χ2 (1, n=1’433) =
4.394, p < .036.
Table 2
User performance data
Task com- Navigation Eye fixations Time for Correct-
plexity type click (msec) ness
M (SD) M log (SD) M (SD) %
Simple Simple 4.89 (6.97) 1.65 (0.64) 5’433 (2’533) 70.7
Menu 5.07 (5.64) 1.75 (0.55) 6’270 (2’451) 54.5
Visible 4.22 (5.39) 1.53 (0.63) 5’185 (2’535) 61.3
Complex Simple 6.22 (8.17) 1.85 (0.62) 7’757 (4’313) 56.5
Menu 6.07 (8.32) 1.86 (0.66) 7’333 (3’651) 50.4
Visible 4.98 (7.16) 1.66 (0.64) 6’234 (3’361) 55.9

Table 3
User performance data: Comparisons between the different navigation designs split
by task complexity
Task com- Navigation type Eye fixations Time for click
plexity comparison
t df p t df p
Simple Simple vs. menu 1.14 108 .128 2.88 119 .003
Simple vs. visible 1.26 108 .105 0.86 119 .197
Menu vs. visible 2.82 108 .003 3.76 119 < .001
Complex Simple vs. menu 0.08 109 .467 0.92 119 .180
Simple vs. visible 1.96 108 .026 3.24 119 .001
Menu vs. visible 2.19 110 .015 2.95 119 .002

4.2 Navigation strategy

Regarding navigation strategy, analysis using chi-square revealed that partici-

pants preferred category navigation items for both simple and complex tasks,

16
but they adapted their behavior dependent on task complexity, choosing sig-
nificantly more service navigation items for complex tasks, compared to simple
tasks: χ2 (1, n=1’433) = 102.21, p < .001 for extended visible navigation, and
χ2 (1, n=1’436) = 71.67, p < .001 for extended menu navigation, cf. Table 4.

Table 4
Navigation strategy chosen
Observed count
Navigation type Task complexity Category navigation Service navigation
items items
Menu Simple 672 46
Complex 560 158
Visible Simple 673 45
Complex 530 185

4.3 Subjective preference

A one-way ANOVA showed there were significant effects of the independent

variable navigation design on all four subjective measures, except for “Effort
needed”. Post hoc tests analyzing “Ease of use” and “Helpfulness” revealed
that extended visible navigation scored significantly higher compared to both
simple and extended menu navigation, with p < .002 for all relationships,
whereas post hoc tests analyzing “Frustration” revealed that extended visible
navigation was only significantly preferred to extended menu navigation, p <
.001, but not to simple navigation, p = .057.

Table 5
Data measured regarding subjective preference
Navigation design ANOVA test results
Measure Simple Menu Visible F p
Ease of use 6.35 (2.21) 6.37 (2.21) 7.26 (1.98) 9.56 (1.98, 235.42) < .001
Helpfulness 5.31 (2.51) 4.87 (2.56) 6.18 (2.33) 13.18 (1.98, 236.04) < .001
Effort 5.27 (2.06) 5.34 (2.04) 5.01 (1.99) 1.80 (1.95, 231.60) .169
Frustration 4.54 (2.59) 4.96 (2.48) 3.94 (2.29) 8.48 (1.99, 237.25) < .001

17
5 Discussion

5.1 Summary and interpretation of findings

The results of the present study strongly suggest that task complexity in-
fluences user performance and navigation strategy, whereas navigation de-
sign influences user performance and subjective preference. The results on
task complexity clearly show that problem solvers perform better with simple
tasks than with complex tasks – but user performance for complex tasks can
be significantly improved with the introduction of service navigation items, if
they are displayed as extended visible navigation and not as menus. Extended
visible navigation outperformed extended menu navigation as well as simple
navigation with respect to performance and subjective preference.

Task complexity influences user performance

Participants were significantly faster in making the first click to load a new
page, needed fewer eye fixations to do so and made significantly more correct
clicks when solving simple tasks compared to solving complex tasks. Users
being faster and making more correct clicks with simple tasks was no surprise:
Complex tasks were conceived such that users needed more mental compar-
isons to determine whether a navigation item could meet all criteria, which
on the one hand made it harder to decide for a click and consequently took
more time, and on the other hand led to more incorrect first clicks.
More interesting is that participants not only needed more time, but also more
eye fixations for complex tasks: They did not just mentally process the per-
ceived items longer – they used some of the additional time to acquire new
navigation items through additional eye fixations. Thus, they did not process
the same amount of visual information for simple and for complex tasks.
This behavior can be explained with Ware’s cognitive cost efficiency men-
tioned in section 2.3: The top-down process of perception triggered more eye
fixations for complex tasks because this was less costly for cognition than con-
ducting more mental comparisons. Additional analyses not reported in the
results section support another aspect of explaining user behavior according
to this line of reasoning: Users switched their navigation strategy even if it
was successful. About 24% of users, after correctly clicking on a category nav-
igation item, switched their navigation strategy and subsequently clicked on a
service navigation item when solving complex tasks (compared to only about
1% for simple tasks). An obvious explanation for this behavior and the large
difference in percentages is that users, after looking at some shopping items,
invested a mouse click on another link instead of additional mental compar-
isons or eye fixations, because trying to match all shopping items on the page

18
against the goal criteria is much harder for complex tasks than for simple
tasks.
Complex tasks were harder indeed: Using simple navigation, users’ first clicks
needed 43% more time for complex tasks than for simple tasks. Extending the
simple navigation with service navigation items fit to users’ tasks narrowed
this gap to 20% (extended visible navigation) or 17% (extended menu naviga-
tion). But user performance data in Table 2 and corresponding results show
that adding service navigation items is not enough. It also matters how they
are displayed: For simple tasks, extended visible navigation did not make users
significantly faster, but the extended menu navigation made users 15% slower,
and this difference was significant. For complex tasks, users were significantly
faster with extended visible navigation, showing a rather large time gain of
24% compared to both simple and extended menu navigation.
Thus, we conclude that our first hypothesis is confirmed: Users performance
depends on task complexity, users perform better with simple tasks than with
complex tasks. Task performance can be improved significantly for complex
tasks with the addition of service navigation items – if they are visible at once
together with the existing navigation items. However, using menus rather than
extended visible navigation slowed participants down.

Task complexity influences navigation strategy

Data in table 4 show a significant difference in service navigation item us-

age that was not caused by navigation design, but by task complexity: Users
chose more than three times as many service navigation items for their first
clicks when solving complex tasks. Interestingly, compared to simple naviga-
tion, users perform complex tasks better only with extended visible navigation
but not with extended menu navigation.
We conclude that our second hypothesis is confirmed: Users adopt their nav-
igation strategy to task complexity for some tasks and, on average, choose
more service navigation items for complex tasks than for simple tasks.

Navigation design influences user performance and satisfaction

Users needed significantly less time and fewer eye fixations to arrive at their
first click to load a new page using extended visible navigation compared to
extended menu navigation, and they made significantly more correct clicks.
Our results demonstrate that regarding user performance and satisfaction,
apart from employing unambiguous familiar navigation labels – as recom-
mended by textbooks like Rosenfeld and Morville (2006) – there is a separa-
ble effect of the navigation design: User performance and satisfaction can be
improved without changing navigation labels, because visible grouped links
seem to be better fitted to perception and cognition than navigation menus.

19
Users navigate in a cognitive-cost-efficient way, as stated by Ware (2008), and
are more efficient processing many navigation items at once instead of making
their minds up about which menu item to open and clicking the appropriate
navigation item within.
Our study also adds to LeCompte’s arguments against the false application of
Miller’s 7±2 rule (LeCompte, 2000). From our own practical work, we share
LeCompte’s experience that ‘Miller’s rule” is often quoted in order to explain
that users would be overwhelmed and could not possibly be able to process
large numbers of items, predicting they would be faster and, of course, more
satisfied if they were presented with no more than 7±2 navigation items at a
time. There are only two ways to design a navigation system compliant to this
rule: Firstly, to use menu navigation – for which our results show “Miller’s
rule” arguments to be false – or secondly, to distribute navigation items in
such a way that the first page shows 1st-level navigation items, the next page
2nd-level navigation items, and so on, introducing several layers of pages in
the web site hierarchy – which can safely be assumed to result in considerably
more clicks and eye fixations than all other design solutions. Based on these
findings, we do not agree with Nielsen (1999) to present navigation items in
menus, but rather side with Fowler and Stanwick (2004), who recommend
showing as many reasonably grouped navigation items as possible on a single
screen.
Regarding subjective preference, users did not judge extended visible naviga-
tion to need significantly less effort than extended menu navigation, but they
thought extended visible navigation was easier to use, more helpful, and caused
them considerably less frustration in executing their tasks. Whereas objective
performance data gathered refers to users’ first clicks on a navigation item,
their subjective preference reflects their whole experience of a given navigation
design. We hypothesized that users have an inherent notion of the “cognitive
costs” of navigation and that the subjective preference reflects this: Opening
menus needed an additional mouse movement and was thus more costly than
just scanning the navigation items.
Thus, we conclude the third hypothesis to be confirmed: Extended visible nav-
igation outperforms extended menu navigation and is subjectively preferred.

5.2 Strengths and limitations

To our knowledge, most eye-tracking studies, e.g. hot spot analyses, are con-
ducted for (comparative) evaluation of designs. They answer questions like
whether users look at certain areas to solve certain tasks, how many fixations
are recorded for a certain area of interest, or whether certain items on the
screen get noticed at all. A strength of our research is that we used eye-tracking
data to generate objective user performance data, using the eye-tracker to get
insight into users’ cognition, explaining for example that the number of eye

20
fixations (instead of only “more thinking”) is what must have been a partial
cause for the longer time it took to make the first click in complex tasks than
in simple tasks.
Usability measures like task completion times or amount of clicks needed can-
not separate the effects of the graphical navigation design employed from
effects of the displayed navigation items’ information scent: One can always
argue that users would need fewer clicks and would be faster if only the nav-
igation labels employed were more meaningful. A strength in this work, the
experimental paradigm chosen ruled out the argument of better labeling and
separated the effect of navigation design on user performance and subjective
evaluation.

A limitation of our work was the focus on the first click, which we introduced
in order to study user cognition (see above). Future research should measure
times and error rates throughout whole tasks – of course displaying real con-
tent instead of empty pages as we did in our research.
Another limitation was the inclusion of mouse movement time in the measure-
ment of the time needed for the first click. Of course, we can argue that users
started moving their mouse from the top of the screen (where they clicked
a main navigation item) towards the first target navigation item, either a
category navigation item, service navigation item or menu item. And, conse-
quently, mouse movement paths should be significantly longer in the simple
and extended visible condition than in the extended menu condition, because
the more navigation items displayed, the longer the mouse movement path
(averaged over all tasks). Considering the third hypothesis, this helps the cor-
responding null hypothesis, because we assume shorter times to first click
for a condition with longer mouse movement paths. But our results would
not only gain validity: improved time measurement without mouse movement
time would also allow researching several follow-up questions enumerated in
the next section.

5.3 Implications for future research

Our results indicate at least five starting points for future research:
Navigation versus search forms. Designing navigation systems for complex
tasks presents a trade-off, as mentioned in section 2.2. Under what circum-
stances (e.g. number of goal criteria, total number of navigation items needed,
number of shopping items to be covered) are users not able to solve navigation
tasks with browsing anymore but need to switch to searching? Following Katz
and Byrne (2003), it should not depend on users’ personal preferences but
on their tasks and information scent whether they use browsing or searching
strategies. In analogy, it should depend on users’ tasks whether they perform
better with web forms (in which they can enter all criteria of a complex task

21
at once) or navigation (with which they can eliminate goal criteria one click
at a time).
Determining the trade-off of eye movements against mouse movements. Now
that we have established, for extended visible navigation and for extended
menu navigation, that differences in execution time are partly attributable to
differences in the number of eye fixations and the associated mental operations
needed (and not only to additional mouse movement), it would be interesting
to actually determine the turning point where users start trading off eye move-
ments against mouse movements and research its dependency on context. It
could be argued, based on values from Ware (2008), that this turning point
could be somewhere around trading 15 eye movements for one mouse move-
ment. Does this depend on task complexity, users’ knowledge, memory load, or
fatigue? Leuthold et al. (2008) observed that blind users avoid trial-and-error
behavior in navigation tasks, whereas sighted users exhibit this behavior in
order to save cognitive resources: They can risk an incorrect click, because –
in stark contrast to blind users – it does not cost them much time and effort
to go back.
Reversal of preference for extended visible navigation. It also remains unclear,
after our study, whether users always prefer navigation designs they can use
with the fewest number of clicks, or whether there really exists a turning point
where users become faster and more satisfied if they can trade off mouse clicks
for eye movements. This could be researched by comparing web sites with dif-
ferent amounts of navigation items: For very large amounts of items, where
users need considerably more than 15 eye movements to arrive at a navigation
decision, their preference for extended visible navigation over extended menu
navigation should reverse, and they should be slowed down in their first click
to load a new page.
Optimizing information architecture for correctness of first click. Different nav-
igation designs and different information architectures (probably optimized for
information scent) should be compared in order to conceive navigation sys-
tems with higher success rates – approximately 50% for problem tasks seems
to be very low. The analysis should then also be extended to cover not only
correctness of the first click, but also correctness of navigation paths to fulfill
whole tasks.
Design features triggering useless eye movements. Do certain design features
exist that trigger useless eye movements? For example, Ware (2008) argues
that features in distinct channels are separated into patterns by the bottom-
up process of visual perception. If the visual design of a navigation uses dif-
ferent channels, maybe fixations are wasted because users need to process the
whole hierarchy of visual patterns before arriving at the point where semantic
information about navigation items can be processed.

22
5.4 Conclusion

Different kinds of navigation items should be displayed in a navigation system

to let users access the same content and functionality through different ways
that correspond to the goal criteria of their tasks.

Our study strongly suggests rethinking the usage of menu navigation on the
web. It confirms the recommendation made by Fowler and Stanwick (2004):
When designing a navigation system, show as many reasonably organized
items as fit on the screen, because grouped links are better fitted to percep-
tion and cognition than menu navigation – especially when users are solving
complex navigation tasks.

References

Alpert, J., Hajaj, N., 07 2008. We knew the web was big...
URL http://googleblog.blogspot.com/2008/07/we-knew-web-was-big.html
Card, S. K., Moran, T. P., Newell, A., 1983. The Psychology of Human-
Computer Interaction. Erlbaum, Hillsdale, N.J.
Fowler, S., Stanwick, V., 2004. Web Application Design Handbook: Best Prac-
tices for Web-Based Software. Morgan Kaufmann Publishers Inc., San Fran-
cisco, CA, USA.
Garrett, J. J., 2003. The elements of user experience: user-centered design for
the Web. New Riders.
Katz, M. A., Byrne, M. D., 2003. Effects of scent and breadth on use of site-
specific search on e-commerce web sites. ACM Transactions on Computer-
Human Interaction (TOCHI) 10 (3), 198–220.
Larson, K., Czerwinski, M., 1998. Web page design: implications of mem-
ory, structure and scent for information retrieval. In: CHI ’98: Proceedings
of the SIGCHI conference on Human factors in computing systems. ACM
Press/Addison-Wesley Publishing Co., New York, NY, USA, pp. 25–32.
LeCompte, D. C., August 2000. 3.14159, 42, and 72: Three numbers that
(should) have nothing to do with user interface design.
URL http://www.internettg.org/newsletter/aug00/contents.html
Leuthold, S., Bargas-Avila, J. A., Opwis, K. Beyond web content accessibility
guidelines: Design of enhanced text user interfaces for blind internet users.
International Journal of Human-Computer Studies 66 (4), 257–270.
Lynch, P. J., Horton, S., 1999. Web Style Guide: Basic Design Principles for
Creating Web Sites. Yale University Press, New Haven, CT, USA.
Miller, G. A., 1956. The magical number seven, plus or minus two: Some limits
on our capacity for processing information. The Psychological Review 63,
81–97.

23
Netcraft, 01 2009. Web server survey archives.
URL http://news.netcraft.com/archives/web server survey.html
Newell, A., Simon, H. A., 1972. Human Problem Solving. Prentice Hall, En-
glewood Cliffs, NJ.
Nielsen, J., 1999. Designing Web Usability: The Practice of Simplicity. New
Riders Publishing, Thousand Oaks, CA, USA.
Rosenfeld, L., Morville, P., 2006. Information Architecture for the World Wide
Web: Designing Large-Scale Web Sites, 3rd Edition. O’Reilly Media.
Shneiderman, B., Plaisant, C., 2004. Designing the User Interface, 4th Edition.
Addison Wesley.
Spool, J. M., Scanlon, T., Snyder, C., Schroeder, W., DeAngelo, T., 1999.
Web site usability: a designer’s guide. Morgan Kaufmann Publishers Inc.,
San Francisco, CA, USA.
Ware, C., 2008. Visual Thinking for Design. Morgan Kaufmann Publishers
Inc., Burlington, MA 01803, USA.

Acknowledgements

The authors would like to thank Olivia Brenzikofer for conducting the majority
of the 120 experiments and David Weber for spending a whole summer with
programming parts of the online shop used for this experiment and trying
to marry the eye-tracking system with the traditional usability setup of the
University of Basel’s usability lab (in order to enable the gathering of all the
data we needed in a useful format).

24
 Interacting with Computers 21 (2009) 257–262

Contents lists available at ScienceDirect

Interacting with Computers

journal homepage: www.elsevier.com/locate/intcom

Error prevention in online forms: Use color instead of asterisks to mark

required-fields
Stefan L. Pauwels *, Christian Hübscher, Stefan Leuthold, Javier A. Bargas-Avila, Klaus Opwis
University of Basel, Faculty of Psychology, Department of Cognitive Psychology and Methodology, Basel, Switzerland

a r t i c l e i n f o a b s t r a c t

Article history: In this study, a simple but important user interface design choice is examined: when marking required-
Received 19 December 2008 fields in online forms, should GUI designers stick with the often used asterisk that many form design
Received in revised form 26 May 2009 guidelines cite as the de-facto web standard, or should they choose a colored background as a new design
Accepted 26 May 2009
solution to visually signal which input fields are required? An experiment with 24 participants was
Available online 23 June 2009
conducted to test the hypotheses that efficiency, effectiveness and satisfaction ratings of colored
required-fields exceed those of asterisk-marked required-fields. Results indicate that colored required
Keywords:
field marking leads to fewer errors, faster form fill-in in and higher user satisfaction.
Online forms
Required-fields
! 2009 Elsevier B.V. All rights reserved.
Error prevention
User feedback
Interaction design

1. Introduction input a process needs in order to proceed and how exactly to enter
data in the correct format are the two main problems form design
Entering data into forms is the information worker’s equivalent needs to address from the information worker’s perspective. This
to the factory worker’s placing items on a conveyor belt. Both com- study examines a part of the first question: how should required
puter applications and machines installed along the conveyor belt input fields for data input on an online form be marked?
are fed input, data or items, based on which they carry out certain Wroblewski (2008) mentions three different usage contexts of
transformations. In order to avoid process errors (e.g. transforma- online forms: e-commerce applications, applications for social
tions applied to the wrong item, yielding undesirable or at least interactions, and productivity-based applications. He finds that
unpredictable results), both computer applications as well as fac- form design should not be neglected by companies with online
tory machines need to stop processing and alert the worker if no user interfaces, stating increased completion rates of between
input is present. If input is present and properly formatted as re- 10% and 40% from improvements indicated by his research and
quired by the transformations applied, both types of machines experience. Spool (in Wroblewski, 2008) mentions a real-life
work with great speed and accuracy. example of an online form redesign yielding increased revenues
As Frederick Winslow Taylor, the founder of scientific manage- of $1.5 million in the first week, with a total of over $300 million
ment, argued, companies should strive for the greatest possible increased revenue in the first year.
productivity, because this leads to the greatest possible prosperity
for company owners and workers alike (Taylor, 1911). Productivity 1.1. Theoretical background
can be defined as data or items processed per time unit (US Federal
Bureau of Justice Assistance, 2008). Thus, for computer applica- Cooper et al. (2007) devote the first chapter of their textbook on
tions, in order to achieve greatest possible productivity, companies user interface design to goal-directed design, which rests on the
need to prevent errors in data input and design input systems to awareness that human behavior is goal-oriented from the age of
maximize input and transformation efficiency. Knowing what three (Klossek et al., 2008). Behaving in a goal-oriented way means
that users choose their actions in order to get as much value as
possible from an IT system with the least amount of effort
* Corresponding author. Address: University of Basel, Faculty of Psychology, expended. As for forms, they will try to enter the least possible
Department of Cognitive Psychology and Methodology, Missionsstrasse 60/62, 4058 amount of data in order to get the desired result, e.g. start or pro-
Basel, Basel-Stadt, Switzerland. Tel.: +41 61 267 35 68; fax: +41 61 267 06 32.
ceed with a process or become authorized to do something. The
E-mail addresses: [email protected] (S.L. Pauwels), [email protected]
(C. Hübscher), [email protected] (S. Leuthold), [email protected] (J.A. only reasons to enter more information are either errors in input
Bargas-Avila), [email protected] (K. Opwis). field validation, or the system’s failure to interpret the user’s input

0953-5438/$ - see front matter ! 2009 Elsevier B.V. All rights reserved.
doi:10.1016/j.intcom.2009.05.007
258 S.L. Pauwels et al. / Interacting with Computers 21 (2009) 257–262

in such a way as to arrive at a single possible path of the process, As for the second and third steps, entering data in the right for-
therefore asking for more information. Cooper et al. (2007) argue mat and navigating between input elements: because expert users
that it is best to provide the user with rich modeless feedback, sig- are fastest if they do not have to home between mouse and key-
naling that required data is missing without interrupting the user’s board, speeding up form input entry is a main priority of form
work. design. Shneiderman and Plaisant (2004) advise enabling the tab
Most web design guidelines are derived from a rather small set key to move the cursor between fields. Many other guidelines for
of heuristics like ‘‘Minimize the user’s memory load” or ‘‘Provide an form entry and form navigation can be found in the works quoted
obvious way to undo actions” (e.g. Nielsen and Landauer, 1993; above, like considering the layout of form input elements or using
Polson and Lewis, 1990). Nielsen (1994) shows that a few heuris- effective default values. For this study, it is sufficient to state that if
tics are indeed enough to explain a remarkable number of observed users are able to enter data as quickly as possible, they are faster
usability errors. Tidwell (2005) presents 12 behavioral patterns the fewer input fields that are required and the fewer optional ones
gathered from user observations, and derives the heuristic of they have to attentively process in order to rule them out and focus
marking required input fields from the pattern of ‘‘deferred on the required ones. The most efficient form is the one that has
choices”, stating that users often want to or have to skip input the fewest possible required-fields grouped together, clearly indi-
fields and may want to come back to them later. Considering the cating that users can submit the form once they have completed
five usability metrics introduced by Nielsen (1993), the marking the fill-in of the group. This line of argument is found in several
of required-fields can be assumed to contribute mainly to effi- works, e.g. Fowler et al. (2004) and Wroblewski (2008).
ciency and error prevention, which, for online forms, translates Concerning the fourth step, validation of form input, Bargas-
into four steps: Avila et al. (2007) researched the presentation of error messages
but not the marking of required-fields. Their Modal Theory of Form
(1) The user’s perceptive and cognitive acts of finding the Completion suggests that users enter data into a form while in
required-fields with respect to the current task. completion mode before being mentally ready to address errors
(2) Entering properly formatted data into the input fields as in revision mode. It is not known whether required-field markings
quickly as possible. support users during completion mode, revision mode or both.
(3) Navigation between form input elements. Bargas-Avila et al. (2007) show that users tend to just overlook
(4) Validation of form input, where each blank required input messages relevant for evaluation during completion mode. Form
field and each formatting error decreases efficiency fill-in performance is not negatively influenced by this. Therefore
substantially. it can be assumed that good required-field design cannot do any
harm even if unnecessarily present in completion mode.
The first step is addressed by all web design guidelines exam- Practitioners in form design mention the trade-off between effi-
ined for this study, which argue that required-fields should be ciency and error prevention (Baxley, 2002): Forms can either be
clearly marked in order to make users efficient and prevent errors optimized for expert users, who know data type and format of
(Fowler et al., 2004; Horton, 2005; Koyani et al., 2004; Shneider- every single entry field, or they can be optimized for novice or
man and Plaisant, 2004; Tidwell, 2005; Wroblewski, 2008). One infrequent users in order to prevent data entry errors. Thus, form
exception is found in the Apple Human Interface Guidelines (Apple design should take into consideration variance in user behavior
Inc., 2008), which states that, to prevent visual clutter, an asterisk (frequent vs. infrequent users; expert vs. novice users) in order
or custom icon next to a required input field should be displayed to maximize efficiency, and de-facto standards, using what users
only after the user attempts to leave the current context (e.g. by already know and understand to prevent errors in data entry.
clicking Continue or OK). The different theoretical and practical considerations men-
Whereas Horton (2005) is satisfied with marking required-fields, tioned above can be summarized into two overarching guidelines:
other guidelines explicitly state design solutions: Wroblewski
(2008) argues use of an asterisk, because it has become the de-facto (1) Make required-fields clearly visible on first sight (Fowler
standard on the web, and, for top-aligned input field labels, even the et al., 2004; Horton, 2005; Koyani et al., 2004; Shneiderman
use of ‘‘* required” instead of the asterisk alone. He warns not to use and Plaisant, 2004; Tidwell, 2005; Wroblewski, 2008) in
the same indicator for required-fields as for optional fields through- order to facilitate fill-in for novice users and speed up per-
out a web site. Fowler et al. (2004) also argue for the asterisk, but ception by expert users. An asterisk is probably not the pre-
mention the use of arrows or other symbols to indicate required- ferred visual mark because it takes more time to perceive
fields. They advise against using color (for either field background (Ware, 2008), although it can be read by a screen reader
or label) or boldface, because screen readers are not able to interpret (Fowler et al., 2004).
their meaning, necessitating a lot of guesswork for the visually im- (2) Add additional visual elements if the user wants to leave the
paired or blind people using screen readers. Shneiderman and Plai- context and a required-field is still empty, in order to draw
sant (2004) state having taken their guidelines from practitioners’ his attention to the correct location on the screen (Apple
works, since there is ‘‘a paucity of empirical work on form fillin”. Inc., 2008). This supports the switch from completion mode
They argue that optional fields should be marked with the word to revision mode (Bargas-Avila et al., 2007) and is modeless
‘‘optional” or another distinct visual mark, and that optional fields (Cooper et al., 2007).
should be placed after required-fields. In addition, they recommend
using a clear completion signal, so users understand at what point This study is concerned with the overarching guideline 1 men-
they can safely submit the form because all necessary information tioned above, namely the marking of required-fields. The aim of
has been entered. the study is to explore an alternative to the visually not very salient
In the only empirical study that has been identified to date, asterisk next to field labels: marking required-fields by coloring
Tullis and Pons (1997) compared several possibilities for mark- their background, a measure that should ease form fill-in for users
ing required input fields. They found only small differences in according to recommendations to clearly mark required-fields. In
completion time between chevrons placed before the field label particular, professional users of a financial services firm were
and colored fields, but users preferred colored fields to chevrons asked to fill out a rather complex online form in a real-life task.
in ratings on scales of visual appearance and overall The number of errors they committed was measured as was the
effectiveness. speed with which they could complete the form when the
 S.L. Pauwels et al. / Interacting with Computers 21 (2009) 257–262 259

required-fields of the online forms were either marked by an aster- 2.2. Participants
isk, as is the convention in web design (Wroblewski, 2008), or
colored. In this study, the following questions are explored: does The study was conducted with 24 participants. Their age ranged
marking required-fields with background color instead of using from 21 to 48 with a mean of 32 (SD = 7.2). Thirteen of the partic-
an asterisk besides the text label lead to fewers errors in form ipants were women. All participants were employees of a financial
fill-in? Are participants able to complete form fill-in faster if institute. They were all users of the CRM application from which
required-fields are marked with background color instead of aster- the manipulated forms were taken. The CRM application imple-
isks? And finally, does marking required-fields with colored back- mented both versions of required-field markings in different
grounds affect user satisfaction positively compared to asterisk screens in an inconsistent manner, therefore participants were
markings? used to both variants.

2. Method 2.3. Apparatus and materials

2.1. Design The two versions of the web based form were recreated as part
of a mock-up of the CRM application using HTML and Adobe Flash
For this study, participants filled out two different versions of a technology. Flash was used so the look and feel of the proprietary
form that is part of a browser-based CRM application. The study CRM application could be mimicked. The mock-up was presented
used a related samples design. The independent variable was the on a laptop computer and sessions were recorded with the usabil-
type of marking applied to required-fields. It had two levels: mark- ity test recording software TechSmith Morae (version 2.0.1). Errors
ing by labels with asterisks and marking by colored field back- were tracked using markers in Morae. Task completion time was
grounds. Figs. 1 and 2 show parts of the forms with different logged automatically by the software. Participants received a paper
marking types for required-fields. interview guide that included a short demographic questionnaire,
Dependent variables were the number of errors a participant the instructions to the two tasks they had to complete, and the
made during the task, task completion time and a post-test question- short version of QUIS after every task. Finally, two additional ques-
naire to assess a participant’s satisfaction with the user interface. tions were added, to explore whether the experimental task was
Once a participant tried to submit a form, each empty required-field realistic and how frequently the participants encounter it during
was counted as an error. A message was presented to the user, con- their work.
taining information about the missing required-fields. The form was
not submitted until all required-fields were complete. The short 2.4. Procedure
form of QUIS (Chin et al., 1988), a validated and widely used ques-
tionnaire (Shneiderman and Plaisant, 2004) for user interface satis- Participants completed the experiment individually. They were
faction, was applied to measure user satisfaction. presented with the start page of the application mock-up and were

Fig. 1. Part of the stock exchange order form with required-fields marked by colored backgrounds.
260 S.L. Pauwels et al. / Interacting with Computers 21 (2009) 257–262

Fig. 2. Part of the stock exchange order form with required-fields marked by asterisks besides labels.

told to complete the questions and tasks in the interview guide. task completion time, t(23) = 1.836, p = .04, and higher satisfaction,
The two tasks in the interview guide had the following form: t(23) = 1.754, p = .047 (one-tailed tests).
‘‘Customer hCi called and asked you to buy hNi registered shares To control the applied sequence effect counter measures, the data
of hStock Xi (Symbol: hYi) at market. Use portfolio number hZi.” were analysed using two-way analyses of variance (ANOVA) with
After completing the task, all participants answered the satis- required-field marking (asterisk, colored background) and experi-
faction questionnaire and the task suitability questions. Every par- mental sequence (asterisk – colored background, colored back-
ticipant completed two tasks, one for every required-field marking ground – asterisk) as the two factors and an alpha level of .05.
type. The order of the two required-field marking types was alter- Dependent variables were, again, number of errors, task time and
nated to counter sequence effects: half the participants’ forms had satisfaction. Fig. 3 shows that in the second trial, participants com-
required-fields marked by asterisks during the first task and mitted fewer errors, F(1, 22) = 6.102, p = .022, were faster,
colored backgrounds during the second task, the other half of the F(1, 22) = 19.844, p < :001, and were more satisfied, F(1, 22) =
participants had forms with required-fields marked by colored 4.510, p = .045, no matter what task they did second. Alternating
backgrounds during the first task and asterisks during the second the order of required-field markings in the tasks successfully coun-
task. tered these sequence effects as all three analyses showed no signif-
icant main effect for experimental sequence for number of errors,
3. Results F(1, 22) = 2.538, p = .125, task time, F(1, 22) = .023, p = .881, and sat-
isfaction F(1, 22) = 2.631, p = .119.
The measured dependent variables of the two required-field
marking conditions are shown in Table 1.
4. Discussion
To test the hypotheses that colored backgrounds as required-
field markings lead to fewer errors, faster form fill-in and greater
Participants completed form fill-in committing significantly
satisfaction compared to asterisk markings, t-tests for related sam-
fewer errors when required-fields were indicated by colored back-
ples and an alpha level of .05 were used. There were statistically
grounds rather than by the usual asterisk. Furthermore, the colored
significant differences: marking required-fields by colored
background let them complete the forms significantly faster and
background caused fewer errors, t(23) = 2.777, p = .006, shorter
more satisfied.
Since we measured task completion time as the duration from
Table 1
initial display of the form to its successful completion, we have
Statistical parameters for different required-field markings.
to assume that task times were influenced directly by whether a
Asterisk Colored background form could be submitted free of errors on the first try or whether
Measures n M (SD) M (SD)
the participants had to deal with it again after an unsuccessful at-
Number of errors 24 2.540 (2.621) .830 (1.239) tempt. The influence of task completion times was therefore to a
Task completion time (s) 24 152.852 (96.008) 108.146 (71.912)
large degree moderated by the number of errors committed. This
Satisfaction (QUIS) 24 6.560 (1.454) 7.342 (1.775)
had the advantage of getting a realistic picture of the actual time
 S.L. Pauwels et al. / Interacting with Computers 21 (2009) 257–262 261

chance of preventing incomplete form submission. The fact that

participants were significantly more satisfied with the forms with
colored required-fields suggests that the raised saliency is a wel-
come help rather than an annoying or unnecessary distraction.
The practical implications of this study are therefore that form
designers can ease form input for users by marking required-fields
with a colored background. One disadvantage of colored required-
fields was mentioned earlier: the inability of screen readers to rec-
ognize the color-markings. Consequently we advise designers to
use colored backgrounds as an additional marker of required-field
rather than an exclusive one.
However, this study does not answer the question, to what de-
gree this finding is applicable to other forms. The form used in this
experiment was quite complex and targeted towards a professional
completing a critical task. Many forms on the Internet (e.g. forms
for account creation) are smaller and have fewer or less dire conse-
quences in the event of errors than an erroneous stock exchange
order. However, many online forms are appearing in various order
and checkout process pages where the support provided by
improved required-field marking could prove helpful. It should be
mentioned, that it is not possible to draw any conclusions on what
color should be used to highlight the required-fields’ backgrounds.
Yellow1 backgrounds provided the results reported here, but
different factors could influence the best color choice. Usage of
colors throughout a page’s or application’s design could influence
saliency of yellow required-fields as well as general or site-specific
information or warnings implied by the use of certain colors.

Acknowledgement

The authors would like to thank the Zürcher Kantonalbank

(ZKB) in Zurich, Switzerland for the support and funding of this re-
search as part of the UCD ZKBconnect project.

References

Apple Inc., 2008. Apple Human Interface Guidelines: User Experience. <http://
developer.apple.com/documentation/UserExperience/Conceptual/AppleHI
Guidelines>.
Bargas-Avila, J.A., Oberholzer, G., Schmutz, P., de Vito, M., Opwis, K., 2007. Usable
error message presentation in the world wide web: Don’t show errors right
away. Interacting with Computers 19 (3), 330–341.
Baxley, B., 2002. Making the Web Work: Designing Effective Web Applications.
Sams Publishing.
Chin, J.P., Diehl, V.A., Norman, K.L., 1988. Development of an instrument measuring
user satisfaction of the human–computer interface. In: CHI’88: Proceedings of
the SIGCHI conference on Human factors in computing systems. ACM, New
York, NY, USA, pp. 213–218.
Cooper, A., Reimann, R., Cronin, D., 2007. About Face 3: The Essentials of Interaction
Design. John Wiley & Sons.
Fowler, S., Stanwick, V., NetLibrary, I., 2004. Web Application Design Handbook:
Best Practices for Web-based Software. Morgan Kaufmann.
Horton, S., 2005. Access by Design: A Guide to Universal Usability for Web
Designers. New Riders Publishing Thousand Oaks, CA, USA.
Fig. 3. Average numbers of errors, task completion times and satisfaction ratings Klossek, U.M.H., Russell, J., Dickinson, A., 2008. The control of instrumental action
following outcome devaluation in young children aged between 1 and 4 years.
for different required-field markings split by task position.
Journal of Experimental Psychology: General 137 (1), 39–51.
Koyani, S.J., Bailey, R.W., Nall, J.R., Allison, S., Mulligan, C., Bailey, K., Tolson, M.,
2004. Research-Based Web Design and Usability Guidelines. Health and Human
loss during form fill-in from the users perspective: Complete Services Department, Washington, DC.
successful form fill-in takes 41% more time if required-fields are Nielsen, J., 1993. Usability Engineering. Academic Press.
not properly highlighted. It was also shown that, although a gen- Nielsen, J., 1994. Enhancing the explanatory power of usability heuristics. In:
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
eral performance increase from first to second form fill-in exists, pp. 152–158.
this benefit is in addition to any required-field-marker related Nielsen, J., Landauer, T., 1993. A mathematical model of the finding of usability
differences. In other words, colored backgrounds are better mark- problems. In: Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems, 206–213.
ers for required-field than asterisks no matter how well a form is Polson, P.G., Lewis, C.H., 1990. Theory-based design for easily learned interfaces.
known. Human–Computer Interaction 5, 191–220.
An important question is why required-fields with colored Shneiderman, B., Plaisant, C., 2004. Designing the User Interface: Strategies for
effective Human-Computer Interaction. fourth ed. Addison-Wesley.
backgrounds help participants to commit fewer errors? Although
the asterisk is a de-facto standard to indicate required-fields, it
seems the additional saliency provided by the colored background, 1
For interpretation of color in Figs. 1 and 3, the reader is referred to the web
which is of course much larger than the asterisk, has a much better version of this article.
262 S.L. Pauwels et al. / Interacting with Computers 21 (2009) 257–262

Taylor, F.W., 1911. The Principles of Scientific Management. Harper & Brothers Publishing. US Federal Bureau of Justice Assistance, 2008. <http://www.ojp.usdoj.gov/BJA/
Tidwell, J., 2005. Designing Interfaces: Patterns for Effective Interaction Design. O’Reilly. evaluation/glossary/>.
Tullis, T.S., Pons, A., 1997. Designating required vs. optional input fields. In: CHI ’97: Ware, C., 2008. Visual Thinking for Design. Morgan Kaufman.
CHI ’97 Extended Abstracts on Human Factors in Computing Systems. ACM, Wroblewski, L., 2008. Web Form Design: Filling in the Blanks. Rosenfield
New York, NY, USA, pp. 259–260. Media.

View publication stats