MANAGEMENT SCIENCE informs ®

Vol. 54, No. 7, July 2008, pp. 1336–1349 doi 10.1287/mnsc.1070.0838

issn 0025-1909
eissn 1526-5501
08
5407
1336 © 2008 INFORMS

Multiple Criteria Decision Making, Multiattribute

Utility Theory: Recent Accomplishments
and What Lies Ahead
Jyrki Wallenius
Helsinki School of Economics, 00101 Helsinki, Finland, jyrki.wallenius@hse.fi

James S. Dyer
College of Business Administration, University of Texas, Austin, Texas 78712,
[email protected]

Peter C. Fishburn
AT&T Laboratories, Florham Park, New Jersey 07932

Ralph E. Steuer
Terry College of Business, University of Georgia, Athens, Georgia 30602, [email protected]

Stanley Zionts
School of Management, State University of New York, Buffalo, New York 14260, [email protected]

Kalyanmoy Deb
Indian Institute of Technology Kanpur, Kanpur 208016, India, and Helsinki School of Economics,
00101 Helsinki, Finland, [email protected]

T his paper is an update of a paper that five of us published in 1992. The areas of multiple criteria decision
making (MCDM) and multiattribute utility theory (MAUT) continue to be active areas of management
science research and application. This paper extends the history of these areas and discusses topics we believe
to be important for the future of these fields.
Key words: decision making; multiattribute; multiple criteria
History: Accepted by David E. Bell, decision analysis; received February 19, 2007. This paper was with the
authors 2 months for 2 revisions. Published online in Articles in Advance May 19, 2008.

1. Introduction has affected our fields and offers intriguing possi-

About 15 years ago, five of us (Dyer et al. 1992) pub- bilities for the future. We now have web-based soft-
lished a paper exploring the next decade for multiple ware for assisting users (even consumers) in applying
criteria decision making (MCDM) and multiattribute MCDM/MAUT approaches.
utility theory (MAUT). The purpose of the present 2. Substantial growth in applications of MCDM/
paper is to review what has been done since that paper MAUT has occurred. See, for example, the website for
was written and to outline interesting future research Expert Choice (http://www.expertchoice.com),2 the
questions. Much has happened since that paper was recent reviews of applications of decision analysis by
published; this has led to increased researcher mobil- Keefer et al. (2004) and Hämäläinen (2004), the survey
ity and to a more open exchange among researchers of applications of multiobjective evolutionary algo-
around the world concerning theory, software, and rithms by Coello and Lamont (2004), and the results
applications. Some of the more important events that of our bibliometric analysis of published papers.
have occurred and trends that have developed since
1992 that relate to the MCDM/MAUT fields include Americans, in a survey conducted in February–March 2007, 69% of
the following: adult Americans used the Internet on an average day, 41% used
1. Use of the Internet has exploded and comput- a search engine to find information, and 56% used the Internet to
send or read e-mail. Obviously the statistics are quite different for
ing power has continued to grow, as evidenced by
third world countries.
what is commonly referred to as Moore’s Law.1 This 2
Weblinks may change over time. We will try to provide names of
individuals to contact, should the links no longer be active. It is
1
According to Pew Internet & American Life Project, a non- also worth noting that Internet sites are, in general, not an equally
profit research center studying the social effects of the Internet on credible source of scientific information as traditional publications.

1336
Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
Management Science 54(7), pp. 1336–1349, © 2008 INFORMS 1337

3. The importance of MCDM/MAUT has been rec- set of solutions may be either small and finite (as in
ognized in professional management journals. See the choice problems) or large and perhaps infinite (as in
article about Thomas Saaty in Fortune (1999) and the design problems). Uncertainty may be involved. Con-
article on “even swaps” as a rational way of making ceptually, we may assume that a decision maker acts
trade-offs in the Harvard Business Review (Hammond to maximize a utility or value function that depends
et al. 1998). on the criteria or attributes. In cases of uncertainty,
4. The importance of behavioral aspects of decision the problem is typically to maximize the expected
making has grown, and this was recognized by the value of a utility function. We believe that an im-
award of the 2002 Nobel Prize in Economics to Daniel portant part—possibly the most important part—of
Kahneman. It is widely believed that his late col- MCDM/MAUT is the support of decision making in a
league Amos Tversky would have shared this honor. broader sense. MCDM/MAUT methods are intended
5. Data envelopment analysis (DEA) has grown in to help a decision maker think about the problem as
importance and its relationship with multiple objec- part of the decision-making process.
tive linear programming (MOLP) has been explored. There are two categories of MCDM/MAUT prob-
6. Evolutionary multiobjective optimization (EMO) lems: multiple criteria discrete alternative problems
has emerged as a new field with strong ties to and multiple criteria optimization problems. Exam-
MCDM/MAUT. ples of discrete alternative problems include choosing
7. Heuristics in MCDM/MAUT have become more the location for a new airport, selecting a computer
important. network, choosing a drug rehabilitation program, and
8. MCDM/MAUT has begun to penetrate many identifying which nuclear power plant to decom-
new areas of research and applications. Such areas are mission. They are “discrete” alternative problems
DEA, negotiation science, e-Commerce, finance, and because sets of alternatives typically consist of mod-
engineering. estly sized collections of choices, although an emerg-
MCDM/MAUT has several neighboring disci- ing class of discrete problems called multiple criteria
plines, such as decision analysis, mathematical pro- sorting problems may have hundreds of alternatives
gramming, DEA, and negotiation analysis. MCDM/ (see Doumpos and Zopounidis 2002).
MAUT concepts and methods are being developed Examples of optimization problems include river
and used in these neighboring disciplines, provid- basin planning, energy planning, engineering com-
ing healthy cross-fertilization, but making it difficult ponent design, portfolio selection, and R&D project
to draw sharp disciplinary boundaries. In this paper, selection. Feasible sets of alternatives for such prob-
we assess how MCDM/MAUT has interfaced with lems usually consist of a very large number of or
these neighboring disciplines. Another interesting infinitely many alternatives, defined by systems of
trend is that MCDM/MAUT concepts and techniques equations and inequalities that identify the feasible
are increasingly being applied in diverse engineering region for the decision variables. Because the vectors
fields and other application areas. of alternatives may have many components and the
Our objectives are to (a) review what has happened number of equations and inequalities may be large,
since the earlier paper was written, and (b) identify the feasible regions may be complex. Consequently,
exciting directions for future research. Although this multiple criteria optimization problems are likely to
paper is not an exhaustive survey, we include many require relatively more computational resources than
additional references published since our earlier paper discrete problems.
was published. Our paper is oriented toward method- In addition to differences in the feasible sets of alter-
ological dimensions rather than application domains, natives, there are other differences between multiple
because generic methodological advances can be criteria discrete alternative and multiple criteria opti-
employed across a broad spectrum of applications. mization problems. One is that discrete alternative
problems are more likely to be modeled with uncer-
tain values for the attributes or criteria than multiple
2. Our Decision-Making Framework criteria optimization problems. Another difference is
We assume a decision maker3 who chooses one (or in the way utility or value functions are taken into
a subset) of a set of alternatives evaluated on the account. Many approaches to multiple criteria dis-
basis of two or more criteria or attributes. The feasible crete problems attempt to represent aspects of a deci-
sion maker’s utility or value function mathematically
3
The decision maker may be an individual or a group that cooper- and then apply these results to estimate the alter-
ates to act according to the same rational decision-making process natives’ (expected) utilities. In multiple criteria opti-
as one that would be followed by an individual. In some cases, we
will also include a discussion of how MCDM/MAUT methods may
mization, there is usually no attempt to capture the
assist with more complex multiperson decision situations such as decision maker’s utility or value function mathemati-
those encountered in negotiations. cally. Instead, the philosophy is to iteratively elicit and
 Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
1338 Management Science 54(7), pp. 1336–1349, © 2008 INFORMS

use implicit information about the decision maker’s Edwards et al. (2007), Ehrgott (2005), Figueira et al.
preferences to help steer the decision maker to her or (2005), Ignizio and Romero (2003), and Caballero et al.
his most preferred solution. (2007).
By implicit information we mean answers to ques-
tions such as (a) which of two solutions is more 3. Bibliometric Analysis of
preferred, (b) which of several trade-off vectors is MCDM/MAUT
most preferred, and (c) which criterion values can be We have conducted a basic bibliometric study of
relaxed to allow improvements in other criteria. If the MCDM/MAUT using the ISI database. The ISI data-
elicited information enables an interactive procedure base, which covers over 8,650 journals, including a
to converge to a “final solution” in a reasonable num- broad range of disciplines, is updated on a weekly
ber of iterations, then the final solution is one that basis. We report basic statistics regarding how our
is either optimal (most preferred) or good enough to fields have developed based on variations of the fol-
terminate the decision process. lowing key words: multiple criteria decision, mul-
Because of different problem types, different fami- tiattribute utility, multiple objective programming/
lies of approaches have evolved for solving discrete optimization, goal programming, AHP, evolutionary/
alternative problems and multiple criteria optimiza- genetic multiobjective, and vector optimization.
tion problems. Topmost among the approaches for Figures 1 and 2 show the number of publications
solving discrete alternative problems is MAUT as and the number of citations of papers in our fields,
described by Keeney and Raiffa (1976). MAUT and respectively, over the 1970–2006 period and until the
the analytic hierarchy process (AHP), as described in end of June 2007. Growth in the number of publica-
Saaty (1999), would basically constitute what some tions and in the number of citations has been rapid
European scholars call the American school, whereas since 1992 when our earlier paper was published and
methods based on a partial ordering of alternatives, has increased even more dramatically since 2000.
such as ELECTRE (in fact a family of methods) and Roughly 10% of the publications have appeared
PROMETHEE, are examples of the French school. The in the European Journal of Operational Research, with
emphasis on different national schools is unfortunate about 1% each in Management Science and Opera-
because it makes it difficult to classify approaches tions Research. The remainder are widely scattered
developed by international teams (see Olson 1996). over many different journals. Tables 1 and 2 pro-
Approaches for solving multiple criteria optimization vide information about publications by country of
problems fall into the following categories: interac- residence of the first author, and subtopical areas
tive methods, goal programming, vector-maximum within the MCDM/MAUT fields. Although authors
algorithms, and evolutionary procedures. However, from the United States have been most prolific (30.3%
these categories are not necessarily mutually exclu- of the total), the other 69.7% have come from all over
sive. For technical discussions of these and other the world underscoring the international nature of
specific MCDM/MAUT methods, see, for example, MCDM/MAUT research. Among the subtopical areas

Figure 1 Published Items in Each Year

800
750
700
650
600
550
500
450
400
350
300
250
200
150
100
50
0
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007

Years
Note. Year 2007 includes published items until the end of June 2007.
Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
Management Science 54(7), pp. 1336–1349, © 2008 INFORMS 1339

Figure 2 Citations in Each Year

6,500
6,000
5,500
5,000
4,500
4,000
3,500
3,000
2,500
2,000
1,500
1,000
500
0
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
Years
Note. Year 2007 includes citations until the end of June 2007.

within MCDM/MAUT, the environment is listed fifth A bibliometric investigation using subheadings from
reflecting its application potential. Also, engineering a management science textbook as key words (but
and computer science are important, reflecting the omitting our MCDM/MAUT key words) reveals that
broad, interdisciplinary nature of our fields. the number of ISI MS/OR publications roughly dou-
Figure 3 provides information by methodological bled from 1976 to 1986, and quadrupled from 1986
area. The growth in AHP-related publications is enor- to 1996. From 1992 to 2006, the growth has been 1.9
mous, as is the recent growth in EMO publications. By times. For MCDM/MAUT, on the other hand, the
contrast, goal programming and mathematical pro- growth from 1992 to 2006 has been 4.2 times (which
gramming have maintained more stable patterns of is largely due to AHP and EMO publications). Over-
growth. The research related to the French School may all the growth path of MCDM/MAUT appears expo-
have been underestimated because of our difficulty in nential. It is also interesting to compare these growth
finding appropriate key words. paths with the growth of science in general. The
We also compared the growth path of MCDM/ number of publications included in the Science Cita-
MAUT publications with the growth of management tion Index has roughly doubled from 1992 to 2006
science/operations research (MS/OR) publications. (Garfield 2007).
We also compared ISI publication figures for years
1970–1990 and 2002–2006 (5-year period) by subtopi-
Table 1 Publications by Country cal area. Clear shifts are noticeable. The relative share
Number Percent of OR/MS and management and business topics has
decreased about 40%. However, the share of computer
United States 2
097 303 science has increased by some 20%, environment has
China 471 68
United Kingdom 441 64
Canada 351 51 Table 2 Subtopical Areas
Taiwan 329 48
Number Percent
Spain 306 44
India 302 44 Operations research and management science 2
415 349
Germany 264 38 Computer science and artificial intelligence 1
829 265
Japan 241 35 Management and business 1
587 230
Italy 235 34 Applied mathematics, interdisciplinary 1
066 154
Australia 202 29 Environmental 689 100
France 195 28 Industrial engineering 641 93
South Korea 189 27 Manufacturing engineering 405 59
Finland 184 27 Economics 308 45
Netherlands 176 25 Civil engineering 289 42
Others 927 134 Computer science and information systems 270 39
Total 6
910 100 Energy and water resources 267 39
 Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
1340 Management Science 54(7), pp. 1336–1349, © 2008 INFORMS

Figure 3 Publication History: Area of Research web protocol. Tim Berners-Lee originally developed
500 “hypertext” and the first web server for CERN at
AHP the end of 1980s, and by 1991 the World Wide Web
450
Goal programming (WWW) as we know it today was born. Its signifi-
EMO cance became apparent a few years later. It is now
400
MAUT influencing many aspects of human life and is pre-
Math programming senting great challenges and opportunities for our
350
French school profession. We comment on its significance in several
Vector optimization places.
300
Another development that we did not foresee is the
penetration of MCDM/MAUT modeling into engi-
neering, electronic commerce, DEA, and negotiation
250
science. This is exciting, but a concern is that many
200
scholars who apply MCDM/MAUT tools and con-
cepts in their own disciplines may lack knowledge
about existing MCDM/MAUT methods. As a result,
150
they may rediscover work that has already been done
or they may not be aware of useful ideas.
100
4.1.1. Decision Support Applications of MCDM/
50 MAUT. The business world has become more com-
petitive and less predictable, accentuating the im-
0 portance of effective decision making and the use of
–1979 1980– 1985– 1990– 1995– 2000– decision support tools. The decision support systems
1984 1989 1994 1999 2004 that are widely used are user friendly and often
employ spreadsheets, such as Excel, at least for data
doubled, and all engineering areas (except for IE) entry. In fact, most OR/MS textbooks, which typically
have increased considerably. include a chapter on MCDM/MAUT, are now built
around spreadsheets. In early work (certainly in the
1970s and early 1980s) it was necessary to do signifi-
4. Contributions Since Our 1992 Paper cant custom programming to carry out applications.
In this section, we focus on the evolution of the However, by 1997, one could usually accomplish es-
MCDM/MAUT fields since our earlier paper. We sentially the same goals with a small amount of visual
begin with a review of the areas that we identified basic for applications (VBA) programming in Excel.
as opportunities for future research and then discuss (See Kirkwood 1997 and Ragsdale 2004, on how
other changes in these fields that we failed to empha- to carry out such analyses in Excel using VBA and
size or anticipate. Solver.) The following web pages provide examples of
MCDM/MAUT software applications: http://www.
4.1. Areas of Future Research Identified in 1992 decisionarium.tkk.fi/, http://faculty.fuqua.duke.edu/
Our 1992 paper identified four major MCDM/MAUT daweb/dasw.htm/, http://www.logicaldecisions.com/,
areas in which we predicted significant future research http://www.krysalis.co.uk/index.html/, http://www.
contributions: namely, decision support applications, logicaldecisions.com/, http://www.krysalis.co.uk/
incorporation of behavioral research, emphasis on index.html/, and http://www.stratadecision.com/.
robust decisions, and the role of heuristics. Similarly, the invention of the Internet has created
These four areas have experienced significant a need for additional decision support (in a dis-
research productivity, although some more than oth- tributed or even a mobile environment). Many schol-
ers. However, work related to heuristics has outper- ars have implemented their favorite MCDM/MAUT
formed our expectations because its growth came procedures on the Internet. For example, the eclec-
from a source that was not expected; it has grown tic AIM method (Lotfi et al. 1992) now has a web
dramatically because of EMO. version, WebAIM: http://mgt2.buffalo.edu/webaim/
In the early 1990s, we could foresee the continuing (see Wang and Zionts 2005). One of the earliest exam-
spread of personal computers and we could predict ples of a web-based MCDM/MAUT decision support
the popularity of spreadsheet modeling and spread- tool is WWW-NIMBUS (Miettinen and Mäkelä 1995).
sheet solvers. We believe that we also anticipated the Decision support is also available for consumers
upcoming advances in computing power, but not nec- online (for example, Active Decisions, which is dis-
essarily all the consequences. However, we did not cussed in §4.3.3) and for decision conferencing.
know about the popularity and the importance of the Advances in MAUT methods to support decision
Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
Management Science 54(7), pp. 1336–1349, © 2008 INFORMS 1341

conferencing have been facilitated by the increases Sayin (2006), Deb and Gupta (2006), and Wang and
in readily available computing power. See Keefer Zionts (2006), along with the work on rough sets,
et al. (2004), the work by Phillips et al. (http://www. based on Pawlak’s original idea (Pawlak 1982; see also
catalyze.co.uk/), and the work by Hämäläinen et al. Slowinski 1992 and the discussion in §4.2). Further-
(http://www.decisionarium.tkk.fi/) for discussions of more, many interval-valued methods provide robust
applications. solution approaches, including PAIRS for value tree
analysis (Salo and Hämäläinen 1992) and robust port-
4.1.2. Behavioral Considerations. Behavioral is-
folio modeling (Liesiö et al. 2007).
sues have not received a great deal of attention by
MCDM/MAUT researchers in recent years, despite 4.1.4. Role of Heuristics. Heuristics have become
the recognition of this topic and calls for further re- more important in recent years, particularly in the
search by Herbert Simon in the 1950s (Simon 1955). MCDM literature. Many real-world problems are so
Korhonen and Wallenius (1996) provide of survey complex that one cannot reasonably expect to find
of salient behavioral issues and outline a 12-point an exact optimal solution. One broad field involved
research agenda emphasizing the design of interactive with such problems is multiple objective combinato-
decision tools. rial optimization, which attempts to address multiple
Recent work related to behavioral issues includes criteria knapsack, traveling salesman, and schedul-
possible biases in the elicitation of weights (Keeney ing problems. Exact solution methods are augmented
2002; Delquié 1993, 1997; Pöyhönen and Hämäläinen by simulated annealing, tabu search, and local search
2001), and issues related to the design of value techniques. The interested reader is referred to the
trees and the selection of attributes (e.g., Hämäläinen growing literature on this subject as illustrated by
and Alaja 2003). Related work may offer additional Ulungu and Teghem (1994), Ehrgott and Gandibleux
opportunities for research. These include studies that (2000), Jaszkiewicz (2001), and Sun (2003). The rapidly
explore whether knowledge of alternatives and other growing area of evolutionary procedures is related to
contextual issues will bias assessments of the impor- the role of heuristics, but is discussed later.
tance of attributes (see, for example, Carlson and
4.2. Computer Related Topics
Pearo 2004, Carlson and Bond 2006). Increases in computing power have been at the heart
Other research related to behavioral issues includes of many of the advances in MCDM/MAUT. Along
an evaluation of alternative strategies for group with algorithmic advances, larger and more complex
decision making and issues related to organizational problems are now solvable in reasonable time. On
decision making. See Hämäläinen (2004) for a dis- the MCDM side, this has had a major impact on
cussion of some MCDM/MAUT group decision mak- procedures used for solving multiple objective linear,
ing approaches and for some suggestions for research multiple objective nonlinear, and multiple objective
agendas related to them. Matheson and Matheson combinatorial optimization problems.
(2007) discuss organizational issues associated with
the application of decision support models. Further- 4.2.1. Impact of Increased Computational Power.
more, problem structuring continues to be of sig- As an MCDM-oriented benchmark, we use the case
nificant interest. See, for example, Kirkwood (1997), of computing all nondominated extreme points of a
Ragsdale (2004), and the recent special issue of the multiple objective linear program (MOLP) to obtain a
Journal of the Operational Research Society (Vol. 57, discretized representation of its nondominated solu-
Issue 7, 2006) on problem structuring. Scheubrein and tion set. In the early 1990s, problems with only a
Zionts (2006) have demonstrated the importance of a few thousand nondominated extreme points were the
front-end (or problem formulation) system for solving largest that could be solved. Because the nondom-
MCDM problems. inated set grows exponentially with the number of
objectives, this placed upper limits on the problems
4.1.3. Robustness Considerations. Robustness can that could be solved at approximately 40 constraints
be defined in many different ways. One can talk by 80 variables with three objectives, 30 constraints
about robust decisions, robust algorithms, or robust by 60 variables with four objectives, and so forth.
optimization in a deterministic or probabilistic setting. Now with faster computers and improved algorithms,
Generally speaking, robustness refers to the ability of problems with approximately one million nondomi-
a solution to cope with uncertainties including those nated extreme points can be solved in about the same
that may not be anticipated. Many papers have been amount of time, thus extending the size of MOLPs
published in recent years that deal with robustness amenable to vector-maximum algorithms, such as
considerations. Steuer’s (2006) ADBASE to problems with hundreds
Hogarth and Karelaia (2005) provide a recent exam- of constraints and variables.
ple of this work for MAUT. For examples of MCDM Algorithms for computing all nondominated facets
robustness research, see Roy (1998), Kouvelis and of a feasible region are still of interest, but because
 Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
1342 Management Science 54(7), pp. 1336–1349, © 2008 INFORMS

of the computation involved they have yet to find groups of researchers: Fonseca and Fleming’s (1993)
a place in practice. Help in this regard may come multiobjective genetic algorithm, Srinivas and Deb’s
from the normal vector identification ideas set forth (1994) nondominated sorting genetic algorithm, and
in Yang and Li (2002). Also, as suggested in Wiecek the Horn et al. (1994) niched Pareto genetic algorithm.
and Zhang (1997), procedures are being developed for This research helped spur the development of evolu-
partitioning an MOLP so that the task of computing tionary multiobjective computation.
all nondominated extreme point solutions can be sub- Starting with an initial population, an EA updates
divided among parallel processors. This will enable the population by using processes designed to mimic
the computation of MOLPs with millions of nondom- natural survival-of-the-fittest principles and genetic
inated extreme points. variation operators to improve the average popula-
When a nondominated set is described by a large tion from generation to generation in a stochastic
number of points, the task of finding a best or accept- manner. The goal is to converge to a diverse final
able solution still remains. The nondominated set is a population of points that represents the nondomi-
portion of the surface of the feasible region in crite- nated set. EAs are suitable for use on parallel proces-
rion space. The dimensionality of the feasible region sors thereby enhancing their potential. The intriguing
in criterion space is never greater than the number ideas of EAs were developed by people, mostly from
of objectives, but should there be several hundred engineering and computing, who had not formerly
thousand points representing a nondominated sur- been connected with the MCDM/MAUT community.
face, they would be like grains of sand on a beach. Representative references on EMO are provided by
The challenge then is how to work one’s way through Corne et al. (2000), Branke et al. (2001), Zitzler et al.
the grains of sand to find the best one. (2001), Deb (2001, 2003), Fonseca et al. (2003), and
Interactive procedures may be used to enable a Coello and Lamont (2004).
decision maker to move to better and better solutions,
4.2.3. Knowledge Discovery, Preference Mod-
while learning much about the problem through inter-
eling. Machine learning and knowledge discovery
actions with points from the nondominated set on the
techniques have entered the field of preference mod-
way. Descriptions of prominent interactive procedures
eling, mainly as a result of the concept of dominance-
are given in Gardiner and Steuer (1994) and Miettinen
based rough sets (Greco et al. 1999, 2005). Slowinski
(1999). Although designed primarily for continuous
et al. (2002) have provided axioms that differentiate
problems, many of them can be modified for navigat-
among the main families of preference models: util-
ing through large numbers of discrete solution points.
ity functions, outranking relations, and decision rules.
In fact, Korhonen and Karaivanova (1999) have pre-
Preference learning from a sample of past decisions
sented an algorithm that is being extended to handle
leads to a preference model, expressed as a set of
more than a million discrete points.
“if–then” decision rules that can be used for descrip-
Although MAUT does not in general have the com-
tive as well as prescriptive purposes. An important
putational demands of MCDM, the main benefit of
feature of the dominance-based rough set approach
advances in computing power has been to facilitate
(DRSA) is the possibility of handling inconsistencies
more responsive user-friendly interfaces with greater
in past decisions. The DRSA concept allows classifi-
functionalities. This has led to a substantial increase
cation of the decision rules into “certain” and “doubt-
in applications of MAUT methods, particularly those
ful” rules, the latter resulting from inconsistencies.
involved in web-based decision support and decision
DRSA, based on the rational principle of dominance,
conferencing.
exploits ordinal properties of evaluations solely and
4.2.2. Evolutionary Multiobjective Optimization. seeks to be transparent to users. In addition to the-
Despite prior advances, multiple criteria optimiza- oretical research, the rough sets research community
tion techniques were unable to solve many highly has vigorously sought to apply its ideas for solv-
nonlinear multiple criteria problems that were begin- ing practical problems. Preference learning underlines
ning to crop up in earnest, mostly in engineer- and strengthens the links between artificial intelli-
ing, in the early 1990s. Whereas the Russian PSI gence and MCDM.
(parameter space investigation) method of Statnikov
and Matusov (2002) was designed earlier for some 4.3. New Application Areas of MCDM/MAUT
of these problems, an entirely new approach—one Methods
drawn from genetic algorithms—called evolutionary MCDM methods and ideas have been integrated
algorithms (EAs) has been developed. Schaffer (1984) into new areas such as DEA, negotiation science,
suggested the first multiobjective evolutionary algo- e-Commerce, spatial modeling, and, on a broad level,
rithm called vector evaluated genetic algorithm. engineering. There is every reason to believe that this
Roughly 10 years later, three working evolution- trend will continue. We now review some of these
ary algorithms were suggested by three different developments.
Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
Management Science 54(7), pp. 1336–1349, © 2008 INFORMS 1343

4.3.1. DEA. Charnes and Cooper conducted the problem area. Teich et al. (1994) classify modeling aids
pioneering research both in goal programming and as follows: (1) whether or not an attempt is made to
DEA (Charnes and Cooper 1961, Charnes et al. 1978). explicitly construct the participants’ value functions;
Yet, MCDM and DEA developed separately; Belton (2) whether participants are requested to make con-
(1992) and Doyle and Green (1993) described the rela- cessions from their preferred positions or whether we
tionships between the two. Subsequently, Joro et al. seek to identify jointly preferred (“win-win”) alterna-
(1998) developed a detailed understanding of the tives from a single negotiating text (Fisher 1978). This
structural (mathematical) relationship between DEA classification leads to four different modeling aids:
and MOLP, and noted the close similarities that exist. 1. Value function and concession based models.
One of the basic differences is the radial projection 2. Value function based and Pareto improvement
used in DEA and the more general nonradial projec- seeking models.
tion used in MOLP. In other words, DEA mechan- 3. Interactive models based on concession making.
ically (radially) extends the ray from the origin via 4. Interactive models seeking Pareto improvements.
the point representing the decision-making unit being
Each category offers interesting research opportu-
evaluated to the efficient envelope when calculating
nities. Ideas and tools have been picked eclectically
efficiency scores. MOLP techniques are more generic
from MCDM/MAUT. See, for example, Ehtamo et al.
and can be used to identify benchmarking units on
(1999) and Ehtamo and Hämäläinen’s (2001) review
the efficient frontier.
paper. The problem area is rich and includes novel
Of course, DEA and MOLP usually have differ-
ent purposes: DEA is used for performance measure- applications of web-based negotiations. Wang and
ment, whereas MOLP is used for aiding choice. The Zionts (2008) have developed an approach for quanti-
observation about the structural similarity between fying certain aspects of negotiations and have shown
DEA and MOLP has sparked synergistic advances in that it is generally better to negotiate on multiple
both models. For example, MOLP can be used to find alternatives at the same time. Yet, documented prac-
nonradially projected targets on the efficient frontier tical applications are few.
(Korhonen and Syrjänen 2004). Originally, DEA did 4.3.3. e-Commerce: Multiattribute Auctions and
not consider the preferences of the decision maker.
Shopping Agents. Geoffrion and Krishnan (2001)
This created problems because a unit could be effi-
have summarized the prospects for operations re-
cient by being best in terms of one output measure,
search in the electronic commerce era. MCDM/
which could be the least important from a decision
MAUT plays an important role in many of these
maker’s point of view. As a way to incorporate prefer-
developments. We point out two novel application
ences into DEA, researchers have suggested the use of
weight restrictions. MOLP models can be used to gen- areas for MCDM/MAUT: multiattribute online auc-
erate novel ways of incorporating a decision maker’s tions and comparison shopping agents. Both topics
preferences into DEA (Halme et al. 1999). On the other are widely discussed in the popular press and the aca-
hand, DEA provides new application areas for MOLP demic literature.
researchers. There is an increasing awareness that price-only
auctions are overly simplistic for most real-world pur-
4.3.2. Negotiation Science. The literature on
chasing situations. One cannot necessarily ignore the
negotiation and group decision making is broad and
attributes of alternatives such as quality, terms of
diverse. The field is multidisciplinary, involving dif-
delivery, and warranty, which naturally leads to the
ferent approaches by social psychologists, economists,
definition of multiattribute auctions. This notion has
and management scientists. A few common threads
been explored in the context of a reverse auction by
have become accepted in all of these areas. One
of them is the concept of a contract curve. It was Chen-Ritzo et al. (2005) and Teich et al. (2004).
introduced by economists, refined in game theory, A central issue underlying multiattribute auctions is
and then reworked by social psychologists. Later, the elicitation of the auction owner’s preferences over
management scientists developed models attempting the relevant attributes/issues. Teich et al. (2004) out-
to locate solutions on the contract curve. Other line approaches for accomplishing this task, includ-
common threads include the assumption of limited ing (a) eliciting a value function, (b) using the auction
negotiator rationality and the misrepresentation of owner specified preference path, and (c) “pricing out”
negotiators’ preferences. (or “costing out”) all other dimensions (attributes)
Raiffa (1982) is a pioneer in this field. For a more besides price and quantity. Approaches (a) and (c)
recent discussion of negotiation theory and its rela- have been implemented in web-based auction sys-
tionship to decision analysis and game theory, see tems. The auction owner specified preference path is
Sebenius (2007). Following Raiffa’s footsteps, several a simple, untested idea. Teich et al. (2004) recommend
MCDM/MAUT researchers have contributed to this the use of pricing out in the context of online auctions,
 Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
1344 Management Science 54(7), pp. 1336–1349, © 2008 INFORMS

since this dramatically simplifies the auctions, effec- 4.3.5. Engineering Applications. MCDM/MAUT
tively reducing bids to quantity-price pairs in multiu- is used in many fields of engineering. Often, how-
nit auctions. In fact, pricing out is an old technique ever, the application of MCDM tools in engineering is
often used by practicing decision analysts (see Keeney based on simple scoring models. Examples of schol-
and Raiffa 1976). arly applications include river basin development
Early shopping agents were designed to find the and management, water regulation, chemical pro-
“best” deals for buyers, focusing solely on the price cess optimization, aircraft wing design, environmen-
of the merchandise. Given that all the price informa- tal planning and management, and radiation therapy
tion was posted on the Internet, the shopping agents planning. See the Systems Analysis Laboratory web
would find the best deals, and consumers would have page (http://www.sal.hut.fi/) at the Helsinki Univer-
no reason to buy elsewhere. As a result, many feared sity of Technology for several such MCDM/MAUT
price wars. These price wars did not occur, however, applications and Coello and Lamont (2004) for EMO
because purchase decisions are based not only on applications. Also see Hobbs and Meier (2000) for
price but also on other attributes related to notions an extensive coverage of the use of MCDM/MAUT
methods in energy and environmental decisions.
of quality and service. More sophisticated shopping
Sophisticated applications of MAUT have appeared
agents, incorporating buyer’s preferences over multi-
in the military/nuclear energy sector, notably related
ple attributes are being developed and evaluated (Sim
to the critical issues associated with nuclear weapons.
and Choi 2003). For example, the tête-à-tête shopping
Von Winterfeldt and Schweitzer (1998) evaluated
agent models and uses the buyer’s utility function
alternatives for the replenishment of tritium in the
(Maes et al. 1999). U.S. nuclear weapons stock pile. Ten alternatives were
Active Decisions (http://www.activebuyersguide. evaluated based on the criteria of production assur-
com) provides a sophisticated example of the im- ance, cost, and environmental impacts. This analysis
plementation of MCDM/MAUT concepts in the busi- was influential in supporting the final recommenda-
ness-to-consumer environment, and offers decision tion by the U.S. Secretary of Energy.
support for a large number of products and services. In related work, a team of U.S. decision analysts
The website allows consumers to select from a list of was commissioned by the Department of Energy’s
possible product attributes and create a multiattribute Office of Fissile Materials Disposition to develop a
preference model to rank order alternatives and facil- multiattribute utility model to evaluate alternatives
itate comparisons. When utilizing the decision guide for disposing excess plutonium. Subsequent to the
(as of September 20, 2003) to analyze digital cam- U.S. study, Russian scientists modified the model with
eras, the user could consider eleven possible product the aid of the U.S. team, and used it to evaluate
attributes. A decision maker could choose a subset of Russian disposition alternatives (Butler et al. 2005).
product attributes from this list, and then answer a The MAUT analysis also highlighted the desirability
series of questions to determine the weights on the of parallelism between U.S. and Russian plutonium
selected attributes. disposition technologies. The Russians have decided
to replicate the design of the U.S. disposition facility
4.3.4. Geographic Information Systems. Many in Russia, contributing to the synergy in the disposi-
real-world spatial planning and management prob- tion policies.
lems give rise to MCDM problems based on the Furthermore, many papers deal with relating a
use of geographic information systems (GIS). Exam- product’s design parameters to performance mea-
ples include environmental planning, urban planning, sures. Typically, a value function is optimized.
facilities location, real estate, and retailing. In fact, the Köksalan and Plante (2003) have proposed a method
Research Group on Geographic Information and Mul- that accounts for multiple criteria (maximization of
ticriteria Decision Analysis (GIMDA) and the online process yield, maximization of process capability,
Journal of Geographic Information and Decision Analy- minimization of process costs) via a procedure that
sis (GIDA) have been established to explore the links interacts with and relies on the preferences of a deci-
and synergies between MCDM/MAUT and GIS. The sion maker. The procedure blends ideas from research
GIS technology offers unique capabilities for analyz- in multiple-response design, multiple criteria opti-
ing spatial decision problems and handling spatial mization, and global optimization. According to the
data. MCDM/MAUT, on the other hand, offers use- authors, the concepts and methods developed can be
ful tools and concepts that incorporate preferences applied to problems such as supply chain manage-
into GIS-based decision making. As the mission of ment and multidisciplinary design optimization.
GIMDA states, there is a need for developing a uni-
fied framework for GIS-based MCDM/MAUT deci- 5. Areas for Future Research
sion making. For further information, see the GIDA The penetration of MCDM/MAUT concepts to the
Journal or the book by Malczewski (1999). areas indicated above continues at an increasing rate,
Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
Management Science 54(7), pp. 1336–1349, © 2008 INFORMS 1345

providing excellent opportunities for involvement MCDM/MAUT models, as discussed by Montibeller

for MCDM/MAUT researchers. The previous section et al. (2008) and by Butler et al. (2006).
commented on past research as well as ongoing and
future research. In many cases, the Internet will con- 5.2. Revisiting Targets
tinue to provide additional challenges and opportu- Another emerging area is the use of targets in prob-
nities. In this section, we identify several other areas lems involving multiple criteria. The idea is that the
with the potential for interesting future research. decision maker’s utility or value may not depend on
the levels of performance on different criteria, but
5.1. Mental Models instead on whether the levels meet a target or thresh-
One attractive area is to develop mental models of old on one or more criteria. MCDM scholars have
decision problems that relate a decision maker’s per- been aware of this notion for many years; the notion
ception of a problem to her fundamental objectives. provides a basis for goal programming. In recent
In his work on value-focused thinking, Keeney (1992) years, several contributions to MAUT have exam-
made the distinction between a hierarchy of funda- ined this idea in more detail, particularly in situa-
mental objectives linked to preferences, and models of tions where performance and goal levels themselves
interactions of attributes of an alternative that might may be uncertain. For example, typical attributes in
be used to predict the performance of an alternative new product choice may be cost, quality, and dif-
on each of these attributes. Keeney identified the latter ferent features, and the corresponding targets may
as means-ends models to distinguish them from true be the best performance on these attributes relative
preference models. The idea of means-ends models to the competition (see Bordley and Kirkwood 2004,
was anticipated by Keeney and Raiffa (1976) in their Tsetlin and Winkler 2007). It would seem natural to
discussion of “proxy attributes.” They noted that sev- revisit MCDM models involving goals or targets for
eral attributes may relate to one fundamental objec- attributes in light of these new studies. Related issues
tive and suggested that the decision maker will have that could be addressed would be methods for select-
to think hard about these interactions when assessing ing targets in ways that are incentive compatible in
attribute weights. However, they did not address the organizations.
question of how these “mental models” of impacts of
5.3. Decision Support in a Distributed
attributes on objectives might be quantified.
Environment
For situations with discrete levels of both attributes
Decision support in a distributed environment is
and objectives, Nadkarni and Shenoy (2001, 2004)
somewhat different from what we had expected in
provide an assessment procedure using a causal map-
the early 1990s. For example, what is a user? Our
ping approach to constructing Bayesian networks.
fields have traditionally sought to support corporate
The advantage of their procedure is that it explic-
managers. However, household consumers need sup-
itly measures uncertainty inherent in mental mod- port for purchasing decisions. What kind of decision
els, but the method becomes complex if the scale of support do they want in an Internet or mobile envi-
measurement is not discrete. When the objectives and ronment? The problem may not be one of having
attributes are measured on a continuous scale, we insufficient information, but rather one of having
need to ask a decision maker how much “impact” “too much” or an unknown quality of informa-
a criterion has on an objective. For example, the tion. We may have to filter information. This is
“appearance” of a car may have some impact on the a potential application and development area for
objective of prestige but not on the objective of mini- MCDM/MAUT.
mum cost. This is a different notion from preference,
but one that could be modeled, is intuitive, and could 5.4. Quadratic and Stochastic Programming
be measured on a ratio scale. Significant progress may be anticipated in computing
Ideas and tools that might be used for explicit as- the nondominated set of a multiple objective problem
sessment of means-ends models include conjoint anal- that is all linear except for one or two of its objec-
ysis or statistical models to estimate trade-offs or tives being quadratic. Instead of the nondominated
“weights” among attributes. Direct assessments by set consisting of flat faces as in an MOLP, we know
subjects might also be used, and it is worthwhile to from the research of Steuer et al. (2005) that the non-
note that such assessments may be ratio judgments, dominated set is made up of paraboloidic platelets.
since the notion of an impact does have a natural With an algorithm possible for computing all such
zero and therefore a natural ratio scale. This perspec- platelets, combined with the option to populate each
tive may provide a link between multiattribute utility platelet with additional points, a densely populated
theory and the AHP, if the AHP is viewed as mea- discretized representation of the nondominated set of
suring the relative impacts of attributes on objectives. such a problem will soon be possible. The task of find-
Some efforts have been made to link these concepts to ing the most preferred nondominated point will then
 Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
1346 Management Science 54(7), pp. 1336–1349, © 2008 INFORMS

be similar to finding the best in a discretized nondom- the solution process (see Phelps and Köksalan 2003,
inated set produced by an evolutionary algorithm. Tan et al. 2005, Köksalan and Phelps 2007, Deb and
The capability to compute nondominated sets Sundar 2006).
for “almost-linear” problems should allow MCDM/ Collaborative efforts between EMO and MCDM/
MAUT to move into the area of multiobjective MAUT researchers, notably the Dagstuhl Confer-
stochastic programming, where the quadratic objec- ences, have begun to focus on using ideas from
tives might represent variance (Caballero et al. 2001). both fields. When solving higher dimensional prob-
This should open up multiobjective stochastic pro- lems, more effective visualization techniques must
grams to new multiple criteria applications in finance, also be developed to represent multiobjective data
as suggested, for instance, by Aouni et al. (2006), to the decision maker. For this problem, the EMO
Bana e Costa and Soares (2001), and Hallerbach and and MCDM/MAUT communities may collaborate in
Spronk (2002). a mutually beneficial manner. Lotov et al. (2004) pro-
vide possibly useful ideas.
5.5. Usefulness of Dotted Representations of
Nondominated Sets 5.7. Challenges from Practice
Since the early days of MCDM, there have basically Kasanen et al. (2000) studied several real-world man-
been two approaches for solving multiple criteria agerial decision situations and drew conclusions for
optimization problems. One is to compute a represen- MCDM/MAUT research. Without making claims of
tation of the nondominated set, often in the form of being “representative,” the authors felt that the pro-
“dots” (i.e., discrete solutions), and then spend time cesses they studied were not uncommon in the real
searching the representation for a final solution. The world. In the absence of a formalized decision pro-
other is to accumulate knowledge about the nondom- cess, decision makers did not know or generate all,
inated set by sequentially probing or sampling from or even a reasonable subset, of options, evaluate
the set until a final solution can be identified. With consequences in terms of explicitly stated criteria,
improved computers and algorithms, we see the two and make truly informed decisions. Even if aided by
approaches blending together. When dotted represen- formal decision models, decision makers may not
tations involve many points, interactive procedures want to or may not be able to act according to
can be modified to sample from large sets of given the MCDM/MAUT paradigm. Hence there may be
points as effectively as from nondominated sets only good reasons to broaden the MCDM/MAUT frame-
known implicitly and for which they were originally work to better reflect real-world decision processes,
designed. Packages such as MATLAB take only a few as described by organizational theorists.
seconds to identify which vector out of a million is
closest to another vector. Benson and Sun (2000) and
Sayin (2003) have worked on new methods that might 6. Conclusions
be able to obtain dotted representations of the non- The conclusions of Dyer et al. (1992) remain valid.
dominated sets of large MOLPs in reasonable time. Extensive research has been done in the fields of
MCDM, MAUT, and related decision support systems
5.6. Evolutionary Multiobjective Optimization in the past decade, as documented by our bibliomet-
We anticipate that EMO and MCDM/MAUT will ric analysis. In addition, our field has matured, and
move closer together to enhance both research pro- developments and methodologies are continuing to
grams. The main thrust of EMO research and appli- move into other fields. Engineering is an example,
cation has been to find well-distributed sets of as discussed earlier, and this increases the benefits
nondominated solutions for problems involving only of expanded communication among research groups
a few (often just two) objectives. When efforts in different disciplines who may be applying these
are made to develop well-distributed sets of non- ideas. Some of the online shopping agents, for exam-
dominated solutions for problems involving more ple, incorporate some MCDM work. One of us has
objectives, EMO methodologies and MCDM/MAUT tried to cooperate with the management of Consumer
techniques may be blended in a synergistic manner. Reports magazine to have them use a state-of-the-art
There are at least two ways to do this. First, an method for ranking products, but has not yet been
EMO may be applied to generate a set of nondom- successful. Nevertheless, there may be similar oppor-
inated solutions, and then an MCDM/MAUT proce- tunities to cooperate with other groups in developing
dure may be used to choose a most preferred solution ranking systems, using ideas similar to the ones illus-
from the set. This process is, however, rather labor trated by Keeney et al. (2006) for ranking academic
intensive. Another way is to embed MCDM/MAUT programs.
principles within an EMO through a utility or fit- Finance may be another field with the potential
ness measure, so that the interaction with the deci- for some additional contributions from the MCDM/
sion maker commences right from the beginning of MAUT community. Monte Carlo simulation and
Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
Management Science 54(7), pp. 1336–1349, © 2008 INFORMS 1347

dynamic programming are examples of management Caballero, R., E. Cerdá, M. M. Muñoz, L. Rey, I. M. Stancu-
science tools that are now widely used to support Minasian. 2001. Efficient solution concepts and their relations
in stochastic multiobjective programming. J. Optim. Theory
research in finance. We would encourage MCDM/ Appl. 110 53–74.
MAUT researchers to become more familiar with the Carlson, K. A., S. D. Bond. 2006. Improving preference assessment:
theory of corporate finance in order to take advantage Limiting the effect of context through pre-exposure to attribute
of similar opportunities. levels. Management Sci. 52 410–421.
We do forecast that MCDM/MAUT will continue to Carlson, K., L. Pearo. 2004. Limiting predecisional distortion by
move into different fields. We see more Internet-based prior valuation of attribute components. Organ. Behav. Human
Decision Processes 94 48–59.
approaches available to support many types of deci-
Charnes, A., W. W. Cooper. 1961. Management Models and Industrial
sions, including consumer search. We believe that the Applications of Linear Programming. Wiley, New York.
Internet will continue to provide challenges as well as Charnes, A., W. W. Cooper, E. Rhodes. 1978. Measuring efficiency
benefits for the MCDM/MAUT research community, of decision making units. Eur. J. Oper. Res. 2 429–444.
besides being a distribution outlet for research and Chen-Ritzo, C., T. Harrison, A. Kwasnica, D. Thomas. 2005. Better,
software. Computers will have ever more sophisti- faster, cheaper: An experimental analysis of a multiattribute
reverse auction mechanism with restricted information feed-
cated programs to elicit interactively and visually the back. Management Sci. 51 1753–1762.
most salient features of decision problems, including Coello, C. A. C., G. B. Lamont. 2004. Applications of Multi-Objective
decision-maker preferences. In fact, we believe that Evolutionary Algorithms. World Scientific, Singapore.
the interactive mode of working with the decision Corne, D., J. Knowles, M. Oates. 2000. The Pareto envelope-based
maker will become standard. We also envision that selection algorithm for multiobjective optimization. Proc. Sixth
several subfields, which developed rather indepen- Internat. Conf. Parallel Problem Solving from Nature, Springer-
Verlag, Heidelberg, Germany, 839–848.
dently, such as EMO, goal programming, and multi-
Deb, K. 2001. Multi-Objective Optimization Using Evolutionary Algo-
objective decision analysis will provide opportunities rithms. Wiley, Chichester, UK.
for collaboration with MCDM/MAUT researchers, Deb, K. 2003. Unveiling innovative design principles by means of
leading to synergistic advances and less fragmenta- multiple conflicting objectives. Engrg. Optim. 35 445–470.
tion of these fields. Deb, K., H. Gupta. 2006. Introducing robustness in multi-objective
optimization. Evolutionary Comput. 14 463–494.
Acknowledgments Deb, K., J. Sundar. 2006. Reference point based multi-objective opti-
mization using evolutionary algorithms. Proc. Eighth Annual
The authors thank Craig W. Kirkwood, Arizona State Uni-
Conf. Genetic and Evolutionary Comput., ACM, New York,
versity; Murat Köksalan, Middle East Technical University; 635–642.
and Roman Slowinski, Poznan University of Technology, as Delquié, P. 1993. Inconsistent trade-offs between attributes: New
well as two anonymous reviewers for valuable comments. evidence in preference assessment biases. Management Sci. 39
1382–1395.
Delquié, P. 1997. Bi-matching: A new preference assessment method
References to reduce compatibility effects. Management Sci. 43 640–658.
Aouni, B., F. Ben Abdelaziz, R. El-Fayedh. 2006. Chance constrained Doumpos, M., C. Zopounidis. 2002. Multicriteria Decision Aid
compromise programming for portfolio selection. Working Classification Methods. Kluwer Academic, Dordrecht, The
paper, Laboratoire LARODEC, Institut Superieur de Gestion, Netherlands.
Tunis, Tunisia. Doyle, R. H., J. R. Green. 1993. DEA and MCDM. Omega 6 713–715.
Bana e Costa, C. A., J. O. Soares. 2001. Multicriteria approaches for Dyer, J. S., P. C. Fishburn, R. E. Steuer, J. Wallenius, S. Zionts. 1992.
portfolio selection: An overview. Rev. Financial Markets 4 19–26. Multiple criteria decision making, multiattribute utility theory:
Belton, V. 1992. Integrating data envelopment analysis with mul- The next ten years. Management Sci. 38 645–654.
tiple criteria decision analysis. A. Goicoechea, L. Duckstein, Edwards, W., R. Miles, D. von Winterfeldt, eds. 2007. Advances
S. Zionts, eds. Proc. IXth Internat. Conf. Multiple Criteria Decision in Decision Analysis. Cambridge University Press, Cambridge,
Making. Springer, Berlin, 71–79. New York.
Benson, H. P., E. Sun. 2000. Outcome space partition of the weight Ehrgott, M. 2005. Multicriteria Optimization. Springer-Verlag, Berlin.
set in multiobjective linear programming. J. Optim. Theory Appl. Ehrgott, M., X. Gandibleux. 2000. An annotated bibliography of
105 17–36. multiobjective combinatorial optimization. OR Spektrum 22
Bordley, R., C. W. Kirkwood. 2004. Multiattribute preference anal- 425–460.
ysis with performance targets. Oper. Res. 52 823–835. Ehtamo, H., R. P. Hämäläinen. 2001. Interactive multiple-criteria
Branke, J., T. Kaussler, H. Schmeck. 2001. Guidance in evolutionary methods for reaching Pareto optimal agreements in negotia-
multi-objective optimization. Adv. Engrg. Software 32 499–507. tions. Group Decision Negotiation 10 475–491.
Butler, J. C., J. S. Dyer, J. Jia. 2006. Using attributes to predict objec- Ehtamo, H., R. P. Hämäläinen, P. Heiskanen, J. Teich, M. Verkama,
tives in preference models. Decision Anal. 3 100–116. S. Zionts. 1999. Generating Pareto solutions in two-party nego-
Butler, J. C., A. N. Chebeskov, J. S. Dyer, T. A. Edmunds, J. Jia, tiations by adjusting artificial constraints. Management Sci. 45
V. I. Oussanov. 2005. The United States and Russia evaluate 1697–1709.
plutonium disposition options with multiattribute utility the- Figueira, J., S. Greco, M. Ehrgott, eds. 2005. Multiple Criteria Decision
ory. Interfaces 35 88–101. Analysis: State of the Art Surveys. Kluwer Academic, Dordrecht,
Caballero, R., T. Gómez, F. Ruiz. 2007. Goal programming: Realis- The Netherlands.
tic targets for the near future. Working paper, Department of Fisher, R. 1978. International Mediation: A Working Guide. Interna-
Economics, University of Malaga, Malaga, Spain. tional Peace Academy, New York.
 Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
1348 Management Science 54(7), pp. 1336–1349, © 2008 INFORMS

Fonseca, C. M., P. J. Fleming. 1993. Genetic algorithms for multi- Keeney, R., H. Raiffa. 1976. Decisions with Multiple Objectives: Pref-
objective optimization: Formulation, discussion, and general- erences and Value Tradeoffs. Wiley, New York.
ization. Proc. Fifth Internat. Conf. Genetic Algorithms, Morgan Keeney, R. L., K. E. See, D. von Winterfeldt. 2006. Evaluating aca-
Kaufmann, San Mateo, CA, 416–23. demic programs: With applications to U.S. graduate decision
Fonseca, C., P. J. Fleming, E. Zitzler, K. Deb, L. Thiele, eds. 2003. science programs. Oper. Res. 54 813–828.
Proc. Second EMO-03 Conf., Lecture Notes in Computer Science, Kirkwood, C. W. 1997. Strategic Decision Making: Multiobjective Deci-
Vol. 2632. Springer, Berlin. sion Analysis with Spreadsheets. Duxbury Press, Belmont, CA.
Fortune. 1999. Staying smart. (May 10), http://money.cnn.com/ Köksalan, M., S. Phelps. 2007. An evolutionary metaheuristic for
magazines/fortune/fortune_archive/1999/05/10/259560/index. approximating preference-nondominated solutions. INFORMS
htm. J. Comput. 19 291–301.
Gardiner, L. R., R. E. Steuer. 1994. Unified interactive multiple Köksalan, M., R. D. Plante. 2003. Interactive multi-criteria optimiza-
objective programming. Eur. J. Oper. Res. 74 391–406. tion for multiple response product and process design. Manu-
Garfield, E. 2007. Charting the growth of science. Presented at the facturing Service Oper. Management 5 334–347.
Chemical Heritage Foundation (May 17). Korhonen, P., J. Karaivanova. 1999. An algorithm for projecting a
Geoffrion, A., R. Krishnan. 2001. Prospects for operations research reference direction onto the nondominated set of given points.
in the e-business era. Interfaces 30 6–36. IEEE Trans. Systems, Man, Cybern. A 29 429–435.
Greco, S., B. Matarazzo, R. Slowinski. 1999. The use of rough Korhonen, P., M. Syrjänen. 2004. Resource allocation based on effi-
sets and fuzzy sets in MCDM. T. Gal, T. Stewart, T. Hanne, ciency analysis. Management Sci. 50 1134–1144.
eds. Advances in Multiple Criteria Decision Making. Kluwer Aca-
Korhonen, P., J. Wallenius. 1996. Letter to the editor: Behavioral
demic, Dordrecht, The Netherlands, 14.1–14.59.
issues in MCDM: Neglected research questions. J. Multi-Criteria
Greco, S., B. Matarazzo, R. Slowinski. 2005. Decision rule approach. Decision Anal. 5 178–182.
J. Figueira, S. Greco, M. Ehrgott, eds. Multiple Criteria Decision
Kouvelis, P., S. Sayin. 2006. Algorithm robust for the bicriteria dis-
Analysis: State of the Art Surveys. Springer, New York, 507–608.
crete optimization problem: Heuristic variations and computa-
Hallerbach, W. G., J. Spronk. 2002. The relevance of MCDM for tional evidence. Ann. Oper. Res. 147 71–85.
financial decisions. J. Multi-Criteria Decision Anal. 11 187–195.
Liesiö, J., P. Mild, A. Salo. 2007. Preference programming for robust
Halme, M., T. Joro, P. Korhonen, S. Salo, J. Wallenius. 1999. A value portfolio modeling and project selection. Eur. J. Oper. Res. 181
efficiency approach to incorporating preference information in 1488–1505.
data envelopment analysis. Management Sci. 45 103–115.
Lotfi, V., T. Stewart, S. Zionts. 1992. An aspiration-level interactive
Hämäläinen, R. P. 2004. Reversing the perspective on applications model for multiple criteria decision making. Comput. Oper. Res.
of decision analysis. Decision Anal. 1 26–31. 19 671–681.
Hämäläinen, R. P., S. Alaja. 2003. The threat of biases in envi- Lotov, A., V. A. Bushenkov, G. K. Kamenev. 2004. Interactive Decision
ronmental decision analysis. Research reports, E12, Systems Maps: Approximation and Visualization of Pareto Frontier. Kluwer
Analysis Laboratory, Helsinki, Finland, http://www.e-reports. Academic, Norwell, MA.
sal.hut.fi.
Maes, P., R. Guttman, A. Moukas. 1999. Agents that buy and sell.
Hammond, J. S., R. L. Keeney, H. Raiffa. 1998. Even swaps: A ratio- Comm. ACM 42 81–91.
nal method for making trade-offs. Harvard Bus. Rev. 76 137–149.
Malczewski, J. 1999. GIS and Multicriteria Decision Analysis. Wiley,
Hobbs, B. F., P. Meier. 2000. Energy Decisions and the Environment:
New York.
A Guide to the Use of Multicriteria Methods. Kluwer Academic,
Norwell, MA. Matheson, D., J. E. Matheson. 2007. From decision analysis to the
decision organization. W. Edwards, R. Miles, D. von Winter-
Hogarth, R., N. Karelaia. 2005. Simple models for multiattribute
feldt, eds. Advances in Decision Analysis. Cambridge University
choice with many alternatives: When it does and does not pay
Press, New York, 419–450.
to face trade-offs with binary attributes. Management Sci. 51
1860–1872. Miettinen, K. 1999. Nonlinear Multiobjective Optimization. Kluwer
Academic, Boston.
Horn, J., N. Nafploitis, D. Goldberg. 1994. A niched Pareto genetic
algorithm for multiobjective optimization. Proc. First IEEE Conf. Miettinen, K., M. M. Mäkelä. 1995. Interactive bundle-based
Evolutionary Comput., IEEE Service Center, Piscataway, NJ, method for nondifferentiable multiobjective optimization:
82–87. NIMBUS. Optimization 34 231–246.
Ignizio, J., C. Romero. 2003. Goal programming. H. Bidgoli, ed. Montibeller, G., V. Belton, F. Ackermann, L. Ensslin. 2008. Rea-
Encyclopedia of Information Systems, Vol. 2. Academic Press, San soning maps for decision aid: An integrated approach for
Diego, 489–500. problem-structuring and multi-criteria evaluation. J. Oper. Res.
Society 59 575–589.
Jaszkiewicz, A. 2001. Multiple objective metaheuristic algorithms
for combinatorial optimization. Habilitation thesis, 360, Poznan Nadkarni, S., P. Shenoy. 2001. A Bayesian network approach to
University of Technology, Poznan, Poland. making inferences in causal maps. Eur. J. Oper. Res. 128
Joro, T., P. Korhonen, J. Wallenius. 1998. Structural comparison of 479–498.
data envelopment analysis and multiple objective linear pro- Nadkarni, S., P. Shenoy. 2004. A causal mapping approach to
gramming. Management Sci. 44 962–970. constructing Bayesian networks. Decision Support Systems 38
Kasanen, E., H. Wallenius, J. Wallenius, S. Zionts. 2000. A study of 259–281.
high-level managerial decision processes, with implications for Olson, D. 1996. Decision Aids for Selection Problems. Springer-Verlag.
MCDM research. Eur. J. Oper. Res. 120 496–510. Pawlak, Z. 1982. Rough sets. Internat. J. Comput. Inform. Sci. 11
Keefer, D., C. W. Kirkwood, J. Corner. 2004. Perspective on decision 341–356.
analysis applications. Decision Anal. 1 4–22. Phelps, S., M. Köksalan. 2003. An interactive evolutionary meta-
Keeney, R. 1992. Value-Focused Thinking: A Path to Creative Decision heuristic for multiobjective combinatorial optimization. Man-
Making. Harvard University Press, Cambridge, MA. agement Sci. 49 1726–1738.
Keeney, R. 2002. Common mistakes in making value trade-offs. Pöyhönen, M., R. P. Hämäläinen. 2001. On the convergence of mul-
Oper. Res. 50 935–945. tiattribute weighting methods. Eur. J. Oper. Res. 129 569–585.
Wallenius et al.: Multiple Criteria Decision Making and Multiattribute Utility Theory
Management Science 54(7), pp. 1336–1349, © 2008 INFORMS 1349

Ragsdale, C. T. 2004. Spreadsheet Modeling and Decision Analysis: Steuer, R. E. 2006. ADBASE: A multiple objective linear program-
A Practical Introduction to Management Science. South-Western ming solver for all efficient extreme points and unbounded
College, Boston. efficient edges. Technical report, Terry College of Business,
Raiffa, H. 1982. The Art and Science of Negotiation. Harvard Univer- University of Georgia, Athens.
sity Press, Cambridge, MA. Steuer, R. E., Y. Qi, M. Hirschberger. 2005. Multiple objectives in
Roy, B. 1998. A missing link in operational research decision aid- portfolio selection. J. Financial Decision Making 1 5–20.
ing: Robustness analysis. Foundations Comput. Decision Sci. 23 Sun, M. 2003. Procedures for finding nondominated solutions for
141–160. multiple objective network programming problems. Transporta-
Saaty, T. L. 1999. Decision Making for Leaders. RWS Publications, tion Sci. 37 139–152.
Pittsburgh. Tan, K. C., E. F. Khor, T. H. Lee. 2005. Multiobjective Evolutionary
Salo, A., R. P. Hämäläinen. 1992. Preference assessment by impre- Algorithms and Applications. Springer, Berlin.
cise ratio statements. Oper. Res. 40 1053–1061. Teich, J., H. Wallenius, J. Wallenius. 1994. Advances in negotiation
Sayin, S. 2003. A procedure to find discrete representations of the science. Trans. Oper. Res. 6 55–94.
efficient set with specified convergence errors. Oper. Res. 51 Teich, J., H. Wallenius, J. Wallenius, O. Koppius. 2004. Emerging
427–436. multiple issue e-auctions. Eur. J. Oper. Res. 159 1–16.
Schaffer, J. D. 1984. Some experiments in machine learning using Tsetlin, I., R. Winkler. 2007. Decision making with multiattribute
vector evaluated genetic algorithms. Ph.D. thesis, Vanderbilt performance targets: The impacts of changes in performance
University, Nashville, TN. and target distributions. Oper. Res. 55 226–233.
Scheubrein, R., S. Zionts. 2006. A problem structuring front end for Ulungu, E., J. Teghem. 1994. Multiobjective combinatorial opti-
a multiple criteria decision support system. Comput. Oper. Res. mization problems: A survey. J. Multi-Criteria Decision Anal. 3
33 18–31. 83–104.
Sebenius, J. K. 2007. Negotiation analysis: Between decisions and Von Winterfeldt, D., E. Schweitzer. 1998. An assessment of tritium
games. W. Edwards, R. F. Miles, D. von Winterfeldt, eds. supply alternatives in support of the U.S. nuclear weapons
Advances in Decision Analysis. Cambridge University Press, stockpile. Interfaces 22 92–118.
New York, 469–488. Wang, J. G., S. Zionts. 2005. WebAIM: An online aspiration
Sim, K. M., C. Y. Choi. 2003. Agents that react to changing market level interactive method. J. Multi-Criteria Decision Anal. 13
conditions. IEEE Trans. Systems, Man, Cybernetics B 33 188–201. 51–63.
Simon, H. 1955. A behavioral model of rational choice. Quart. J. Wang, J. G., S. Zionts. 2006. The aspiration level interactive method
Econom. 69 99–118. (AIM) reconsidered: Robustness of solutions to multiple crite-
Slowinski, R. 1992. Intelligent Decision Support: Handbook of Applica- ria problems. Eur. J. Oper. Res. 17(2) 948–958.
tions and Advances of the Rough Sets Theory. Kluwer Academic, Wang, J. G., S. Zionts. 2008. Negotiating wisely: Considerations
Dordrecht, The Netherlands. based on MCDM/MAUT. Eur. J. Oper. Res. 188 191–205.
Slowinski, R., S. Greco, B. Matarazzo. 2002. Axiomatization of Wiecek, M. M., H. Zhang. 1997. A parallel algorithm for multiple
utility, outranking and decision-rule preference models for objective linear programs. Comput. Optim. Appl. 8 41–56.
multiple-criteria classification problems under partial incon- Yang, J. B., D. Li. 2002. Normal vector identification and interac-
sistency with the dominance principle. Control Cybernetics 31 tive tradeoff analysis using minimax formulation in multiob-
1005–1035. jective optimization. IEEE Trans. Systems, Man, Cybernetics A 32
Srinivas, N., K. Deb. 1994. Multiobjective optimization using non- 305–319.
dominated sorting in genetic algorithms. Evolutionary Comput. Zitzler, E., K. Deb, L. Thiele, C. A. C. Coello, D. Corne.
J. 2 221–248. 2001. Proc. First Evolutionary Multi-Criterion Optimization
Statnikov, R. B., J. B. Matusov. 2002. Multicriteria Analysis in Engi- Conf., Lecture Notes in Computer Science, Vol. 1993. Springer,
neering. Kluwer Academic, Dordrecht, The Netherlands. New York.