Series

Series
SeriesISSN:
ISSN:
ISSN:1935-4185
1935-4185
1935-4185

MM

MENACHE ••• OZDAGLAR

MENACHE
MENACHE
SSYNTHESIS
YNTHESISL
LECTURES
&CC
YNTHESIS ECTURES
ECTURESON
ON
ON
CCOMMUNICATION
OMMUNICATION N
OMMUNICATION NETWORKS
ETWORKS
ETWORKS & Mor
Mor gan&
Morgan
gan &Cl
Cl
Claypool
aypool
aypool Publishers
Publishers
Publishers

OZDAGLAR
OZDAGLAR
Series
Series
SeriesEditor:
Editor:
Editor:Jean
Jean
JeanWalrand,
Walrand,
Walrand,University
University
Universityofof
ofCalifornia,
California,
California,Berkeley
Berkeley
Berkeley

Network
Network Games
Theory,
Theory,
Games
Theory,Models,
Models,
Models,and
and
andDynamics
Dynamics
Dynamics
Network
Network Games
Games
Ishai
Ishai
IshaiMenache,
Menache,
Menache,Microsoft
Microsoft
MicrosoftResearch
Research
ResearchNew
New
NewEngland
England
England
Theory,
Theory, Models,
Models, and
and Dynamics
Dynamics
Asuman
Asuman
AsumanOzdaglar,
Ozdaglar,
Ozdaglar,Massachusetts
Massachusetts
MassachusettsInstitute
Institute
Instituteofof
ofTechnology
Technology
Technology

Traditional
Traditional
Traditionalnetworknetwork
networkoptimization
optimization
optimizationfocuses focuses
focuseson on
onaaasingle
single
singlecontrol
control
controlobjective
objective
objectivein in
inaaanetwork
network
networkpopulated
populated
populatedby by
by
obedient
obedient
obedientusersusers
usersandand
andlimited
limited
limiteddispersion
dispersion
dispersionof of
ofinformation.
information.
information.However,However,
However,most most
mostof of
oftoday’s
today’s
today’snetworks
networks
networksare are
arelarge-
large-
large-
scale
scale
scalewith
with
withlack
lack
lackofof
ofaccess
access
accessto to
tocentralized
centralized
centralizedinformation,
information,
information,consistconsist
consistof of
ofusers
users
userswith
with
withdiverse
diverse
diverserequirements,
requirements,
requirements,and and
and

NETWORK GAMES
NETWORK
NETWORK GAMES
are
are
aresubject
subject
subjectto to
todynamic
dynamic
dynamicchanges.
changes.
changes.These
These
Thesefactors
factors
factorsnaturally
naturally
naturallymotivate
motivate
motivateaaanew new
newdistributed
distributed
distributedcontrol
control
controlparadigm,
paradigm,
paradigm,
where
where
wherethe the
thenetwork
network
networkinfrastructure
infrastructure
infrastructureisisiskeptkept
keptsimple
simple
simpleandand
andthe the
thenetwork
network
networkcontrol
control
controlfunctions
functions
functionsare are
aredelegated
delegated
delegatedto to
to
individual
individual
individualagents
agents
agentswhich
which
whichmakemake
maketheir
their
theirdecisions
decisions
decisionsindependently
independently
independently(“selfishly”).The
(“selfishly”).The
(“selfishly”).Theinteraction
interaction
interactionof of
ofmultiple
multiple
multiple
independent
independent
independentdecision-makers
decision-makers
decision-makersnecessitates
necessitates
necessitatesthe the
theuse
use
useofof
ofgame
game
gametheory,
theory,
theory,including
including
includingeconomic
economic
economicnotions
notions
notionsrelated
related
related

GAMES
to
totomarkets
markets
marketsand and
andincentives.
incentives.
incentives.
This
This
Thismonograph
monograph
monographstudies studies
studiesgamegame
gametheoretic
theoretic
theoreticmodels
models
modelsof of
ofresource
resource
resourceallocation
allocation
allocationamong among
amongselfish
selfish
selfishagents
agents
agentsin in
in
networks.
networks.
networks.The
isisisgiven
given
givento to
The
Thefirst
tothe
the
first
firstpart
part
theanalysis
partof
analysis
analysisof
of
ofthe
of
the
themonograph
monograph
monographintroduces
ofdynamics
dynamics
dynamicsin in
ingame
game
introduces
introducesfundamental
gametheoretic
theoretic
fundamental
fundamentalgame
theoreticsituations,
situations,
game
gametheoretic
situations,which
which
theoretic
theoretictopics.
whichisisiscrucial
crucial
topics.
topics.Emphasis
crucialforfor
Emphasis
Emphasis
fordesign
design
designand and
and
Ishai
Ishai
IshaiMenache
Menache
Menache
control
control
controlof of
ofnetworked
networked
networkedsystems.
systems.
systems.The The
Thesecond
second
secondpart part
partofof
ofthe
the
themonograph
monograph
monographapplies applies
appliesthe the
thegame
game
gametheoretic
theoretic
theoretictools
tools
tools Asuman
Asuman
AsumanOzdaglar
Ozdaglar
Ozdaglar
for
for
forthe
the
theanalysis
analysis
analysisof of
ofresource
resource
resourceallocation
allocation
allocationin in
incommunication
communication
communicationnetworks. networks.
networks.We We
Weset set
setup
up
upaaageneral
general
generalmodel
model
modelof of
of
routing
routing
routingin in
inwireline
wireline
wirelinenetworks,
networks,
networks,emphasizing
emphasizing
emphasizingthe the
thecongestion
congestion
congestionproblemsproblems
problemscaused
caused
causedby by
bydelay
delay
delayandand
andpacket
packet
packetloss.
loss.
loss.
In
InInparticular,
particular,
particular,we we
wedevelop
develop
developaaasystematic
systematic
systematicapproach
approach
approachto to
tocharacterizing
characterizing
characterizingthe the
theinefficiencies
inefficiencies
inefficienciesof of
ofnetwork
network
network
equilibria,
equilibria,
equilibria,and and
andhighlight
highlight
highlightthe the
theeffect
effect
effectofof
ofautonomous
autonomous
autonomousservice service
serviceproviders
providers
providerson on
onnetwork
network
networkperformance.
performance.
performance.We We
We
then
then
thenturn
turn
turntoto
toexamining
examining
examiningdistributed
distributed
distributedpower power
powercontrol
control
controlin in
inwireless
wireless
wirelessnetworks.
networks.
networks.We We
Weshowshow
showthat
that
thatthe
the
theresulting
resulting
resulting
Nash
Nash
Nashequilibria
equilibria
equilibriacan can
canbebe
beefficient
efficient
efficientifififthe
the
thedegree
degree
degreeof of
offreedom
freedom
freedomgiven given
givento to
toend-users
end-users
end-usersisisisproperly
properly
properlydesigned.
designed.
designed.

About
About
AboutSYNTHESIs
SYNTHESIs
SYNTHESIs
This
This
Thisvolume
volume
volumeisisisaaaprinted
printed
printedversion
version
versionofof
ofaaawork
work
workthat
that
thatappears
appears
appearsinin
inthe
the
theSynthesis
Synthesis
Synthesis
Digital
Digital
DigitalLibrary
Library
LibraryofofofEngineering
Engineering
Engineeringand and
andComputer
Computer
ComputerScience.
Science.
Science.Synthesis
Synthesis
SynthesisLectures
Lectures
Lectures

MOR GAN
MOR
MOR
provide
provide
provideconcise,
concise,
concise,original
original
originalpresentations
presentations
presentationsofof ofimportant
important
importantresearch
research
researchand
and
anddevelopment
development
development
topics,
topics,
topics,published
published
publishedquickly,
quickly,
quickly,inin
indigital
digital
digitaland
and
andprint
print
printformats.
formats.
formats.For
For
Formore
more
moreinformation
information
information

GAN &
GAN
visit
visit
visitwww.morganclaypool.com
www.morganclaypool.com
www.morganclaypool.com
SSYNTHESIS
YNTHESISL
YNTHESIS LECTURES
ECTURES
ECTURESON
ON
ON

& CL
&
CCOMMUNICATION
OMMUNICATION NNETWORKS

CL AYPOOL
CL
ISBN:
ISBN:
ISBN:978-1-60845-408-2
978-1-60845-408-2
978-1-60845-408-2
Mor
Mor
Morgan
gan
gan Cl
Cl
Claypool
aypool &
&
&
aypool Publishers
Publishers
Publishers 90000
90000
90000
OMMUNICATION ETWORKS
ETWORKS

AYPOOL
AYPOOL
wwwwwwwww. .m
.m
mooorrrgggaaannncccl llaaayyypppooooool ll. .c. ccooom
mm
999781608
781608
781608454082
454082
454082
Jean
Jean
JeanWalrand,
Walrand,
Walrand,Series
Series
SeriesEditor
Editor
Editor
Network Games
Theory, Models, and Dynamics
 Synthesis Lectures on
Communication Networks
Editor
Jean Walrand, University of California, Berkeley
Synthesis Lectures on Communication Networks is an ongoing series of 50- to 100-page publications
on topics on the design, implementation, and management of communication networks. Each lecture is
a self-contained presentation of one topic by a leading expert. The topics range from algorithms to
hardware implementations and cover a broad spectrum of issues from security to multiple-access
protocols. The series addresses technologies from sensor networks to reconfigurable optical networks.
The series is designed to:
• Provide the best available presentations of important aspects of communication networks.
• Help engineers and advanced students keep up with recent developments in a rapidly evolving
technology.
• Facilitate the development of courses in this field

Network Games: Theory, Models, and Dynamics

Ishai Menache and Asuman Ozdaglar
2011

An Introduction to Models of Online Peer-to-Peer Social Networking

George Kesidis
2010

Stochastic Network Optimization with Application to Communication and Queueing

Systems
Michael J. Neely
2010

Scheduling and Congestion Control for Wireless and Processing Networks

Libin Jiang and Jean Walrand
2010

Performance Modeling of Communication Networks with Markov Chains

Jeonghoon Mo
2010
 iii
Communication Networks: A Concise Introduction
Jean Walrand and Shyam Parekh
2010

Path Problems in Networks

John S. Baras and George Theodorakopoulos
2010

Performance Modeling, Loss Networks, and Statistical Multiplexing

Ravi R. Mazumdar
2009

Network Simulation
Richard M. Fujimoto, Kalyan S. Perumalla, and George F. Riley
2006
Copyright © 2011 by Morgan & Claypool

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in
any form or by any means—electronic, mechanical, photocopy, recording, or any other except for brief quotations in
printed reviews, without the prior permission of the publisher.

Network Games: Theory, Models, and Dynamics

Ishai Menache and Asuman Ozdaglar
www.morganclaypool.com

ISBN: 9781608454082 paperback

ISBN: 9781608454099 ebook

DOI 10.2200/S00330ED1V01Y201101CNT009

A Publication in the Morgan & Claypool Publishers series

SYNTHESIS LECTURES ON COMMUNICATION NETWORKS

Lecture #9
Series Editor: Jean Walrand, University of California, Berkeley
Series ISSN
Synthesis Lectures on Communication Networks
Print 1935-4185 Electronic 1935-4193
Network Games
Theory, Models, and Dynamics

Ishai Menache
Microsoft Research New England

Asuman Ozdaglar
Massachusetts Institute of Technology

SYNTHESIS LECTURES ON COMMUNICATION NETWORKS #9

M
&C Morgan & cLaypool publishers
ABSTRACT
Traditional network optimization focuses on a single control objective in a network populated by
obedient users and limited dispersion of information. However, most of today’s networks are large-
scale with lack of access to centralized information, consist of users with diverse requirements, and
are subject to dynamic changes. These factors naturally motivate a new distributed control paradigm,
where the network infrastructure is kept simple and the network control functions are delegated to
individual agents which make their decisions independently (“selfishly"). The interaction of multiple
independent decision-makers necessitates the use of game theory, including economic notions related
to markets and incentives.
This monograph studies game theoretic models of resource allocation among selfish agents in
networks. The first part of the monograph introduces fundamental game theoretic topics. Emphasis
is given to the analysis of dynamics in game theoretic situations, which is crucial for design and
control of networked systems. The second part of the monograph applies the game theoretic tools
for the analysis of resource allocation in communication networks. We set up a general model of
routing in wireline networks, emphasizing the congestion problems caused by delay and packet
loss. In particular, we develop a systematic approach to characterizing the inefficiencies of network
equilibria, and highlight the effect of autonomous service providers on network performance. We
then turn to examining distributed power control in wireless networks. We show that the resulting
Nash equilibria can be efficient if the degree of freedom given to end-users is properly designed.

KEYWORDS
game theory, Nash equilibrium, dynamics, communication networks, routing, power
control
 To Helena, Sophia, and Rami
Ishai Menache

To Daron and my parents for their

unconditional love and support
Asuman Ozdaglar
 ix

Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

PART I Game Theory Background . . . . . . . . . . . . . . . . . . . . . 1

1 Static Games and Solution Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Strategic Form Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Solution Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 Dominant and Dominated Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.2 Iterated Elimination of Strictly Dominated Strategies . . . . . . . . . . . . . . . . . . 8
1.2.3 Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2.4 Correlated Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.3 Existence of a Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.3.1 Games with Finite Pure Strategy Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.3.2 Games with Infinite Pure Strategy Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.3.3 Continuous Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.3.4 Discontinuous Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.4 Uniqueness of a Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.A Appendix: Metric Spaces and Probability Measures . . . . . . . . . . . . . . . . . . . . . . . . . 35
1.B Appendix: Nonlinear Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2 Game Theory Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.1 Extensive Form Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.2 Learning Dynamics in Games – Fictitious Play . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.2.1 Convergence of Fictitious Play . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.2.2 Non-convergence of Fictitious Play . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.2.3 Convergence Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.3 Games with Special Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.3.1 Supermodular Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.3.2 Potential Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.A Appendix: Lattices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
x

PART II Network Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3 Wireline Network Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.1 Selfish Routing, Wardrop Equilibrium and Efficiency . . . . . . . . . . . . . . . . . . . . . . . 71
3.1.1 Routing Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.1.2 Wardrop Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.1.3 Inefficiency of the Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.1.4 Multiple Origin-Destination Pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.2 Partially Optimal Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.2.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.2.2 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.2.3 Efficiency of Partially Optimal Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.2.4 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.3 Congestion and Provider Price Competition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.3.1 Pricing and Efficiency with Congestion Externalities . . . . . . . . . . . . . . . . . 88
3.3.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.3.3 Monopoly Pricing and Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.3.4 Oligopoly Pricing and Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.3.5 Efficiency Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.3.6 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

4 Wireless Network Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

4.1 Noncooperative Transmission Scheduling in Collision Channels . . . . . . . . . . . . . 102
4.1.1 The Model and Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.1.2 Equilibrium Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.1.3 Achievable Channel Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.1.4 Best-Response Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4.1.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.2 Noncooperative Power Control in Collision Channels . . . . . . . . . . . . . . . . . . . . . . 117
4.2.1 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.2.2 Equilibrium Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.2.3 Best-Response Dynamics and Convergence to the Power Efficient
Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.2.4 Equilibrium (In)Efficiency and Braess-Like Paradoxes . . . . . . . . . . . . . . . 126
4.2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
4.3 Related Work and Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
4.4 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
 xi

5 Future Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Authors’ Biographies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

 Preface
Traditional network optimization focuses on a well-defined control objective in a network
populated by obedient users and limited dispersion of information, and uses convex optimization
techniques to determine efficient allocation of resources (see [8], [20], [22], [102]). Most of today’s
networked systems, such as the Internet, transportation networks, and electricity markets, differ
from this model in their structure and operation. First, these networks are large-scale with lack of
access to centralized information and subject to dynamic changes. Hence, control policies have to be
decentralized, scalable, and robust against unexpected disturbances. Second, these networks consist
of interconnection of heterogeneous autonomous entities and serve users with diverse requirements,
so there is no central party with enforcement power or accurate information about user needs.
This implies that selfish incentives and private information of users need to be incorporated into the
control paradigm. Finally, these networks are subject to continuous upgrades and investments in new
technologies, making economic incentives of service and content providers much more paramount.
These new challenges have naturally motivated a new distributed control paradigm, where the
network infrastructure is kept simple and the network control functions are delegated to individual
agents, which make their decisions independently (“selfishly"), according to their own performance
objectives. The key aspect of this approach is to view the network as a resource to be shared by
a number of heterogeneous users with different service requirements. The interaction of multiple
independent decision-makers necessitates the use of game theory (the study of multi-agent problems)
and also some ideas from economics related to markets and incentives. Consequently, the recent
engineering literature considers a variety of game-theoretic and economic market models for resource
allocation in networks.
This monograph studies game theoretic models for analysis of resource allocation among
heterogeneous agents in networks. Given the central role that game theory plays in the analysis
of networked systems, the first part of the monograph will be devoted to a systematic analysis
of fundamental game theoretic topics. Emphasis will be placed on game theoretic tools that are
commonly used in the analysis of resource allocation in current networks or those that are likely to
be used in future networks. We start in the next chapter with strategic form games, which constitute
the foundation of game theoretic analysis and enable us to introduce the central concept of Nash
equilibrium, which makes predictions about equilibrium behavior in situations in which several
agents interact. We also introduce several related concepts, such as correlated equilibria, both to
clarify the conditions under which Nash equilibria are likely to provide a good approximation to
behavior in various different circumstances and as alternative concepts of equilibrium that might be
useful in the communication area in future research.
xiv PREFACE
The Nash equilibrium is by its very definition a static concept, and as such, the study of its
properties does not cover the analysis of dynamics, namely, if and how an equilibrium is reached. Issues
of dynamic resource allocation and changes in behavior of users in wireline and wireless networks
are of central importance in the analysis of communication networks. We present in Chapter 2 two
complementary ways of introducing dynamic analysis in game theoretic situations. First, we study
extensive form (dynamic) games where the relevant concept of (subgame perfect) Nash equilibrium
will exhibit some dynamics itself. Second, we look at dynamics induced by the repeated play of
the same strategic form game when agents are not so sophisticated to play Nash equilibrium, but
follow simple myopic rules or rules of thumb. The game theory literature has established that this
type of myopic play has several interesting properties in many games and, in fact, converges to
Nash equilibrium for certain classes of games. Interestingly, these classes of games, which include
potential games and supermodular games, have widespread applications for network game models,
so the last part of this chapter includes a detailed analysis of potential and supermodular games and
the dynamics of play under various myopic and reactive rules.
The second part of the monograph applies game theoretic tools to the analysis of resource
allocation in wireline and wireless communication networks. Chapter 3 focuses on wireline networks,
with special attention drawn to the consequences of selfish routing in communication networks. We
first show how the notions of Nash equilibrium and subgame perfect Nash equilibrium enable
us to develop simple models of routing and resource allocation in wireline networks. We set up
a general model of routing in wireline networks emphasizing the congestion problems caused by
delay and packet loss. We show how a variety of different models of congestion can be modeled
as a static strategic form game with the cost of congestion captured by latency functions. We first
establish existence of equilibria and provide basic characterization results. We then turn to the
question of efficiency of equilibria in wireline communication problems. It is well known since
the work of Alfred Pigou that equilibria in such situations with congestion problems can involve
significant externalities. We provide examples illustrating these inefficiencies, both demonstrating
that inefficiencies could be significantly very large (unbounded) and the possibilities of paradoxical
result such as the Braess’ paradox. We then develop a systematic approach to characterizing the
inefficiencies of these equilibria.
Classic models of wireline communication ignore the fact that autonomous service providers
are active participants in the flows of communication and do so by either (i) redirecting traffic
within their own networks to achieve minimum intradomain total latency or (ii) charging prices
for maximizing their individual revenues. We then show how the presence of autonomous service
providers can be incorporated into the general framework of wireline games. We demonstrate how
existence of pure strategy and mixed strategy equilibria can be established in the presence of service-
provider routing and pricing, and develop an alternative mathematical approach to quantifying
inefficiency of equilibria in several networking domains. The interesting set of results is that in a
variety of network topologies, the presence of prices ameliorates the potential inefficiencies that
exist in wireline communication networks. However, we also show that in complex communication
 PREFACE xv
networks, even in the presence of prices and optimal intradomain routing decisions, inefficiencies
could be substantial, partly because of the double marginalization problem in the theory of oligopoly.
Chapter 4 deals with wireless communication games. Perhaps the most lucid example for
the consequences of selfish behavior in wireless networks is the case where a mobile captures a
shared collision channel by continuously transmitting packets, hence effectively nullifying other
users’ throughput. We show that this undesired scenario can be avoided under a natural power-
throughput tradeoff assumption, where each user minimizes its average transmission rate (which is
proportional to power investment) subject to minimum-throughput demand. An important element
in our models is the incorporation of fading effects, assuming that the channel quality of each mobile
is time-varying and available to the user prior to the transmission decisions. Our equilibrium analysis
reveals that there are at most two Nash equilibrium points where all users obtain their demands,
with one strictly better than the other in terms of power investment for all users. Furthermore, we
suggest a fully distributed mechanism that leads to the better equilibrium. The above model is then
extended to include wireless platforms where mobiles are allowed to autonomously control their
transmission power. We demonstrate the existence of a power-superior equilibrium point that can
be reached through a simple distributed mechanism. On the negative side, however, we point to
the possibility of Braess-like paradoxes, where the use of multiple power levels can diminish system
capacity and also lead to larger per-user power consumption, compared to the case where only a
single level is permitted.
We conclude the monograph in Chapter 5 by outlining high-level directions for future work
in the area of network games, incorporating novel challenges not only in terms of game-theoretic
modeling and analysis, but also with regard to the proper exploitation of the associated tools in
current and future networking systems.
Large parts of Chapters 3 and 4 are based on our individual research in the area. We are
naturally indebted to our coauthors whose advice, knowledge, and deep insight have been invaluable.
Ishai Menache wishes to particularly thank Nahum Shimkin for his guidance and collaboration.
Asuman Ozdaglar would like to thank Daron Acemoglu for his collaboration and support.
We are also grateful to Ermin Wei for making detailed comments on drafts of this monograph.
Finally, we wish to acknowledge the research support of NSF grants CMMI-0545910 and SES-
0729361, AFOSR grantsFA9550-09-1-0420 and R6756-G2,ARO grant 56549NS, the DARPA
ITMANET program, and a Marie Curie International Fellowship within the 7th European Com-
munity Framework Programme.

Ishai Menache and Asuman Ozdaglar

March 2011
 PART I

Game Theory Background

 3

CHAPTER 1

Static Games and Solution

Concepts
This chapter presents the fundamental notions and results in noncooperative game theory. In Section
1.1, we introduce the standard model for static strategic interactions, the strategic form game. In
Section 1.2, we define various solution concepts associated with strategic form games, including
the Nash equilibrium. We then proceed in Sections 1.3–1.4 to address the issues of existence and
uniqueness of a Nash equilibrium.

1.1 STRATEGIC FORM GAMES

We first introduce strategic form games (also referred to as normal form games). A strategic form
game is a model for a static game in which all players act simultaneously without knowledge of other
players’ actions.

Definition 1.1 (Strategic Form Game) A strategic form game is a triplet

I , (Si )i∈I , (ui )i∈I  where

1. I is a finite set of players, I = {1, . . . , I }.

2. Si is a non-empty set of available actions for player i.


3. ui : S → R is the payoff (utility) function of player i where S = i Si .1

For strategic form games, we will use the terms action and (pure) strategy interchangeably.2
We denote by si ∈ Si an action for player i, and by s−i = [sj ]j =i a vector of actions for all players
except i. We refer to the tuple (si , s−i ) ∈ S as an action (strategy) profile, or outcome. We also denote

by S−i = j =i Sj the set of actions (strategies) of all players except i. Our convention throughout
will be that each player i is interested in action profiles that “maximize" his utility function ui .
1 Here we implicitly assume that players have preferences over the outcomes and that these preferences can be captured by assigning
a utility function over the outcomes. Note that not all preference relations can be captured by utility functions (see Chapters 1,3
and 6 of [60] for more on this issue).
2 We will later use the term “strategy" more generally to refer to randomizations over actions, or contingency plans over actions in
the context of dynamic games.
4 1. STATIC GAMES AND SOLUTION CONCEPTS
The next two examples illustrate strategic form games with finite and infinite strategy sets.

Example 1.2 Finite Strategy Sets A two-player game where the strategy set Si of each player is
finite can be represented in matrix form. We adopt the convention that the rows (columns) of the
matrix represent the action set of player 1 (player 2).The cell indexed by row x and column y contains
a pair, (a, b), where a is the payoff to player 1 and b is the payoff to player 2, i.e., a = u1 (x, y) and
b = u2 (x, y). This class of games is sometimes referred to as bimatrix games. For example, consider
the following game of “Matching Pennies.”

HEADS TAILS
HEADS í 1, 1 1, í 1
TAILS 1, í 1 í 1, 1
Matching Pennies.
This game represents “pure conflict,” in the sense that one player’s utility is the negative of
the utility of the other player, i.e., the sum of the utilities for both players at each outcome is “zero.”
This class of games is referred to as zero-sum games (or constant-sum games) and has been extensively
studied in the game theory literature [15].

Example 1.3 Infinite Strategy Sets The strategy sets of players can also have infinitely many ele-
ments. Consider the following game of Cournot Competition, which models two firms producing the
same homogeneous good and seeking to maximize their profits.The formal game G = I , (Si ), (ui )
consists of:
1. A set of two players, I = 1, 2.
2. A strategy set Si = [0, ∞) for each player i, where si ∈ Si represents the amount of good that
the player produces.
3. A utility function ui for each player i given by its total revenue minus its total cost, i.e.,

ui (s1 , s2 ) = si p(s1 + s2 ) − ci si

where p(q) represents the price of the good (as a function of the total amount of good q), and
ci is the unit cost for firm i.
For simplicity, we consider the case where both firms have unit cost, c1 = c2 = 1, and the price
function is piecewise linear and is given by p(q) = max{0, 2 − q}.
We analyze this game by considering the best-response correspondences for each of the firms.
For firm i, the best-response correspondence Bi (s−i ) is a mapping from the set S−i into set Si such
that
Bi (s−i ) = {si ∈ Si | ui (si , s−i ) ≥ ui (si , s−i ), for all si ∈ Si }.
 1.2. SOLUTION CONCEPTS 5
Note that for this example, the best response correspondences are unique-valued (hence can be
referred to as best-response functions), since for all s−i , there is a unique action that maximizes the
utility function for firm i. In particular, we have:

Bi (s−i ) = arg max (si p(si + s−i ) − si ) .

si ≥0

When s−i > 1, the action that maximizes the utility function of firm i, i.e., its best response, is 0.
More generally, it can be seen (by using first order optimality conditions, see Appendix 1.B) that for
any s−i ∈ S−i , the best response of firm i is given by

Bi (s−i ) = arg max (si (2 − si − s−i ) − si )

 si ≥0
1−s−i
if s−i ≤ 1,
= 2
0 otherwise.

s2
1

B1(s2)
1/2

B2(s1)

1/2
1
s1

Figure 1.1: Best response functions for the Cournot Competition game.

Figure 1.1 illustrates the best response functions as a function of s1 and s2 . Intuitively, we
expect the outcome of this game to be at the point where both of these functions intersect. In the
next section, we will argue that this intersection point is a reasonable outcome in this game.

1.2 SOLUTION CONCEPTS

This section presents the main solution concepts for strategic form games: strictly and weakly dom-
inant and dominated strategies, pure and mixed Nash equilibrium, and correlated equilibrium.
6 1. STATIC GAMES AND SOLUTION CONCEPTS
1.2.1 DOMINANT AND DOMINATED STRATEGIES
In some games, it may be possible to predict the outcome assuming that all players are rational and
fully knowledgeable about the structure of the game and each other’s rationality. This is the case,
for instance, for the well-studied Prisoner’s Dilemma game. The underlying story of this game is as
follows: Two people are arrested for a crime, placed in separate rooms, and questioned by authorities
trying to extract a confession. If they both remain silent (i.e., cooperate with each other), then the
authorities will not be able to prove charges against them and they will both serve a short prison
term, say 2 years, for minor offenses. If only one of them confesses (i.e., does not cooperate), his
term will be reduced to 1 year and he will be used as a witness against the other person, who will get
a sentence of 5 years. If they both confess, they both get a smaller sentence of 4 years. This game
can be represented in matrix form as follows:

COOPERATE DON’T COOPERATE

COOPERATE í 2, í 2 í 5, í 1
DON’T COOPERATE í 1, í 5 í 4, í 4
Prisoner’s Dilemma.
In this game, regardless of the other players decision, playing Don’t Cooperate yields a
higher payoff for each player. Hence, the strategy Don’t Cooperate is strictly dominant, i.e., no
matter what strategy the other player chooses, this strategy always yields a strictly better outcome.
We can therefore infer that both players will choose Don’t Cooperate and spend the next four years
in jail while if they both chose the strategy Cooperate, they could have ended up in jail only for two
years! Prisoner’s Dilemma is a paradigmatic example of self-interested rational behavior not leading
to jointly (socially) optimal outcomes. We will see in Chapters 3 and 4 that many network games
exhibit such inefficiencies in equilibrium due to selfish nature of players.
A compelling notion of equilibrium in games would be the dominant strategy equilibrium,
where each player plays a dominant strategy as formalized in the next definition.

Definition 1.4 Dominant Strategy A strategy si ∈ Si is a dominant strategy for player i if

ui (si , s−i ) ≥ ui (si , s−i ) for all si ∈ Si and for all s−i ∈ S−i .

It is strictly dominant if this relation holds with a strict inequality.

Definition 1.5 Dominant Strategy Equilibrium A strategy profile s ∗ is a (strictly) dominant

strategy equilibrium if for each player i, si∗ is a (strictly) dominant strategy.

In the Prisoner’s Dilemma game, (Don’t Cooperate, Don’t Cooperate) is a strictly dom-
inant strategy equilibrium. Though compelling, dominant strategy equilibria do not always exist,
 1.2. SOLUTION CONCEPTS 7
as illustrated by the Matching Pennies game (cf. Example 1.2) and the next example. Consider a
slightly modified Prisoner’s Dilemma game in which players also have the strategy Suicide leading
to the following payoff structure:

COOPERATE DON’T COOPERATE SUICIDE

COOPERATE í 2, í 2 í 5, í 1 0, í 20
DON’T COOPERATE í 1, í 5 í 4, í 4 í 4, í 20
SUICIDE í 20, 0 í 20, í 4 í 20, í 20
Prisoner’s Dilemma with Suicide.

This payoff matrix models a scenario in which if one player chooses the strategy Suicide,
then, due to lack of witnesses, the other player gets off free if he remains silent (cooperates). In this
game, there is no dominant strategy equilibrium because of the additional strategy Suicide. Notice,
however, that the strategy Suicide is the worst possible option for a player, no matter what the other
player does. In this sense, Suicide is strictly dominated by the other two strategies. More generally,
we say that a strategy is strictly dominated for a player if there exists some other strategy that yields
a strictly higher payoff regardless of the strategies of the other players.

Definition 1.6 Strictly Dominated Strategy A strategy si ∈ Si is strictly dominated for player i
if there exists some si ∈ Si such that

ui (si , s−i ) > ui (si , s−i ) for all s−i ∈ S−i .

Next, we define a weaker version of dominated strategies.

Definition 1.7 Weakly Dominated Strategy A strategy si ∈ Si is weakly dominated for player i
if there exists some si ∈ Si such that

ui (si , s−i ) ≥ ui (si , s−i ) for all s−i ∈ S−i ,

and
ui (si , s−i ) > ui (si , s−i ) for some s−i ∈ S−i .

It is plausible to assume that no player chooses a strictly dominated strategy. Moreover, com-
mon knowledge of payoffs and rationality leads players to do iterated elimination of strictly domi-
nated strategies, as illustrated next.
8 1. STATIC GAMES AND SOLUTION CONCEPTS
1.2.2 ITERATED ELIMINATION OF STRICTLY DOMINATED STRATEGIES
In the Prisoner’s Dilemma with Suicide game, the strategy Suicide is a strictly dominated strategy
for both players. Therefore, no rational player would choose Suicide. Moreover, if player 1 is certain
that player 2 is rational, then he can eliminate her opponent’s Suicide strategy. We can use a similar
reasoning for player 2. After one round of elimination of strictly dominated strategies, we are back to
the Prisoner’s Dilemma game, which has a dominant strategy equilibrium.Thus, iterated elimination
of strictly dominated strategies leads to a unique outcome, (Don’t Cooperate, Don’t Cooperate)
in this modified game. We say that a game is dominance solvable if iterated elimination of strictly
dominated strategies yields a unique outcome.
Consider next another game.

LEFT MIDDLE RIGHT

UP 4, 3 5, 1 6, 2
MIDDLE 2, 1 8, 4 3, 6
DOWN 3, 0 9, 6 2, 8

Example for iterated elimination of strictly dominated strategies.

In this game, there are no strategies that are strictly dominated for player 1 (the row player). On
the other hand, the strategy Middle is strictly dominated by the strategy Right for player 2 (the
column player). Thus, we conclude that it is not rational for player 2 to play Middle and we can
therefore remove this column from the game, resulting in the following reduced game.

LEFT RIGHT
UP 4, 3 6, 2
MIDDLE 2, 1 3, 6
DOWN 3, 0 2, 8

Game after one removal of strictly dominated strategies.

Now, note that both strategies Middle and Down are strictly dominated by the strategy Up for
player 1, which means that both of these rows can be removed, resulting in the following game.

LEFT RIGHT
UP 4, 3 6, 2

Game after three iterated removals of strictly dominated strategies.

 1.2. SOLUTION CONCEPTS 9
We are left with a game where player 1 does not have any choice in his strategies, while player 2 can
choose between Left and Right. Since Left will maximize the utility of player 2, we conclude that
the only rational strategy profile in the game is (Up,Left).3
More formally, we define the procedure of iterated elimination of strictly dominated strategies
(or iterated strict dominance) as follows:
• Step 0: For each i, let Si0 = Si .
• Step 1: For each i, define
Si1 = {si ∈ Si0 |  doesnot exist si ∈ Si such that
there 0

ui si , s−i > ui (si , s−i ) for all s−i ∈ S−i

0
}.

...
• Step k: For each i, define
Sik = {si ∈ Sik−1 |  doesnot exist si ∈ Si such that k−1
there k−1

ui si , s−i > ui (si , s−i ) for all s−i ∈ S−i }.

• For each i, define

Si∞ = ∩∞ k
k=0 Si .

It is immediate that this procedure yields a nonempty set of strategy profiles under some
assumptions stated in the next theorem.

Theorem 1.8 Suppose that either (1) each Si is finite, or (2) each ui (si , s−i ) is continuous and each Si
is compact. Then Si∞ is nonempty for each i.

We next apply iterated elimination of strictly dominated strategies to the Cournot Compe-
tition game (cf. Example 1.3). Recall that we use the notation Sik to denote the set of strategies of
player i that survive iterated elimination of strictly dominated strategies at step k. In the first step,
we note that both firms must choose a quantity between [0, ∞), i.e.,
S11 = [0, ∞),
S21 = [0, ∞).
Since the range of the best response function of player 1 is [0, 1/2], any strategy outside this range
is never a best response and therefore is strictly dominated. The same reasoning holds for player 2.
Thus, at the second step, we have
S12 = [0, 1/2],
S22 = [0, 1/2].
3 One might worry that different orders for the removal of dominated strategies can yield different results. However, it can be
shown that the order in which strategies are eliminated does not affect the set of strategies that survive iterated elimination of
strictly dominated strategies.
10 1. STATIC GAMES AND SOLUTION CONCEPTS
Given that player 2 only chooses actions in the interval [0, 1/2], then player 1 can restrict the domain
of his best response function to only these values. Using his best response function, this implies that
the strategies outside the interval [1/4, 1/2] are strictly dominated for player 1. Applying the same
reasoning for player 2, we obtain

S13 = [1/4, 1/2],

S23 = [1/4, 1/2].

(see Figure 1.2). It can be shown that in the limit, the endpoints of the intervals converge to the
point where the two best response functions intersect. Hence, the Cournot Competition game is
another example of a dominance solvable game. Most games, however, are not solvable by iterated
strict dominance; therefore, we need a stronger equilibrium notion to predict the outcome.

s2 s2
1 1

B1(s2) B1(s2)
1/2 1/2

1/ B2(s1) 1/ B2(s1)
4 4

1/ 1/2 1 1/ 1/2 1
4 4
s1 s1

Figure 1.2: Elimination of strictly dominated strategies for the Cournot competition game.

1.2.3 NASH EQUILIBRIUM

We next introduce the fundamental solution concept for strategic form games, the notion of a Nash
equilibrium. A Nash equilibrium captures a steady state of the play in a strategic form game such
that each player acts optimally and forms correct conjectures about the behavior of the other players.

Definition 1.9 Nash Equilibrium A (pure strategy) Nash equilibrium of a strategic form game
I , (Si ), (ui )i∈I  is a strategy profile s ∗ ∈ S such that for all i ∈ I , we have

ui (si∗ , s−i
∗ ∗
) ≥ ui (si , s−i ) for all si ∈ Si .
 1.2. SOLUTION CONCEPTS 11
Hence, a Nash equilibrium is a strategy profile s ∗ such that no player i
can profit by unilaterally
deviating from his strategy si∗ , assuming
every other player j follows his strategy sj∗ . The definition
of a Nash equilibrium can be restated in terms of the best-response correspondences.

Definition 1.10 Nash Equilibrium - Restated Let I , (Si ), (ui )i∈I  be a strategic game. For any
s−i ∈ S−i , consider the best-response correspondence of player i, Bi (s−i ), given by

Bi (s−i ) = {si ∈ Si | ui (si , s−i ) ≥ ui (si , s−i ) for all si ∈ Si } .

We say that an action profile s ∗ is a Nash equilibrium if

si∗ ∈ Bi (s−i
∗
) for all i ∈ I .

This implies that for two player games, the set of Nash equilibria is given by the intersection
of the best response correspondences of the two players (e.g., recall the Cournot Competition game).
Below we give two other examples of games with pure strategy Nash equilibria.

Example 1.11 Battle of the Sexes

Consider a two player game with the following payoff structure:

BALLET SOCCER
BALLET 2, 1 0, 0
SOCCER 0, 0 1, 2
Battle of the Sexes.

This game, referred to as the Battle of the Sexes game, represents a scenario in which the two
players wish to coordinate their actions, but have different preferences over their actions. This game
has two pure Nash equilibria, i.e., the strategy profiles (Ballet, Ballet) and (Soccer, Soccer).

Example 1.12 Second Price Auction – with Complete Information

We consider a second price auction: There is a single indivisible object to be assigned to one
of n players. Player i’s valuation of the object is denoted by vi . We assume without loss of generality
that v1 ≥ v2 ≥ · · · ≥ vn > 0 and that each player knows all the valuations v1 , . . . , vn , i.e., it is a
complete information game.4 The rules of this auction mechanism are described as follows:

• The players simultaneously submit bids, b1 , .., bn .

4The analysis of the incomplete information version of this game, in which the valuations of other players are unknown (or
probabilistically known), is similar.
12 1. STATIC GAMES AND SOLUTION CONCEPTS
• The object is given to the player with the highest bid (or to a random player among the ones
bidding the highest value).

• The winner pays the second highest bid.

This mechanism induces a game among the players in which the strategy of each player is given by
her bid, and her utility for a bid profile is given by her valuation of the object minus the price she
pays, i.e., if player i is the winner, her utility is vi − bj where j is the player with the second highest
bid; otherwise, her utility is zero.
We first show that the strategy profile (b1 , .., bn ) = (v1 , .., vn ) is a Nash equilibrium. First
note that if indeed everyone plays according to this strategy profile, then player 1 receives the object
and pays a price v2 . Hence, her payoff will be v1 − v2 > 0, and all other payoffs will be 0. Now,
player 1 has no incentive to deviate since her utility cannot increase. Similarly, for all other players
i  = 1, in order for player i to change her payoff, she needs to bid more than v1 , in which case her
payoff will be vi − v1 < 0. Therefore, no player has an incentive to unilaterally deviate, showing that
this strategy profile is a Nash equilibrium.
We next show that the strategy profile (v1 , 0, 0, ..., 0) is also a Nash equilibrium. As before,
player 1 will receive the object, and will have a payoff of v1 − 0 = v1 . Using the same argument as
before, we conclude that none of the players have an incentive to deviate, and this strategy profile is
a Nash equilibrium. We leave this as an exercise to show that the strategy profile (v2 , v1 , 0, 0, ..., 0)
is also a Nash equilibrium.
So far, we have shown that the game induced by the second price auction has multiple Nash
equilibria. We finally show that for each player i, the strategy of bidding her valuation, i.e., bi = vi ,
in fact, weakly dominates all other strategies. Given a bid profile, let B ∗ denote the maximum of all
bids excluding player i’s bid, i.e.,
B ∗ = max bj .
j  =i

Assume that player i’s valuation is given by v∗. Figure 1.3 illustrates the utility of player i as a function
of B ∗ , when she bids her valuation, bi = v ∗ , less than her valuation, bi < v ∗ , and more than her
valuation bi > v ∗ . In the second graph, which represents the case when she bids bi < v ∗ , notice that
whenever bi ≤ B ∗ ≤ v ∗ , player i receives zero utility since she loses the auction to whoever bid B ∗ .
If she would have bid her valuation, she would have positive utility in this region (as depicted in the
first graph). These figures show that for each player, bidding her own valuation weakly dominates
all her other strategies.
An immediate implication of the preceding analysis is that there exist Nash equilibria (e.g.,
the strategy profile (v1 , 0, 0, ..., 0)) that involve the play of weakly dominated strategies.

Given that Nash equilibrium is a widely used solution concept in strategic form games, a
natural question is why one should expect the Nash equilibrium outcome in a strategic form game.
One justification is that since it represents a steady state situation, rational players should somehow
reason their way to Nash equilibrium strategies; that is, Nash equilibrium might arise through
 1.2. SOLUTION CONCEPTS 13

ui(bi) ui(bi) ui(bi)

v* bi v* v* b
B* B* i B*
bi = v* bi < v* bi > v*

Figure 1.3: The utility of player i as a function of B ∗ , the maximum of all bids except i’s bid.

introspection. This justification requires that players are rational and know the payoff functions of
all players, that they know their opponents are rational and know the payoff functions, and so on. A
second justification is that Nash equilibria are self-enforcing. That is, if players agree on a strategy
profile before independently choosing their actions, then no player has an incentive to deviate if the
agreed strategy profile is a Nash equilibrium. A final justification of the Nash equilibrium outcome
is through learning dynamics or evolution, which will be discussed in Chapter 2.

1.2.3.1 Mixed Strategy and Mixed Strategy Nash Equilibrium

Recall the game of Matching Pennies:

HEADS TAILS
HEADS 1, í 1 í 1, 1
TAILS í 1, 1 1, í 1
Matching Pennies.
It is easy to see that this game does not have a pure Nash equilibrium, i.e., for every pure strategy
in this game, one of the parties has an incentive to deviate. However, if we allow the players to
randomize over their choice of actions, we can determine a steady state of the play. Assume that
player 1 picks Heads with probability p and Tails with probability 1 − p, and that player 2 picks
both Head and Tail with probability 21 . Then, with probability
1 1 1
p + (1 − p) =
2 2 2
player 1 will receive a payoff 1. Similarly, she will receive a payoff -1 with the same probability. This
implies that if player 2 plays the strategy ( 21 , 21 ), then no matter what strategy player 1 chooses,
14 1. STATIC GAMES AND SOLUTION CONCEPTS
she will get the same payoff. Due to the symmetry of the game, we conclude that the randomized
strategy (( 21 , 21 ), ( 21 , 21 )) is a stable play of the game.
We next formalize the notion of “randomized" or mixed strategies. We first introduce some
notation. Let i denote the set of probability measures over the pure strategy (action) set Si . We
use σi ∈ i to denote the mixed strategy of player i. When Si is a finite set, a mixed strategy is
a finite dimensional probability vector, i.e., a vector whose elements denote the probability with
which a particular action will be played. If Si has two elements, the set of mixed strategies i
is the one-dimensional probability simplex, i.e., i = {(x1 , x2 ) | xi ≥ 0, x1 + x2 = 1}. We use

σ ∈  = i∈I i to denote a mixed strategy profile. Note that this implicitly assumes that players

randomize independently. We similarly denote σ−i ∈ −i = j =i j .
Following von Neumann-Morgenstern expected utility theory, we extend the payoff functions
ui from S to  by 
ui (σ ) = ui (s)dσ (s),
S
i.e., the payoff of a mixed strategy σ is given by the expected value of pure strategy payoffs under
the distribution σ .
We are now ready to define the notion of a mixed strategy Nash equilibrium.

Definition 1.13 Mixed Strategy Nash Equilibrium A mixed strategy profile σ ∗ is a mixed strategy
Nash equilibrium (or mixed Nash equilibrium) if for each player i,

ui (σi∗ , σ−i
∗ ∗
) ≥ ui (σi , σ−i ) for all σi ∈ i .

∗ )=
 ∗
Note that since ui (σi , σ−i Si ui (si , σ−i )dσi (si ), it is sufficient to check only pure strategy
“deviations” when determining whether a given profile is a Nash equilibrium. This leads to the
following characterization of a mixed Nash equilibrium.

Proposition 1.14 A mixed strategy profile σ ∗ is a (mixed strategy) Nash equilibrium if and only if for
each player i,
ui (σi∗ , σ−i
∗ ∗
) ≥ ui (si , σ−i ) for all si ∈ Si .

We also have the following useful characterization of a mixed Nash equilibrium in finite games
(i.e., games with finite strategy sets).

Proposition 1.15 Let G = I , (Si )i∈I , (ui )i∈I  be a finite strategic form game. Then, σ ∗ ∈  is a
Nash equilibrium if and only if for each player i ∈ I , every pure strategy in the support of σi∗ is a best
∗ .
response to σ−i
 1.2. SOLUTION CONCEPTS 15
Proof. Let σ ∗ be a mixed strategy Nash equilibrium, and let Ei∗ = ui (σi∗ , σ−i
∗ ) denote the expected

utility for player i. By Proposition 1.14, we have

Ei∗ ≥ ui (si , σ−i
∗
) for all si ∈ Si .
We first show that Ei∗ = ui (si , σ−i ∗ ) for all s in the support of σ ∗ (combined with the preceding
i i
relation, this proves one implication). Assume to arrive at a contradiction that this is not the case,
i.e., there exists an action si in the support of σi∗ such that ui (si , σ−i ∗ ) < E ∗ . Since u (s , σ ∗ ) ≤ E ∗
i i i −i i
for all si ∈ Si , this implies that

σi∗ (si )ui (si , σ−i
∗
) < Ei∗ ,
si ∈Si

– a contradiction. The proof of the other implication is similar and is therefore omitted. 2
It follows from this characterization that every action in the support of any player’s equilibrium
mixed strategy yields the same payoff. Note that this characterization result extends to games with
infinite strategy sets: σ ∗ ∈  is a Nash equilibrium if and only if for each player i ∈ I , given σ−i ∗ ,

no action in Si yields a payoff that exceeds his equilibrium payoff, and the set of actions that yields
a payoff less than his equilibrium payoff has σi∗ -measure zero.
Example 1.16 Let us return to the Battle of the Sexes game.

BALLET SOCCER
BALLET 2, 1 0, 0
SOCCER 0, 0 1, 2
Battle of the Sexes.
Recall that this game has 2 pure Nash equilibria. Using the characterization result in Propo-
sition 1.15, we show that it has a unique mixed strategy Nash equilibrium (which is not a pure
strategy Nash equilibrium). First, by using Proposition 1.15 (and inspecting the payoffs), it can be
seen that there are no Nash equilibria where only one of the players randomizes over its actions.
Now, assume instead that player 1 chooses the action Ballet with probability p ∈ (0, 1) and Soccer
with probability 1 − p, and that player 2 chooses Ballet with probability q ∈ (0, 1) and Soccer
with probability 1 − q. Using Proposition 1.15 on player 1’s payoffs, we have the following relation
2 × q + 0 × (1 − q) = 0 × q + 1 × (1 − q) .
Similarly, we have
1 × p + 0 × (1 − p) = 0 × p + 2 × (1 − p) .
We conclude that the only possible mixed strategy Nash equilibrium is given by q = 1
3 and p = 23 .

We conclude this section by discussing strict dominance by a mixed strategy. Consider the
following game:
16 1. STATIC GAMES AND SOLUTION CONCEPTS

LEFT RIGHT
UP 2, 0 í 1, 0
MIDDLE 0, 0 0, 0
DOWN í 1, 0 2, 0

Note that Player 1 has no pure strategy that strictly dominates Middle. However, the mixed
strategy ( 21 , 0, 21 ) of player 1 yields a strictly higher payoff than the pure strategy Middle. In this
case, we say that the strategy Middle is strictly dominated by the strategy ( 21 , 0, 21 ).

Definition 1.17 Strict Domination by Mixed Strategies A strategy si is strictly dominated for
player i if there exists a mixed strategy σi ∈ i such that ui (σi , s−i ) > ui (si , s−i ), for all s−i ∈ S−i .

It can be shown that strictly dominated strategies are never used with positive probability in a
mixed strategy Nash Equilibrium. In contrast, as we have seen in the Second Price Auction example,
weakly dominated strategies can be used in a Nash Equilibrium.

1.2.4 CORRELATED EQUILIBRIUM

In a Nash equilibrium, players choose strategies (or randomize over strategies) independently. For
games with multiple Nash equilibria, one may want to allow for randomizations between Nash
equilibria by some form of communication prior to the play of the game.

Example 1.18 Battle of the Sexes Suppose that in the Battle of the Sexes game, the players flip
a coin and go to the Ballet if the outcome of the coin flip is heads, and to the Football game if the
outcome is tails, i.e., they randomize between two pure strategy Nash equilibria, resulting in a payoff
of (3/2, 3/2).

The coin flip is one way of communication prior to the play. A more general form of commu-
nication is to find a trusted mediator who can perform general randomizations, as illustrated in the
next example.

Example 1.19 Traffic Intersection Game Consider a game where two cars arrive at an intersection
simultaneously. Row player (player 1) has the option to play U or D, and the column player (player
2) has the option to play L or R with payoffs given by
L R
U 5, 1 0, 0
D 4, 4 1, 5
Traffic Intersection Game.
 1.2. SOLUTION CONCEPTS 17
There are two pure strategy Nash equilibria: the strategy profiles (U, L) and (D, R). To find
the mixed strategy Nash equilibria, assume player 1 plays U with probability p and player 2 plays R
with probability q. Using Proposition 1.15, we obtain
5q = 4q + (1 − q),
5p = 4p + (1 − p),
which imply that p = q = 1/2. This shows that there is a unique mixed strategy equilibrium with
expected payoffs (5/2,5/2).
Case 1: Assume that there is a publicly observable random variable, e.g., a fair coin, such that if the
outcome is heads, player 1 plays U and player 2 plays L, and if the outcome is tails, player 1 plays
D and player 2 plays R. The expected payoffs for this play of the game is given by (3,3). We show
that no player has an incentive to deviate from the “recommendation" of the coin. If player 1 sees
heads, he believes that player 2 will play L, and, therefore, playing U is his best response (a similar
argument holds when he sees tails). Similarly, if player 2 sees a heads, he believes that player 1 will
play U , and, therefore, playing L is his best response (a similar argument holds when he sees tails).
Case 2: Next, consider a more elaborate signaling scheme. Suppose the players find a mediator who
chooses x ∈ {1, 2, 3} with equal probability 1/3. She then sends the following messages:
• If x = 1, player 1 plays U , player 2 plays L.
• If x = 2, player 1 plays D, player 2 plays L.
• If x = 3, player 1 plays D, player 2 plays R.
We show that no player has an incentive to deviate from the “recommendation" of the mediator:
• If player 1 gets the recommendation U , he believes player 2 will play L, so his best response
is to play U .
• If player 1 gets the recommendation D, he believes player 2 will play L, R with equal probability,
so his best response is to play D.
• If player 2 gets the recommendation L, he believes player 1 will play U, D with equal probability,
so his best response is to play L.
• If player 2 gets the recommendation R, he believes player 1 will play D, so his best response
is to play R.
Thus, the players will follow the mediator’s recommendations.With the mediator, the expected
payoffs are (10/3, 10/3), strictly higher than what the players could get by randomizing between
Nash equilibria.

The preceding examples lead us to the notions of correlated strategies and “correlated equi-
librium". Let (S) denote the set of probability measures over a metric space S. Let R be a random
18 1. STATIC GAMES AND SOLUTION CONCEPTS
variable taking values in S = ni=1 Si distributed according to π . An instantiation of R is a pure
strategy profile and the i th component of the instantiation will be called the recommendation to player
i. Given such a recommendation, player i can use conditional probability to form posterior beliefs
about the recommendations given to the other players. A distribution π is defined to be a correlated
equilibrium if no player can ever expect to unilaterally gain by deviating from his recommendation,
assuming the other players play according to their recommendations.

Definition 1.20 A correlated equilibrium of a finite game is a joint probability distribution π ∈ (S)
such that if R is a random variable distributed according to π then

Prob(R = s|Ri = si ) ui (ti , s−i ) − ui (s) ≤ 0 (1.1)
s−i ∈S−i

for all players i, all si ∈ Si such that Prob(Ri = si ) > 0, and all ti ∈ Si .

We have the following useful characterization for correlated equilibria in finite games.

Proposition 1.21 A joint distribution π ∈ (S) is a correlated equilibrium of a finite game if and only
if

π(s) ui (ti , s−i ) − ui (s) ≤ 0 (1.2)
s−i ∈S−i

for all players i and all si , ti ∈ Si such that si  = ti .

Proof. Using the definition of conditional probability, we can rewrite the definition of a correlated
equilibrium as
 π(s)
ui (ti , s−i ) − ui (s) ≤ 0
t−i ∈S−i π(si , t−i )
s−i ∈S−i

for all i, all si ∈ Si such that t−i ∈S−i π(si , t−i ) > 0, and all ti ∈ Si . The denominator does not
depend on the variable of summation so it can be factored out of the sum and canceled, yielding
the simpler condition that (1.2) holds for all i, all si ∈ Si such that t−i ∈S−i π(si , t−i ) > 0, and all
ti ∈ Si . But if t−i ∈S−i π(si , t−i ) = 0 then the left-hand side of (1.2) is zero regardless of i and
ti , so the equation always holds trivially in this case. The equation (1.2) also holds trivially when
si = ti , so we only need to check it in the case si  = ti . 2
We can alternatively think of correlated equilibria as joint distributions corresponding to
recommendations which will be given to the players as part of an extended game. The players are
then free to play any function of their recommendation (this is called a departure function) as their
strategy in the game. If it is a Nash equilibrium of this extended game for each player to play
his recommended strategy (i.e., to use the identity departure function), then the distribution is a
correlated equilibrium.This interpretation is justified by the following alternative characterization of
 1.3. EXISTENCE OF A NASH EQUILIBRIUM 19
correlated equilibria, which is useful in defining correlated equilibria for games with infinite strategy
spaces (see [98]).

Proposition 1.22 A joint distribution π ∈ (S) is a correlated equilibrium of a finite game if and only
if 
π(s) ui (ζi (si ), s−i ) − ui (s) ≤ 0 (1.3)
s∈S
for all players i and all functions ζi : Si → Si .

Proof. By substituting ti = ζi (si ) into (1.2) and summing over all si ∈ Si , we obtain (1.3) for any i
and any ζi : Si → Si . For the converse, define ζi for any si , ti ∈ Si by

ti ri = si
ζi (ri ) =
ri else.

Then all the terms in (1.3) except the si terms cancel, yielding (1.2). 2

1.3 EXISTENCE OF A NASH EQUILIBRIUM

In this section, we study the existence of a Nash equilibrium in both games with finite and infinite pure
strategy sets. We start with an example pricing-congestion game, where players have infinitely many
pure strategies. We consider two instances of this game, one with a unique pure Nash equilibrium,
and the other with no pure Nash equilibria. This model will be studied in more detail in Chapter 3.

Example 1.23 Pricing-Congestion Game We consider a price competition model that was stud-
ied in [5] (see also [4], [6], [83], and [27])5 .

l1(x) ,p1

1 unit of traffic

Reservation utility R
l2(x) ,p2

Figure 1.4: A price competition model over congested networks.

Consider a parallel link network with I links. Assume that d units of flow are to be routed
through this network. We assume that this flow is the aggregate flow of many infinitesimal users.
5This model will be studied in detail in Chapter 3.
20 1. STATIC GAMES AND SOLUTION CONCEPTS
Let li (xi ) denote the latency function of link i, which represents the delay or congestion cost as
a function of the total flow xi on link i. Assume that the links are owned by independent service
providers, which set a price pi per unit of flow on link i (see Figure 1.4). The effective cost of using
link i is pi + li (xi ). Users have a reservation utility equal to R, i.e., if pi + li (xi ) > R, then no traffic
will be routed on link i.
We consider a special case of this model with two links and latency functions l1 (x1 ) = 0
and l2 (x2 ) = 3x22 . For simplicity, we assume that R = 1 and d = 1. Given the prices (p1 , p2 ), we
assume that the flow is allocated according to a Wardrop equilibrium [37], i.e., the flows are routed
along minimum effective cost paths and the effective cost cannot exceed the reservation utility (see
also Def. 3.2 in Chapter 3). Formally, a flow vector x = [xi ]i=1,...,I is a Wardrop equilibrium if
I
i=1 xi ≤ 1 and

pi + li (xi ) = min{pj + lj (xj )}, for all i with xi > 0,

j

pi + li (xi ) ≤ 1, for all i with xi > 0,

I
with = 1 if minj {pj + lj (xj )} < 1.
i=1 xi
We use the preceding characterization to determine the flow allocation on each link given
prices 0 ≤ p1 , p2 ≤ 1:
2
3 (p1 − p2 ), p1 ≥ p2 ,
x2 (p1 , p2 ) =
0, otherwise,

and x1 (p1 , p2 ) = 1 − x2 (p1 , p2 ). The payoffs for the providers are then given by:
u1 (p1 , p2 ) = p1 × x1 (p1 , p2 ),
u2 (p1 , p2 ) = p2 × x2 (p1 , p2 ).
We find the pure strategy Nash equilibria of this game by characterizing the best response
correspondences, Bi (p−i ) for each player6 . In particular, for a given p2 , B1 (p2 ) is the optimal solution
set of the following optimization problem

maximize 0≤p1 ≤1, 0≤x1 ≤1 p1 x1

3
subject to p1 = p2 + (1 − x1 ).
2
Solving the preceding optimization problem, we find that

3 p2
B1 (p2 ) = min 1, + .
4 2
p1
Similarly, B2 (p1 ) = 2 .
6The following analysis assumes that at the Nash equilibria (p , p ) of the game, the corresponding Wardrop equilibria x satisfies
1 2
x1 > 0, x2 > 0, and x1 + x2 = 1. For the proofs of these statements, see Acemoglu and Ozdaglar [5].
 1.3. EXISTENCE OF A NASH EQUILIBRIUM 21
Best Response Functions
1

0.8

0.6
2
p

0.4

0.2

0
0 0.2 0.4 0.6 0.8 1
p
1

Figure 1.5: Best response correspondences.

Figure 1.5 illustrates the best response correspondences as a function of p1 and p2 . These
correspondences intersect at the unique point (p1 , p2 ) = (1, 21 ), which is the unique pure strategy
Nash equilibrium.
We next consider a similar example with latency functions given by

0 if 0 ≤ x ≤ 1/2
l1 (x) = 0, l2 (x) = x−1/2
 x ≥ 1/2,

for some sufficiently small  > 0. For this example, we show that no pure Nash equilibrium strategy
exists for small . The following list considers all candidate Nash equilibria (p1 , p2 ) and profitable
unilateral deviations for  sufficiently small, thus establishing the nonexistence of a pure strategy
Nash equilibrium:

1. p1 = p2 = 0: A small increase in the price of provider 1 will generate positive profits, thus
provider 1 has an incentive to deviate.

2. p1 = p2 > 0: Let x be the corresponding flow allocation. If x1 = 1, then provider 2 has an

incentive to decrease its price. If x1 < 1, then provider 1 has an incentive to decrease its price.

3. 0 ≤ p1 < p2 : Player 1 has an incentive to increase its price since its flow allocation remains
the same.
22 1. STATIC GAMES AND SOLUTION CONCEPTS
4. 0 ≤ p2 < p1 : For  sufficiently small, the profit function of player 2, given p1 , is strictly
increasing as a function of p2 , showing that provider 2 has an incentive to increase its price.

We are interested in establishing conditions which guarantee that a strategic form game has
a (mixed) Nash equilibrium. Recall that a mixed strategy profile σ ∗ is a Nash equilibrium if

ui (σi∗ , σ−i
∗ ∗
) ≥ ui (σi , σ−i ), for all σi ∈ i .

In other words, σ ∗ is a Nash equilibrium if and only if σi∗ ∈ Bi∗ (σ−i ∗ ) for all i, where B ∗ (σ ∗ ) is
i −i
∗ . We define the
the best response of player i, given that the other players use the strategy profile σ−i
correspondence B :  ⇒  such that for all σ ∈ , we have

B(σ ) = [Bi (σ−i )]i∈I . (1.4)

The existence of a Nash equilibrium is then equivalent to the existence of a mixed strategy σ such
that σ ∈ B(σ ), i.e., existence of a fixed point of the correspondence B. We will use Kakutani’s fixed
point theorem to establish conditions under which there exists a Nash equilibrium (see [53]).

Theorem 1.24 Kakutani’s Fixed Point Theorem Let f : A ⇒ A be a correspondence, with x ∈

A → f (x) ⊂ A, satisfying the following conditions:

1. A is a compact, convex, and non-empty subset of a finite dimensional Euclidean space.

2. f (x) is non-empty for all x ∈ A.

3. f (x) is a convex-valued correspondence: for all x ∈ A, f (x) is a convex set.

4. f (x) has a closed graph: If {x n , y n } → {x, y} with y n ∈ f (x n ), then y ∈ f (x).

Then, there exists some x ∈ A, such that x ∈ f (x).

1.3.1 GAMES WITH FINITE PURE STRATEGY SETS

In this section, we show that a finite game always has a mixed strategy Nash equilibrium. In proving
this, we use the following theorem that establishes the existence of an optimal solution for an
optimization problem (see [21] for the proof ).

Theorem 1.25 Weierstrass Let A be a nonempty compact subset of a finite dimensional Euclidean
space and let f : A → R be a continuous function.Then there exists an optimal solution to the optimization
problem

minimize f (x)
subject to x ∈ A.
 1.3. EXISTENCE OF A NASH EQUILIBRIUM 23

We proceed to the main equilibrium existence result, established by Nash [78].

Theorem 1.26 Any finite strategic game has a mixed strategy Nash equilibrium.

Proof. We will apply Kakutani’s theorem to the best response correspondence B :  ⇒  defined
in Eq. (1.4). We show that B(σ ) satisfies the conditions of Kakutani’s theorem.
1. The set  is compact, convex, and non-empty.

By definition,  = i∈I i , where each i is a probability simplex of dimension |Si | − 1,
showing the desired property.
2. The set B(σ ) is non-empty for all σ ∈ .
By definition,
Bi (σ−i ) ∈ arg max ui (x, σ−i ).
x∈i

Here i is non-empty and compact, and ui is linear in x. Therefore, Weirstrass’ theorem applies,
showing that B(σ ) is non-empty.
3. The correspondence B(σ ) is a convex-valued correspondence.
We show that for all σ ∈ , the set B(σ ) is a convex set, or equivalently, Bi (σ−i ) is a convex
set for all i. Let σi , σi ∈ Bi (σ−i ). Then, for all λ ∈ [0, 1] ∈ Bi (σ−i ), we have

ui (σi , σ−i ) ≥ ui (τi , σ−i ) for all τi ∈ i ,

ui (σi , σ−i ) ≥ ui (τi , σ−i ) for all τi ∈ i .

The preceding relations imply that for all λ ∈ [0, 1], we have

λui (σi , σ−i ) + (1 − λ)ui (σi , σ−i ) ≥ ui (τi , σ−i ) for all τi ∈ i .

By the linearity of ui ,

ui (λσi + (1 − λ)σi , σ−i ) ≥ ui (τi , σ−i ) for all τi ∈ i .

Therefore, λσi + (1 − λ)σi ∈ Bi (σ−i ), showing that B(σ ) is convex-valued.

4. The correspondence B(σ ) has a closed graph.
Assume, to arrive at a contradiction, that B(σ ) does not have a closed graph. Then, there
exists a sequence (σ n , σ̂ n ) → (σ, σ̂ ) with σ̂ n ∈ B(σ n ), but σ̂ ∈
/ B(σ ), i.e., there exists some i such
that σ̂i ∈
/ Bi (σ−i ). This implies that there exists some σi ∈ i and some  > 0 such that

ui (σi , σ−i ) > ui (σ̂i , σ−i ) + 3.

n → σ , we have for sufficiently large n,
By the continuity of ui and the fact that σ−i −i

n
ui (σi , σ−i ) ≥ ui (σi , σ−i ) − .
24 1. STATIC GAMES AND SOLUTION CONCEPTS
Combining the preceding two relations, we obtain
n
ui (σi , σ−i ) > ui (σ̂i , σ−i ) + 2 ≥ ui (σ̂in , σ−i
n
) + ,

where the second relation follows from the continuity of ui . This contradicts the assumption that
σ̂in ∈ Bi (σ−i
n ), and completes the proof. 2
Since every mixed Nash equilibrium is a correlated equilibrium, the existence of a correlated
equilibrium in a finite strategic form game follows from this result.

1.3.2 GAMES WITH INFINITE PURE STRATEGY SETS

In the previous section, we showed that every finite strategic form game has a mixed strategy Nash
equilibrium. The proof relies on Kakutani’s fixed point theorem and the fact that using mixed
strategies essentially “convexifies" the strategy spaces in finite games. In this section, we investigate
the question of the existence of a pure Nash equilibrium in games where the players have infinitely
many pure strategies. In particular, we show that under convexity assumptions on the strategy sets
and the payoff functions, there exists a pure strategy Nash Equilibrium.
We first introduce some basic convexity concepts. For a given set C ⊂ Rn and function f :
C → R, denote the t-upper level set of f by

Lf (t) = {x ∈ C | f (x) ≥ t}.

Definition 1.27 Let C be a nonempty convex subset of Rn .

(i) A function g : C → R is called concave if for all x, y ∈ C, and for all λ ∈ [0, 1], we have

f (λx + (1 − λ)y) ≥ λf (x) + (1 − λ)f (y).

(ii) A function g : C → R is called quasi-concave if for all t, Lg (t) is a convex set, i.e., for all
x, y ∈ C with g(x) ≥ t and g(y) ≥ t, and for all λ ∈ [0, 1], we have

g(λx + (1 − λ)y) ≥ t.

We next show that under convexity assumptions, games with infinite strategy sets have pure
Nash equilibria.

Theorem 1.28 Debreu, Glicksberg and Fan Consider a strategic form game I , (Si ), (ui ), where
I is a finite set. Assume that the following holds for each i ∈ I :
1. Si is a non-empty, convex, and compact subset of a finite-dimensional Euclidean space.

2. ui (s) is continuous in s.
 1.3. EXISTENCE OF A NASH EQUILIBRIUM 25
3. ui (si , s−i ) is quasi-concave in si .

Then, the game I , (si ), (ui ) has a pure strategy Nash equilibrium.

I
Proof. (outline) Consider the best response correspondence B : S ⇒ S, where S = i=1 Si , defined
by
B(s) = [Bi (s−i )]i∈I .

Following a similar argument as in the proof of Theorem 1.26, it can be seen that the correspondence
B satisfies the conditions of Kakutani’s theorem: in step 1, use the convexity assumption on the
strategy sets, and in step 3, use the quasi-concavity assumption on the utility functions instead of
linearity. 2
The existence of a mixed Nash equilibrium in finite games is a special case of the preceding
theorem: for finite games, the utility function of a player is linear in its mixed strategy, hence satisfies
the assumptions of the preceding theorem. What happens when we relax quasi-concavity? The
following example shows that without this assumption, a pure strategy Nash equilibrium may fail
to exist.

Example 1.29 Unit Circle Example

Two players pick points s1 and s2 on the unit circle. The payoffs for the two players are

u1 (s1 , s2 ) = d(s1 , s2 )

u2 (s1 , s2 ) = −d(s1 , s2 )

where d(x, y) denotes the Euclidean distance between the vectors x, y ∈ R2 . It can be seen that
there is no pure strategy Nash equilibrium in this game, i.e., if both players pick the same location,
player 1 has an incentive to deviate. If they pick different locations, player 2 has an incentive to
deviate.

Despite the non-existence of a pure strategy Nash equilibrium in the preceding example, one
can show that each player choosing a uniformly random point on the circle is a mixed strategy Nash
equilibrium. In the next section, we will address the question of existence of a mixed Nash equilibrium
in games where each player has infinitely many pure strategies. We will first study “continuous
games". We will then focus on “discontinuous games", which arise in competition models among
service providers in communication networks, and delineate the conditions under which a mixed
Nash equilibrium exists.
26 1. STATIC GAMES AND SOLUTION CONCEPTS
1.3.3 CONTINUOUS GAMES
In this section, we study games in which players have infinitely many pure strategies. In particular,
we want to include the possibility that the pure strategy set of a player may be a bounded interval
on the real line, such as [0,1].7

Definition 1.30 A continuous game is a game I , (Si ), (ui ) where I is a finite set, the Si are
nonempty compact metric spaces, and the ui : S → R are continuous functions.

A compact metric space is a general mathematical structure for representing infinite sets that can
be well approximated by large finite sets. One important fact is that, in a compact metric space, any
infinite sequence has a convergent subsequence. Any closed bounded subset of a finite-dimensional
Euclidean space is an example of a compact metric space. More specifically, any closed bounded
interval of the real line is an example of a compact metric space, where the distance between two
points x and y is given by |x − y|. In our treatment, we will not need to refer to any examples more
complicated than these (see Appendix 1.A for basic definitions of a metric space and convergence
notions for probability measures).
We next state the analogue of Nash’s Theorem for continuous games.

Theorem 1.31 Glicksberg Every continuous game has a mixed strategy Nash equilibrium.

With continuous strategy spaces, the space of mixed strategies  is infinite-dimensional;

therefore, we need a more powerful fixed point theorem than the version of Kakutani we have
used before. Here we adopt an alternative approach to prove Glicksberg’s Theorem, which can be
summarized as follows:

• We approximate the original game with a sequence of finite games, which correspond to
successively finer discretizations of the original game.

• We use Nash’s Theorem to produce an equilibrium for each approximation.

• We use the weak topology and the continuity assumptions to show that these converge to an
equilibrium of the original game.

1.3.3.1 Closeness of Two Games and -Equilibrium

Let u = (u1 , . . . , uI ) and ũ = (ũ1 , . . . , ũI ) be two profiles of utility functions defined on S such
that for each i ∈ I , the functions ui : S → R and ũi : S → R are bounded measurable functions.
We may define the distance between the utility function profiles u and ũ as

max sup |ui (s) − ũi (s)|.

i∈I s∈S

7 Our development follows that of Myerson [77].

 1.3. EXISTENCE OF A NASH EQUILIBRIUM 27
Consider two strategic form games defined by the two profiles of utility functions:

G = I , (Si ), (ui ), G̃ = I , (Si ), (ũi ).

Even if u and ũ are very close, the equilibria of G and G̃ may be far apart. For example, assume
there is only one player, S1 = [0, 1], u1 (s1 ) = s1 , and ũ1 (s1 ) = −s1 , where  > 0 is a sufficiently
small scalar. The unique equilibrium of G is s1∗ = 1, and the unique equilibrium of G̃ is s1∗ = 0, even
if the distance between u and ũ is only 2.
However, if u and ũ are very close, there is a sense in which the equilibria of G are “almost"
equilibria of G̃.

Definition 1.32 (-equilibrium) Given  ≥ 0, a mixed strategy σ ∈  is called an -equilibrium

if for all i ∈ I and si ∈ Si ,
ui (si , σ−i ) ≤ ui (σi , σ−i ) + .
Clearly, an -equilibrium with  = 0 is a Nash equilibrium.

The following result shows that such -equilibria have a continuity property across games.

Proposition 1.33 Let G be a continuous game. Assume that σ k → σ ,  k → , and for each k, σ k is an
 k -equilibrium of G. Then σ is an -equilibrium of G.

Proof. For all i ∈ I , and all si ∈ Si , we have

k
ui (si , σ−i ) ≤ ui (σ k ) + .

Taking the limit as k → ∞ in the preceding relation, and using the continuity of the utility functions
together with the convergence of probability distributions under weak topology [see Eq. (1.14)], we
obtain,
ui (si , σ−i ) ≤ ui (σ ) + ,
establishing the result. 2
We next define formally the closeness of two strategic form games.

Definition 1.34 Let G and G be two strategic form games with

G = I , (Si ), (ui ), G̃ = I , (Si ), (ũi ).

Assume that the utility functions ui and ũi are measurable and bounded. Then G is an
α−approximation to G if for all i ∈ I and s ∈ S, we have

|ui (s) − ui (s)| ≤ α.

28 1. STATIC GAMES AND SOLUTION CONCEPTS

We next relate the −equilibria of close games.

Proposition 1.35 If G is an α-approximation to G and σ is an -equilibrium of G , then σ is an

( + 2α)-equilibrium of G.

Proof. For all i ∈ I and all si ∈ Si , we have

ui (si , σ−i ) − ui (σ ) = ui (si , σ−i ) − ui (si , σ−i ) + ui (si , σ−i ) − ui (σ ) + ui (σ ) − ui (σ )

≤  + 2α.

The next proposition shows that we can approximate a continuous game with an essentially
finite game to an arbitrary degree of accuracy.

Proposition 1.36 For any continuous game G and any α > 0, there exists an “essentially finite" game
which is an α-approximation to G.

Proof. Since S is a compact metric space, the utility functions ui are uniformly continuous, i.e., for
all α > 0, there exists some  > 0 such that

ui (s) − ui (t) ≤ α,

for all d(s, t) ≤ . Since Si is a compact metric space, it can be covered with finitely many open balls
j
Ui , each with radius less than . Assume without loss of generality that these balls are disjoint and
j j
nonempty. Choose an si ∈ Ui for each i, j . Define the “essentially finite" game G with the utility
functions ui defined as

j j

I
j
ui (s) = ui (s1 , . . . , sI ), for all s ∈ U j = Uk .
k=1

Then for all s ∈ S and all i ∈ I , we have

|ui (s) − ui (s)| ≤ α,

since d(s, s j ) ≤  for all j , implying the desired result. 2

 1.3. EXISTENCE OF A NASH EQUILIBRIUM 29
1.3.3.2 Proof of Glicksberg’s Theorem
We now return to the proof of Glicksberg’s Theorem. Let {α k } be a scalar sequence with α k ↓ 0.

• For each α k , there exists an “essentially finite" α k -approximation Gk of G by Proposition 1.36.

• Since Gk is “essentially finite" for each k, it follows using Nash’s Theorem that it has a 0-
equilibrium, which we denote by σ k .

• Then, by Proposition 1.35, σ k is a 2α k -equilibrium of G.

• Since  is compact, {σ k } has a convergent subsequence. Without loss of generality, we assume

that σ k → σ .

• Since 2α k → 0, σ k → σ , by Proposition 1.33, it follows that σ is a 0-equilibrium of G.

1.3.4 DISCONTINUOUS GAMES

There are many games in which the utility functions are not continuous (e.g., price competition
models, congestion-competition models in networks). The next theorem shows that for discontin-
uous games, under some mild semicontinuity conditions on the utility functions, it is possible to
establish the existence of a mixed Nash equilibrium (see Dasgupta and Maskin [40]-[41]). The key
assumption is to allow discontinuities in the utility function to occur only on a subset of measure
zero, in which a player’s strategy is “related" to another player’s strategy. To formalize this notion,
we introduce the following set: for any two players i and j , let D be a finite index set and for d ∈ D,
let fijd : Si → Sj be a bijective and continuous function. Then, for each i, we define

S ∗ (i) = {s ∈ S | there exists j  = i such that sj = fijd (si ).} (1.5)

Before stating the theorem, we first introduce some weak continuity conditions.

Definition 1.37 Let X be a subset of Rn , Xi be a subset of R, and X−i be a subset of Rn−1 .

(i) A function f : X → R is called upper semicontinuous (respectively, lower semicontinuous) at a

vector x ∈ X if f (x) ≥ lim supk→∞ f (xk ) (respectively, f (x) ≤ lim inf k→∞ f (xk )) for every
sequence {xk } ⊂ X that converges to x. If f is upper semicontinuous (lower semicontinuous)
at every x ∈ X, we say that f is upper semicontinuous (lower semicontinuous).
∗ ⊂
(ii) A function f : Xi × X−i → R is called weakly lower semicontinuous in xi over a subset X−i
∗ ,
X−i , if for all xi there exists λ ∈ [0, 1] such that, for all x−i ∈ X−i

λ lim inf f (xi , x−i ) + (1 − λ) lim inf f (xi , x−i ) ≥ f (xi , x−i ).
xi ↑xi xi ↓xi
30 1. STATIC GAMES AND SOLUTION CONCEPTS
Theorem 1.38 (Dasgupta and Maskin) Let Si be a closed interval of R. Assume that ui is continuous
except on a subset S ∗∗ (i) of the set S ∗ (i) defined in Eq. (1.5). Assume also that ni=1 ui (s) is upper
semicontinuous and that ui (si , s−i ) is bounded and weakly lower semicontinuous in si over the set {s−i ∈
S−i | (si , s−i ) ∈ S ∗∗ (i)}. Then the game has a mixed strategy Nash equilibrium.

The weakly lower semicontinuity condition on the utility functions implies that the function
ui does not jump up when approaching si , either from below or above. Loosely, this ensures that
player i can do almost as well with strategies near si as with si , even if his opponents put weight on
the discontinuity points of ui .
In the following examples, we consider games with discontinuous utility functions and study
their pure and mixed strategy Nash equilibria. The first example is the Bertrand competition, which
is a standard model of competition among firms selling a homogeneous good. It is particularly
relevant in network games where service providers set prices over resources and compete over user
demand.

Example 1.39 Bertrand Competition with Capacity Constraints Consider two firms that
charge prices p1 , p2 ∈ [0, 1] per unit of the same good. Assume that there is unit demand and
all customers choose the firm with the lower price. If both firms charge the same price, each firm
gets half the demand. All demand has to be supplied. The payoff functions of each firm is the profit
they make (we assume for simplicity that cost of supplying the good is equal to 0 for both firms).

(a) We first study the pure strategy Nash equilibria of this game. For this, we consider all possible
candidate strategy profiles and check if there is any profitable unilateral deviation:

– p1 = p2 > 0: each of the firms has an incentive to reduce their price to capture the whole
demand and increase profits.
– p1 < p2 : Firm 1 has an incentive to slightly increase his price.
– p1 = p2 = 0: Neither firm can increase profits by changing its price unilaterally. Hence,
(p1 , p2 ) = (0, 0) is the unique pure strategy Nash equilibrium.

(b) Assume now that each firm has a capacity constraint of 2/3 units of demand (since all demand
has to be supplied, this implies that when p1 < p2 , firm 2 gets 1/3 units of demand). It can be
seen in this case that the strategy profile (p1 , p2 ) = (0, 0) is no longer a pure strategy Nash
equilibrium: either firm can increase his price and still have 1/3 units of demand due to the
capacity constraint on the other firm, thus making positive profits.
It can be established using Theorem 1.38 that there exists a mixed strategy Nash equilibrium.
Let us next proceed to construct a mixed strategy Nash equilibrium. We focus on symmetric
Nash equilibria, i.e., both firms use the same mixed strategy. We use the cumulative distribution
function F (·) to represent the mixed strategy used by either firm. It can be seen that the
 1.3. EXISTENCE OF A NASH EQUILIBRIUM 31
expected payoff of player 1, when he chooses p1 and firm 2 uses the mixed strategy F (·), is
given by
p1 2
u1 (p1 , F (·)) = F (p1 ) + (1 − F (p1 )) p1 .
3 3
Using the fact that each action in the support of a mixed strategy must yield the same payoff
to a player at the equilibrium (cf. Proposition 1.15), we obtain for all p in the support of F (·),

p 2
−F (p) + p = k,
3 3

for some k ≥ 0. From this we obtain:

3k
F (p) = 2 − .
p

Note next that the upper support of the mixed strategy must be at p = 1, which implies that
F (1) = 1. Combining with the preceding, we obtain
⎧
⎪
⎨ 0, if 0 ≤ p ≤ 21 ,
F (p) = 2 − p1 , if 21 ≤ p ≤ 1,
⎪
⎩ 1, if p ≥ 1.

The analysis of the equilibria of the next game is similar and left as an exercise.

Example 1.40 Hoteling Competition Each of n candidates chooses a position to take on the real
line in the interval [0,1]. There is a continuum of citizens, whose favorite positions are uniformly
distributed between [0,1]. A candidate attracts votes of citizens whose favorite positions are closer to
his position than to the position of any other candidate; if k candidates choose the same position, then
each receives the fraction 1/k of the votes that the position attracts. The payoff of each candidate is
his vote share.

(a) Find all pure strategy Nash equilibria when n = 2.

(b) Show that there does not exist a pure strategy Nash equilibrium when n = 3. Find a mixed
strategy Nash equilibrium.
32 1. STATIC GAMES AND SOLUTION CONCEPTS
1.4 UNIQUENESS OF A NASH EQUILIBRIUM
In the previous section, we showed that under some convexity assumptions, games with infinite pure
strategy sets have a pure Nash equilibrium (see Theorem 1.28). The next example shows that even
under strict convexity assumptions, there may be infinitely many pure strategy Nash equilibria.

Example 1.41 Consider a game with 2 players, where Si = [0, 1] for i = 1, 2, and the payoffs are
given by
s12
u1 (s1 , s2 ) = s1 s2 − ,
2
s22
u2 (s1 , s2 ) = s1 s2 − .
2
Note that ui (s1 , s2 ) is strictly concave in si . It can be seen in this example that the best response
correspondences (which are unique-valued) are given by

B1 (s2 ) = s2 , B2 (s1 ) = s1 .

Plotting the best response curves shows that any pure strategy profile (s1 , s2 ) = (x, x) for x ∈ [0, 1]
is a pure strategy Nash equilibrium.

We will next establish conditions that guarantee that a strategic form game has a unique pure
strategy Nash equilibrium. We will follow the development of the classical paper by Rosen [88].
In order to discuss the uniqueness of an equilibrium, we provide a more explicit description
of the strategy sets of the players. In particular, we assume that for player i ∈ I , the strategy set Si
is given by
Si = {xi ∈ Rmi | hi (xi ) ≥ 0}, (1.6)
where hi : Rmi → R is a concave function. Since hi is concave, it follows that the set Si is a convex
 
set. Therefore, the set of strategy profiles S = Ii=1 Si ⊂ Ii=1 Rmi , being a Cartesian product of
convex sets, is a convex set.The following analysis can be extended to the case where Si is represented
by finitely many concave inequality constraints, but we do not do so here for clarity of exposition.

Given these strategy sets, a vector x ∗ ∈ Ii=1 Rmi is a pure strategy Nash equilibrium if and
only if for all i ∈ I , xi∗ is an optimal solution of the optimization problem
∗
maximizeyi ∈Rmi ui (yi , x−i ) (1.7)
subject to hi (yi ) ≥ 0.

(for a brief review of standard notation and results for nonlinear optimization problems, see Appendix
1.B). In the following, we use the notation ∇u(x) for the gradient vector, namely

∇u(x) = [∇1 u1 (x), . . . , ∇I uI (x)]T . (1.8)

 1.4. UNIQUENESS OF A NASH EQUILIBRIUM 33
We next introduce the key condition for uniqueness of a pure strategy Nash equilibrium.

Definition 1.42 We say that the payoff functions (u1 , . . . , uI ) are diagonally strictly concave for
x ∈ S, if for every x ∗ , x̄ ∈ S, we have

(x̄ − x ∗ )T ∇u(x ∗ ) + (x ∗ − x̄)T ∇u(x̄) > 0.

Theorem 1.43 Consider a strategic form game I , (Si ), (ui ). For all i ∈ I , assume that the strategy
sets Si are given by Eq. (1.6), where hi is a concave function, and there exists some x̃i ∈ Rmi such that
hi (x̃i ) > 0. Assume also that the payoff functions (u1 , . . . , uI ) are diagonally strictly concave for x ∈ S.
Then the game has a unique pure strategy Nash equilibrium.

Proof. Assume that there are two distinct pure strategy Nash equilibria. Since for each i ∈ I , both
xi∗ and x̄i must be an optimal solution for an optimization problem of the form (1.7), Theorem 1.46
implies the existence of nonnegative vectors λ∗ = [λ∗1 , . . . , λ∗I ]T and λ̄ = [λ̄1 , . . . , λ̄I ]T such that
for all i ∈ I , we have
∇i ui (x ∗ ) + λ∗i ∇hi (xi∗ ) = 0, (1.9)

λ∗i hi (xi∗ ) = 0, (1.10)

and
∇i ui (x̄) + λ̄i ∇hi (x̄i ) = 0, (1.11)

λ̄i hi (x̄i ) = 0. (1.12)

Multiplying Eqs. (1.9) and (1.11) by (x̄i − xi∗ )T and (xi∗ − x̄i )T , respectively, and adding over
all i ∈ I , we obtain

0 = (x̄ − x ∗ )T ∇u(x ∗ ) + 
(x ∗ − x̄)T ∇u(x̄) 
+ λ∗i ∇hi (xi∗ )T (x̄i − xi∗ ) + λ̄i ∇hi (x̄i )T (xi∗ − x̄i )
i∈I
 i∈I

> λ∗i ∇hi (xi∗ )T (x̄i − xi∗ ) + λ̄i ∇hi (x̄i )T (xi∗ − x̄i ), (1.13)
i∈I i∈I

where to get the strict inequality, we used the assumption that the payoff functions are diagonally
strictly concave for x ∈ S. Since the hi are concave functions, we have

hi (xi∗ ) + ∇hi (xi∗ )T (x̄i − xi∗ ) ≥ hi (x̄i ).

34 1. STATIC GAMES AND SOLUTION CONCEPTS
Using the preceding together with λ∗i > 0, we obtain for all i,

λ∗i ∇hi (xi∗ )T (x̄i − xi∗ ) ≥ λ∗i (hi (x̄i ) − hi (xi∗ ))

= λ∗i hi (x̄i )
≥ 0,

where to get the equality we used Eq. (1.10), and to get the last inequality, we used the facts λ∗i ≥ 0
and hi (x̄i ) ≥ 0. Similarly, we have

λ̄i ∇hi (x̄i )T (xi∗ − x̄i ) ≥ 0.

Combining the preceding two relations with the relation in (1.13) yields a contradiction, thus
concluding the proof. 2
Let U (x) denote the Jacobian of ∇u(x) [see Eq. (1.8)]. In particular, if the xi are all 1-
dimensional, then U (x) is given by
⎛ 2 ⎞
∂ u1 (x) ∂ 2 u1 (x)
2 · · ·
⎜ ∂x1 ∂x1 ∂x2
⎟
⎜ ∂ 2 u2 (x) .. ⎟
U (x) = ⎜ . ⎟.
⎝ ∂x2 ∂x1 ⎠
..
.

The next proposition provides a sufficient condition for the payoff functions to be diagonally
strictly concave.

Proposition 1.44 For all i ∈ I , assume that the strategy sets Si are given by Eq. (1.6), where hi is a
concave function. Assume that the symmetric matrix (U (x) + U T (x)) is negative definite for all x ∈ S,
i.e., for all x ∈ S, we have

y T (U (x) + U T (x))y < 0, ∀ y  = 0.

Then, the payoff functions (u1 , . . . , uI ) are diagonally strictly concave for x ∈ S.

Proof. Let x ∗ , x̄ ∈ S. Consider the vector

x(λ) = λx ∗ + (1 − λ)x̄, for some λ ∈ [0, 1].

Since S is a convex set, x(λ) ∈ S.

Because U (x) is the Jacobian of ∇u(x), we have
d dx(λ)
∇u(x(λ)) = U (x(λ))
dλ d(λ)
= U (x(λ))(x ∗ − x̄),
 1.A. APPENDIX: METRIC SPACES AND PROBABILITY MEASURES 35
or  1
U (x(λ))(x ∗ − x̄)dλ = ∇u(x ∗ ) − ∇u(x̄).
0

Multiplying the preceding by (x̄ − x ∗ )T yields

(x̄ − x ∗ )T ∇u(x ∗ ) + (x ∗ −
 x̄)
T
∇u(x̄)
1 1
= − (x − x̄)T [U (x(λ)) + U T (x(λ))](x ∗ − x̄)dλ
∗
2 0
> 0,

where to get the strict inequality we used the assumption that the symmetric matrix (U (x) + U T (x))
is negative definite for all x ∈ S. 2
Recent work has shown that the uniqueness of a pure strategy Nash equilibrium can be
established under conditions that are weaker than those given above. This analysis is beyond our
scope; for more information on this, see the papers [96], [97], [95].

1.A APPENDIX: METRIC SPACES AND PROBABILITY

MEASURES
This appendix summarizes the basic definitions related to metric spaces and probability measures
for completeness. A metric space is a set M together with a function d : M × M → R, that defines
the “distance" d(x, y) between any two points x, y in the set. The distance function satisfies the
following properties for every x, y, z ∈ M:

• d(x, y) = d(y, x) ≥ 0,

• d(x, y) = 0 if and only if x = y,

• d(x, y) + d(y, z) ≥ d(x, z).

In a metric space M, a point y ∈ M is the limit of a sequence of points {x k }∞ k=1 ⊂ M if and

only if the distance d(x , y) → 0 as k → ∞. An open ball of radius  around a point x, denoted by
k

B(x, ), is the set of all points in the metric space that have a distance less than  from x, i.e.,

B(x, ) = {y | d(x, y) < }.

A set S is an open subset of a metric space M if and only if for every x ∈ S, there exists some  > 0
such that B(x, ) ⊂ S. A metric space is compact if and only if every collection of open sets that
“covers" M (i.e., their union includes all of M) has a finite sub-collection that also covers M.
When there are infinitely many actions in the set Si , a mixed strategy for player i can no
longer be described by just listing the probability of each individual action (i.e., by a finite dimen-
sional probability vector). For example, suppose that Si is the interval [0, 1]. If player i selected his
36 1. STATIC GAMES AND SOLUTION CONCEPTS
action from a uniform probability distribution over the interval [0,1], then each individual action
in [0,1] would have zero probability; but the same would be true if he selected his action from
a uniform probability distribution over the interval [0.5,1]. To describe a probability distribution
over Si , we must list the probabilities of subsets of Si . Unfortunately, for technical reasons, it may be
mathematically impossible to consistently assign probabilities to all subsets of an infinite set, so some
weak restriction is needed on the class of subsets whose probabilities can be meaningfully defined.
These are called the measurable sets. Here, we let the measurable subsets of S and of each set Si be the
smallest class of subsets that includes all open subsets, all closed subsets, and all finite or countably
infinite unions and intersections of sets in the class. These are the Borel subsets (and they include
essentially all subsets that could be defined without the use of very sophisticated mathematics). Let
Bi denote the set of such measurable or Borel subsets of Si .
Let i denote the set of probability distributions over Si , i.e., σi ∈ i if and only if σi is a
function that assigns a nonnegative number σi (Q) to each Q that is a Borel subset of Si , σ (Si ) = 1,
and for any countable collection (Qk )∞ k=1 of pairwise-disjoint Borel subsets of Si ,
∞  ∞
 
σi Q =k
σi (Qk ).
k=1 k=1

We define convergence for mixed strategies by assigning the weak topology to i . Two impli-
cations of this topology are important for our purposes:
• The set of probability distributions i is a compact metric space, i.e., every sequence {σik } ⊂ i
has a convergent subsequence.
• A sequence {σik } ⊂ i of mixed strategies converges to σi ∈ i if and only if for all continuous
functions fi : Si → R, we have
 
lim f (si )dσi (si ) =
k
f (si )dσi (si ). (1.14)
k→∞ Si Si

The preceding condition asserts that if s̃ik is a random strategy drawn from Si according to the σik
distribution, and s̃i is a random strategy drawn from Si according to the σi distribution, then the
expected value of f (s̃ik ) must converge to the expected value of f (s̃i ) as k → ∞.
A function g : S → R is called (Borel) measurable if for every scalar t, the set {x ∈ S | g(x) ≥ t}
is a Borel subset of S. A function g : S → R is bounded if there exists some scalar K such that
|g(x)| ≤ K for all x ∈ S. To be able to define expected utilities, we must require that player’s utility
functions are measurable and bounded in this sense. Note that this assumption is weaker than the
continuity of the utility functions. With this assumption, the utility functions ui are extended from
 
S = Ij =1 Sj to the space of probability distributions  = Ij =1 j as follows:

ui (σ ) = ui (s)dσ (s),
S
where σ ∈ .
 1.B. APPENDIX: NONLINEAR OPTIMIZATION 37
1.B APPENDIX: NONLINEAR OPTIMIZATION
This appendix provides some notation and standard results for nonlinear optimization problems.
Given a scalar-valued function f : Rn → R, we use the notation ∇f (x) to denote the gradient
vector of f at point x, i.e.,
 
∂f (x) ∂f (x) T
∇f (x) = ,..., ,
∂x1 ∂xn
(recall our convention that all vectors are column vectors). Given a scalar-valued function u :
I
i=1 R
mi → R, we use the notation ∇ u(x) to denote the gradient vector of u with respect to
i
xi at point x, i.e.,
 T
∂u(x) ∂u(x)
∇i u(x) = ,..., . (1.15)
∂xi1 ∂ximi
We next state necessary conditions for the optimality of a feasible solution of a nonlinear
optimization problem. These conditions are referred to as the Karush-Kuhn-Tucker conditions in the
optimization literature.

Theorem 1.45 (Karush-Kuhn-Tucker conditions) Let x ∗ be an optimal solution of the optimization

problem

maximize f (x)
subject to gj (x) ≥ 0, j = 1, . . . , r,

where the cost function f : Rn → R and the constraint functions gj : Rn → R are continuously differ-
entiable. Denote the set of active constraints at x ∗ as A(x ∗ ) = {j = 1, . . . , r | gj (x ∗ ) = 0}. Assume that
the active constraint gradients, ∇gj (x ∗ ), j ∈ A(x ∗ ), are linearly independent vectors. Then, there exists
a nonnegative vector λ∗ ∈ Rr (Lagrange multiplier vector) such that


r
∇f (x ∗ ) + λ∗j ∇gj (x ∗ ) = 0,
j =1

λ∗j gj (x ∗ ) = 0, ∀ j = 1, . . . , r. (1.16)

One can view the preceding theorem as a generalization of the Lagrange multiplier theorem
for equality constrained optimization problems to inequality constrained optimization problems.
The condition that the active constraint gradients, ∇gj (x ∗ ), j ∈ A(x ∗ ), are linearly independent
vectors is a regularity condition that eliminates “degenerate" cases where there may not exist Lagrange
multipliers. These type of conditions are referred to as constraint qualifications. The condition in Eq.
(1.16) implies that if λ∗j > 0, then gj (x ∗ ) = 0, and if gj (x ∗ ) > 0, then λ∗j = 0. It is therefore referred
38 1. STATIC GAMES AND SOLUTION CONCEPTS
to as the complementary slackness condition and captures loosely the intuitive idea that a multiplier is
used only when the constraint is active and the problem is locally unconstrained with respect to the
inactive constraints.
For convex optimization problems (i.e., minimizing a convex function over a convex constraint
set, or maximizing a concave function over a convex constraint set), we can provide necessary and
sufficient conditions for the optimality of a feasible solution:

Theorem 1.46 Consider the optimization problem

maximize f (x)
subject to gj (x) ≥ 0, j = 1, . . . , r,

where the cost function f : Rn → R and the constraint functions gj : Rn → R are concave functions.
Assume also that there exists some x̄ such that gj (x̄) > 0 for all j = 1, . . . , r. Then a vector x ∗ ∈ Rn is
an optimal solution of the preceding problem if and only if gj (x ∗ ) ≥ 0 for all j = 1, . . . , r, and there
exists a nonnegative vector λ∗ ∈ Rr (Lagrange multiplier vector) such that


r
∗
∇f (x ) + λ∗j ∇gj (x ∗ ) = 0,
j =1

λ∗j gj (x ∗ ) = 0, ∀ j = 1, . . . , r.

Note that the condition that there exists some x̄ such that gj (x̄) > 0 for all j = 1, . . . , r is a
constraint qualification (referred to as the Slater’s constraint qualification) that can be used for convex
optimization problems.
 39

CHAPTER 2

Game Theory Dynamics

In the previous chapter, we have studied strategic form games which are used to model static games
where each player chooses his action once and for all, simultaneously. This chapter presents two
complementary ways for introducing dynamic analysis in competitive situations. First, we study
extensive form (dynamic) games, which are used to model dynamic decision making in multi-agent
environments. Second, we analyze dynamics induced by the repeated play of the same strategic form
game by boundedly rational agents, following simple myopic rules.
The last part of the chapter studies special classes of games with appealing dynamic properties,
supermodular and potential games, which have widespread applications for network game models.

2.1 EXTENSIVE FORM GAMES

In this section, we present extensive form games which model multi-agent sequential decision
making. Our focus will be on multi-stage games with observed actions where: (1) all previous actions
are observed, i.e., each player is perfectly informed of all previous events; (2) some players may move
simultaneously at some stage k. Extensive form games can be conveniently represented by game trees
as illustrated in the next examples.

Example 2.1

Entrant

In Out

Incumbent
(1,2)
AF

(2,1) (0,0)

Figure 2.1: Entry deterrence game.

We first consider an entry deterrence game, in which there are two players (see Figure 2.1).
Player 1, the entrant, can choose to enter the market or stay out. Player 2, the incumbent, after
observing the action of the entrant, chooses to accommodate him or fight with him. The payoffs for
40 2. GAME THEORY DYNAMICS
each of the action profiles (or histories) are given by the pair (x, y) at the leaves of the game tree: x
denotes the payoff of player 1 (the entrant) and y denotes the payoff of player 2 (the incumbent).

Example 2.2

Player 1

Invest Not Invest

Player 2

CournotGame I CournotGame II
c1 = 0 c1 = 2
c2 = 2 c2 = 2

Figure 2.2: Investment in duopoly.

Our next example considers a duopoly investment game, in which there are two players in the
market (see Figure 2.2). Player 1 can choose to invest or not invest. After player 1 chooses his action,
both players engage in a Cournot competition (cf. Example 1.3). If player 1 invests, then they will
engage in a Cournot game with c1 = 0 and c2 = 2. Otherwise, they will engage in a Cournot game
with c1 = c2 = 2. We can also assume that there is a fixed cost of f for player 1 to invest.

We next formally define the extensive form game model.

Definition 2.3 Extensive Form Game An extensive form game G consists of the following
components:

1. A set of players, I = {1, . . . , I }.

2. A set H of sequences, referred to as histories, defined as follows:

h0 = ∅ initial history
s 0 = (s10 , . . . , sI0 ) stage 0 action profile
h1 = s 0 history after stage 0
.. ..
. .
hk+1 = (s 0 , s 1 , . . . , s k ) history after stage k
Let Hk= be the set of all possible stage k histories. Then, H = ∪∞
{hk } k
k=0 H is the set of all
possible histories. If the game has a finite number (K + 1) of stages, then it is a finite horizon
game. We use H K+1 to denote the set of all possible terminal histories.
 2.1. EXTENSIVE FORM GAMES 41
3. A set of pure strategies for each player, defined as a contingency plan of how to play in each stage
k for every possible history hk . Let Si (H k ) = hk ∈H k Si (hk ) be the set of actions available
to player i at stage k. Define sik : H k → Si (H k ) such that si (hk ) ∈ Si (hk ). Then the pure
strategy of player i is the set of sequences si = {sik }∞ k=0 , i.e., a pure strategy of a player is a
collection of functions from all possible histories into available actions. A strategy profile s
is given by the tuple s = (s1 , . . . , sI ). Given a strategy profile s, we can find the sequence
of actions generated: the stage 0 actions are s 0 = s 0 (h0 ), the stage 1 actions are s 1 = s 1 (s 0 ),
the stage 2 actions are s 2 = s 2 (s 0 , s 1 ), and so on. This is called the path or outcome of strategy
profile s.

4. A set of preferences for each player. Since for finite horizon games, terminal histories H K+1
specify an entire sequence of play, we can represent the preferences of player i by a utility
function ui : H K+1 → R. In most applications, the utility functions are additively separable
over stages, i.e., each player’s utility is some weighted average of single stage payoffs. As the
strategy profile s determines the path s 0 , . . . , s k and hence hK+1 , we will use the notation
ui (s) as the payoff to player i under the strategy profile s.

The next example illustrates pure strategies of players in extensive form games.

Example 2.4

Player 1

C D

Player 2

EFGH

(2,1) (3,0) (0,2) (1,3)

Figure 2.3: Strategies in an extensive form game.

We consider the extensive form game illustrated in Figure 2.3. Player 1’s strategies are given
by functions s1 : H 0 = ∅ → S1 = {C, D}, which can be represented as two possible strategies; C,D.
Player 2’s strategies are given by functions s 2 : H 1 = {{C}, {D}} → S2 , which can be represented as
four possible strategies; EG, EH , F G and F H . For the strategy profile s = (C, EG), the outcome
is given by {C, E}. Similarly, for the strategy profile s = (D, EG), the outcome will be {D, G}.
42 2. GAME THEORY DYNAMICS
In the next example, we determine the pure strategies of players in an extensive form game
and use it to define the strategic (or normal) form representation of an extensive form game.

Example 2.5 Consider the following two-stage extensive form version of matching pennies.

Player 1

H T

Player 2

H T H T

(-1,1) (1,-1) (1,-1) (-1,1)

Figure 2.4: Two-stage extensive form version of matching pennies.

Since a strategy should be a complete contingency plan, player 2 has 4 different pure strategies:
heads following heads, heads following tails HH; heads following heads, tails following tails HT; tails
following heads, tails following tails TT; tails following heads, heads following tails TH. Identifying
the pure strategies of players allows us to go from the extensive form game to its strategic form
representation:

HH HT TT TH
H í 1, 1 í 1, 1 1, í 1 1, í 1
T 1, í 1 í 1, 1 í 1, 1 1, í 1

Strategic form representation of the extensive form version of matching pennies given in
Figure 2.4.

A Nash equilibrium of an extensive form game can be defined through its strategic form
representation, i.e., a pure strategy Nash equilibrium of an extensive form game is a strategy profile
s such that no player i can do better with a different strategy, which is the familiar condition that
ui (si , s−i ) ≥ ui (si , s−i ) for all si .1 The next example finds the Nash equilibria of an extensive form
game and argues that not all Nash equilibria represent reasonable outcomes in extensive form games.

Example 2.6 We consider the entry deterrence game described in Example 2.1. The equivalent
strategic form representation of this game is given by:
1 A mixed strategy Nash equilibrium is defined similarly.
 2.1. EXTENSIVE FORM GAMES 43
ACCOMMODATE FIGHT
IN (2, 1) (0, 0)
OUT (1, 2) (1, 2)

Strategic form representation of the entry deterrence game illustrated in Figure 2.1.

Hence, this game has two pure Nash equilibria: the strategies (In, Accommodate) and (Out,
Fight). However, the equilibrium (Out, Fight) does not seem reasonable: the entrant chooses “Out"
because he believes that the incumbent will fight if he enters the market. However, if this happens,
it is more profitable for the incumbent to choose “Accommodate". Hence, the equilibrium (Out,
Fight) is sustained by a noncredible threat of the incumbent.

This motivates a new equilibrium notion for extensive form games, Subgame Perfect (Nash)
Equilibrium, which requires each player’s strategy to be “optimal,” not only at the start of the game,
but also after every history. To define subgame perfection formally, we first define the notion of a
subgame. In perfect information extensive form games (i.e., each player is perfectly informed about
all the actions chosen at the previous stages when choosing an action), each node of the game tree
corresponds to a unique history, and is referred to as a subgame.2 We use the notation hk or G(hk )
to denote a subgame. A restriction of a strategy s to subgame G , s|G is the action profile implied
by s in the subgame G .

Definition 2.7 Subgame Perfect Equilibrium A strategy profile s ∗ is a Subgame Perfect Nash
equilibrium (SPE) in game G if for any subgame G of G, s ∗ |G is a Nash equilibrium of G .

Subgame perfection (i.e., the fact that a strategy profile restricted to any subgame should be
a Nash equilibrium of the subgame) will remove noncredible threats since these will not be Nash
equilibria in the appropriate subgames. In the entry deterrence game, following entry, the action
“Fight" is not a best response, and thus not a Nash equilibrium of the corresponding subgame.
Therefore, (Out, Fight) is not an SPE.
To find the SPE of an extensive form game, one can find all the Nash equilibria and eliminate
those that are not subgame perfect. However, for finite horizon games, the SPE can be found
more economically by using backward induction. Backward induction refers to starting from the last
subgames of a finite game, then finding the best response strategy profiles or the Nash equilibria in
the subgames, then assigning these strategies profiles and the associated payoffs to the subgames, and
moving successively towards the beginning of the game. It is straightforward to see that backward
induction provides the entire set of SPE.
2 In imperfect information extensive form games, subgames are defined through information sets, which model the information that
players have when they are choosing their actions. Information sets partition the nodes of a game tree and can be viewed as a
generalization of a history; see [46].
44 2. GAME THEORY DYNAMICS
For general extensive form games (i.e., finite or infinite horizon games), we will rely on a useful
characterization of the subgame perfect equilibria given by the “one stage deviation principle," which
is essentially the principle of optimality of dynamic programming. We first state it for finite horizon
games.

Theorem 2.8 One stage deviation principle For finite horizon multi-stage games with observed
actions , s∗ is a subgame perfect equilibrium if and only if for all i, t and ht , we have
ui (si∗ , s−i
∗ ∗
|ht ) ≥ ui (si , s−i |ht )
for all si satisfying
si (ht )  = si∗ (ht ),
∗
si|ht (ht+k ) = si|h t (h
t+k
), for all k > 0, and all ht+k ∈ G(ht ).

Informally, a strategy profile s is an SPE if and only if no player i can gain by deviating from
s in a single stage and conforming to s thereafter. We omit the proof of the one stage deviation
principle for finite horizon games, which uses a recursive argument to show that if a strategy satisfies
the one stage deviation principle, then that strategy cannot be improved upon by a finite number
of deviations. This leaves open the possibility that a player may gain by an infinite sequence of
deviations, which we exclude using the following condition.

Definition 2.9 Consider an extensive form game with an infinite horizon, denoted by G∞ . Let
h denote an ∞-horizon history, i.e., h = (s 0 , s 1 , s 2 ...), is an infinite sequence of actions. Let ht =
(s 0 , . . . , s t−1 ) be the restriction to first t periods. The game G∞ is continuous at infinity if for all
players i, the payoff function ui satisfies

sup |ui (h) − ui (h̃)| → 0 as t → ∞.

i,h,h̃ s.t. ht =h̃t

The continuity at infinity condition is satisfied when the overall payoffs are a discounted sum
of stage payoffs, i.e.,
∞
ui = δit git (s t ),
t=0
(where git (s t )
are the stage payoffs, and the positive scalar δi < 1 is a discount factor), and the stage
payoff functions are uniformly bounded, i.e., there exists some B such that maxt,s t |git (s t )| < B.

Theorem 2.10 Consider an infinite-horizon game, G∞ , that is continuous at infinity. Then, the one
stage deviation principle holds, i.e., the strategy profile s ∗ is an SPE if and only if for all i, ht , and t, we
have
ui (si∗ , s−i
∗ ∗
|ht ) ≤ ui (si , s−i |ht ),
 2.2. LEARNING DYNAMICS IN GAMES – FICTITIOUS PLAY 45
for all si that satisfies si (ht ) = si∗ (ht ) and si|h t (h
t+k ) = ∗ (ht+k )
si|ht for all ht+k ∈ G(ht ) and for all
k > 0.

2.2 LEARNING DYNAMICS IN GAMES – FICTITIOUS PLAY

Most economic theory relies on equilibrium analysis, based on Nash equilibrium or its refinements.
The traditional explanation for when and why equilibrium arises is that it results from analysis and
introspection by the players in a situation where the rules of the game, the rationality of the players,
and the payoff functions of players are all common knowledge.
In this section, we develop an alternative explanation why equilibrium arises using a dynamic
process in which less than fully rational players grope for optimality over time. For this, we rely
on adaptive models studied in learning in games literature which prescribe rules for how players
form beliefs and select strategies over time (see [45]). Our focus will be on fictitious play (and its
variants), introduced by Brown [28], which is a widely used model of learning. In this process, agents
behave assuming they are facing a stationary, but unknown, distribution of opponents’ strategies. We
examine whether fictitious play is a sensible model of learning and the asymptotic behavior of play
when all players use fictitious play learning rules.
The most compelling justification of fictitious play is as a “belief-based" learning rule, i.e.,
players form beliefs about opponent play (from the entire history of past play) and behave rationally
with respect to these beliefs.
The model for fictitious play is given below. Here we focus on the case where we have two
players only, see [45] for the analysis of fictitious play for multiple players.
• We have two players playing the strategic form game G at times t = 1, 2, . . ..

• We denote the stage payoff of player i by gi (si , s−i ).

• For t = 1, 2, . . . and i = 1, 2, we define the function ηit : S−i → R, where ηit (s−i ) is the
number of times player i has observed the action s−i before time t. We use ηi0 (s−i ) to represent
a starting point (or fictitious past).

Example 2.11 Assume that S2 = {U, D}. If η10 (U ) = 3 and η10 (D) = 5, and player 2 plays U, U, D
in the first three periods, then η13 (U ) = 5 and η13 (D) = 6.

Each player assumes that his opponent is using a stationary mixed strategy. Players choose
actions in each period (or stage) to maximize that period’s expected payoff given their prediction of
the distribution of opponent’s actions, which they form according to:

ηit (s−i )
μti (s−i ) = t ,
s̄−i ∈S−i ηi (s̄−i )
46 2. GAME THEORY DYNAMICS
i.e., player i forecasts player −i’s strategy at time t to be the empirical frequency distribution of past
play.
Given player i’s belief or forecast about his opponents play, he chooses his action at time t to
maximize his payoff, i.e.,
sit ∈ arg max gi (si , μti ).
si ∈Si

This choice is myopic, i.e., players are trying to maximize current payoff without considering their
future payoffs. Note that the myopic assumption is consistent with the assumption that players are
using stationary mixed strategies.

Example 2.12 Consider the fictitious play of the following game:

L R
U 3, 3 0, 0
D 4, 0 1, 1

Note that this game is dominance solvable (D is a strictly dominant strategy for the row
player), and the unique Nash equilibrium is (D, R). Assume that η10 = (3, 0) and η20 = (1, 2.5).
Period 1: Then, μ01 = (1, 0) and μ02 = (1/3.5, 2.5/3.5), so play follows s10 = D and s20 = L.
Period 2: We have η11 = (4, 0) and η21 = (1, 3.5), so play follows s11 = D and s21 = R.
Period 3: We have η11 = (4, 1) and η21 = (1, 4.5), so play follows s12 = D and s22 = R.
Since D is a dominant strategy for the row player, he always plays D, and μt2 converges to
(0, 1) with probability 1. Therefore, player 2 will end up playing R.

The striking feature of the fictitious play is that players do not have to know anything about
their opponent’s payoff. They only form beliefs about how their opponents will play.

2.2.1 CONVERGENCE OF FICTITIOUS PLAY

Let {s t }
be a sequence of strategy profiles generated by fictitious play. In this section, we study the
asymptotic behavior of the sequence {s t }, i.e., the convergence properties of the sequence {s t } as t
goes to infinity.
We first define the notion of convergence to pure strategies.

Definition 2.13 The sequence {s t } converges to s if there exists T such that s t = s for all t ≥ T .

The next proposition formalizes the property that if the fictitious play sequence converges,
then it must converge to a Nash equilibrium of the game.

Proposition 2.14 Let {s t } be a sequence of strategy profiles generated by fictitious play.

(a) If {s t } converges to s̄, then s̄ is a pure strategy Nash equilibrium.

 2.2. LEARNING DYNAMICS IN GAMES – FICTITIOUS PLAY 47
(b) Suppose that for some t, st = s∗, where s∗ is a strict Nash equilibrium of G. Then sτ = s∗ for all
τ > t.

The proof of part (a) is straightforward. We provide a proof for part (b):

Proof of part (b): Let s t = s ∗ . We will show that s t+1 = s ∗ . Note that
∗
μt+1
i = (1 − α)μti + αs−i
t
= (1 − α)μti + αs−i ,
t to denote the degenerate probability distribution and
where, abusing the notation, we used s−i

1
α= .
s−i ηit (s−i ) + 1

Therefore, by the linearity of the expectation, we have for all si ∈ Si ,

∗
i ) = (1 − α)gi (si , μi ) + αgi (si , s−i ).
gi (si , μt+1 t

Since si∗ maximizes both terms (in view of the fact that s ∗ is a strict Nash equilibrium), it follows
that si∗ will be played at t + 1. 2

Note that the preceding notion of convergence only applies to pure strategies. We next provide
an alternative notion of convergence, i.e., convergence of empirical distributions or beliefs.

Definition 2.15 The sequence {s t } converges to σ ∈  in the time-average sense if for all i and for
all si ∈ Si , we have

[number of times sit = si for t ≤ T ]

lim = σ (si ),
T →∞ T +1

i.e., μT−i (si ) converges to σi (si ) as T → ∞.

The next example illustrates convergence of the fictitious play sequence in the time-average
sense.

Example 2.16 Matching Pennies Consider the fictitious play of the following matching pennies
game:

H T
H 1, í 1 í 1, 1
T í 1, 1 1, í 1
48 2. GAME THEORY DYNAMICS
Consider the following sequence of play:

Time η1t η2t Play

0 (0, 0) (0, 2) (H, H )
1 (1, 0) (1, 2) (H, H )
2 (2, 0) (2, 2) (H, T )
3 (2, 1) (3, 2) (H, T )
4 (2, 2) (4, 2) (T , T )
5 (2, 3) (4, 3) (T , T )
6 ... ... (T , H )

! Play continues as (T,H),

" (H,H), (H,H) - a deterministic cycle. The time average converges
to (1/2, 1/2), (1/2, 1/2) , which is the unique Nash equilibrium.

The next proposition extends the basic convergence property of fictitious play, stated in Propo-
sition 2.14, to mixed strategies.

Proposition 2.17 Suppose a fictitious play sequence {s t } converges to σ in the time-average sense. Then
σ is a Nash equilibrium of G.

Proof. Suppose s t converges to σ in the time-average sense. Assume that σ is not a Nash equilibrium.
Then there exist some i, si , si ∈ Si with σi (si ) > 0 such that

gi (si , σ−i ) > gi (si , σ−i ).

Choose  > 0 such that

1# $
< gi (si , σ−i ) − gi (si , σ−i ) ,
2
and T sufficiently large that for all t ≥ T , we have
% %
% T % 
%μi (s−i ) − σ−i (s−i )% < ,
maxs∈S gi (s)
which is possible since μti → σ−i by assumption. Then, for any t ≥ T , we have

gi (si , μti ) = gi (si , s−i )μti (s−i )
s−i

≤ gi (si , s−i )σ−i (s−i ) + 
s−i

< gi (si , s−i )σ−i (s−i ) − 
s−i

≤ gi (si , s−i )μti (s−i ) = gi (si , μti ).
s−i
 2.2. LEARNING DYNAMICS IN GAMES – FICTITIOUS PLAY 49
This shows that after T , si is never played, implying that as T → ∞, μt−i (si ) → 0. But this con-
tradicts the fact that σi (si ) > 0, completing the proof. 2
It is important to realize that convergence in the time-average sense is not necessarily a natural
convergence notion, as illustrated in the following example.

Example 2.18 Mis-coordination Consider the fictitious play of the following game:
A B
A 1, 1 0, 0
B 0, 0 1, 1
! "
Note that this game had a unique mixed Nash equilibrium (1/2, 1/2), (1/2, 1/2) . Consider
the following sequence of play:

Time η1t η2t Play

0 (1/2, 0) (0, 1/2) (A, B)
1 (1/2, 1) (1, 1/2) (B, A)
2 (3/2, 1) (1, 3/2) (A, B)
3 ... ... (B, A)
4 ... ... (A, B)

! Play continues as (A,B),

" (B,A), . . . - again a deterministic cycle. The time average converges
to (1/2, 1/2), (1/2, 1/2) , which is a mixed strategy equilibrium of the game. But players never
successfully coordinate!

The following proposition provides some classes of games for which fictitious play converges
in the time-average sense (see Fudenberg and Levine [45]; see also [87], [72], [75] for the proofs).

Proposition 2.19 Fictitious play converges in the time-average sense for the game G under any of the
following conditions:
(a) G is a two player zero-sum game.
(b) G is a two player nonzero-sum game where each player has at most two strategies.
(c) G is solvable by iterated strict dominance.
(d) G is an identical interest game, i.e., all players have the same payoff function.
(e) G is an (exact) potential game (see Def. 2.36).

In Section 2.2.3, we consider a continuous time variant of fictitious play and provide the proof
of convergence for two player zero-sum games and identical interest games.
50 2. GAME THEORY DYNAMICS
2.2.2 NON-CONVERGENCE OF FICTITIOUS PLAY
The next proposition shows that there exists games in which fictitious play does not converge in the
time-average sense.

Proposition 2.20 Shapley [94] Fictitious play does not converge in Shapley’s modified version of
Rock-Scissors-Paper game (described below).

Example 2.21 Shapley’s modified version of Rock-Scissors-Paper game has payoffs:

R S P
R 0, 0 1, 0 0, 1
S 0, 1 0, 0 1, 0
P 1, 0 0, 1 0, 0

Suppose that η10 = (1, 0, 0) and that η20 = (0, 1, 0). Then in period 0, play is (P,R). In period 1,
player 1 expects R, and 2 expects S, so play is (P,R). Play then continues to follow (P,R) until player
2 switches to S (suppose this lasts for k periods). Play then follows (P,S), until player 1 switches to R
(for βk periods, where β is a scalar such that β > 1). Play then follows (R,S), until player 2 switches
to P (for β 2 k periods). It can be shown that play of the game cycles among 6 (off-diagonal) profiles
with periods of ever-increasing length, thus non-convergence.

Shapley’s proof of the non-convergence of fictitious play in the above game explicitly computes
time spent in each stage of the sequence. Here, we provide an alternative insightful proof due to
Monderer et al. [74]. For this we will make use of the following lemma, which relates realized payoffs
to expected payoffs under the empirical distributions. Define the time-average payoffs through time
t as:
1 
t
Ui =
t
gi (siτ , s−i
τ
).
t +1
τ =0
Define the expected payoffs at time t as:

Ũit = gi (sit , μti ) = max gi (si , μti ).

si ∈Si

Lemma 2.22 Given any  > 0 and any i ∈ I , there exists some T such that for all t ≥ T , we have

Ũit ≥ Uit − .
 2.2. LEARNING DYNAMICS IN GAMES – FICTITIOUS PLAY 51
Proof. Note that

Ũit = gi (sit , μti ) ≥ gi (sit−1 , μti )

1 t
= gi (sit−1 , s−i
t−1
)+ gi (sit−1 , μt−1
i )
t +1 t +1
1 t
= gi (sit−1 , s−i
t−1
)+ Ũ t−1 .
t +1 t +1 i

Expanding Ũit−1 and iterating, we obtain

1 
t
1
Ũit ≥ gi (siτ , s−i
τ
)− gi (sit , s−i
t
).
t +1 t +1
τ =0

Choosing T such that

1
> max gi (s),
T + 1 s∈S
completes the proof. 2

We can now use the preceding lemma to prove Proposition 2.20 (Shapley’s result).

Proof of Proposition 2.20. In the Rock-Scissors-Paper game, there is a unique Nash equilibrium with
expected payoffs of 1/3 for both players. Therefore, if the fictitious play converged, then Ũit → 1/3
for i = 1, 2, implying that Ũ1t + Ũ2t → 2/3. But under fictitious play, realized payoffs always sum
to 1, i.e.,
U1t + U2t = 1, for all t,
thus contradicting the lemma above, and showing that fictitious play does not converge. 2

2.2.3 CONVERGENCE PROOFS

We now consider a continuous time version of fictitious play and provide a proof of convergence for
zero-sum games and identical interest games.

2.2.3.1 Continuous Time Fictitious Play

Let us introduce some notation to facilitate the subsequent analysis. We denote the empirical dis-
tribution of player i’s play up to (but not including) time t when time intervals are of length t
by
(t−t)/t
τ =0 I {siτ = si }
pit (si ) = .
t/t
We use p t ∈  to denote the product distribution formed by the pit . We can now think of making
time intervals t smaller, which will lead us to a version a fictitious play in continuous time.
52 2. GAME THEORY DYNAMICS
In continuous time fictitious play (CTFP), the empirical distributions of the players are updated
in the direction of a best response to their opponents’ past action:
dpit
∈ BRi (p−i
t
) − pit ,
dt
where
t
BRi (p−i ) = arg max gi (σi , p−i
t
).
σi ∈i

Another variant of the CTFP is the perturbed CTFP defined by

dpit
= Ci (p−i
t
) − pit ,
dt
where # $
t
Ci (p−i ) = arg max gi (σi , p−i
t
) − Vi (σi ) , (2.1)
σi ∈i

and Vi : i → R is a strictly convex function and satisfies a “boundary condition" (see Fudenberg
and Levine [45] for examples of perturbation functions). Note that because the perturbed best-
response Ci is uniquely defined, the perturbed CTFP is described by a differential equation rather
than a differential inclusion.

2.2.3.2 Convergence of (perturbed) CTFP for Zero-sum Games

In this section, we provide a proof of convergence of perturbed CTFP for two player zero-sum
games.
We consider a two player zero-sum game with payoff matrix M, where the payoffs are per-
turbed by the functions Vi (defined above), i.e., the payoffs are given by

1 (σ1 , σ2 ) = σ1 Mσ2 − V1 (σ1 ), (2.2)

2 (σ1 , σ2 ) = −σ1 Mσ2 − V2 (σ2 ). (2.3)

Let {p t } be generated by the perturbed CTFP,
dpit
= Ci (p−i
t
) − pit ,
dt
[cf. Eq. (2.1)]. We use a Lyapunov function approach to prove convergence. In particular, we consider
the function
W (t) = U1 (pt ) + U2 (pt ),
where the functions Ui :  → R are defined as

Ui (σi , σ−i ) = max i (σi , σ−i ) − i (σi , σ−i ),

σi ∈i
 2.2. LEARNING DYNAMICS IN GAMES – FICTITIOUS PLAY 53
(i.e., the function Ui gives the maximum possible payoff improvement player i can achieve by a
unilateral deviation in his own mixed strategy). Note that Ui (σ ) ≥ 0 for all σ ∈ , and Ui (σ ) = 0
for all i implies that σ is a mixed Nash equilibrium.
For the zero sum game with payoffs (2.2)–(2.3), the function W (t) takes the form

W (t) = max 1 (σ1 , p2t ) + max i (p1t , σ2 ) + V1 (p1t ) + V2 (p2t ).

σ1 ∈1 σ2 ∈2

We will show that dWdt(t) ≤ 0 with equality if and only if pit = Ci (p−i
t ), showing that for all
0
initial conditions p , we have
! "
lim pit − Ci (p−i
t
) =0 i = 1, 2.
t→∞

We need the following lemma.

Lemma 2.23 (Envelope Theorem) Let F : Rn × Rm → R be a continuously differentiable function.

Let U ⊂ Rm be an open convex subset, and u∗ (x) be a continuously differentiable function such that

F (x, u∗ (x)) = min F (x, u).

u∈U

Let H (x) = minu∈U F (x, u). Then,

∇x H (x) = ∇x F (x, u∗ (x)).

Proof. The gradient of H (x) is given by

∇x H (x) = ∇x F (x, u∗ (x)) + ∇u F (x, u∗ (x))∇x u∗ (x)

= ∇x F (x, u∗ (x)),

where we use the fact that ∇u F (x, u∗ (x)) = 0 since u∗ (x) minimizes F (x, u). 2
Using the preceding lemma, we have
 
d dp2t
max 1 (σ1 , p2t ) = ∇σ2 1 (C1 (p2t ), p2t )
dt σ1 ∈1 dt
dp t
= C1 (p2t ) M 2
dt
= C1 (p2t ) M (C2 (p1t ) − p2t ).

Similarly,  
d
max 2 (p1t , σ2 ) = −(C1 (p2t ) − p1t ) MC2 (p1t ).
dt σ2 ∈2
54 2. GAME THEORY DYNAMICS
Combining the preceding two relations, we obtain

dW (t) dp1t dp2t

= −C1 (p2t ) Mp2t + (p1t ) MC2 (p1t ) + ∇V1 (p1t ) + ∇V2 (p2t ) . (2.4)
dt dt dt
t ) is a perturbed best response, we have
Since Ci (p−i

C1 (p2t ) Mp2t − V1 (C1 (p2t )) ≥ (p1t ) Mp2t − V1 (p1t ),

−(p1t ) MC2 (p1t ) − V2 (C2 (p1t )) ≥ −(p1t ) Mp2t − V2 (p2t ),

t ) = p t , i = 1, 2 (the latter claim follows by the uniqueness of the
with equality if and only if Ci (p−i i
perturbed best response). Combining these relations, we have

−C1 (p2t ) Mp2t + (p1t ) MC2 (p1t ) ≤ [Vi (pit ) − Vi (Ci (p−i
t
))]

i
≤ ∇Vi (pit ) (Ci (pit ) − pit )
i
 dpit
= − ∇Vi (pit ) ,
dt
i

where the second inequality follows by the convexity of Vi . The preceding relation and Eq. (2.4)
dt ≤ 0 for all t, with equality if and only if Ci (p−i ) = pi for both players, completing
imply that dW t t

the proof.

2.2.3.3 Convergence of CTFP for Identical Interest Games

We finally provide a proof of convergence of CTFP for identical interest games. Consider an I -player
game with identical interests, i.e., a game where all players share the same payoff function . Recall
the continuous time fictitious play (CTFP) dynamics:

dpit
∈ BRi (p−i
t
) − pit .
dt
Let {pit } denote the sequence generated by the CTFP dynamics and let σit = pit + dpit /dt. Note
that σit ∈ BRi (p−i
t ).

Theorem 2.24 For all players i and regardless of the initial condition p 0 , we have
 
lim t
max (σi , p−i ) − (pit , p−i
t
) = 0;
t→∞ σi ∈i

namely, pit is asymptotically a best response to p−i

t .
 2.2. LEARNING DYNAMICS IN GAMES – FICTITIOUS PLAY 55
Proof. We again consider the function W (t) ≡ i Ui (p t ), where

Ui (σi , σ−i ) = max (σi , σ−i ) − (σi , σ−i ).

σi ∈

Observe that
⎡ ⎤
d d ⎣  
((pt )) = ··· p1t (s1 ) · · · pnt (sn )(s)⎦
dt dt
si ∈Si sn ∈Sn
⎛ ⎞
  dpt 
= ··· i
(si ) ⎝ pjt (sj )⎠ (s)
dt
i si ∈Si sn ∈Sn j  =i
 * dpt +
=  i t
, p−i .
dt
i

The preceding explicit derivation essentially follows from the fact that  is linear in its
arguments, because these are mixed strategies of players. Therefore, the time derivative can be
directly applied to the arguments. Now, observe that
* t +
dpi t
 , p−i = (σit − pit , p−i
t
) = (σit , p−i
t
) − (p t ) = Ui (p t ),
dt
where the second equality again follows by the linearity of  in mixed strategies. The last equality
uses the fact that σit ∈ BRi (p−i
t ). Combining this relation with the previous one, we have

d 
((pt )) = Ui (pt ) = W (t).
dt
i

Since W (t) is nonnegative everywhere, we conclude (pt ) is nondecreasing as t increases;

thus ∗ = limt→∞ (pt ) exists (since  is bounded above, ∗ < ∞). Moreover, we have
 
∗
 − (p ) ≥ (p
t t+
) − (p ) =
t
W (t + τ )dτ ≥ 0,
0

where the first inequality uses the fact that  is nondecreasing; the middle inequality follows from the
fundamental theorem of calculus, and the last inequality simply uses the fact that W (t) is everywhere
nonnegative. Since the left-hand side converges to zero, we conclude that W (t) → 0 as t → ∞.
This establishes that for each i and for any initial condition p 0 ,
 
lim t
max (σi , p−i ) − (pit , p−i
t
) = 0.
t→∞ σi ∈i

2
56 2. GAME THEORY DYNAMICS
2.3 GAMES WITH SPECIAL STRUCTURE
This section studies two special classes of games with appealing equilibrium and dynamic properties:
supermodular games and potential games.

2.3.1 SUPERMODULAR GAMES

Supermodular games are games that are characterized by “strategic complementarities". Informally,
this means that the marginal utility of increasing a player’s strategy raises with increases in the other
players’ strategies. The implication is that the best response function of a player is a nondecreasing
function of other players’ strategies.
The machinery needed to study supermodular games is lattice theory and monotonicity re-
sults in lattice programming (see Appendix 2.A for more on lattices). The methods used are non-
topological, and they exploit order properties. The existence of a pure Nash equilibrium in super-
modular games can be established through a lattice-theoretic fixed point theorem, Tarski’s fixed point
theorem, which shows the existence of a fixed point for increasing functions (see [46] and Appendix
2.A for a statement of Tarski’s fixed point theorem). Since for supermodular games the best-response
correspondences of players have a monotone increasing selection, this implies existence of a pure
Nash equilibrium. Below, we present an analysis that does not rely on Tarski’s fixed point theo-
rem, but instead uses ideas from the monotonicity results in lattice programming. This analysis also
provides more insight into the structure of the equilibrium set in supermodular games.
Supermodular games are interesting for a number of reasons:

• They arise in many network models.

• We can establish the existence of a pure strategy equilibrium without requiring the quasi-
concavity of the payoff functions.

• Many solution concepts yield the same predictions.

• The equilibrium set has a smallest and a largest element and there exists a simple algorithm
to compute these.

• They have nice sensitivity (or comparative statics) properties and behave well under a variety
of learning rules.

Much of the theory is due to Topkis [101], [100], Vives [103], [104], Milgrom and Roberts [69],
and Milgrom and Shannon [70].

2.3.1.1 Monotonicity of Optimal Solutions

We first study the monotonicity properties of optimal solutions of parametric optimization problems.
In particular, we consider
x(t) = arg max f (x, t),
x∈X
 2.3. GAMES WITH SPECIAL STRUCTURE 57
where f : X × T → R, X ⊂ R, and T is some partially ordered set (we will mostly focus on T ⊆ RK
with the usual vector order, i.e., for some x, y ∈ T , x ≥ y if and only if xi ≥ yi for all i = 1, . . . , k;
see Appendix 2.A for more general partially ordered sets). We are interested in conditions under
which we can establish that x(t) is a nondecreasing function of t. We next define the key property
of increasing differences.3

Definition 2.25 Let X ⊆ R and T be some partially ordered set. A function f : X × T → R has
increasing differences in (x, t) if for all x ≥ x and t ≥ t, we have

f (x , t ) − f (x, t ) ≥ f (x , t) − f (x, t).

The preceding definition implies that if f has increasing differences in (x, t), then the incre-
mental gain to choosing a higher x (i.e., x ≥ x rather than x) is greater when t is higher. That is
f (x , t) − f (x, t) is nondecreasing in t. One may verify that the property of increasing differences
is symmetric: an equivalent statement is that if t ≥ t, then f (x, t ) − f (x, t) is nondecreasing in x.
Note that the function f need not be nicely behaved for the above definition to hold, nor
do X and T need to be intervals. For instance, we could have X = {0, 1} and just a few parameter
values, e.g., T = {0, 1, 2}. If, however, f is nicely behaved, we can rewrite increasing differences in
terms of partial derivatives.

Lemma 2.26 Let X ⊆ R and T ⊆ Rk for some k, a partially ordered set with the usual vector order.
Let f : X × T → R be a twice continuously differentiable function. Then, the following statements are
equivalent:

(a) The function f has increasing differences in (x, t).

(b) For all t ≥ t and all x ∈ X, we have

∂f (x, t ) ∂f (x, t)
≥ .
∂x ∂x

(c) For all x ∈ X, t ∈ T , and all i = 1, . . . , k, we have

∂ 2 f (x, t)
≥ 0.
∂x∂ti
3The following analysis can be extended to the case where X is a lattice (see Topkis [101], and Milgrom and Roberts [69] for these
extensions).
58 2. GAME THEORY DYNAMICS
Before presenting our key result about monotonicity of optimal solutions, we introduce some
examples, which satisfy increasing differences property.

Example 2.27 Network Effects

A set I of users can use one of two products X and Y (e.g., Blu-ray and HD DVD). We use
Bi (J, k) to denote the payoff to i when a subset J of users use k and i ∈ J . There exists a positive
externality if
Bi (J, k) ≤ Bi (J , k), when J ⊂ J ,
i.e., player i is better off if more users use the same technology as him. This leads to a strategic form
game with actions Si = {X, Y }. Let us define the order Y  X, which induces a lattice structure.
Given s ∈ S, let X(s) = {i ∈ I | si = X}, Y (s) = {i ∈ I | si = Y }. We define the payoff functions
as
Bi (X(s), X) if si = X,
ui (si , s−i ) =
Bi (Y (s), Y ) if si = Y
It can be verified that the payoff functions satisfy increasing differences.

Example 2.28 Wireless Power Control We consider the problem of power control in cellular
CDMA wireless networks. In a CDMA system, all users access the channel using orthogonal codes at
the same time, utilizing the entire available frequency spectrum (unlike time division multiple access
(TDMA) or frequency division multiple access (FDMA) schemes). Direct sequence CDMA systems
require stringent power control in the uplink channel to reduce the residual interference between
users due to use of semi-orthogonal spreading codes. In the current IS-95 CDMA standard, the
power control is done centrally by the base-station to mitigate the “near-far" problem by equalizing
the received power of all mobile units regardless of their distance from the receiver.
With emerging multimedia applications, power control will be increasingly aimed at achieving
different quality of service (QoS) requirements of applications. In the presence of heterogeneity in
QoS requests, the resource (power) allocation problem becomes nonstandard and traditional network
optimization techniques cannot be applied directly to come up with efficient distributed methods
that would yield optimal allocations. To address this problem, recent literature has used game-
theoretic models for resource control among heterogeneous users. This literature uses the “utility-
maximization” framework of market economics, to provide different access privileges to users with
different QoS requirements in a distributed manner (see [54], [57]). In this framework, each user (or
equivalently application) is represented by a utility function that is a measure of his preferences over
transmission rates. The consideration of game-theoretic methods for resource allocation in wireless
networks is relatively more recent; see Chapter 4 for an overview of wireless network games.
It has been well-recognized that in the presence of interference, the strategic interactions
between the users is in some scenarios that of strategic complementarities (see [93], [10], and [50]).
Below, we present a simple game model for power allocation and show that the payoff functions
 2.3. GAMES WITH SPECIAL STRUCTURE 59
satisfy increasing differences property. A different model for power allocation in wireless networks
will be considered in the sequel in Chapter 4.
• Let L = {1, 2, ..., n} denote the set of wireless nodes and

P = [Pimin , Pimax ] ⊂ Rn
i∈L

denote the set of power vectors p = [p1 , . . . , pn ] such that each node i ∈ L transmits at a
power level pi .
• Received signal-to-interference ratio (SINR) for each node i, γi : P → R is given by
pi hi
γi (p) = ,
σ2 + j  =i,1≤j ≤n pj hj

where σ 2 is the noise variance (assuming an additive white Gaussian noise channel), and hi is
the channel gain from mobile i to the base station.
• Each user is endowed with a function fi (γi ) as a function of its SINR γi .
• The payoff function of each user represents a tradeoff between the payoff obtained by the
received SINR and the power expenditure, and it takes the form

ui (pi , p−i ) = fi (γi ) − cpi .

Assume that each function fi satisfies the following assumption regarding its coefficient of
relative risk aversion:
−γi fi (γi )
≥ 1, ∀ γi ≥ 0.
fi (γi )
We show that for all i = 1 . . . , n, the function ui (pi , p−i ) has increasing differences in (pi , p−i ).
For this, we check the partial derivatives (see Lemma 2.26):
∂ui γi
(pi , p−i ) = f (γi ) − c,
∂pi pi
and for all j  = i,
∂ 2 ui γ 2 hj # $
(pi , p−i ) = − i2 γi f (γi ) + f (γi ) .
∂pi ∂pj pi hi
In view of our assumption on the coefficient of relative risk aversion of the utility functions fi , i.e.,
−γi fi (γi ) 2
fi (γi )
≥ 1, it follows that ∂p∂ i ∂p
ui
j
(pi , p−i ) ≥ 0 for all j  = i, showing the desired claim.
Consider the following class of utility functions:

γ 1−α
f (γ ) = , α > 1.
1−α
60 2. GAME THEORY DYNAMICS
It can be seen that
−γi fi (γi )
= α > 1.
fi (γi )

Example 2.29 Oligopoly Models

We first consider Bertrand competition: Suppose firms 1, . . . , I simultaneously choose prices,
and the demand function is given by

Di (pi , p−i ) = ai − bi pi + dij pj ,
j  =i

where bi and dij are nonnegative constants. Let the strategy space be Si = [0, ∞) and the payoff
function be ui (pi , p−i ) = (pi − ci )Di (pi , p−i ) (where as usual ci is the cost of producing one unit
2
of good). Then, ∂p∂ i ∂p
ui
j
(pi , p−i ) = dij ≥ 0, showing that ui (pi , p−i ) has increasing differences in
(pi , p−i ).
We next consider Cournot competition in a duopoly: two firms choose the quantity they
produce qi ∈ [0, ∞). We denote the inverse demand function by P (qi , qj ), and assume that it is a
function of Q = qi + qj , and it is twice continuously differentiable in Q . We further assume that

P (Q) + qP (Q) ≤ 0.

Let the payoff function of each firm be ui (qi , qj ) = qi P (qi + qj ) − cqi .Then, it can be seen that the
payoff functions of the transformed game defined by s1 = q1 , s2 = −q2 has increasing differences
in (s1 , s2 ).

2.3.1.2 Main Result

The next theorem presents the key result of our development, which is due to Topkis [101].

Theorem 2.30 Let X ⊂ R be a compact set and T be some partially ordered set. Assume that the function
f : X × T → R is upper semicontinuous in x for all t ∈ T and has increasing differences in (x, t). Define
x(t) = arg maxx∈X f (x, t). Then, we have:

1. For all t ∈ T , x(t) is nonempty and has a greatest and least element, denoted by x̄(t) and x(t),
respectively.

2. For all t ≥ t, we have x̄(t ) ≥ x̄(t) and x(t ) ≥ x(t).

 2.3. GAMES WITH SPECIAL STRUCTURE 61
Proof. (1) By the assumptions that for all t ∈ T , the function f (·, t) is upper semicontinuous and
X is compact, it follows by the Weierstrass’ Theorem that x(t) is nonempty. For all t ∈ T , x(t) ⊂ X,
therefore is bounded. Since X ⊂ R, to establish that x(t) has a greatest and lowest element, it suffices
to show that x(t) is closed.
Let {x k } be a sequence in x(t). Since X is compact, x k has a limit point x̄. By restricting to
a subsequence if necessary, we may assume without loss of generality that x k converges to x̄. Since
x k ∈ x(t) for all k, we have
f (x k , t) ≥ f (x, t), ∀ x ∈ X.
Taking the limit as k → ∞ in the preceding relation and using the upper semicontinuity of f (·, t),
we obtain
f (x̄, t) ≥ lim sup f (x k , t) ≥ f (x, t), ∀ x ∈ X,
k→∞
thus showing that x̄ belongs to x(t), and proving the desired closedness claim.
(2) Let t ≥ t. Let x ∈ x(t) and x = x̄(t ). By the fact that x maximizes f (x, t), we have

f (x, t) − f (min(x, x ), t) ≥ 0.

This implies (by verifying the two cases: x ≥ x and x ≥ x) that

f (max(x, x ), t) − f (x , t) ≥ 0.

By increasing differences of f , this yields

f (max(x, x ), t ) − f (x , t ) ≥ 0.

Thus, max(x, x ) maximizes f (·, t ), i.e, max(x, x ) belongs to x(t ). Since x is the greatest element
of the set x(t ), we conclude that max(x, x ) ≤ x , thus x ≤ x . Since x is an arbitrary element of
x(t), this implies x̄(t) ≤ x̄(t ). A similar argument applies to the lowest maximizers.

2
The above theorem suggests that if f has increasing differences, then the set of maximizers
x(t) is nondecreasing in t, in the sense that both the greatest maximizers x̄(t) and the lowest
maximizers x(t) are nondecreasing in t.

2.3.1.3 Supermodular Games

We now introduce the class of supermodular games.

Definition 2.31 Supermodular game The strategic form game I , (Si ), (ui ) is a supermodular
game if for all i:

1. Si is a compact subset of R (or more generally Si is a sublattice of Rm ),

62 2. GAME THEORY DYNAMICS
2. ui is upper semicontinuous in si , continuous in s−i ,

3. ui has increasing differences in (si , s−i ).4

Applying Topkis’ theorem in this context immediately implies that each player’s best response
correspondence is increasing in the actions of other players.

Corollary 2.32 Assume I , (Si ), (ui ) is a supermodular game. Let

Bi (s−i ) = arg max ui (si , s−i ).

si ∈Si

Then:

1. Bi (s−i ) has a greatest and least element, denoted by B̄i (s−i ) and Bi (s−i ).

2. If s−i ≥ s−i , then B̄i (s−i ) ≥ B̄i (s−i ) and Bi (s−i ) ≥ Bi (s−i ).

We now use the properties of the supermodular games to show that various solution concepts
that we considered in Chapter 1 for strategic form games yield the same predictions in supermodular
games.

Theorem 2.33 Let I , (Si ), (ui ) be a supermodular game. Then the set of strategies that survive
iterated strict dominance in pure strategies (i.e., iterated elimination of strictly dominated pure strategies,
see Section 1.2.2) has greatest and least elements s̄ and s, coinciding with the greatest and the least pure
strategy Nash Equilibria.

The preceding theorem immediately yields the following corollary.

Corollary 2.34 Supermodular games have the following properties:

1. Pure strategy NE exist.

2. The largest and smallest strategies that are compatible with iterated strict dominance (ISD), correlated
equilibrium, and Nash equilibrium are the same.

3. If a supermodular game has a unique NE, it is dominance solvable.5

We now return to the proof of Theorem 2.33.

4 More generally, u is supermodular in (s , s ); see Fudenberg and Tirole [46] for the more general definition of supermodularity,
i i −i
which is an extension of the property of increasing differences to games with multi-dimensional strategy spaces.
5 Consequently, several learning and adjustment rules, such as best-response dynamics, converge to it.
 2.3. GAMES WITH SPECIAL STRUCTURE 63
Proof. We iterate the best response mapping. Let = S, and let
S0 s0= (s10 , . . . , sI0 ) be the largest
element of S. Let si = B̄i (s−i ) and Si = {si ∈ Si | si ≤ si1 }. We
1 0 1 0 show that any si > si1 , i.e, any
si ∈
/ Si1 , is strictly dominated by si1 . For all s−i ∈ S−i , we have
ui (si , s−i ) − ui (si1 , s−i ) ≤ ui (si , s−i
0
) − ui (si1 , s−i
0
)
< 0,
where the first inequality follows by the increasing differences of ui (si , s−i ) in (si , s−i ), and the
0 . Note that s 1 ≤ s 0 . Iterating
strict inequality follows by the fact that si is not a best response to s−i i i
this argument, We define
sik = B̄i (s−i
k−1
), Sik = {si ∈ Sik−1 | si ≤ sik }.
Assume s k ≤ s k−1 . Then, by Corollary 2.32, we have
sik+1 = B̄i (s−i
k
) ≤ B̄i (s−i
k−1
) = sik .
This shows that the sequence {sik } is a decreasing sequence, which is bounded from below, and hence
it has a limit, which we denote by s¯i . Only the strategies si ≤ s̄i are undominated.
Similarly, we can start with s 0 = (s10 , . . . , sI0 ) the smallest element in S and identify s.
To complete the proof, we show that s̄ and s are NE. By construction, for all i and si ∈ Si , we
have
ui (sik+1 , s−i
k
) ≥ ui (si , s−i
k
).
Taking the limit as k → ∞ in the preceding relation and using the upper semicontinuity of ui in si
and continuity of ui in s−i , we obtain
ui (s̄i , s̄−i ) ≥ ui (si , s̄−i ),
showing the desired claim. 2

2.3.2 POTENTIAL GAMES

In this section, we present the family of potential games, introduced by Monderer and Shapley [76],
and study their properties. A strategic form game is a potential game if there exists a function
 : S → R such that  (si , s−i ) gives information about the utility functions ui (si , s−i ) for each
i ∈ I . The function  is referred to as the potential function. The potential function has a natural
analogy to “energy” in physical systems. It will be useful both for locating pure strategy Nash equilibria
and also for the analysis of “myopic” dynamics.
We next formally define different classes of potential games. Let G = I , (Si ), (ui ) be a
strategic form game.

Definition 2.35 A function  : S → R is called an ordinal potential function for the game G if for
each i ∈ I and all s−i ∈ S−i ,
ui (x, s−i ) − ui (z, s−i ) > 0 iff (x, s−i ) − (z, s−i ) > 0, for all x, z ∈ Si .
64 2. GAME THEORY DYNAMICS
G is called an ordinal potential game if it admits an ordinal potential.

Definition 2.36 A function  : S → R is called an (exact) potential function for the game G if for
each i ∈ I and all s−i ∈ S−i ,

ui (x, s−i ) − ui (z, s−i ) = (x, s−i ) − (z, s−i ), for all x, z ∈ Si .

G is called an (exact) potential game if it admits a potential.

A potential function assigns a real value for every s ∈ S. Thus, for bimatrix games (i.e.,
finite games with two players), we can also represent the potential function as a matrix, each entry
corresponding to the vector of strategies from the payoff matrix.

Example 2.37 The matrix P represents a potential function for the “Prisoner’s dilemma" game
described below: * + * +
(1, 1) (9, 0) 4 3
G= , P =
(0, 9) (6, 6) 3 0

The next theorem establishes the existence of a pure strategy Nash equilibrium in finite ordinal
potential games.

Theorem 2.38 Every finite ordinal potential game has at least one pure strategy Nash equilibrium.

Proof. The global maximum of an ordinal potential function is a pure strategy Nash equilibrium.
To see this, suppose that s ∗ corresponds to the global maximum. Then, for any i ∈ I , we have, by
definition, (si∗ , s−i
∗ ) − (s, s ∗ ) ≥ 0 for all s ∈ S . But since  is a potential function, for all i
−i i
and all s ∈ Si ,

ui (si∗ , s−i
∗ ∗
) − ui (s, s−i )≥0 iff (si∗ , s−i
∗ ∗
) − (s, s−i ) ≥ 0.

Therefore, ui (si∗ , s−i

∗ ) − u (s, s ∗ ) ≥ 0 for all s ∈ S and for all i ∈ I . Hence, s ∗ is a pure strategy
i −i i
Nash equilibrium. 2
Note, however, that there may also be other pure strategy Nash equilibria corresponding to
local maxima of the potential function. The next two examples present examples of ordinal and exact
potential games.

Example 2.39 Cournot Competition I firms choose quantity qi ∈ [0, ∞). The payoff function
for player i given by ui (qi , q−i ) = qi (P (Q) − c). We define the function
 I 

(q1 , · · · , qI ) = qi (P (Q) − c).
i=1
 2.3. GAMES WITH SPECIAL STRUCTURE 65
Note that for all i and all q−i > 0,
ui (qi , q−i ) − ui (qi , q−i ) > 0 iff (qi , q−i ) − (qi , q−i ) > 0, ∀ qi , qi > 0.

The function  is therefore an ordinal potential function for this game.

Example 2.40 Cournot Competition Suppose now that P (Q) = a − bQ and costs ci (qi ) are
arbitrary. We define the function


I 
I 
I 
I
∗ (q1 , · · · , qn ) = a qi − b qi2 − b qi ql − ci (qi ).
i=1 i=1 1≤i<l≤I i=1

It can be shown that for all i and all q−i ,

ui (qi , q−i ) − ui (qi , q−i ) = ∗ (qi , q−i ) − ∗ (qi , q−i ), for all qi , qi > 0.

The function  is an exact potential function for this game.

We next study the convergence behavior of simple myopic dynamics in finite ordinal potential
games. The next definition formalizes the strategy profile trajectories generated by such dynamics
in potential games.

Definition 2.41 A path in strategy space S is a sequence of strategy vectors (s 0 , s 1 , · · · ) such that
every two consecutive strategies differ in one coordinate (i.e., exactly in one player’s strategy). An
improvement path is a path (s 0 , s 1 , · · · ) such that, uik (s k ) < uik (s k+1 ) where s k and s k+1 differ in
the ikth coordinate. In other words, the payoff improves for the player who changes his strategy.

An improvement path can be thought of as generated dynamically by “myopic players", who

update their strategies according to one-sided better reply dynamic.

Proposition 2.42 In every finite ordinal potential game, every improvement path is finite.

Proof. Suppose (s 0 , s 1 , · · · ) is an improvement path. Therefore, we have,

(s 0 ) < (s 1 ) < · · · ,

where  is the ordinal potential. Since the game is finite, i.e., it has a finite strategy space, the
potential function takes on finitely many values and the above sequence must end in finitely many
steps. 2
66 2. GAME THEORY DYNAMICS
This result implies that in finite ordinal potential games, every “maximal" improvement path
must terminate in an equilibrium point. That is, the simple myopic learning process based on one-
sided better reply dynamic converges to the equilibrium set.
We conclude this section by presenting a widely applicable class of games, congestion games,
introduced by Rosenthal [89]. Congestion games are characterized by resources shared among many
agents, which experience negative externalities due to the resulting congestion effects. Hence, this
class of games is a standard model of strategic interactions in many network games. We show that
congestion games are potential games and therefore share the appealing properties of these games.

2.3.2.1 Congestion Games

We define the congestion model as a tuple C = I , M, (Si )i∈I , (cj )j ∈M  where:
• I = {1, 2, · · · , I } is the set of players.
• M = {1, 2, · · · , m} is the set of resources.
• Si is the set of nonempty resource combinations (e.g., links or common resources) that player
i can take or use. A strategy for player i is si ∈ Si , corresponding to the subset of resources
that this player is using.
• cj (k) is the benefit for the negative of the cost to each user who uses resource j if k users are
using it.
Given this model, we define the congestion game I , (Si ), (ui ) with utilities

ui (si , s−i ) = cj (kj ),
j ∈si

where kj is the number of users of resource j under strategy s.

The next theorem shows that every congestion game is a potential game.

Theorem 2.43 Every congestion game is a potential game and thus has a pure strategy Nash equilibrium.

Proof. For each j define k̄ji as the usage of resource j excluding player i, i.e.,

k̄ji = I [j ∈ si ] ,
i =i

where I [j ∈ si ] is the indicator for the event that j ∈ si . With this notation, the utility difference
of player i from two strategies si and si (when others are using the strategy profile s−i ) is
 
ui (si , s−i ) − ui (si , s−i ) = cj (k̄ji + 1) − cj (k̄ji + 1).
j ∈si j ∈si
 2.A. APPENDIX: LATTICES 67
Now consider the function
⎡ ⎤
kj
 
(s) = ⎣ cj (k)⎦ .
j∈ i ∈I si k=1

We can also write ⎡ i ⎤

k̄j
 ⎢ ⎥  j i
(si , s−i ) = ⎣ cj (k)⎦ + c (k̄j + 1).
j∈ si k=1 j ∈si
i  =i

Therefore, we have
⎡ i ⎤
k̄j
 ⎢ ⎥  j i
(si , s−i ) − (si , s−i ) = ⎣ cj (k)⎦ + c (k̄j + 1)
j∈ si k=1 j ∈si
i  =i
⎡ i ⎤
k̄j
 ⎢ ⎥  j i
− ⎣ cj (k)⎦ + c (k̄j + 1)
j∈ si k=1 j ∈si
i  =i
 
= cj (k̄ji + 1) − cj (k̄ji + 1)
j ∈si j ∈si
= ui (si , s−i ) − ui (si , s−i ).

2.A APPENDIX: LATTICES

Let ≥ be a binary relation on a nonempty set S. The pair (S, ≥) is a partially ordered set if ≥ is
reflexive (x ≥ x for all x ∈ S), transitive (x ≥ y and y ≥ z implies that x ≥ z), and antisymmetric
(x ≥ y and y ≥ x implies that x = y). A partially ordered set (S, ≥) is (completely) ordered if for
x ∈ S and y ∈ S, either x ≥ y or y ≥ x.
A lattice is a partially ordered set (S, ≥) in which any two elements x, y have a least upper
bound (supremum), supS (x, y) = inf{z ∈ S | z ≥ x, z ≥ y}6 , and a greatest lower bound (infimum),
inf S (x, y) = sup{z ∈ S | z ≤ x, z ≤ y}7 , in the set. For example, any interval of the real line with the
usual order is a lattice since any two points have a supremum and infimum in the interval. However,
the set S ⊂ R2 , S = {(1, 0), (0, 1)} is not a lattice with the vector ordering (the usual component-
wise ordering: x ≤ y if and only if xi ≤ yi for any i) since (1, 0) and (0, 1) have no joint upper
bound in S. The set S = {(0, 0), (0, 1), (1, 0), (1, 1)} is indeed a lattice with the vector ordering.
6 Supremum of {x, y} is denoted by x ∨ y and is called the join of x and y.
7 Infimum of {x, y} is denoted by x ∧ y and is called the meet of x and y.
68 2. GAME THEORY DYNAMICS
Similarly, the simplex in Rn (again with the usual vector ordering) {x ∈ Rn | i xi = 1, xi ≥ 0} is
not a lattice, while the box {x ∈ Rn | 0 ≤ x1 ≤ 1} is.
A lattice (S, ≥) is complete if every nonempty subset of S has a supremum and an infimum
in S. Any compact interval of the real line with the usual order is a complete lattice, while the open
interval (a, b) is a lattice but is not complete (indeed, the supremum of (a, b) does not belong to
(a, b)).
A subset L of the lattice S is a sublattice of S if the supremum and infimum of any two
elements of L (with the supremum and infimum is taken with respect to S) belong also to L. That
is, a sublattice L of the lattice S is a subset of S that is closed under the operations of supremum
and infimum. The sublattice L of S is complete if every nonempty subset of L has a supremum and
infimum in L. A subset that is a lattice or complete lattice in its own right may not be a sublattice or
complete sublattice of a larger lattice, because the relevant suprema and infima are defined relative
to the larger lattice. Thus, the set T = [0, 1) ∪ {2} is a complete lattice under the usual ordering; the
supremum in T for the set [0, 1) is 2 ∈ T . However, T is not a complete sublattice of the lattice [0,2],
because then sup[0, 1) = 1 ∈ / T.
A function f : S → R is supermodular on S if for all x, y ∈ S

f (x) + f (y) ≤ f (x ∧ y) + f (x ∨ y).

Note that supermodularity is automatically satisfied if Si is single dimensional. We next

provide an alternative characterization of supermodularity for smooth functions.

Theorem 2.44 Let I be an interval Rn . Assume that f : Rn → R is twice continuously differentiable

on some open set containing I . Then f is supermodular on I if and only if for all x ∈ I and all i = j ,

∂ 2f
≥ 0.
∂xi ∂xj

In general, supermodularity uses only the order structure of the lattice. It entails no assumptions
of convexity or connectedness of the domain, nor does it require any convexity, differentiability of
the function itself. However, in view of this theorem it is easy to check whether smooth functions
on Euclidean intervals are supermodular.
Let (S, ≥) be a partially ordered set. A function f from S to S is increasing if for all x, y ∈ S,
x ≥ y implies f (x) ≥ f (y).
We finally state the following lattice-theoretical fixed point theorem due to Tarski.

Theorem 2.45 Tarski Let (S, ≥) be a complete lattice and f : S → S an increasing function. Then,
the set of fixed points of f , denoted by E, is nonempty and (E, ≥) is a complete lattice.
 PART II

Network Games
 71

CHAPTER 3

Wireline Network Games

Many games are played over networks, in the sense that players’ payoffs depend on others through a
network-like structure. Classical examples are the allocation of network flows in a communication
network, or of traffic in a transportation network. Since the routing decisions of each user usually
affect the performance of other users (through commonly shared links), this leads to a noncooperative
routing game among the users. Our focus in this chapter is on the equilibria of such routing games.
In particular, we shall consider the important engineering aspect of the efficiency of the equilibrium,
namely how “good" the equilibrium point is compared to the socially optimal operating point (with
respect to a properly defined quality measure).
In Section 3.1, we formally define a general routing model and the associated equilibrium
concepts. We demonstrate that inefficiencies might occur at equilibrium due to selfish behavior.
Moreover, due to self-interested decisions of users, we show that the addition of network resources
may deteriorate network performance, a phenomenon known as the Braess’ paradox [27].
The basic routing game ignores the service provider role in routing traffic. Most large-scale
communication networks, such as the Internet, consist of interconnected administrative domains.
While source (or selfish) routing, where transmission follows the least cost path for each source,
is reasonable across domains, service providers typically engage in traffic engineering to improve
operating performance within their own network. Motivated by this observation, we develop and
analyze in Section 3.2 a model of partially optimal routing, where optimal routing within subnetworks
is overlaid with selfish routing across domains.
Another aspect that is missing from a basic routing game is the possibility that a service
provider charges a price for utilizing its resources. Prices are often set by multiple service providers in
control of their administrative domains with the objective of maximizing their (long-run) revenues.
In Section 3.3, we investigate the implications of profit-maximizing pricing by multiple decentralized
service providers. Finally, several extensions to the models considered in this chapter are outlined in
Section 3.4.

3.1 SELFISH ROUTING, WARDROP EQUILIBRIUM AND

EFFICIENCY
Before precisely formulating the routing model that will be the subject of this chapter, consider the
motivating network example depicted in Figure 3.1.The example in this figure is due to Pigou (1920).
There is one unit of load that needs to be routed from a source node to a destination node. This load
corresponds to aggregate load of infinitesimal users (e.g., motorists in a transportation network).
72 3. WIRELINE NETWORK GAMES
There are two alternative links (routes) that may carry the traffic. Each link is characterized by a
per-unit cost function. In the transportation context, the cost may correspond to the (average) delay
of a vehicle1 . Note that the cost in the upper link depends on the amount of load it accommodates,
while the load in the lower link is a constant, thus congestion independent.

delay depends on congestion

l1 (x) = x

1 unit of traffic

l2 (x) = 1
no congestion effects

Figure 3.1: The Pigou example.

Observe first that the socially optimal solution, i.e., the flow allocation which minimizes the
aggregate cost is to split traffic equally between the two routes, giving
 1 1 3
min Csystem (x S ) = li (xiS )xiS = + = .
x1 +x2 ≤1 4 2 4
i

However, in a noncooperative framework, the above solution does not correspond to a Nash
equilibrium. Indeed, consider the corresponding game between the infinitesimal players (such games
between an infinite number of “small" users are often referred to as non-atomic games). The players
that use the bottom link experience a cost which is strictly higher than the cost experienced by players
in the upper link, and would therefore modify their routing decision. Consequently, the unique Nash
equilibrium of the game, also referred to as Wardrop equilibrium (WE)2 is x1 = 1 and x2 = 0 (since
for any x1 < 1, l1 (x1 ) < 1 = l2 (1 − x1 )), giving an aggregate cost of

Ceq (x W E ) = li (xiW E )xiW E = 1 + 0 = 1.
i

Note that the aggregate cost at the Wardrop equilibrium is larger than the optimal cost by a factor
of 43 . We will establish later in Section 3.1.3 that 43 is actually the worst-case factor for the efficiency
loss when all network topologies are taken into account (as long as the delay costs are linear, as in
our example).
The rest of this section is organized as follows. We present the routing model in Section 3.1.1,
restricting attention to a single source-destination pair. We define and characterize the Wardrop
1 Accordingly, the terms “delay" and “cost" are interchangeably used in this chapter.
2 Named after John Glen Wardrop, an English transport analyst who developed this equilibrium concept in 1952 [105].
 3.1. SELFISH ROUTING, WARDROP EQUILIBRIUM AND EFFICIENCY 73
equilibrium in Section 3.1.2. We then study in Section 3.1.3 the efficiency loss incurred by self-
interested behavior. We conclude this section by extending the framework of the routing problem
to multiple source-destinations.

3.1.1 ROUTING MODEL

We consider in this section networks with a single origin-destination pair. Let N = (V , A) be the
directed network with V being the set of nodes and A being the set of links. Denote by P the set
of paths between the origin and the destination. Let xp denote the flow on path p ∈ P . We assume
that each link i ∈ A has a latency function li (xi ), where

xi = xp . (3.1)
{p∈P |i∈p}

Here the notation p ∈ P |i ∈ p denotes the paths p that traverse link i ∈ A. The latency function
captures congestion effects, hence the latency is a function of the total flow on the link. We assume
for simplicity that li (xi ) is nonnegative, differentiable, and nondecreasing. We further assume that
all traffic is homogeneous, in the sense that all players (e.g., drivers) have the same experience when
utilizing the link. The total traffic is normalized to one, and the set of players is accordingly given
by I = [0, 1].

Remark 3.1 So far in this monograph, we took the set of players, I , to be a finite set. Nonetheless,
this is not essential for a proper definition of a game. In non-atomic games, I is typically taken to
be some interval in R, as in the above.

We denote a routing pattern by the vector x. If x satisfies (3.1), and furthermore p∈P xp = 1
and xp ≥ 0 for all p ∈ P , then x is a feasible routing pattern. The total delay (latency) cost of a
routing pattern x is:

C(x) = xi li (xi ).
i∈A

That is, it is the sum of latencies li (xi ) for each link i ∈ A multiplied by the flow over this link, xi ,
summed over all links A.
The socially optimal routing x S is a feasible routing pattern that minimizes the aggregate cost;
it can be obtained as a solution of the following optimization problem

minimize xi li (xi )
i∈A
subject to xp = xi , for all i ∈ E,
{p∈P |i∈p}

xp = 1 and xp ≥ 0 for all p ∈ P .
p∈P
74 3. WIRELINE NETWORK GAMES
3.1.2 WARDROP EQUILIBRIUM
As indicated above, an underlying assumption in our routing model is that each player is “infinitesi-
mal", i.e., has a negligible effect on overall performance. A Wardrop equilibrium, which we formally
define below, is a convenient modeling tool when each participant in the game is small. This equi-
librium notion can be regarded as a Nash equilibrium in this game, where the strategies of the other
players are replaced by aggregates, due to the non-atomic nature of the game; in our specific context,
the aggregates correspond to the total traffic on different routes.
We now proceed to formally define the notion of a Wardrop equilibrium. For concreteness, we
define the equilibrium in the context of a transportation network, although the context can obviously
be more general, and correspond to any non-atomic game.
Since a Wardrop equilibrium can be viewed as a Nash equilibrium with an infinite number
of small decision makers, it has to be the case that for each motorist their routing choice must be
optimal. This implies that if a motorist k ∈ I is using path p, then there does not exist path p such
that  
li (xi ) > li (xi ).
i∈p i∈p

A Wardrop equilibrium is formally defined as follows.

Definition 3.2 A feasible flow patters x is a Wardrop equilibrium if

 
li (xi ) = li (xi ) for all p, p ∈ P with xp , xp > 0, and
i∈p i∈p
 
li (xi ) ≥ li (xi ) For all p, p ∈ P with xp > 0 and xp = 0.
i∈p i∈p

A fundamental property of the Wardrop equilibrium is that it can be obtained via the solution
of a convex optimization problem. This important property suggests that a Wardrop equilibrium can
be characterized (i.e., computed) efficiently. We state this result below and provide an outline for its
proof.

Theorem 3.3 Beckmann, McGuire and Winsten [18] A feasible routing pattern x W E is a Wardrop
equilibrium if and only if it is a solution to
 xi
minimize li (z) dz
i∈A0

subject to xp = xi , for all i ∈ A,

{p∈P |i∈p}

xp = 1 and xp ≥ 0 for all p ∈ P . (3.2)
p∈P
 3.1. SELFISH ROUTING, WARDROP EQUILIBRIUM AND EFFICIENCY 75
Moreover, if each li is strictly increasing, then xW E is unique.

Proof. Observe first that by Weierstrass’ Theorem, a solution to (3.2) exists, and thus a Wardrop
equilibrium always exists. Next, note that (3.2) can be written as

 i∈p xp
minimize li (z) dz
0

i∈A
subject to xp = 1 and xp ≥ 0 for all p ∈ P .
p∈P

Since each li is nondecreasing, this is a convex program. Therefore, first-order conditions are neces-
sary and sufficient. The first-order conditions with respect to xp are
 ! "
li xiW E ≥ λ
i∈p

with the complementary slackness, i.e., with equality whenever xpW E > 0. Here λ is the Lagrange
multiplier on the constraint p∈P xp = 1. Consequently, the Lagrange multiplier will be equal
to the lowest cost path, which then implies the result that for all p, p ∈ P with xpW E , xpW E > 0,
WE) = W E ); clearly, for paths with x W E = 0, the cost can be higher.
i∈p li (xi i∈p li (xi p
Finally, if each li is strictly increasing, then the set of equalities WE) =
i∈p li (xi
W E ) admits unique solution, establishing uniqueness.
i∈p li (xi 2

3.1.3 INEFFICIENCY OF THE EQUILIBRIUM

We saw from the Pigou example that the Wardrop equilibrium fails to minimize total delay, hence it
is generally inefficient when compared to the performance at the social optimum. More generally, it
is well known that equilibria exhibit inefficiencies in diverse noncooperative scenarios. Koutsoupias
and Papadimitriou [55] introduced the term Price of Anarchy (POA) to quantify these inefficiencies
in games over all possible instances.
In our routing context, let R denote the set of all routing instances, covering all possible
network topologies and all latency functions which belong to a given family of functions (e.g.,
affine). Then the PoA is defined as the worst-case efficiency over all instances, namely

C(x S (R))
inf .
R∈R C(x W E (R))

The Pigou example establishes that when restricting ourselves to affine latency functions, the PoA
is at least 43 . The following theorem establishes that the PoA is exactly 43 , namely 43 is the worst
possible ratio between the social optimum cost and the cost at a Wardrop equilibrium.
76 3. WIRELINE NETWORK GAMES
Theorem 3.4 Roughgarden and Tardos [91]
Let Rconv and Raff denote the class of all routing instances where latency functions are convex and
affine, respectively.
(a)
C(xSO (R))
infconv = 0.
R∈R C(xW E (R))
(b) Consider a routing instance R = (V , A, P , s, t, X, l) where lj is an affine latency function for all
j ∈ A. Then,
C(xSO (R)) 3
W E
≥ .
C(x (R)) 4
Furthermore, the bound above is tight.

Proof. We shall prove part (b) of the theorem later in Section 3.2.3 under a more general setup
(Theorem 3.9).
We prove part (a) by means of a simple example, demonstrating that the Wardrop equilibrium
can be arbitrarily inefficient when allowing the larger set of convex increasing latency functions.
Consider the example in Figure 3.2, which is the same as the Pigou example, except with a different
latency on link 1 which is now l1 (x) = x k , k ≥ 1.

l1( x ) = x k

1 unit of traffic

l2( x ) = 1

Figure 3.2: Extension of the Pigou example to non-linear latency functions.

In this example, the socially optimal routing is obtained by the following equation
l1 (x1 ) + x1 l1 (x1 ) = l2 (1 − x1 ) + (1 − x1 ) l2 (1 − x1 ) ,
leading to
x1k + kx1k = 1 .
Therefore, the system optimum sets x1 = (1 + k)−1/k and x2 = 1 − (1 + k)−1/k , so that

li (xiS )xiS = (1 + k)− k + 1 − (1 + k)−1/k .
k+1
min Csystem (x S ) =
x +x ≤1
1 2
i
 3.1. SELFISH ROUTING, WARDROP EQUILIBRIUM AND EFFICIENCY 77
The Wardrop equilibrium again has x1 = 1 and x2 = 0 (since once again for any x1 < 1,
l1 (x1 ) < 1 = l2 (1 − x1 )). Thus

Ceq (x W E ) = li (xiW E )xiW E = 1 + 0 = 1.
i

Therefore, the PoA is upper bounded by

Csystem (x S )
= (1 + k)− + 1 − (1 + k)−1/k .
k+1
k
Ceq (x W E )

This limits to 0 as k → ∞ (the first term tends to zero, while the last term limits to one). Thus, the
equilibrium can be arbitrarily inefficient relative to the social optimum.
There has been extensive research on the Price of Anarchy in related selfish routing models.
We do not attempt to cover all models in this survey. We refer the reader to a survey book by
Roughgarden [90] on the subject.
The Price of Anarchy notion allows one to quantify the extent to which selfish behavior affects
network performance. A related consequence of selfish behavior in networks is captured through the
celebrated Braess’ Paradox [27], which demonstrates that the addition of an intuitively helpful route
negatively impacts network users at equilibrium. Figure 3.3 depicts the example used by Braess. A
link with zero cost is added to the network, nonetheless the equilibrium, which coincided with the
social optimum before the addition of the new link, becomes inefficient; this can be immediately seen,
as the overall cost increases. This outcome is paradoxical since the addition of another route should
help traffic; obviously, the addition of a link can never increase aggregate delay in the social optimum.
An active research direction has been to identify conditions under which the Braess’ paradox would
never occur, e.g., based on the network topology or other factors; see [90] and references therein for
more details.

1
1/2
x 1 x 1
1 unit of 1 unit of
traffic traffic 0

1 x 1 x

1/2

Ceq = 1/2 (1/2+1) + 1/2 (1/2+1) = 3/2 Ceq = 1 + 1 = 2

Csys = 3/2 Csys = 3/2

Figure 3.3: The Braess’ Paradox.

78 3. WIRELINE NETWORK GAMES
3.1.4 MULTIPLE ORIGIN-DESTINATION PAIRS
We conclude this section by briefly considering the generalization of the network routing model to
multiple origin-destination pairs. Suppose that there are K such pairs (some of them having possibly
the same origin or destination). Assume that origin-destination pair j has total traffic rj . Let us
denote the set of paths for origin-destination pair j by Pj , and now P = ∪j Pj . Then the socially
optimal routing pattern is a solution to

minimize xi li (xi )
i∈E
subject to xp = xi , i ∈ E,
{p∈P |i∈p}

xp = rj , j = 1, . . . , k, and xp ≥ 0 for all p ∈ P .
p∈Pj

Turning now to the equilibrium behavior, it can be easily shown that essentially the same
characterization theorem for Wardrop equilibrium applies with multiple origin-destination pairs.
Formally,

Theorem 3.5 A feasible routing pattern x W E is a Wardrop equilibrium if and only if it is a solution to
  xi
minimize li (z) dz
i∈E0

subject to xp = xi , i ∈ E,
{p∈P |i∈p}

xp = rj , j = 1, . . . , k, and xp ≥ 0 for all p ∈ P .
p∈Pj

Moreover, if each li is strictly increasing, then x W E is uniquely defined.

Regarding equilibrium efficiency, it turns out that the PoA bounds obtained for single source-
destination remains so when allowing for multiple source-destination pairs; see [90] for further
details.

3.2 PARTIALLY OPTIMAL ROUTING

3.2.1 BACKGROUND AND MOTIVATION
Since the passage of the Telecommunications Act in 1996, the Internet has undergone a dramatic
transformation and experienced increasing decentralization. Today, thousands of network providers
cooperate and compete to provide end-to-end network service to billions of users worldwide. While
end-users care about the performance across the entire network, individual network providers opti-
mize their own objectives.The Internet’s architecture provides no guarantees that provider incentives
will be aligned with end-user objectives.
 3.2. PARTIALLY OPTIMAL ROUTING 79
The emergence of overlay routing over the past decade has further highlighted the potentially
conflicting objectives of the service provider and the end-users. In overlay routing, end-user software
(such as peer-to-peer file-sharing software) makes route selection decisions on the basis of the
best end-to-end performance available at any given time, while administrative domains control
the routing of traffic within their own (sub)networks. Network operators use traffic engineering to
optimize performance, and also to react to the global routing decisions of overlay networks (e.g.,
[85]).
These considerations make it clear that the study of routing patterns and performance in
large-scale communication networks requires an analysis of partially optimal routing, where end-to-
end route selection is selfish and responds to aggregate route latency, but network providers redirect
traffic within their own networks to achieve minimum intradomain total latency.
We develop and analyze in this section a model of partially optimal routing, combining
selfish across-domain routing and traffic engineering by service providers within their administrative
domains.
We consider routing flows between multiple source-destination pairs through a network.
As in the previous section, each link is endowed by a latency function describing the congestion
level (e.g., delay or probability of packet loss) as a function of the total flow passing through the
link. Each source-destination pair in the network has a fixed amount of flow, and flows follow the
minimum delay route among the available paths as captured by the notion of Wardrop equilibrium. Our
innovation is to allow subsets of the links in the network (“subnetworks”) to be independently owned
and operated by different providers, and consider the possibility that these providers engage in traffic
engineering and route traffic to minimize the total (or average) latency within their subnetworks.
Source-destination pairs sending traffic across subnetworks perceive only the effective latency resulting
from the traffic engineering of the service providers. The resulting equilibrium, which we call a
partially optimal routing (POR) equilibrium, is a Wardrop equilibrium according to the effective
latencies seen by the source-destination pairs. This model provides a stylized description of the
practice of traffic engineering in the Internet. The content of this section is based on [3].

3.2.2 THE MODEL

The network model, the definition of the socially optimal solution and the definition of the Wardrop
equilibrium remain the same as in Section 3.1. We now assume that a single network provider
controls a subnetwork with unique entry and exit points; within this domain, the provider optimizes
performance of traffic flow. Formally, we assume there is a disjoint collection of directed subgraphs
(subnetworks) inside of G. Within a subnetwork G0 = (V0 , A0 ), a service provider optimally routes
all incoming traffic. Let s0 ∈ V0 denote the unique entry point to G0 , and let t0 ∈ V0 denote the
unique exit point from G0 . Let P0 denote the set of available paths from s0 to t0 using the edges in A0 .
We make the assumption that every path in P passing through any node in V0 must contain a path
in P0 from s0 to t0 ; this is consistent with our assumption that G0 is an independent autonomous
80 3. WIRELINE NETWORK GAMES
system, with a unique entry and exit point. We call R0 = (V0 , A0 , P0 , s0 , t0 ) a subnetwork of G, and
with a slight abuse of notation, we say that R0 ⊂ R.
Given an incoming amount of flow X0 , the network provider chooses a routing of flow to
solve the following optimization problem to minimize total (or average) latency:

minimize xj lj (xj ) (3.3)
j ∈A0

subject to yp = xj , j ∈ A0 ;
p∈P0 :j ∈p

yp = X0 ;
p∈P0
yp ≥ 0, p ∈ P0 .

In this optimization problem, the subnetwork owner sees an incoming traffic amount X0 , and chooses
the optimal routing of this flow through the subnetwork. This is a formal abstraction of the process
of traffic engineering carried out by many network providers to optimize intradomain performance.
Let L(X0 ) denote the optimal value of the preceding optimization problem. We define
l0 (X0 ) = L(X0 )/X0 as the effective latency of partially optimal routing in the subnetwork R0 , with
flow X0 > 0. If traffic in the entire network G routes selfishly, while traffic is optimally routed
within G0 , then replacing G0 by a single link with latency l0 will leave the Wardrop equilibrium
flow unchanged elsewhere in G.
We have the following simple lemma that provides basic properties of l0 and L.

Lemma 3.6 Assume that every latency function, lj , is a strictly increasing, nonnegative, and continuous
function. Then:
(a) The effective latency l0 (X0 ) is a strictly increasing function of X0 > 0.
(b) Assume further that each lj is a convex function. The total cost L(X0 ) is a convex function of X0 .

In light of the preceding lemma, we can extend the definition of l0 so that l0 (0) =
limx0 ↓0 l0 (x0 ); the preceding limit is well defined since l0 is strictly increasing.
To define the overall network performance under partially optimal routing, first sup-
pose that there is a single independently-operated subnetwork. Given a routing instance R =
(V , A, P , s, t, X, l), and a subnetwork R0 = (V0 , A0 , P0 , s0 , t0 ) defined as above, we define a new
routing instance R = (V , A , P , s, t, X, l ) as follows:

V = (V \ V0 ) {s0 , t0 };

A = (A \ A0 ) {(s0 , t0 )};

P corresponds to all paths in P , where any subpath in P0 is replaced by the link (s0 , t0 ); and l
consists of latency functions lj for all edges in A \ A0 , and latency l0 for the edge (s0 , t0 ). Thus, R
 3.2. PARTIALLY OPTIMAL ROUTING 81
is the routing instance R with the subgraph G0 replaced by a single link with latency l0 ; we call R
the equivalent POR instance for R with respect to R0 . The overall network flow in R with partially
optimal routing in R0 , xP OR (R, R0 ), is defined to be the Wardrop equilibrium flow in the routing
instance R :
xP OR (R, R0 ) = xW E (R ).
In other words, it is equilibrium with traffic routed selfishly given the effective latency l0 of the
subnetwork R0 . Note also that this formulation leaves undefined the exact flow in the subnetwork
R0 ; this is to be expected since problem (3.3) may not have a unique solution.
The total latency cost of the equivalent POR instance for R with respect to R0 is given by

C(xP OR (R, R0 )) = xjP OR (R, R0 )lj (xjP OR (R, R0 )).
j ∈A

3.2.3 EFFICIENCY OF PARTIALLY OPTIMAL ROUTING

We first consider the effect of optimal routing within subnetworks on the performance of the overall
network. One might conjecture that optimally routing traffic within subnetworks should improve
the overall performance. The following example shows that this need not be the case.

Example 3.7 Consider the network G = (V , A) with source and destination nodes s, t ∈ V illus-
trated in Figure 3.4(a). Let R = (V , A, P , s, t, 1, l) be the corresponding routing instance, i.e., one
unit of flow is to be routed over this network. The subnetwork G0 consists of the two parallel links
in the middle, links 5 and 6, with latency functions

l5 (x5 ) = 0.31, l6 (x6 ) = 0.4 x6 .

The latency functions for the remaining links in the network are given by

l1 (x1 ) = x1 , l2 (x2 ) = 3.25,

l3 (x3 ) = 1.25, l4 (x4 ) = 3x4 .

Assume first that the flow through the subnetwork G0 is routed selfishly, i.e., according to the
Wardrop equilibrium. Given a total flow X0 through the subnetwork G0 , the effective Wardrop
latency can be defined as
1
l˜0 (X0 ) = C(x W E (R0 )), (3.4)
X0
where R0 is the routing instance corresponding to the subnetwork G0 with total flow X0 . The
effective Wardrop latency for this example is given by

l˜0 (X0 ) = min{0.31, 0.4X0 }.

82 3. WIRELINE NETWORK GAMES

l1(x1) = x1 l2(x2) = 3.25

s t
1 unit l (x ) = 0.31 l (x ) = 0.4 x
5 5 6 6 6

l (x ) = 1.25 l (x ) = 3x
3 3 4 4 4

(a)

l (x ) l (x ) l (x ) l (x )
1 1 2 2 1 1 2 2

~ l0(X0)
l0(X0)

l3(x3) l4(x4) l3(x3) l4(x4)

(b) (c)

Figure 3.4: A network for which POR leads to a worse performance relative to selfish routing. Figures (b)
and (c) illustrate representing the subnetwork with a single link with Wardrop effective latency l˜0 (X0 )
and optimal effective latency l0 (X0 ), respectively.

Substituting the subnetwork with a single link with latency function l˜0 yields the network in Figure
3.4(b). It can be seen that selfish routing over the network of Figure 3.4(b) leads to the link flows
x1W E = 0.94 and X0W E = 0.92, with a total cost of C(xW E (R)) = 4.19. It is clear that this flow
configuration arises from a Wardrop equilibrium in the original network.
Assume next that the flow through the subnetwork G0 is routed optimally, i.e., as the optimal
solution of problem (3.3) for the routing instance corresponding to G0 . Given a total flow X0 through
the subnetwork G0 , the effective latency of optimal routing within the subnetwork G0 can be defined
as

L(X0 )
l0 (X0 ) = ,
X0
 3.2. PARTIALLY OPTIMAL ROUTING 83
where L(X0 ) is the optimal value of problem (3.3). The effective optimal routing latency for this
example is given by

0.4X0 , if 0 ≤ X0 ≤ 0.3875;
l0 (X0 ) =
0.31 − 0.0961
1.6X0 , if X0 ≥ 0.3875.

Substituting the subnetwork with a single link with latency function l0 yields the network in Figure
3.4(c). Note that selfish routing over this network leads to the partially optimal routing (POR)
equilibrium. It can be seen that at the POR equilibrium, the link flows are given by x1P OR = 1 and
X0P OR = 1, with a total cost of C(xP OR (R)) = 4.25, which is strictly greater than C(xW E (R)).

In this section, we quantify the inefficiency of partially optimal routing. Our metric of effi-
ciency is the ratio of the total cost at the social optimum to the total cost at the partially optimal
routing solution, C(xSO )/C(xP OR ). Throughout, we assume that all independently-operated sub-
networks can be represented as subgraphs with unique entry and exit points.
We will establish two main theorems. The first provides a tight bound on the loss of efficiency
when all latency functions are affine; and the second provides a tight bound on the loss of efficiency
when all latency functions are polynomials of bounded degree.
We start with a simple result that compares the worst-case efficiency loss of partially optimal
routing with that of selfish routing. These relations will be useful in finding tight bounds on the
efficiency loss of partially optimal routing. Recall that Rconv , Raff , and Rconc denote the class of
all routing instances where latency functions are convex, affine, and concave, respectively.

Proposition 3.8

(a) For all R ∈ {Rconv , Raff , Rconc }, we have

C(xSO (R)) C(xSO (R))

inf ≤ inf . (3.5)
R∈R C(xP OR (R, R0 )) R∈R C(xW E (R))
R0 ⊂R

(b)
C(xSO (R)) C(xSO (R))
inf = inf .
R∈R C(xP OR (R, R0 )) R∈R C(xW E (R))
R0 ⊂R

(c)
C(xSO (R)) C(xSO (R))
inf ≥ inf .
R∈Raff C(xP OR (R, R0 )) R∈Rconc C(xW E (R))
R0 ⊂R
84 3. WIRELINE NETWORK GAMES
Our main theorem in this section is an extension of the results in Theorem 3.4 to the setting
of partially optimal routing.

Theorem 3.9

(a)
C(xSO (R))
infconv = 0.
R∈R C(xP OR (R, R0 ))
R0 ⊂R

(b) Consider a routing instance R = (V , A, P , s, t, X, l) where lj is an affine latency function for all
j ∈ A; and a subnetwork R0 of R. Then:

C(xSO (R)) 3
P OR
≥ .
C(x (R, R0 )) 4
Furthermore, the bound above is tight.

Proof. Part (a) of the theorem is an immediate corollary of Proposition 3.8(a) (for R = Rconv ) and
Theorem 3.4(a).
The remainder of the proof establishes part (b) of the theorem by proving two lemmas. The
first provides a tight bound of 3/4 on the ratio of the optimal routing cost to the selfish routing cost
for routing instances in which the latency function of each link is a concave function. This lemma is
relevant because when all latency functions are affine, the effective latency of any subnetwork under
partially optimal routing is concave, as shown in the second lemma.
The proof of the following lemma uses a geometric argument that was used in [36]. This
result also follows from the analysis in [35]. Here, we provide an alternative proof, which will be
useful in our subsequent analysis.

Lemma 3.10 Let R ∈ Rconc be a routing instance where all latency functions are concave. Then,

C(xSO (R)) 3
W E
≥ .
C(x (R)) 4
Furthermore, this bound is tight.

Proof of Lemma. Consider a routing instance R ∈ Rconc , with R = (V , A, P , s, t, X, l). Let

xW E be the flow configuration at a Wardrop equilibrium. Recall that x W E is a Wardrop equilibrium
if and only if it satisfies 
lj (xjW E )(xjW E − xj ) ≤ 0, (3.6)
j ∈A
 3.2. PARTIALLY OPTIMAL ROUTING 85
for all feasible solutions x for the same routing instance (see, e.g., [39]).
By Eq. (3.6), for all feasible solutions x of Problem (3.2), we have

C(xW E ) = xjW E lj (xjW E ) (3.7)
j ∈A

≤ xj lj (xjW E ) (3.8)
j ∈A
 
= xj lj (xj ) + xj (lj (xjW E ) − lj (xj )).
j ∈A j ∈A

We next show that for all j ∈ A, and all feasible solutions x of Problem (3.2), we have
1 WE
xj (lj (xjW E ) − lj (xj )) ≤ x lj (xjW E ). (3.9)
4 j
If xj ≥ xjW E , then since lj is nondecreasing, we have lj (xjW E ) ≤ lj (xj ), establishing the desired
relation (3.9). Assume next that xj < xjW E . The term xj (lj (xjW E ) − lj (xj )) is equal to the area of
the shaded rectangle in Figure 1. Consider the triangle formed by the three points

(0, lj (xjW E )), (0, lj (xj ) − lj (xj )xj ),

 
lj (xjW E ) − lj (xj ) + lj (xj )xj
, lj (xjW E ) .
lj (xj )
Denote this triangle by T . It can be seen that
1
xj (lj (xjW E ) − lj (xj )) ≤ Area(T ).
2
By the concavity of lj , we further have
 xjW E  lj (xjW E ) xjW E lj (xjW E )
Area(T ) ≤ dydx ≤ ,
0 lj (x) 2

i.e., the area of triangle T is less than or equal to the area between the curves y = lj (x) and
y = lj (xjW E ) in the interval x ∈ [0, xjW E ], which in turn is less than or equal to half of the area
xjW E lj (xjW E ). Combining the preceding two relations, we obtain Eq. (3.9), which implies
 1  WE 1
xj (lj (xjW E ) − lj (xj )) ≤ xj lj (xjW E ) = C(xW E ).
4 4
j ∈A j ∈A

Combining with Eq. (3.8), we see that for all feasible solutions x of Problem (3.2), we have
3 
C(xW E ) ≤ xj lj (xj ).
4
j ∈A
86 3. WIRELINE NETWORK GAMES
Since the socially optimal flow configuration xSO is a feasible solution for Problem (3.2), we obtain
the desired result. 

lj ( x )
lj ( x WE )
j

lj (xj)

x
x x WE
j j

Figure 3.5: Illustration of the proof of Proposition 3.10.

The following lemma, which establishes that the effective latency l0 of a subnetwork under
partially optimal routing is concave when the latency functions are affine, completes the proof of
part (b) of the theorem.

Lemma 3.11 Let R0 = (V0 , A0 , P0 , s0 , t0 ) be a subnetwork. Assume that the latency functions of all
links in the subnetwork are nonnegative affine functions, i.e., for all j ∈ A0 , lj (xj ) = aj xj + bj , where
aj ≥ 0 and bj ≥ 0. Let l0 (X0 ) denote the effective latency of partially optimal routing of X0 units of flow
in the subnetwork R0 . Then l0 (X0 ) is a concave function of X0 .

Proof of Lemma. Since the lj are affine, for all X0 ≥ 0, we have

 aj xj2 bj xj
l0 (X0 ) = minyp ≥0, p∈P +
X0 X0
j ∈A0

subject to yp = xj , j ∈ A0 ;
p∈P0 :j ∈p

yp = X0 .
p∈P0
 3.2. PARTIALLY OPTIMAL ROUTING 87
yp xj
Using the change of variables ŷp = X0 for all p ∈ P0 , and x̂j = X0 for all j ∈ A0 in the preceding
optimization problem, we obtain

l0 (X0 ) = minŷp ≥0, p∈P0 aj X0 x̂j2 + bj x̂j (3.10)
j ∈A0

subject to ŷp = x̂j , j ∈ A0 ;
p∈P0 :j ∈p

ŷp = 1.
p∈P0

Denote the feasible set of problem (3.10) by Y , i.e.,

⎧ ⎫
⎨ %  ⎬
%
Y = y % yp ≥ 0, ∀ p ∈ P0 , yp = 1 .
⎩ ⎭
p∈P0

Then by defining xj (y) = p∈P0 :j ∈p yp , we

can write (3.10) equivalently as:
⎡⎛ ⎞ ⎛ ⎞⎤
 
l0 (X0 ) = inf ⎣⎝ aj xj (y)2 ⎠ X0 + ⎝ bj xj (y)⎠⎦ .
y∈Y
j ∈A0 j ∈A0

But now observe that l0 (X0 ) is the infimum of a collection of affine functions of X0 . By a stan-
dard result in convex analysis (see, e.g., [21], Proposition 1.2.4(c)), it follows that l0 (X0 ) is concave. 

Combining Lemmas 3.10 and 3.11 with Proposition 3.8 completes the proof of part (b) of
Theorem 3.9. 2

3.2.4 EXTENSIONS
To conclude this section, we next provide a brief description of additional results that were obtained
in [3]. We first mention that this paper provides tight bounds on the loss of efficiency when all
latency functions are polynomials of bounded degree.
In contrast to the results in Theorem 3.9 that match the corresponding bounds for selfish
routing throughout the whole network, when subnetworks have multiple entry-exit points, the
performance of partially optimal routing can be arbitrarily bad, even with linear latencies. This
result suggests that special care needs to be taken in the regulation of traffic in large-scale networks
overlaying selfish source routing together with traffic engineering within subnetworks.
The paper [3] also provides conditions for service providers to prefer to engage in traffic
engineering rather than allowing all traffic to route selfishly within their network. The latter is a
possibility because selfish routing may discourage entry of further traffic into their subnetwork,
reducing total delays within the subnetwork, which may be desirable for the network provider when
there are no prices per unit of transmission.
88 3. WIRELINE NETWORK GAMES
Overall, we believe that the model of partially optimal routing presented in this section is a
good approximation to the functioning of large-scale communication networks, such as the Internet.

3.3 CONGESTION AND PROVIDER PRICE COMPETITION

3.3.1 PRICING AND EFFICIENCY WITH CONGESTION EXTERNALITIES
We now construct a model of resource allocation in a network with competing selfish users and
profit-maximizing service providers. The central question is whether the equilibrium prices that
emerge in such a framework affect system efficiency. The class of models incorporating strategic
behavior by service providers introduces new modeling and mathematical challenges. These models
translate into game-theoretic competition models with negative congestion externalities,3 whereby
the pricing decision of a service provider affects the level of traffic and thus the extent of congestion in
other parts of the network. Nevertheless, tractable analysis of pricing decisions and routing patterns
are possible under many network topologies.
Models incorporating for-profit service providers have been previously investigated in [16],
[17], and [4]. Here, we develop a general framework for the analysis of price competition among
providers in a congested (and potentially capacitated) network building on [5] and [6]. We will
see that despite its conceptual simplicity, this framework has rich implications. We illustrate some
of these, for example, by showing the counterintuitive result that increasing competition among
providers can reduce efficiency, which is different from the results of the most common models of
competition in economics. Most importantly, we also show that it is possible to quantify the extent
to which prices set by competing service providers affect system efficiency. While generally service
provider competition does not lead to an equilibrium replicating the system optimum, the extent of
inefficiency resulting from price competition among service providers can often be bounded.
We start with a simple example which shows the efficiency implications of competition be-
tween two for-profit service providers.

Example 3.12 One unit of traffic will travel from an origin to a destination using either route 1
or route 2 (cf. Figure 3.6). The latency functions of the links, which represent the delay costs as a
function of the total link flow, are given by

x2 2
l1 (x) = , l2 (x) = x.
3 3
It is straightforward to see that the efficient allocation [i.e., one that minimizes the total delay cost
i li (xi )xi ] is x1 = 2/3 and x2 = 1/3, while the (Wardrop) equilibrium allocation that equates
S S

delay on the two paths is x1 ≈ .73 > x1S and x2W E ≈ .27 < x2S . The source of the inefficiency is
W E

that each unit of traffic does not internalize the greater increase in delay from travel on route 1, so
there is too much use of this route relative to the efficient allocation.
3 An externality arises when the actions of the player in a game affects the payoff of other players.
 3.3. CONGESTION AND PROVIDER PRICE COMPETITION 89

l1(x)=x2/3

1 unit of
traffic

l2(x)=(2/3)

Figure 3.6: A two link network with congestion-dependent latency functions.

Now consider a monopolist controlling both routes and setting prices for travel to maximize
its profits. We show below that in this case, the monopolist will set a price including a markup,
which exactly internalizes the congestion externality. In other words, this markup is equivalent to
the Pigovian tax that a social planner would set in order to induce decentralized traffic to choose the
efficient allocation.
 Consequently,
 in this simple example, monopoly prices will be p1ME = (2/3)3 +
k and p2ME = 2/32 + k, for some constant k. The resulting traffic in the Wardrop equilibrium will
be identical to the efficient allocation, i.e., x1ME = 2/3 and x2ME = 1/3.
Finally, consider a duopoly situation, where each route is controlled by a different profit-
maximizing
 provider.
 In this case, it can be shown that equilibrium prices will take the form piOE =
xiOE l1 + l2 [see Eq. (3.16) in Section 3.3.4], or more specifically, p1OE ≈ 0.61 and p2OE ≈ 0.44.
The resulting equilibrium traffic is x1OE ≈ .58 < x1S and x2OE ≈ .42 > x2S , which also differs from
the efficient allocation. It is noteworthy that although the duopoly equilibrium is inefficient relative
to the monopoly equilibrium, in the monopoly equilibrium k is chosen such that all of the consumer
surplus is captured by the monopolist, while in the oligopoly equilibrium users may have positive
consumer surplus.4

The intuition for the inefficiency of the duopoly relative to the monopoly is related to a new
source of (differential) monopoly power for each duopolist, which they exploit by distorting the
pattern of traffic: when provider 1, controlling route 1, charges a higher price, it realizes that this
will push some traffic from route 1 to route 2, raising congestion on route 2. But this makes the
traffic using route 1 become more “locked-in,” because their outside option, travel on the route 2, has
become worse. As a result, the optimal price that each duopolist charges will include an additional
markup over the Pigovian markup. Since the two markups are generally different, they will distort
the pattern of traffic away from the efficient allocation.

4 Consumer surplus is the difference between users’ willingness to pay (reservation price) and effective costs, p + l (x ), and is
i i i
thus different from the social surplus (which is the difference between users’ willingness to pay and latency cost, li (xi ), thus also
takes into account producer surplus/profits).
90 3. WIRELINE NETWORK GAMES
3.3.2 MODEL
We consider a network with I parallel links. Let I = {1, . . . , I } denote the set of links. Let xi denote
the total flow on link i, and x = [x1 , . . . , xI ] denote the vector of link flows. Each link in the network
has a flow-dependent latency function li (xi ), which measures the delay as a function of the total
flow on link i. We assume that the latency function li is convex, nondecreasing, and continuously
differentiable. The analysis can be extended to the case when the links are capacity-constrained, see
[5]. We also assume that li (0) = 0 for all i 5 . We denote the price per unit flow (bandwidth) of link
i by pi . Let p = [p1 , . . . , pI ] denote the vector of prices.
We are interested in the problem of routing d units of flow across the I links. We assume
that this is the aggregate flow of many “small" users and thus adopt the Wardrop’s principle in
characterizing the flow distribution in the network; i.e., the flows are routed along paths with
minimum effective cost, defined as the sum of the latency at the given flow and the price of that
path. We also assume that the users have a homogeneous reservation utility R and decide not to send
their flow if the effective cost exceeds the reservation utility.
More formally, for a given price vector p ≥ 0, a vector x W E ∈ RI+ is a Wardrop equilibrium
(WE) if

li (xiW E ) + pi = min{lj (xjW E ) + pj }, ∀ i with xiW E > 0, (3.11)

j
li (xiW
E
) + pi ≤ R, ∀ i with xiW E > 0,
xiW E ≤ d,
i∈I
1 2
with xiW E = d if minj lj (xjW E ) + pj < R. We denote the set of WE at a given p by W (p).6
i∈I
We next define the social problem and the social optimum, which is the routing (flow alloca-
tion) that would be chosen by a planner that has full information and full control over the network.
A flow vector x S is a social optimum if it is an optimal solution of the social problem
! "
max R − li (xi ) xi . (3.12)
x≥0
i∈I
i∈I xi ≤d

Hence, the social optimum is the flow allocation that maximizes the social surplus, i.e., the difference
between users’ willingness to pay and total latency. For two links, let x S be a social optimum with
xiS > 0 for i = 1, 2. Then it follows from first order optimality conditions that

l1 (x1S ) + x1S l1 (x1S ) = l2 (x2S ) + x2S l2 (x2S ). (3.13)

This implies that the prices xiS li (xiS ), i.e., the marginal congestion prices, can be used to enforce the
system optimum [cf. Eq. (3.11)].
5This assumption is a good approximation to communication networks where queueing delays are more substantial than propagation
delays. We will talk about the efficiency implications of relaxing this assumption in different models.
6 It is possible to account for additional constraints, such as capacity constraints on the links, by using a variational inequality
formulation (see [5], [36]).
 3.3. CONGESTION AND PROVIDER PRICE COMPETITION 91
For a given vector x ≥ 0, we define the value of the objective function in the social problem,

S(x) = (R − li (xi )) xi , (3.14)
i∈I

as the social surplus, i.e., the difference between users’ willingness to pay and the total latency.

3.3.3 MONOPOLY PRICING AND EQUILIBRIUM

We first assume that a monopolist service provider owns the I links and charges a price of pi per
unit bandwidth on link i. The monopolist sets the prices to maximize his profit given by

(p, x) = p i xi ,
i∈I

where x ∈ W (p). This defines a two-stage dynamic pricing-congestion game, where the monopolist
sets prices anticipating the demand of users, and given the prices (i.e., in each subgame), users choose
their flow vectors according to the WE. We define a vector (pME , x ME ) ≥ 0 to be a Monopoly
Equilibrium (ME) if x ME ∈ W (pME ) and

(pME , x ME ) ≥ (p, x), ∀ p ≥ 0, ∀ x ∈ W (p) .7

In [5], it was shown that price-setting by a monopolist internalizes the negative externality
and achieves efficiency. In particular, a vector x is the flow vector at an ME if and only if it is a
social optimum. This result was extended to a model that incorporates a general network topology
in [51]. This is a significant departure from the existing performance results of selfish routing in the
literature which assert that the efficiency losses with general latency functions can be arbitrarily bad.

3.3.4 OLIGOPOLY PRICING AND EQUILIBRIUM

We next assume that there are S service providers, denote the set of service providers by S , and
assume that each service provider s ∈ S owns a different subset Is of the links. Service provider s
charges a price pi per unit bandwidth on link i ∈ Is . Given the vector of prices of links owned by
other service providers, p−s = [pi ]i ∈/ Is , the profit of service provider s is

s (ps , p−s , x) = pi xi ,
i∈Is

for x ∈ W (ps , p−s ), where ps = [pi ]i∈Is .

The objective of each service provider, like the monopolist in the previous section, is to
maximize profits. Because their profits depend on the prices set by other service providers, each
service provider forms conjectures about the actions of other service providers, as well as the behavior
7 Our definition of the ME is stronger than the standard subgame perfect Nash equilibrium concept for dynamic games. In [5],
we show that the two solution concepts coincide for this game.
92 3. WIRELINE NETWORK GAMES
of users, which, we assume, they do according to the notion of (subgame perfect) Nash equilibrium.
We refer to the game among service providers as the price competition game. We  define OE
a vector

(p OE , x OE ) ≥ 0 to be a (pure strategy) Oligopoly Equilibrium (OE) if x OE ∈ W psOE , p−s and
for all s ∈ S ,

s (psOE , p−s
OE
, x OE ) ≥ s (ps , p−s
OE
, x), ∀ ps ≥ 0, ∀ x ∈ W (ps , p−s
OE
). (3.15)

We refer to p OE as the OE price.

Analysis of the optimality conditions for the oligopoly problem [cf. (3.15)] allows us to
characterize the OE prices (see [5]). In particular, let (pOE , x OE ) be an OE such that piOE xiOE > 0
for some i ∈ I . Then, for all s ∈ S and i ∈ Is ,
⎧ OE
⎪
⎪ x l (x OE ), if lj (xjOE ) = 0 ⎫ / Is ,
for some j ∈
⎨ i ⎧i i
piOE = ⎨ OE ⎬
j ∈ I s xj
⎪
⎪ min R − li (xiOE ) , xiOE li (xiOE ) + , otherwise.
⎩ ⎩ / Is l (x OE ) ⎭
j∈
1
j j

The preceding characterization implies that in the two link case with minimum effective cost
less than R, the OE prices satisfy

piOE = xiOE (l1 (x1OE ) + l2 (x2OE )) (3.16)

as claimed before. Intuitively, the price charged by an oligopolist consists of two terms: the first,
xiOE li (xiOE ), is equal to the marginal congestion price that a social planner would set [cf. Eq.
(3.13)] because the service provider internalizes the further congestion caused by additional traffic.
The second, xiOE lj (xjOE ), reflects the markup that each service provider can charge users because
of the negative congestion externality (as users leave its network, they increase congestion in the
competitor network).

3.3.5 EFFICIENCY ANALYSIS

We investigate the efficiency properties of price competition games that have pure strategy equilibria
8 . Given a price competition game with latency functions {l }
i i∈I , we define the efficiency metric at
some oligopoly equilibrium flow x OE as the ratio of the social surplus in the oligopoly equilibrium
to the surplus in the social optimum [cf. Eq. 3.14 for the definition of the social surplus], i.e., the
efficiency metric is given by
S(x OE )
rI ({li }, x OE ) = , (3.17)
S(x S )
where x S is a social optimum given the latency functions {li }i∈I and R is the reservation utility.
In other words, the efficiency metric is the ratio of the social surplus in an equilibrium relative to
the surplus in the social optimum. As in the previous sections, we are interested in the worst-case
8This set includes, but is substantially larger than, games with linear latency functions, see [6].
 3.3. CONGESTION AND PROVIDER PRICE COMPETITION 93
performance (or Price of Anarchy) of an oligopoly equilibrium, so we look for a lower bound on
rI ({li }, x OE ) over all price competition games and all oligopoly equilibria.
We next give an example of an I link network which has positive flows on all links at the OE
and an efficiency metric of 5/6.

Example 3.13 Consider an I link network where each link is owned by a different provider. Let
the total flow be d = 1 and the reservation utility be R = 1. The latency functions are given by
3
l1 (x) = 0, li (x) = (I − 1)x, i = 2, . . . , I.
2
The unique social optimum for this example
# is x S = [1, 0,$. . . , 0]. It can be seen that the flow
allocation at the unique OE is x OE = 23 , 3(I1−1) , . . . , 3(I1−1) . Hence, the efficiency metric for this
example is rI ({li }, x OE ) = 56 .

The next theorem establishes the main efficiency result.

Theorem 3.14 Consider a general parallel link network with I ≥ 2 links and S service providers, where
provider s owns a set of links Is ⊂ I . Then, for all price competition games with pure strategy OE flow
x OE , we have
5
rI ({li }, x OE ) ≥ ,
6
and the bound is tight.
The idea behind the proof of the theorem is to lower bound the infinite dimensional optimiza-
tion problem associated with the definition of the Price of Anarchy by a finite dimensional problem.
Then, one can use the special structure of parallel links to analytically solve the finite-dimensional
optimization problem. Further details can be found in [5].
A notable feature of Example 3.13 and this theorem is that the (tight) lower bound on
inefficiency is independent of the number of links I and how these links are distributed across
different oligopolists (i.e., of market structure). Thus, arbitrarily large networks can feature as much
inefficiency as small networks.9

3.3.6 EXTENSIONS
In this subsection, we extend the preceding analysis in two directions: First, we consider elastic traffic,
which models applications that are tolerant of delay and can take advantage of even the minimal
amounts of bandwidth (e.g., e-mail). We then focus on more general network topologies.
Elastic Traffic. To model elastic traffic, we assume that user preferences can be represented
! by an
"
increasing, concave, and twice continuously differentiable aggregate utility function u i∈I xi ,
9This result superficially contrasts with theorems in the economics literature that large oligopolistic markets approach competitive
behavior. These theorems do not consider arbitrary large markets but replicas of a given market structure.
94 3. WIRELINE NETWORK GAMES
which represents the amount of utility gained from sending a total amount of flow i∈I xi through
the network.
We assume that at a price vector, the amount of flow and the distribution of flow across the
links is given by the Wardrop equilibrium (cf. Def. 3.2). In particular, for a given price vector p ≥ 0,
a vector x ∗ ∈ RI+ is a Wardrop equilibrium if
! "
li (xi∗ ) + pi = u xj∗ , ∀ i with xi∗ > 0,
j ∈I
! "
li (xi∗ ) + pi ≥ u xj∗ , ∀ i ∈ I.
j ∈I

We define the social optimum and the efficiency ! metric

" as in Eqs. (3.12) and (3.17), replacing
R i∈I xi (i.e., users’ willingness to pay) by u i∈I xi .
It can be shown that for elastic traffic with a general concave utility function, the efficiency
metric can be arbitrarily close to 0 (see [82]). The two-stage game with multiple service providers
and elastic traffic with a single user class was first analyzed by Hayrapetyan, Tardos, and Wexler [49].
Using an additional assumption on the utility function (i.e., the utility function has a concave first
derivative), their analysis provides non-tight bounds on the efficiency loss10 . Using mathematical
tools similar to the analysis in [5], the recent work [82] provides a tight bound on the efficiency loss
of this game, as established in the following theorem.

Theorem 3.15 Consider a parallel link network with I ≥ 1 links, where each link is owned by a different
provider. Assume that the derivative of the utility function, u is a concave function. Then, for all price
competition games with elastic traffic and pure strategy OE flow x OE , we have

2
rI (u, {li }, x OE ) ≥ ,
3
and the bound is tight.

Parallel-Serial Topologies. Most communication networks cannot be represented by parallel link

topologies, however. A given source-destination pair will typically transmit through multiple inter-
connected subnetworks (or links), potentially operated by different service providers. Existing results
on the parallel-link topology do not address how the cooperation and competition between service
providers will impact on efficiency in such general networks.
Here, we take a step in this direction by considering the simplest network topology that al-
lows for serial interconnection of multiple links/subnetworks, which is the parallel-serial topology
(see Figure 3.7). It was shown in [6] that the efficiency losses resulting from competition are con-
siderably higher with this topology. When a particular provider charges a higher price, it creates
10 For example, they provide the non-tight bound of 1/5.064 in general, and the bound of 1/3.125 for the case when latency without
congestion is zero.
 3.3. CONGESTION AND PROVIDER PRICE COMPETITION 95
a negative externality on other providers along the same path, because this higher price reduces
the transmission that all the providers along this path receive. This is the equivalent of the double
marginalization problem in economic models with multiple monopolies and is the source of the
significant degradation in the efficiency performance of the network.

l j (x )1, p j

x1
x2
d units
Reservation utility : R
x3

Figure 3.7: A parallel-serial network topology.

In its most extreme form, the double marginalization problem leads to a type of “coordination
failure,” whereby all providers, expecting others to charge high prices, also charge prohibitively high
prices, effectively killing all data transmission on a given path. We may expect such a pathological
situation not to arise since firms should not coordinate on such an equilibrium (especially when
other equilibria exist). For this reason, we focus on a stronger concept of equilibrium introduced by
Harsanyi, the strict equilibrium. In strict OE, each service provider must play a strict best response to
the pricing strategies of other service providers. We also focus our attention on equilibria in which
all traffic is transmitted (otherwise, it can be shown that the double marginalization problem may
cause entirely shutting down transmission, resulting in arbitrarily low efficiency, see [6]).
The next theorem establishes the main efficiency result for this topology.

Theorem 3.16 Consider a general I ≥ 2 path network, with serial links on each path, where each link
is owned by a different provider. Then, for all price competition games with strict OE flow x OE , we have

1
rI (x OE ) ≥ ,
2
and the bound is tight.
Despite this positive result, it was shown in [6] that when the assumption li (0) = 0 is relaxed,
the efficiency loss of strict OE relative to the social optimum can be arbitrarily large. This suggests
that unregulated competition in general communication networks may have considerable costs in
terms of the efficiency of resource allocation and certain types of regulation may be necessary to
make sure that service provider competition does not lead to significant degradation of network
performance.
96 3. WIRELINE NETWORK GAMES
3.4 CONCLUDING REMARKS
This chapter has focused on basic models of network routing in which users are selfish. We have
studied the extent to which selfish behavior affects network performance. In addition to the basic
routing game which involves only the end-users themselves, we have addressed networking scenarios
in which network providers play an important role. In particular, we have considered communication
networks in which service providers engage in traffic engineering, as well as networks where the
service providers are profit maximizers.
We conclude this chapter by listing several venues that have not been covered herein, which
have been active research areas within the general framework of wireline networks games.
User heterogeneity and service differentiation. The models in this chapter assumed a single user-
class in the sense that users are indistinguishable regarding the performance they obtain while using
the network, and also regarding their preferences and tradeoffs (e.g., money vs. latency tradeoff ).
In practice, the user population is often heterogenous, and multi-class user models need to be
investigated. From the network side, user heterogeneity requires proper adjustments. A “one-service-
class per all" might fail short to accommodate the needs of diverse user types. Consequently, several
service classes with different characteristics often need to be offered. Service differentiation brings
in a clear need for offering incentives to users to encourage them to choose the service appropriate
for their needs, hence preventing over utilization of network resources. Pricing mechanisms provide
an efficient way to ensure QoS guarantees and regulate system usage. One of the key debates in
network pricing area is whether charges should be based on fixed access prices or usage-based prices.
While usage-based pricing has the potential to fulfill at least partially the role of a congestion control
mechanism, there were criticisms in view of the apparent disadvantages of billing overheads and the
resulting uncertainties in networking expenses (see [42]). A variety of pricing mechanisms have been
proposed over the last decade, ranging from simple price differentiation (e.g., Odlyzko Paris Metro
Pricing proposal [80]) to auction-based mechanisms and dynamic pricing schemes; see [38, 42] for
an overview of various pricing mechanisms. A related engineering issue of service differentiation
with a fixed number of service classes, is what kind of QoS guarantees can be offered and advertised,
see, e.g., [62] and references therein.
Models for investment and capacity upgrade decisions. Most of the work in this literature of rout-
ing games investigates the efficiency losses resulting from the allocation of users and information
flows across different paths or administrative domains in an already established network. Arguably,
the more important economic decisions in large-scale communication networks concern the in-
vestments in the structure of the network and in bandwidth capacity. In fact, the last 20 years have
witnessed significant investments in broadband, high-speed and optical networks. It is therefore of
great importance to model price and capacity competition between service providers and investigate
the efficiency properties of the resulting equilibria. See [2] and references therein for recent work in
the area.
Two-sided markets. The models considered in this section can be viewed as an abstraction of end-
users or consumers who route traffic over the Internet. However, the Internet which offers diverse
 3.4. CONCLUDING REMARKS 97
economic opportunities for different parties, involves other important “players", such as Content
Providers 11 (CPs). Consumers and Content providers (CP) base their choice of Internet Service
Providers (ISPs) not only on prices but also on other features such as speed of access, special add-ons
like spam blocking and virus protection. Therefore, the competition between service providers is not
only over price but also over quality. In addition, the pricing decisions become more involved, as
ISPs compete in prices for both CP’s and consumers. From an economic perspective, one can treat
these models as a competition between interconnected two-sided-market platforms in the presence
of quality choice; accordingly, game-theory is a natural paradigm for the analysis of the complex
interaction between the different parties. See [79] and references therein for recent work on the
subject.
Atomic players.The routing games considered in the section assume that the user population consists
of an infinitesimal number of “small" non-atomic users. Such modeling assumption might not be
appropriate for certain networking domains, in which each player may control a non-negligible
fraction of the total flow. Examples of such domains include logistic and freight companies that
transport goods between different points in the world to serve their clients [34].
In general, atomic routing games are harder to analyze when compared to non-atomic ones:
The underlying Nash equilibrium might not be unique (see [23, 81, 86]), and except for very special
cases, there is no potential function that can be used to characterize the equilibrium point. Atomic
network games thus incorporate many research challenges that still need to be explored.
Stochasticity and queuing effects.The use of latency functions as in the routing games considered in
this chapter, abstract away the stochastic elements in a communication network. A significant part of
the delay in a communication network link is due to the queueing overhead. It is therefore important
to incorporate the queueing effects into the selfish routing models. There has been extensive work
on decision making in queueing systems, see [48] for a survey book on the subject.
Additional efficiency measures. In this chapter, we have studied the Price of Anarchy (the worst-
case ratio between equilibrium and optimal performance) for different selfish routing models. Nat-
urally, there are other performance measures that can be considered. One such is the Price of Sta-
bility [90], which is the worst-case ratio between performance at the best equilibrium and the social
optimum. However, both these measure provide worst-case guarantees; one may rightly ask whether
networks leading to worst case performance are likely. In other words, worst-case analysis should
be supplemented with statistical analysis (e.g., average and variance of the efficiency loss). A related
direction is to rule out, or give less significance to equilibria that are less likely to be the outcome of
natural user dynamics, thereby replacing the static PoA measure with dynamic PoA analysis.

11 A content provider is defined as an organization or individual that creates information, educational or entertainment content for
the Internet.
 99

CHAPTER 4

Wireless Network Games

The emerging use of wireless technologies (such as WiFi and WiMAX) for data communication
has brought to focus novel system characteristics which are of less importance in wireline platforms.
Power control and the effect of mobility on network performance are good examples of topics which
are prominent in the wireless area. An additional distinctive feature of wireless communications is
the possible time variation in the channel quality between the sender and the receiver, an effect
known as channel fading [25].
Current wireless networks consist of a relatively large number of users with heterogeneous
Quality of Service (QoS) requirements (such as bandwidth, delay, and power). To reduce the man-
agement complexity, decentralized control of such networks is often to be preferred to centralized
one. This requirement leads to distributed (or at least partially distributed) network domains, in
which end-users take autonomous decisions regarding their network usage, based on their individ-
ual preferences. This framework is naturally formulated as a non-cooperative game, and has gained
much interest in recent literature (e.g., see [59] for a recent collection of papers on game theory in
communication systems).
In the context of wireless networks, self-interested user behavior can be harmful, as network
resources are often limited, and might be abused by a subset of greedy users. In many cases, an
individual user can momentarily improve its Quality of Service (QoS) metrics, such as delay and
throughput, by accessing the shared channel more frequently. Aggressiveness of even a single user
may lead to a chain reaction, resulting in possible throughput collapse.
The wireless domain is quite complex and incorporates many recent technological advances
and diverse modeling features. Naturally, we do not intend to cover all of those in this survey. We
do provide below a basic taxonomy for classification of the different aspects that define a wireless-
network game.
Network structure. A wireless network in its full generality can be described by a set of transmitters
and a set of receivers. Each transmitter may in principle be associated with one or more receivers.
A transmission to a particular receiver interferes not only with other transmissions to that receiver,
but it may also affect data reception at other receivers; see Figure 4.1 for an illustration.
In the bulk of this section, we will focus on the uplink network model, where there exists a
single receiver (or base station), with multiple nodes transmitting to it.
Multipath propagation and fading. Unlike wired communications that take place over a relatively
stable medium, the wireless transmission medium varies strongly with time. The signal that is
obtained at the receiver is, in fact, a superposition of multi-path components, which correspond
to different propagation paths. If there are phase shifts between the components (due to different
100 4. WIRELESS NETWORK GAMES

Figure 4.1: A multi-cell wireless network.The transmission power of each mobile affects the throughput
of all other mobiles, even if they transmit to different base-stations (interference is illustrated by dotted
lines). The higher the power of the mobile the higher its throughput, and the lower the throughput of
other mobiles.

interacting objects along the paths), the transmitted signal might not be properly decoded at the
receiver.This effect is known as (small-scale) fading. In addition, if there is no line of sight between the
transmitter and the receiver, the signal quality obviously diminishes, an effect known as shadowing
(or large-scale fading). This effect is often caused by the mobility of the wireless terminals. More
generally, the signal’s quality at the receiver is inversely proportional to the distance between the
transmitter and the receiver, which also varies due to mobility.
The above factors indicate that the channel quality between the transmitter and the receiver
is time varying. In this section, we will examine how wireless mobile use the information about the
channel quality (referred to as Channel State Information – CSI) in order to adjust their transmission
parameters. CSI can naturally be exploited for enhanced performance in a centralized setup; however,
the consequences of its use under self-interested behavior requires sophisticated game-theoretic
analysis.
Reception model. The throughput (i.e., the number of bits that are properly received at the receiver
per unit time) of a wireless devise which transmits to some base station depends on numerous factors.
First, it depends on the underlying domain (e.g., UMTS cellular system or a wireless LAN network).
Within each domain, the throughput could be a function of the transmission rates, the modulation
schemes, the packet sizes, the receiver technology, the multi-antenna processing algorithms, and
the carrier allocation strategies. In this section, we will not describe the effect of each of the above
elements on performance. Nevertheless, we provide below a basic classification of reception models.
1) Capture. As mentioned above, simultaneous transmissions of different mobiles interfere
with each other. Perhaps the most basic reception model is that of a collision channel, in which a
transmission can be successful only if no other user attempts transmission simultaneously. Thus, at
each time slot, at most one user can successfully transmit to the base station.
 101
Some wireless LAN receivers are able to decode a single transmission, even if other transmis-
sion took place at the same time. The so-called capture effect takes place when a single (the strongest)
user can be successfully received even in the presence of other simultaneous transmissions, provided
that its power dominates the others’ transmissions.
A broadly studied capture model is based on the signal to interference plus noise ratio (SINR)
(see, e.g., [106] and references therein). Consider an uplink network, and let Pi be user i’s transmission
power. Denote by hi the channel gain of user i. Then the SINR for user i is given by

hi Pi
SINR i (P) = . (4.1)
j  =i hj Pj + σ0

A transmission is successful if the SINR is large enough, namely if SINR i > β > 1 (where σ0 is the
ambient noise power).
2) Multi-packet reception. In some wireless systems, such as cellular networks, multiple simulta-
neous receptions are possible. In the broad sense, these systems rely on spread-spectrum techniques,
such as Code Division Multiple Access – CDMA [73]. The throughput of each user is, in general,
a function of its SINR (4.1). There are several different ways to model this function, depend-
ing on the specific underlying domain. For example, a well-studied model assumes that the user
throughput is a logarithmic, concave function of the SINR, namely ri (P) = log(1 + SINR i (P)).
This expression approximates the case where the transmitter can adjust its coding scheme to obtain
rates approaching the Shannon capacity. Another common model for the user throughput is given
by ri (P) = Ri f (SINR i (P)), where Ri is the transmission rate, and f (·) is the packet success proba-
bility, which is an increasing function of the SINR (usually sigmoid-shaped, see [68] and references
therein).
The choice of the reception model greatly affects the characteristics of the underlying non-
cooperative game between the mobile users. Our focus in this section will be on single-packet
reception, with emphasis on collision channels. A survey of noncooperative games in multi-packet
reception domains can be found in [68].
Preferences and utilities. As mentioned earlier, the choice of utilities in network games reflects
certain tradeoffs between QoS measures. In the context of wireless network, the salient measures
are throughput and power consumption. Accordingly, the most commonly studied user objectives
consist of combinations of these two measures:

• Maximize throughput subject to an average power constraint.

• Minimize the average power consumption subject to a minimum throughput constraint.

• Maximize the ratio between throughput and power consumption. The corresponding utility
in this case is referred to as the bit per joule utility.

Our focus in this section will be on the objective of minimizing power consumption subject to
throughput constraint. In often cases, a pricing term based on resource usage is incorporated into
102 4. WIRELESS NETWORK GAMES
the user utility, in order to regulate the network to the desired operating point. As we shall see in
this section, pricing is not always required for social efficiency as the objective of minimizing the
power consumption could be self-regulating on its own.
User control and protocol constraints. Depending on the wireless domain, mobile users may in
principle be able to self determine various parameters of their transmissions. These include, for
example, the timing of the transmission, the transmission rate, the modulation schemes, the packet
sizes, and the choice of frequency bands in multi-carrier systems.
If users are given the freedom to control all transmission parameters, then it is expected that
their self-interested behavior would lead to undesirable outcomes. Thus, it is often assumed that
users may self tune only a subset of the parameters. Moreover, in some cases, the user behavior is
regulated by an underlying Medium Access Control (MAC) protocol. A desirable protocol is one
which prevents the users from reaching bad equilibria yet still gives each user the right freedom to
adjust their transmission parameters based on their heterogenous preferences, thereby leading the
network to desirable operating points.
As mentioned above, our focus in this chapter will be on noncooperative mobile interaction
in an uplink collision channel. A basic assumption in our models is that the channel quality between
each user and the base-station is time-varying, and that each user can monitor its channel quality and
adjust its transmission parameters accordingly. The objective of each user is to minimize its energy
investment, subject to a minimal throughput constraint. The structure of this chapter is as follows.
In Section 4.1, we consider the scenario where each mobile decides on its transmission schedule as
a function of its underlying channel state. In Section 4.2, we assume that users can not only adjust
their transmission schedule but also control the power level that they employ for every realization
of the channel state. Section 4.3 briefly describes related work and highlights several interesting
extensions of the above mentioned models. Finally, we provide in Section 4.4 our view for important
future directions in the area of wireless network games.
The content in Sections 4.1– 4.2 relies mostly on [64, 65, 66, 67]. We do not provide herein
complete proofs for some of the analytical results. Those could be found in the above references.

4.1 NONCOOPERATIVE TRANSMISSION SCHEDULING IN

COLLISION CHANNELS
In this section, we consider a shared uplink in the form of a collision channel. A basic assumption
of our user model is that each user has some throughput requirement, which it wishes to sustain
with a minimal power investment. The required throughput of each user may be dictated by its
application (such as video or voice which may require fixed bandwidth), or mandated by the system.
A distinctive feature of our model is that the channel quality between each user and the base station
is stochastically varying. For example, the channel quality may evolve as a block fading process [25]
with a general underlying state distribution (such as Rayleigh, Rice, and Nakagami-m, see [25]).
A user may base its transmission decision upon available indications on the channel state, known
as channel state information (CSI). This decision is selfishly made by the individual without any
 4.1. NONCOOPERATIVE TRANSMISSION SCHEDULING IN COLLISION CHANNELS 103
coordination with other users, giving rise to a non-cooperative game. Our focus in this section is on
stationary transmission strategies, in which the decision whether to transmit or not can depend (only)
on the current CSI signal. Non-stationary strategies are naturally harder to analyze, and moreover,
their advantage over stationary strategies is not clear in large, distributed and selfish environments.
The technological relevance for our work lies, for example, in Wireless Local Area Network
(WLAN) systems where underlying network users have diverse (application-dependent) throughput
requirements. The leading standard, namely the 802.11x [1], employs a random access protocol,
whose principles are based on the original Aloha. Interestingly, recent IEEE standardization activity
(the 802.11n standard) focuses on the incorporation of CSI for better network utilization. This last
fact further motivates to study the use of CSI in distributed, self-optimizing user environments.
The main contributions of this section can be summarized as follows:
• Our equilibrium analysis reveals that when the throughput demands are within the network
capacity, there exist exactly two Nash equilibrium points in the resulting game.
• One equilibrium is strictly better than the other in terms of power investment for all users,
and, in fact, coincides with the socially optimal solution. The performance gap (in terms of
the total power investment) between the equilibrium points is potentially unbounded.
• We describe a fully distributed mechanism which converges to the better equilibrium point.
The suggested mechanism is natural in the sense that it relies on the user’s best response to
given network conditions.
The structure of the section is as follows. We first present the general model (Section 4.1.1),
and identify basic properties related to stationary transmission strategies. A detailed equilibrium
analysis is provided in Section 4.1.2. Section 4.1.3 focuses on the achievable network capacity. In
Section 4.1.4, we present a mechanism which converges to the better equilibrium. We discuss several
aspects of our results in Section 4.1.5 and highlight further research directions.

4.1.1 THE MODEL AND PRELIMINARIES

We consider a wireless network, shared by a finite set of mobile users I = {1, . . . , n} who transmit at
a fixed power level to a common base station over a shared collision channel. Time is slotted, so that
each transmission attempt takes place within slot boundaries that are common to all. A transmission
can be successful only if no other user attempts transmission simultaneously. Thus, at each time slot,
at most one user can successfully transmit to the base station. To further specify our model, we start
with a description of the channel between each user and the base station (Section 4.1.1.1), ignoring
the possibility of collisions. In Section 4.1.1.2, we formalize the user objective and formulate the
non-cooperative game which arises in a multi-user shared network.

4.1.1.1 The Single-User Channel

Our model for the channel between each user and the base station is characterized by two basic
quantities.
104 4. WIRELESS NETWORK GAMES
a. Channel state information. At the beginning of each time slot k, every user i obtains a channel
state information (CSI) signal ζi,k ∈ Zi ⊂ R+ , which provides an indication (possibly partial) of the
quality of the current channel between the user and the base station (a larger number corresponds
to a better channel quality). We assume that each set Zi of possible CSI signals for user i is finite1
and denote its elements by {zi1 , zi2 , . . . , zixi }, with zi1 < zi2 < · · · < zixi .
b. Expected data rate. We denote by Ri (zi ) > 0 the expected data rate (in bits per slot) that user i
can sustain at any given slot as a function of the current CSI signal zi ∈ Zi . We assume that the
function Ri (zi ) strictly increases in zi .
Throughout this chapter, we make the following assumption:
Assumption 1 (i) Zi = {ζi,k }∞ k=1 is an ergodic Markov chain; We denote by πi the row vector of steady
state probabilities of the Markov chain Zi , and by πim > 0 its m-th entry corresponding to state zim ∈ Zi
(signals with zero steady-state probability are excluded from the set Zi ). (ii) The Markov chains Zi ,
i = 1, . . . n, are independent.
Interpretation: The above model may be used to capture the following network scenario. The quality
(or state) of the channel between user i and the base station may vary over time. Let wi denote
an actual channel state for user i at the beginning of some slot (time indexes are omitted here for
simplicity). Instead of the exact channel state, user i observes a CSI signal zi , which is some (possibly
noisy) function of wi . As already noted, larger zi ’s indicate better channel conditions. After observing
the CSI at the beginning of a slot, user i may respond by adjusting its coding scheme in order to
maximize its data throughput on that slot. The expected data rate Ri (zi ) thus takes into account
the actual channel state distribution (conditioned on zi ), including possible variation within the slot
duration, as well as the coding scheme used by the user. Specifically, let R̃i (wi , zi ) be the expected
data rate for channel state wi , as determined by the coding scheme  that corresponds to zi . Then
the expected data rate is given by Ri (zi ) = E(R̃i (wi , zi )|zi ) = P̃i (wi |zi )R̃i (wi , zi )dwi , where E
is the expectation operator and P̃i (wi |zi ) is the conditional probability that the actual channel state
is wi when zi is observed. Assuming that the actual channel quality forms a Markov chain across
slots, this property is clearly inherited by the CSI sequence Zi .
Our modeling assumptions accommodate, in particular, the so-called block-fading model,
which is broadly studied in the literature (see [25, 84] and references therein). Note, however, that
our model does not require the i.i.d. assumption, nor does it require the actual channel state to be
fixed within each interval.

Example 4.1 Gaussian Channel Consider the case where the channel quality between user i
and the base station evolves as a discrete Markov process, with white Gaussian noise being added
to the transmitted signal. Specifically, at each time t, the received signal yi (t) is given by yi (t) =
√
wi (t)xi (t) + n(t), where xi (t) and wi (t) are, respectively, the transmitted signal and channel gain
(which is the physical interpretation for channel quality). Let T be the length of a slot. Then wi (t)
1This is assumed for convenience only. Note that the channel quality may still take continuous value, which the user reasonably
classifies into a finite number of information states.
 4.1. NONCOOPERATIVE TRANSMISSION SCHEDULING IN COLLISION CHANNELS 105
remains constant within slot boundaries, i.e., wi (t) ≡ wi,k , t ∈ [(k − 1)T , kT ), k ≥ 1. Suppose user
i is able to obtain the underlying channel quality with high precision, namely ζi,k ≈ wi,k . Let Si
be the maximal energy per slot, and let N0 /2 be the noise power spectral density. Then if the user
can optimize its coding scheme for rates approaching the Shannon capacity, the expected data rate
which can be reliably transmitted is given by the well known formula Ri (zi ) = Bi log(1 + NSi0 zi ),
where Bi is the bandwidth.

4.1.1.2 User Objective and Game Formulation

We turn now to describe the user objective and the non-cooperative game which arises as a conse-
quence of the user interaction over the collision channel.
Basic Definitions. We associate with each user i has a throughput demand ρi (in bits per slot)
which it2 wishes to deliver over the network. The objective of each user is to minimize its average
transmission power (which is equivalent in our model to the average rate of transmission attempts,
as users transmit at a fixed power level), while maintaining the effective data rate at (or above) this
user’s throughput demand. We further assume that users always have packets to send, yet they may
postpone transmission to a later slot to accommodate their required throughput with minimal power
investment.
A general transmission schedule, or strategy, σi for user i specifies a transmission decision at
each time instant, based on the available information that includes the CSI signals and (possibly) the
transmission history for that user. A transmission decision may include randomization (i.e., transmit
with some positive probability). Since this section focuses on stationary transmission strategies, we
will not bother with a formal definition of a general strategy. For our purpose, it suffices to assume
that the collection of user strategies (σi )i∈I together with the channel description, induce a well
defined stochastic process of user transmissions.
Obviously, each user’s strategy σi directly affects other users’ performance through the com-
monly shared medium. The basic assumption of our model is that users are self-optimizing and
are free to determine their own transmission schedule in order to fulfill their objective. We further
assume that users are unable to coordinate their respective decisions. This situation is modeled and
analyzed in this section as a non-cooperative game between the n users.
We denote by σ = (σ1 , . . . , σn ) the strategy-profile comprised of all users’ strategies. The
notation σ−i is used for the transmission strategies of all users but the i-th one. For each user i, let
pi (σ ) be the average transmission rate (or transmission probability), and let ri (σ ) be the expected
average throughput as determined by the user’s own strategy σi and by the strategies of all other
users σ−i . Further denote by ci,k the indicator random variable which equals one if user i transmits
at slot k and zero, otherwise, and by ri,k the number of data bits successfully transmitted by user i at

2The user here should be interpreted as the algorithm that manages the transmission schedules and is accordingly referred to in
the third person neuter.
106 4. WIRELESS NETWORK GAMES
the same slot. Then

1 
K

pi (σ ) = lim Eσ ci,k , (4.2)
K→∞ K
k=1
1 K

ri (σ ) = lim Eσ ri,k , (4.3)
K→∞ K
k=1

where Eσ stands for the expectation operator under the strategy-profile σ . If the limit in (4.2) does
not exist, we may take the lim sup instead, and similarly the lim inf in (4.3).
A Nash equilibrium (NE) is a strategy-profile σ = (σ1 , . . . , σn ), which is self-sustaining in
the sense that all throughput constraints are met, and no user can lower its transmission rate by
unilaterally modifying its transmission strategy. Formally,

Definition 4.2 Nash equilibrium point A strategy-profile σ = (σ1 , . . . , σn ) is a Nash equilibrium

point if
σi ∈ argmin {pi (σ̃i , σ−i ) : ri (σ̃i , σ−i ) ≥ ρi } . (4.4)
σ̃i

The transmission rate pi can be regarded as the cost which the user wishes to minimize. Using

game-theoretic terminology, a Nash equilibrium is a strategy-profile σ = (σ1 , . . . , σn ) so that each

σi is a best response of user i to σ−i , in the sense that the user’s cost is minimized.
Our focus in this section is on locally stationary transmission strategies, in which the user’s
decision whether to transmit or not can depend (only) on its current CSI signal (for simplicity, we
shall henceforth refer to such strategy just as stationary strategy, yet recall that local information
only is employed by each user). A formal definition for a stationary strategy is provided below.

Definition 4.3 Stationary strategies A stationary strategy for user i is a mapping σi :

Zi → [0, 1]. Equivalently, a stationary strategy will be represented by an xi -dimensional vector
si = (si1 , . . . , sixi ) ∈ [0, 1]xi , where the m-th entry corresponds to the user i’s transmission prob-
ability when the observed CSI signal is zim . For example, the vector (0, . . . , 0, 1) represents the

strategy of transmitting (w.p. 1) only when the CSI signal is the highest possible. Note that the
transmission probability in a slot, which is a function of si only, is given by


xi
pi (si ) = sim πim . (4.5)
m=1


Let s = (s1 , . . . , sn ) denote a stationary strategy-profile for all users. Evidently, the probability

that no user from the set I \i transmits in a given slot is given by j =i (1 − pj (sj )). Since the
 4.1. NONCOOPERATIVE TRANSMISSION SCHEDULING IN COLLISION CHANNELS 107
transmission decision of each user is independent of the decisions of other users, the expected
average rate ri (si , s−i ) is given by
 x 
 i  
ri (si , s−i ) = sim πim Ri (zim ) 1 − pj (sj ) , (4.6)
m=1 j  =i

where the expression xm=1

i
sim πim Ri (zim ) stands for the average rate which is obtained in a collision-
free environment under the same strategy si .

Remark 4.4 As noted above, we restrict attention here to stationary strategies. When the CSI
process is i.i.d., it may be shown that a Nash equilibrium in stationary strategies is, in fact, a Nash
equilibrium in general strategies. For more general state processes (e.g., Markovian), this need not
be the case, and the restriction to stationary strategies is upheld for simplicity of implementation.

Threshold Strategies. A subclass of stationary strategies which is central in our analysis is defined
below.

Definition 4.5 Threshold strategies A threshold strategy is a stationary strategy of the form
si = (0, 0, . . . , 0, simi , 1, 1 . . . , 1), simi
∈ (0, 1], where zimi is a threshold CSI level above which
user i always transmits, and below which it never transmits. An important observation, which we

summarize next, is that users should always prefer threshold strategies.

Lemma 4.6 Assume that all users access the channel using a stationary strategy. Then a best response
strategy of any user i is always a threshold strategy.

As a result of the above lemma, we may analyze the non-cooperative game by restricting the
strategies of each user i to the set of threshold strategies, denoted by Ti . We proceed by noting
that every threshold strategy can be identified with a unique scalar value pi ∈ [0, 1], which is the
transmission probability in every slot, i.e., pi ≡ pi (si ). More precisely:

Lemma 4.7 The mapping si = (0, 0, . . . , 0, simi , 1, 1 . . . , 1) ∈ Ti → pi ≡ pi (si ) ∈ [0, 1], is a sur-
jective (one-to-one and onto) mapping from the set of threshold strategies Ti to the interval [0, 1].

Proof. The claim follows directly from (4.5), upon recalling that m πim = 1 and πim > 0 by as-
sumption. Note first that under a threshold strategy si , pi (si ) = simi πimi + xm=m
i
π m . Evidently,
i +1 i
0 ≤ pi (si ) ≤ m πi = 1. Conversely, every pi ∈ [0, 1] corresponds to a unique threshold strategy
m

as follows: Given pi , the corresponding mi is such that xm=m i

π m < pi and xm=m
i +1 i
i
π m ≥ pi ;
i i
xi
pi − m
m=mi +1 πi
the transmission probability for the threshold CSI is given by simi = mi . 2
πi
108 4. WIRELESS NETWORK GAMES
Given this mapping, the stationary policy of each user will be henceforth represented through
a scalar pi ∈ [0, 1], which uniquely determines the CSI threshold and its associated transmission
probability, denoted by zimi (pi ) and simi (pi ), respectively. Consequently, the user’s expected through-
put per slot in a collision free environment, denoted by Hi , can be represented as a function of pi
only, namely

 
xi
Hi (pi ) = simi (pi )πimi (pi )Ri (zimi (pi )) + πim Ri (zim ), (4.7)
m=mi (pi )+1

where mi (pi ) denotes the index of the threshold CSI and πimi (pi ) denotes its probability. This
function will be referred to as the collision-free rate function. Using this function, we may obtain an
explicit expression for the user’s average throughput, as a function of p = (p1 , . . . , pn ), namely

ri (pi , p−i ) = Hi (pi ) (1 − pj ). (4.8)
j  =i

Example 4.8 Null CSI A special important case is when no CSI is available. This corresponds
to xi = 1 in our model. In this case, the collision-free rate function is simply Hi (pi ) = R̄i pi , where
R̄i = Ri (zi1 ) is the expected data rate that can be obtained in any given slot.

H i ( pi )
Ri ( zi1 )

Ri ( zi3 )

pi
1

Figure 4.2: An example of the collision-free rate function Hi (pi ). In this example, there are four CSI
signals. Note that the slope of Hi (pi ) is exactly the rate of the threshold CSI which corresponds to pi .

Some useful properties of the rate function (4.7) are summarized in the next lemma.

Lemma 4.9 The collision-free rate function Hi satisfies the following properties.
 4.1. NONCOOPERATIVE TRANSMISSION SCHEDULING IN COLLISION CHANNELS 109
(i) Hi (0) = 0.
(ii) Hi (pi ) is a continuous and strictly increasing function over pi ∈ [0, 1].
(iii) Hi (pi ) is concave.

Proof. Noting (4.5), pi = 0 means no transmission at all, thus an average rate of zero. It can be
easily seen that Hi (pi ) in (4.7) is a piecewise-linear (thus continuous), strictly increasing function.
As to concavity, note that the slope of Hi is determined by Ri (zimi ) which decreases with pi (see
Figure 4.2), as mi decreases in pi (from Eq. (4.7)) and zim is increasing in m (by definition). 2

4.1.2 EQUILIBRIUM ANALYSIS

In this section, we describe several basic properties of the Nash equilibria under stationary trans-
mission strategies. In Section 4.1.2.1, we define the equilibrium equations, which must be satisfied
at every feasible equilibrium point and are essential for our analysis. In Section 4.1.2.2, we charac-
terize the feasible region of throughput demands, which is the set of vectors ρ = (ρ1 , . . . , ρn ) for
which a Nash equilibrium point exists. We further show that there exist exactly two equilibrium
point for each ρ within the feasible region. We then prove that one equilibrium is better than the
other for all users in terms of the invested power (Section 4.1.2.3) and quantify the efficiency loss
incurred by selfish behavior of users (Section 4.1.2.4). We conclude this section by commenting on
computational aspects related to the efficient calculation of equilibria.

4.1.2.1 The Equilibrium Equations

A key property which is useful for the analysis is that every Nash equilibrium point can be repre-
sented via a set of n equations in the n variables p = (p1 , . . . , pn ). This is summarized in the next
proposition.

Proposition 4.10 The equilibrium equations A strategy-profile p = (p1 , . . . , pn ) is a Nash equi-

librium point if and only if it solves the following set of equations

ri (pi , p−i ) = Hi (pi ) (1 − pj ) = ρi , i ∈ I . (4.9)
j  =i

Proof. Adapting the Nash equilibrium definition (4.4) to stationary threshold strategies, a NE is a
strategy-profile p = (p1 , . . . , pn ) such that
1 2
pi = min p̃i ∈ [0, 1], subject to ri (p̃i , p−i ) ≥ ρi , i ∈ I , (4.10)
where ri is defined in (4.8). Since ri (p̃i , p−i ) is strictly increasing in p̃i (by Lemma 4.9), (4.10) is
equivalent to ri (pi , p−i ) = ρi , i ∈ I , which is just (4.9). 2
Due to the above result, we shall refer to the set of equations (4.9) as the equilibrium equations.
110 4. WIRELESS NETWORK GAMES
4.1.2.2 Two Equilibria or None
Obviously, if the overall throughput demands of the users are too high, there cannot be an equilibrium
point since the network naturally has limited traffic capacity (the capacity of the network will be
considered in Section 4.1.3).
Denote by ρ = (ρ1 , . . . , ρn ) the vector of throughput demands, and let  be the set of feasible
vectors ρ, for which there exists at least one Nash equilibrium point (equivalently, for which there
exists a feasible solution to (4.9)). Figure 4.3 illustrates the set of feasible throughput demands for
a simple two-user case, with Hi (pi ) = pi .

ρ2
(0,1)

(1/4,1/4)
Ω0

(1,0) ρ1

Figure 4.3: The set of feasible throughput demands for a two user network with Hi (pi ) = pi , i = 1, 2.

To specify some structural properties of , it is convenient to define the set of basis vectors , ˆ
ˆ
where each ρ̂ = (ρ̂1 , . . . , ρ̂n ) ∈  is such that ρ̂i > 0 for every i ∈ I and ρ̂2 = 1, i.e., i ρ̂i = 1.
2

Proposition 4.11 The feasible set  obeys the following properties.

ˆ α ρ̂ ∈  for all α ∈ [0, α(ρ̂)].
(i) Closed cone structure: For every ρ̂ ∈ ,
(ii) Let ρ ≤ ρ̃ be two throughput demand vectors. Then if ρ̃ ∈ , it follows that ρ ∈ .

In particular, note that  is a closed set with nonempty interior. We can now specify the num-
ber of equilibrium points for any throughput demand vector ρ = (ρ1 , . . . , ρn ). When throughput
demands are within the feasible region , we establish that there are exactly two Nash equilibria.

Theorem 4.12 Consider the non-cooperative game model under stationary transmission strategies. Let
 be the set of feasible throughput demand vectors ρ = (ρ1 , . . . , ρn ), and let 0 be its (non-empty)
interior. Then for each ρ ∈ 0 , there exist exactly two Nash equilibria.
 4.1. NONCOOPERATIVE TRANSMISSION SCHEDULING IN COLLISION CHANNELS 111
The idea of the proof is to reduce the equation set (4.9) to a single scalar equation in a
single variable pi , for some arbitrarily chosen user i. The right-hand side of the equation remains
ρi ; unimodality of the left-hand side will establish the required result; further details can be found
in [64, 66].

4.1.2.3 The Energy Efficient Equilibrium

Going beyond the basic questions of existence and number of equilibrium points, we wish to further
characterize the properties of the equilibrium points. In particular, we are interested here in the
following question: How do the two equilibrium points compare: is one “better" than the other? The
next theorem shows that, indeed, one equilibrium point is power-superior for all users.

Theorem 4.13 Assume that the throughput demand vector ρ is within the feasible region 0 , so that
there exist two equilibria in stationary strategies. Let p and p̃ be these two equilibrium points. If pi < p̃i
for some user i, then pj < p̃j for every j ∈ I .

 ρi
Proof. Define aik = ρk . For every user k  = i divide the i-th equation in the set (4.9) by the k-th
one. We obtain
Hi (pi )(1 − pk ) Hi (p̃i )(1 − pk )
aik = < , (4.11)
Hk (pk )(1 − pi ) Hk (pk )(1 − p̃i )
Hi (p̃i )(1−p̃k )
since Hi is increasing. Now since H k (p̃k )(1−p̃i )
= aik , it follows that (1− p̃k ) (1−pk )
Hk (p̃k ) < Hk (pk ) . Since Hk is
increasing in pk , we conclude from the last inequality that pk < p̃k . 2
The last result is significant from the network point of view. It motivates the design of a
network mechanism that will avoid the inferior equilibrium point, which is wasteful for all users.
This will be our main concern in Section 4.1.4. Henceforth, we identify the better equilibrium point
as the Energy Efficient Equilibrium (EEE).
We now turn to examine the quality of the EEE relative to an appropriate social cost. Re-
call that each user’s objective is to minimize its average transmission rate subject to a throughput
demand. Thus, a natural performance criterion for evaluating any strategy-profile s = (s1 , . . . , sn )
(in particular, an equilibrium strategy-profile) is given by the sum of the user’s average transmission
rates induced by s, namely 
Q(s) = pi (si ). (4.12)
i
The next theorem addresses the quality of the EEE with respect to that criterion.

Theorem 4.14 Let p be an EEE. Then i pi ≤ 1.

An immediate conclusion from the above theorem is that the overall power investment at
the EEE is bounded, as the sum of transmission probabilities is bounded. This means, in particular,
that the average transmission power of all users is bounded by the maximal transmission power of a
single station.
112 4. WIRELESS NETWORK GAMES
4.1.2.4 Social Optimality and Efficiency Loss
We proceed to examine the extent to which selfish behavior affects system performance. That it, we
are interested to compare the quality of the obtained equilibrium points to the centralized, system-
optimal solution (still restricted to stationary strategies). To that end, we shall use the notions of
price of anarchy (PoA) (see Chapter 3, Section 3.1.3) and price of stability (PoS). Recall that the
price of anarchy (PoA) is (an upper bound on) the performance ratio (in terms of a relevant social
performance measure) between the global optimum and the worst Nash equilibrium, while the price
of stability (PoS) is (an upper bound on) the performance ratio between the global optimum and the
best Nash equilibrium.
Returning to our specific network scenario, consider the case where a central authority, which is
equipped with user characteristics H = (H1 , . . . , Hn ) and ρ = (ρ1 , . . . , ρn ) can enforce a stationary
transmission strategy for every user i ∈ I . We consider (4.12) as the system-wide performance
criterion, and we compare the performance of this optimal solution to the performance at the Nash
equilibria. A socially optimal strategy-profile denoted s∗ (H, ρ), is a strategy that minimizes (4.12),
while obeying all user throughput demands ρi . Similarly, denote by sa (H, ρ) and sb (H, ρ) the
multi-strategies at the better NE and at the worse NE, respectively. Then the PoA and PoS are given
by
   
Q sb (H, ρ) Q sa (H, ρ)
P oA = sup  ∗  , P oS = sup 
∗
. (4.13)
H,ρ Q s (H, ρ) H,ρ Q s (H, ρ)

We next show that the PoA is generally unbounded, while the PoS is always one.

Theorem 4.15 Consider the non-cooperative game under stationary transmission strategies. Then (i) the
PoS is always one, and (ii) the PoA is generally unbounded.

Proof. (i) This claim follows immediately, noting that (1) the socially optimal stationary strategy
is a threshold strategy (by applying a similar argument to the one used in Lemma 4.6), and (2)
the socially optimal stationary strategy obeys the equilibrium equations (4.9) (following a similar
argument to the one used in Proposition 4.10). Hence, by Proposition 4.10, the optimal solution is
also an equilibrium point. Equivalently, this means that P oS = 1.
(ii) We establish that the price of anarchy is unbounded by means of an example. Consider a network
with n identical users with Hi (pi ) = R̄pi (this collision-free rate function corresponds to users who
cannot obtain any CSI). Each user’s throughput demand is ρi =  → 0. Recall that the throughput
demands are met with equality at every equilibrium point (Proposition 4.10). Then, by symmetry,
we obtain a single equilibrium equation, namely R̄p(1 − p)n−1 = . As  goes to zero, the two
equilibria are pa → 1 and pb → 0. Obviously, the latter point is also a social optimum; it is readily
seen that the price of anarchy here equals in the limit to ppab → ∞. 2
The above theorem clearly motivates the need for a mechanism that will induce the EEE, as
this equilibrium point coincides with the socially-optimal solution, while the gap between the two
equilibria could be arbitrarily large. Such mechanism is considered in Section 4.1.4.
 4.1. NONCOOPERATIVE TRANSMISSION SCHEDULING IN COLLISION CHANNELS 113
4.1.3 ACHIEVABLE CHANNEL CAPACITY
The aim of this section is to provide explicit lower bounds for the achievable channel capacity. The
term “capacity" is used here for the total throughput (normalized to successful transmission per slot),
which can be obtained in the network. We focus here on the case where users have no CSI, and then
relate our result to general CSI.
Consider the null-CSI model, where no user can observe any CSI (see the example at the end
of Section 4.1.1). Recall that the collision-free rate in this case is given by Hi (pi ) = R̄i pi , where

R̄i is the expected data rate in case of a successful transmission. Define yi = R̄ρi , which we identify
i
henceforth as the normalized throughput demand for user i; indeed, yi stands for the required rate
of successful transmissions. Then the equilibrium equations (4.9) become

pi (1 − pj ) = yi , 1 ≤ i ≤ n. (4.14)
j  =i

We shall first consider the symmetric case, i.e., yi = y for every user i, and then relate the results to
the general non-symmetric case. The theorem below establishes the conditions for the existence of
an equilibrium point in the symmetric null-CSI case.

Theorem 4.16 Symmetric users Let yi = y for every 1 ≤ i ≤ n. Then (i) A Nash equilibrium exists
if and only if
1
ny ≤ (1 − )n−1 . (4.15)
n
(ii) In particular, a Nash equilibrium exists if ny ≤ e−1 .

Proof. (i) By dividing the equilibrium equations (4.14) of any two users, it can be seen that every
symmetric-users equilibrium satisfies pi = pj = p (∀ i, j ). Thus, the equilibrium equations (4.14)
reduce to a single (scalar) equation:


h(p) = p(1 − p)n−1 = y. (4.16)

We next investigate the function h(p). Its derivative is given as h (p) = (1 − p)n−2 (1 − np). It can
be seen that the maximum value of the function h(p) is obtained at p = 1/n. An equilibrium exists
if and only if the maximal value of h(p) is greater than y. Substituting the maximizer p = 1/n in
(4.16) implies the required result.
(ii) It may be easily verified that the right-hand side of (4.15) decreases with n. Since limn→∞ (1 −
1 n−1
n) = e−1 , the claim follows from (i). 2
In [66], it is shown that the simple bound obtained above holds for non-symmetric users as
well, implying that the symmetric case is the worst in terms feasible channel utilization. Formally,
114 4. WIRELESS NETWORK GAMES
Theorem 4.17 Asymmetric users For any set of n null-CSI users with normalized throughput
demands {yi }, an equilibrium point exists if


n
1
yi ≤ (1 − )n−1 . (4.17)
n
i=1

The quantity e−1 is also the well-known maximal throughput of a slotted Aloha system with
Poisson arrivals and an infinite set of nodes [19]. In our context, if the normalized throughput
demands do not exceed e−1 , an equilibrium point is guaranteed to exist. Thus, in a sense, we may
conclude that noncooperation of users, as well as restricting users to stationary strategies, do not
reduce the capacity of the collision channel.
We conclude this section by noting that Eq. (4.17) serves as a global sufficient condition
for the existence of an equilibrium point, which holds for any level of channel observability. This
observation follows by showing that the capacity can only increase when users obtain channel state
information (see [66] for details).

4.1.4 BEST-RESPONSE DYNAMICS

A Nash equilibrium point for our system represents a strategically stable working point, from which
no user has incentive to deviate unilaterally. Still, the question of if and how the system arrives at
an equilibrium remains open. Furthermore, since our system has two Nash equilibria with one (the
EEE) strictly better than the other, it is of major importance (from the system viewpoint, as well as
for each individual user) to employ mechanisms that converge to the better equilibrium rather than
the worse.
The distributed mechanism we consider here relies on a user’s best-response (BR), which is
generally the optimal user reaction to a given network condition (see [46]). Specifically, the best
response of a given user is a transmission probability which brings the obtained throughput of that
user (given other user strategies) to its throughput demand ρi . Accordingly, observing (4.8), the best
response of user i for any strategy-profile p = (p1 , . . . , pn ) is given by
 
−1 ρi
pi := Hi  , (4.18)
j  =i (1 − pj )

where Hi−1 is the inverse function of the collision-free rate function Hi (if the argument of Hi−1
is larger than maximal value of Hi , pi can be chosen at random). Note that Hi−1 is well defined,
since Hi is continuous and monotone (Lemma 4.9). It is important to notice that each user is not
required to be aware of the transmission probability of every other user. Indeed, only the overall idle

probability of other users j =i (1 − pj ) is required in (4.18). Our mechanism can be described as
follows. Each user updates its transmission probability from time to time through its best response (4.18).
The update times of each user need not be coordinated with other users.
 4.1. NONCOOPERATIVE TRANSMISSION SCHEDULING IN COLLISION CHANNELS 115
This mechanism reflects what greedy, self-interested users would naturally do: repeatedly
observe the current network situation and react to bring their costs to a minimum.
The analysis of the best-response mechanism will be carried out under the following assump-
tions that apply throughout this section.

Assumption 2

(i) Fixed Demands: The user population and the users’ throughput requirements ρ1 , . . . , ρn are fixed.
Furthermore, ρ1 , . . . , ρn are within the feasible throughput region.

(ii) Persistent Updates: Each user updates its transmission probabilities using Eq. (4.18) at arbitrarily
chosen time instants, and the number of updates is unbounded.

Furthermore, in order to guarantee convergence to the EEE (denoted henceforth as pa ), we must

impose restrictions on the initial strategy profile of the users, denoted p0 . Such restrictions are clearly
essential due to the inherent non-uniqueness of the equilibrium in our model; indeed, if we start at
the worse equilibrium point (denoted henceforth as pb ), we will stay there indefinitely under BR.
For initial conditions above pb , we might observe throughput collapse, as some users increase their
transmission probabilities to their maximal values of 1.
To specify the required condition, let us define the following sets of strategy profiles:
1
1 = p ∈ [0, 1]n : ri (p) ≥ ρi for every i, and
2
pj < pjb for some j ;
1 2
0 = p ∈ [0, 1]n : p ≤ p̄ for some p̄ ∈ 1 .

Our main convergence result is summarized below.

Theorem 4.18 BR Convergence Let the initial request probabilities p0 = (p10 . . . , pn0 ) satisfy p0 ∈
0 . Then the best response dynamics asymptotically converges to the better equilibrium point pa . The

proof proceeds by applying a “sandwich" argument. We first show that BR converges monotonously
(from below) to pa when started at p0 = 0. We then show that it converges monotonously (from
above) to pa when started with p0 in 1 . Finally, we conclude by monotonicity of the BR that
convergence must occur all for initial conditions between 0 and 1 , namely for all p0 ∈ 0 . Details
can be found in [67].
We briefly list here some considerations regarding the presented mechanism.

1. It is important to notice that each user is not required to be aware of the transmission probability

of every other user. Indeed, only the overall idle probability of other users j =i (1 − pj ) is
required in (4.18). This quantity could be estimated by each user by monitoring the channel
utilization.
116 4. WIRELESS NETWORK GAMES
2. The update-rule (4.18) entails the notion of a quasi-static system, in which each user responses
to the steady state reached after preceding user updates.This assumption approximates a natural
scenario where users update their transmission probabilities at much slower time-scales than
their respective transmission rates. A recent paper [92] demonstrates that the quasi-static
assumption can be relaxed while still ensuring convergence of user dynamics, by employing
stochastic approximation tools.

In [66], we show that the appealing convergence properties continue to hold when the user
population changes over time. We notice that the convergence results obtained in the section would
still hold for a relaxed variation of (4.18), given by
 
−1 ρi
pi := βi Hi  + (1 − βi )pi , (4.19)
j  =i (1 − pj )

where 0 ≤ βi ≤ 1. This update rule can be more robust against inaccuracies in the estimation of

j =i (1 − pj ) , perhaps at the expense of slower convergence to the desired equilibrium.
Our convergence results are obviously idealized and should be supplemented with further
analysis of the effect of possible deviations from the model and possible remedies. In case that a
worst equilibrium point is reached (or no equilibrium is obtained after a reasonably long time),
users can reset their probabilities and restart the mechanism (4.18) for converging to the better
equilibrium. This procedure resembles the basic ideas behind TCP protocols. The exact schemes for
detecting operation at suboptimal equilibria, and consequently directing the network to the EEE,
are beyond the scope of the present research.

4.1.5 DISCUSSION
We briefly discuss here some consequences of our results, emphasizing network management aspects,
and highlight some interesting future research directions. Our equilibrium analysis has revealed that
within the feasible region the system has two Nash equilibrium points with one strictly better than
the other. The better equilibrium (the EEE) is socially optimal, hence the network should ensure
that users indeed operate at that equilibrium. An important step in this direction is the above
suggested distributed mechanism which converges to the EEE. It should be mentioned, however,
that fluctuations in the actual system might clearly bring the network to an undesired equilibrium.
Hence, centralized management (based on user feedbacks) may still be required to identify the
possible occurrence of the worse equilibrium, and then direct the network to the EEE. Possible
mechanisms for this purpose remain a research direction for the future.
In this section, we mainly considered the throughput demands ρi as determined by the user
itself. Alternatively, ρi may be interpreted as a bound on the allowed throughput which is imposed by
the network (as part of a resource allocation procedure).The advantage of operating in this “allocated-
rate" mode is twofold. First, the network can ensure that user demands do not exceed the network
capacity (e.g., by restricting the allocated rate, or through call admission control). Second, users
 4.2. NONCOOPERATIVE POWER CONTROL IN COLLISION CHANNELS 117
can autonomously reach an efficient working point without network involvement, as management
overhead is reduced to setting the user rates only. The rate allocation phase (e.g., through service
level agreements) is beyond the scope of the present model.
Another important comment relates to elastic users that may lower their throughput demand
based on a throughput–power tradeoff. An obvious effect of demand elasticity would be to lower the
throughput at inefficient equilibria. It remains to be verified whether other properties established
here remain valid in this case.
The framework and results of this section may be extended in several ways. An interesting
research extension is to consider non-stationary user strategies. A central question is whether the
system benefits from the use of more complex policies by selfish individuals. The incorporation of
non-stationary strategies seems to add considerable difficulty to the analysis and may require more
elaborate game theoretic tools than the ones used here.

4.2 NONCOOPERATIVE POWER CONTROL IN COLLISION

CHANNELS
This section considers the combined power-control and transmission scheduling problem in a time-
slotted collision channel. At every time slot, each user may observe its own channel quality and
decide whether to transmit or not, and if so at which power level (chosen from a discrete set). Our
focus in this section is on local stationary transmission strategies, in which this decision can depend
(only) on the current channel state of the mobile. As in Section 4.1, we assume that each mobile
user has some throughput requirement, which it wishes to sustain with a minimal power investment.
Users cannot coordinate their transmissions, and adjust their transmission decisions to minimize
their power investment based on network conditions. This situation is modeled and analyzed as a
noncooperative game between the mobiles. The basic objective of this section is to obtain structural
properties for the associated Nash equilibria and investigate the effects of the physical parameters
(e.g., the channel state distribution, the available power levels) on equilibrium performance.
The main findings of this section are the following:
• In contrast to Section 4.1, there are possibly more than two equilibria for the underlying game.
Importantly, the equilibria are completely ordered in terms of the per-user power investment;
hence, as before there is one equilibrium which is best for all. This equilibrium can be reached
through best response dynamics that requires minimal information structure for each mobile.
• On the negative side, the power-efficient equilibrium is usually inferior to the centrally assigned
power schedule. In addition, we demonstrate that the freedom given to users in the form of
multiple power levels might have negative effects on network performance, in terms of both
channel capacity and overall power consumption.These properties could be regarded as Braess-
like paradoxes (see Chapter 3, Section 3.1.3) in wireless networks.
The structure of the section is as follows. We first present the general model (Section 4.2.1)
and define the Nash equilibrium point of the noncooperative game. Structural results of the Nash
118 4. WIRELESS NETWORK GAMES
equilibria are derived in Section 4.2.2. In addition, we show that if the required rates are feasible, there
exists an equilibrium point which is uniformly best for all users in terms of the power investment.
Accordingly, we suggest in Section 4.2.3 a simple distributed mechanism that converges to that
equilibrium. In Section 4.2.4, we study the efficiency loss incurred by selfish user behavior and
identify a couple of Braess-like paradoxes. Section 4.2.5 discusses some consequences of our results,
and outlines further research.

4.2.1 THE MODEL

We consider a time-slotted wireless network, shared by a finite set of mobile users I = {1, . . . , n}
who transmit to a common base station over a shared collision channel. A finite set of power
levels3 Qi = {Q0i , Q1i , . . . , QJi i } is available to each mobile i, where 0 = Q0i < Q1i , · · · < QJi i . A
transmission at any (positive) power level is successful only if no other user attempts transmission
simultaneously.
As in Section 4.1, we first describe the channel characteristics between each user and the base
station (Section 4.2.1.1), ignoring the possibility of collisions. In Section 4.2.1.2, we formalize the
user objective and formulate the non-cooperative power control game between the users, and the
associated Nash equilibrium.

4.2.1.1 The Single-User Channel

Our model for the channel between each user and the base station is characterized by two basic
quantities.
a. Channel State. We use an identical channel state model as in Section 4.1.1.1. For simplicity, we
shall often use the term “channel state" when actually referring to the associated CSI signal, yet we
recall that partial information only may be available to the user. For completeness, we briefly repeat
the basic modeling assumptions used in Section 4.1 with regard to the channel state process.
We assume that the channel state (or quality) between mobile i and the base station evolves
as an ergodic Markov chain Zi , taking values in a finite set Zi = (zi1 , zi2 , . . . , zixi ) of xi states. For
convenience, we shall assume that the states are ordered from worst (zi1 ) to best (zixi ) and denote this
relation by zi1 < zi2 < . . . < zixi . The Markov chains Zi , i = 1 . . . n, are assumed to be independent.
We denote by πi the row vector of steady state probabilities of the Markov chain Zi , and by πim its
m-th entry corresponding to state zim ∈ Zi .
m,j
b. Expected data rate. Let Ri ≥ 0 denote the expected data rate (in bits per second) that user i can
j
sustain at a given slot as a function of the current channel state zim and the power level Qi assigned
for the transmission. We assume that the data rate strictly increases with the channel quality, and

3 A continuum of power levels can be treated as well, and in fact turns out to be analytically simpler. Here we choose to focus on
the finite case which is more realistic.
 4.2. NONCOOPERATIVE POWER CONTROL IN COLLISION CHANNELS 119
further that it strictly increases with the transmission power. That is,
1,j 2,j x ,j
Ri < Ri < · · · < Ri i , j ∈ {1, . . . Ji } (4.20)
Rim,1 < Rim,2 < · · · < Rim,Ji , m ∈ {1, . . . xi }; (4.21)

naturally, Rim,0 = 0, since Q0i = 0 by definition.

Example 4.19 Assume that white Gaussian noise with a spectral density of N0 /2 is added to the
transmitted signal. Then if a user can optimize its coding scheme to approach the Shannon capacity,
the average rate that can be sustained is given by the following rate function
m,j j
Ri = log(1 + zim Qi /N0 ). (4.22)
We shall consider this specific rate function in Section 4.2.4.

4.2.1.2 User Objective and Game Formulation

In this subsection, we describe the user objective and the non-cooperative game which arises as a
consequence of the user interaction over the collision channel. We characterize stationary transmis-
sion strategies, which are central in this section, and then define the Nash equilibrium of the game
within this class of strategies.
As in Section 4.1, we associate with each user i a throughput demand ρi (in bits per second)
which it wishes to deliver over the network. The objective of each user is to minimize its average
transmission power while maintaining the effective data rate at (or above) this user’s throughput
demand. We further assume that users always have packets to send, yet they may delay transmission
to a later slot to accommodate their required throughput with minimal power investment.
As in the previous section, our focus is on local and stationary transmission strategies, in which
the transmission power decision (which includes the decision not to transmit at all) can depend only
on the current state of the mobile zim . The user does not have any information regarding the channel
state of other users. For any given channel state, the mobile decision may include randomization
over the available powers. A formal definition is provided below.

Definition 4.20 Locally stationary strategy A locally stationary strategy for user i is represented
m,j
by an xi × (Ji + 1) matrix qi , where its (m, j ) entry, denoted qi , corresponds to the probability
j
that user i will transmit at power Qi when the observed channel state is zim . As such, the set of
feasible locally stationary strategies is given by
⎧ ⎫
⎨ Ji ⎬
m,j m,j
i = qi ≥ 0, qi = 1 ∀m = 1, . . . xi .
⎩ ⎭
j =0

For simplicity, we shall refer to the above defined strategy as stationary strategy. We also define two

user-specific quantities that are derived from a given stationary strategy qi .

120 4. WIRELESS NETWORK GAMES
• The transmission probability pi (qi ) in a slot, given by
⎛ ⎞
 xi Ji
πim ⎝ qi ⎠ .
m,j
pi (qi ) = (4.23)
m=1 j =1

• The collision-free data-rate Hi (qi ), which stands for the average data-rate of successful trans-
missions, namely ⎛ ⎞

xi  Ji
πim ⎝ qi Ri ⎠ .
m,j m,j
Hi (qi ) = (4.24)
m=1 j =1


A strategy-profile, namely a collection of (stationary) user strategies, is denoted by q = (q1 , . . . , qn ).
The notation q−i will be used for the transmission strategies of all users but for the i-th one. Note that

the probability that no user from the set I \{i} transmits in a given slot is given by l=i (1 − pl (ql )).
Since the transmission decision of each user is independent of the decisions of other users, the
expected data rate of user i, denoted ri (q), is given by

ri (q) = Hi (qi ) (1 − pl (ql )). (4.25)
l =i

A Nash equilibrium point (NE) for our model is defined below.

Definition 4.21 Nash equilibrium point A strategy-profile q = (q1 , . . . , qi ) is a Nash equilibrium

point if for every i ∈ I ,
⎧ ⎫
⎨xi
 
Ji
 ⎬
m,j j
qi ∈ argmin πim q̃i Qi : ri (q̃i , q−i ) ≥ ρi . (4.26)
q̃i ∈i ⎩ m=1 j =1
⎭

Noting (4.25), it is important to emphasize that the only interaction between users is through
the effective collision rate over the shared channel. This observation is significant in the context of
the game dynamics, as the only external information that is required for best response adaptation is
not user-specific and can be relatively easily measured.

4.2.2 EQUILIBRIUM ANALYSIS

In this section, we characterize the Nash equilibria (4.26) of the network under stationary trans-
mission strategies. Our analysis starts by examining the best-response strategy of each mobile. In
particular, we demonstrate that classic “water-filling" properties for optimal power control carry over
to the noncooperative-game framework. We then show that one of the possible equilibrium points
is best for all users in terms of their power investment.
 4.2. NONCOOPERATIVE POWER CONTROL IN COLLISION CHANNELS 121
4.2.2.1 Basic Properties
This subsection provides some basic properties of the best-response strategy. Our first result states
that the throughput demands should be met with equality at every equilibrium point.

Lemma 4.22 Let q = (q1 , . . . , qn ) be a Nash equilibrium point (4.26). Then

ri (q) = ρi . (4.27)

Proof. The result immediately follows from (4.24)–(4.26) by noting that both the user rate and
m,j
average power consumption are monotonously increasing functions of qi , j ∈ {1, . . . Ji }, m ∈
{1, . . . xi }. Indeed, if the current rate is strictly higher than required, each user may unilaterally
m,j
reduce its power investment (and still hold to its requirement) by slightly decreasing some qi > 0,
j ∈ {1, . . . Ji }. 2
Noting (4.25), we observe that the best-response strategy can be analyzed for each user
in isolation, as the overall effect of other users is manifested only through a multiplicative term
which modulates the effective rate. Indeed, Lemma 4.22 indicates that the user’s best response
ρi
must obey Hi (qi ) = ρ̄i , where ρ̄i =  (1−p (q ))
. Consequently, we shall henceforth use the latter
l=i l l
transformation for the best-response characterization.
Consider the (discrete) rate function obtained for a given state zim , and let fim (Qi ) be its
continuous linear interpolation. Let gim (Qi ) be the lower concave hull of fim (Qi ), i.e, the smallest
concave function such that gim (Qi ) ≥ fim (Qi ) (see Figure 4.4). We next assert that power levels
that are not on the lower concave hull gim (Qi ) would not be used in any best-response strategy.
j
Lemma 4.23 Consider any channel state zim . Let Qi be a power level whose associated rate is below
m,j
gim (Qi ). Then qi = 0 under every best-response strategy.

As a consequence, of the above lemma, for any given channel state zim , the interpolation of
power levels that can be used (with the associated rates) creates a piecewise-linear concave function.
Due to concavity, the next result immediately follows.

Lemma 4.24 Let qi be a best-response strategy. Then for every state zim , there are at most two non-zero
m,j j
elements qi and qim,k . Moreover, these elements correspond to adjacent power-levels Qi and Qki on the
lower concave hull gim .

The significance of Lemma 4.24 is that the best-response for each state can be represented
by a point on the concave hull graph.
Until now, we have focused on the optimal strategy within each channel state. We next
provide a characterization of the best-response across the different states. To that end, we require
the following definition.
122 4. WIRELESS NETWORK GAMES

lower concave hull

rate

Qi3 QiJ i Qi

Figure 4.4: The data rate as a function of the transmission power for some fixed state zim . Observe that
a power level which obtains a rate which is not on the lower concave hull of the interpolation (Q3i in
the figure) would not be used: A convex combination of Q2i and Q4i can lead to a better rate at the same
power cost.

j j
Definition 4.25 rate-gain Let Qi and Qki (Qi < Qki ) be two adjacent power-levels on the lower

concave hull of gim . The rate-gain (under state zim ) for these two power levels is defined as σim,k =
m,j
Rim,k −Ri
j . The next result states that higher rate-gain power levels should always be preferred. This
Qki −Qi

property is central in the characterization of the best-response, and it would have implications on
the efficient calculation of the best-response, as well as on the equilibrium structure.
m,j
Lemma 4.26 Consider two channel states zim and zim . A power allocation with qi < 1 (j ≥ 1) and
m,j
qim ,k > 0 (k ≥ 1) such that σi > σim ,k is always suboptimal.

m,j
Proof. (outline) The idea of the proof is to raise the transmission probability qi and decrease qim ,k
while preserving the same data rate. Such strategy would lower the average power investment, and
the result will follow. Further details can be found in [65]. 2

4.2.2.2 Equilibrium Structure

Lemma 4.26 leads to several significant properties regarding the structure of the equilibrium. As
expected, better channel states would be always preferred. In addition, there always exists a best-
response strategy with a single randomization of power levels. These properties are summarized in
the next proposition.
 4.2. NONCOOPERATIVE POWER CONTROL IN COLLISION CHANNELS 123
Proposition 4.27 The following properties are valid for every best-response strategy: (i) There exists
m,j
some state zim in which the user transmits with positive probability (i.e., 0 < Jj i=1 qi ≤ 1), and further
Ji k,j k,j
j =1 qi = 1 for k > m and Jj i=1 qi = 0 for k < m. (ii) There exists a best response with a single
randomization; that is, for every channel state m but one, there exists some power level in Qi which is used
m,j
w.p. 1, i.e., qi = 1.

Proof. (i) Let zim be the lowest channel quality at which the user transmits with positive probability.
Ji k,j jk
Then by construction, j =1 qi = 0 for k < m. For every k > m, let Qi be the smallest power
k,j k m,j
level that is on the lower concave hull gik . It can be easily seen that σi > σi for every Qj that is
on the lower concave hull gim . This property essentially follows from (4.21). Assume by contradiction
k,j k,j k
that Jj i=1 qi < 1; this means that qi < 1. It then follows by Lemma 4.26 that such strategy
cannot be optimal.
(ii) In view of Lemma 4.26, a best-response with more than a single randomization is possible if
and only if two (or more) rate-gains, corresponding to active power levels of two (or more) different
channel states, are the same. In that case, the same interchange argument used in the proof of
Lemma 4.26 can be applied for constructing an equal power strategy that eliminates one of the
randomizations. Repeating such procedure will result in a single randomization. 2

Remark 4.28 Classic results on “water-filling" optimization consider the specific rate function
(4.22). For this function, it can be shown that higher powers are used for better channel states.
Under our general model assumptions this is not necessarily true. Generally, an additional property
of “increasing differences" is required for monotonicity in the power levels as described above. In
our case, the increasing differences property holds if and only if
m ,j m,j m ,j m,j
Ri − Ri > Ri − Ri (4.28)

for every two indices pairs m > m and j > j .

Based on Lemma 4.26, the following iterative procedure can be carried out to efficiently
calculate the best-response:

1. Arrange the rate-gains in decreasing order.

m,j
Rim,k −Ri m,j
2. Starting with the highest rate-gain, say σim,k = j , set qim,k := 1, qi := 0, j  = k.
Qki −Qi
m,j
Calculate the average rate via (4.24). If the total rate exceeds ρ̄i , find qim,k < 1, qi > 0 such
that required rate is met with equality.

3. Otherwise, raise the rate by examining the next highest rate-gain and setting the associated
probabilities as in Step 2. Repeat this step until obtaining the required data rate ρ̄i .
124 4. WIRELESS NETWORK GAMES

rate

Better state

QiJ i Qi

Figure 4.5: A two-state channel with two power levels available. This graph depicts the rate as a function
of the used power for both states. The procedure for optimizing the power allocation starts with using
the lower power level at the better state. If the obtained rate is not satisfactory, note that in this case
the preceding choice is to transmit with the lower power level also at the worse state. This example
demonstrates a scenario where users might transmit frequently, resulting in frequent collisions.

It immediately follows from Lemma 4.26 and Proposition 4.27 that step 3 above proceeds
by either augmenting the power-level for a channel state which is at use, or initiating transmissions
for a new channel state (with lower quality than the ones used so far). See Figure 4.5 for a graphi-
cal interpretation. The above procedure guarantees convergence to the best-response strategy; this
property easily follows from Lemma 4.26.

4.2.2.3 Energy Efficient Equilibrium

The previous subsections established certain properties that are common to all equilibrium points.
The aim of this section is to compare the possible equilibria in terms of power investment. We shall
establish that the equilibrium point are ordered with respect to the individual users’ power investment
uniformly across all users. That is, if some user spends more power in one equilibrium than in the
other, so do other users. This property immediately implies that if there exist several equilibrium
points, one of them is best for all users in terms of the power investment. As before, we shall refer
to this equilibrium as the energy efficient equilibrium (EEE).
We establish below that equilibria are ordered (component-wise) in terms of the power in-
vestment. For this result, the following lemma is required.
 4.2. NONCOOPERATIVE POWER CONTROL IN COLLISION CHANNELS 125
Lemma 4.29 Assume that user i is a unique network user. Consider two different rate demands ρi
and ρ̃i for that user so that ρi < ρ̃i . Let qi and q̃i be best-responses for ρi and ρ̃i , respectively. Then
pi (qi ) ≤ pi (q̃i ).

The above lemma immediately leads to the next theorem.

Theorem 4.30 Let q and q̃ be two equilibria so that

Hi (qi ) < Hi (q̃i )

for some user i. Then Hl (ql ) < Hl (q̃l ) for every other user l  = i. Consequently, the power investments
that correspond to q̃ are strictly higher for all users, compared to the power investments under q.

Proof. Recall from Lemma 4.22 that

ri (q) = ρi (4.29)

for every equilibrium point q. Noting (4.25), dividing the equations (4.29) of any two users i and l
H (q )(1−p (q ))
results in the following relation Hil (qli )(1−pil (qil )) = ρρil , or
! "
Hi (qi )
1−pi (qi ) ρi
! " = . (4.30)
Hl (ql ) ρl
1−pl (ql )

If Hi (qi ) < Hi (q̃i ), it follows by Lemma 4.29 that pi (qi ) ≤ pi (q̃i ). Hence, in order to keep the
fixed ratio for q̃, it follows that Hl (ql ) < Hl (q̃l ). It remains to verify that the power investment at
q̃ is strictly higher compared to the equilibrium q. Evidently, if this was not the case, users can use
m,j
q˜i as a starting point, by subtracting a small amount from some q̃i > 0, j > 1 and obtaining a
strictly lower-power allocation for Hi (qi ); this contradicts q being an equilibrium point. 2
The significance of Theorem 4.30 is that all mobiles are better-off at one of the equilibrium
points, the EEE. The next subsection is thus dedicated to finding a simple distributed mechanism
which converges to this equilibrium point.

4.2.3 BEST-RESPONSE DYNAMICS AND CONVERGENCE TO THE POWER

EFFICIENT EQUILIBRIUM
As in Section 4.1.4, it turns out that the natural course of asynchronous best-response dynamics
converges to the EEE. Notably, these dynamics would not require specific knowledge on other user
strategies, thus they can be applicable in wireless systems.
The distributed mechanism we consider here relies on a user’s best-response, which was
comprehensively studied in Section 4.2.2.2 when users were not able to control their powers. Recall
126 4. WIRELESS NETWORK GAMES
that in our model, the best response (BR) for user i is a solution to the following optimization
problem ⎧ ⎫
⎨xi
 
Ji
 ⎬
m,j j
min πim qi Qi : Hi (qi ) = ρ̄i , (4.31)
qi ∈i ⎩ ⎭
m=1 j =1
ρi
where ρ̄i =  .
l=i (1−pl (ql ))
It can be shown that the BR dynamics converges to the best equilibrium under the same
conditions that were required in Section 4.1.4 (see Theorem 4.18 for a precise statement of the
convergence result, which is not repeated herein). Notice that the equilibrium qb in our context
stands for the second best equilibrium point (or a vector of ones in case of a single equilibrium
point).

4.2.4 EQUILIBRIUM (IN)EFFICIENCY AND BRAESS-LIKE PARADOXES

In the previous section, we have indicated that there exists a best equilibrium in terms of power
investment. Furthermore, this equilibrium can be reached through best-response dynamics. Our
first objective in this section is to examine whether the best equilibrium is also socially optimal. In
Section 4.1, it was shown that this is the case when a single power level is available to each user. We
next demonstrate by means of an example that this property generally does not carry over to our
current model. We then investigate the consequences of providing users with multiple power levels,
and relate our observations to the Braess’ paradox (see Chapter 3, Section 3.1.3).
Consider the case where a central authority, which has full information regarding the channel
state distributions of every user can enforce a stationary transmission strategy (see Definition 4.20)
for every user i ∈ I . We consider the total power consumption as the system-wide performance
criterion, namely
   m   m,j j 
xi Ji
C(q) = πi qi Qi . (4.32)
i∈I m=1 j =1
Our aim is to compare the performance of the optimal centrally assigned strategy-profile to the
performance at the Nash equilibrium with respect to the quantity C(q). To that end, we use the
concepts of price of anarchy (PoA) and price of stability (PoS), as in Section 4.1.2.4.
Recall that a simple example was used in the proof of Theorem 4.15, Section 4.1, to show that
the PoA is generally unbounded. The example obviously holds for the current (more general) model
as well. The fact that the energy efficient equilibrium can be reached by a distributed asynchronous
mechanism makes the price of stability more significant, as the price of anarchy can be avoided by
employing the mechanism.
We next show through a numeric example that unlike the single power case, there can be a
gap between the energy efficient equilibrium and the optimal solution.
Example 4.31 We consider a wireless network of two symmetric users, with identical chan-
nel conditions and rate requirements ρ1 = ρ2 = ρ (hence user indexes are omitted in the fol-
 4.2. NONCOOPERATIVE POWER CONTROL IN COLLISION CHANNELS 127
lowing). The rate per (state, power) pair is given by (4.22). The possible channel states are
Z = {0.1, 0.5, 5, 30, 80, 200}, and the corresponding steady-state probability vector is

π = (0.3, 0.25, 0.2, 0.12, 0.08, 0.05) .

We consider two different game instances:

• Instance 1: Multiple power levels are allowed for each user;

Q(1) = {0, 1, 2, 3, 6, 8, 15} .

• Instance 2: A single power level (besides zero) is allowed; Q(2) = {0, 2}.

For both of the above instances, Figure 4.6 depicts the per-user energy at equilibrium as a
function of the required data rate ρ (note that multiple equilibria are possible for a given ρ).
The interesting region of required rates is ρ ∈ [0.78, 0.92] (emphasized in the figure itself ).
For rates at this region, the EEE of Instance 2 (a single power level) obtains a lower energy investment
compared to the energy efficient equilibrium of Instance 1 (multiple power levels). Noting that the
single power level of 2 is one of the available powers in Q(1) , indicates that the EEE for the multiple-
power case is not a system-wide optimal strategy-profile. Indeed, a simple power strategy that uses
only a single power outperforms the equilibrium of Instance 1. Hence, the optimal centrally assigned
strategy would obviously outperform the equilibrium policy of Instance 1 as well.
We have demonstrated through an example that the price of stability is generally larger than
1. A precise quantification for this measure obviously depends on the channel characteristics, the
available powers and the assumptions on the rate functions. Explicit models and their associated
price of stability are an interesting subject for future research.
A classic example for the consequences of self-interested behavior in networks is the Braess’
paradox [27] (see Chapter 3, Section 3.1.3), which shows that the addition of a link in a transportation
network might increase the overall traffic delay at equilibrium. We next point to Braess-like paradoxes
in our network model, which concern the addition of available power levels to each user.
The first Braess-like paradox has already been demonstrated in Example 4.31. Recall that for
required rates of ρ ∈ [0.78, 0.92], Instance 2 (a single power level) outperforms Instance 1 (multiple
power levels, which include the one used in Instance 2). Apparently, the addition of power levels in
this example worsens user performance in terms of the average energy investment.
We next demonstrate an additional type of Braess-like paradox, which relates to the network’s
capacity. We use the term “capacity" for the (total) maximal rate that can be obtained in the network.
Consider the following example.

Example 4.32 As in Example 4.31, we compare two scenarios that differ in the allowed power
levels. All conditions are identical to the ones of Example 1. The only difference is that instead of
Instance 2, we consider the following instance:
128 4. WIRELESS NETWORK GAMES


    













      



Figure 4.6: Braess-like paradox with regard to power investment. The average energy as a function
of the required rate. Note that multiple equilibria are possible for a given ρ. In the marked region of
required rates, the energy efficient equilibrium with a single power level outperforms the respective one
with multiple power levels.

• Instance 3: A single power level (besides zero) is allowed; Q(3) = {0, 8}. Note that Q(3) ⊂ Q(1) ;

Figure 4.7 presents the throughput ρ that is obtained as a function of the collision-free data
rate Hi (4.24). Observe that the use of a single power level (Instance 3) increases the maximal rate
ρ that can be accommodated in the network (compared to Instance 1). This example indicates that
the use of multiple power levels might decrease the network capacity, due to selfish user behavior.

4.2.5 DISCUSSION
The explanation for the Braess-like paradoxes, as well as for the sub-optimality of the energy efficient
equilibrium is quite intuitive, given the nature of the collision channel. In some cases, user strategies
would result in frequent transmissions; this would be the case if the rate-gain at low power levels
corresponding to inferior channel states is higher than the rate-gain in switching to higher power
levels at good quality states. Figure 4.5 illustrates this idea. Consequently, the shared channel would
be subject to frequent collisions that would lead to both unnecessary power investment and a decrease
in network capacity.
It is well known that the adjustment of power levels increases capacity in single-user chan-
nels [47]. A challenging direction for future research, inspired by the above observations, is to
prevent such Braess-like paradoxes in multiuser wireless networks, and even better, enlarge capacity
and reduce power investment. A key role here could be given to network management that would
determine the right tradeoff between user flexibility and overall performance, by assigning the power
 4.3. RELATED WORK AND EXTENSIONS 129
1.4
Six Levels (1,2,3,5,8,15)
Single Level (8)
1.2

Required Rate (Rho)

0.8

0.6

0.4

0.2

0
0 0.5 1 1.5 2 2.5 3
Collision Free Rate (H)

Figure 4.7: Braess-like paradox with regard to the wireless channel capacity. The obtained rate ρ as a
function of the collision free rate H . Note that several equilibrium points (up to six in this particular case)
are possible for each ρ. It is clearly seen that the use of a single power level may accommodate larger ρ’s,
compared to the case where multiple power levels are used.

levels appropriately. A central issue for future research would be how to detect suboptimal equilibria
and lead the network to the best equilibrium point. A related direction is to examine the robustness
of best-response mechanisms, such as the one suggested here, to changes in the transmitters pop-
ulation, which might occur in wireless networks. An additional research direction is to extend the
reception model beyond the collision model studied in this chapter. In particular, capture models
(which sometimes better represent WLAN systems) are of obvious interest. In these models, the use
of higher power levels increases not only the data rate but also the chances of the individual trans-
mission for being properly received at the base station. Hence, selfish mobiles may have a natural
incentive to transmit at high power levels, which is usually not the case for the collision model stud-
ied here. It will be therefore interesting to examine whether some of the negative consequences of
selfish behavior reported here will disappear, thereby reducing the gap between the energy-efficient
operating point and the optimal one.

4.3 RELATED WORK AND EXTENSIONS

In this section, we briefly describe some related literature and also focus on several important model
extensions.
130 4. WIRELESS NETWORK GAMES
Exploiting channel state information for increasing the network’s capacity has been an on-
going research topic within the information theory community (see [25] for a survey). Several papers
([7, 84] and references therein) consider uplink decentralized approaches, in which each station’s
transmission decision can be based on private CSI only. Nodes are assumed to operate in a cooperative
manner, thus willing to accept a unified throughput-maximizing transmission policy. Papers that
have considered Aloha-like random access networks from a non-cooperative perspective include
[13, 44, 52, 52, 58, 99].
Numerous papers (e.g., [9, 10, 93]) have studied the equilibrium properties of power allo-
cation games where mobile users adjust their transmission power to meet some objective (such as
maximizing the throughput, or the energy investment per bit). The above papers consider a static
setup where mobiles adjust their power based on average network conditions. Consequently, an
equilibrium is characterized by fixed transmission powers for all users. The papers [11, 12, 56]
consider the power allocation game under time-varying channel conditions, which require users
to adjust their transmission power as a function of the channel state. The user objective in these
references is to maximize throughput subject to power constraints. In [56], it is assumed that the
channel state of a particular user is available to all others, an assumption that might be hard to justify
in practice. [11, 12] consider the case where only private channel state information is available. The
reception model that is studied in the above references supports multipacket-reception, as opposed
to the collision channel studied in this chapter.
We list below several variations and extensions for the model considered in the previous two
sections.
Capture and Multi-packet reception. An immediate extension to Sections 4.1–4.2 is to consider
different reception models in the form of capture channels (i.e., a single reception is possible, even
with simultaneous transmissions) or multi-packet reception. Such models are considered in [63]. It is
shown in this paper that while multiple equilibrium points exist in general, one of these equilibria is
uniformly best (for all users) and can be reached by best-response dynamics. A negative phenomenon,
which is demonstrated in [63], is the possibility of a partial-equilibrium with starvation where
stronger users (in terms of received power) satisfy their data rates, while preventing weaker ones
from obtaining their respective rates.
State correlation. A recent paper [31] considers the case where the channel quality of every user
is affected by global and time-varying conditions at the base station. Each user wishes to optimize
its individual network utility that incorporates a natural tradeoff between throughput and power.
It is shown that the equilibrium performance can be arbitrarily bad (in terms of aggregate utility);
however, the efficiency loss at the best equilibrium can be bounded as a function of a technology-
related parameter. Convergence to the best equilibrium is possible in special cases, however, not
guaranteed in general.
Reservation and back-off mechanisms. A recent paper [67] considers the model studied in Section
4.1, with the addition of a reservation mechanism, which is used in the 802.11 standard. The
medium access protocol incorporates channel reservation that relies on RTS (request-to-send) and
 4.4. FUTURE DIRECTIONS 131
CTS (clear-to-send) control packets. Consequently, collisions are reduced to the relatively short
periods where mobiles request channel use. The analysis reveals that the structural results that were
obtained in Section 4.1 continue to hold: For feasible throughput demands, there exist exactly
two Nash equilibrium points in stationary strategies, with one superior to the other uniformly
over all users. Additionally, it is shown that the better equilibrium point can be obtained through
best-response dynamics. However, this dynamics require individual knowledge which might no be
available. Consequently, simpler myopic dynamics are suggested and shown to converge to the better
equilibrium under proper initial conditions.
An additional feature of the 802.11 (and essential for its success) is the exponential back-
off mechanism. It may be possible to extend the scope of the models studied in this chapter to
include this important feature. As a first step in this direction, one may adopt the model suggested
by Bianchi [24], which provides the means for tractable, yet fairly accurate analysis of the 802.11
performance, including the back-off mechanism.

4.4 FUTURE DIRECTIONS

To conclude this chapter, we discuss a few possible research venues within the area of wireless network
games. One “persistent" direction is to keep examining the consequences of self-interested behavior
in the context of new emerging technologies (e.g., WiMax, MIMO antennas). We however wish to
highlight below some basic research directions, which would require the employment of additional
distinctive sets of game-theoretic tools.
Battery-State Dependent Power Control. In sensor network domains, the battery of the mobile is
limited and can be charged only occasionally (e.g., by solar energy). Hence, instead of considering the
long-term power average (either as an objective or as a constraint), the transmitter has to be aware of
its actual remaining energy. Consequently, together with varying channel conditions (as in Sections
4.1–4.2), the underlying single-user optimization task becomes a dynamic power control problem
(rather than a static mapping from channel states to power levels). In a multiuser competitive setup,
the underlying game becomes a dynamic, stochastic game, which is naturally much more involved
and harder to analyze. Preliminary attempts to understand the properties of such dynamic games
can be found in [61]. Recent papers [29, 30] consider the dynamic power allocation problem under
arbitrarily varying channels (i.e., there is no a-priori distribution on the channel states), yet the setup
is cooperative, in the sense that mobiles are interested in maximizing the total system throughput.
Much remains to be explored in the noncooperative context.
Pricing, dynamics and near-potential games. Despite extensive research efforts, there is lack of
predictability regarding the outcome of selfish power control in multi-packet reception domains.
Since the underlying game might not have a special structure (e.g., potential or supermodular game),
it becomes difficult to regulate the network to required operating points. A recent paper [32] tackles
these difficulties by introducing a novel approach, termed the potential-game approach, which relies
on approximating the underlying noncooperative game with a “close" potential game, for which
prices that induce an optimal power allocation can be derived. The authors use the proximity of the
132 4. WIRELESS NETWORK GAMES
original game with the approximate game to establish through Lyapunov-based analysis that natural
user-update schemes (applied to the original game) converge within a neighborhood of the desired
operating point, thereby inducing near-optimal performance in a dynamical sense. It is of interest to
use the potential game approach for other wireless network games for improving the performance
under selfish decision makers.
Cognitive Radios. A Cognitive Radio (CR) is a revolutionary concept that was first defined by
Mitola [71] as a radio that can adapt its transmitter parameters to the environment in which it
operates. It is based on the concept of Software Defined Radio (SDR) [26] that can alter parameters
such as frequency band, transmission power, and modulation scheme through changes in software.
According to the Federal Communications Commission (FCC), a large portion of the assigned
spectrum is used only sporadically [43]. Due to their adaptability and capability to utilize the wireless
spectrum opportunistically, CRs are key enablers to efficient use of the spectrum. Under the basic
model of cognitive-radio networks, Secondary Users (SUs) can use white spaces that are not used by
the Primary Users (PUs), but they must avoid interfering with active PUs. From a game-theoretic
perspective, one may view the SUs as competing for the available spectrum while gearing their sensing
capabilities towards using the spectrum in an optimal way. These perspectives open an exciting area
of research, as users actions involve not only power control and spectrum decisions, but also sensing
decisions (which spectrum channels to sense and when). The latter set of decisions is closely related
to the theoretical frameworks of experts and multi-armed bandits. Indeed, some recent papers (e.g.,
[14] and references therein) study the consequences of distributed sensing on network performance
by using these frameworks for algorithm design and analysis.
 133

CHAPTER 5

Future Perspectives
We conclude this monograph by outlining some high-level directions for future research in the field
of network games. As this field is interdisciplinary by nature, we believe that research should consist
of two commensurate paths:
1. Develop new game-theoretic models to represent interactions among heterogenous users in
current and future communication networks. This also involves identifying the right equilib-
rium notion that captures the strategic interactions among the players.
2. Develop novel game-theoretic tools for the analysis of complex user interaction in noncoop-
erative domains. The objective here is to employ such tools in the design of new network
architectures and protocols.
Game theory provides numerous solution concepts for the understanding of noncooperative
user interaction. However, some of the notions that have been developed are not yet fully deployed
in networked systems. These include:
Correlated equilibrium. To the best of our knowledge, there are no convincing applications in
networked systems that exploit the advantages of the correlated equilibrium concept, despite their
appealing static and dynamic properties. It is, therefore, of great interest to identify potential net-
working applications in which correlation between users can be used to improve system efficiency.
Dynamic (Markov) games. As indicated in Chapter 4, static one-shot games fail short to model
certain user interactions in networks. As exemplified therein in the context of sensor networks the
actual battery condition of a mobile sensor could be the most important factor in its current behavior
and future actions. This gives rise to a dynamic game with an evolving state. Another example is
a peer-to-peer network, in which the decision of whether to download a certain content from a
central server determines if and how the peers would be able to benefit from that content. An
important research direction is therefore to employ the theory of dynamic games in the relevant
network application.
In our opinion, the need for new game-theoretic tools arises from two related reasons. First,
many games over networks do not have a special structure (e.g., potential or supermodular), hence
these games are often hard to analyze. Second, little can be said about the convergence properties of
user dynamics, unless, again, the underlying game has special structure. Motivated by these issues,
we suggest the following research directions:
Analyzing games by their relation to games with structure. As mention earlier, we have recently
introduced a novel potential game approach and applied it in the context of distributed power allo-
cation in multi-cell wireless networks [32]. This approach relies on approximating the underlying
134 5. FUTURE PERSPECTIVES
noncooperative game with a “close” potential game, for which simple incentive schemes that induce
a system-optimal power allocation can be derived. We believe that this approach can be applied to
other resource allocation problems, such as power control in multi-channel systems, and routing in
wireline and wireless networks.
An important direction is to extend the above idea beyond potential games. As we elabo-
rated in this monograph, there are additional classes of games with predictable dynamics outcomes.
These include ordinal potential games and supermodular games. By extending the potential game
approach to other games with special structure, the hope is to be able to cover numerous networking
applications, thereby improving the analysis and controllability thereof.
Dynamic efficiency. In relation to the above item, the ability to enhance the predictability of the
dynamics in noncooperative networks motivates the study of the efficiency of such system. The bulk
of the research that has dealt with quantifying the efficiency (or efficiency loss) in networks focused
on static measures. The well-studied notions of PoA and PoS compare the (Nash) equilibrium
performance to the socially optimal operating point. However, these measures do not capture the
underlying dynamics of the system. For example, some of the equilibria might not be reached by
natural user dynamics, thereby diminishing the relevance of the static efficiency loss measures. It
is therefore of great interest to define and study measures of efficiency loss in the dynamical sense;
e.g., comparing the possible limiting outcomes of best-response dynamics to the socially-optimal
operating point.While this and other related topics could have been considered under very specialized
game structures, the potential game approach and similar approaches may allow one to make progress
for general, unstructured, non-cooperative games.
An additional topic for future work relates to the underlying assumptions on utility functions
in network games. As indicated in [33], the commonly-adopted assumption of selfishness has been
repeatedly questioned by economists and psychologists. Experiments have shown that in some
network environments, the users do not necessarily act selfishly to optimize their own performance,
and their behavior can be regarded as either altruistic or malicious. Accordingly, an active research
area has been to incorporate altruistic or malicious characteristics into the utility functions, and
examine the corresponding equilibrium properties; see, e.g., [33].
As a closing comment, we mention that while the emphasis in this monograph has been on
communication networks, game theory has become a dominant tool for the analysis and design
of emerging applications such as sponsored search auctions and social networks. Indeed, the self-
interested behavior of users plays a central role in these domains. Therefore, we strongly believe that
the research in the field of networking games will continue playing an important role in the better
understanding and design of present and future network systems.
 135

Bibliography
[1] IEEE 802.11 standards, Available from http://standards.ieee.org/getieee802/
802.11.html. 103
[2] D. Acemoglu, K. Bimpikis, and A. Ozdaglar, Price and capacity competition, Games and Eco-
nomic Behavior 66 (2009), no. 1, 1–26. DOI: 10.1016/j.geb.2008.06.004 96
[3] D. Acemoglu, R. Johari, and A. Ozdaglar, Partially optimal routing, Journal on Selected Areas
in Communications 25 (2007), no. 6, 1148–1160. DOI: 10.1109/JSAC.2007.070809 79, 87
[4] D. Acemoglu and A. Ozdaglar, Flow control, routing, and performance from service provider
viewpoint, LIDS report, WP-1696, 2004. 19, 88
[5] , Competition and efficiency in congested markets, Mathematics of Operations Research
32 (2007), no. 1, 1–31. DOI: 10.1287/moor.1060.0231 19, 20, 88, 90, 91, 92, 93, 94
[6] , Competition in parallel-serial networks, Journal on Selected Areas in Communications
25 (2007), no. 6, 1180–1192. DOI: 10.1109/JSAC.2007.070812 19, 88, 92, 94, 95
[7] S. Adireddy and L. Tong, Exploiting decentralized channel state information for ran-
dom access, IEEE Transactions on Information Theory 51 (2005), no. 2, 537–561.
DOI: 10.1109/TIT.2004.840878 130
[8] R.K. Ahuja, T.L. Magnanti, and J.B. Orlin, Network flows, Prentice-Hall, Englewood Cliffs,
New Jersey, 1993. xiii
[9] T. Alpcan, T. Başar, R. Srikant, and E. Altman, CDMA uplink power control as a noncooperative
game, Wireless Networks 8 (2002), 659–670. DOI: 10.1023/A:1020375225649 130
[10] E. Altman and Z. Altman, S-modular games and power control in wireless networks, IEEE Trans-
actions on Automatic Control 48 (2003), no. 5, 839–842. DOI: 10.1109/TAC.2003.811264
58, 130
[11] E. Altman, K. Avrachenkov, I. Menache, G. Miller, B. Prabhu, and A. Shwartz, The water-
filling game in fading multiple access channel, IEEE Transactions on Automatic Control 54
(2009), no. 10, 2328–2340. DOI: 10.1109/TIT.2008.920340 130
[12] E. Altman, K. Avrachenkov, G. Miller, and B. Prabhu, Discrete power con-
trol: Cooperative and non-cooperative optimization, INFOCOM, 2007, pp. 37–45.
DOI: 10.1109/INFCOM.2007.13 130
136 BIBLIOGRAPHY
[13] E. Altman, R. El-Azouzi, and T. Jimenez, Slotted Aloha as a game with partial information,
Computer Networks 45 (2004), no. 6, 701–713. DOI: 10.1016/j.comnet.2004.02.013 130

[14] A. Anandkumar, N. Michael, and A.K.Tang, Opportunistic Spectrum Access with Multiple Users:
Learning under Competition, INFOCOM, 2010. DOI: 10.1109/INFCOM.2010.5462144
132

[15] T. Basar and G.J. Olsder, Dynamic noncooperative game theory, Academic Press, London/New
York, 1989. 4

[16] T. Basar and R. Srikant, Revenue-maximizing pricing and capacity expansion in a many-users
regime, Proc. of INFOCOM, 2002. DOI: 10.1109/INFCOM.2002.1019271 88

[17] , A stackelberg network game with a large number of followers, Journal of Optimization
Theory and Applications 115 (2002), no. 3, 479–490. DOI: 10.1023/A:1021294828483 88

[18] M. Beckmann, C.B McGuire, and C. B. Winsten, Studies in the economics of transportation,
Yale University Press, 1956. 74

[19] D. Bertsekas and R. Gallager, Data networks, Prentice-Hall, 1992. 114

[20] D.P. Bertsekas and R.G. Gallager, Data networks, Prentice-Hall, Englewood Cliffs, New
Jersey, 1992. xiii

[21] D.P. Bertsekas, A. Nedić, and A.E. Ozdaglar, Convex analysis and optimization, Athena Sci-
entific, Cambridge, Massachusetts, 2003. 22, 87

[22] D.P. Bertsekas and J.N. Tsitsiklis, Parallel and distributed computation, Prentice-Hall, Engle-
wood Cliffs, New Jersey, 1989. xiii

[23] U. Bhaskar, L. Fleischer, D. Hoy, and C. C. Huang, Equilibria of atomic flow games are not
unique, SODA, SIAM, 2009, pp. 748–757. DOI: 10.1023/A:1021294828483 97

[24] G. Bianchi, Performance analysis of the IEEE 802.11 distributed coordination func-
tion, IEEE Journal on Selected Areas in Communications 18 (2000), no. 3, 535–547.
DOI: 10.1109/49.840210 131

[25] E. Biglieri, J. Proakis, and S. Shamai, Fading channels: Information-theoretic and communi-
cations aspects, IEEE Transactions on Information Theory 44 (1998), no. 6, 2619–2692.
DOI: 10.1109/18.720551 99, 102, 104, 130

[26] Vanu G. Bose, Alok B. Shah, and Michael Ismert, Software Radios for Wireless Networking,
Proc. IEEE INFOCOM’98, Apr. 1998. 132

[27] D. Braess, Uber ein paradoxon aus der verkehrsplanung, Unternehmensforschung 12 (1969),
258 – 268. 19, 71, 77, 127
 BIBLIOGRAPHY 137
[28] G. Brown, Iterative solution of games by fictitious play, Activity Analysis of Production and
Allocation (T. Koopmans, ed.), John Wiley and Sons, New York, New York, 1951. 45
[29] N. Buchbinder, L. Lewin-Eytan, I. Menache, J. Naor, and A. Orda, Dynamic power alloca-
tion under arbitrary varying channels - an online approach, INFOCOM, 2009, pp. 145–153.
DOI: 10.1109/INFCOM.2009.5061916 131
[30] , Dynamic power allocation under arbitrary varying channels - the multi-user case, IN-
FOCOM, 2010. DOI: 10.1109/INFCOM.2010.5462067 131
[31] O. Candogan, I. Menache, A. Ozdaglar, and P. A. Parrilo, Competitive schedul-
ing in wireless collision channels with correlated channel state, GameNets, 2009.
DOI: 10.1109/GAMENETS.2009.5137452 130
[32] , Near-optimal power control in wireless networks - a potential game approach, INFO-
COM, 2010. DOI: 10.1109/INFCOM.2010.5462017 131, 133
[33] P. Chen and D. Kempe, Altruism, selfishness, and spite in traffic routing, Proc. of Electronic
Commerce (EC), 2008. DOI: 10.1145/1386790.1386816 134
[34] R. Cominetti, J. R. Correa, and N. E. Stier-Moses, The impact of oligopolistic competition in
networks, Operations Research 57 (2009), no. 6, 1421–1437. DOI: 10.1287/opre.1080.0653
97
[35] J. R. Correa, A. S. Schulz, and N. E. Stier-Moses, Selfish routing in capacitated networks, Math-
ematics of Operations Research 29 (2004), no. 4, 961–976. DOI: 10.1287/moor.1040.0098
84
[36] , A geometric approach to the price of anarchy in nonatomic congestion games, Games and
Economic Behavior 64 (2008), 457–469. DOI: 10.1016/j.geb.2008.01.001 84, 90
[37] J. R. Correa and N. E. Stier-Moses, Wardrop equilibria, Wiley Encyclopedia of Operations
Research and Management Science ( J. J. Cochran, ed.), John Wiley and Sons, 2010, Forth-
coming. 20
[38] C. Courcoubetis and R. Weber, Pricing communication networks: Economics, technology and
modelling, Wiley, 2003. 96
[39] S. C. Dafermos and F. T. Sparrow, The traffic assignment problem for a general network, J. of
Research of the National Bureau of Standards 73 (1969), no. 2, 91–118. 85
[40] P. Dasgupta and E. Maskin, The existence of equilibrium in discontinuous economic games. 1:
Theory, Review of Economic Studies 53 (1986), 1–26. 29
[41] , The existence of equilibrium in discontinuous economic games. 2: Applications, Review of
Economic Studies 53 (1986), 27–42. 29
138 BIBLIOGRAPHY
[42] L. A. DaSilva, Pricing for qos-enabled networks: a survey, IEEE Communication Surveys and
Tutorials 3 (2000), no. 2, 2–8. DOI: 10.1109/COMST.2000.5340797 96

[43] FCC, ET Docket No. 03-222, Notice of Proposed Rule Making and Order, Dec. 2003. 132

[44] A. Fiat, Y. Mansour, and U. Nadav, Efficient contention resolution for selfish agents, Symposium
on Discrete Computing (SODA), 2007, pp. 179 – 188. 130

[45] D. Fudenberg and D.K. Levine, The theory of learning in games, MIT Press, Cambridge,
Massachusetts, 1999. 45, 49, 52

[46] D. Fudenberg and J. Tirole, Game theory, MIT Press, Cambridge, Massachusetts, 1991. 43,
56, 62, 114

[47] A. J. Goldsmith and P. P. Varaiya, Capacity of fading channels with channel side in-
formation, IEEE Transactions on Information Theory 43 (1997), no. 6, 1986–1992.
DOI: 10.1109/18.641562 128

[48] M. Haviv and R. Hassin, To queue or not to queue: Equilibrium behavior in queueing systems,
2003. 97

[49] A. Hayrapetyan, E. Tardos, and T. Wexler, A network pricing game for selfish traffic, Proc.of
ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing, 2005.
DOI: 10.1145/1073814.1073869 94

[50] J. Huang, R. Berry, and M. L. Honig, Distributed interference compensation for wireless net-
works, IEEE Journal on Selected Areas in Communications 24 (2006), no. 5, 1074–1085.
DOI: 10.1109/JSAC.2006.872889 58

[51] X. Huang, A. Ozdaglar, and D. Acemoglu, Efficiency and braess’ paradox under pricing in general
networks, IEEE Journal on Selected Areas in Communication 24 (2006), no. 5, 977–991.
DOI: 10.1109/JSAC.2006.872879 91

[52] Y. Jin and G. Kesidis, Equilibiria of a noncooperative game for heterogeneous

users of an ALOHA network, IEEE Comm. Letters 6 (2002), no. 7, 282–284.
DOI: 10.1109/LCOMM.2002.801326 130

[53] S. Kakutani, A generalization of brouwerÕs fixed point theorem, Duke Mathematical Journal 8
(1941), no. 3, 457–459. DOI: 10.1215/S0012-7094-41-00838-4 22

[54] F.P. Kelly, A.K. Maulloo, and D.K.Tan, Rate control for communication networks: Shadow prices,
proportional fairness, and stability, Journal of the Operational Research Society 49 (1998), 237–
252. 58
 BIBLIOGRAPHY 139
[55] E. Koutsoupias and C. Papadimitriou, Worst-case equilibria, Proc. of the 16th An-
nual Symposium on Theoretical Aspects of Computer Science, 1999, pp. 404–413.
DOI: 10.1007/3-540-49116-3_38 75

[56] L. Lai and H. El-Gamal, The water-filling game in fading multiple access channel, IEEE Trans-
actions on Information Theory (2006), Submitted. DOI: 10.1109/TIT.2008.920340 130

[57] S. Low and D.E. Lapsley, Optimization flow control, I: Basic algorithm and
convergence, IEEE/ACM Transactions on Networking 7 (1999), no. 6, 861–874.
DOI: 10.1109/90.811451 58

[58] B. MacKenzie and S. B. Wicker, Stability of slotted aloha with multipacket

reception and selfish users, Proceedings of INFOCOM, 2003, pp. 1583–1590.
DOI: 10.1109/INFCOM.2003.1209181 130

[59] N. B. Mandayam, S. B. Wicker, J. Walrand, T. Basar, J. Huang, and D. P. Palomar (editors),

Special issue on game theory in communication systems, Journal of Selected Areas in Communi-
cations 26 (2008), no. 7. 99

[60] A. Mas-Colell, M.D. Whinston, and J.R. Green, Microeconomic theory, Oxford University
Press, 1995. 3

[61] I. Menache and E. Altman, Battery-state dependent power control as a dynamic game, WiOpt,
2008. DOI: 10.1109/WIOPT.2008.4586072 131

[62] I. Menache and N. Shimkin, Capacity management and equilibrium for propor-
tional qos, IEEE/ACM Transactions on Networking 16 (2008), no. 5, 1025–1037.
DOI: 10.1109/TNET.2007.911430 96

[63] , Decentralized rate regulation in random access channels, INFOCOM, 2008, pp. 394–
402. DOI: 10.1109/INFOCOM.2008.86 130

[64] , Efficient rate-constrained nash equilibrium in collision channels with state information,
INFOCOM, 2008, pp. 403–411. DOI: 10.1109/INFOCOM.2008.87 102, 111

[65] , Noncooperative power control and transmission scheduling in wireless collision channels,
SIGMETRICS, 2008, pp. 349–358. DOI: 10.1145/1384529.1375497 102, 122

[66] , Rate-based equilibria in collision channels with fading, IEEE Journal on Selected Areas
in Communications 26 (2008), no. 7, 1070–1077. DOI: 10.1109/JSAC.2008.080905 102,
111, 113, 114, 116

[67] , Reservation-based distributed medium access in wireless collision channels, Telecommu-

nications Systems (2010), Available online. DOI: 10.4108/ICST.VALUETOOLS2008.4475
102, 115, 130
140 BIBLIOGRAPHY
[68] F. Meshkati, H. V. Poor, and S. C. Schwartz, Energy-efficient resource allocation
in wireless networks, IEEE Signal Processing Magazine 24 (2007), no. 3, 58–68.
DOI: 10.1109/MSP.2007.361602 101

[69] P. Milgrom and J. Roberts, Rationalizability, learning and equilibrium in games with strategic
complementarities, Econometrica 58 (1990), 1255–1278. DOI: 10.2307/2938316 56, 57

[70] P. Milgrom and C. Shannon, Monotone comparative statics, Econometrica 62 (1994), 157–180.
56

[71] J. Mitola III, Cognitive Radio for Flexible Mobile Multimedia Communications, ACM/Kluwer
MONET 6 (2001), no. 5, 435–441. DOI: 10.1109/MOMUC.1999.819467 132

[72] K. Miyasawa, On the convergence of learning processes in a 2x2 nonzero sum game, Research
Memo 33, Princeton, 1961. 49

[73] A. F. Molisch, Wireless communications, John Wiley and Sons, 2005. 101

[74] D. Monderer, D. Samet, and A. Sela, Belief affirming in learning processes, Mimeo, Technion,
1994. 50

[75] D. Monderer and L.S. Shapley, Fictitious play property for games with identical interests, Journal
of Economic Theory 68 (1996), 258–265. DOI: 10.1006/jeth.1996.0014 49

[76] , Potential games, Games and Economic Behavior 14 (1996), 124–143.

DOI: 10.1006/game.1996.0044 63

[77] R. Myerson, Game theory: Analysis of conflict, Harvard University Press, Cambridge, Mas-
sachusetts, 1991. 26

[78] J. Nash, Non-cooperative games, Annals of Mathematics 54 (1951), no. 2, 286–295. 23

[79] P. Njoroge, A. Ozdaglar, N. Stier-Moses, and G. Weintraub, Competition, market coverage, and
quality choice in interconnected platforms, in Proc. of NetEcon, 2009, Available online: http://
netecon.seas.harvard.edu/NetEcon09/index09.html. 97

[80] A. M. Odlyzko, Paris metro pricing for the internet, Proc. of the first ACM conference on
Electronic Commerce (EC), 1999, pp. 140 – 147. DOI: 10.1145/336992.337030 96

[81] A. Orda, R. Rom, and N. Shimkin, Competitive routing in multi-user communication networks,
IEEE/ACM Trans. on Networking 1 (1993), no. 3, 510–521. DOI: 10.1145/336992.337030
97

[82] A. Ozdaglar, Price competition with elastic traffic, Networks (2008), Published online.
DOI: 10.1002/net.v52:3 94
 BIBLIOGRAPHY 141
[83] A. Ozdaglar and R. Srikant, Incentives and pricing in communication networks, Algorithmic
Game Theory (N. Nisan, T. Roughgarden, E. Tardos, and V. Vazirani, eds.), Cambridge
University Press, 2007. 19

[84] X. Qin and R. Berry, Distributed approaches for exploiting multiuser diversity in wire-
less networks, IEEE Transactions on information theory 52 (2006), no. 2, 392–413.
DOI: 10.1109/TIT.2005.862103 104, 130

[85] J. Rexford, Route optimization in ip networks, in Handbook of Optimization in Telecommu-

nications, Springer Science + Business, Kluwer Academic Publishers, 2006. 79

[86] O. Richman and N. Shimkin, Topological uniqueness of the nash equilibrium for
atomic selfish routing, Mathematics of Operations Research 32 (2007), no. 1, 215–232.
DOI: 10.1287/moor.1060.0229 97

[87] J. Robinson, An iterative method of solving a game, Annals of Mathematical Statistics 54 (1951),
296–301. 49

[88] J.B. Rosen, Existence and uniqueness of equilibrium points for concave n-person games, Econo-
metrica 33 (1965), no. 3. 32

[89] R.W. Rosenthal, A class of games possessing pure-strategy nash equilibria, International Journal
of Game Theory 2 (1973), 65–67. DOI: 10.1007/BF01737559 66

[90] T. Roughgarden, Selfish routing and the price of anarchy, MIT Press, 2005. 77, 78, 97

[91] T. Roughgarden and E. Tardos, How bad is selfish routing, Journal of the ACM 49 (2002),
236–259. DOI: 10.1145/506147.506153 76

[92] E. Sabir, R. El-Azouzi, V. Kavitha, Y. Hayel, and E. Bouyakhf, Stochastic learning solution
for constrained nash equilibrium throughput in non saturated wireless collision channels, Proc. of
VALUETOOLS, 2009, pp. 1–10. DOI: 10.4108/ICST.VALUETOOLS2009.7764 116

[93] C. U. Saraydar, N. B. Mandayam, and D. J. Goodman, Efficient power control via pricing in
wireless data networks, IEEE Transactions on Communications 50 (2002), no. 2, 291–303.
DOI: 10.1109/26.983324 58, 130

[94] L.S. Shapley, Some topics in two-person games, Advances in Game Theory (M. Drescher, L.S.
Shapley, and A.W. Tucker, eds.), Princeton University Press, Princeton, 1964. 50

[95] A. Simsek, A. Ozdaglar, and D. Acemoglu, Uniqueness of generalized equilibrium for box con-
strained problems and applications, Proc. of Allerton Conference, 2005. 35

[96] , A generalized Poincare-Hopf theorem for compact nonsmooth regions, Mathematics of

Operations Research 32 (2007), no. 1, 193–214. DOI: 10.1287/moor.1060.0235 35
142 BIBLIOGRAPHY
[97] , Local indices for degenerate variational inequalities, Mathematics of Operations Re-
search 33 (2008), no. 2, 291–301. DOI: 10.1287/moor.1070.0299 35

[98] N. Stein, P.A. Parrilo, and A. Ozdaglar, Correlated equilibria in continuous games: Char-
acterization and computation, to appear in Games and Economic Behavior (2010).
DOI: 10.1109/CDC.2007.4434890 19

[99] J. Sun and E. Modiano, Opportunistic power allocation for fading channels with non-
cooperative users and random access, IEEE BroadNets Wireless Networking Symposium, 2005.
DOI: 10.1109/ICBN.2005.1589639 130

[100] D. Topkis, Supermodularity and complementarity, Princeton University Press, Princeton, 1998.
56

[101] D.M. Topkis, Equilibrium points in nonzero-sum n-person submodular games, SIAM Journal of
Control and Optimization 17 (1979), no. 6, 773—-787. DOI: 10.1137/0317054 56, 57, 60

[102] J.N. Tsitsiklis and D.P. Bertsekas, Distributed asynchronous optimal routing in data
networkss, IEEE Transactions on Automatic Control 31 (1986), no. 4, 325–332.
DOI: 10.1109/TAC.1986.1104261 xiii

[103] X. Vives, Nash equilibrium with strategic complementarities, Journal of Mathematical Eco-
nomics 19 (1990), no. 3, 305–321. DOI: 10.1016/0304-4068(90)90005-T 56

[104] , Oligopoly pricing, MIT Press, Cambridge, Massachusetts, 2001. 56

[105] J. G. Wardrop, Some theoretical aspects of road traffic research, Proc. of the Institute of Civil
Engineers, Part II 1 (1952), 325–378. 72

[106] M. Zorzi, Mobile radio slotted ALOHA with capture and diversity, INFOCOM, 1995, pp. 121–
128. DOI: 10.1007/BF01202545 101
 143

Authors’ Biographies

ISHAI MENACHE
Ishai Menache received his PhD degree in Electrical Engineering from the Technion, Israel Institute
of Technology, in 2008. Prior to his graduate studies, he worked for a couple of years in Intel, as an
engineer in the networks communication group. Until recently, he was a postdoctoral associate at
the Laboratory for Information and Decision Systems in MIT. He is currently a visiting researcher
at Microsoft Research New England, focusing on pricing and resource allocation aspects of Cloud
Computing. Dr. Menache’s broader areas of interest include communication networks, game theory
and machine learning. He is a recipient of the Marie Curie Outgoing International Fellowship.

ASUMAN OZDAGLAR
Asuman Ozdaglar received the B.S. degree in electrical engineering from the Middle East Technical
University, Ankara, Turkey, in 1996, and the S.M. and the Ph.D. degrees in electrical engineering
and computer science from the Massachusetts Institute of Technology, Cambridge, in 1998 and
2003, respectively.
Since 2003, she has been a member of the faculty of the Electrical Engineering and Computer
Science Department at the Massachusetts Institute of Technology, where she is currently the Class
of 1943 Associate Professor. She is also a member of the Laboratory for Information and Decision
Systems and the Operations Research Center. Her research interests include optimization theory,
with emphasis on nonlinear programming and convex analysis, game theory, with applications in
communication, social, and economic networks, and distributed optimization and control. She is
the co-author of the book entitled “Convex Analysis and Optimization” (Athena Scientific, 2003).
Professor Ozdaglar is the recipient of a Microsoft fellowship, the MIT Graduate Student
Council Teaching award, the NSF Career award, and the 2008 Donald P. Eckman award of the
American Automatic Control Council. She served on the Board of Governors of the Control Sys-
tem Society in 2010. She is currently the chair of the working group “Game-Theoretic Methods
in Networks” under the Technical Committee “Networks and Communications Systems” of the
IEEE Control Systems Society and serves as an associate editor for the area Optimization Theory,
Algorithms and Applications for the Asia-Pacific Journal of Operational Research.