ECON 221 - Introduction to Strategic Thinking

Sequential-Move Games

Markus von Wartburg

The University of British Columbia

Outline

1 Sequential-Move Games
Game Tree and Strategies
Subgame(s)
Backward Induction and the Subgame Perfect Equilibrium (Rollback)
Credibility of Threats and Promises
Additional Moves / Players
Nature as a Player
Backward Induction as a Behavioral Principle
Sequential-Move Games With Perfect Information

Model strategic situations with a strict order of play.

Perfect information means that players know everything that has happened prior
to making the decision.

Sequential-move games are represented in extensive form using a game tree.

Constructing a game tree:

1 Who are the players?

2 When does each player get to move?

3 What are the actions available to each player?

Game Tree

A game tree consists of decision nodes, terminal nodes and branches

connecting them.

The game starts at the root of the game tree with the initial (decision) node.

At each decision node, a player (possibly nature) chooses an action.

Actions are shown as branches emerging from a node and leading to another
decision or terminal node.

Each terminal node (where no player has another action) is associated with an
outcome of the game and payoffs for the players.
Entry Game

Entrant Players

Stay Out Enter

Actions Monopolist
( 0 , 10 )
Accommodate Fight
Decision Nodes
Entrant Entrant
Payoffs

In Out In Out

Terminal Nodes

( 6 ,6 ) ( -3 , 10 ) ( -4 , -2 ) ( -3 , 5 )
Strategies in Sequential-Move Games

A pure strategy is a complete plan of action that specifies the choice to be made
at each decision node (even at nodes precluded by previous play).

- Strategies at nodes not reached can influence earlier play

- Small mistakes.

Entrant Monopolist Entrant

Stay Out Enter

Monopolist
Accommodate (Stay Out, In, In) (Enter, In, In)
( 0 , 10 )
Accommodate Fight Fight (Stay Out, In, Out) (Enter, In, Out)
Entrant Entrant
(Stay Out, Out, In) (Enter, Out, In)
In Out In Out
(Stay Out, Out, Out) (Enter, Out, Out)

( 6 ,6 ) ( -3 , 10 ) ( -4 , -2 ) ( -3 , 5 )

The Entrant’s strategy is listed as (<root>,<accommodate>,<fight>).

Strategy as a COMPLETE Plan of Action

Why does a strategy have to include actions at nodes that are precluded by own previous play?
Why are (Stop, Stop) and (Stop, Share) proper strategies for Player 1 but not simply Stop?

Player 1

Stop Continue

Player 2
( 1 ,0 )
Stop Continue

Player 1
( 0 ,2 )

Stop Share

( 4 ,1 ) ( 3 ,3 )

1 Stop does not uniquely describe Player 1’s strategy.

2 It is impossible to find the optimal strategy of Player 2 against Stop since it depends on the
choice of Player 1 at the last node.

3 Without the optimal strategy of Player 2 we cannot determine the optimal action for Player 1 at
the first node.
Subgame(s)

Subgame
With perfect information, a subgame consists of a subset of the nodes and branches of
the original game that, when taken together, constitute a game in themselves.

Since a subgame must be a game, it has a unique initial node (called subroot).

The subgame consists of the subroot and all its successors.

Every game is a (trivial) subgame of itself.

Entrant

Stay Out Enter

Monopolist
( 0 , 10 )
Accommodate Fight

Entrant Entrant

In Out In Out

( 6 ,6 ) ( -3 , 10 ) ( -4 , -2 ) ( -3 , 5 )
Subgame Perfect Equilibrium

Reinhard Selten invented and formalized the solution concept called subgame perfect
equilibrium.
Born in Breslau (Wroclaw), now Poland, Reinhard Selten (1930 - ) studied The Theory of
Games and Economic Behavior by von Neumann and Morgenstern subsequently earned
his Ph.D in mathematics at the University of Frankfurt (Germany). Selten is a professor
emeritus at the University of Bonn (Germany) and his pathbreaking work on extensive form
games led to his sharing the 1994 Nobel Memorial Prize in Economics (with John Nash and
John Harsanyi).
Reinhard Selten

Subgame Perfect Equilibrium

A set of strategies (one for each player) is a subgame perfect equilibrium of a game,
if this set of strategies is optimal in every subgame of the game.

In games with perfect information, the subgame perfect equilibria consist of those selected by
backward induction (rollback).
Backward Induction (Rollback)

"It is true that life must be understood backward, but . . . it must be lived forward."
Søren Kierkegaard

Look Forward, Reason Back.

Find the outcome of the game using Backward Induction (Rollback):

1 Find the optimal choice at last decision node.

2 Eliminate ("prune") non-optimal actions.

3 Find optimal choice at previous decision node (actions now lead directly to terminal node).

4 Work backwards to the start of the game.

Backward Induction: Step 1

Find the optimal choice at last decision node.

Entrant

Stay Out Enter

Monopolist
( 0 , 10 )
Accommodate Fight

Entrant Entrant

In Out In Out

( 6 ,6 ) ( -3 , 10 ) ( -4 , -2 ) ( -3 , 5 )

Eliminate ("prune") non-optimal actions.

Backward Induction: Step 2

Find optimal choice at previous decision node (actions now lead directly to
terminal node).

Entrant

Stay Out Enter

Monopolist
( 0 , 10 )
Accommodate Fight

Entrant Entrant

In Out In Out

( 6 ,6 ) ( -3 , 10 ) ( -4 , -2 ) ( -3 , 5 )
Backward Induction: Final Step

Work backwards to the start of the game.

Entrant

Stay Out Enter

Monopolist
( 0 , 10 )
Accommodate Fight

Entrant Entrant

In Out In Out

( 6 ,6 ) ( -3 , 10 ) ( -4 , -2 ) ( -3 , 5 )
Outcome Entry Game

Entrant

Stay Out Enter

Monopolist
( 0 , 10 )
Accommodate Fight

Entrant Entrant

In Out In Out

( 6 ,6 ) ( -3 , 10 ) ( -4 , -2 ) ( -3 , 5 )

Equilibrium Outcome:
1 Entrant enters the market.

2 Monopolist will accommodate entrant.

3 Entrant will stay in the market.

Equilibrium Strategies:

Entrant: (Enter, In, Out)

Monopolist: Accommodate
The Credibility of Threats I

Entrant

Stay Out Enter

Monopolist
( 0 , 10 )
Accommodate Fight

Entrant Entrant

In Out In Out

( 6 ,6 ) ( -3 , 10 ) ( -4 , -2 ) ( -3 , 5 )

Suppose the monopolist threatens to fight if entrant enters. The entrant should
stay out if he believes the threat, but should he believe it?

Credibility of Threats
A threat is only credible, if it is in the best interest of the player to carry out the threat.

Once entrant entered, monopolist is better off accommodating the entrant.

Monopolist’s threat to fight if entrant enters is not credible.
The Credibility of Threats II

Entrant

Stay Out Enter

Monopolist
( 0 , 10 )
Accommodate Fight

Entrant Entrant

In Out In Out

( 6 ,6 ) ( -3 , 10 ) ( -4 , -2 ) ( -3 , 8 )

Monopolist’s cost to fight are lower, is the threat of fighting now credible?

It is in the best interest of the monopolist to fight after the entrant has entered,
hence the threat is credible. The credibility of the threat alters the outcome of the
game, the entrant will stay out.

Equilibrium Strategies: {Entrant, Monopolist} = {(Stay Out, In, Out), Fight}

Credibility of Promises

Extension of Credit

The debtor asks to borrow $100, promising to pay the lender $105 back in a year’s
time.

Debtor can earn $110 over the course of the year. Then he must decide whether
to pay back the lender or default.

It is in the joint interest of the parties that the loan be made.

Lendor

Don’t make loan Make loan

Debtor
( 0 ,0 )
Default Pay back

( -100 , 110 ) (5, 5)

Extension of Credit

Lendor

Don’t make loan Make loan

Debtor
( 0 ,0 )
Default Pay back

( -100 , 110 ) (5, 5)

The lender will not make a loan since the debtor will not pay back the loan (plus
interest). The promise to pay back the loan is not credible.

Credibility of Promises
A promise is only credible, if it is in the best interest of the player to follow through on
the promise.

Suppose there exists a legal system that allows the lender to enforce the loan in
the event of default and makes the debtor pay the lender’s litigation costs of $10.
Enforcement of Contracts Through Legal System

Lendor

Don’t make loan Make loan

Debtor
( 0 ,0 )
Default Pay back

( 5 , -5 ) ( 5 ,5 )

The lender makes the loan, since the debtor will find it in his best interest to pay
back the loan (promise is credible).

It is in the interest of both parties to allow one to call upon the legal system to
enforce a promise that the other made.

In the absence of legally enforceable contracts, reputation, long-term cooperation

or the threat of violence (mafia) can ensure that each party performs.
Costly Enforcement of Contracts I

Suppose that the lender must engage in extremely costly litigation (with no cost
recovery) in the event of debtor default.

Litigation imposes costs on lender and debtor of $125.

Lender

Don’t make loan Make loan

Debtor
( 0 ,0 )
Default Pay Back

Lender
(5, 5)
Sue Do nothing

( -120 , -120 ) ( -100 , 110 )

Costly Enforcement of Contracts II

Lender

Don’t make loan Make loan

Debtor
( 0 ,0 )
Default Pay Back

Lender
(5, 5)
Sue Do nothing

( -120 , -120 ) ( -100 , 110 )

The subgame perfect equilibrium strategies are for the lender (Don’t Make Loan,
Do Nothing) and for the debtor to Default.

If the legal enforcement of contracts is too costly, the legal system loses the ability
to encourage mutually beneficial interactions.
Additional Players

The framework can easily be extended to accommodate additional players.

Backward induction (Rollback) can still be used to determine the equilibrium.

Team Project

Three players work on a project in sequential order.

1 Players can exert effort (E) or slack off (S).

2 If project succeeds, all players get a benefit of 10, but exerting effort costs 4.

3 The project succeeds if at least two players exert effort.

Team Project I

Player 1

Slack Effort

Player 2 Player 2

Slack Effort Slack Effort

Player 3 Player 3 Player 3 Player 3

Slack Effort Slack Effort Slack Effort Slack Effort

0 0 0 10 -4 6 6 6
0 0 -4 6 0 10 6 6
0 -4 0 6 0 6 10 6
Team Project II

Player 1

Slack Effort

Player 2 Player 2

Slack Effort Slack Effort

Player 3 Player 3 Player 3 Player 3

Slack Effort Slack Effort Slack Effort Slack Effort

0 0 0 10 -4 6 6 6
0 0 -4 6 0 10 6 6
0 -4 0 6 0 6 10 6
Additional Moves

Additional moves make the game more complex.

The Game of Nim

1 The game consists of columns of pennies. Two players take turns removing any
number (at least one) of the pennies from a single column.

2 The winner is the player removing the last penny from any column.

For simplicity, consider Nim(2,1): There are two pennies in the first column and
one penny in the second column.
Nim(2,1) I

Player 1

Player 2 Player 2 Player 2

Player 1 Player 1 Player 1

( -1 , 1 ) ( -1 , 1 )

( 1 , -1 ) ( 1 , -1 ) ( 1 , -1 )
Nim(2,1) II

Player 1

Player 2 Player 2 Player 2

Player 1 Player 1 Player 1

( -1 , 1 ) ( -1 , 1 )

( 1 , -1 ) ( 1 , -1 ) ( 1 , -1 )

Equilibrium Outcome:
1 Player 1 removes one penny from first column.
2 Player 2 removes one penny from either column.
3 Player 1 removes the remaining penny and wins (first-mover advantage).

- If there exists more than one optimal action for a player, mark all optimal actions.
- There could be multiple outcomes if the path runs through the whole tree.
Nature as a Player

Nature can be introduced as a special player to make random decisions.

Nature is used to model uncertainties inherent to the game that do not arise from
behavior of players.

Nature chooses between actions based on a fixed probability distribution.

Nature does not receive a payoff, only real players do.

Players choose their action based on their expected payoff if they move before
Nature.
The Beach I

The Beach

The Park Board decides whether to open a large or small concession stand at the beach
selling ice cream.

Nature decides on the weather, sunny or rainy, with equal probability.

After observing the weather, Giorgio decides whether to bring his ice cream cart to the beach
or stay home.

If the weather is sunny, there are ice cream sales of 10 to be made. There are no sales if it is
raining.

If only one ice cream seller is open, it attracts all the sales. If both are open, the concession
stand gets 70% of the sales if it is large, and 40% if it is small.

Giorgio has operating costs of 2 for his ice cream cart, the small concession stand has
operating costs of 2, the large stand has costs of 4.
The Beach II

Park Board

Large Small

Nature Nature

Sunny Rainy Sunny Rainy

50% 50% 50% 50%

Giorgio Giorgio Giorgio Giorgio

Stay Stay Stay Stay
Beach Beach Beach Beach
Home Home Home Home

( 3 ,1 ) ( 6 , 0 ) ( -4 , -2 ) ( -4 , 0 ) ( 2 ,4 ) ( 8 ,0 ) ( -2 , -2 ) ( -2 , 0 )
The Beach III

Park Board

Large Small

Nature Nature

Sunny Rainy Sunny Rainy

50% 50% 50% 50%

Giorgio Giorgio Giorgio Giorgio

Stay Stay Stay Stay
Beach Beach Beach Beach
Home Home Home Home

( 3 ,1 ) ( 6 , 0 ) ( -4 , -2 ) ( -4 , 0 ) ( 2 ,4 ) ( 8 ,0 ) ( -2 , -2 ) ( -2 , 0 )

What happens if Giorgio has to decide before he knows the weather (but after the concession stand opened)?
Backward Induction as a Behavioral Principle

Backward induction may be useful to explain actual outcomes in simple games

with few players and moves.

The concept is less useful in complex sequential-move games such as Chess:

Game tree is too large to draw.

Number of possible chess moves is estimated to be of order 10120 .

Rule required to assign payoffs to nonterminal nodes (intermediate value function).

Backward induction may not predict behavior well if players fail to look ahead far
enough or fail to draw the appropriate conclusions from looking ahead.

Players might care about fairness and not merely about monetary payoffs.
Centipede Game I

Experimental tests of backward induction have been conducted using variants of

the Centipede Game:

1 Continue 2 Continue 1 Continue 2 Continue 1 Continue

( 50, 50 )

Stop Stop Stop Stop Stop

( 20, 0 ) ( 0, 40 ) ( 60, 0 ) ( 0, 80 ) ( 100, 0 )

How would you play this game?

Centipede Game II

1 Continue 2 Continue 1 Continue 2 Continue 1 Continue

( 50, 50 )

Stop Stop Stop Stop Stop

( 20, 0 ) ( 0, 40 ) ( 60, 0 ) ( 0, 80 ) ( 100, 0 )

Backward induction leads to paradoxical outcome: Player 1 stops the game with
the first move.

In most experimental settings, the centipede game continues for at least a few
rounds.

By behaving "irrationally ", the players as a group fare better than if they followed
the logic of backward reasoning!