Scribd
Upload
19 views

Lecture 3: Martingales and Hitting Probabilities For Random Walk

This lecture discusses martingales and their application to random walk processes. The key points are: 1) A martingale is a sequence of random variables where the expected value of the next term, given all previous terms, is equal to the current term. 2) For a fair coin random walk that is absorbed upon reaching 0 or some value b, the process forms a martingale. 3) For an unfair coin with probabilities p and q, the walk will be a martingale if the terms are multiplied by the ratio q/p. 4) Martingale properties allow determining the probabilities of absorption at the boundaries for the gambler's ruin problem, both for fair and unfair coins.

Uploaded by

spitzersglare
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
19 views

Lecture 3: Martingales and Hitting Probabilities For Random Walk

This lecture discusses martingales and their application to random walk processes. The key points are: 1) A martingale is a sequence of random variables where the expected value of the next term, given all previous terms, is equal to the current term. 2) For a fair coin random walk that is absorbed upon reaching 0 or some value b, the process forms a martingale. 3) For an unfair coin with probabilities p and q, the walk will be a martingale if the terms are multiplied by the ratio q/p. 4) Martingale properties allow determining the probabilities of absorption at the boundaries for the gambler's ruin problem, both for fair and unfair coins.

Uploaded by

spitzersglare
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Stat 150 Stochastic Processes

Spring 2009

Lecture 3: Martingales and hitting probabilities for random walk


Lecturer: Jim Pitman

A sequence of random variables Sn , n {0, 1, 2, . . . }, is a martingale if 1) E|Sn | < for each n = 0, 1, 2, . . . 2) E(Sn+1 |(S0 , S1 , . . . , Sn )) = Sn E.g., Sn = S0 + X1 + + Xn , where Xi are dierences, Xi = Si Si1 . Observe that (Sn ) is a martingale if and only E(Xn+1 |S0 , S1 , . . . , Sn ) = 0. Then say (Xi ) is a martingale dierence sequence. Easy example: let Xi be independent random variables (of each other, and of S0 ), where E(Xi ) 0 = Xi are martingale dierences = Sn is a martingale. Some general observations: If Sn is a martingale, then E(Sn ) E(S0 ) (Constant in n). Proof: By induction, suppose true for n, then E(Sn+1 ) = E(E(Sn+1 |(S0 , S1 , . . . , Sn ))) = E(Sn ) = E(S0 ) Illustration: Fair coin tossing walk with absorption at barriers 0 and b. Process: Start at a. Run until the sum of 1s hits 0 or b, then freeze the value. By construction, 0 Sn b = E|Sn | b < . Given S0 , S1 , . . . , Sn , with 0 < Si < b for 0 i n, we assume that Sn+1 = So E(Sn+1 |S0 , S1 , . . . , Sn with 0 < Si < b, 0 i n) 1 1 = (Sn + 1) + (Sn 1) = Sn 2 2 whereas given S0 , S1 , . . . , Sn , with either Sn = 0 or Sn = b, then Sn+1 = Sn . So also E(Sn+1 |S0 , S1 , . . . , Sn with Si = 0 or b) = Sn 1 Sn + 1 Sn 1 with probability 1/2 with probability 1/2

Lecture 3: Martingales and hitting probabilities for random walk

Since the only possible values of Sn are in the range from 0 to b, no matter what the value of Sn , E(Sn+1 |S0 , S1 , . . . , Sn ) = Sn So the fair random walk with absorbtion at barriers 0 and b is a martingale. Revisit the Gamblers ruin problem for a fair coin. Observe (Sn = b) = (Sn+1 = b) P(Sn = b) P(Sn+1 = b) P(Sn = b) is an increasing sequence which is bounded above by 1. Proof: We know E(Sn ) a. But
b1

a = E(Sn ) = 0 E(Sn = 0) + b P(Sn = b) +


i=1

i P(Sn = i)

b1 Observe that i =1 P(Sn = i) = P(T > n) 0 as n , where T is the time to hit the barriers. Then a a n P(Sn = b) = as n b b b

Gamblers ruin for an unfair coin p q , p + q = 1. Idea: Choose an increasing function g so that we make the game fair; that is, we will make an Mn = g (Sn ) so that (Mn ) is a MG. Look at rSn instead of Sn for some ratio r to be determined. Suppose you are given rS0 , . . . rSn , with 0 < Sn < b, then rSn+1 = Compute 1 E(rSn+1 |rS1 , . . . rSn ) = rSn (rp + q ) = rSn r 1 q rp + q = 1 r = or r = 1. r p r Sn r r Sn 1 r with prob p with prob q

So r = q/p implies rSn is a martingale. Again, the walk is stopped if it reaches the barriers at 0 or b, and given such values the martingale property for the next step is obvious. The martingale property gives: ErSn ra if S0 = a. q q q ( )a = ( )0 P(Sn = 0) + ( )b P(Sn = b) + p p p
b1

q ( )i P(Sn = i) p i=1

Lecture 3: Martingales and hitting probabilities for random walk

Let Pup = limn P(Sn = b), 1 Pup = limn P(Sn = 0), q q q ( )a = ( )0 (1 Pup ) + ( )b Pup p p p (q/p)a 1 = Pup = (q/p)b 1

Case p > q , x a and let b , q lim Pup = 1 ( )a b p q a lim Pdown = ( ) b p

You might also like