1 INTRODUCTION 4
1 Introduction
• Markov chains
– are a fundamental class of stochastic models for sequences of non–independent random variables, i.e.
of random variables possessing a specific dependency structure.
– have numerous applications e.g. in insurance and finance.
– play also an important role in mathematical modelling and analysis in a variety of other fields such as
physics, chemistry, life sciences, and material sciences.
• Questions of scientific interest often exhibit a degree of complexity resulting in great difficulties if the
attempt is made to find an adequate mathematical model that is solely based on analytical formulae.
• In these cases Markov chains can serve as an alternative tool as they are crucial for the construction of
computer algorithms for the Markov Chain Monte Carlo simulation (MCMC) of the mathematical models
under consideration.
This course on Markov chains and Monte Carlo simulation will be based on the methods and models introduced
in the course “Elementare Wahrscheinlichkeitsrechnung und Statistik”. Further knowledge of probability theory
and statistics can be useful but is not required.
• The main focus of this course will be on the following topics:
– discrete–time Markov chains with finite state space
– stationarity and ergodicity
– Markov Chain Monte Carlo (MCMC)
– reversibility and coupling algorithms
• Notions and results introduced in “Elementare Wahrscheinlichkeitsrechnung and Statistik” will be used
frequently. References to these lecture notes will be labelled by the prefix “WR” in front of the number
specifying the corresponding section, theorem, lemma, etc.
• The following list contains only a small collection of introductory texts that can be recommended for in–
depth studies of the subject complementing the lecture notes.
– E. Behrends (2000) Introduction to Markov Chains. Vieweg, Braunschweig
– P. Bremaud (2008) Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues. Springer,
New York
– B. Chalmond (2003) Modeling and Inverse Problems in Image Analysis. Springer, New York
– D. Gamerman, H. Lopes (2006) Markov Chain Monte Carlo: Stochastic Simulation for Bayesian
Inference. Chapman & Hall, London
– O. Häggström (2002) Finite Markov Chains and Algorithmic Applications. Cambridge University
Press, Cambridge
– D. Levin, Y. Peres, E. Wilmer (2009) Markov chains and mixing times. Publications of the AMS,
Riverside
– S. Resnick (1992) Adventures in Stochastic Processes. Birkhäuser, Boston
– C. Robert, G. Casella (2009) Introducing Monte Carlo Methods with R. Springer, Berlin
– T. Rolski, H. Schmidli, V. Schmidt, J. Teugels (1999) Stochastic Processes for Insurance and Finance.
Wiley, Chichester
– Y. Suhov, M. Kelbert (2008) Probability and Statistics by Example. Volume 2. Markov Chains: A
Primer in Random Processes and their Applications. Cambridge University Press, Cambridge
– H. Thorisson (2002) Coupling, Stationarity, and Regeneration. Springer, New York
– G. Winkler (2003) Image Analysis, Random Fields and Dynamic Monte Carlo Methods. Springer,
Berlin
