Skip to main content

Optional background reading

Introduction to Markov Processes, 2021/22, Semester 2
Dr Valentine Nlebedim
Tutor information is taken from the Module Catalogue

You can do well on this module by reading the notes and watching the videos, attending the lectures and workshops, and working on the problem sheets, assignments and practicals, without any further reading. However, students can benefit from optional extra background reading or an alternative view on the material.

My favourite book on Markov chains, which I used a lot while planning this course and writing these notes, is Norris, Markov Chains. This a whole book just on Markov processes, including some more detailed material that goes beyond this module. Its coverage of of both discrete and continuous time Markov processes is very thorough.

Other good books with sections on Markov processes that I have used include Grimmet and Stirzaker, Probability and Random Processes; Grimmet and Walsh, Probability: An Introduction; and Stirzaker, Elementary Probability. Grimmett and Stirzaker is an excellent handbook that covers most of undergraduate probability -- I bought a copy when I was a second-year undergraduate and still keep it next to my desk.

A gentler introduction with plenty of examples is provided by Jones and Smith, Stochastic Processes: An Introduction, although it doesn't cover everything in this module.


G.R. Grimmett and D.R. Stirzaker, Probability and Random Processes, 4th edition, Oxford University Press, 2020. Chapter 6. Chapter 6 on Markov processes is available as an Online Course Reading in Minerva.     

G. Grimmet and D. Walsh, Probability : an introduction ISBN: 0198532644 (pbk ; cased) : £9.95; 0198532725, 2nd edition, Oxford University Press, 2014. Chapter 12.

P.W. Jones and P. Smith, Stochastic Processes: an introduction, 3rd edition, Texts in Statistical Science, CRC Press, 2018. Chapters 2-7.  Whole book available online from the University library.

J.R. Norris, Markov Chains ISBN: 0521481813 (pbk), Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press, 1997. Chapters 1-3. Chapter 1 on discrete time Markov chains is available online.

D.R. Stirzaker, Elementary Probability, 2nd edition, Cambridge University Press, 2003. Chapter 9. Whole book available online from the University library.

This list was last updated on 24/01/2021