Notes on Stochastic Processes

My handwritten notes/summaries for an undergraduate stochastic processes course that I took in can be found HERE.

I went through a lot of books, but I didn’t like the order in which they introduced different topics. These notes are organized in a way that I thought was most intuitive and logical. So if you can read them (and that’s a big if😊) you shouldn’t have any problem following the arguments and the chain of thought.

Below you can find a rough outline of the syllabus. My notes cover the first half of the course, up until the end of the branching process section.

Errata: In my notes for the first chapter, I sometimes misused “recursive state” when in fact I should have said “recurrent state”.

Syllabus

1. Discrete Time Markov Chains
   1. Finite State Space
   2. Return Time
   3. Countable State Space
   4. Mixing Time
2. Continuous Time Markov Chains
   1. Exponential and Poisson Distributions
   2. Poisson Process
   3. Continuous Time Markov Chains
3. Markov Chain Monte Carlo (MCMC)
   1. Detailed Balance Equation
   2. Metropolis Algorithm
   3. Metropolis-Hastings Algorithm
   4. Glauber Dynamics (Gibbs Sampler)
4. Probabilistic Models
   1. Branching Process
   2. Random Graphs
   3. Percolation
   4. Uniform Spanning Tree