Hi.
This is our second lecture on Markov chains.
We will concentrate on developing further
the general principles and tools behind Markov chains.
We will first briefly review the main definitions,
and apply them to illustrate some additional calculations
one can do, as a way to warm up.
We will then concentrate most of the time
on the central topic of today-- steady-state behavior
of chains-- that is, on what a Markov chain does
when it runs for a long time, and on what
we can say about the probabilities of being
in different states.
And in order to do that, we will review recurrent states,
transient states, and recurrent classes,
talk a bit more about periodic states,
concentrate most of the time on the convergence theorem here,
and the consequence associated with the balanced equations
of Markov chains.
And finally, we will end the lecture
with an important, special class--
the so-called birth/death processes.
So let us start.