3

Here's my understanding of it: Assume we have an $n\times n$ stochastic matrix $P$ that represents our Markov chain such that $x$ and $y$ are stationary distributions for $P$. Then

$P(x) = x$

$P(y) = y$

$P(ax+by) = P(ax) + P(by) = aP(x) + bP(y)$ where $ax+by$ is a convex linear combination

$ = ax + by$

meaning that $ax+by$ is a stationary distribution, so there is an infinite amount of stationary distributions of P if there are at least 2.

Does this mean a Markov chain either has one or infinitely many stationary distributions?

Especially Lime
  • 46,692
user665214
  • 87
  • For a perhaps unenlightening example, let $$ P = \begin{pmatrix}1&0\0&1\end{pmatrix}. $$ Then $\pi P = \pi$ for any row vector $\pi$, so any distribution is a stationary distribution for $P$. – Math1000 Feb 17 '20 at 05:27
  • So here it all depends in which state we are initially. If we're at s0, we will forever remain at s0. The same goes for s1.

    Assuming we start at a fixed state x, can we end up with multiple stationary distributions then?

     – mercury0114 Jun 12 '23 at 12:49

2 Answers2

5

Yes, the set of stationary distributions will always be convex, for the reason you give.

Some Markov chains (like simple random walk on the integers) have no stationary distributions. But finite MCs have stationary distributions, so your statement about "1 or infinitely many" is correct, in this case.

kimchi lover
  • 24,981
4

Yes. Let $\mu$ and $\nu$ be two distinct stationary distributions. Now choose randomly between $\mu$ and $\nu$ with probabilities $p$ and $1-p$, and whatever distribution is chosen, choose according to it an initial state. Then the distribution of the state at time $0$ is $p\mu+(1-p)\nu$. If we chose $\mu$, then our distribution at time $1$ is still $\mu$ (because $\mu$ is stationary), and if we chose $\nu$ our distribution at time $1$ is $\nu$. Therefore our distribution at time $1$ is still $p\mu+(1-p)\nu$, so $p\mu+(1-p)\nu$ is stationary. This is just a probabilistic proof of what you proved algebraically.

Jack M
  • 28,518
  • 7
  • 71
  • 136