Given an i.i.d. sequence $(\theta_{n}^{i})$ with binomial distribution $\mathcal{B}_{3,\frac{1}{3}}$ we define the Markov Chain $$X_{n+1}:= \sum_{i=0}^{X_{n}}\theta_{n}^{i}\:\:\: \mbox{mod }5.$$
Question: Calculate the transition matrix.
Motivation of the question: In stochastic processes it is very difficult to verify if the answer of an exercise is correct, one way is to use the software R, but I am not good at programming, so I turn to this community to verify that my attempt is correct.
My attempt: Note that $\theta_{n}^{i}$ take values in $\left\{0,1,2,3\right\}$ and $X_{n}$ take values in $\left\{0,1,2,3,4\right\}$, then $$\sum_{i=0}^{X_{n}}\theta_{n}^{i} \in \left\{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15\right\}.$$ Furthermore, if $X_{n}=k$ is fixed, then $$\sum_{i=0}^{k}\theta_{n}^{i} \in \left\{0,1,\ldots, 3(k+1)\right\}.$$
Let $x_{n+1},\ldots,x_{1},x_{0}\in \left\{0,1,2,3,4\right\}$, then $$ \begin{array}{rcl} \mathbb{P}(X_{n+1}=x_{n+1} \left|X_{n}=x_{n},\ldots,X_{0}=x_{0} \right. )&=&{\displaystyle \mathbb{P}\left(\left.\sum_{i=0}^{X_{n}}\theta_{n}^{i}\:\:\: \mbox{mod }5=x_{n+1} \right|X_{n}=x_{n},\ldots,X_{0}=x_{0} \right) }\\ &=& {\displaystyle \mathbb{P}\left(\left.\sum_{i=0}^{x_{n}}\theta_{n}^{i}\:\:\: \mbox{mod }5=x_{n+1} \right|X_{n}=x_{n},\ldots,X_{0}=x_{0} \right) } \\ &=& {\displaystyle \mathbb{P}\left(\sum_{i=0}^{x_{n}}\theta_{n}^{i}\:\:\: \mbox{mod }5=x_{n+1} \right) } \end{array} $$ This show that $(X_{n})$ is a Markov Chain, futhermore, if $\Pi$ is the transition matrix, then for all $k,t\in\left\{0,1,2,3,4\right\}$ we have $$\Pi(k,t)={\displaystyle \mathbb{P}\left(\sum_{i=0}^{k}\theta_{n}^{i}\:\:\: \mbox{mod }5=t \right) }. $$ In this sense, note that $$\left\{\sum_{i=0}^{k}\theta_{n}^{i}\:\:\: \mbox{mod }5=t \right\}= \bigcup_{m=0}^{\left\lfloor \frac{3(k+1)}{5} \right\rfloor} \left\{\sum_{i=0}^{k}\theta_{n}^{i}=5m+t \right\}.$$ This is a disjoint union, therefore, we have $$\mathbb{P}\left(\sum_{i=0}^{k}\theta_{n}^{i}\:\:\: \mbox{mod }5=t \right)=\sum_{m=0}^{\left\lfloor \frac{3(k+1)}{5} \right\rfloor}\mathbb{P}\left( \sum_{i=0}^{k}\theta_{n}^{i}=5m+t\right)$$ But we know that $\sum_{i=0}^{k}\theta_{n}^{i}$ has binomial distribution $\mathcal{B}_{3(k+1),\frac{1}{3}}$ (see this Link), then $$\mathbb{P}\left(\sum_{i=0}^{k}\theta_{n}^{i}\:\:\: \mbox{mod }5=t \right)= \sum_{m=0}^{\left\lfloor \frac{3(k+1)}{5} \right\rfloor} \binom{3(k+1)}{5m+t}\left(\frac{1}{3}\right)^{5m+t}\left(\frac{2}{3}\right)^{3(k+1)-5m-t}.$$ Therefore, $$\Pi(k,t)=\sum_{m=0}^{\left\lfloor \frac{3(k+1)}{5} \right\rfloor} \binom{3(k+1)}{5m+t}\left(\frac{1}{3}\right)^{5m+t}\left(\frac{2}{3}\right)^{3(k+1)-5m-t}. $$ Using Wolfram alpha we have
- For $k=0$, $$\Pi(0,t)=\frac{2^{3-t}}{27}\binom{3}{t}.$$
- For $k=1$, $$\Pi(1,t)=\frac{2^{6-t}}{27^2}\binom{6}{t}+\frac{2^{1-t}}{27^2}\binom{6}{t+5}.$$
- For $k=2$, $$\Pi(2,t)=\frac{2^{9-t}}{27^3}\binom{9}{t}+\frac{2^{4-t}}{27^3}\binom{9}{t+5}.$$
- For $k=3$, $$\Pi(3,t)=\frac{2^{12-t}}{27^4}\binom{12}{t}+\frac{2^{7-t}}{27^4}\binom{12}{t+5}+\frac{2^{2-t}}{27^4}\binom{12}{t+10}.$$
- For $k=4$, $$\Pi(4,t)=\frac{2^{15-t}}{27^5}\binom{15}{t}+\frac{2^{10-t}}{27^5}\binom{15}{t+5}+\frac{2^{5-t}}{27^5}\binom{15}{t+10}+\frac{2^{-t}}{27^5}\binom{15}{t+15}.$$
Therefore, $\Pi$ is given by $$\Pi =\left(\begin{array}{lllll} \frac{8}{27} & \frac{12}{27} & \frac{6}{27} & \frac{1}{27} & 0 \\ \frac{76}{27^{2}} & \frac{193}{27^{2}} & \frac{240}{27^{2}} & \frac{160}{27^{2}} & \frac{60}{27^{2}} \\ \frac{2528}{27^{3}} & \frac{2976}{27^{3}} & \frac{4752}{27^{3}} & \frac{5394}{27^{3}} & \frac{4033}{27^{3}} \\ \frac{105736}{27^{4}} & \frac{83736}{27^{4}} & \frac{92926}{27^{4}} & \frac{120560}{27^{4}} & \frac{128480}{27^{4}} \\ \frac{3203937}{27^{5}} & \frac{2830160}{27^{5}} & \frac{2511160}{27^{5}} & \frac{2687780}{27^{5}} & \frac{3115870}{27^{5}} \\ \end{array}\right).$$
Remark: I would like to know if there is a simpler way to solve this question and write $ \Pi $.
