3

Consider a birth and death process with infinitesimal parameter $\lambda_n$, $\mu_n$. Then the expected length of time for reaching state $r + 1$ starting from state 0 is $$\sum_{n=0}^r \frac{1}{\lambda_n\pi_n} \sum_{k=0}^n \pi_k$$

For the definition $\pi_n$ see the following: $$ \sum_{n=0}^{\infty} \pi_n \sum_{k=0}^n \frac{1}{\lambda_n\pi_n} = \infty$$

where $\pi_0=1$ and $\pi_n= \frac{\lambda_0\lambda_1\cdots \lambda_{n-1}}{\mu_0\mu_1\cdots\mu_{n-1}}$, $n=1,2,\ldots$.

In most practical examples of birth and death processes the last condition is met and the birth and death process associated with the prescribed parameters is uniquely determined.

Well, this is a very complicated exercise for me, first I tried to let $T^{\ast}_n$ denote the elapsed time of first entering state $n + 1$ starting from state $n$.

And then trying to derive a recursion relation for $E[T^{\ast}_n]$ but I can not solve it, I stuck trying to derive this recursion relation....

Could someone help me with hints, suggestions to solve this... Thanks for your time and help everyone.

Rosa Maria Gtz.
  • 3,886
  • 3
  • 29
  • 55
  • Yea, I’m trying to show that the expected length of time for reaching state $r + 1$ starting from state 0 is $$\sum_{n=0}^r \frac{1}{\lambda_n\pi_n} \sum_{k=0}^n \pi_k$$ – Rosa Maria Gtz. Apr 13 '18 at 20:38

1 Answers1

2

For consistency of notation, define $\mu_0=0$. (Normally in birth-death processes $\mu_0$ is just not defined.)

Now the expected time of a transition from $i$ is $(\lambda_i+\mu_i)^{-1}$, the probability of the first transition from $i$ being a birth is $\frac{\lambda_i}{\lambda_i+\mu_i}$, and the probability of the first transition from $i$ being a death is $\frac{\mu_i}{\lambda_i+\mu_i}$. Therefore the expected time to hit $r+1$ started from $i$ satisfies

$$u(i)=(\lambda_i+\mu_i)^{-1} \left ( 1 + \lambda_i u(i+1) + \mu_i u(i-1) \right ),0 \leq i \leq r \\ u(r+1)=0.$$

(Here again for consistency of notation we arbitrarily define $u(-1)$; this has no effect because it is multiplied by zero anyway.) This is a tridiagonal linear system. Because it has Dirichlet boundary conditions ($u(-1)$ and $u(r+1)$ are given), you can solve it recursively instead of by elimination: plug in $i=r$ and use the boundary condition to get a relation between $u(r)$ and $u(r-1)$. Plug in this relation at $i=r-1$, to get a relation between $u(r-1)$ and $u(r-2)$, and so on. When you get to $i=0$ you can actually solve for $u(0)$, because $\mu_0=0$. This is actually what you want in this case, but if it weren't you could back-substitute to find the others.

This kind of procedure is the subject of renewal theory.

Ian
  • 104,572
  • I haven't seen this renewal theory just continuous time Markov chains, this problem is from this subject. – Rosa Maria Gtz. Apr 13 '18 at 20:49
  • @Knight It is just conditioning on the first jump and integrating against the jump time distribution. You don't need to already know anything about renewal theory. – Ian Apr 13 '18 at 20:50
  • It is a little consusing how you get this $L_{ij}$ but I will try it.... – Rosa Maria Gtz. Apr 13 '18 at 21:01
  • @Knight In effect I am writing down a big recursion: $-\lambda_0 u(0) + \lambda_0 u(1) = -1,\mu_1 u(0) - (\lambda_1+\mu_1) u(1) + \lambda_1 u(2) = -1,\dots,\mu_r u(r-1) - (\lambda_r + \mu_r) u(r) + \lambda_r u(r+1)=-1,u(r+1)=0$. Since you have this type of boundary condition, you can proceed recursively starting from $r+1$: plug in $u(r+1)=0$ into the previous equation, solve for $u(r)$ in terms of $u(r-1)$, and then plug that into the next equation, etc. until you get all the way down. – Ian Apr 13 '18 at 21:04
  • @Knight As for where the relation came from, the intuition is that a jump starting at a state $i$ takes an average time of $(\lambda_i+\mu_i)^{-1}$ (where $\mu_0=0$ by definition), is a birth with probability $\frac{\lambda_i}{\lambda_i+\mu_i}$, and is a death with probability $\frac{\mu_i}{\lambda_i+\mu_i}$. So another way to write the same equation for $0<i<r+1$ is $u(j)=(\lambda_i+\mu_i)^{-1} + \frac{\lambda_i}{\lambda_i+\mu_i} u(j+1) + \frac{\mu_i}{\lambda_i+\mu_i} u(j-1)$. Then $u(r+1)$ is trivial and $u(0)$ satisfies a slightly different equation because $\mu_0=0$. – Ian Apr 13 '18 at 21:07
  • Ok, What do you think about this : Let $E[T_0]=\frac{1}{\lambda_0}$ be expected time when there is a birth when there is 0 in the population. If $n \geq 1$ then $$ E[T_n]= P_{n,n+1}[\frac{1}{\lambda_n}] + P_{n,n-1}[E[[T_{n-1}]+E[T_n]]] $$, where $P_{n,n+1}= \frac{\lambda_n}{\lambda_n+\mu_n}$ and $P_{n,n-1}=\frac{\mu_n}{\lambda_n+\mu_n}$. – Rosa Maria Gtz. Apr 20 '18 at 19:58
  • continues my comment... So when we are in state 0 we can only go to state 1 in an expected time $\frac{1}{ \lambda_0}$. If it is in another state then it is the probability that it will pass directly to the next state for the time it will take to move to that state (at this time I have my doubts if it is that $\frac{1}{\lambda_n}$) plus the probability that it will pass to the previous state for the time it takes to reach the state where it starts $E (T_ {n-1}) +$ the time it makes the transition to the other state $E (T_n)$. So is it correct? – Rosa Maria Gtz. Apr 20 '18 at 19:58
  • @Knight If you're describing how to calculate the time to hit $r+1$ starting from $n$, no that is not right. The expected time to hit $r+1$ starting from $n$ is the expected time of a jump (any jump, not necessarily a birth), plus the weighted average of the expected times after a jump, where the weights in the average are given by the distribution after a jump. So $E[T_n]=(\lambda_n+\mu_n)^{-1}+P_{n,n+1} E[T_{n+1}]+P_{n,n-1} E[T_{n-1}]$. – Ian Apr 20 '18 at 20:58
  • I could solve the tridiagonal linear system that you gave me and I solved the renewal system L L using this theory. Thank you very much. Can you help me with this renewal theory (https://math.stackexchange.com/questions/2756382/renewal-process-with-underlying-distribution) – Rosa Maria Gtz. May 02 '18 at 14:03