6

Suppose $X \sim Bin(n,p)$ and $Y \sim Bin(n,1-p)$, How is $X+Y$ distributed? I know that for independent variables one can do the same as:

Sum of two independent binomial variables

Furthermore i have seen this post:

Addition of two Binomial Distribution

However $X$ and $Y$ do not necessarily need to be independent of each other.

Backstory: I am trying to calculate the entropy of some $Z = X_1 + X_2 + \dots + X_n$ where each $X_i$ is either $Bin(n,p)$ or $Bin(n,1-p)$ depending on its parent node.

For example lets take a graph that has only outgoing edges of degree 2 beginning from some source $X_0 \sim (\frac{1}{2},\frac{1}{2})$. If we compare layer $2$ and layer $3$, we sent $2^3$ nodes to $2^4$ nodes. The probability for a set of child nodes to get certain states is $Bin(n,p)$ when the parent has state $1$ and $Bin(n,1-p)$ if the parent has state $-1$. We proceed this until we reached some threshold layer $d$.

What is the probability distribution of $Z = \sum X_i^{(d)}$

Kees Til
  • 1,978
  • 2
    Refer to this https://math.stackexchange.com/questions/1153576/addition-of-two-binomial-distribution – Ahmad Bazzi Jun 21 '18 at 15:47
  • 2
    Consider $X = n-Y.$ – William M. Jun 21 '18 at 15:55
  • hmmm considering your tip i dont see how to simplify

    $\sum_{k=0}^n P(X + Y = n) = \sum_{k=0}^n P(Y = i, X = n - i) = ?$

     – Kees Til Jun 21 '18 at 17:07
  • 1
    The title is odd since, already if $X$ and $Y$ are independent and $p\ne\frac12$, $X+Y$ is not binomial. – Did Jun 25 '18 at 18:26
  • i will edit this right away this was not what i meant thanks for the notice :D, any idea how i should go further with the generating probability functions at my answer to get for example $P(\sum X_i^{(3)} = 1)$ – Kees Til Jun 25 '18 at 18:40

2 Answers2

2

Assume a tree with binary offspring. Assume the root is generation 0. Take last generation $X_{n,1},\ldots,X_{n,2^n}$ be the bernoulli variables for generation, let $N_n = \sum X_{n,i}$, let $\phi_p$ be the characteristic function for $Bin(2,p)$, and $\phi_q$ the moment generation function for $Bin(2,q)$. Then we have (as can be seen by conditioning on the last generation) $$ E(\exp(sN_n)) = E\phi_p(s)^{X_{n-1,1}}\phi_q(s)^{1-X_{n-1,1}}\cdots = E\left(\phi_q(s)^{2^{n-1}}\left ( \frac{\phi_p(s)}{\phi_q(s)} \right ) ^{\sum_i X_{n-1,i}}\right) $$ Now this can be rewriten as $$ CE\exp(s_{n-1}N_{n-1}) $$ with $$ C = \phi_q(s)^{2^{n-1}} $$ and $$ s_{n-1} = \ln \left (\frac{\phi_p(s)}{\phi_q(s)} \right ) $$ Now use induction. If one is after probabilities consider using https://en.wikipedia.org/wiki/Probability-generating_function and note $$ Ez^N = E(\exp(ln(z) N)) $$

Stefan
  • 899
  • right moment generation function is the term. Sorry – Stefan Jun 21 '18 at 18:27
  • No problem, i almost undertstand it, however the term of $s_{n-1}$ is not clear to me. It seems you want to have that $s_{n-1} N_{n-1} = \left ( \frac{\phi_p(s)}{\phi_q(s)} \right ) ^{\sum_i X_{n-1,i}}\right) – Kees Til Jun 21 '18 at 18:59
  • 1
    The sum is $N_{n-1}$ and you essentially have $z^{N_{n-1}}$ now $z^x = e^{ln(z)x}$ – Stefan Jun 21 '18 at 19:07
  • I understand it know, however it is so abstract so im trying to write down an example where we go from layer 2 to layer 3. How do i calculate the probability that the sum is for example 4? – Kees Til Jun 21 '18 at 19:13
  • 1
    use the probability generating function and se the reference how to calculate the probability. Have fun! – Stefan Jun 21 '18 at 19:18
0

Assume a tree with binary offspring. Assume the root is generation 0. Take the last generation $X_{n,1},…,X_{n,2^n}$ and let $N_n=\sum X_{n,i}$. Then we see that:

$E[Z^{\sum X_{n,i}}] = E[Z^{X_{n,1}} \dots Z^{X_{n,2^n}}]$

if we condition on the $n-1$-th generation we with the law of total expectation https://en.wikipedia.org/wiki/Law_of_total_expectation, where G(Z)_q is the probability generating function of Bin(2,q):

$E[Z^{X_{n,1}} \dots Z^{X_{n,2^n}}] = E[E[Z^{X_{n,1}} \dots Z^{X_{n,2^n}}|X_{n-1}]] = E[G(z)_{q_1}^{X_{n-1,1}}G(z)_{q_{-1}}^{1-X_{n-1,1}} \dots G(z)_{q_1}^{X_{n-1,2^{n-1}}}G_{q_{-1}}^{1-X_{n-1,2^{n-1}}}] = \left(G(z)_{q_{-1}}\right)^{2^{n-1}}E[\left(\frac{G(z)_{q_1}}{G(z)_{q_-1}}\right)^{\sum X_{n-1,i}}] = \left((1-q_{-1}) + q_{-1} z\right)^{2^{n-1}}E[\left(\frac{G(z)_{q_1}}{G(z)_{q_-1}}\right)^{\sum X_{n-1,i}}]$

I don't know for sure if one can use the law of total expectation there and how one should write out the second expectation term.

Kees Til
  • 1,978