A program is divided into 3 blocks that are being compiled on 3 parallel computers. Each block takes an Exponential amount of time, 5 minutes on the average, independently of other blocks. The program is completed when all the blocks are compiled. Compute the expected time it takes the program to be compiled.
I know X follows an exponential distribution, so E(X) = 1/λ. But how do I define "λ" based on the parameters? The answer should compute to 55/6.
Let there are $n$ blocks to be compiled on parallel computers. Each of the compiled time $X_i$ are independent to each other and have the common exponential distribution with same mean $1/\lambda$
The program is completed when every block is compiled, so the completion time is $$ \max\{X_1, X_2, \ldots, X_n\} $$
So the expected value is
$$ \begin{align} E[\max\{X_1, X_2, \ldots, X_n\}] &= \int_0^{+\infty} \Pr\{\max\{X_1, X_2, \ldots, X_n\} > x\} dx \\ &= \int_0^{+\infty} 1 - \Pr\{\max\{X_1, X_2, \ldots, X_n\} \leq x\} dx \\ &= \int_0^{+\infty} 1 - \Pr\left\{\bigcap_{i=1}^n X_i \leq x\right\} dx \\ &= \int_0^{+\infty} 1 - \prod_{i=1}^n \Pr\{ X_i \leq x\} dx \\ &= \int_0^{+\infty} 1 - \prod_{i=1}^n(1 - e^{-\lambda x}) dx \\ &= \int_0^{+\infty} 1 - (1 - e^{-\lambda x})^n dx \\ &= \int_0^{+\infty} 1 - \sum_{i=0}^n \binom {n} {i} (-1)^i e^{-i\lambda x} dx \\ &= \sum_{i=1}^n \binom {n} {i} (-1)^{i+1} \int_0^{+\infty} e^{-i\lambda x} dx \\ &= \sum_{i=1}^n \binom {n} {i} (-1)^{i+1} \frac {1} {i \lambda} \\ &= \frac {1} {\lambda} \sum_{i=1}^n \binom {n} {i} \frac {(-1)^{i+1}} {i} \end{align} $$
So when $n = 3$, the expected value is
$$ \frac {1} {\lambda} \left[\binom {3} {1} \frac {1} {1} - \binom {3} {2} \frac {1} {2} + \binom {3} {3} \frac {1} {3}\right] = \frac {11} {6} \frac {1} {\lambda} $$
That is $11/6$ times the individual mean, and that is why you got $55/6$ minutes in your answer when individual mean is $5$ minutes.
On the other hand, the expected value of the minimum of i.i.d. exponential is still exponential with mean $1/(n\lambda)$. Here $\min\{X_1, X_2, \ldots, X_n\}$ represent the minimum compile time of the $n$ blocks (the quickest one). After the first one is compiled, by the memory-less property of exponential, the remaining $n-1$ compile time reset and we wait for the second compile time, which again is a minimum of exponential, with mean $1/((n-1)\lambda)$. This process repeat until we wait the last one finish compile, so inductively
$$E[\max\{X_1, X_2, \ldots, X_n\}] = \frac {1} {\lambda} \sum_{i=1}^n \frac {1} {i} $$
Note that both expression are identical. The latter one here shows that the mean of the maximum is multiplying the $n$-th harmonic number to the individual mean.
