Prove that the estimators are biased.

Question

Given the following sentences:

Let $X_1,..., X_n$ be a random sample from a $Pois(\mu)$ distribution. Consider the following estimator for $e^{-\mu}=P(X_i=0)$: $T=e^{-\overline{X_n}}$.

The independent random variables $X_1;...;X_n$ have a geometric distribution with parameter $p$. Look at the following estimator for $p$: $S=\frac{1}{\overline{X_n}}$.

Prove that the estimators are biased.

In my opinion both estimators are unbiased:
$E[T]=e^{E[\overline{X_n}]}=e^{-\mu}$ that is unbiased for the parameter $e^{-\mu}$.
$E[S]=\frac{1}{E[\overline{X_n}]}=\frac{1}{1/p}=p$ that is unbiased for the parameter $p$.
Why I'm wrong in both cases? Where are my mistakes? Thanks.

Answer 1

You can't write$$E\left\{e^{\overline{X_n}}\right\}=e^{E\{\overline{X_n}\}}$$but we have $$E\left\{e^{\overline{X_n}}\right\}{=E\left\{e^{X_1\over n}e^{X_2\over n}\cdots e^{X_n\over n}\right\}\\=\left(E\left\{e^{X_1\over n}\right\}\right)^n\\=(\exp(\mu(e^{1\over n}-1)))^n\\=\exp\Big(\mu n(\sqrt[n]e-1)\Big)}$$according to Poisson distribution which is biased.

As with the first one, we can argue similarly for the second one as following$$E\{S\}{=E\left\{{n\over X_1+\cdots + X_n}\right\}\\=nE\left\{{1\over X_1+\cdots + X_n}\right\}\\=nE\left\{{1\over Y}\right\}}$$where $Y$ has negative binomial distribution with parameters $p $ and $n$. Further calculations are nasty in this case but you can refer to Geometric distribution section Parameter estimation.

Answer 2

The problem is that the exponential and the reciprocal aren't linear. You can't get an unbiased estimator for the standard deviation by taking the square root of an unbiased estimator for the variance, and we have the same problem here.

In order to see these examples more clearly, consider one-element samples. In the Poisson example, we get a probability of $e^{-\mu}$ of sample value $0$ and estimate $1$, a probability of $\mu e^{-\mu}$ of sample value $1$ and estimate $e$, a probability of $\frac{\mu^2}{2}e^{mu}$ of sample value $2$ and estimate $e^2$, and so on. Add them up, and the expected value of what we get is a value of the moment-generating function $\sum_{n=0}^{\infty} \frac{\mu^n e^n}{n!}e^{-\mu} = e^{\mu(e-1)}$. That's not the mean we wanted, and there's no simple way to correct it.

The second example, on the geometric distribution, has similar issues. The probability of getting $n$ is $(1-p)\cdot p^n$, so our one-element estimate returns $\frac1n$ with that probability. Sum that, and we get $-(1-p)\ln(1-p)$. That's definitely not what we want - it goes to zero as $p\to 1^-$.

Answer 3

Here $n < +\infty$. We compute $\Bbb E (\exp(\bar X_n))$ where $\bar X_n$ is the mean of a i.i.d. sample $(X_1,\dots,X_n)$ with distribution $\text{Poisson}(\mu)$: \begin{aligned} \Bbb E (\exp(\bar X_n)) &= \Bbb E \left(\prod_{i=1}^n \exp \frac{X_i}{n}\right) \\ &= \prod_{i=1}^n \Bbb E \left( \exp \frac{X_i}{n}\right) \\ &= \Bbb E \left( \exp \frac{X_1}{n}\right)^n \\ &= \left(e^{-\mu}\sum_{k=0}^\infty \frac{(e^{1/n} \mu)^k}{k!} \right)^n \\ &= \exp\left(n (e^{1/n}-1)\mu\right) & \neq \exp (\mu) \end{aligned} One can note that the bias vanishes as $n\to +\infty$. For the second example, one can prove that the estimator is biased by using Jensen's inequality (see this post).