3

Consider the following probability function of a random variable $Y$: $$ f(y \mid \theta)=e^{-(y-\theta)},\quad y\ge\theta $$ and $0$ otherwise. We take a random sample $(Y_1,Y_2,...,Y_k)$ and want to find a sufficient statistic and a maximum likelihood estimator for $\theta$.

Now, the likelihood is given by $$ L\left(y_1, y_2, \ldots, y_k \mid \theta\right)=\prod_{i=1}^k e^{-\left(y_i-\theta\right)}=\exp \left(-\sum_{i=1}^k y_i+k \theta\right) $$

Obviously, this is maximized when $\theta$ is maximized. Since the density function is nonzero only when $y\ge\theta$, my first intuition is that the MLE for $\theta$ is $\min(y_1,y_2,...,y_k)$, although I am not sure that it is correct.

For the sufficient statistic, I believe we can choose $S=-\sum_{i=1}^k Y_i$, in which case the likelihood function can be written as the product of $g(s, \theta)=e^{s+k \theta}$ and $h(y_1,y_2,...,y_k)=1$, and a theorem then tells us that $S$ is a sufficient statistic.

Can someone tell me if I have made a mistake or misunderstood something?

Logi
  • 863
  • "and a theorem then tells us that $S$ is a sufficient statistic". What theorem? – lafinur Dec 24 '22 at 01:29
  • @lafinur: Let $U$ be a statistic based on the random sample $Y_1, Y_2, ..., Y_n$. Then $U$ is a sufficient statistic for $\theta$ iff the likelihood can be written as the product of a function $g(u,\theta)$ and a function $h(y_1,y_2,...,y_n)$. – Logi Dec 24 '22 at 01:34
  • That is correct by factorization theorem on sufficient statistics. – user45765 Dec 24 '22 at 01:41
  • @user45765, how about my intent of finding the MLE for $\theta$? – Logi Dec 24 '22 at 01:46
  • 1
    Sanity check: Your MLE is not a function of your sufficient statistic, so at least one of these is incorrect. – angryavian Dec 24 '22 at 01:47
  • @angryavian, maybe I did not describe well my objectives, but I do not think that the MLE and the sufficient statistic need to be linked. – Logi Dec 24 '22 at 01:53
  • @Logi I am wrong as I assumed that your likelihood function is correct here and the support is fixed. As angryavian suggested, you need to incorporate support in the distribution function. The point of sufficient statistics is to reduce the work of extracting $\theta$ to sufficient statistics. – user45765 Dec 24 '22 at 02:08

1 Answers1

4

Your argument for the maximum likelihood estimator is fine, since the likelihood is $$e^{k\theta -\sum_i y_i} \mathbf{1}_{\theta \le \min_i y_i}.$$

As I mentioned in a comment, your MLE $\min_i y_i$ (actually, any estimator) should be a function of any sufficient statistic (so, contrary to your comment, the MLE and sufficient statistics are definitely related). This is a fundamental property of sufficient statistics. If you don't believe me, see this excerpt from Wikipedia:

A sufficient statistic is a function of the data whose value contains all the information needed to compute any estimate of the parameter (e.g. a maximum likelihood estimate).

Since $\min_i y_i$ is not a function of $\sum_i y_i$, we see that $\sum_i y_i$ is not a sufficient statistic. This is a good lesson to always encode the support of densities with an indicator function (as I have above) before doing further operations like maximizing the likelihood or applying the Fisher-Neyman factorization theorem. With the indicator function, you can see that the factorization $$e^{k\theta} \mathbf{1}_{\theta \le \min_i y_i} \cdot e^{-\sum_i y_i} = g(\min_i y_i, \theta) \cdot h(y_1, \ldots, y_n)$$ shows that $\min_i y_i$ is a sufficient statistic.

angryavian
  • 93,534
  • It is not that I do not trust you, but I would like to understand what I have done wrong in the construction of my intented sufficient statistic. Theorem 9.4 of the book Mathematical Statistics with Applications, 7th Ed states the following: Let $U$ be a statistic based on the random sample $Y_1, Y_2, ...,Y_n$. Then $U$ is a a sufficient statistic for $\theta$ if and only if the likelihood can be written as a product $L=g(u,\theta)\times h(y_1,y_2,...,y_n)$, where $g(u,\theta)$ is a function of only $u$ and $\theta$, and $h(y_1, y_2,..., y_n)$ is not a function of $\theta$. – Logi Dec 24 '22 at 02:10
  • I believe my statistic satisfies that or is something going under my nose? – Logi Dec 24 '22 at 02:11
  • 1
    @Logi your choice of $g$ and $h$ do not have product equal to the likelihood $e^{k\theta - \sum_i y_i} \mathbf{1}_{\theta \le \min_i y_i}$. In particular it leads to a nonzero value when $\theta > \min_i y_i$. – angryavian Dec 24 '22 at 02:37
  • Ah, okay. I think I get it now. Thanks for your help, and merry Christmas. :) – Logi Dec 24 '22 at 15:11
  • The comment by Wikipedia is disputable. See this question: Why does a sufficient statistic contain all the information needed to compute any estimate of the parameter?. Relating to this example you could re-use your same argument the other way around "Since $\sum_i y_i$ is not a function of $\min_i y_i$ , we see that $\min_i y_i$ is not a sufficient statistic." It is true however that the MLE, if it exists, is a function of the sufficient statistic. – Sextus Empiricus Jun 17 '24 at 12:01
  • "the MLE, if it exists, is a function of the sufficient statistic." I am not sure about the other direction, the MLE may not need to be a sufficient statistic. It is the likelihood function that is a sufficient statistic. If the mapping from likelihood functions to MLE's is not injective, then the MLE is not a sufficient statistic. (I actually have to figure out an example with this, I am writing out this second comment based on intuition) – Sextus Empiricus Jun 17 '24 at 12:06