Conditional expectation of exponential random variable given minimum of sample

Question

Suppose $X_1, \dots , X_n \sim f_{\theta}(x) = e^{-(x-\theta)}$ are IID.

I'm interested in the conditional expectation $E[X_1 | X_{(1)} = t]$, where $X_{(1)} = \min(X_1, \dots, X_n)$.

Let $f_{X_1|T} (x|t)$ be the conditional pdf of $X_1$ given $X_{(1)} = t$. By definition:

\begin{equation} f_{X_1|T} (x|t) = \frac{f_{X_1,T} (x,t)}{f_T(t)} \end{equation}

I have $f_T(t) = ne^{-n(t-\theta)}$ for $t \in (\theta, \infty)$.

For the joint pdf $f_{X_1,T}$ I consider two cases.

If $X_1 = X_{(1)}$, then I argue $f_{X_1,T}(x,t)$ is $0$ almost everywhere because it only has support on the diagonal where $x=t$.

If $X_1 \neq X_{(1)}$, then $X_{(1)}$ is some $X_i$ for $i \neq 1$ and thus $X_1, X_{(1)}$ are independent. In this case the joint pdf splits:

\begin{equation} f_{X_1|T} (x|t) = \frac{f_\theta(x)f_T(t)}{f_T(t)} = f_\theta(x) \end{equation}

Thus $E[X_1 | X_{(1)} = t] = E[X_1] = \int_{\theta}^\infty xe^{-(x-\theta)}dx = [-xe^{-(x-\theta)} - e^{-(x-\theta)}]_\theta^\infty = \theta + 1$.

Intuitively, this would mean knowing the minimum of the sample doesn't give me any information about $X_1$.

Is this argument correct?

Answer 1

There is no joint density of $(X_1,X_{(1)})$ in the usual sense (with respect to Lebesgue measure) because $X_1=X_{(1)}$ occurs with a positive probability of $\frac1n$.

Here, as @user806050 suggests, you can use the law of total expectation.

For any $t\in [\theta,\infty)$, you have

\begin{align} E\left[X_1\mid X_{(1)}=t\right]&=E\left[X_1\mid X_1=t\right]\cdot\frac1n + E\left[X_1\mid X_1>t\right]\cdot\left(1-\frac1n\right) \\&=\frac{t}{n}+(1+t)\left(1-\frac1n\right) \\&=t+1-\frac1n. \end{align}

That $E\left[X_1\mid X_1>t\right]=1+t$ can be shown directly from $E\left[X_1\mid X_1>t\right]=\frac{E\left[X_1\mathbf1_{X_1>t}\right]}{P(X_1>t)}$ or equivalently by noting that the conditional distribution of $X_1$ given $X_1>t$ is another shifted exponential with shift $t$:

$$f_{X_1\mid X_1>t}(x)=e^{-(x-t)}\mathbf1_{x>t}.$$

Hence, $$E\left[X_1\mid X_{(1)}\right]=X_{(1)}+1-\frac1n \quad,\text{ a.e. }$$

There are also other indirect ways of finding this conditional expectation (compare with Computing the UMVUE for Uniform$(0,\theta)$). For more formal derivations, have a look at Finding an efficient estimator for $\theta$ in $U[0, \theta]$ in terms of the sample maximum and Finding $E(|X_1|\ |\max |X_i| )$ for similar discussions on a $\text{Uniform}(0,\theta)$ model.

Edit:

In case it is not clear, the law of total expectation is used in the following way:

$$E\left[X_1\mid X_{(1)}=t\right]=E\left[X_1\mid X_{(1)}=t,A \right]P(A\mid X_{(1)}=t)+E\left[X_1\mid X_{(1)}=t,A^c\right]P(A^c\mid X_{(1)}=t)\,,$$ where $A$ is the event $\{X_1=X_{(1)}\}$.

---

As an aside, here $X_1-X_{(1)}$ happens to be independent of $X_{(1)}$, which can be argued using Basu's theorem. This immediately gives $$E\left[X_1\mid X_{(1)}\right]=E\left[X_1-X_{(1)}\mid X_{(1)}\right]+X_{(1)}=E\left[X_1-X_{(1)}\right]+X_{(1)}\,,$$ and all that remains is to find $E\left[X_1\right]$ and $E\left[X_{(1)}\right]$.