Conditional expectation for multivariate normal: Independence trick

Question

Let $n\ge2$ be an integer, let $\Sigma$ be a positive semidefinite, symmetric $n\times n$ matrix of real numbers partitioned as $$\Sigma=\begin{pmatrix}\Sigma_{a,a}&\Sigma_{a,b}\\\Sigma_{b,a}&\Sigma_{b,b}\end{pmatrix},$$ where $\Sigma_{a,a}$ is $1\times1$ and $\Sigma_{b,b}$ is $(n-1)\times(n-1),$ assume $\Sigma_{b,b}$ is positive definite (i.e., invertible) and let $X=(X_1,\dots,X_n)$ be $N(0,\Sigma),$ normal with mean zero and covariance matrix $\Sigma.$ I wish to find $E(X_1\mid X_2,\dots,X_n).$ In addition, I am using the Radon-Nikodym-derivative definition of conditional expectation, so I would prefer not to compute conditional densities $f_{X_a\mid X_b}(x_a\mid x_b)=f_{X_a,X_b}(x_a,x_b)/f_{X_b}(x_b).$

From Conditional Expectation Multivariate Normal, I can guess that $E(X_1\mid X_2,\dots,X_n)=\Sigma_{a,b}\Sigma_{b,b}^{-1}(X_2,\dots,X_n)^T.$ To prove this result, I tried reasoning as follows, similar to user357269's answer to "Conditional expectation of a joint normal distribution": If $X_1-\Sigma_{a,b}\Sigma_{b,b}^{-1}(X_2,\dots,X_n)^T$ and $\sigma(X_2,\dots,X_n)$ are independent, then we have $$E(X_1\mid X_2,\dots,X_n)$$ $$=E(X_1-\Sigma_{a,b}\Sigma_{b,b}^{-1}(X_2,\dots,X_n)^T+\Sigma_{a,b}\Sigma_{b,b}^{-1}(X_2,\dots,X_n)^T\mid X_2,\dots,X_n)$$ $$=E(X_1-\Sigma_{a,b}\Sigma_{b,b}^{-1}(X_2,\dots,X_n)^T)+\Sigma_{a,b}\Sigma_{b,b}^{-1}(X_2,\dots,X_n)^T=\Sigma_{a,b}\Sigma_{b,b}^{-1}(X_2,\dots,X_n)^T,$$ where the last equality follows from $EX_1=0$ and $E((X_2,\dots,X_n))=0.$

However, I am stuck on showing independence. For the case $n=2,$ we can compute the covariance $\text{Cov}(X_1-\Sigma_{a,b}\Sigma_{b,b}^{-1}X_2,X_2)=0$ and appeal to a theorem. However, I am unsure what to do for larger $n,$ since $(X_2,\dots,X_n)$ is vector-valued rather than real-valued.

Answer 1

Let $V_1:=X_1-\Sigma_{a,b}\Sigma_{b,b}^{-1}(X_2,\dots,X_n)^T$; then $X':=(V_1,X_2,\dots,X_n)$ is also Gaussian and its covariance matrix has the form $$ \Sigma'=\begin{pmatrix}\Sigma'_{a,a}&0 \\ 0&\Sigma_{b,b}\end{pmatrix}. $$ Writing the density of this new Gaussian vector, we can see that we can factorize with respect to the first variable and the independence between $V_1$ and $(X_2,\dots,X_n)$ follows.

Answer 2

There is a $1\times n$ matrix $A$ such that

$$ A(X_1,\ldots,X_n)^\top = \big(X_1-\Sigma_{a,b}\Sigma_{b,b}^{-1}(X_2,\dots,X_n)^T, \, X_2, \, X_3, \, \ldots, \, X_n\big) $$ The covariance matrix of $A(X_1,\ldots,X_n)^\top$ is $A\Sigma A^\top.$ If you observe that all entries in the first row and first column of this matrix are $0$ except the variance of the first component in the random vector, then that implies something about the factorization of the joint density function.

Let us look at a useful definition and a useful lemma:

Definition: $$ \operatorname{cov}\left( \left[ \begin{array}{c} Y_1 \\ \vdots \\ Y_m \end{array} \right], \left[ \begin{array}{c} X_1 \\ \vdots \\ X_n \end{array} \right] \right) = \text{a certain } m\times n \text{ matrix}. $$ (Details are an exercise.)

Definition: $$ \operatorname{var}\left[ \begin{array}{c} Y_1 \\ \vdots\\ Y_m \\ X_1 \\ \vdots \\ X_n \end{array} \right] = \text{a certain } (m+n)\times(m+n) \text{ matrix}. $$ The former matrix is found within the latter.

Lemma:

If the former matrix is the $m\times n$ zero matrix, then one can deduce something about factoring the multivariate normal density, and hence about independence.

You wrote:

For the case $n=2,$ we can compute the covariance ... and appeal to a theorem.

But it is not only in the case $n=2$ that that works.