Uncorrelating random variables.

Question

I was reading this answer, and the first sentence seemed more intuitive at first than after thinking through it:

If $\pmatrix{X\\ Y}$ is bivariate normal with mean $\pmatrix{0\\0}$ and covariance matrix $\Sigma=\pmatrix{1&\rho\\\rho&1}$, then $\pmatrix{U\\V}=\Sigma^{-1/2} \pmatrix{X\\Y}$ is bivariate normal with mean $\pmatrix{0\\0}$ and covariance matrix $\pmatrix{1&0\\ 0&1}.$ That is, $U$ and $V$ are independent, standard normal random variables.

At first I thought that $\Sigma^{-1/2}= \begin{pmatrix}1&&\frac{1}{\sqrt{\rho}}\\\frac{1}{\sqrt{\rho}}&&1\end{pmatrix}$.

But this is clearly not the case as the answers so far explain.

This mistake corrected, I still would like to understand why:

$\frac{1}{2 \sqrt{1-\rho^2}}\begin{pmatrix}\sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho} \\ \sqrt{1-\rho}-\sqrt{1+\rho} & \sqrt{1-\rho}+\sqrt{1+\rho}\end{pmatrix}\begin{pmatrix}\mathbf X\\ \mathbf Y\end{pmatrix}$ manages to de-correlate $\bf X$ and $\bf Y$.

Or, $\frac{1}{2 \sqrt{1-\rho^2}}\begin{pmatrix}\sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho} \\ \sqrt{1-\rho}-\sqrt{1+\rho} & \sqrt{1-\rho}+\sqrt{1+\rho}\end{pmatrix} \mathbf A^T$ with $\mathbf A$ corresponding to the two correlated values arranged in two columns.

Answer 1

For a start:

$$\begin{bmatrix}1 & \rho\\\rho & 1\end{bmatrix}^{-1}~=~\dfrac1{1-\rho^2}\begin{bmatrix}-1 & \rho\\\rho & -1\end{bmatrix}$$

So

$$\begin{bmatrix}1 & \rho\\\rho & 1\end{bmatrix}^{-1/2}~=~\dfrac1{2}\begin{bmatrix}\dfrac 1{\sqrt{1+\rho}}+\dfrac 1{\sqrt{1-\rho}} & \dfrac 1{\sqrt{1+\rho}}-\dfrac 1{\sqrt{1-\rho}}\\\dfrac 1{\sqrt{1+\rho}}-\dfrac 1{\sqrt{1-\rho}} & \dfrac 1{\sqrt{1+\rho}}+\dfrac 1{\sqrt{1-\rho}}\end{bmatrix}$$

Answer 2

Your problem is that $\Sigma^{-1/2}$ is not $\begin{pmatrix}1 & \frac{1}{\sqrt{\rho}} \\ \frac{1}{\sqrt{\rho}} & 1 \end{pmatrix}$, it is a matrix $T$ such that $$T \cdot T \cdot \Sigma = \Sigma \cdot T \cdot T = \begin{pmatrix}1&0\\0&1\end{pmatrix},$$ where $\cdot$ denotes matrix multiplication.

One possible such $T$ is (credit to Wolfram Alpha):

$$T := \frac{1}{2 \sqrt{1-\rho^2}}\begin{pmatrix}\sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho} \\ \sqrt{1-\rho}-\sqrt{1+\rho} & \sqrt{1-\rho}+\sqrt{1+\rho}\end{pmatrix}.$$

Observe that $$T^2 = \frac{1}{1-\rho^2}\begin{pmatrix}1 & -\rho \\ -\rho & 1 \end{pmatrix},$$ which is the inverse of your $\Sigma$.

Answer 3

I did not read the answer you linked to, but here is what I think. Since the covariance is a bilinear form, under a change of basis $$ \begin{pmatrix} X' \\ Y' \end{pmatrix} = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} X \\ Y \end{pmatrix} $$the covariance matrix will transform as $$ \Sigma' = S \; \Sigma \; S^T $$ where $ S $ is the $ a,b,c,d $ matrix. So you want to find the $ S $ that will make $ \Sigma' $ identity. It is not hard to read off the eigenvectors and eigenvalues of $ \Sigma $, from which it follows that $$ S= \frac{1}{\sqrt{2}} \begin{pmatrix} \frac{1}{\sqrt{1 + \rho}} & \frac{1}{\sqrt{1 + \rho}} \\ -\frac{1}{\sqrt{1 - \rho}} & \frac{1}{\sqrt{1 - \rho}} \end{pmatrix} $$ I hope this will help!

Answer 4

When I computed this square-root matrix, I first started by diagonalizing it as, with $\Sigma^{-1/2} = PDP^{-1}$, $\Sigma^{-1/2}\Sigma^{-1/2} = PDP^{-1}PDP^{-1} = PD^2P^{-1} = \Sigma^{-1}$. Thus, as $D^2$ is diagonal, we know $D$ too.

We start with $\Sigma = \begin{pmatrix} 1 & \rho\\ \rho & 1 \end{pmatrix}$, thus $\Sigma^{-1} = \frac1{1-\rho^2} \begin{pmatrix} 1 & -\rho\\ -\rho & 1 \end{pmatrix}$.

Finding eigenvalues ($1-\rho$ and $1+\rho$ positive as they are the same as $\Sigma$'s) and the associated eigenvectors ($(1,1)^T)$ and $(1,-1)^T$) allows us to write, with $P = \begin{pmatrix} 1 & 1\\ 1 & -1 \end{pmatrix}$ (check that $P^{-1} = 1/2P$), $$\Sigma^{-1} = \frac1{1-\rho^2}P\begin{pmatrix} 1-\rho & 0\\ 0 & 1+\rho \end{pmatrix}P^{-1}.$$

We are allowed to take the square root of these eigen values, then, $$\Sigma^{-1/2} = \frac1{\sqrt{1-\rho^2}}P\begin{pmatrix} \sqrt{1-\rho} & 0\\ 0 & \sqrt{1+\rho} \end{pmatrix}P^{-1}.$$

This should yield to $$\Sigma^{-1/2} = \frac1{2\sqrt{1-\rho^2}}\begin{pmatrix} \sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho}\\ \sqrt{1+\rho}-\sqrt{1-\rho} & \sqrt{1-\rho}+\sqrt{1+\rho} \end{pmatrix}.$$

By squaring that, we end up on $\Sigma^{-1}$.

Now, computing the variance of $\Sigma^{-1/2}\begin{pmatrix}X\\Y\end{pmatrix}$ gives us a bivariate normal law with covariance matrix identity. Writing this density down as a separable function of $x$ and $y$, we get the independence.