0

Theorem (Best Linear Prediction of $Y$ outcomes): Let $(X,Y)$ have moments of at least the second order, and let $Y'=a+bX$. Then the choices of $a$ and $b$ that minimize $Ed^2(Y,Y')=E(Y-(a+bX))^2$ are given by $$a= \left(E(Y) - \dfrac{cov(x,y)}{var(x)}\right)E(X)$$ and $$b=\dfrac{cov(x,y)}{var(x)}$$

Proof: Left to the reader.

I want to prove this theorem, so I see that this $a$ and $b$ are very similar to the case when correlation is equal $1$, except that $cov(x,y)$ is not $= std(x)std(y)$, but I can do no more.

Also, below the theorem: Now define $V=Y-Y'$ to represent deviation....Since $EY=EY'$, $EV=0$ (there is no mention of why $EV=0$) $std(Y) = var(Y')+var(V)+Cov(Y',V)$ (I get this one)

where: $var(V)=EV^2=Ed^2(Y,Y')=Ed^2(Y,a+bX)=var(Y)-\dfrac{cov(X,Y)^2}{var(X)^2}$ (Why?Why?Why? I have sat down for nearly 1 hour and can't understand this expression. Sometimes, the book exceeds too fast that I can't understand)

Empy2
  • 52,372
SON TO
  • 653
  • 5
  • 13

1 Answers1

1

We want to minimize the expected value of $(Y-a-bX)^2$. That expands to $Y^2+a^2+(bX)^2-2aY-2bXY+2abX$. So we want to minimize $$E(Y^2)+a^2+b^2E(X^2)-2aE(Y)-2bE(XY)+2abE(X)$$ That is a quadratic in $a$ and $b$. Do you know how to minimize that?

For the answer, you don't want the large brackets in the expression for $a$.
Then $E(Y')=E(a+bX)=a+bE(X)=E(Y)-bE(X)+bE(X)=E(Y)$.
So $E(V)=E(Y-Y')=E(Y)-E(Y')=0$.

Empy2
  • 52,372
  • I am sorry, but this does not seem the quadratic because the form is ax^2+bxy+cy^2=0... – SON TO Jul 15 '13 at 14:32
  • Also do you know how to derive var(V). I have not been able to derive that var(V). – SON TO Jul 15 '13 at 14:33
  • In order to minimize the quadratic, we set the determinant coefficient ad-bc>=0, right? – SON TO Jul 15 '13 at 14:41
  • @SONTO The extreme value of a quadratic occurs at the average of the roots of the quadratic. For $Ax^2 + Bx + C$, this average is $-\frac{B}{2A}$. Note that $C$ is irrelevant. This will give you one equation relating $a$ and $b$. The other equation comes from noting that $E[(Z-\alpha)^2]$ has minimum value $\operatorname{var}(Z)$ at $\alpha = E[Z]$, that is, if $Z$ is chosen such that $E[Z-\alpha] = 0$. Thus, we need to have $a$ and $b$ such that $E[Y-a-bX] = 0$. – Dilip Sarwate Jul 15 '13 at 16:20