The matrix equation for linear regression is: $$ \vec{y} = X\vec{\beta}+\vec{\epsilon} $$ The Least Square Error solution of this forms the normal equations:
$$ ({\bf{X}}^T \bf{X}) \vec{\beta}= {\bf{X}}^T \vec{y} $$
Using the Moore-Penrose pseudoinverse: $$ X^+ = ({\bf{X}}^T \bf{X})^{-1}{\bf{X}}^T $$ this can be written as: $$ \vec{\beta}= A^+ \vec{y} $$ Decomposing using SVD: $$ X=U\Sigma V^T$$ alledgely leads to: $$ X^+=V \Sigma^+U^T $$
where the pseudoinverse of $ \Sigma $ is just found by taking the inverse of all nonzero diagonal elements.
The point of doing things this way is that I will be doing linear regression n >> 1 times and $\vec{y}$ will vary but X (and therefore $X^+$ also) will be constant.
In particular I am interested in doing "scaled" linear regression: $$ \vec{y} = WX\vec{\beta}+\vec{\epsilon} $$ where W is a diagonal matrix (and in my case the diagonal elements are integer values) which change over n. The reason for this is that sometimes I have multiple measurements of $y_i$ for the same X.
I understand that I could solve this using the pseudoinverse of WX but this pseudoinverse would then have to be found for each n and that defeats the point.
In the case of scaled linear regression the normal equations are: $$ ({\bf{X}}^T \bf{W}^2 \bf{X}) \vec{\beta}= ({\bf{X}}^T \bf{W}^T) \vec{y} $$
If I substitute SVD into this equation what is then the equation for $ \vec{\beta} $?
I am hoping for some cheap operation so that: $$ \vec{\beta}= operation(W,X^+,\vec{y}) $$
As an example something like $$ \vec{\beta}= (WX^+)\vec{y} $$ would be ideal (this example is probable not correct though), since I would just find the pseudoinverse of X once (costly) and then multiply it with a diagonal matrix and a vector (cheap) for each n.
