Matrix notation for weighted sum of squares

Question

While going through page 1 of Lecture 24: Weighted and Generalized Least Squares [PDF], I got the following questions. Weighted sum of squares is defined as below: $$ \sum_{i = 0}^{n}{w_i(Y_i - X_ib)^2}$$ And this could be written in matrix notation as follows $$(Y−Xb)'W(Y−Xb)$$

Only thing I did not get is how $W$ (weights) got into the middle? And why should it be a diagonal matrix?

I get that $\sum_{}x^2$ could be written as $X^T * X $. But how $\sum_{}wx^2$ is written as $X^TWX$ , any clue?

@RodrigodeAzevedo Thank you for the edits, I get that $\sum_{}x^2$ could be written as $X^T * X $. But how $\sum_{}wx^2$ is written as $X^TWX$ , any clue?? — Sonnu, Jul 08 '18 at 00:51
To weight the rows of a matrix, left-multiply by a diagonal matrix. To weight the columns of a matrix, right-multiply by a diagonal matrix. — Rodrigo de Azevedo, Jul 08 '18 at 00:57

score 2 · Answer 1 · answered Jul 09 '18 at 03:57

\begin{align} \begin{bmatrix} x_1 & x_2 & \cdots & x_n \end{bmatrix} \begin{bmatrix} w_1 & 0 & \cdots & 0 \\ 0 & w_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & w_n \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} &= \begin{bmatrix} x_1 & x_2 & \cdots & x_n \end{bmatrix} \begin{bmatrix} w_1x_1 \\ w_2x_2 \\ \vdots \\ w_nx_n \end{bmatrix} \\[3mm] &= \sum_{i=1}^n w_i x_i^2 \end{align}

Matrix notation for weighted sum of squares

1 Answers1