I know that if I want to minimize $ |Ax -b| $ then I have to solve $ A^T (Ax - b) = 0 $ because the error term $ (Ax - b)$ suppose to be orthogonal to the columns of $A$ and the justification to this is typically given by geometrical arguments in 3 dimensions (Protagoras theorem). Can someone provide a pure algebraic argument why this is true ?
Asked
Active
Viewed 28 times
0
-
The solution here is purely algebraic (linear algebra) and it is based on orthogonal projections on to sublunar spaces Prove uniqueness of solutions of different OLS matrix cases – Mittens Mar 27 '22 at 00:05
-
I guess I don't understand why $ \nabla | Ax -b | = 2 A^T (Ax -b ) $ – Tomer Mar 27 '22 at 00:13
-
That is must likely the gradient of a function. That solution uses a little bit of Calculus. The link here you uses only linear algebra (geometric) methods. – Mittens Mar 27 '22 at 00:14