1

Let $A$ be an $m\times n$ matrix that describes an injective linear transformation $T_A:\mathbb{R}^n\to\mathbb{R}^m$ and fix $b\in\mathbb{R}^n$ then there exists a unique $x\in\mathbb{R}^n$ that minimizes $\|Ax-b\|$. And then show that $x$ is the unique solution of the system $A^tAx=A^tb$.

Now the solution is trivial if $b$ is in the image of $T_A$. Now I couldn't figure out what to do for the other case.

UserA
  • 1,699

2 Answers2

4

Hint: $\text{Im }T_A$ will be a vector subspace of $\Bbb R^m$. Think in the orthogonal projection of $b$ over $\text{Im }T_A$.

Community
  • 1
Martín-Blas Pérez Pinilla
  • 42,722
  • 4
  • 51
  • 94
1

The fact that there exists a unique $\hat x\in\mathbb R^n$ which minimizes $\|Ax-b\|$ is a special case of the Hilbert projection theorem. In particular, $A\hat x$ is the orthogonal projection of $b$ onto $\mathsf{Im}(T_A)$. Since $\mathbb R^m = \mathsf{Im}(T_A)\oplus \mathsf{Im}(T_A)^\perp$, it follows that $b-A\hat x\in\mathsf{Im}(T_A)^\perp$. Since $\mathsf{Im}(T_A)^\perp=\mathsf{Ker}(T_{A^T})$, we have $A^T(b-A\hat x)=0$ and hence $A^TA\hat x=A^T b$. $A$ is injective, so $A$ and $A^T$ are invertible, and hence we may write $\hat x$ uniquely as $$\hat x = (A^TA)^{-1}A^Tb. $$

Math1000
  • 38,041
  • The Hilbert theorem requires a closed and convex subset $C$ of $\mathbb{R}^m$ and does not use injectivity of $T_A$. I don't see how it can be used here. @Math1000 – UserA Apr 05 '16 at 01:54
  • @AdelSaleh Every subspace of a normed vector space is convex (due to closure under vector addition), and every subspace of a finite-dimensional space is closed (as a finite-dimensional space is complete). So $\mathsf{Im}(T_A)$ is indeed a closed and convex subset of $\mathbb R^m$. – Math1000 Apr 05 '16 at 02:26