Find the minimum norm least squares solution of Ax = b using QR factorization when $A \in R^{m×n}$ is not full rank

Question

• I can only solve this when $rank(A)=n$ (full rank)
  What I did, I derived the least square solution first and I got $\tilde{x}=(A^TA)^{-1}A^Tb$
  Then use $A=QR$, and finally, $\tilde{x}=R^{-1}Q^Tb$
• For $rank(A)=r<n$, the QR factorization will be $A =\begin{bmatrix}Q_1 & Q_2\end{bmatrix}$$\begin{bmatrix}R_1 & R_2\\0 & 0\end{bmatrix}$ , $Q_1 \in R^{m\times r},R_1\in R^{r\times r}$
  It seems like I cannot solve it the same way as $rank(A)=n$ or am I wrong? I don't know how to move forward. Please show me the way. Thank you.

Answer 1

Let $A=QR$ be the QR decompisition of the matrix $A_{m\times n}$, and $$u=\arg\min_x\|Ax-b\|^2,\qquad \min_x\|Ax-b\|^2\neq 0.$$ It follows that $b\not\in Im(A)$, $rank(A)<m$, and $rank(A)=r< n$.

Let us suppouse that $n\leq m$. Then $$Q =\begin{bmatrix}Q_1 & Q_2\end{bmatrix},\qquad R=\begin{bmatrix}R_1 \\ 0\end{bmatrix}$$ and $$\|Ax-b\|^2=\|QRx-b\|^2=\|Q^*(QRx-b)\|^2=\|Rx-Q^*b\|^2.$$

If we denote $$Qb=\begin{bmatrix}c_1 \\c_2\end{bmatrix},\quad Rx=\begin{bmatrix}R_1 \\ 0\end{bmatrix}x=\begin{bmatrix}R_1x\\ 0\end{bmatrix}.$$

Therefore $$u=\arg\min_x\|Ax-b\|^2=\arg\min_x\|Rx-Q^*b\|^2=\arg\min_x\|R_1x-c_1\|^2+\|c_2\|^2.$$

Perhaps you can find some related results searching for "\(A=QR\) generalized inverse" on SearchOnMath, like Moore-Penrose pseudoinverse as an orthogonal projection, for instance. Please see also this discussion.