11

Consider a matrix $A \in\mathbb R^{n\times m}$ with $n>m$. It has full column rank, i.e. $\operatorname{rank}(A)=m$. Its left pseudo-inverse is given by; $$A^{-1}_\text{left}=(A^TA)^{-1}A^T $$

From two different results during my studies, I have realized the following: $$ \|A^{-1}_\text{left}\|_2 = \frac{1}{\sigma_{\min}(A)} $$ just like the case as if $A$ is square invertible matrix.

I have seen a similar question, however I couldn't relate the answer with the equality given above.

My question is: How can we show that the L2 norm of left pseudo-inverse of $A$ is related to its minimum singular value?

Thank you in advance for your help.

Community
  • 1
Metehan Yayla
  • 364

2 Answers2

8

Hint: It suffices to prove the following facts:

  • If $U,V$ are orthogonal (and square), then $\|AV\| = \|UA\| = \|A\|$
  • $(U\Sigma V^T)^\dagger = V\Sigma^{\dagger}U^T$ (where $\dagger$ denotes the pseudo-inverse)
  • If $\Sigma$ is a diagonal matrix of singular values, then $\|\Sigma^{\dagger}\| = 1/\sigma_{min}(\Sigma)$

Perhaps you can put the pieces together from here. Happily, this approach still works when $A$ does not have full column-rank.

Ben Grossmann
  • 234,171
  • 12
  • 184
  • 355
2

For brevity, denote this left inverse $A^{-1}_{\text{left}}$ by $B$. It is well-known that $AB$ is the orthogonal projection onto the column space of $A$. Therefore, $ABy\perp(I-AB)y$ for every vector $y$. It follows that \begin{aligned} \|B\|_2&=\max_{y\ne0}\frac{\|By\|}{\|y\|}\\ &=\max_{y\ne0}\frac{\|By\|}{\|ABy+(I-AB)y\|}\\ &=\max_{By\ne0}\frac{\|By\|}{\|ABy+(I-AB)y\|}\\ &\le\max_{By\ne0}\frac{\|By\|}{\|ABy\|}\\ &\le\max_{x\ne0}\frac{\|x\|}{\|Ax\|}\\ &=\left(\min_{x\ne0}\frac{\|Ax\|}{\|x\|}\right)^{-1}\\ &=\frac{1}{\sigma_\min(A)}. \end{aligned}

user1551
  • 149,263