22

It is well known that eigenvalues for real symmetric matrices are the same for matrices $A$ and its transpose $A^\dagger$.

This made me wonder: Can I say the same about the singular values of a rectangular matrix? So basically, are the eigenvalues of $B B^\dagger$ the same as those of $B^\dagger B$?

glS
  • 7,963
Lagerbaer
  • 3,556
  • I know that $BB^T$ is not the transpose of $B^T B$. I was using the definition of the singular values of $B$ as the square-root of the eigenvalues of $B^T B$. Then, the singular values of $B^T$ should be $B B^T$, shouldn't they? – Lagerbaer Mar 31 '11 at 04:56
  • Oh, okay then. Sorry about that. – Arturo Magidin Mar 31 '11 at 04:58

1 Answers1

29

Both eigenvalues and singular values are invariant to matrix transpose no matter a matrix is square or rectangular.

The definition of eigenvalues of $A$ (must be square) is the $\lambda$ makes $$\det(\lambda I-A)=0$$ For $A^T$, $\det(\lambda I-A^T)=0$ is equivalent to $\det(\lambda I-A)=0$ since the determinant is invariant to matrix transpose. However, transpose does changes the eigenvectors.

It can also be demonstrated using Singular Value Decomposition. A matrix $A$ no matter square or rectangular can be decomposed as $$A=U\Sigma V^T$$ Its transpose can be decomposed as $A^T=V \Sigma^T U^T$. The transpose changes the singular vectors. But the singular values are persevered.

Shiyu
  • 5,378
  • Thanks. I should have looked at the SVD from the beginning. – Lagerbaer Mar 31 '11 at 04:58
  • 2
    Maybe it's a bit pedantic, but since it confused me for a moment maybe it's worth to point it out: if you define the singular values of $A$ as the square roots of the eigenvalues of $A^TA$ - as the OP does in the comment - then this is not true in general. A could have zero as singular value and $A^T$ not (or viceversa)...you probably define here singular values as square roots of the NON-ZERO eigenvalues of $A^TA$. – Hans Aug 20 '12 at 20:43
  • 3
    You brought up a good point. I didn't notice this before. I also would like to add a point. The most exact definition is this: non-zero singular values of $A$ are the square roots of the non-zero eigenvalues of $A^TA$ or $AA^T$. Note the singular value can also be zero. – Shiyu Aug 24 '12 at 03:24
  • 1
    "since the determinant is invariant to matrix transpose" doesn't seem sufficient. That shows $A$ and $A^T$ would have the same determinant, but those are not the matrices we're taking the determinant of -- we're subtracting those from $\lambda I$. For that to apply you would have to demonstrate that $(\lambda I - A)^T = \lambda I - A^T$. – Joseph Garvin Jan 29 '18 at 01:20
  • ... which I've now convinced myself must be true because the elements on the diagonal stay the same under transpose, and adding/subtracing $I$ can only affect those. Still this comment may help someone else understand ;) – Joseph Garvin Jan 29 '18 at 01:30
  • In this answer, https://stats.stackexchange.com/a/134283/134555, @amoeba used $S$ (same as $\Sigma$ here) and $S^T$ the way that looks like they are replaceable. Is it correct? – Belter Jan 01 '22 at 14:24