Characteristic polynomials identity as corollary of a general identity

Question

Well, given two matrices $A,B$ of size $k\times n$ and $n\times k$ the characteristic polynomials of $AB$ and $BA$ are related, i.e., $\chi_{BA}=\lambda^{n-k}\chi_{AB}$. There are a lot of nice proofs on math.stackexchange, but I found one on math.overflow here: highest score answer, which is presented as corollary of the following general identity on rings: $(1-ba)^{-1}=1+b(1-ab)^{-1}a$. The identity on characteristic polynomials should follow from $(\lambda I_{n}-BA)^{-1}=\lambda^{-1}(I_n+B(\lambda I_k-AB)^{-1}A)$. Of course $\det(\lambda I_{n}-BA)^{-1}=\lambda^{-n}\det(I_n+B(\lambda I_k-AB)^{-1}A)$ and, in order to get the desidered equality, $\det(I_n+B(\lambda I_k-AB)^{-1}A)$ should be equal to $\lambda^{k}\det(\lambda I_k-AB)^{-1}$ but, no matter how much I wrap my head around this, I can't find a way to prove it.

Answer 1

This may not fully satisfy you as an answer, but it was a little too long for a comment:

If you're willing to use the Sylvester determinant identity, or first prove it which can be done in a variety of ways, then you have $$\det(I_n+B(\lambda I_k-AB)^{-1}A)=\det(I_k+AB(\lambda I_k-AB)^{-1})$$ But $$I_k+AB(\lambda I_k-AB)^{-1}=I_k+(\lambda I_k-(\lambda I_k-AB))(\lambda I_k-AB)^{-1}=\lambda(\lambda I_k-AB)^{-1}$$ so $$\det(I_n+B(\lambda I_k-AB)^{-1}A)=\det(\lambda(\lambda I_k-AB)^{-1})=\lambda^k\det(\lambda I_k-AB)^{-1}$$ which is the thing you wanted.

Answer 2

Case of square matrices ($n=k$) and $A$, $B$ invertible:

The inversion formula can be rewritten as $$ (\lambda I_n - BA)^{-1}BA = B(\lambda I_k - AB)^{-1}A. $$ Without any assumptions, we can take determinants to get $$\tag{1} \det(\lambda I_n - BA)^{-1}\det (BA) = \det(B(\lambda I_k - AB)^{-1}A). $$ In the special case of square matrices ($n=k$) and assuming $A$ and $B$ are invertible, we may conclude $\det(\lambda I_n - BA) = \det(\lambda I_n - AB)$.

---

Generalization to non-invertible square matrices (sketch):

Since the set of singular matrices is nowhere dense, we may pick two sequences of matrices $A_i$ and $B_i$ converging to $A$ and $B$, such that all factors in $(1)$ are invertible when substituting $A_i$ and $B_i$. Hence we get $\det(\lambda I_n - B_iA_i) = \det(\lambda I_n - A_iB_i)$ for all $i$. Take the limit (since everything is continuous) to conclude $\det(\lambda I_n - BA) = \det(\lambda I_n - AB)$.

Note: For such a continuity argument to work, the matrices must be over a certain kind of domain, e.g. $\Bbb R$. But from this we get a polynomial identity, which proves that it holds as well over any (commutative unitary) ring.

---

Generalization to rectangular matrices:

Assume (without loss of generality) that $k<n$. Then we can make the following block matrix calculation to use the above result: $$\begin{split} \det(\lambda I_n - BA) &= \det\left(\lambda I_n - \begin{pmatrix}B & 0\end{pmatrix} \begin{pmatrix}A \\ 0\end{pmatrix} \right) \\ &= \det\left(\lambda I_n - \begin{pmatrix}A \\ 0\end{pmatrix} \begin{pmatrix}B & 0\end{pmatrix} \right) \\ &= \det\left(\lambda I_n - \begin{pmatrix}AB & 0 \\ 0 & 0\end{pmatrix} \right) \\ &= \det \begin{pmatrix}\lambda I_k - AB & 0 \\ 0 & \lambda I_{n-k}\end{pmatrix} \\ &= \lambda^{n-k}\det(\lambda I_k - AB). \end{split}$$