What is the codimension of matrices of rank $r$ as a manifold?

Question

I'm reading through G&P's Differential Topology book, but I hit a wall at the end of section 4. There is a result stating

The set $X=\{A\in M_{m\times n}(\mathbb{R}):\mathrm{rk}(A)=r\}$ is a submanifold of $\mathbb{R}^{m\times n}$ with codimension $(m-r)(n-r)$.

There is a suggestion: Let $A\in M_{m\times n}(\mathbb{R})$ have form $$ A=\begin{pmatrix} B & C \\ D & E\end{pmatrix} $$ where $B$ is an invertible $r\times r$ matrix. Then right multiply by $$ \begin{pmatrix} I & -BC^{-1} \\ 0 & I \end{pmatrix} $$ and show $\mathrm{rk}(A)=r$ iff $E-DB^{-1}C=0$.

I multiplied out and got the matrix $$ M:=\begin{pmatrix} B & 0 \\ D & E-DB^{-1}C\end{pmatrix}. $$

Since I multiplied by a nonsingular matrix, I know that $\mathrm{rk}(A)=\mathrm{rk}(M)$. If $E-DB^{-1}C=0$, then $$ M=\begin{pmatrix} B & 0 \\ D & 0 \end{pmatrix} $$ has rank $r$, so $A$ has rank $r$. For the converse, if $A$ has rank $r$, then $M$ has rank $r$, so by performing row operations, $M$ is row equivalent to a matrix of the form $$ \begin{pmatrix} I_r & 0 \\ 0 & 0 \end{pmatrix}. $$ This would imply $E-DB^{-1}C$ is row equivalent to $0$, and I think this implies $E-DB^{-1}C=0$.

My main concern is then, how does this approach imply $\mathrm{codim}(X)=(m-r)(n-r)$? Is there some special map I can apply the Preimage Theorem to?

Thank you.

Answer 1

Since you asked, I've replaced my hint with a full solution:

First consider matrices of the form

$$A = \begin{pmatrix} B & C\\ D & E \end{pmatrix}$$

Where $B$ is an $r \times r$ nonsingular matrix. Since invertibility is an open condition, this set of such matrices, denoted $Z$, is a submanifold of $M_{m \times n}$. Postmultiply by the nonsingular matrix

$$\begin{pmatrix} I & -B^{-1}C\\ 0 & I \end{pmatrix}$$

to obtain the matrix

$$\begin{pmatrix} B & 0\\ D & -DB^{-1}C + E \end{pmatrix}$$

the original matrix has rank $r$ iff this new matrix has rank $r$, which is clearly only the case if $-DB^{-1}C + E = 0$. Thus we can define a map $f$ from $Z$ to matrices of size $(m-r) \times (n-r)$ that sends $A$ as above to $-DB^{-1}C + E$. This is clearly smooth, so it suffices to check that it is a submersion. Now, the tangent space of the image is the same space as the image, since the image is a linear space. Let $X$ be an $(m-r) \times (n-r)$ matrix. Consider the curve passing through any matrix $A \in Z$.

$$\gamma(t) = \begin{pmatrix} B & C\\ D & E+tX \end{pmatrix}$$

The derivative of $f \circ \gamma$ at $0$ is $X$, and this is equal to

$$df_{A}(\begin{pmatrix} 0 & 0\\ 0 & X \end{pmatrix})$$

so that at any arbitrary point $A$ we have shown the existence of a tangent vector at $A$ that is mapped by $df$ to $X$. This verifies that $f$ is submersion, and hence $f^{-1}(0)$ is a smooth submanifold of $\mathbb{R}^{mn}$. The dimension $f^{-1}(0)$ is $mn - (m-r)(n-r)$, i.e. of codimension $(m-r)(n-r)$.

Of course, we have only shown that matrices of rank $r$ contained in $Z$ form a smooth submanifold. However, any matrix can be put into the form of matrices in $Z$ by rearranging rows and columns, which is just a linear isomorphism. Thus if $A$ is matrix of rank $r$, we have a map $R$ to a matrix in $Z$ contained in chart $\psi$. Then we have that $\psi \circ R$ is a smooth chart around $A$ inherited from a chart on $M_{m \times n}$. The collection of these charts then extends to a maximal atlas giving the set of rank-$r$ matrices the structure of a smooth submanifold.

Answer 2

This answer was originally going to be a comment on Bruno Joyal's answer, but it got too long.

As you suggest, a natural way to try proving something like this is to use the preimage theorem. So we want a map $F$ from $m \times n$ matrices to $(m-r)(n-r)$ space such that $d F$ is surjective and $F^{-1}(0)$ is, more or less, the matrices of rank $r$. (Why do I say more or less? First of all, the matrices of rank $r$ contain the matrices of rank $<r$ in their closure, and $F^{-1}(0)$ will be closed, so we can't exactly hope to get the matrices of rank $r$. And, in fact, our solution will only work in the open submanifold $Z$ introduced by Isaac Solomon.)

One natural idea is to send a matrix $M$ to all of its $(r+1) \times (r+1)$ minors; $M$ is rank $r$ if and only if all of these minors vanish. But there are $\binom{m}{r+1} \times \binom{n}{r+1}$ of these, way more than $(m-r)(n-r)$, and $dF$ is not surjective.

Can we cleverly choose $(m-r) \times (n-r)$ of the minors to use? Yes! Let $Z$ be the open submanifold where the upper leftt $r \times r$ sumbmatrix is invertible. I'll write $B$ for this upper left matrix. Let $F: \mathrm{Mat}_{m \times n} \to \mathrm{Mat}_{(m-r) \times (n-r)}$ send a matrix $M$ to the matrix $F(M)$ whose $(i,j)$ entry is the minor using row $i+r$ together with the first $r$ rows, and column $j+r$ together with the first $r$ columns.

I claim $dF$ is surjective. Proof: We'll show that the $(m-r) \times (n-r)$ matrix $\left( \partial F_{ij}/\partial M_{k \ell} \right)$ where $k$ and $\ell>r$ is an isomorphism. If $(k, \ell) \neq (i+r, j+r)$, then $M_{k \ell}$ does not occur in the minor $F_{ij}$, so $\partial F_{ij}/\partial M_{k \ell}=0$. If $(k, \ell) = (i+r, j+r)$, then expanding by minors in the last row, $\partial F_{ij}/\partial M_{k \ell} = \det B \neq 0$. So the matrix of partials $\left( \partial F_{ij}/\partial M_{k \ell} \right)$ is diagonal with nonzero entries on the diagonal.

Now, I claim that $F^{-1}(0) \cap Z$ is the set of rank $r$ matrices in $Z$. I'm going to skip this one. Basically, we can apply downward and leftward row and column operations to $M$ to put it in the form $\left( \begin{smallmatrix} B & 0 \\ 0 & N \end{smallmatrix} \right)$ without altering the minors $F_{ij}$. So we have a proof by using a set of $(m-r) \times (n-r)$ minors, rather than by G and P's trick.

The fun thing is that I believe these proofs are basically the same proof! I think that $F(M) = (\det B) (E - D B^{-1} C)$ and the row and columns operations I referred to in the previous paragraph are the $2 \times 2$ matrices G and P use. So the question is whether to write the proof using

Answer 3

Another approach: A matrix has rank $<r$ if and only if all of its $r \times r$ minors have zero determinant. An $m\times n$ matrix has $(m-r)(n-r)$ minors of size $r \times r$ (choose which rows and columns to exclude). Together, the determinants of these minors give a polynomial map $\mathbf R^{m \times n} \to \mathbf R^{(m-r) \times (n-r)}$ whose zero set is precisely the set of matrices of rank $< r$...

Answer 4

Here is a more algorithmic elementary answer building off of those above for anyone who is a numerical analyst at heart.

Given a rank $r$ matrix $A \in \mathbb{C}^{m \times n}$ one can use, e.g., the SVD of $A$ to find a full rank $U \in \mathbb{C}^{m \times r}$ and a full rank $V \in \mathbb{C}^{r \times n}$ such that

$$A = UV.$$

If we denote the first $r$ columns of $V$ as the matrix $V_1 \in \mathbb{C}^{r \times r}$ and the last $n-r$ columns of $V$ as the matrix $V_2 \in \mathbb{C}^{r \times (n-r)}$ we have $V = [V_1~ V_2]$ so that

$$A = U~[V_1~ V_2].$$

Note that, w.l.g., $V_1$ will be invertible since $V$ is full rank (if not, we could pick some other subset of columns to be $V_1$ to get a different map...). Thus, we have that

$$A = (U V_1)~V^{-1}_1~ [V_1 ~V_2] = (U V_1)~[I ~|~ (V^{-1}_1)V_2]$$

where $I$ is the $r \times r$ identity matrix.

Now we can see that $A$ can be stored and reconstructed at will using the equation above once we record both $(U V_1) \in \mathbb{C}^{m \times r}$ and $(V^{-1}_1)V_2 \in \mathbb{C}^{r \times (n-r)}$. And, storing these two matrices requires us to record only

$$mr + r(n - r) = mr + nr - r^2$$

complex values. As such, any rank $r$ matrix $A$ is intrinsically at most $(mr + nr - r^2)$-dimensional (i.e., since we just explicitly constructed a continuous invertible map...). We further see, I suppose, that this way of storing a rank $r$ matrix via the SVD is near-optimal given the earlier posts.

Answer 5

Even though this is an older post, I thought I'd post a simpler answer. Suppose $r\leq m \leq n$. Notice if $M \in Mat_{m\times n}(\mathbb{R})$ has rank $r$ then we can find (full rank) $A \in Mat_{m\times k}(\mathbb{R})$ and (full rank) $B \in Mat_{k\times n}(\mathbb{R})$ such that $$M = AB.$$

On the other hand, there's an ambiguity of a (full rank) $U \in Mat_{k\times k}(\mathbb{R})$,

$$M = (AU) (U^{-1}B)$$

Counting free parameters, and noting that everything we did is on an open set, this shows that the dimension the set of rank $m\times n$ matrices of rank $r$ is $m\cdot r + n\cdot r - r^{2}$, or codimension $n\cdot m - (m\cdot r + n\cdot r - r^{2}) = (n-r)(m-r)$. There are some other technical points, but I think this is the core of the argument you need.

Answer 6

Ref: Link

Consider ${ \mathbb{R} ^{m \times n} . }$ Let ${ r \leq \min \lbrace m, n \rbrace . }$

We are interested in the set

$${ \mu _r := \lbrace A \in \mathbb{R} ^{m \times n} : \text{rk}(A) = r \rbrace . }$$

We are to show

$${ \text{To show:} \quad \mu _r \text{ is a submanifold of } \mathbb{R} ^{m \times n} . }$$

We can first consider ${ U \cap \mu _r , }$ where

$${ U := \left \lbrace A = \begin{pmatrix} A _{11} & A _{12} \\ A _{21} & A _{22} \end{pmatrix} \in \mathbb{R} ^{m \times n} : A _{11} \in \mathbb{R} ^{r \times r} \text{ is invertible} \right \rbrace . }$$

Note that ${ U \subseteq _{\text{open}} \mathbb{R} ^{m \times n} . }$

Note that ${ U = f ^{-1} (\mathbb{R} \setminus \lbrace 0 \rbrace ), }$ where ${ f : \mathbb{R} ^{m \times n} \to \mathbb{R}, }$ ${ f(A) = \det(A _{11}) .}$

Note that

$${ {\begin{aligned} &\, U \cap \mu _r \\ = &\, \lbrace A \in \mathbb{R} ^{m \times n} : \det(A _{11}) \neq 0, \text{rk}(A) = r \rbrace \\ = &\, \lbrace A \in \mathbb{R} ^{m \times n} : \det(A _{11}) \neq 0, \text{ Schur complement } \Delta _{A _{11}} = O \rbrace \\ = &\, \lbrace A \in \mathbb{R} ^{m \times n} : \det(A _{11}) \neq 0, A _{22} - A _{21} A _{11} ^{-1} A _{12} = O \rbrace \\ = &\, \left \lbrace \begin{pmatrix} A _{11} & A _{12} \\ A _{21} & A _{21} A _{11} ^{-1} A _{12} \end{pmatrix} : \det (A _{11}) \neq 0 \right \rbrace . \end{aligned}} }$$

Hence

$${ U \cap \mu _r = \text{im}(\Phi) }$$

where

$${ {\begin{aligned} &\, \Phi : GL(r, \mathbb{R}) \times \mathbb{R} ^{r \times (n-r)} \times \mathbb{R} ^{(m-r) \times r} \longrightarrow \mathbb{R} ^{m \times n}, \\ &\, \Phi (A, B, C) := \begin{pmatrix} A & B \\ C & C A ^{-1} B \end{pmatrix} . \end{aligned}} }$$

Note that ${ \Phi }$ is a smooth embedding.

Note that ${ \Phi }$ is a smooth immersion, and ${ \Phi : GL(r, \mathbb{R}) \times \mathbb{R} ^{r \times (n-r)} \times \mathbb{R} ^{(m-r) \times r} \longrightarrow \text{im}(\Phi) }$ is a homeomorphism of topological spaces.

Hence

$${ {\begin{aligned} &\, U \cap \mu _r = \text{im}(\Phi) \text{ is a submanifold of } \mathbb{R} ^{m \times n} \\ &\, \text{of dimension } r ^2 + r(n-r) + (m-r) r . \end{aligned}} }$$

Let ${ A \in \mu _r . }$ We can pick permutation matrices ${ P _A , Q _A }$ such that ${ P _A A Q _A \in U \cap \mu _r . }$

It suffices to show there are index sets ${ I, J }$ such that the submatrix ${ A _{I, J} }$ is an ${ r \times r }$ invertible matrix.

Pick rows with indices ${ i _1, \ldots, i _r }$ such that the ${ r \times n }$ matrix formed is of rank ${ r . }$ Now pick columns with indices ${ j _1 , \ldots , j _r }$ such that the ${ r \times r }$ matrix formed is of rank ${ r . }$ Note that the submatrix ${ A _{I, J} }$, where ${ I = \lbrace i _1, \ldots, i _r \rbrace }$ and ${ J = \lbrace j _1, \ldots, j _r \rbrace , }$ is an ${ r \times r }$ invertible matrix.

Hence ${ A \in P _A ^{-1} (U \cap \mu _r) Q _A ^{-1} . }$

Hence

$${ \mu _r \subseteq \bigcup _{P, Q \text{ perm matrices}} P(U \cap \mu _r) Q . }$$

Hence

$${ \mu _r = \bigcup _{P, Q \text{ perm matrices}} P(U \cap \mu _r)Q . }$$

This gives an atlas of ${ \mu _r . }$

Hence ${ \mu _r }$ is a submanifold of ${ \mathbb{R} ^{m \times n} }$ of dimension ${ r ^2 + r(n-r) + (m-r) r .}$

Answer 7

Here is a solution only using elementary mathematics. You need $mn$ parameters to completely specify and $m$-by$n$ matrix. If this matrix is of rank (at most) $r$, then $m-r$ rows and $n-r$ columns are over-specified, given $(m-r)(n-r)$ over-specified parameters in all. Thus all you really need is $mn - (m-r)(n-r) = (m + n - r)r$ free parameters.

There the sought-for co-dimension is the correction term $(m-r)(n-r)$. $\quad\quad\quad\quad\quad\quad\Box$