In one of my L.A. homeworks, wanted me to prove it:
problem:
Assume that $A,B \in M_n(\mathbb{R})$ and $A^{m} = 0$. prove: $\hspace{0.2cm}$ $rank(AB) \leq n - rank (A^{m-1})$ .
so I know that as A is a nilpotent matrix, I have $0 \in Im(A^m) = 0 \Rightarrow Im(A^m) \subseteq Im(A^{m−1})$ and $rank(AB) \leq rank (A)$ . but I'm little confused in usage of rank-nullity theorem to prove it. Does it help to do the proof or Is there another reasoning?
Basically, all lies in $\operatorname{rank}(A) \geq \operatorname{rank}(AB) $ (result you already got)
For intuition:
Think about Jordan decomposition for nilpotent matrices. Basically every non zero entry of $A^{m-1}$ belongs to one block (not necessarily all blocks are filled, but the important thing is that two different entries don't correspond to one block). Now for each of these block you get at least one element of the kernel of $A$, which gives you much info on $\ker A$, just from the rank of $A^{m-1}$.
Now you can solve this directly: take any element $x \in \mathbb{R}^n$ such that $A^{m-1} x\neq 0$. Since $A^mx = 0$, $A^{m-1}x \in \ker A $.
Now if you take elements $x_1,\dots ,x_r$ such that $A^{m-1}x_1,\dots ,A^{m-1}x_r$ forms a basis of the image of $A^{m-1}$ (given by the stronger version of the rank the applied on $A^{m-1}$, you get a free family of cardinality $\operatorname{rank} A^{m-1}$ which lies in $\ker A$.
This gives you $\dim \ker A \geq \operatorname{rank} A^{m-1}$ and by the rank thm $$n - \operatorname{rank} A \geq \operatorname{rank} A^{m-1}$$ or $$\operatorname{rank} A \leq n - \operatorname{rank} A^{m-1}$$
