Geometric intuition and visualization of matrix exponential

Question

What is the geometric intuition and visualization of the matrix exponential $e^{\textbf{X}}$ ? Does that explain why matrix exponential is only valid with square matrices ?

Answer 1

How do we understand $\exp(x)$ for $x$ a real number? It can't be understood as repeated multiplication, since $x$ is not a counting number. One way to understand it is by the example of compound interest. If $x$ is a simple interest rate, then after the term is over we have $1+x$ times the principal, after interest is applied. If instead the interest is compounded $n$ times over the term, then our balance is increased to $\left(1+\frac{x}{n}\right)^n.$ If we want continuously compounded interest, we take the limit as $n\to\infty,$ getting $\exp(x).$

Think of the interest as a small addition to the principal, and then exponentiation accumulates all those small infinitesimal changes into a large finite change, in a continuous and differentiable fashion. In doing so, it turns additive changes into multiplicative factors $\exp(x+y)=\exp(x)\exp(y).$

You may be more familiar with a description of exponentiation as repeated multiplication, but this alternate description, as the accumulation of infinitesimal additions, is a more generally applicable one, and still implies the repeated multiplication picture via this formula,

$$ \exp(n) = \exp(1+\cdots+1) = \exp(1)\cdot\dotsb\cdot\exp(1)=\exp(1)^n. $$

So it is more fundamental and should be taken as the definition of exponentiation: it is the operation which turns addition into multiplication.

It is the same for matrices. Think of $X$ as a change to the matrix $1$. Perhaps it is to add real numbers along the diagonal, or perhaps it is to shift the off-diagonal by opposite amounts. Then $\exp(X)$ accumulates these infinitesimal matrix shifts. It can be the scaling of the diagonal, or rotation in the off-diagonal entries.

Here too the exponential satisfies a formula of turning addition into multiplication, but with a wrinkle. For a single matrix $X$ we still have the right formula for scalar multiples: $\exp(sX+tX)=\exp(sX)\exp(tX).$ And again, this formula can be taken as the defintion of exponential in an arbitrary Lie group, and gives the geometric intuition. We also have $\exp(X+Y)=\exp(X)\exp(Y)$ if $X$ and $Y$ are commuting matrices. But if $X$ and $Y$ do not commute, the formula becomes more complicated. We can no longer say exponentiation turns addition into multiplication. We might instead say that it turns commutators into conjugations.

To get even more geometric, the above picture is an example of a basic construction from differential topology. Given a tangent vector to a manifold at a point, we may find at least in a small neighborhood, a curve in the manifold whose tangent vector at zero is the given vector. In other words, exponentiation is the operation which turns flattened approximations back into their curvilinear originals.

Please also see Nick Alger's answer here, which seems good.

I will also note that I guess you could have a homomorphism between algebras of different dimension. It would be represented as a non-square matrix. It can still have an exponential.

Answer 2

The best way to vizualize is by power series:

$$e^A = 1 + A + A^2/2!...$$

The presence of matrix multiplication explain why it's only valid with square matrices.