3

The Fréchet derivative is a generalised definition of the derivative for normed vector spaces:

Let $(X, \lVert.\rVert_X)$, $(Y,\lVert.\rVert_Y)$ be normed vector spaces. A map $F:X\to Y$ is called Fréchet differentiable with derivative $\mathrm{D}F(x_0)$ at $x_0\in X$ if there exists a bounded operator $\mathrm{D}F(x_0):X\to Y$ such that for all $h\in X$ $$ \lim_{\lVert h\rVert_X\to 0}\frac{\lVert F(x_0+h)-F(x_0)-\mathrm{D}F(x_0)(h) \rVert_Y}{\lVert h\rVert_X} = 0. $$

I can see the intuition behind this definition as follows: the usual limit definition in one-variable calculus is $$ \lim_{x \to x_0}\frac{f(x)-f(x_0)}{x-x_0} = f'(x_0) \equiv \mathrm{D}f(x_0). $$ Move terms and add $|\cdot|$ to get $$ \lim_{x\to x_0}\frac{\left|f(x)-f(x_0)-\mathrm{D}f(x_0)(x-x_0)\right|}{|x-x_0|} = 0,$$ which has the same form as the Fréchet derivative.

However, I find it hard to conceptualise the derivative for more abstract spaces, such as in the space of matrices or integral of functions. An example I came across another day is the derivative in $\mathrm{GL}_n(\mathbb{R})$. Consider the inverse map $$ F:\mathrm{GL}_n(\mathbb{R})\to\mathrm{GL}_n(\mathbb{R})\equiv A \mapsto A^{-1}, $$ where $$\mathrm{GL}_n(\mathbb{R}) = \{ A \;|\;\mathrm{det}(A) \neq 0 \}.$$ Following the above definition, we can differentiate $F$ (with respect to what?!) and find that for all $A\in\mathrm{GL}_n(\mathbb{R})$, $H\in M_n(\mathbb{R})$ $$ \mathrm{D}F(A)(H) = -A^{-1}HA^{-1}.$$

There have been several posts about this (1, 2), using various methods to arrive at the result, but none of them seem to give an intuitive interpretation. What does it even mean to differentiate a matrix-valued function or a functional that takes in functions as its input?

Jonathan Huang
  • 453

1 Answers1

1

It is the linear map that approximates $F$ the best in the point $A^{-1}$. You can see $H \mapsto F(A+H)-DF(A)[H]$ as the "tangent" that passes through $A$ in $H = 0$. It is very similar to derivatives in $\mathbb{R}$: Imagine if $A \in \mathbb{R}$. Then $F'(A) = -A^{-1} A^{-1}$. and the best linear approximation is thus $h \mapsto -A^{-1} A^{-1}h$.

In abstract derivatives, as the objects usually don't commute, you sometimes have to rearrange $H$ and $A$. In this case, $F(A+H)-F(A) = \left(\sum_{k=1}^\infty (-A^{-1}H)^k \right)A^{-1}$ for small $\lVert H \rVert$, so you want $A^{-1}$ "on the right side" of $DF(A)$, so you can factor it out, when computing $F(A+H)-F(A) - DF(A)[H]$

Hyperbolic PDE friend
  • 6,919