If $f$ is uniformly differentiable then $f '$ is uniformly continuous?

Question

The following theorem is true?

Theorem. Let $U\subset \mathbb{R}^m$ (open set) and $f:U\longrightarrow \mathbb{R}^n$ a differentiable function.

If $f$ is uniformly differentiable $ \Longrightarrow$ $f':U\longrightarrow \mathcal{L}(\mathbb{R}^m,\mathbb{R}^n)$ is uniformly continuous.

Note that $f$ is uniformly differentiable if

$\forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,\color{blue}{[x,x+h]\subset U} \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!|<\epsilon |\!|h|\!| $ (edited)

$\forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,\color{blue}{x,x+h\in U} \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!|<\epsilon |\!|h|\!|\qquad \checkmark$

Any hints would be appreciated.

Answer 1

Let's build off of Tomas' last remark, slightly modified:

Let $t>0$ be small. Then \begin{eqnarray} \|f'(x)-f'(y)\| &=& \frac{1}{t}\sup_{\|w\|=1}\|\langle f'(x)-f'(y),tw\rangle\| \nonumber \\ &\leq& \frac{1}{t}\sup_{\|w\|=1}\|f(x+tw)-f(x)-[f(y+tw)-f(y)]\| + 2\epsilon \nonumber \end{eqnarray}

It suffices to show that this weighted combination of four close points on a parallelogram can be bounded by $C\epsilon t$.

Let us bound $\|f(x+h) - f(x) + f(x+k) - f(x+h+k)\|_2 \leq C\epsilon(\|h\|+\|k\|)$, and then in this case $\|h\|=t$ and $\|k\|\leq \delta$, so if $t=\delta$ the whole expression is bounded by a constant times $\epsilon$.

Note applying uniform differentiability three times in directions $h,k,$ and $h+k$, for small $\|h\|,\|k\|$ we have

\begin{eqnarray*} \|f(x+h) - f(x) + f(x+k) - f(x+h+k)\| &\leq& \|f'(x)h + f'(x)k - f'(x)(h+k)\|_2 + 3\epsilon(\|h\|+\|k\|)\\ &=& 3\epsilon(\|h\|+\|k\|) \end{eqnarray*}

Answer 2

This answer only the case $f:U\subset\mathbb{R}\to \mathbb{R}^n$. By hypothesis we have that: given $\epsilon>0$, there exist $\delta>0$ such that if $|x-y|<\delta$ $$\left|\frac{f(x)-f(y)}{x-y}-f'(y)\right|<\frac{\epsilon}{2}\tag{1}$$

and

$$\left|\frac{f(y)-f(x)}{y-x}-f'(x)\right|<\frac{\epsilon}{2}\tag{2}$$

From $(2)$ we get that $$\left|\frac{f(x)-f(y)}{x-y}-f'(x)\right|<\frac{\epsilon}{2}\tag{3}$$

From $(1)$ and $(3)$ $$|f'(x)-f'(y)|<\epsilon$$

I dont know how to tackle the case $U\subset \mathbb{R}^m$.

Update: I would like to add here some thoughts, maybe it can help someone to give a full answer. First note that

\begin{eqnarray} \|f'(x)-f'(y)\| &=& \sup_{\|w\|=1}\|\langle f'(x)-f'(y),w\rangle\| \nonumber \\ &=& \sup_{\|w\|=1}\|f(x+w)-f(x)-[f(y+w)-f(y)]+o(x,w)-o(y,w)\| \nonumber \end{eqnarray}

By hypothesis, $\|o(x,w)-o(y,w)\|\leq \epsilon\|w\|+\epsilon\|w\|$, hence for $\|w\|=1$ we have that $\sup_{\|w\|=1}\|o(x,w)-o(y,w)\|$ is small, independetly of $x,y$. Therefore, it remains to prove that $f$ is uniformly continuous. One way to prove it is for example showing that $f'$ is bounded, which I think is true when $U$ is bounded. When $U$ is unbounded, I think we need a direct argument to show that $f$ is uniformly continuous.

Answer 3

The difficulty here is that the obvious estimate (as in Tomás' answer) only gives a result along lines - i.e. it can't say anything about how $f'(x)h$ changes as you move in any direction but $h$. Thus I think we need some kind of "polarization argument" to get an isotropic result. When $f$ is $C^2$, this is literally polarizing the Hessian matrix $H(x) = f''(x)$ to show it is bounded from the fact that it is bounded on the diagonal. While we cannot use this literally for general $f$, hopefully we can use a finite-difference analogy and the analysis will go through. Here's the proof for the $C^2$ case and an optimistic starting point for the general case.

For any $\epsilon>0$ there is a $\delta>0$ such that for all $x,h$ with $|h|<\delta$, we have (cf. Tomás' answer)

$$ \begin{align} |f(x+h) - f(x) - f'(x)h| &< \epsilon |h| \\ |f(x) - f(x+h) + f'(x+h)h| &< \epsilon |h| \end{align} $$

which combine to give $$t|(f'(x+t v)-f'(x))v| < 2 \epsilon $$ for $v$ a unit vector, $t<\delta$.

In the $C^2$ case the left side is $t^2 |H(x)(v,v)| + o(t^2)$, so we have a uniform bound $$|H(x)(v,v)| < M \tag{1}.$$ The polarization formula is $$ H(x)(v,w) = \frac12 \left( H(x)(v+w,v+w) - H(x)(v,v) - H(x)(w,w) \right). \tag{2}$$

We want to show uniform continuity of $f'$. A line integral estimate gives $$ |f'(x)w-f'(y)w| \le |x-y| \sup_{z \in U} |H(z)\left(v,w\right)|$$ where $|x-y|v = x-y$; so it suffices to show that $\sup_{z \in U, |v| = 1} H(z)(v,w)$ is finite. Applying $(1)$ and $(2)$ we estimate

$$ \begin{align} |H(z)(v,w)| & \le \frac12 M \left( |v+w|^2 + |v|^2 + |w|^2 \right)\\ & \le M \left( 1 + |w| + |w|^2 \right) \end{align} $$

so we are done.

Now to try generalising this. Attempting to write down the polarization formula using finite difference terms gives

$$ \begin{align} &(f'(x+v+w) - f'(x+v))w + (f'(x+w+v)-f'(x+w))v \\ = &(f'(x+v+w) - f'(x))(v+w) - (f'(x+v)-f'(x))v-(f'(x+w) - f'(x))w \\ \end{align}$$

which (since $f$ is uniformly differentiable) is bounded by $2 \epsilon (|v+w| + |v| + |w|)$.

Now we would like to equate the two terms on the LHS, which is the reason polarization works for symmetric forms. Our expression is of course not symmetric but we can hope it is close enough for small $u,v$ - I have no idea whether or not this can be done without more assumptions, however. If anyone has any ideas (or has spotted any mistakes) please let me know.

Answer 4

From the uniform differentiabilty, we can find a $t>0$ such that, for every ${\bf v}$ with $\|{\bf v}\| \le 2t$ and ${\bf x} \in U$, ${\bf x} + {\bf v} \in U$,

$$ t^{-1}\| f({\bf x}) - f({\bf x}+{\bf v}) + f'({\bf x}){\bf v}\| < \frac{\epsilon}{5}$$

Now, let take any ${\bf x}$, ${\bf y}$ such that $\|{\bf x} - {\bf y}\| < t$, and ${\bf x}, {\bf y} \in U$. Define ${\bf k}$ = ${\bf y} - {\bf x}$.

Note that for any $t{\bf w}$ with $\| {\bf w} \|=1$, we have $\|t{\bf w} + {\bf k}\| < 2t$.

We then have, for every ${\bf x}$ such that an open ball of radius $2t$ is contained in $U$,

$$\begin{align} \left\| f'({\bf x}) - f'({\bf y})\right\| &= t^{-1}\max_{\|{\bf w}\|=1}\left\| f'({\bf x}){t\bf w} - f'({\bf x} +{\bf k})t{\bf w}\right\| \\ &\le t^{-1}\max_{|{\bf w}|=1} \left\| ( f({\bf x}+t{\bf w}) - f({\bf x}) + f({\bf x} + {\bf k}) - f({\bf x} + {\bf k} + t{\bf w}) \right\| + \frac{2}{5}\epsilon \\ &\le t^{-1}\max_{|{\bf w}|=1}\left\| f'({\bf x})({t\bf w} + {\bf h} -t{\bf w} -{\bf h})\right\| + \epsilon\\ &= \epsilon \end{align}$$

However, I do not know how to prove it for ${\bf x}$ less than $2t$ away from a point outside $U$.

Maybe some sort of extension theorem would help?

Otherwise, we can strengthen the definition of uniform differentiability to the following:

$\forall \epsilon>0\,,\exists \delta>0:|\!|h|\!|<\delta,\color{blue}{x \in U} \Longrightarrow |\!|f(x+h)-f(x)-f'(x)(h)|\!|<\epsilon |\!|h|\!|\qquad $

in which case it does work.

Answer 5

Okay, let's get rid of the $R'$ and try the same thing this way. f is differentiable at $X_0$ means that $\exists$ a g which is linear throughout some neighborhood U of $X_0$, g(X) = $f(X_0)$ + (M,$(X-X_0)$ [ where the second term is the inner product of $(X-X_0)$ with a constant vector M] and for which $\lim_{X \rightarrow X_0}\frac{f(X) -g(X)}{X - X_0}$ = 0. This means that f is practically linear throughout U.

Now if f were precisely linear we would have no problem, since $f(X) - f(X_0)$ = (M,$X-X_0)$ and this is uniformly continuous throughout U. The problem is that f is not exactly linear. I am using R(X) to represent the difference between f, the not quite linear, and g the exactly linear.

So the question of uniform continuity of f in the neighborhood U comes down to the behavior of R(X) in U. What do we know about R? We know that $\lim_{X \rightarrow X_0} \frac{R(X)}{X-X_0}$ = 0 and further that the convergence is uniform in U. That means that if you pick an $\epsilon$, there is a $\delta$ independent of X such that in $V \subset U$ a neighborhood of $X_0$ of radius $\delta$, if $X_1$ and $X_2$ are in V then $\frac {|R(X_1) - R(X_2)|}{|X_1-X_2|}$ < $\epsilon$. Substituting back the definition of R we have

$R(X_1)$ = $f(X_1)-g(X_1)$ = $f(X_1) - f(X_0)- (M,(X_1-X_0))$ and similarly for $X_2$. So

$\frac {|R(X_1) - R(X_2)|}{|X1-X_2|}$ = $\frac{f(X_1) -f(X_2) - [(M,(X_1-X_0))-(M,(X_2-X_0))]}{X_1 - X_2}$ < $\epsilon$ when $X_1, X_2$ are in V.

We get $|f(X_1)-f(X_2)|$ < $\epsilon|(X_1-X_2)|$ + $[(M,(X_1-X_0))-(M,(X_2-X_0))]$. The terms in square brackets go to 0 uniformly as $X_1 \rightarrow X_2$ because they are the difference of a linear function at 2 points in V, and the $\epsilon|(X_1-X_2)|$ likewise .

Answer 6

The proof of theorem $3.4$ in your reference contains the idea to solve the problem: one has to prove the continuity of partial derivatives.
The reasoning is valid apart from the non-Archimedean and local context of the writing.

In the $\varepsilon-\delta \,$ game let $\delta \,$ be a reply to $\varepsilon /4$ as for the differentiability of $\mathbf f$ and let $\delta_1$ be a reply to $\varepsilon \delta/4$ as for the continuity of $\mathbf f$.

In the following $f_j^i$ is the derivative of $f^i$ respect to $x_j \,$.

Now choose $\mathbf x,\mathbf y \in U$ such that $||\mathbf x-\mathbf y||<\min \{\delta,\delta_1\}$.
Then $$|f_j^i(\mathbf x)\delta-f_j^i(\mathbf y)\delta| \le$$$$|f^i(\mathbf x+\delta\hat {\mathbf e}_j)-f^i(\mathbf x)-f_j^i(\mathbf x)\delta|$$$$+ \,|f^i(\mathbf y+\delta\hat {\mathbf e}_j)-f^i(\mathbf y)-f_j^i(\mathbf y)\delta|$$$$+ \,|f^i(\mathbf x)-f^i(\mathbf y)|+|f^i(\mathbf x+\delta\hat {\mathbf e}_j)-f^i(\mathbf y+\delta\hat {\mathbf e}_j)|$$$$< \varepsilon \delta$$from which the continuity of $f_j^i \,$.
The continuity of the (total) derivative follows from $$||\mathbf f'(\mathbf x)-\mathbf f'(\mathbf y)|| \le \left \{\sum_{i,j} \;[f_j^i(\mathbf x)-f_j^i(\mathbf y)]^2 \right \} ^\frac 12 $$by the Schwarz inequality.

Edit:$\;$ OP advises that the proof doesn't work: in fact the uniform continuity of $\mathbf f$ is not assured because $U$ is open.

new attempt

Let $U$ be simply open: I want to prove that $f'$ is uniformly continuous on the set$$U_{\Delta}=\{x\in U:\ {\rm dist}\,(x,\Bbb R^m\backslash U)>\Delta\}$$with $\Delta>0$ chosen sufficiently small.

Avoiding some detail, let $x \in U_{\Delta}$ and let $\delta>0$ corresponding to $\varepsilon>0$ and such that $\delta<\Delta$.

We have $$(f'(x+h)-f'(x))(k)=$$$$-[f(x+k)-f(x+h)-f'(x+h)(k-h)]+$$$$f(x+k)-f(x)-f'(x)(k)+$$$$f(x)-f(x+h)-f'(x+h)(-h)$$$$$$Take $h$ and $k$ so that $\|h\|<\frac \delta 2$ and $\|k\|=\|h\|\,$: hence $x+h,x+k \in U$.

Then $$\|(f'(x+h)-f'(x))(k)\|<4\varepsilon \|k\|$$

Now let $u \in \mathbb R^m$ such that $\|u\|=1$ and let $k=\|h\|\,u\,$: we have $$\|(f'(x+h)-f'(x))(u)\|<4\varepsilon$$Then $$\|f'(x+h)-f'(x)\|\le4\varepsilon$$$$$$

Answer 7

Let us start with the simplest case f:$R^1 \rightarrow R^1$. The discussion below will generalize nicely to more dimensions.

Then f is differentiable at $x_0$ iff $ \lim_{x \rightarrow x_0} {\frac{f(x)-f(x_0)}{x-x_0}}$ exists and is finite. We call that limit $f'(x_0)$ . So if f is differentiable at $x_0$ then we may write

f(x) = $f(x_0) + f'(x_0)(x-x_0) + R(x)$ where $\lim_{x \rightarrow x_0} R(x)/(x-x_0)$ = 0.     (1)

If f is uniformly differentiable that means

$\lim_{x \rightarrow x_0} R(x)/(x-x_0)$ converges uniformly to zero.     (2)

From (1) we have $f'(x)$ = $f'(x_0)$ + R'(x). [We know R is differentiable because it is the difference between f which is differentiable and a linear polynomial.]

So $\lim_{x \rightarrow x_0} (f'(x)-f'(x_0))$ = $\lim_{x \rightarrow x_0} R'(x)$ = $\lim_{x \rightarrow x_0} R(x)/(x-x_0)$ = 0. And this convergence is uniform as per our hypothesis as stated in (2).

Moving on to the case where $f:R^n \rightarrow R^m$. For this part of the discussion capital letters will be used to represent vectors (in whatever dimension).

According to many sources (and I will state specifically Robert Sealey "Calculus of Several Variables") f is differentiable at $X_0$ iff there exists a linear function g (X) = $f(X_0)$ + $(M,X-X_0)$ [where M is a constant vector and (a,b) means the inner product], such that

$ \lim_{X \rightarrow X_0} {\frac{f(X)-g(X)}{|X-X_0|}}$ = 0.   (3)

If f is uniformly differentiable that means simply that this limit converges uniformly.We could have stated differentiability this way in the $R^1 \rightarrow R^1$ case, but I stated it as I did in that case to throw some light on what g actually is.

Let R(X) = f(X) - g(X) $\Rightarrow$ f(X) = g(X) + R(X). Since R = f - g then $R/|x-x_0| \rightarrow 0$ uniformly. Notice also that $R(X_0)$ = 0, since it is $f(X_0) - g(X_0)$ and by (3) this must be 0. Since g is linear and f is differentiable R is also differentiable and from the above $R'(X_0)$ = 0.

Looking at $f'$ we have $f'(X) = g'(X) + R'(X) = M + R'(X)$;
$f'(X_0) = M + R'(X_0)$ and thus $f'(X_0) = M$.

Finally $\lim_{X \rightarrow X_0} (f'(X)-f'(X_0))$ = $\lim_{X \rightarrow X_0} R'(X)$ = $\lim_{X \rightarrow X_0}{\frac{R(X)-R(X_0)}{X-X_0}}$ = $\lim_{X \rightarrow X_0}{\frac{R(X)}{X-X_0}}$ and this goes to 0 uniformly because f is uniformly differentiable, R = f - g, and $R(X_0)$ is 0.