1

Suppose that $$h(x,z):= (x-z)^T\nabla f(z)+\frac{1}{2}(x-z)^TH(x-z),$$ where $f$ is a smooth function in $\mathbb R^n$. Also, suppose $H\succ 0$ and $\|\nabla f(u)\|\le \gamma; \forall u\in \mathbb R^n$. Can we conclude that $h$ is bounded below for all $(x,z)\in \mathbb R^n \times \mathbb R^n$?

I am trying to avoid imposing other conditions (for example convexity on $f$, having a lower bound for $f$ itself, or even bounding $x$ and $z$ to a bounded set) to achieve the existence of a lower bound but if we have to, that is okay.

If we consider the KKT conditions, we see that at a local minimizer, taking gradient with respect to $x$ and setting it equal to zero implies $x-z=-H^{-1}\nabla f(z)$ and therefore the following:

\begin{equation*} \begin{aligned} h(x,z) &= (x-z)^T\nabla f(z) +\frac{1}{2}(x-z)^TH(x-z) \\ & = -\nabla f(z)^TH^{-1}\nabla f(z) + \frac{1}{2} \nabla f(z)^TH^{-1}\nabla f(z) \\ & = -\frac{1}{2}\nabla f(z)^TH^{-1}\nabla f(z) \\ &\ge -\frac{1}{2} \|H^{-1}\|_2 \|\nabla f(z)\|_2^2 \\ & \ge -\frac{1}{2} \lambda^{-1}_{\min}(H) \|\nabla f(z)\|^2 \\ & \ge - \frac{1}{2} \gamma^2\lambda^{-1}_{\min}(H) \end{aligned} \end{equation*}

But does this necessarily mean that the whole function $h(x,z)$ is bounded below?

By the way, setting the gradient of $h$ with respect to $z$ equal to zero shows that $(\nabla^2 f(z)-H)(x-z)=\nabla f(z)$.

Is $h(x,z)$ coercive?

Sam
  • 323

1 Answers1

0

Let $c := \frac14\lambda_{\min}(H) > 0$.

By Cauchy-Bunyakovsky-Schwarz and AM-GM, we have $$(x-z)^\top\nabla f(z) \ge - \sqrt{\|x-z\|^2 \|\nabla f(z)\|^2} \ge - 2c \|x-z\|^2 - \frac{1}{8c}\|\nabla f(z)\|^2.$$

Thus, we have \begin{align*} h(x, z) &\ge - 2c \|x-z\|^2 - \frac{1}{8c}\|\nabla f(z)\|^2 + \frac12(x-z)^\top H (x - z)\\ &= \frac12(x-z)^\top (H - 4c I)(x - z) - \frac{1}{8c}\|\nabla f(z)\|^2\\ &\ge - \frac{1}{8c}\gamma^2. \end{align*}

River Li
  • 49,125