Computationally checking if a function is convex

Question

Consider a real-valued function $f : \mathbb{R} \to \mathbb{R}$. Given any $x \in \mathbb{R}$, we can compute $f(x)$. We do not know the analytical form of $f$, and it should be treated as a black box. However we do know that $f$ is first-order differentiable and can (numerically) compute its derivative $f'$.

Question: Is there an efficient way to numerically check if $f$ is convex using computational methods?

I would ideally use something like a second-derivative test, however there is no guarantee that $f$ is second-order differentiable and this method needs to generalize to arbitrary $C^1$ functions.

---

One "brute force" idea was to just literally apply the definition of convexity, $$ f(\lambda x_1 + (1-\lambda)x_2) \leq \lambda f(x_1) + (1-\lambda)f(x_2) $$ for all $\lambda \in [0, 1], x_1, x_2 \in \mathbb{R}$. However this is somewhat impractical as it may require testing many different points across $\mathbb{R}$, and there might be local regions where convexity is violated but this test does not pick up on it.

Answer 1

No. It's impossible, for any finite algorithm. The reason is exactly as you list: there may be a tiny interval where the algorithm didn't evaluate $f,f'$ and where the function is non-convex.

It's easy to formally prove that no such algorithm can exist, by a standard adversary argument. Let's assume for simplicity that your algorithm is deterministic (the argument can be extended to apply to randomized algorithms as well, with a little messier proof). Let $f$ be some convex function. Suppose your algorithm has evaluated $f$ and $f'$ at some set of points, say $x_1,\dots,x_n$ (assume these are listed in increasing order), and your algorithm has terminated with the conclusion that $f$ is convex. Now let's define a new point $x'=(x_i+x_{i+1})/2$. Suppose we define a new function $g$ that is identical to $f$, except on a tiny interval $(x'-\epsilon,x'+\epsilon)$, where on that interval, $g$ is non-convex. Notice that running the algorithm on $g$ will evaluate $g$ and $g'$ on exactly the same points, namely $x_1,\dots,x_n$, and at those points, $f$ and $g$ are identical and indistinguishable. Therefore, when run on $g$, your algorithm will also terminate with the conclusion that $g$ is convex. But $g$ is (by construction) not convex. Therefore, your algorithm produces the wrong answer when run on $g$. This proves that there is no algorithm that is correct on all functions and always terminates after finitely many steps.

Answer 2

Brute force is as effective as you will get, but it hinges on choice of the "test set"

Since your function is a black-box, and you are unwilling to stipulate any structure for it beyond the fact that it is first-order differentiable, any computational check of convexity is essentially going to boil down to a "brute force" check of convexity over some limited number of distinct input points $x_1 < \cdots < x_n$ (which I will call the "test set").

In your case you can compute the function value and the first-derivative at each point, so there are essentially two things you need to confirm: (1) that the function is convex with respect to every triple of points;$^\dagger$ and (2) that the first derivative is nondecreasing at consecutive points. It is worth noting that this first test is actually significantly less computationally intensive than you are presenting in your question. Because you are dealing with a univariate function, all you need to check is the convexity of consecutive triplets of points. (For a univariate function, if consecutive triples are convex then all triples must be convex.) This means that you will need to make $n-2$ convexity checks and $n-1$ derivative checks, which is computationally linear in $n$.

Suppose you do that and you confirm convexity over the test set. That is great, but of course the test set is only a finite number of values in the domain of the function. Ultimately, you are going to have to decide on what is a "good" way to generate the test set, which is going to require some assumptions about which parts of the domain of the function are "better" to test. With no further information about the function this is an arbitrary choice, but you might prefer a test set that is highly diffuse (i.e., has high variance). Ultimately, if you just have a black-box function over the entire real line then nothing can really mitigate this problem. No matter how many points you test, there will still be infinitely long regions of the real line that you did not test, and there will still be gaps between the points where you did not test. There is always the possibility that convexity fails in an area that you did not test (either beyond the boundary points of the test set or within the gaps between points in the test set).

---

$^\dagger$ To be clear, what I mean here by testing convexity for a triple of points is checking the inequality constraint that defines convexity with the middle-point of the triple being the interpolated point. For any triple of points $a<b<c$ in the test set, this would mean checking that:

$$f(b) \leqslant \frac{b-a}{c-a} \cdot f(a) + \frac{c-b}{c-a} \cdot f(c).$$

Answer 3

Depending on your $f$ probably no but if one has very strong information on the function then it is possible and there are techniques in rigorous computation that can allow you to do this. As a couple of people have mentioned if you know nothing about the function then it is impossible, since you can just modify the function on a small interval that "evades" your mesh.

If you know something like analyticity, for instance, then something may be possible. There is a theorem that is widely used in rigorous computations. Suppose that you are interested in a function $$f: \mathbb{R} \rightarrow \mathbb{R}$$ on an interval I. Suppose that that function is analytic in an ellipse in $\mathbb{C}$ containing $I$, and you can bound the maximum modulus of $f$ on the ellipse. Then you can get a bound on the error in Chebyshev interpolation of the function on the interval $I$ that is exponential in the number of nodes. This can be used to give a rigorous computer assisted proof that (for instance) the second derivative is strictly positive.

For instance suppose your $f(\lambda)$ arose from solving a linear ODE in which $\lambda$ enters as a parameter. Then $f(\lambda)$ is entire and it is relatively straightforward to estimate $f$ in the complex $\lambda$ plane.

There is a paper by Tadmore in SIAM J Numerical Analysis from 1986 (Volume 23, #1) that gives some theorems.

Answer 4

As others have noticed, it’s impossible to explicitly check this due to small intervals potentially having erratic behavior, but if you’re just trying to do something reasonable, it’s pretty easy to do an approximate empirical check.

If you only need to double check $f$ on some interval $[a,b]$, then you can easily subdivide it equally into equal intervals $x_i=a+(b-a)\cdot i/N$ and then check adjacent triples, i.e. $f(x_{i-1})+f(x_{i+1})\geq 2f(x_i)$ for all $i$.

If you can compute $f’$, and it’s not much more work, you can also just check whether $f’$ is monotonically increasing ($f’(x_i)\leq f’(x_{i+1})$). I don’t think this gets you much more than just checking triples though and it’s probably easier to just do the former check with smaller intervals.

If you want any kind of guarantees, you’re going to need to bound $f’’’$ from below. If your estimates give that $f’’\geq A$ (or in terms of just $f$, that $f(x_{i-1})+f(x_{i+1})-2f(x_i)\geq A(x_{i+1}-x_i)^2$), and you can somehow bound $f’’’\geq -B$, then you only need regions of size smaller than $A/B$ to guarantee convexity. If you want to be clever to save computation, you can subdivide regions with convexity close to $0$ or more negative third derivative more, but that seems like more effort than it’s worth.