4

When I read somewhere about the importance of convexity for optimization most of the time it deals with the nice property of convex functions that local and global minima are the same.

This is a very neat property to have but if this is the only reason we value convexity so highly then why don't we look more into pseudoconvexity since it is less strict than convexity?

For the definition of pseudoconvexity, I will cite Wikipedia:

Consider a differentiable function $ f:X\subseteq \mathbb {R} ^{n}\rightarrow \mathbb {R}$, defined on a (nonempty) convex open set $X$ of the finite-dimensional Euclidean space $\mathbb {R} ^{n}$. This function is said to be pseudoconvex if the following property holds:

$$ \forall x,y\in X:\quad \nabla f(x)\cdot (y-x)\geq 0\Rightarrow f(y)\geq f(x).$$

Pseudoconvex functions also have the property that local and global minima are the same. Otherwise we have: $$Convex \Rightarrow pseudoconvex \Rightarrow quasiconvex$$ There has to be a reason that pseudoconvexity isn't discussed that much.

Sen90
  • 477
  • 2
    +1, I'm also curious to see the answers here. However, as an initial guess, requiring $f$ to be differentiable and defined on an open convex subset of $\mathbb{R}^n$ seems rather restrictive to me. In my area (geometric mechanics), we almost exclusively work on Riemannian manifolds, where convexity has a natural generalization. The optimization problems often come in the form of minimizing some action functional over an infinite-dimensional Hilbert manifold, and complex analysis is baked into the theory – infinitylord May 22 '24 at 21:25
  • 1
    Convexity is easier to work with. – copper.hat May 22 '24 at 21:40
  • Related: https://math.stackexchange.com/q/4363840/1105283 – JJMae May 22 '24 at 21:55

2 Answers2

3

Convexity is not the decisive property to evaluate the difficulty of an optimization problem, in practice. There are lots of optimization problems that are feasible in practice even though they are not convex.

I suspect you are drawing a faulty conclusion from reading a lot of theoretical materials. It is easier to prove theorems about optimizing convex functions than about optimizing general functions. Convexity is a simple, clean property that is familiar and easy to state, so a natural property to appear in a theorem. People often focus more attention on theorems that are simple to state and/or that have an elegant proof.

Just because you've seen a lot of theorems that mention convexity as a precondition does not mean that convexity is the best way to assess the difficulty of an optimization problem. It implies that in many cases convexity can be a sufficient condition for optimization to be feasible, but it says nothing about whether it is a necessary condition.

Why don't you come across lots of theorems that list pseudoconvexity as an assumption? I don't know, but my guess is: Because pseudoconvexity is less well-known and less simple/clean than convexity (i.e., is not as amenable to clean proofs).

D.W.
  • 5,958
  • +1 to this! Just to name an example: Mixed integer programs are a class of nonconvex problems which have a lot of applicability, research interest, and (at least for modest scales) are not too difficult to solve in-practice (e.g., using SCIP/Gurobi/Express/etc). However, even though these problems are solvable in-practice, it is very difficult to prove pure theoretical results about them. So, many of the publications on this topic are not in math journals; they appear in engineering/CS/OR articles – Zim May 23 '24 at 07:31
  • I know that convex and non-convex optimization is NP-hard. So from a theoretical standpoint there is in general no difference between those two classes in terms of "hardness".

    But when I read topics which discuss convex vs non-convex optimization, it seems that convex optimization is practically "on average" the easier task. Can I see this somewhat similar to the simplex algo which in theory is exponential but in practice still very usable most of the time.

     – Sen90 May 23 '24 at 17:24
2

Just to add to the other answers, convexity is a property which (loosely speaking) admits both nice theory and efficient algorithms. However,

(A) there are plenty of nonconvex problems which are solvable with efficient algorithms (which sometimes lack proper theory, e.g., mixed integer programs have some very pessimistic worst-case-scenario bounds); and

(B) there is a huge desire within the research communiuty for more theoretical results for nonconvex functions. For instance, the SIAM-Optimization community recently awarded a "best paper" award for theoretical advancement for the optimization of weakly convex functions.

Zim
  • 4,623