0

When we have a standard minimization problem expressed in the form

\begin{align} &minimize \,\, \, f_0(x)\\ &s.t. \, \, \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, f_i(x)\le0 \qquad i=1, \dots,m\\ &\, \, \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, \, \, h_i(x)=0 \qquad i=1, \dots,p \end{align}

then we can take account of the constraints by considering a weighted sum of them, considering lagrangian

$$L(x,\lambda,\nu) = f_0(x) + \sum_{i=1}^m\lambda_if_i(x) + \sum_{i=1}^p \nu_ih_i(x) $$

and also define the dual function $g$ as $$ g(\lambda,\nu) := \inf_{x}L(x,\lambda,\nu)$$ Of course, when the lagrangian is unbounded below in $x$ the dual formulation takes the value $-\infty$.

What I don't understand is that since the dual function is the pointwise infimum of a family of affine functions of $(\lambda,\nu)$it is concave, even when the problem is not convex.

I understand that $g$ pointwise infimum of affine functions, (because they are basically linear functions), but I'm not relating this with the notion of always being concave.

Also my intuition brings me to consider that if $g$ is concave, then we end up with a maximization problem which will take us to the same optimal value of the original minimization problem, maybe under some additional conditions?

Thank you

James Arten
  • 2,023
  • 1
    To get some intuition as to why this thing is concave, draw some straight lines on a 2d plane and take their pointwise infimum (as if they were affine functions $ y = a x + b$). You will find that this is always a concave function. – Rammus Nov 23 '20 at 09:33
  • 1
    Note the supremum of $g$ may not be exactly equal to the infimum of the original problem. If this holds then we say the two problems are strongly dual. However, we will always have that the supremum of $g$ is a lower bound on the infimum of $f_0(x)$, this is known as weak duality. – Rammus Nov 23 '20 at 09:37
  • @Rammus that's exactly what I've just done. Drawing a bunch of straight lines on a plane and taking the pointwise infimum then it's clear that you obtain a concave function.. thank you everyone – James Arten Nov 23 '20 at 09:49
  • Duplicate of https://math.stackexchange.com/questions/1374399/why-is-the-lagrange-dual-function-concave – Brannon Mar 06 '24 at 15:47

1 Answers1

4

As mentioned, the function $L(x, \lambda, \nu)$ is affine with respect to $(\lambda, \nu)$. Therefore it is also concave. Hence, since $g(\lambda, \nu)$ is the minimum of concave functions, it is also concave. This is equivalent to the case of maximum of convex functions, which is convex.

Does this make sense? Or you still have a missing piece?

iarbel84
  • 1,623
  • 8
  • 9