9

Let $I$ and $J$ be compact intervals. Let $f:I\times J\to\mathbb R$ be differentiable and strictly convex. Is the function $g:I\to\mathbb R$ defined by $$ g(x) = \min_{y\in J} f(x,y) $$ convex?

Remarks:

  • I know that minimum of convex functions is in general not convex. However, I can't find a counter example in which $f$ is convex.

  • The regularity ensures that the minimizer $y^*(x)$ of $f(x, \cdot)$ is unique.

  • Assume $y^*$ as function is convex, $y^*$ maps $I$ into an interval $J^*$, and $f(x, \cdot)$ is increasing on $J^*$ for every $x\in I$. Then, $g$ is convex.

Thanks for any input :)

user251257
  • 9,417

2 Answers2

10

It is convex!

Your first statement that the minimum of convex functions is in general not convex is true, but here you have a lot more structure! In a sense you are projecting onto $x$. In fact, $g$ is also called the inf-projection of $f$. Let $\lambda \in (0,1)$ and $y_1, y_2 \in J$ arbitrary:

$$ \begin{aligned} g(\lambda x_1 + (1-\lambda) x_2) &= \min_{y} f(\lambda x_1 + (1-\lambda)x_2, y) \\ &\leq f(\lambda x_1 + (1-\lambda)x_2, \lambda y_1 + (1-\lambda)y_2)\\ &\leq \lambda f(x_1, y_1) + (1-\lambda) f(x_2,y_2)\\ \end{aligned} $$

Now first minimize with respect to $y_1$, then with respect to $y_2$ to finally get: $$g(\lambda x_1 + (1-\lambda) x_2) \leq \lambda g(x_1) + (1-\lambda) g(x_2)$$

Also notice that you do not need the regularity conditions you imposed onto $f$.

air
  • 2,871
  • Hi, can you clarify the last two step. In my understanding, $y_1, y_2$ are arbitrary value of $y$, thus if you first minimize wrt y1, you will have $ f(x_1, y_1) \ge Min_{y} f(x_1, y)$, and same as wrt y2, therefore, it should be $(1,_1)+(1−)(_2,_2) \ge Min{y} f(x_1,y) + (1 - )Min_{y}f(x_2, y)$not the other way around. – sundaycat Aug 25 '24 at 22:56
  • The left hand side has no $y$ in it and the inequality holds for all $y_1$, $y_2$. – air Aug 27 '24 at 03:07
7

Theorem: Let $X,Y$ be real linear spaces and $f\colon X\times Y\to [-\infty,+\infty]$ be convex. Then $$ \phi(x)=\inf_{y\in Y}f(x,y) $$ is convex.

Proof: Let $E$ be the image of $\text{epi}(f)$ under the projection $(x,y,\alpha)\to (x,\alpha)$. Then by definition of infimum $$ \text{epi}(\phi)=\{(x,\alpha)\in X\times\mathbb{R}\colon \ (x,\beta)\in E,\ \forall\beta>\alpha\}.\tag1 $$ The epigraph of $f$ is convex, then $E$ is convex (as a linear image of the convex set), and $(1)$ yields then that $\text{epi}(\phi)$ is convex as an intersection of convex sets $E_\epsilon=E-(0,\epsilon)$, i.e. $$ \text{epi}(\phi)=\bigcap_{\epsilon>0}E_\epsilon. $$

P.S. Since a convex function $f$ on any set $S\subset X\times Y$ can be extended to the whole space by (re)defining $f=+\infty$ outside $S$, the generality is not lost by assuming that $f$ is defined on $X\times Y$.

A.Γ.
  • 30,381
  • Irrespective of whether your answer is correct or wrong, here you're talking about linear spaces; how does it link back to the user's question where $X$ and $Y$ are compact sets ? – dohmatob Aug 24 '15 at 02:43
  • 1
    @dohmatob I am talking about functions to the extended real line. One can always (re)define $f=+\infty$ outside a set of one's particular interest, so no generality is lost to replace it with the whole space. – A.Γ. Aug 24 '15 at 03:07
  • I'm a fan of extended real treatment myself. But yes, I'd agree this doesn't quite answer the question without that clarification. – Michael Grant Aug 24 '15 at 03:43
  • @A.G.: Indeed you can always do thusly. I was worried about pedagogy for the user. Still concerning pedagogy, there is a significant amount of non-trivial detail buried in your answer, for example: $\alpha \ge \phi(x) \iff \exists y_0 \in Y | \beta \ge f(x, y_0) \forall \beta > \alpha$, etc. Also, your extension to linear spaces does no good to the remainder of the problem. – dohmatob Aug 24 '15 at 03:46
  • @user251257: Proposition A.22 of the PhD thesis of Bertsekas http://web.mit.edu/dimitrib/www/phdthesis.pdf provides a extenstion to Danskin's Theorem, which establishes the first parts of your result, plus much more. Think of Danskin's Theorem (and generalizations thereof) whenever you see problems like yours :) – dohmatob Aug 24 '15 at 04:33
  • @A.G. could you explain, why ${(x,α) \in X\times \mathbb R: (x,\beta)\in E, \forall \beta > \alpha }$ is convex? – user251257 Aug 24 '15 at 13:40
  • @dohmatob: thanks for your comment. I can find proposition A.22 in your the thesis your referred. Could you tell me the page number? As concerning Danskin's Theorem, I am don't really need the differentiablility. I was only wondering about the convexity. – user251257 Aug 24 '15 at 13:45
  • @user251257 Jawohl :-) Added the explanation. – A.Γ. Aug 24 '15 at 16:37
  • @dohmatob Could you, please, explain how Danskin's Theorem helps in establishing convexity of the infimum? – A.Γ. Aug 24 '15 at 16:44
  • @user251257 and A.G.: Oops! My comment makes no sense whatsoever. At one point I misread the problem and saw the fantôme of a supremum. Of course Danskin's Theorem is only applicable to problems of "sup-type" (which is not your case). Plus, one doesn't need such machinery here (for example, as seen in A.G.'s solution). Please disregard. – dohmatob Aug 25 '15 at 06:52
  • @user251257: Is the source of your problem publicly available ? This could be helpful in attacking the remainder. – dohmatob Aug 25 '15 at 06:55
  • @dohmatob: I am quite happy with the answers I got. What is the remainder you referred to? It is not an assignment or something. I was just curious. – user251257 Aug 25 '15 at 13:42
  • @user251257: OK, the remainder sounded a bit adhoc to me, and so I was wondering whether there was a source for it, to get the original formulation, etc. – dohmatob Aug 25 '15 at 15:24
  • @dohmatob: You meant the remarks? That was what I have tried and came up with. Nothing of particular importance. – user251257 Aug 25 '15 at 15:26
  • Ah!, those were remarks ? I think I misread your question entirely then. Luckily, others paid more attention and you got an answer :) – dohmatob Aug 25 '15 at 15:35