3

I'm working through a support vector machines tutorial. In eventually deriving the solvable objective function, the following objective function (to be maximized) was proposed, but dismissed as non-convex (i.e., it's negative is non-convex). Why is it non-convex?

$$ \newcommand{\norm}[1]{\lVert#1\rVert} f(\hat{\gamma}, w, b) = \frac{\hat{\gamma}}{\norm{w}} = \min_{i=1, \dots, m} y^{(i)} \left( \frac{w^{T}}{\norm{w}} x^{(i)} + \frac{b}{\norm{w}} \right) $$

where the label variable $y^{(i)} \in \{-1,1\}$, the feature vector $x^{(i)} \in \mathbb{R}^{n}$, $w \in \mathbb{R}^{n}$, and $b \in \mathbb{R}$.

The (convex) constraints are: $$ y^{(i)} (w^{T}x^{(i)} + b) \ge \hat{\gamma},\; i=1, \dots, m. $$

JefferyRPrice
  • 153
  • Please define what spaces your variables live in and any constraints on them. And in which variable is it non-convex with respect to. – muaddib May 23 '15 at 13:10
  • @muaddib Updated to define spaces and added the constraints. I do not know in which variable it is non-convex. – JefferyRPrice May 23 '15 at 13:52
  • Perhaps I'm overcomplicating this. If I were to consider minimizing $(-1/\lvert w \rvert)$ (i.e., let $w \in \mathbb{R}^{1}$), it's clear that this is not convex wrt $w$. – JefferyRPrice May 23 '15 at 23:06
  • 1
    I think you're right (re: overcomplicating). It seems a bit odd to me to ask "why is this function non-convex?" It is non-convex because, well, it is not convex. Convex functions are the exception, not the rule. If you can't prove something is not convex, you must necessarily assume that it is not. Have you made any attempt at all to prove that it is convex? – Michael Grant May 24 '15 at 20:50
  • 1
    Ah, see that you have, given your analysis of $-1/|w|$... carry on, jolly good, :-) – Michael Grant May 24 '15 at 21:13

1 Answers1

2

As noted in the comments (thanks Michael Grant), I was overcomplicating things. It's easy to show non-convexity with respect to $w$. First, consider $w \in \mathbb{R}^{1}$ and show that (the negative of) $f$ is not convex by checking a couple of values of $w$. Secondly, note that $f$ is undefined for $w = \vec{0}$, implying that the $w$ domain of $f$ is non-convex in $\mathbb{R}^{n}$.

JefferyRPrice
  • 153