I hear that $f(x)=x^4$ is a strictly convex function $\forall x \in \Re$. However, strict convexity condition is that the second derivative should be positive $\forall x \in \Re$. For the mentioned function, the second derivative is zero at $x=0$, which is in the domain of $f$. Therefore, it should not be a strict convex function. But I am pretty sure it is because I heard it in Prof. Boyd's lecture.
Am I missing something obvious?
If the second derivative is strictly positive, then the function is strictly convex. However, the converse need not be true. A function $f:\Bbb R\to\Bbb R$ is strictly convex if and only if for all $x,y\in\Bbb R$ with $x\neq y$ we have $$f\bigl(tx+(1-t)y\bigr)<tf(x)+(1-t)f(y)$$ for all $0<t<1$. $f(x)=x^4$ is indeed strictly convex.
This is a common misconception. Many make the same mistake regarding the relationship between positive first derivative and increasing functions (the former implies the latter, but not vice versa). See here and here, for examples of people making such errors.
x=0 is zero, we say that $x^4$ is strictly convex. Now my question is how to analytically see if the function is indeed strictly convex? (In case, we cannot plot the function and see if the line segment joining any two points lies above the function)
– Autonomous
Mar 05 '13 at 05:00
Strict Convexity is when $f(tx+(1-t)y)<tf(x)+(1-t)f(y)$. The fact that $f''(x)\geq 0$ implies $f$ is convex, however, it does not necessarily imply that $f$ is not strictly convex. In the case of $f(x)=x^4$, you get $f''(x)=12x^2\geq 0$ but in fact $f$ is strictly convex. Indeed, any line segement lies above the curve.
You can use the condition $\langle \nabla f(y) - \nabla f(x), y - x \rangle > 0$ to check the strict convexity. For $f(x)=x^4$, the above condition becomes $4(y-x)^2 \left[(y+ \frac{1}{2}x)^2 + \frac{3}{4}x^2\right] > 0$ with $x \neq y$. This example reminds us that the positive definite Hessian is sufficient, but not necessary, for strict convexity.
To check that a function is strictly convex, you can use the following criterion: the graph of a function that is convex but not strictly convex contains a straight line interval. In other words, such a function coincides with a linear function on some interval, or yet in other words, its second derivative vanishes identically on some interval. Since this is not the case for $x^4$, and it is convex, we can conclude that it is strictly convex.
Proof of the criterion: If a function is convex but not strictly convex, then if satisfies $$f(tx+(1-t)y)=tf(x)+(1-t)f(y)$$ for some $x,y$ and $t\in(0,1).$ Geometrically, it means that the point $Z=(z,f(z))$ lies on the straight line segment $[XY]$ connecting $X=(x,f(x))$ and $Y=(y,f(y))$, for some $z\in(x,y)$. Now, pick any $u,v$ with $x<u<z<v<y.$ By convexity, both points $U=(u,f(u))$ and $V=(v,f(v))$ are below or on $[XY]$, hence so is the whole segment $[UV]$ between them. Moreover, if at least one of $U,V$ were not on $[XY]$, then the whole open segment $(UV)$ would be strictly below $[XY]$. But by convexity, the graph of $f$ between $u$ and $v$ is below $[UV]$, in particular, so is the point $Z$ on that graph. Therefore, $Z$ could not be on $[XY]$, which is a contradiction showing that both $U$ and $V$ must be on $[XY]$. But since $u$ and $v$ were arbitrary, this means that the whole graph of $f$ between $x$ and $y$ is just the segment $[XY]$.
Second derivative being positive is for strong convexity. There is a subtle difference between strict convexity and strong convexity. A strongly convex function is strictly convex but the converse need not be true.
The condition for strict convexity is strict Jensen's inequality as pointed out by Alex R.
