A non-convex optimization problem is one where either the objective function is non-convex in a minimization problem (or non-concave in a maximization problem) or where the feasible region is not convex.
Questions tagged [non-convex-optimization]
695 questions
35
votes
3 answers
What is the definition of a first order method?
The term "first order method" is often used to categorize a numerical optimization method for say constrained minimization, and similarly for a "second order method".
I wonder what is the exact definition of a first (or second) order method.
Does…
Wazowski
- 537
22
votes
1 answer
Gradient descent on non-convex function works. How?
For Netflix Prize competition on recommendations one method used a stochastic gradient descent, popularized by Simon Funk who used it to solve an SVD approximately. The math is better explained here on pg 152. A rating is predicted by…
BBSysDyn
- 16,513
19
votes
7 answers
Maximize the value of $v^{T}Av$
Let $A$ be a symmetric, real matrix. The goal is to find a unit vector $v$ such that the value $v^{T}Av$ is
maximized, and
minimized.
The answer is that $v$ should be the eigenvector of $A$ with
largest eigenvalue, and
smallest eigenvalue.…
user4205580
- 2,213
15
votes
3 answers
How do I solve $\min_x \max(c_1^Tx, c_2^Tx, \dots, c_k^Tx)$ for $\lVert x \rVert_2 = 1$.
Let $f(x) = \max(c_1^Tx, c_2^Tx, \dots, c_k^Tx)$.
where $x, c_1, c_2, \dots, c_k \in \mathbb R^n$.
What fast iterative methods are available for finding the (approximate) min of $f$ with the constraint $\lVert x \rVert_2 = 1$?
Notes:
$f$ is convex…
cairnc
- 183
14
votes
1 answer
Why is the non-negative matrix factorization problem non-convex?
Supposing $\mathbf{X}\in\mathbb{R}_+^{m\times n}$, $\mathbf{Y}\in\mathbb{R}_+^{m\times r}$, $\mathbf{W}\in\mathbb{R}_+^{r\times n}$, the non-negative matrix factorization problem is defined…
no_name
- 445
14
votes
5 answers
Stable strict local minimum implies local convexity
Let $\bar{x}\in\mathbb{R}^n$ and $f:\mathbb{R}^n\rightarrow\mathbb{R}$ be a $C^2$ function.
We have known that if $\nabla f(\bar{x})=0$ and $\nabla^2f(\bar{x})>0$, i.e. $\nabla^2f(\bar{x})$ is positive definite, then $\bar{x}$ is a strict local…
Blind
- 1,076
14
votes
3 answers
Solve least-squares minimization from overdetermined system with orthonormal constraint
I would like to find the rectangular matrix $X \in \mathbb{R}^{n \times k}$ that solves the following minimization problem:
$$
\mathop{\text{minimize }}_{X \in \mathbb{R}^{n \times k}} \left\| A X - B \right\|_F^2 \quad \text{ subject to } X^T X =…
Alec Jacobson
- 573
11
votes
2 answers
Linear programming with one quadratic equality constraint
I have a problem that can be formulated as a linear program with one quadratic equality constraint:
where variable $x$ is an $n$-dimensional vector and $H$ is a positive semidefinite $n \times n$ matrix.
I know this optimization problem can always…
user123346
- 111
11
votes
0 answers
Balanced linear partitioning of a set of points in $R^d$
Suppose we have a set of points in $R^d$ and for a given constant $\epsilon>0$ we want to find a hyperplane such that it divides the dataset into two balanced partitions, and that the number of points that are $\epsilon$-close the hyperplane is…
kvphxga
- 725
10
votes
1 answer
Does there exist a strongly convex function that is strongly convex with respect to norm $\|\cdot\|_p$ for any $p > 2$?
A function $f$ is said to be strongly convex with respect to a norm $\|\cdot\|_p$ if for all $x,y$, $$f(x) \geq f(y) + \nabla f(y)^T(x-y) + \frac{1}{2}\|x-y\|^2_p.$$
There are a bunch of functions used in machine learning, statistics, etc. that are…
Norman
- 1,246
10
votes
3 answers
Box-constrained orthogonal matrix
Given constants $\ell, u \in \mathbb{R}^{3 \times 3}$ and the following system of constraints in $P \in \mathbb{R}^{3 \times 3}$
$$
P^T P = I_{3 \times 3},\quad \ell_{ij} \leq P_{ij} \leq u_{ij},
$$
I would like to find a matrix $P$ which satisfies…
Alex Shtoff
- 168
9
votes
2 answers
Question about KKT conditions and strong duality
I am confused about the KKT conditions. I have seen similar questions asked here, but I think none of the questions/answers cleared up my confusion.
In Boyd and Vandenberghe's Convex Optimization [Sec 5.5.3] , KKT is explained in the following…
user2020
- 93
8
votes
1 answer
Clarke's tangent cone, Bouligand's tangent cone, and set regularity
For a set $C$ (which may not be convex) and a point $x\in C$:
Bouligand's Tangent cone is defined as
$$
T(C,x) = \left\{v : \lim_{\theta\to 0_+} \inf \frac{d(x+\theta v, C)}{\theta} = 0\right\}
$$
and where $d(x,C) = \min_{y\in C} \|x-y\|$ the…
Y. S.
- 1,876
8
votes
1 answer
Minimizing Quadratic Form with Norm and Positive Orthant Constraints
Let $ M $ be a positive semi definite matrix.
I want to solve
$$ \arg \min_{x} {x}^{T} M x \quad \mathrm{s.t.} \quad \left\| x \right\| = 1, \ x \succeq 0 $$
where $ x \succeq 0 $ means each coordinate of $x$ is nonnegative.
Is there a standard…
user7530
- 50,625
8
votes
3 answers
Why is the constraint $\|w\| = 1$ non-convex?
Related: Why is this function, related to SVM derivation, non-convex?
I am studying notes which cover the derivation of SVM. The intuition is the geometric margin should be maximized in order to result in more confident predictions. The notes pose…
Alex
- 285