8

Let $ M $ be a positive semi definite matrix.

I want to solve $$ \arg \min_{x} {x}^{T} M x \quad \mathrm{s.t.} \quad \left\| x \right\| = 1, \ x \succeq 0 $$ where $ x \succeq 0 $ means each coordinate of $x$ is nonnegative.

Is there a standard approach for attacking this problem?
Without the inequality constraint, the usual approach is to expand $ x $ in the eigenvector basis of $ M $ and to notice that the solution must be the eigenvector with least eigenvalue.
Without the equality constraint, obviously the solution is $ x = \boldsymbol{0} $.

Royi
  • 10,050
user7530
  • 50,625

1 Answers1

5

I would use the Projected Gradient Descend for this case.
Though the problem isn't Convex it will work nicely.

The algorithm is as following:

  1. Calculate the Gradient at the current point.
  2. Update the solution $ x = x - 2 t M x $ where $ 2 M x $ is the Gradient of the Objective Function and $ t $ is the step size.
  3. Project the output of previous step into $ {\mathbb{R}}_{+} $ by $ {x}_{i} = \max \left\{ {x}_{i}, 0 \right\} $.
  4. Project the output of previous step onto the Unit Sphere by $ {x}_{i} = \frac{ {x}_{i} }{ \left\| x \right\|_{2} } $.
  5. Go back to (1) (Or check validity of the point, KKT will do even the problem isn't Convex).

In a simple 2D example I created which worked pretty well:

enter image description here

The code is available at my StackExchange Mathematics Q2699867 GitHub Repository.

Remark 001
I'd even considering starting with the solution of the Convex Problem when you replace the equality constraint $ \left\| x \right\|_{2} = 1 $ by $ \left\| x \right\|_{1} = 1 $ (This will make the problem basically constraining $ x $ to the Unit Simplex). You can either use it as a starting point for the above algorithm or approximated solution by itself.

Remark 002
Another approach might be something like I did in - Solution for $ \arg \min_{ {x}^{T} x = 1} { x}^{T} A x - {c}^{T} x $.
Yet after each iteration of updating $ \lambda $ you should also project the output $ x $ into $ \mathbb{R}_{+} $.

Royi
  • 10,050
  • Note to my self - This might be related - https://math.stackexchange.com/a/2407669/33. – Royi Mar 24 '18 at 20:15
  • Regarding remark 1, even the 1 norm equals 1 is a non-convex constraint right? – Sridhar Thiagarajan Aug 16 '18 at 06:38
  • @SridharThiagarajan, Yep. Any non linear equality constraint makes the problem non convex. – Royi Aug 16 '18 at 06:49
  • Why does your remark say 'I'd even considering starting with the solution of the Convex Problem when you replace the equality constraint ‖x‖2=1 by ‖x‖1=1' then? – Sridhar Thiagarajan Aug 16 '18 at 08:46
  • I wasn't clear on that. But what I meant is if you have $ x \succeq 0 $ and $ {\left| x \right|}_{1} = 1 $ then the problem become Convex as its equivalent is having $ x $ in the Unit Simplex. – Royi Aug 16 '18 at 09:34
  • Shouldn't that be $||x||_{1} \leq 1$, that equality constraint of yours looks non-convex. – Sridhar Thiagarajan Aug 16 '18 at 11:31
  • Have a look at the following constraints: $ x \succeq 0 $ and $ \sum_{i = 1}^{n} {x}_{i} = 1 $. Those are the constraints of being in the Unit Simplex which is a convex set. – Royi Aug 16 '18 at 11:51
  • Now, what do you think about $ x \succeq 0 $ and $ {\left| x \right|}{1} = \sum{i = 1}^{n} \left| {x}_{i} \right| = 1 $? Do you see the equivalence when $ x \succeq 0 $? – Royi Aug 16 '18 at 11:53
  • Got it! I suppose it's non convex only in the absence of that first inequality. This would be $x \geq 0$ and $1'x=1$ – Sridhar Thiagarajan Aug 16 '18 at 18:54