4

I am attempting to maximize a positive semidefinite quadratic form over the standard simplex.

Given a symmetric positive semidefinite (Hessian) matrix $A \in \Bbb R^{d \times d}$ and a matrix $W \in \Bbb R^{d \times n}$,

$$\begin{array}{ll} \underset{z \in \Bbb R^n}{\text{maximize}} & z^\top W^\top A W z\\ \text{subject to} & \Bbb 1_n^\top z = 1\\ & z \geq \Bbb 0_n\end{array}$$

where $z_i \in [0,1]$ is a probability value used to proportionally weight each column of $W$.

I tried to solve this problem by utilizing the fact that given a constraint $z^\top z = 1$, the $z$ that maximizes $z^\top W^\top A W z$ is the first eigenvector of the matrix $A$. But I'm not sure whether this is the right way.

Thank you.

Rodrigo de Azevedo
  • 23,223
JJumSSu
  • 65

1 Answers1

2

In general your problem is a non-convex QP with potentially many local maxima (consider for instance that even in the simplest case $W^TAW = I$, every nontrivial corner of the simplex is a local maximum).

If $n$ is sufficiently small, testing all $2^n$ active sets will definitely work. I don't know an analytic solution for your specific constraints; branch-and-bound techniques might be able to solve it numerically in practice.

user7530
  • 50,625
  • Thank you for the answer. Seems that my problem is tricky. Unfortunately n is not sufficiently small. – JJumSSu Aug 21 '20 at 02:00
  • If A is a positive definite matrix instead of PSD, would my problem be solved more easily? – JJumSSu Aug 21 '20 at 02:00
  • @JJumSSu No, that wouldn't help. To make the problem convex you'd need the matrix to be negative (semi)-definite. – user7530 Aug 21 '20 at 02:01
  • I see. Thank you for the novel answer and the comment. – JJumSSu Aug 21 '20 at 02:06
  • The suggested approach obviously won't work: consider $A=Id+J$ where $J$ is the matrix of all $1$'s, so the first eigenvector is unique and equals $(1,\dots,1)$. Do you really believe that $(1/n,\dots,1/n)$ is a maximizer? If so, compute the value and compare with that at $(1,0,\dots,0)$. – fedja Aug 22 '20 at 13:28
  • @fedja I assume that the OP intends the largest-eigenvalue eigenvector when they say "first", as that's what makes sense. – user7530 Aug 22 '20 at 14:43
  • @user7530 Yes, that is what I mean too and it gives you a wrong answer in the example I pointed out. – fedja Aug 22 '20 at 16:47
  • @fedja I see; you're right. – user7530 Aug 22 '20 at 17:20
  • What are the $2^n$ points of the active set to be tested? Must not the max occur at one of the $n$ corners of the simplex? – Hasse1987 Feb 07 '22 at 21:46
  • @Hasse1987 You are right; in general there are $2^n$ active sets, but in this problem you can skip checking all but the $n$ corresponding to simplex corners. – user7530 Feb 07 '22 at 21:58