11

Let $A \in S^{n}_{+}$ be a positive semi-definite matrix with all entries being non-negative. I wonder if there is an analytical solution to the following SDP in correlation matrix $X \in S^{n}_{+}$

$$\begin{array}{ll} \underset{X \in S^{n}_{+}}{\text{minimize}} & \mbox{Tr} (A X)\\ \text{subject to} & X_{ii} = 1, \quad \forall i \in [n]\end{array}$$

Does this optimization problem have an analytical solution?

To share some idea on the objective, consider the spectral decomposition of matrix $A$

$$A = \sum_{k} \lambda_k y_k y^{T}_k$$

with $\lambda_k > 0$ being the positive eigenvalues of matrix $A$ and $y_k \in \mathbb{R}^{n}$ the corresponding eigenvectors. The objective is to minimize the weighted sum of variances, i.e.,

$$\mbox{Tr} (A X) = \sum_{k} \lambda_k y^{T}_k X y_k$$

The dual problem is

$$\begin{array}{ll} \underset{D \text{ is diagonal}}{\text{maximize}} & \mbox{Tr}(D)\\ \text{subject to} & A \succeq D\end{array}$$

The problem is so neat, so I wonder if there is any hope to have an analytical solution.

Rodrigo de Azevedo
  • 23,223
zxzx179
  • 1,577
  • One other way to notate the minimization is as $\min_{X\in S_+^n}$. That's a small thing but it makes the domain a bit more transparent. – Semiclassical Feb 24 '21 at 18:25
  • @Semiclassical Thanks! I have adopted the notation. – zxzx179 Feb 24 '21 at 18:37
  • $A$ is https://mathworld.wolfram.com/DoublyNonnegativeMatrix.html – Rodrigo de Azevedo Feb 24 '21 at 18:47
  • 2
    Since $X$ is a correlation matrix, it can be written as $V^T V$ where the columns of $V$ are all unit vectors. (This is a special case of what's stated in Rodrigo's link.) As such, your weighted sum of variances can be expressed as $$\sum_k \lambda_k y_k^\top V^\top V y_k = \sum_k \lambda_k |V y_k|^2.$$ Here I am less confident about how to proceed, but this is certainly is bounded below by 0 (as it should be---$X\succeq 0$, after all) and only achieves this value if $Vy_k=0$ for all $k$. So we want to pick $V$ with kernel containing most of the $y_k$. – Semiclassical Feb 24 '21 at 20:36
  • By the way, the set of n-by-n real PSD matrices with ones on the diagonal is known as the elliptope. See for instance the definition here. – Semiclassical Feb 24 '21 at 21:06
  • @Semiclassical Thanks a lot! Very helpful! – zxzx179 Feb 24 '21 at 23:49
  • 1
    I just realized: The elliptope terminology originates from papers of Laurent and Poljak in the 90s. The second such paper (as far as I know) is Laurent and Poljak 1996, and section 5 is explicitly about this exact minimization problem. They don't solve it, but they do give the context in which it appears. Moreover, that seems a good starting point for a literature hunt to determine the present status of the problem. – Semiclassical Feb 25 '21 at 00:15
  • For a more recent source, see the opening paragraphs of this paper: https://powei.tw/maxcut.pdf. Thus this SDP is actually considered a canonical example of such, with applications to obtaining approximate solutions to the (NP-hard) MAXCUT problem. However, I'll stress that this is posed as an algorithmic problem rather than an analytical one. The only advantage I can see is the additional assumption that $A$ is doubly-nonnegative, but it's not clear if this actually helps. – Semiclassical Feb 25 '21 at 01:45
  • @Semiclassical Thanks a ton! I lose my hope from the algorithm paper you pointed. Yeah, the fact that matrix $A$ is positive semi-definite gives the problem more physical meaning (minimize the weighted sum of variances), but indeed it is hard to see how this helps. – zxzx179 Feb 25 '21 at 03:11
  • Take a look at this. – Rodrigo de Azevedo Feb 27 '21 at 16:07
  • @Semiclassical I started a bounty. In case you feel like giving it a go. – Rodrigo de Azevedo Feb 27 '21 at 23:47
  • @RodrigodeAzevedo Thank you! The edit is much better than the earlier one. – zxzx179 Feb 28 '21 at 18:42
  • @RodrigodeAzevedo I have also read your link. It is interesting: the optimization problem here is an SDP relaxation to the binary program in the link. There you mentioned that the binary program is hard (maybe you can be more clear what hard means there?) even when $Q \succeq 0$. Does this mean that even the SDP relaxation does not provide a good approximation? I'm also considering things like random hyperplane rounding to get a feasible solution to the binary problem, which for the maximization case, provides a $\frac{2}{\pi}$ approximation. Seems like it's not true for the minimization case. – zxzx179 Feb 28 '21 at 18:44
  • @Stupid_Guy What I meant is that even when the objective function is easy to minimize over the reals — due to convexity — it is hard over ${ \pm 1}$. Which makes sense, because restricting to ${ \pm 1}$ is equivalent to imposing (non-convex) quadratic equality constraints. – Rodrigo de Azevedo Feb 28 '21 at 20:44
  • @Stupid_Guy Take a look at this. If you use a hyperplane, you will find the hyperplane that is tangent to the spectrahedron. If the point at which they touch is a "vertex", then you have a rank-$1$ matrix and, thus, you have solved the original binary program. – Rodrigo de Azevedo Feb 28 '21 at 20:46

0 Answers0