3

Consider several linearly independent matrices $A_k \in \mathbb R^{m \times n}$ and the following equation $$ \operatorname{rank} \left(A_0 + \sum_{k=1}^r c_k A_k\right) = 1. $$ Here $A_k$ are fixed, $c_k$ are unknown real values and $r = n m - n - m + 1$.

I wonder what methods may be used to solve this type of equation.

I though about relation between singular values and trace: let $B = A_0 + \sum_{k=1}^r c_k A_k$. $$ \|B\|_F^2 = \operatorname{tr}(B^\top B) = \sum_i \sigma_i^2 = \sigma_0^2\\ \|B^\top B\|_F^2 = \operatorname{tr}(B^\top B B^\top B) = \sum_i \sigma_i^4 = \sigma_0^4 = \|B\|_F^4. $$ The latter equation is a forth degree equation in $c_k$. For square matrices probably it could be reduced to second degree multivariate equation, but still remains a complex problem.

uranix
  • 7,773
  • 1
  • 22
  • 57
  • Interesting problem. If I may ask, where does this problem come from? – Rodrigo de Azevedo Aug 14 '20 at 19:26
  • 1
    @RodrigodeAzevedo Originally it was solving $\sum_{ij} a_{pij} x_i y_j = b_p$. Considering $Z_{ij} = x_i y_j$ as unknown gives a linear problem $\sum_{ij} a_{pij} Z_{ij} = b_p$ with constraint $\operatorname{rank} Z = 1$ – uranix Aug 14 '20 at 20:48
  • So, if I understand correctly, originally, you had a system of bilinear equations ${\rm x}^\top {\rm A}_p {\rm y} = b_p$, right? You may want to take a look at this. – Rodrigo de Azevedo Aug 14 '20 at 20:51
  • @RodrigodeAzevedo Yes, it is exactly the same problem and I understand your solution. Unfortunately I don't see any benefit in rewriting a set of quadratic equation into a minimization problem which itself requires solving nonlinear problems to find the optimum. – uranix Aug 14 '20 at 21:06
  • Nonlinear, but convex. It's a heuristic. Better than nothing. The problem is that the minimizer found by the optimization solver is a vector of float values and that may not be enough. – Rodrigo de Azevedo Aug 14 '20 at 21:08
  • 1
    On the other thought your method might provide a good guess for Newton's iterations - that what I was lacking before. – uranix Aug 14 '20 at 21:11

1 Answers1

1

Given linearly independent matrices ${\rm A}_k \in \mathbb R^{m \times n}$, we have the following equation in ${\rm x} \in \mathbb R^r$

$$\operatorname{rank} \left( {\rm A}_0 + \sum_{k=1}^r x_k {\rm A}_k \right) = 1$$

Since the nuclear norm is a convex proxy for the rank, we could solve the following convex program in ${\rm x} \in \mathbb R^r$

$$\begin{array}{ll} \underset{{\rm x} \in \mathbb R^r}{\text{minimize}} & \left\| {\rm A}_0 + \displaystyle\sum_{k=1}^r x_k {\rm A}_k \right\|_* \end{array}$$

Let ${\rm x}^{\min}$ be the minimizer of this convex program. If

$$\operatorname{rank} \left( {\rm A}_0 + \displaystyle\sum_{k=1}^r x_k^{\min} {\rm A}_k \right) = 1$$

we are done. If not, we wasted a few minutes of our lives.

  • Do I understand correctly, "convex proxy for the rank" means that sometimes minimum of the rank is attained at the minimum of the nuclear norm? I've tried this approach for random problems, but the optimal solution has rank 2 or 3, never 1. – uranix Aug 16 '20 at 11:02
  • 1
    @uranix Maryam Fazel wrote a thesis about it. Some talks of hers can also be found on YouTube. I am not acquainted with the theory, but I have used nuclear norm minimization, for example, to find "sparse" sum of squares (SOS) decompositions of nonnegative polynomials. – Rodrigo de Azevedo Aug 16 '20 at 11:17
  • Hi I am currently working on a similar problem where the nuclear norm of the weighed sum of known matrices is minimized. But solvers like CVX takes too much time solving it when $r$ and the size of $A_k$ is large. I wonder if you happens to know some fast first order method for such problem? – Zieac Jan 04 '23 at 02:09
  • @Zieac Are the weights nonnegative? If the matrices are symmetric and PSD, then their conic combinations should also be symmetric and PSD — do not assume I am correct and please verify — in which case, the nuclear norm is just the trace. – Rodrigo de Azevedo Jan 04 '23 at 11:03
  • @Zieac How large are your matrices? 100 x 100? 100,000 x 100,000? – Rodrigo de Azevedo Jan 04 '23 at 11:10
  • @RodrigodeAzevedo My matrices are hermitian NxN with N~100, and there are N matrices in total. The weights are arbitrary so PSD assumption does not hold. Would be great if you can move to my post yesterday for more details. Thanks a lot for your help! – Zieac Jan 05 '23 at 07:51