4

Consider the following equation:

$$\left (A + \text{sgn}(x)\text{sgn}(x)^T \right )x= y \, x \tag{1} $$

where $A$ is an $n \times n$ symmetric matrix. The variables are $y \in \mathbb R, x \in \mathbb R ^n$ with $\| x \|_2=1$. Note this can be equivalently written as

$$Ax +\|x\|_1 \text{sgn}(x) =y \, x.$$

To avoid trivial or degenerate solutions, we restrict ourselves to the solutions $x \neq 0$ with $\| x \|=1$; indeed, any solution $x$ of this equation remains a solution after a linear transformation.

Such an equation appears in non-convex optimization, where all of its solutions need to be found so that the problem is fully solved. In fact the original system that needs to be solved is of the form

$$\color{blue}{\left [ \left(A + \text{sgn}(x)\text{sgn}(x)^T \right )x- y \, x \right ] \odot \text{sgn}(x) =0.}, \tag{2} $$

which reduces to the above system if we assume $x_i \neq 0, i=,\dots,n$. Also, if $A$ is a diagonal matrix, the two systems (1) and (2) become the same.

A naïve approach to solve the equation (1) is to fix $\text{sgn}(x)=a$ for any $a \in \{-1,1 \}^n$, and then solve the system. One can see that if $a$ is considered, $-a$ is not needed to be assessed as both produce the same matrix of $aa^T$. Hence, $2^{n-1}$ cases should be examined. After fixing $\text{sgn}(x)=a$, each solution $y$ and $x$ of the resulting system:

$$(A + aa^T )x= y x$$

is nothing but an eigenvalue and its normalized eigenvector of matrix $A+aa^T.$ Such a solution is a correct solution for our equation only if $\text{sgn}(x)=a$ or $\text{sgn}(x)=-a.$

When $A=0$, for each $a$, $\text{rank}(aa^T)=1$, and the system has the following $n$ solutions $(x,y)$:

$$(e_1,1),\dots, (e_n,1).$$

I have two related questions:

How many solutions can this system have (my guess the number of solutions is of order $O(n)$ though $2^{n-1}$ different possibilities can be obtained by fixing $\text{sgn}(x)$)?

Is there any efficient procedure to find all solutions of the above system (the computational complexity of the naïve approach is of exponential order)?

Amir
  • 11,124
  • 1
    Shouldn't $\operatorname{sgn}(x) \in {-1,0,1}^n$? – Brian Moehring Mar 11 '24 at 16:40
  • @BrianMoehring I think the $0$ case is covered by assuming that if a solution with $x_i=0$ for some $i$ is obtained, it is accepted in the checking phase, while $\text{sgn}(x_i)$ is fixed to $-1$ or $1$. Is that correct? – Amir Mar 11 '24 at 16:57
  • If $x_i=0$, the situation of the equation is the same as the case where the $i$th row of the matrix $\text{sgn}(x)\text{sgn}(x)^T$ becomes zero by setting $\text {sgn}(x_i)=0$. It is why I think we can ignore the 0 case to simplify the search procedure. – Amir Mar 11 '24 at 17:12
  • Where did this question come from. Context please – Mike Mar 11 '24 at 18:17
  • @Mike I obtained this equation after applying the KKT conditions in this answer: https://math.stackexchange.com/a/4877362/1231520. I think it is an interesting equation. I am wondering whether the naive approach described in the OP is the only approach or not? – Amir Mar 11 '24 at 18:41
  • Do we really need to convert the problem to an equivalent problem to discuss the KKT conditions? I think this can be written without this step and nondifferentiability should not be an issue! See: https://www.cs.cmu.edu/~ggordon/10725-F12/slides/16-kkt.pdf. – Sam Mar 16 '24 at 08:09
  • @Sam I think using subgradients, a similar system can be obtained, while I cannot see how to deal with the fact that at $0$ the subdifferential of $|x|$ is $ [-1,1]$. – Amir Mar 16 '24 at 18:41

0 Answers0