1

I am trying to solve a linear programming with the $\ell_0$-norm constraints, which give a constrain on the element of the variables. E.g., I have $3$ variables, $x_1$, $x_2$, $x_3$ and I have some "normal" constraints, like below

$x_1 > 0,$

$x_1 < 0.3,$

$x_2 > 0,$

$x_1 < 0.4,$

$x_3 > 0,$

$x_1 < 0.5.$

Besides I have a $\ell_0$-norm like constraints.

$$|x_1|_{0} + |x_2|_{0} +|x_3|_{0} \leq 2$$

That is the maxinum amount of the chosen variable from $x_1$, $x_2$, $x_3$ is $2$.

The objective function (to get mininum) is

$$ f = -(x_1+x_2+x_3) $$

So the answer should be $[0, 0.3, 0.4]$, that is $x_2$ and $x_3$ chosen.

How to deal with this kind of linear programming? I know it may be a non-convex problem. Also a $\text{LP}$ problem. Could it be change to a Mixed-integer linear programming ($\text{MILP}$) problem? Or is there any usual way to solve it? Any idea is appreciated. Just some key words or workflows is well, too. I need to know more about it.

Rodrigo de Azevedo
  • 23,223
Zhang Wei
  • 147

1 Answers1

1

You can make it into a MILP by introducing binary variables indicating if the corresponding $x$ values are zero or not. In your case, assuming $x$ are nonnegative: $$ \begin{array}{l} x_i\leq Mz_i\\ z_i\in\{0,1\}\\ z_0+z_1+z_2\leq 2 \end{array} $$ where $M$ is a reasonable constant, will give you the bound you are looking for. See for instance https://docs.mosek.com/modeling-cookbook/mio.html#implication-of-positivity for more on this.

In general problems with $L_0$ norm are hard. A typical approximation is to replace the $0$-norm with $1$-norm and include the $1$-norm as penalty in the objective. This is just a heuristic but it tends to prefer sparse solutions. See for instance from slide 17 in https://web.stanford.edu/class/ee364b/lectures/l1_slides.pdf.

Michal Adamaszek
  • 4,147
  • At first, the method in mosek as you mentioned is great. I transfer the problem to MILP successfully. But to the 0-norm replacement, I need to ensure my idea. I think I could get a heuristic solution, as initial point in MILP, from the 1-norm penalty. But I need a search from the heuristic solution or change the penalty term's weight to get a global optimal result. Am I right? – Zhang Wei Aug 20 '20 at 06:50
  • @ZhangWei You are right, it requires more work to get the global optimum. There is a whole lot of literature about algorithms and heuristics for 0-norm, and a lot depends on the application. I can't really help here. Just wanted to indicate the 1-norm as a possible starting point since it fits well to LPs. – Michal Adamaszek Aug 20 '20 at 06:58
  • I want to change this problem to MILP problem because there is ready-made MILP function in Matlab or Mosek. It is hard for me to set the Branch and Bound method by myself. Thank you for your advise. I really learn much from it. The general idea you mentioned is not easily got from book. Really grateful. – Zhang Wei Aug 20 '20 at 07:12