4

I am having trouble understanding the logic behind optimization of cost function of the form $$\min (|x| + |y| + |z|) \,$$ subject to constraints $$Ax \le b \qquad Cx = d $$ such as $$ x + y \le 1 \qquad 2x + z = 3.$$

I have seen methods involving representing absolute values as a new variable and putting constraints on them, $i.e$ $$|x| = a \qquad -a \le x \le a$$ but I don't understand why should we represent an equality as inequality. Aren't we changing the equation itself?

I have tried to think but am unable to grasp it. Please help.

The method is available on Wikipedia as a numerical example:- https://optimization.mccormick.northwestern.edu/index.php/Optimization_with_absolute_values

EDIT:- The relevant example

Rodrigo de Azevedo
  • 23,223
Manish
  • 565
  • Eliminating $x_1$ with $x_1 = x_2 +5$ it's easy to see that the example problem is equivalent to minimizing $|x_2|$ subject to $x_2 \geq \frac{5}{3}$. Therefore the absolute value can be dropped and the solution is $x_1 = \frac{20}{3}$, $x_2 = \frac{5}{3}$ and the optimum value is $\frac{35}{3}$. – Fabio Somenzi Feb 22 '17 at 18:17
  • Do you have to solve this problem using LP? If don't, you can just solve a convex problem by changing the objective by the quadratic form $x^2+y^2+z^2$. – Alex Silva Feb 23 '17 at 10:44
  • @Alex, actually yes. I just wanted to know the logic behind the method, since people have done it. – Manish Feb 25 '17 at 09:58
  • how do we get the values of x1, x2? – ayush singhal Aug 12 '21 at 01:27

2 Answers2

10

From section 6.1.1 of Boyd & Vandenberghe:

Sum of absolute residuals approximation

When the $\ell_1$-norm is used, the norm approximation problem $$ \text{minimize} \quad \| A x - b \|_1 = | r_1 | + \dots + | r_m | $$ is called the sum of (absolute) residuals approximation problem, or, in the context of estimation, a robust estimator (for reasons that will be clear soon). Like the Chebyshev approximation problem, the $\ell_1$-norm approximation problem can be cast as an LP $$ \begin{array}{ll} \text{minimize} & {\bf 1}^T t \\ \text{subject to} & -t \preceq A x - b \preceq t, \end{array} $$ with variables $x \in {\bf R}^n$ and $t \in {\bf R}^m$.

Rodrigo de Azevedo
  • 23,223
2

On that page, they are solving another problem. They are trying to make the constraint |x| < b, so it's true that they can split that inequation in those two. In your case, you can't, because you need it to be either x, or -x, not any value in between.

What you can do, is using a bivalent variable. Those are also called logical variables.

With them, you can define another variable, like u, and restrain them like this.

Let's say V is a bivalent variable. And M a big number.

x - MV ≤ u ≤ x + MV

-x - M*(1-V) ≤ u ≤ -x + M*(1-V)

This way, when V is 0, the second constraint does nothing. And the first one forces u to be x.

When V is 1, the first constraint does nothing. And the second one forces u to be -x.

Then, when the problem is solved, u can only take one of those values.

Fab
  • 68
  • I still think that it represents a similar problem. I have added the example. Is this the one you were referring to? – Manish Feb 22 '17 at 16:46
  • 1
    On that example, U can take values that aren't right. So, it could not work as you want. The only way I know to restrain something to two values is using some bivalents variables. But maybe you're trying not to use them. If that's the case, it depends on the problem. If the function you're trying to maximize or minimize "pulls" the variable to either X or -X, then it will work. But if there's a better solution in between. Then it will not. Sometimes the problem itself won't allow that something in the middle is the best solution, so you can exploit that. But it won't work always. – Fab Feb 22 '17 at 16:59
  • Let's do a quick review. On that problem, if you say the best solution is x1 = 25/3 and x2 = -5/3, -25/3 < U1 <25/3 and -5/3 < U2 < 5/3. And those are both the only restraints on U. That means that optimal solution is going to be -25/3 - 3*5/3 = - 40/3. Which isn't 40/3. Maybe the values are fine anyways, maybe they are not. But it's not the same function. And we will have to do a little analysis to check if on that problem it's the same or not. – Fab Feb 22 '17 at 17:13
  • yes the interval should be as you mentioned, but shouldn't it be U1=25/3 and U2=5/3; which does give us 40/3 and not its negative – Manish Feb 22 '17 at 17:19
  • If you have any doubts of what the model does, you can always do a script on GLPK and see what it does. You can use any GLPK compiler, GUSEK on windows is pretty easy to use, and you can compile a model with a glpsol -o mySolution.sol. You'll have to add that line on GUSEK too. I'll write you a script with your model, so you can also see how it works by yourself, and try new things – Fab Feb 22 '17 at 17:52
  • 1
    var x1; var x2; var x3; var u1; var u2; var u3;

    s.t. cst0: x1+2*x2 >= 10; s.t. cst1: x1 - x2 = 5; s.t. cstu1a: -x1 <= u1; s.t. cstu1b: u1 <= x1; s.t. cstu2a: -x2 <= u2; s.t. cstu2b: u2 <= x2; s.t. cstu3a: -x3 <= u3; s.t. cstu3b: u3 <= x3;

    minimize z: u1 + 3*u3;

    end;

     – Fab Feb 22 '17 at 17:53
  • Thanks a lot for your prompt response. – Manish Feb 23 '17 at 04:31