Converting sum of infinity norm and L1 norm to linear programming

Question

So I'm trying to convert this minimization problem,

min $\parallel Ax-y \parallel_{\infty}$ + $\parallel x \parallel_{1}$ where $A$ is $m$ by $n$, $y$ is $m$ by $1$ and $x$ is $n$ by $1$.

into a linear program. My attempted solution is to rewrite it as follows,

$$\min\,\, t + \textbf{1}^T z : t \in R, z \in R^n $$ subject to $$z \geq x$$ $$-z \leq x$$ $$\textbf{1}t \geq Ax - y$$ $$-\textbf{1}t \leq Ax - y$$

However, I'm having doubts about this solution since it sort of ignores the relationship between $z$ and $t$ and my hunch is to introduce some sort of constraint to capture that relationship. It'd be great if someone could confirm my answer or if I'm wrong, at least point me in the right direction?

Answer 1

This is absolutely right. Well done! You're not ignoring the relationship between $z$ and $t$; it is naturally captured by the model. The fact that they are both present in the objective ensures that an increase in one must be accompanied by a decrease in the other. It is possible to prove that at the optimal point, it must be the case that $z_i=|x_i|$, $i=1,2,\dots,n$, and $t=\|Ax-b\|_\infty$.

Answer 2

I think its best to do the conversion stepwise. There are basically just three ingredients you need to reformulate a minimization problem involving the 1 or the sup norm into a linear programming problem. These are: \begin{align} \|v\|_1 &= \min_z (\mathbf{1}^T z) \quad \mathrm{s.t.}\; z\geq |v|\\ \|v\|_\infty &= \min_t t \quad \mathrm{s.t.}\; \mathbf{1}t\geq |v|\\ \min_{(u,v)} f(u,v) &= \min_u \min_v f(u,v) \end{align} for any vector $v$ and $u$, where $\leq$ and $|\cdot|$ are taken to be element-wise. All of these are trivial to prove (the form is always $a=\min_{b\in B} b$, so show $a\leq b\,\forall b\in B$ and $a\in B$).

Plugging that into your minimisation problem gives \begin{align} \min_x \|Ax-y\|_\infty + \|x\|_1 &= \min_x (\min_z (\mathbf{1}^T z) \; \mathrm{s.t.}\; z\geq |Ax-y|)+ (\min_t t \; \mathrm{s.t.}\; \mathbf{1}t\geq |x|) \\ &= \min_{x,z,t} (\mathbf{1}^T z) + t \; \mathrm{s.t.}\; z\geq |Ax-y| \wedge \mathbf{1}t\geq |x|) \end{align} Now you just replace the $|u|$-terms with inequalities with inequalities involving $u$ and $-u$ and you are done. From here it is also clear that you have to minimize over all three variables $x$, $z$ and $t$ and that $z$ and $t$ are coupled via $x$.