Reference for $\ell_1$-$\ell_2$ regularization

Asked Oct 29 '21 at 03:47

Active Nov 02 '21 at 23:26

Viewed 85 times

Given matrices $A, W \in \mathbb{R}^{n \times n}$ and vector $y \in \mathbb{R}^n$, consider the following optimization problem

$$ \min_{x \in \mathbb{R}^n} \| W x \|_1 + \| Ax - y \|_2^2,$$

Is there any general optimization framework for this kind of problem? I have found that some papers only deal with the case where $W = I$ (identity matrix). Could anyone give me some references?

edited Oct 29 '21 at 07:07

Rodrigo de Azevedo

23,223

asked Oct 29 '21 at 03:47

stander Qiu

This is called $l_1$ or, in some circles, lasso regularization. – copper.hat Oct 29 '21 at 05:19
@copper.hat Yes, but from my limited understanding lasso treats $W$ as identity matrix. I want to find out references for general situation. – stander Qiu Oct 29 '21 at 06:06
Have you tried introducing optimization variables $t_1$ and $t_2$, and then minimizing $t_1 + t_2$ subject to $| W x |_1 \leq t_1$ and $| Ax - y |_2^2 \leq t_2$? – Rodrigo de Azevedo Oct 29 '21 at 07:13
@RodrigodeAzevedo Hi, thanks for your advice. Let me try to formulate this problem as standard form. But how is the $\ell_1$ norm solved? – stander Qiu Nov 01 '21 at 01:33
@standerQiu Take a look at this. – Rodrigo de Azevedo Nov 01 '21 at 03:36
Did you make any progress? – Rodrigo de Azevedo Nov 03 '21 at 06:54
@RodrigodeAzevedo Hi, sorry for late reply. Busy these days. I find that it is possible to approximate the $\ell_1$ norm as Huber norm ( $\ell_2$ norm close to the non-smooth regions) and then apply the ISTA algorithm for the problem. – stander Qiu Nov 05 '21 at 04:22

Reference for $\ell_1$-$\ell_2$ regularization

0 Answers0