Is there a closed form solution for the following least squares problem: $$ \min_\mathbf{x}\|\mathbf{a+Bx}\|^2 ~~\text{s.t}~~\|\mathbf{x}\|^2 \leq \alpha^2$$ where $\mathbf{a} \in \mathbb{C^{M\times 1}}$ and $\mathbf{B} \in \mathbb{C^{M\times N}}$.
Asked
Active
Viewed 1,425 times
1 Answers
4
It depends.
Consider the unconstrained least squares problem
$$ \min_\mathbf{x}\|\mathbf{a+Bx}\|^2. $$
This problem has the solution $\mathbf{x}^\star = - (\mathbf{B}^H\mathbf{B})^{-1}\mathbf{B}^H\mathbf{a} $. It is clear that $\mathbf{x}^\star $ is also the solution to your problem if $\|\mathbf{x}^\star\|^2 \leq \alpha^2$. In this case, you have an analytical solution.
However, if $\|\mathbf{x}^\star\|^2 > \alpha^2$, you have to consider the constrained problem
$$ \min_\mathbf{x}\|\mathbf{a+Bx}\|^2 + \lambda \|\mathbf{x}\|^2, $$ where $\lambda >0 $ is the Lagrange-Multiplier. This problem has the solution
$$\mathbf{x}^\star = - (\mathbf{B}^H\mathbf{B} + \lambda \mathbf{I})^{-1}\mathbf{B}^H\mathbf{a}$$
For the proper Lagrange-Multiplier $\lambda$, $\|\mathbf{x}^\star\|^2 = \alpha^2$ holds true. However, there is no analytical way to determine $\lambda$ which can be found by using bi-section search.
Rodrigo de Azevedo
- 23,223
The Pheromone Kid
- 889
- 7
- 18
-
You are welcome! plz vote up :-) – The Pheromone Kid Jul 31 '14 at 09:07
-
I believe the unconstrained solution is $\mathbf{x}^\star = - (\mathbf{B}^H\mathbf{B})^{-1}\mathbf{B}^H\mathbf{a} $. It's missing the inverse, correct? – user4259 Aug 01 '14 at 03:55
-
correct, sorry for the mistake. I edit my post! – The Pheromone Kid Aug 01 '14 at 06:34
-
How do I find $\lambda$? What should be the starting value? how to iterate it? – Sagi Buchbinder Shadur Aug 22 '16 at 12:31
-
@SagiShadur, Have a look at my solution: https://math.stackexchange.com/a/2402360/33. – Royi Aug 27 '17 at 18:29