In this answer Derivation of soft thresholding operator how can I derive that $\nabla(||x-b||_2^2)=b-x$?
Asked
Active
Viewed 2,597 times
2 Answers
2
$\nabla(||x-b||_2^2)=b-x$
Proof:
$\nabla(||x-b||_2^2) = \nabla(||b-x||_2^2) = \nabla[(x-b)^T(x-b)] =
\nabla[(x^T-b^T)(x-b)] = \nabla[x^Tx-x^Tb-b^Tx-b^Tb] = 2x-b-b=2(x-b)$
- I think there should be $2$ there.
- I think $x$ should be positive since whatever you expand it, $x^x$ must be positive.
You can see the following vedio: (a similar derivation)
https://www.youtube.com/watch?v=jbdyR7ESOh4&list=PL7y-1rk2cCsDOv91McLOnV4kExFfTB7dU&index=6
(roughly around 40:00)
sleeve chen
- 8,576
2
The following fact will be helpful in your problem.
$$\frac{\partial}{\partial {\bf x}}||{\bf x}||_2^2 = \frac{\partial}{\partial {\bf x}}||{\bf x}^T{\bf x}||_2 = 2{\bf x}$$
To obtain the above result (namely the last equality), write the definition of the $2$-norm of ${\bf x}^T{\bf x}$ and differentiate wrt to each $x_i$. Now, just replace ${\bf x}$ with ${\bf x}-{\bf a}$ and carefully step through the process again to obtain $\frac{\partial}{\partial {\bf x}}||{\bf x}-{\bf a}||_2^2$.
Erik M
- 2,441
-
but it says that the solution is $a-x$ not $x-a$ – volperossa Mar 26 '14 at 23:13
-
I don't see anywhere in the link you provided that states that '$\nabla||x-b||_2^2 = b-x$' (which I don't believe to be true). All that is stated is equivalence of optimality conditions. – Erik M Mar 26 '14 at 23:31
-
here: $0 \in \nabla(|x-z|^2_2) + \partial(\lambda|z|_1) \Leftrightarrow 0 \in z-x + \lambda\partial|z|_1$ – volperossa Mar 26 '14 at 23:39
-
$(0 \in \nabla(|x-z|^2_2) + \partial(\lambda|z|_1) \Leftrightarrow 0 \in z-x + \lambda\partial|z|_1) \not\Rightarrow \nabla(|x-z|^2_2) = z-x$ – Erik M Mar 26 '14 at 23:41
-
:o so...why z-x?! :o – volperossa Mar 26 '14 at 23:46
-
$\frac{\partial}{\partial {\bf x}}||{\bf x}-{\bf a}||_2^2 = 2({\bf x} - {\bf a})$ – Erik M Mar 26 '14 at 23:55
-
It may be difficult to see the text, but that $\not\Rightarrow$ is does not imply – Erik M Mar 27 '14 at 00:12