Frechet derivative of shift operator in $l_2$?

Question

Let $x \in l_2$ and $J(x) = \sum_{k = 1}^{+\infty} x_k x_{k + 1}$. Find $DJ(u)$ and $D(DJ(u))$.

Attempted solution

Since $x \in l_2$, then $\sum_{k = 1}^{+\infty}x_k < \infty$. Another fact: $\|x\|_{l_2} = (\sum_{k=1}^{+\infty}x_k^{2})^{\frac{1}{2}}$.

By the definition of Frechet derivative: $$ \lim_{h\to0} \frac{|J(u + h) - J(u) - (DJ(u))(h)|}{\|h\|_{l_2}} = \lim_{h\to0} \frac{|\sum_{k = 1}^{+\infty}(u_k + h_k)(u_{k + 1} + h_{k + 1}) - \sum_{k = 1}^{+\infty}u_ku_{k+1} - (DJ(u))(h)|}{\|h\|_{l_2}} = \lim_{h\to0} \frac{|\sum_{k = 1}^{+\infty}u_kh_{k+1} + \sum_{k = 1}^{+\infty}u_{k + 1}h_k + \sum_{k = 1}^{+\infty}h_kh_{k + 1} - (DJ(u))(h)|}{\|h\|_{l_2}} $$

Now I am quite tempted to say that the following is the Frechet derivative

$$(DJ(u))(h) = \sum_{k = 1}^{+\infty}u_kh_{k+1} + \sum_{k = 1}^{+\infty}u_{k + 1}h_k$$

and then provide an extremely akward proof that

$$\lim_{h\to0} \frac{|\sum_{k = 1}^{+\infty}h_kh_{k+ 1}|}{\|h\|_{l_2}} = 0$$

Another way to go

In my seminar notes, it's said that $J(x) = \left<Ax, x\right>$ where $A$ is the right-shift opertor: $Ax = [0, x]$. Then there is magic: ordinary rules of matrix calculus are applied and $DJ(u) = (A + A^{*})u$. The second-order Frechet derivative is left as homework.

Questions

1. Can the first solution be somehow linked with the second?
2. Am I right that $D(DJ(u)) = DJ(u)$? Here is my proof (this has been shown incorrect but yet unclear to me why): $$ \lim_{h\to0} \frac{|(DJ(u))(s + h) - (DJ(u))(s) - D(DJ(u)(s))(h)|}{\|h\|_{l_2}} = \lim_{h\to0} \frac{|\left<(A + A^*)u, s + h\right> - \left<(A + A^*)u, s\right> - D(DJ(u)(s))(h)|}{\|h\|_{l_2}} = \lim_{h\to0} \frac{|\left<(A + A^{*})u, h\right> - D(DJ(u)(s))(h)|}{\|h\|_{l_2}} $$
3. Why is it suddenly possible to apply matrix calculus to an operator which isn't working with finite spaces? I mean, $A$ is not just a matrix, right?

Answer 1

If you indeed find extremely awkward proofs tempting, then go for it... but a proof of $$\lim_{h\to0} \frac{|\sum_{k = 1}^{+\infty}h_kh_{k+ 1}|}{\|h\|_{l_2}} = 0$$ need not be awkward. Since $2h_{k}h_{k+1}\le h_k^2+h_{k+1}^2 $, the numerator is at most $\|h\|_{l_2}^2$.

Can the first solution be somehow linked with the second?

In the second approach, instead of manipulating with a particular quadratic form, we work with a general one. It's like solving $x^2+5x+6$ with factorization trick versus deriving a formula for solution of $ax^2+bx+c=0$ and then just using it.

Why is it suddenly possible to apply matrix calculus to an operator which isn't working with finite spaces?

Because it's been proved to work. And if it wasn't, it should have been done. When a professor skips a proof, it's not as much magic as laziness or lack of time in lecture.

The proof isn't difficult, though: $$\left< A(x+h),x+h \right>-\left<Ax,x\right> = \left<Ax,h\right>+\left<Ah,x\right>+\left<Ah,h\right>$$ The second term on the right is $\left<A^*x,h\right>$, and the last one is bounded by $\|A\| \|h\|^2$.

Am I right that $D(DJ(u)) = DJ(u)$

No, the second derivative is never equal to the first derivative. They have different domains. The second derivative at $x$ takes two vectors as arguments, the first derivative takes only one. Correct formula: $$D^2J(u)(h,k) = \left<(A+A^*)h,k\right>$$ This is using the idea of second derivative as a bilinear form $Q$ on vectors $h,k$ such that $$J(u+h+k) - J(u) - DJ(u)(h+k) = Q(h,k)+o(\|h\|^2+\|k\|^2) $$ In your case the computation is simple because the $o$ term is actually zero.

Above, "bilinear form" is a linear map from $X$ to the space of linear functionals. So, the second derivative is a map from $X$ to $L(X\to L(X,\mathbb R))$, which happens to be constant in this case (there is no $u$ in the second derivative).