Derivative of $\sqrt{x^TAx}$

Question

Let $x\in \mathbb{R}^n$ and $A\in\mathbb{R}^{n\times n}$ be a positive semi-definite matrix.

Is there a way to express in closed form the following derivative?

$$\frac{\partial}{\partial x} \sqrt{x^TAx}$$

Answer 1

First of all, using the chain rule, we have $$ \frac{\partial }{\partial x} \sqrt{x^TAx} = \frac{1}{2 \sqrt{x^TAx}}\cdot \frac{\partial }{\partial x} x^TAx. $$ One approach to this partial derivative is to write the expression $f(x + h)$ in the form $f(x) + g(x)^Th + o(h)$; by definition, the $g(x)$ for which this holds is equal to $\frac{\partial f}{\partial x}$. With that in mind, $$ (x+h)^TA(x + h) = x^TAx + x^TAh + h^TAx + h^TAh\\ = x^TAx + x^TAh + [h^TAx]^T + o(h)\\ = x^TAx + x^TAh + x^TA^Th + o(h)\\ = x^TAx + [(A + A^T)x]^Th + o(h). $$ With that, the derivative of $x \mapsto x^TAx$ is $(A + A^Tx)$, and we have $$ \frac{\partial }{\partial x} \sqrt{x^TAx} = \frac{1}{2 \sqrt{x^TAx}}\cdot (A + A^T)x. $$

Answer 2

$ \def\o{{\tt1}}\def\p{\partial} \def\L{\left}\def\R{\right}\def\LR#1{\L(#1\R)} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\grad#1#2{\frac{\p #1}{\p #2}} $The simplest approach is to square the function and then use implicit differentiation $$\eqalign{ f^2 &= x^TAx \\ 2f\;df &= \LR{dx^TAx+x^TA\,dx} = \LR{Ax+A^Tx}^Tdx \\ \grad{f}{x} &= \frac{\LR{A+A^T}x}{2f} = \frac{\LR{A+A^T}x}{2\sqrt{x^TAx}} \\ }$$

Answer 3

If $A$ is symmetric, you can rewrite it as $$A = B^T B$$ for some matrix $B$ (see more information on how to find $B$ here).

Hence we have $$x^TAx = x^TB^TBx = (Bx)^TBx = \|Bx\|^2$$

Taking the square root and differentiating should yield a result.
Where naturally $\|Bx\|^2$ is a scalar.

Hope that helps!