0

I am trying to calculate the partial derivative of $x_i$ from the following:

$$\frac{\partial}{\partial x} (\mathbb{x}^T\mathbb{A}\mathbb{x})$$

For some vector $\mathbb{x}$ and matrix $\mathbb{A}$.

I can show this a sum of sums such that:

$$\sum_{k=1}\sum_{j=1}x_k a_{kj} x_j$$

From this I can show the cases for which $k$ and $j$ respectively it not equal to $i$, and then add the remaining parts to the sum. This will yield:

$$\sum_{k\neq 1} (x_k a_{ki}) + \sum_{j\neq 1} (a_{ij} x_i) + 2a_{ii} x_i$$

However I cannot seem to break what I need to do to finish finding the partial derivative with respect to $x_i$. Any suggestions/help would be appreciated.

Rodrigo de Azevedo
  • 23,223
NewDev90
  • 161
  • It would be instructive to try the case when $A$ is a $2\times 2$ matrix. –  Sep 12 '19 at 14:30

1 Answers1

2

I will use Einstein notation here: every repeating index in a product implies summation over it.

The trick here is that we should treat $x_j(x_i)$ and $x_k(x_i)$ as general functions.

$$ \frac{\partial}{\partial x_i}\left(a_{jk}x_jx_k\right)=a_{jk}\frac{\partial x_j}{\partial x_i}x_k+a_{jk}x_j\frac{\partial x_k}{\partial x_i} $$

Here we used the fact that $(uv)'=u'v+uv'$. Now we need to simplify the partial derivatives. What is important here is that $x_j$ doesn't depend on $x_i$ unless $j=i$. Or in other words: $\partial x_j/\partial x_i=\delta_{ij}$, where $\delta_{ij}$ is a Kronecker delta function (a unit matrix): $$ a_{jk}\frac{\partial x_j}{\partial x_i}x_k+a_{jk}x_j\frac{\partial x_k}{\partial x_i}=a_{jk}\delta_{ij}x_k+a_{jk}x_j\delta_{ik}=a_{ik}x_k+a_{ji}x_j $$

Finally, we can notice that in the second term we can replace index $j$ for whatever we want except $i$ (since it is just a name of summation index): $$ a_{ik}x_k+a_{ji}x_j = a_{ik}x_k+a_{ki}x_k = (a_{ik}+a_{ki})x_k $$

We can rewrite it in matrix form as this: $$ \frac{\partial}{\partial x}(x^TAx)=(A+A^T)x $$

Vasily Mitch
  • 10,414
  • Thanks a lot for the explanation. I have not yet come upon the Kronecker delta thus far - but this shows how I would engage the problem. – NewDev90 Sep 12 '19 at 18:54