0

In Gaussian Mixture Models, in order to derive the M-step for covariance matrix, I need this result. I have poor knowledge of matrix calculus and result does not exist in https://en.wikipedia.org/wiki/Matrix_calculus

What is the partial derivative of $x^T A x$ w.r.t $A$?

Rodrigo de Azevedo
  • 23,223
Tuna Özateş
  • 31
  • The section on scalar-by-matrix seems as if it would work here, because $x^TAx$ is a scalar. It might be tedious to write out all the components but it should work. – RobertTheTutor Apr 25 '21 at 23:09
  • 1
    You can just do it component wise in the worst case scenario. In this case, if you do it component wise you see that $\partial{x^T Ax}/\partial{A_{ij}} = x_i \cdot x_j$. So the ultimate answer would be $xx^T$ – Vercingetorix Apr 25 '21 at 23:10
  • Here I'm assuming $x$ is a column vector. And I'm using the "denominator convention" as written in the wikipedia article. Otherwise it would be the transpose of that in the other notation. – Vercingetorix Apr 25 '21 at 23:11
  • 1
    This function is linear in $A$, so it is its own derivative. – Jannik Pitt Apr 25 '21 at 23:36
  • Echoing @JannikPitt's answer. – copper.hat Apr 25 '21 at 23:47

1 Answers1

1

Note that $$x'Ax = \mathrm{tr}(xx'A) = xx':A,$$ where $:$ is a scalar product on the space of matrices. The derivative wrt $A$ is thus $xx'$.

lmaosome
  • 682