2

Let $X$ be a $n\times K$ full-column rank matrix, with $n>K$. Let $P$ be a symmetric and idempotent matrix of dimension $n\times n$, hence positive semi-definite with eigenvalues equal to 0 or 1 (only $K$ of them are actually equal to 1 in my setting). Define $W$ as a diagonal positive semi-definite matrix of dimension $n\times n$, with elements between 0 and 1. All matrices are real.

What are the conditions for $X'PWX$ to be positive semi-definite? More specifically, I want to be sure that the eigenvalues are non-negative.

This is a quadratic form, $PW$ is positive semi-definite (in the sense that its eigenvalues are non-negative) from the properties of $P$ and $W$, but $PW$ is not necessarily symmetric. First, I am not sure what it the definition of a positive semi-definite matrix that is not symmetric. Also, I have considered using several decompositions but non-symmetry always end up being an issue. I have run simulations to check this condition, and the eigenvalues of $X'PWX$ are always positive, so I figured this should hold more generally. I would be happy to clarify further if needed.

dv_bn
  • 187
  • Isn't it possible to write $y^\top X^\top PW X y = \left(Xy\right)^\top PW \left(Xy\right) \geq 0$ for any $y$, where the inequality follows from the positive semidefiniteness of $PW$? – sven svenson Oct 07 '20 at 18:36
  • @svensvenson, I would like to agree but I am not sure what is a definition of PSD for a non-symmetric matrix. What is known about $PW$ is that it is a square non-symmetric matrix with non-negative eigenvalues. – dv_bn Oct 07 '20 at 18:49
  • 1
    If $x \mapsto x^t M x$ is a quadratic form, then $x \mapsto x^t K x$ , where $K = \frac12(M + M^t)$, is the same quadratic form. So you might as well "symmetrize" your matrix, and then you can apply your favorite definition of SPD to it. – John Hughes Oct 07 '20 at 22:20
  • @JohnHughes, your comment seems to be on point. Thanks! I would be happy to accept your answer if you could propose a full answer about why the eigenvalues of $X'PWX$ are the same as those of $X'\frac{1}{2}(PW+W'P)X$. I am not sure to see why the two quadratic forms are actually the same. – dv_bn Oct 07 '20 at 23:28
  • 1
    Look at $\pmatrix{x&y} \pmatrix{3 & 0 \ 4 & 2}\pmatrix{x\y} = 3x^2 + 4xy + 2 y^2$. The numbers in the right-hand expression come from the matrix entries. But the upper right matrix entry also contributes an $xy$ term, so this is exactly the same as $\pmatrix{x&y} \pmatrix{3 & 2 \ 2 & 2} \pmatrix{x\y} = 3x^2 + 4xy + 2 y^2$, which comes from "symmetrizing" the matrix (averaging it with its transpose). This is a fairly standard trick when you mess with quadratic forms, BTW -- worth tucking in your back pocket for later use. – John Hughes Oct 08 '20 at 02:37
  • This is very helpful, thanks @JohnHughes. It seems to imply that any quadratic form with a non-symmetric positive semi definite matrix is also positive semi definite. I wonder why all standard results (that I know, so not so many actually) are then stated only for symmetric matrices. In light of this result, the answer to this question: https://math.stackexchange.com/questions/516533/why-do-mathematicians-use-only-symmetric-matrices-when-they-want-positive-semi-d?noredirect=1&lq=1 is not general enough. – dv_bn Oct 08 '20 at 11:41

0 Answers0