Log det of covariance and entropy

Question

I understand log of determinant of covariance matrix bounds entropy for gaussian distributed data. Is this the case for non gaussian data as well and if so, why?

What does Determinant of Covariance Matrix give? and http://web.ntpu.edu.tw/~phwang/teaching/2012s/IT/slides/chap08.pdf show connection between 'differential entropy' and log of determinant of covariance matrix for Gaussian case. Eqn. 26 of https://arxiv.org/pdf/1604.03924.pdf?fbclid=IwAR1tDOzgZ2iXSo3lDbXnr8TUkxawCA8NikHFlfY4E5OWmbmJ3_WHeVPotFE has some relations (I guess for non-Gaussian case, but yet to check that)

For context, please mention some of the bounds (names/statements) of the bounds that you have seen. This will help others attract your attention towards similar bounds for possibly non-Gaussian data. — Sarvesh Ravichandran Iyer, Feb 09 '19 at 05:39

leonbloy · Accepted Answer · 2019-03-28T22:09:38.630

2

I understand log of determinant of covariance matrix bounds entropy for gaussian distributed data

Wrong. That's not a bound, it's the value of entropy for a gaussian. And it's not exactly log of determinant of covariance matrix but

$$H(Z)=\frac{k}{2} \ln(2 \pi e)+\frac12 \ln (| \Sigma|)$$

Is this the case for non gaussian data as well and if so, why?

It's a well known result that the gaussian distribution maximizes the entropy, for a given covariance. See eg here.

Hence, yes, that value is a bound.

edited Mar 28 '19 at 22:09

answered Feb 09 '19 at 14:40

leonbloy

66,202

What does 'previous value' refer to in your answer? For the non gaussian case it being a bound , is what you meant ? – hearse Feb 09 '19 at 15:38
Yes, I meant the value of the equation above. – leonbloy Feb 09 '19 at 16:15
1

is there a similar bound or any bound for that sake on cross-entropy based on determinants? – hearse May 02 '19 at 16:09

Log det of covariance and entropy

1 Answers1