0

I have a family of probability distributions on the n-dim sphere $\mathbb S^n \subset \mathbb R^{n+1}$ defined in the following way:

$D_0$ is the uniform distribution, which is constructed by sampling $k$ points $z_i \in \mathbb R^{n+1}$ from a normal distribution of mean $m_0 = (0,0, \dots, 0)$ and covariance matrix $\Sigma = diag(1,1, \dots,1)$. In order to transform this Gaussian distribution into a uniform distribution on the sphere, we normalize every $z_i$. Our points will then be $x_i = \frac{z_i}{\| z_i \|}$.

The other distributions $D_t$ are constructed in the same way, the only difference is that they have mean $m_t = (t,0, \dots, 0)$. The covariance matrix remains constant.

How can I compute the entropy of these distributions?

I know that the entropy $E$ has a maximum under the uniform distribution, and intuitively I can see that $E(D_t) > E(D_s)$ if $t<s$, but I don't know how to show it formally.

Thank you very much!

Alfred
  • 167
  • 10
  • It's not clear for me how is $D_0$ constructed . Do we have $n$ (or $n+1$?) iid standard gaussian (say $z=(z_1,z_2 \cdots z_n)$ and we (square) normalize them? ($x_i= z_i/\sqrt{\sum z_i^2}$) such that $\sum x_i^2= 1$ ? – leonbloy Jul 29 '19 at 12:24
  • No, every $x \in \mathbb R^n$ is a single gaussian random variable sampled from a multivariate distribution. Then, it gets normalised the way you do it. – Alfred Jul 29 '19 at 14:59
  • Well... it looks exactly the same to me. – leonbloy Jul 29 '19 at 18:04
  • Ok we are maybe talking about the same thing. What I mean is that i sample $m$ points $z_i \in \mathbb R^{n+1}$ and then normalise each one of them computing $\frac{z_i}{| z_i |}$. This way, i obtain $m$ points that lay on the surface of the sphere $\mathbb S^{n}$. – Alfred Jul 29 '19 at 18:13
  • The differential entropy is not a very well behaved beast... Besides other things (like being sensitive to scale), the common definition does not (directly) apply to degenerate densities (spec: where the support has zero measure). See eg this answer https://math.stackexchange.com/a/2634605/312 – leonbloy Jul 29 '19 at 18:25
  • In your case, to avoid the differential entropy over the sphere surface from going to $-\infty$ I guess you could somewhat "normalize" it by substracting $\log(\epsilon)$ from the common defintion, where $\epsilon$ is the width of the shell (with $\epsilon \to 0$).... – leonbloy Jul 29 '19 at 18:28
  • Ok, thank you very much! Is there a way to estimate these entropies, somehow? Thanks! – Alfred Jul 30 '19 at 11:36

0 Answers0