0

Consider the multi-variate normal distribution $N(0, 1)$. The probability density shows that as the norm of samples decreases, the probability increases. However, in high-dimensional settings, random samples tend to be concentrated more in a thin shell. Empirical evidence:

import torch

d = 4
distribution = torch.distributions.multivariate_normal.MultivariateNormal(torch.zeros(d), torch.eye(d))


distribution.sample().norm()
#=> 1.2142


distribution.sample().norm()
#=> 2.6995


distribution.sample().norm()
#=> 1.6052


d = 512
distribution = torch.distributions.multivariate_normal.MultivariateNormal(torch.zeros(d), torch.eye(d))


print(distribution.sample().norm())
#=> 22.2805


print(distribution.sample().norm())
#=> 23.4046


print(distribution.sample().norm())
#=> 22.5938

How do I model the probability that a sample is from the multi-variate distribution $N(0, 1)$ given its dimensionality $d$ and its norm $r$?

Similar question: Distribution of the squared norm

This question instead concerns the distribution of the (unsquared) norm.

Richie Bendall
  • 163
  • 11

1 Answers1

0

The distribution of the radial distance $r = \Vert x \Vert$ for a multivariate normal distribution $N(0_d, I_d)$ follows a chi-distribution with $d$ degrees of freedom. Thus, we can find the probability that a sample with norm $r$ was sampled from $N(0_d, I_d)$ with its probabiliy density function:

$$ \text{pdf}_d(r) = \frac{2^{-d/2}}{\Gamma(d/2)} \cdot r^{d-1} \cdot e^{-r^2/2} $$

where $\Gamma(x) = (x-1)!$ is the gamma function and the log pdf:

\begin{align} \log \text{pdf}_d(r) &= \log\left(\frac{2^{-d/2}}{\Gamma(d/2)} \cdot r^{d-1} \cdot e^{-r^2/2}\right) \\ &=\log\left(2^{-d/2}\right) - \log \Gamma\left(\frac{d}{2}\right) + \log\left(r^{d-1}\right) + \log\left(e^{-r^2/2}\right) \\ &= -\frac{d}{2} \log 2 - \log \Gamma\left(\frac{d}{2}\right) + (d-1)\log r -\frac{r^2}{2} \end{align}

Richie Bendall
  • 163
  • 11