0

I am trying to work out the polynomial expression in multivariate normal distribution. The inner product inside the epsilon for univariate (assuming 0 mean and unit variances/covariances for simplicity) is as follows:

$$p(x)\approx \exp\ \left(-\frac{1}{2}(x^2)\right)$$ so basically the concerned polynomial is $x^2$ For bivariate case the concerned polynomial is: $$\left(x^2\ -2xy\ + y^2\right)$$ This is the expanded form of $(x-y)^2$. Now for trivariate case, the concerned polynomial is: $$\left(x^2\ + y^2+z^2 -2xy\ -2xz -2yz\right)$$

This is now bit confusion as what is the factored form for this? Also in general what is the rule if there are more variables? There is a matrix form for it which requres inverse of the matrix but I am looking for polynomial case for intuitive understanding.

GENIVI-LEARNER
  • 543
  • 1
    Why do you expect a factorization? Could you please explain what you mean and intend in the bivariate case? Starting from, what are $x,y$? – Lutz Lehmann Feb 06 '20 at 11:37
  • $x,y$ are the two random variables under examination for bivariate normal distribution. We are looking to see how one changes when other changes. I am choosing unit variance and covariance to understand how underlying polynomial equation enforces this correlation. for three random variables we have $x,y,z$ and this is an example of trivariate normal distribution case. – GENIVI-LEARNER Feb 06 '20 at 11:42
  • Factorization will help me understand how the expanded form is being generated. Also loosely speaking what do we have to do to random variables to encode a correlation between them in plain algebra (not matrices). (it appears that we simply square their arthematic subtraction)! it works for univariate and bivariate case but not trivariate case – GENIVI-LEARNER Feb 06 '20 at 11:45
  • 1
    What is "inside the epsilon", do you mean the exponential? You know that the general form is $\exp(-\frac12 \vec x^T\Sigma^{-1}\vec x)$? Usually $\Sigma$ has full rank, which is necessary to have an inverse, which directly prohibits any factorization, as the existence of a factorization would mean rank $1$. – Lutz Lehmann Feb 06 '20 at 11:50
  • yes, i meant exponential. So translating this notion to algebra, does it mean that if the rank of the matrix is full, all the terms in the covariace matrix is 1, by the very definitation of the formula $\exp(-\frac12 \vec x^T\Sigma^{-1}\vec x)$ cant we factorize the terms as $(x+y+z)^2$ if the $\Sigma$ matrix is all 1, implying all the random variables are correlated to each other? – GENIVI-LEARNER Feb 06 '20 at 11:59
  • 1
    If $Σ$ is all $1$ it has rank 1 and is not invertible. Essentially, all 3 variables are identical, so that $(x+y+z)^2=9x^2$. – Lutz Lehmann Feb 06 '20 at 12:29
  • yes makes so much sense. the thing that confuses me is that despite enforcing full rank, I am able to get x,y,z samples from multivariate function in python: X = [0,1,2] samples = np.random.multivariate_normal([0,0,0], [[1,1,1],[1,1,1],[1,1,1]]) plt.scatter(X, samples) plt.show() – GENIVI-LEARNER Feb 06 '20 at 14:13
  • and yes as you mentioned I am getting same values for all x, y, z but it is computing the inverse in the formula, the test code above has 0 mean and unit variance and covariance. Thats why I am confused even more is how it is inverting rank 1 matrix – GENIVI-LEARNER Feb 06 '20 at 14:21
  • This was the whole point of my question to understand it in polynomial domain, because covariance matrix with all 1's in the entry works! – GENIVI-LEARNER Feb 06 '20 at 14:28
  • 1
    The generation of the random samples with that covariance matrix does not require the inverse. The distribution function however will be singular, having infinite weight along the diagonal and zero weight everywhere else. – Lutz Lehmann Feb 06 '20 at 14:30
  • I see, i want to understand what you said more. Can you point me to the right direction, I really cant visualize how does it puts infinite weight on the diagonal and zero everywhere. The generation of random samples require Cholesky decomposition as here but matrix [[1,1,1],[1,1,1],[1,1,1]] is not positive definite. – GENIVI-LEARNER Feb 06 '20 at 14:53
  • @LutzLehmann, to understand this intuitively, I added this question. I would really like to see your take on this – GENIVI-LEARNER Feb 11 '20 at 12:32

0 Answers0