3

Context: Consider the problem of identifying the error probability associated with a given classifier. Assume that the points produced by the two classes, namely A and B, are characterized by bivariate Gaussians. In this context, the classifier may be characterized by a line and the probability of error is related to the probability that a point is beyond that line.

Question: Given a line $y=ax+b$, what is the probability that a bivariate Gaussian generates a point beyond that line?

Case 1) In the simplest case, we assume that the bivariate Gaussian is normalized, with mean (0, 0), correlation 0, and variances 1. We also assume $b=0$. Intuitively, I think in this case the answer is that the sampled point will be in each side of the line with probability 1/2

Case 2a) In this case, we assume that the bivariate Gaussian is normalized, with mean (0, 0), correlation 0, and variances 1. We assume $b \neq 0$

Case 2b) In this case, we assume that the bivariate Gaussian is general, with mean $\mu$ and covariance matrix $\Sigma$. We assume $b = 0$

Case 3) Now, we assume that the bivariate Gaussian is general with mean $\mu$ and covariance matrix $\Sigma$. We assume $b \neq 0$

Daniel S.
  • 863

1 Answers1

3

You have to work all the cases in one shot as follows. First you consider $(X,Y)=N(\mu_1,\mu_2, \Sigma)$ where $$\Sigma=\left[\begin{array}{cc}\sigma_1^2&\rho \sigma_1\sigma_2\\\rho \sigma_1\sigma_2&\sigma_2^2\end{array}\right].$$ Next you write$$X=\mu_1+\sigma_1X_1,\ Y=\mu_2+\sigma_2Y_1$$ where $X_1, Y_1 \sim N(0,1)$ with correlation coefficient $\rho.$ Second step you write $$Y_1=\rho X_1+\sqrt{1-\rho^2}Z$$ where $Z\sim N(0,1)$ is independent of $X_1.$ Third step: you rewrite the event $Y>aX+b$ under the form $$X_1\geq \alpha Z+\beta.$$ Fourth step you consider $$Z_1=\frac{X_1-\alpha Z}{\sqrt{1+\alpha^2}}, z_1=\frac{\beta}{\sqrt{1+\alpha^2}}$$ and you observe that $Z_1\sim N(0,1)$ and the probability of the event $Y>aX+b$ is $\Pr(Z_1>z_1).$

Gérard Letac
  • 4,542
  • $$X_1\geq \alpha Z+\beta.$$

    Indeed,

    $$Y > a X + b$$

    $$\mu_2 + \sigma_2 \left( \rho X_1 + \sqrt{1-\rho^2} Z \right) > a (\mu_1 + \sigma_1 X_1) + b$$

    $$\sigma_2 \rho X_1 > a (\mu_1 + \sigma_1 X_1) + b - \mu_2 - \sigma_2\sqrt{1-\rho^2} Z $$

    $$(\sigma_2 \rho - a \sigma_1) X_1 > a \mu_1 + b - \mu_2 - \sigma_2\sqrt{1-\rho^2} Z $$

    Therefore, $$\beta=\frac{a \mu_1 + b - \mu_2 }{\sigma_2 \rho - a \sigma_1}$$

    $$\alpha=\frac{- \sigma_2\sqrt{1-\rho^2}}{\sigma_2 \rho - a \sigma_1} $$

     – Daniel S. Jan 14 '24 at 19:44
  • I guess that the result above holds if $\sigma_2 \rho - a \sigma_1 > 0$. I think that if that is not the case, than we should replace $P(Z_1 > z_1)$ by $P(Z_1 < z_1)$. Are there any other conditions that must be checked for the above result to hold?

    @Gérard Letac

     – Daniel S. Jan 15 '24 at 15:15
  • 1
    Daniel you are right of course. Thanks for having computed all these constants. – Gérard Letac Jan 15 '24 at 15:21
  • follow up to generalize to multiple dimensions here https://math.stackexchange.com/questions/4848267/what-is-the-probability-that-a-multivariate-gaussian-generates-a-point-above-a-c – Daniel S. Jan 20 '24 at 16:16