4

I'm trying to understand why the quadratic equation can approximate the log likelihood ratio, and how it is derived: $$\mathrm{Log}(\mathrm{LR})=\frac{1}{2}\left(\frac{\mathrm{MLE}-\theta}{S}\right)^2$$ Is this approximated using Taylor's series or normal distribution equation or anything else?

Using Taylor's expansion, I get $ -\frac{1}{2}(MLE-\theta)^2 (\frac{1}{\theta^2}+\frac{1}{MLE^2})$, instead of $ -\frac{1}{2}(\frac{MLE-\theta}{S})^2$

This was brought up in book 'Essential Medical statistics' Chapter $28$, the main goal was to derive a supported range (similar to the $95\%$ CI) for the likelihood ratio.

It was mentioned in the book that the log of the likelihood ratio (LR) is used instead of the likelihood itself, because the $\log(\mathrm{LR})$ can be approximated by a quadratic equation (the one shown above), for easier calculation. It is also said that this equation is chosen so as to meet the curve of and to have the same curvature as the $\log(\mathrm{LR})$ at the MLE.

Ela
  • 43

1 Answers1

1

Let's say $\ell(\theta; x)$ is our proposed log-likelihood function model (with likelihood function $f(\theta;x)$) for a given vector of observations $x$.

Then the second order Taylor Series expansion of $\ell(\theta;x)$ about the MLE ($\theta^*$) is:

$$\ell(\theta; x)\lvert_{\theta^*} = \ell(\theta^*; x) + \ell^{(1)}(\theta^*; x)(\theta-\theta^*) + \frac{\ell^{(2)}(\theta^*; x)}{2}(\theta-\theta^*)^2$$

Now, we normally use the likelihood ratio $LR(\theta;x):=\frac{f(\theta;x)}{f(\theta^*;x)}$ so we can subtract $\ell(\theta^*;x)$ from the series to get:

$$\ell(\theta; x)\lvert_{\theta^*} = \ell^{(1)}(\theta^*; x)(\theta-\theta^*) + \frac{\ell^{(2)}(\theta^*; x)}{2}(\theta-\theta^*)^2$$

Finally, by the definition of the MLE, $\frac{d}{d\theta}f(\theta;x)\lvert_{\theta = \theta^*} = 0$. By monotonicity of $\ln$, this implies $\frac{d}{d\theta}\ell(\theta;x)\lvert_{\theta = \theta^*} = 0$. Therefore, our first derivative term drops out, leaving:

$$\ell(\theta; x)\lvert_{\theta^*} = \frac{\ell^{(2)}(\theta^*; x)}{2}(\theta-\theta^*)^2$$

Finally, under some mild regularity conditions (as defined by Cramer) we can show that $\ell^{(2)}(\theta^*; x)<0$ and so we get:

$$\ell(\theta; x)\lvert_{\theta^*} \approx -\frac{\ell^{(2)}(\theta^*; x)}{2}(\theta-\theta^*)^2$$

Finally, the negative second derivative of the log-likelihood has a special name, the observed Fisher Information $i(\theta^*)$, which asymptotically approaches $\frac{1}{S^2}$, where $S$ is the standard deviation (under the same regularity conditions linked to above).

Therefore, re-arranging our formula we get what is in your textbook:

$$\ell(\theta; x)\lvert_{\theta^*} \approx -\frac{1}{2}\left(\frac{\theta-\theta^*}{S}\right)^2$$