3

The most statistics I ever took was a few lessons on it back in high school. What always bothered me is how arbitrary the definitions seemed. For instance, I remember having trouble with the definition of standard deviation.

The standard deviation of a set of values $X$ is the square root of the average of the squared of the differences between each value in $X$ and the average of $X$.

At school, standard deviation was never given any more precise definition than "a number that gives you a rough idea how 'diffuse' the dataset is". While I can see in a very approximate way that this is basically correct, it's a long way from how definitions of concepts in math are usually explained. Usually there's a precise notion that we're trying to capture, and a clear explanation as to how our definition captures that notion.

But here, when I asked for further information, I was told things like "you square the differences to make them positive", when what I was hoping for was something like:

  1. A specific real-world concept that the definition captures.
  2. A class of problems in which the definition arises naturally.
  3. A specific mathematical property that we would like to have, which leads necessarily to this particular definition.

Is there any rigorous basis for the definitions in statistics, or do we genuinely just make up formulae that kinda sorta get us something like what we're trying to calculate? Have I just never seen statistics as it really is, or is it actually very different to every other field of mathematics? If the former, can you recommend a book that explains statistics in the way I'd like?

Jack M
  • 28,518
  • 7
  • 71
  • 136
  • 1
    Anyone teaching statistics who says "you square the distances in order to make them positive" is either an imbecile or a person who assumes the student is an imbecile. I heard that same statement in an undergraduate psychology course. I concluded --- and I still think several decades later --- that that professor was an imbecile. He was perfectly capable of talking intelligently about other subjects, but on that topic he was profoundly stupid at best. – Michael Hardy Dec 06 '13 at 23:16
  • @EricAuld My question is broader, but that's certainly relevant. Thanks. – Jack M Dec 06 '13 at 23:18
  • @JackM Right, at least it's a jumping off point for a more specific question. – Eric Auld Dec 06 '13 at 23:20
  • @MichaelHardy It is a disturbing and sad truth that in a field such as psychology, which is entirely dependent on statistical rigor to maintain any shred of scientific validity, a professor cannot understand even the most basic principles of statistical theory. It is even sadder that SPSS is a tool so easy to use that most practitioners avoid developing such a background. It is saddest, perhaps, the frequency with which the analysis is fed to graduate students to plug and chug. – Emily Dec 07 '13 at 00:41

2 Answers2

1

The obvious alternative to the standard deviation as a measure of dispersion is the mean absolute deviation. In the 18th century Abraham de Moivre wrote a book called The Doctrine of Chances. ("The doctrine of chances" is 18th-century English for the theory of probability. De Moivre wrote in English because he had fled to England to escape the persecution of Protestants in France.) He considered this problem (about which he had written an article in Latin while still in France):

Toss a fair coin 1800 times. What is the probability that the number of heads you get is between (for example) 880 and 910?

If you toss a fair coin once, the set of equally probable outcomes is just $\{0,1\}$ and the standard deviation is $1/2$, and the variance is $1/4$. If you toss a coin $1800$ times, the variance of the total number of heads you get is $1800$ times that, i.e. $1800\cdot(1/4)$, so the standard deviation is the square root of that. You can't do anything like that with the mean absolute deviation. De Moivre showed that the distribution is given by the bell-shaped curve $y=c e^{-x^2/2}$ but re-located and rescaled to have a mean of $900$ and a standard deviation of $\sqrt{1800/4\,{}}$. Later his friend James Stirling showed that $c=1/\sqrt{2\pi}$, and de Moivre included that result in his book.

With root-mean-square deviations this sort of thing can be done; with mean absolute deviations is cannot.

The reason things like this are not made clear in the most elementary statistics courses is that those courses are for the many people who may have occasion to use statistical methods, not for the far fewer people who want to understand the theory behind them.

Michael Hardy
  • 1
  • Standard deviation was only an example, I'm really asking about statistics in general, as a discipline. How is it developed? Where do these formulae come from? – Jack M Dec 07 '13 at 01:06
0

The definition of the "standard deviation of a set of values" is not at all what you write. A definition would be like "Let $A$ be a set of cardinality $n$ containing as elements real numbers, denoted $x_1,...,x_n$, ordered or unordered. Define the magnitude "Sample mean", denoted $\bar x$, by $$\bar x = \frac 1n\sum_{i=1}^nx_i$$ Then the "standard deviation" of the elements of $A$, denoted $S(A)$, is defined as

$$S(A) = \left(\frac 1n\sum_{i=1}^n (x_i-\bar x)^2\right)^{1/2} $$

I don't see any imprecision or ambiguity here.

Perhaps you meant to discuss the intuition behind statistical concepts, or even, their "real-world" usefulness, as a tool or as a mapping, as it looks later on your post. In that case, the comparison with mathematics in general is rather unfair to mathematics -since their great power is that they don't really give a damn about them being directly "intuitively useful" or "related" to any "real-world situation" whatsoever -and probably that's why they are indirectly indispensable to almost all real-world matters.

Alecos Papadopoulos
  • 10,546