Universal meaning of 'normal'

Question

The word 'normal' is used in many different areas of math, with about 20 different uses identified by Wikipedia. Is there any overarching meaning explaining why the same word is used for, say, 'perpendicular', 'normal subgroup' and 'normal distribution'?

Answer 1

In parts, perhaps this question could also belong on History of Science and Mathematics Stack Exchange and/or English Language and Usage, but I think it can fit here as well.

General Meanings/Origin

"Normal" comes to English ultimately from Latin "nōrma", meaning "carpenter's square" which also had a meaning like "rule, pattern" even in Latin. See Wikitionary or the Online Etymology Dictionary, for instance.

This combination basically explains all related meanings you come across in English mathematics.

Perpendicular

Carlo Beenakker 's answer to How did "normal" come to mean "perpendicular"? on Math Overflow quotes a fifteenth century commentary:

"Angulus normalis est idem qui angulus rectus" = "a normal angle is the same as a right angle"

And Brian M. Scott's answer to Etymology of the word "normal" (perpendicular) quotes "OED Third Edition, an entry updated December 2003" (Here OED refers to the Oxford English Dictionary, not the aforementioned Online Etymology Dictionary.):

classical Latin normālis right-angled

Normal Subgroup

Martin Brandenburg's answer to Why are normal subgroups called "normal"? quotes Jeff Miller's Earliest Known Uses of Some of the Words of Mathematics page:

According to The Genesis of the Abstract Group Concept (1984) by Hans Wussing, "The German Normalteiler (normal subgroup) goes back to Weber [H., Lehrbuch der Algebra, vol. 1, Braunschweig, 1895. p.511] and is possibly linked to Dedekind's term Teiler (divisor), which was employed in ideal theory" [Dirk Schlimm].

If true, then "normal" for "normal subgroup" may have come through German, but Wikitionary says that the German word "normal" has essentially the same source and meanings as the English.

user810157 conjectured in their answer:

"normal" means "inducing some regularity/order" and hence "some structure": think of the group structure induced in the quotient when the subgroup is (indeed) "normal". Of course this is just my guess.

Normal Distribution

Wikipedia cites Probability Theory: The Logic of Science, Ch 7 by Edwin J. Jaynes, which I quote here:

The literature gives conflicting evidence about the origin of the term "Normal distribution"...However, the term had long been associated with the general topic: given a linear model $y=X\beta+e$ where the vector $y$ and the matrix $X$ are known, the vector of parameters $\beta$ and the noise vector $e$ unknown, Gauss (1823) called the system of equations $X'X\hat\beta=X'y$ which give the least squares parameter estimates $\hat\beta$, the "normal equations" and the ellipsoid of constant probability density was called the "normal surface." It appears that somehow the name got transferred from the equations to the sampling distribution that leads to those equations.

Presumably, Gauss meant "normal" in its mathematical sense of "perpendicular" expressing the geometric meaning of those equations. The minimum distance from a point (the estimate) to a plane (the constraint) is the length of the perpendicular.

If this is the ultimate reason why people started calling it the "normal distribution", then it is a coincidence that the central limit theorem makes this distribution occupy a central role which could seem related to the meanings "standard"/"usual" that the word "normal" can have. In fact, Jaynes says:

This leads many to think — consciously or subconsciously — that all other distributions are in some way abnormal.

Actually, it is quite the other way; it is the so-called 'normal' distribution that is abnormal in the sense that it has many unique properties not possessed by any other. Almost all of our experience in inference has been with this abnormal distribution, and much of the folklore that we must counter here was acquired as a result. For decades, workers in statistical inference have been misled, by that abnormal experience, into thinking that methods such as coence intervals, that happen to work satisfactorily with this distribution, should work as well with others.