The normal distribution and z-scores

Why one curve shows up everywhere

Measure the heights of thousands of adults, the weights of 1-kg rice packets from a factory, or small errors in a scale, and draw how often each value occurs. You often get the same shape: a single hump in the middle that falls away evenly on both sides, like a bell. This is the normal distribution (also called the Gaussian or bell curve). It appears when a value is the result of many small, independent influences adding up.

Its features

• It is symmetric: the left half mirrors the right half.
• The mean, median and mode are all at the centre.
• Two numbers describe it completely: the mean (where the centre is) and the standard deviation (how wide the bell is).
• The tails go on forever but get very thin: values far from the mean are rare, not impossible.

The 68–95–99.7 rule

For normal data, about 68% of values lie within 1 SD of the mean, about 95% within 2 SDs, and about 99.7% within 3 SDs. Suppose adult heights in a group are normal with mean 165 cm and SD 7 cm. Then about 68% are between 158 and 172 cm, about 95% between 151 and 179 cm, and almost everyone (99.7%) between 144 and 186 cm.

from statistics import NormalDist

heights = NormalDist(mu=165, sigma=7)
print(round(heights.cdf(172) - heights.cdf(158), 4))   # 0.6827
print(round(heights.cdf(179) - heights.cdf(151), 4))   # 0.9545
print(round(heights.cdf(172), 4))                      # 0.8413

z-scores

A z-score (or standard score) measures how far a value is from the mean in units of standard deviation. In words: subtract the mean, then divide by the SD. The formula is z = (x − mean) / sd. A z of 0 is exactly average, +1 is one SD above, −2 is two SDs below. Because z has no units, you can compare things measured on completely different scales.

marks = [55, 60, 65, 70, 75, 80, 85]
mean = sum(marks) / len(marks)                                   # 70.0
sd = (sum((m - mean) ** 2 for m in marks) / len(marks)) ** 0.5   # 10.0
print([(m - mean) / sd for m in marks])
# [-1.5, -1.0, -0.5, 0.0, 0.5, 1.0, 1.5]

Comparing scores from different exams

Ravi scored 82 out of 100 in his board exam, where the mean was 70 and the SD was 8. In a mock entrance test he scored 180 out of 300, where the mean was 120 and the SD was 50. (Made-up numbers.) Which was the stronger performance? Raw marks cannot be compared, but z-scores can: board z = (82 − 70) / 8 = 1.5; mock z = (180 − 120) / 50 = 1.2. Relative to the other students, his board result was stronger.

from statistics import NormalDist

board = NormalDist(mu=70, sigma=8)
print(board.zscore(82))                       # 1.5
print(round(board.cdf(82), 4))                # 0.9332
print(round(NormalDist().inv_cdf(0.975), 2))  # 1.96

Flagging unusual values

Since about 95% of normal data has |z| ≤ 2 (|z| means the size of z, ignoring the sign), values with |z| > 2 are often flagged as unusual, and |z| > 3 as very unusual. This is a quick check, with two catches. In a big dataset about 5% of perfectly normal values have |z| > 2, so "unusual" is not "wrong". And a huge outlier inflates the SD itself, which can hide other outliers. For skewed data the 1.5 × IQR rule from the previous topic is safer.

Common mistakes

• Applying the 68–95–99.7 rule to skewed data. Fix: check the shape first.
• Forgetting that z can be negative. A z of −2 is as unusual as +2. Fix: compare abs(z).
• Dividing by zero when every value is the same (SD = 0). Fix: handle that case separately; all z-scores are then 0.
• Calling every |z| > 2 point an error. Fix: treat it as a flag to investigate.