Conditional probability and Bayes' theorem

Why "given" matters

The chance that a random email is spam might be 0.2. But if the email contains the words "you won a lottery", the chance is much higher. New information changes probabilities. Conditional probability is how we update a chance after we learn something.

The definition

P(A | B) is read "the probability of A given B". It means: look only at the cases where B happened, and ask how often A also happened. The formula is P(A | B) = P(A and B) ÷ P(B). With counts it is even simpler: (count of A and B) ÷ (count of B).

emails_with_lottery = 50          # B happened
spam_with_lottery = 45            # A and B happened
print(spam_with_lottery / emails_with_lottery)

The 2 × 2 table method

Imagine 10,000 people are tested for a disease. 1% have it (the base rate, also called the prior). The test catches 99% of sick people (its sensitivity) but also wrongly flags 5% of healthy people (the false positive rate). Fill a table with whole people instead of percentages.

Of the 594 people who test positive, only 99 are sick. So P(sick | positive) = 99 ÷ 594 ≈ 0.167, about 17%. The test is good, but the disease is rare, so most positives come from the huge healthy group. This surprise is called the base rate fallacy.

Bayes' theorem

Bayes' theorem writes the same table as a formula: P(A | B) = P(B | A) × P(A) ÷ P(B). Here A is "sick" and B is "positive". The bottom part, P(B), is every way to get a positive: sick and positive, plus healthy and positive.

def bayes(prior, sensitivity, false_positive_rate):
    true_pos = prior * sensitivity
    false_pos = (1 - prior) * false_positive_rate
    return true_pos / (true_pos + false_pos)

print(round(bayes(0.01, 0.99, 0.05), 4))

Where data scientists use it

• Spam filters: the Naive Bayes classifier combines P(word | spam) for many words.
• Medical and fraud alerts: deciding how many alerts are true and how many staff are needed to check them.
• A/B tests and Bayesian statistics: updating a belief about which version is better as data comes in.

Common mistakes

• Ignoring the base rate. Fix: always ask "how common is this to begin with?" and use the 2 × 2 table.
• Swapping P(A | B) and P(B | A). Fix: say the sentence aloud: "out of the people who tested positive, how many are sick?"
• Dividing by the wrong total. Fix: in P(A | B), the bottom is always the count of B.