Probability rules and Bayes' theorem

Statistics · Senior school statistics · about 10 min

Probability lets us measure uncertainty and reason from incomplete information. The same event can have a different likelihood once new evidence is known. In exam questions, the main challenge is choosing the correct denominator and distinguishing an event from evidence about that event.

The key idea

Bayes' theorem reverses a conditional probability:

P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}

Here $A$ is the event of interest, $B$ is observed evidence, and $P(B)>0$ . The numerator is the joint probability of $A$ and $B$ .

Draw one rectangular sample space labelled S. Inside it place two partially overlapping circles labelled A on the left and B on the right. Shade the overlap with a denser fill than the non-overlapping parts of the circles. Place A∩B only in the overlap. Do not draw numbers, arrows, or additional events. — The overlap is subtracted once when calculating a union.

Addition and multiplication rules

For any events $A$ and $B$ , the addition rule is $P(A\cup B)=P(A)+P(B)-P(A\cap B)$ . Subtract the overlap so it is not counted twice. The multiplication rule is $P(A\cap B)=P(A)P(B\mid A)$ , provided $P(A)>0$ . If events are independent, this becomes $P(A)P(B)$ ; independence is an assumption to justify, not a default. Complements are often quicker: $P(A^c)=1-P(A)$ . When a question says at least one, calculate the probability of none and subtract it from one whenever that is simpler.

Conditional probability

A conditional probability restricts the sample space to cases where the condition occurred. Formally, $P(A\mid B)=P(A\cap B)/P(B)$ . Therefore $P(A\mid B)$ need not equal $P(B\mid A)$ . On a tree diagram, multiply probabilities along each path to obtain its joint probability, then add paths that lead to the same observed result. When a test returns a positive result, include both true positives and false positives in the denominator. The initial prevalence, sometimes called the prior probability, can make a large difference to the answer.

Using Bayes in an exam

Label the hypothesis and evidence before inserting numbers. First obtain the total probability of the evidence using $P(B)=P(B\mid A)P(A)+P(B\mid A^c)P(A^c)$ for a two-case partition. Next divide the probability of the path containing both $A$ and $B$ by that total. A frequency table with a convenient total such as $1\,000$ is an alternative to algebra and provides a useful check. Finally, interpret the posterior as a probability after observing the evidence, not the accuracy of the test in isolation.

Worked example: A positive screening result

In a large group, $10\%$ of items have a particular defect. A screening test is positive for $80\%$ of defective items and for $5\%$ of non-defective items. If a randomly selected item tests positive, find the probability that it is defective.

Draw a left-to-right probability tree with a starting dot at left. First split into an upper branch D labelled 0.10 and a lower branch not D labelled 0.90. Each of these branches splits into a positive upper path and a negative lower path. From D label the conditional positive branch 0.80 and the negative branch 0.20; from not D label positive 0.05 and negative 0.95. Align four endpoints vertically in this order: D and positive, D and negative, not D and positive, not D and negative. No joint probabilities or posterior answer is shown. — A two-stage probability tree separates true and false positives.

Show the solution

Let $D$ mean defective and $+$ mean a positive result. The data give $P(D)=0.10$ , $P(+\mid D)=0.80$ , and $P(+\mid D^c)=0.05$ .

Find the two positive-result paths:

P(D\cap +)=0.10(0.80)=0.080

P(D^c\cap +)=0.90(0.05)=0.045

Add these paths to obtain the probability of a positive result:

P(+)=0.080+0.045=0.125

Apply Bayes' theorem:

P(D\mid +)=\frac{0.080}{0.125}=0.64

Answer: $64\%$ of positive items are defective. As a check, among $1\,000$ items there would be $80$ true positives and $45$ false positives, giving $80/125=0.64$ .

Common mistakes

Reversing the condition

Do not replace $P(D\mid +)$ with $P(+\mid D)$ . Write the desired probability first.

Ignoring false positives

Calculate the probability of every route to a positive result before forming the denominator.

Assuming independence

Only use the product of unconditional probabilities when independence is stated or justified.

Adding overlapping events without subtracting overlap

Use $P(A\cup B)=P(A)+P(B)-P(A\cap B)$ .

Check your understanding

1. Suppose $P(A)=0.30$ , $P(B\mid A)=0.60$ and $P(B\mid A^c)=0.20$ . What is $P(A\mid B)$ ?

2. Independent events satisfy $P(A)=0.40$ and $P(B)=0.30$ . Find $P(A\cup B)$ .

3. If $P(A)=0.15$ , what is $P(A^c)$ ?

Still not sure? Talk it through.

Waypoint Mentor's tutor explains this out loud, works through your own problems on a whiteboard, and checks you've got it. Free to start.

Explain this to me