Mathematics 9709 · Probability & Statistics 2

The hypothesis-test machine

A hypothesis test asks one question: is my result surprising enough to doubt the claim? Assume the claim is true, work out the probability of a result this extreme or worse, and if that's below your agreed line, it's too unlikely to be luck — so you reject the claim.

The 6-step machine (binomial)

The whole chapter rides on one idea you can see. Take a die you suspect is loaded towards six and roll it 16 times. One six proves nothing; sixteen sixes would be damning. Somewhere between is a cut-off — the smallest count of sixes too unlikely to happen by chance on a fair die. That cut-off is the entire machine.

Assuming the die is fair, $X\sim B(16,\tfrac16)$ . Add up the tail: $P(X\ge5)=11.3\%$ but $P(X\ge6)=3.78\%$ . If we draw the line at 5%, then 6 sixes is the first result that drops below it — that is where “unlucky” becomes “suspicious”.

Roll a die 16 times: how many sixes would convince you it's loaded? Anything in the gold region (≥6) would happen under 5% of the time by luck — that's the rejection region.

The significance level is the line you draw before looking; the critical value is the first outcome whose tail falls below it.

Define: “Let $X=$ number of … , $X\sim B(n,p)$ .”
Hypotheses about the parameter $p$ (never the data): $H_0{:}\ p=p_0$ , $H_1{:}\ p<p_0$ or $p>p_0$ .
State the significance level (usually 5%).
Test statistic = the tail probability assuming $H_0$ : $P(X\le x)$ for a suspected decrease, $P(X\ge x)$ for an increase.
Compare with the significance level.
Conclude in context: tail < level → reject; tail > level → accept. Never say “proved.”

Worked example

The machine end-to-end (drivers crashing)

Drivers crash 40% of the time on a simulator; after training, 4 of 20 crash. Test at 5% for a reduction.

Let $X=$ number who crash, $X\sim B(20,0.4)$ .
$H_0{:}\ p=0.4$ , $H_1{:}\ p<0.4$ (training aims to reduce).
Significance level 5%.
Suspected decrease → lower tail: $P(X\le4)=0.0510=5.10\%$ .
$5.10\%>5\%$ .
Do not reject $H_0$ .

Answer

5.10\%>5\%\ \Rightarrow\ \text{do not reject }H_0

There is insufficient evidence that the crash rate has fallen.

With 3 crashes instead, P(X≤3) = 1.60% < 5% → reject. One fewer crash flips the verdict — that gap is exactly where the critical value sits.

Worked example

A cleaner increase, at a 4% level (spinner)

A regular pentagonal spinner is spun 10 times and lands on red 5 times. Test at 4% whether it is biased towards red.

Let $X=$ spins landing red, $X\sim B(10,0.2)$ (one face in five).
$H_0{:}\ p=0.2$ , $H_1{:}\ p>0.2$ (suspected increase).
Increase → upper tail: $P(X\ge5)=1-P(X\le4)=0.0328=3.28\%$ .
$3.28\%<4\%$ → reject $H_0$ .

Answer

3.28\%<4\%\ \Rightarrow\ \text{reject }H_0

There is evidence the spinner is biased towards red.

Note the upper-tail shortcut $P(X\ge5)=1-P(X\le4)$ , and that the level need not be 5%. Had it landed red only 4 times, P(X≥4) = 12.1% > 4% → accept.

Ten spins, B(10, 0.2): the gold region (≥5) holds under 4% of the probability. The observed 5 reds lands inside it, so the test rejects 'fair'.

The test statistic is a TAIL (a region), not a single P(X = x). Comparing P(X = x) with the level is the single most common way to lose every mark on the question.

Worked example

A real exam trap: tail, not point

A researcher believes boys are equally likely to pick a green or orange cube; she offers both to 10 boys and 8 pick green. She tests $H_0{:}\ p=0.5$ vs $H_1{:}\ p>0.5$ at 5%.

A student writes $P(X=8)=0.0439<0.05$ and rejects $H_0$ . This scores zero method: the test statistic must be the tail $P(X\ge8)$ , not the point.

$P(X\ge8)=0.0439+\tbinom{10}{9}(0.5)^{10}+(0.5)^{10}=0.0547$ . Since $0.0547>0.05$ , do not reject $H_0$ .

Answer

P(X\ge8)=0.0547>5\%\ \Rightarrow\ \text{do not reject }H_0

So there is insufficient evidence that boys prefer green.

(9709/62/O/N/23 Q3) — the examiner explicitly awards the “invalid” mark for spotting that a point probability is the wrong statistic.

Why a tail, never a single bar

Any single outcome is rare when $n$ is large — even the most likely count has a small $P(X=x)$ . What makes a result evidence is that it sits in a band of equally-or-more-extreme outcomes that together are unlikely. That band is the tail, so the test statistic is always cumulative: $P(X\le x)$ or $P(X\ge x)$ .

Finding the rejection region (and the critical value)

Instead of testing one observation, a question may ask for the whole rejection region: every outcome that would lead to rejection. Walk the tail inward until it first exceeds the level — the last outcome still inside is the critical value.

Worked example

Build the rejection region, then reverse it

10% of people are left-handed; among 50 players you suspect fewer are. $X\sim B(50,0.1)$ , $H_1{:}\ p<0.1$ , level 5%.

$P(X\le1)=3.38\%<5\%$ but $P(X\le2)=11.2\%>5\%$ , so the region stops at 1.

Answer

\text{Reject if }X\le1\ (\text{critical value }1).

Reverse version: “if the critical value is 2, what is the least integer significance level?” Since P(X≤2) = 11.2%, you need at least a 12% level for 2 to be the cut-off.

Drag the significance level below and watch the rejection region grow — a lower level means a smaller region and a more demanding test.

α5%

H₁: p < 0.4 (lower tail)·critical region X ≤ 3, P(X ≤ 3) = 1.60%

Wording	H₁	Test statistic
decrease / fewer / lower	$p<p_0$	$P(X\le x)$
increase / more / higher	$p>p_0$	$P(X\ge x)$
changed / different / biased	$p\ne p_0$	relevant tail vs α/2

Examiner note

Compare a tail (

P(X\ge x)

), never the point

P(X=x)

; for an increase use

P(X\ge x)=1-P(X\le x-1)

; and conclude non-definitely in context (“insufficient evidence that …”), never “proved”. (9709/62/O/N/23 Q3) (9709/62/O/N/24 Q3)

Common mistake

Stating

H_0

about the data (“4 drivers crash”) instead of the parameter

p

; comparing a single

P(X=x)

instead of the tail; the off-by-one in

P(X\ge x)=1-P(X\le x-1)

; and a definite conclusion (“the die is biased”) instead of an evidence statement.

NextOne/two-tailed & the normal test

The hypothesis-test machine

The 6-step machine (binomial)

Roll a die 16 times: how many sixes would convince you it's loaded? Anything in the gold region (≥6) would happen under 5% of the time by luck — that's the rejection region.

The significance level is the line you draw before looking; the critical value is the first outcome whose tail falls below it.

Define: “Let $X=$ number of … , $X\sim B(n,p)$ .”
Hypotheses about the parameter $p$ (never the data): $H_0{:}\ p=p_0$ , $H_1{:}\ p<p_0$ or $p>p_0$ .
State the significance level (usually 5%).
Test statistic = the tail probability assuming $H_0$ : $P(X\le x)$ for a suspected decrease, $P(X\ge x)$ for an increase.
Compare with the significance level.
Conclude in context: tail < level → reject; tail > level → accept. Never say “proved.”

Worked example

The machine end-to-end (drivers crashing)

Drivers crash 40% of the time on a simulator; after training, 4 of 20 crash. Test at 5% for a reduction.

Let $X=$ number who crash, $X\sim B(20,0.4)$ .
$H_0{:}\ p=0.4$ , $H_1{:}\ p<0.4$ (training aims to reduce).
Significance level 5%.
Suspected decrease → lower tail: $P(X\le4)=0.0510=5.10\%$ .
$5.10\%>5\%$ .
Do not reject $H_0$ .

Answer

5.10\%>5\%\ \Rightarrow\ \text{do not reject }H_0

There is insufficient evidence that the crash rate has fallen.

With 3 crashes instead, P(X≤3) = 1.60% < 5% → reject. One fewer crash flips the verdict — that gap is exactly where the critical value sits.

Worked example

A cleaner increase, at a 4% level (spinner)

A regular pentagonal spinner is spun 10 times and lands on red 5 times. Test at 4% whether it is biased towards red.

Let $X=$ spins landing red, $X\sim B(10,0.2)$ (one face in five).
$H_0{:}\ p=0.2$ , $H_1{:}\ p>0.2$ (suspected increase).
Increase → upper tail: $P(X\ge5)=1-P(X\le4)=0.0328=3.28\%$ .
$3.28\%<4\%$ → reject $H_0$ .

Answer

3.28\%<4\%\ \Rightarrow\ \text{reject }H_0

There is evidence the spinner is biased towards red.

Note the upper-tail shortcut $P(X\ge5)=1-P(X\le4)$ , and that the level need not be 5%. Had it landed red only 4 times, P(X≥4) = 12.1% > 4% → accept.

Ten spins, B(10, 0.2): the gold region (≥5) holds under 4% of the probability. The observed 5 reds lands inside it, so the test rejects 'fair'.

The test statistic is a TAIL (a region), not a single P(X = x). Comparing P(X = x) with the level is the single most common way to lose every mark on the question.

Worked example

A real exam trap: tail, not point

A researcher believes boys are equally likely to pick a green or orange cube; she offers both to 10 boys and 8 pick green. She tests $H_0{:}\ p=0.5$ vs $H_1{:}\ p>0.5$ at 5%.

A student writes $P(X=8)=0.0439<0.05$ and rejects $H_0$ . This scores zero method: the test statistic must be the tail $P(X\ge8)$ , not the point.

$P(X\ge8)=0.0439+\tbinom{10}{9}(0.5)^{10}+(0.5)^{10}=0.0547$ . Since $0.0547>0.05$ , do not reject $H_0$ .

Answer

P(X\ge8)=0.0547>5\%\ \Rightarrow\ \text{do not reject }H_0

So there is insufficient evidence that boys prefer green.

(9709/62/O/N/23 Q3) — the examiner explicitly awards the “invalid” mark for spotting that a point probability is the wrong statistic.

Why a tail, never a single bar

Finding the rejection region (and the critical value)

Worked example

Build the rejection region, then reverse it

10% of people are left-handed; among 50 players you suspect fewer are. $X\sim B(50,0.1)$ , $H_1{:}\ p<0.1$ , level 5%.

$P(X\le1)=3.38\%<5\%$ but $P(X\le2)=11.2\%>5\%$ , so the region stops at 1.

Answer

\text{Reject if }X\le1\ (\text{critical value }1).

Reverse version: “if the critical value is 2, what is the least integer significance level?” Since P(X≤2) = 11.2%, you need at least a 12% level for 2 to be the cut-off.

Drag the significance level below and watch the rejection region grow — a lower level means a smaller region and a more demanding test.

α5%

H₁: p < 0.4 (lower tail)·critical region X ≤ 3, P(X ≤ 3) = 1.60%

Wording	H₁	Test statistic
decrease / fewer / lower	$p<p_0$	$P(X\le x)$
increase / more / higher	$p>p_0$	$P(X\ge x)$
changed / different / biased	$p\ne p_0$	relevant tail vs α/2

Examiner note

Compare a tail (

P(X\ge x)

), never the point

P(X=x)

; for an increase use

P(X\ge x)=1-P(X\le x-1)

; and conclude non-definitely in context (“insufficient evidence that …”), never “proved”. (9709/62/O/N/23 Q3) (9709/62/O/N/24 Q3)

Common mistake

Stating

H_0

about the data (“4 drivers crash”) instead of the parameter

p

; comparing a single

P(X=x)

instead of the tail; the off-by-one in

P(X\ge x)=1-P(X\le x-1)

; and a definite conclusion (“the die is biased”) instead of an evidence statement.