Mathematics 9709 · Probability & Statistics 2

Type I and Type II errors

A test can be wrong in two opposite ways. Type I = false alarm: reject a true claim (convict the innocent). Type II = missed it: accept a false claim (free the guilty). The fact that ties it together: $P(\text{Type I})=$ the significance level itself.

False alarm vs missed it

	Accept H₀	Reject H₀
H₀ true	correct	Type I (false alarm)
H₀ false	Type II (missed it)	correct

$P(\text{Type I})=P(\text{reject }H_0\mid H_0\text{ true})$ = the significance level. $P(\text{Type II})=P(\text{accept }H_0\mid H_1\text{ true})$ — computable only with the true alternative value, working in that model. The asymmetry matters: rejecting $H_0$ rules out a Type II; accepting it rules out a Type I.

P(Type I) lives in the H₀ model; P(Type II) lives in the TRUE H₁ model. You can never make a Type II if you rejected, nor a Type I if you accepted.

Worked example

Type I from a binomial test

16 coin tosses, suspect bias towards tails. $X\sim B(16,0.5)$ , $H_1{:}\ p>0.5$ at 5%.

$P(X\ge12)=3.84\%<5\%$ but $P(X\ge11)=10.5\%$ , so the critical region is $X\ge12$ . The Type I probability is the achieved size of that region.

Answer

P(\text{Type I})=P(X\ge12\mid p=0.5)=3.84\%

Not exactly 5%: a discrete binomial only reaches the nearest achievable tail ≤ 5%. In context, a Type I error here means “deciding the coin is biased when it is actually fair.”

Worked example

Type I = the critical region's probability (real paper)

Back to the cube test, $X\sim B(10,0.5)$ , $H_1{:}\ p>0.5$ at 5%. We found $P(X\ge8)=5.47\%>5\%$ , so 8 is not in the region; $P(X\ge9)=1.07\%<5\%$ , so the region is $X\ge9$ .

$P(\text{Type I})=P(X\ge9\mid p=0.5)=\tbinom{10}{9}(0.5)^{10}+(0.5)^{10}=\tfrac{11}{1024}$ .

Answer

P(\text{Type I})=\tfrac{11}{1024}=0.0107

(9709/62/O/N/23 Q3) — once the rejection region is fixed, P(Type I) is just its total probability under H₀, never the nominal 5%.

Type II needs the true model

A Type II probability only exists once you are told what is actually true. Swap to that distribution and find the chance the test still lands in the acceptance region.

Flag anyone with protein > 17. Type I = the healthy curve poking past the line (false alarm); Type II = the 'has it' curve falling short of it (missed). Slide the line and one shrinks as the other grows.

Worked example

Type I AND Type II from two normal models

A protein level $X$ screens for a condition. Healthy people have $X\sim N(10,3^2)$ ; those with it have $X\sim N(22,7^2)$ . The doctor flags anyone with $X>17$ .

Type I (healthy, flagged): use $N(10,9)$ . $P(X>17)=1-\Phi(\tfrac{17-10}{3})=1-\Phi(2.333)=0.0098$ .
Type II (has it, missed): use $N(22,49)$ . $P(X<17)=\Phi(\tfrac{17-22}{7})=\Phi(-0.714)=0.238$ .

Answer

P(\text{Type I})=0.0098,\quad P(\text{Type II})=0.238

A continuous variable (protein level) needs NO continuity correction. The two probabilities use different curves — switching models is the whole skill.

Worked example

Reverse Type II: how good must the test be?

Same condition model $X\sim N(22,7^2)$ . The doctor wants a cut-off $a$ (flag when $X>a$ ) so the test misses a true case with probability under 0.03. Find the range of $a$ .

A miss is $X<a$ under the true model, so require $P(X<a)<0.03$ .
Standardise: $\Phi\!\left(\tfrac{a-22}{7}\right)<0.03$ , and $\Phi^{-1}(0.03)=-1.881$ .
$\tfrac{a-22}{7}<-1.881\Rightarrow a<22-1.881\times7=8.83$ .

Answer

a<8.83

A lower cut-off catches more true cases (smaller Type II) — but that same move lets more healthy people past the line, so Type I climbs. You are choosing where on the trade-off to sit.

Worked example

The balanced cut-off: equal Type I and Type II

Both models again, $N(10,3^2)$ healthy and $N(22,7^2)$ with the condition. Find the cut-off $x$ where $P(\text{Type I})=P(\text{Type II})$ .

Type I (healthy past $x$ ): $1-\Phi\!\left(\tfrac{x-10}{3}\right)$ . Type II (condition below $x$ ): $\Phi\!\left(\tfrac{x-22}{7}\right)$ .
Set equal: $1-\Phi\!\left(\tfrac{x-10}{3}\right)=\Phi\!\left(\tfrac{x-22}{7}\right)$ . By symmetry of $\Phi$ , the arguments are negatives: $\tfrac{10-x}{3}=\tfrac{x-22}{7}$ .
Cross-multiply: $7(10-x)=3(x-22)\Rightarrow 70-7x=3x-66\Rightarrow x=13.6$ .

Answer

x=13.6\ (\text{each error then }\approx0.115)

This single value is the balance point the slider below has to land on — push the line either side and one error grows while the other shrinks.

Drag the cut-off below: push it right and false alarms (Type I) shrink while misses (Type II) balloon. You trade them — you cannot kill both at once.

cutoff

P(Type I) = 0.010·P(Type II) = 0.238·raising the bar misses more

Why cutting one error grows the other

Both errors are areas cut by the same critical line: slide it to shrink the false-alarm tail (Type I) and the missed-detection area (Type II) grows — you trade them, not remove both. The honest way to cut both is a larger $n$ , which narrows both curves so they overlap less.

Examiner note

P(\text{I})=P(\text{reject }H_0\mid H_0)

= the achieved tail under

H_0

;

P(\text{II})=P(\text{accept }H_0\mid H_1)

, so switch to the true model. A discrete test rarely hits the level exactly. (9709/62/M/J/23 Q6) (9709/62/O/N/24 Q3)

Common mistake

Computing

P(\text{Type II})

under

H_0

instead of the true

H_1

model (you must switch distributions); quoting the nominal level as

P(\text{Type I})

when the discrete region is smaller; mixing up the two errors (Type I rejects a true

H_0

); and adding a continuity correction to a continuous variable.

Your turn— tap to reveal the worked answer (Type II under the true alternative)

Sunflower heights are $N(1.8,0.32^2)$ . A friend claims singing raises the mean; the gardener will believe it if one plant exceeds $2.25$ m. If the true mean of sung-to plants is $2.4$ m (same SD), find $P(\text{Type II})$ .

Use the true model $N(2.4,0.32^2)$ : $P(X<2.25)=\Phi(\tfrac{2.25-2.4}{0.32})=\Phi(-0.469)$ .

Answer

P(\text{Type II})=\Phi(-0.469)=0.320

PreviousOne/two-tailed & the normal test NextThe Poisson formula & scaling λ

Type I and Type II errors

False alarm vs missed it

	Accept H₀	Reject H₀
H₀ true	correct	Type I (false alarm)
H₀ false	Type II (missed it)	correct

P(Type I) lives in the H₀ model; P(Type II) lives in the TRUE H₁ model. You can never make a Type II if you rejected, nor a Type I if you accepted.

Worked example

Type I from a binomial test

16 coin tosses, suspect bias towards tails. $X\sim B(16,0.5)$ , $H_1{:}\ p>0.5$ at 5%.

$P(X\ge12)=3.84\%<5\%$ but $P(X\ge11)=10.5\%$ , so the critical region is $X\ge12$ . The Type I probability is the achieved size of that region.

Answer

P(\text{Type I})=P(X\ge12\mid p=0.5)=3.84\%

Not exactly 5%: a discrete binomial only reaches the nearest achievable tail ≤ 5%. In context, a Type I error here means “deciding the coin is biased when it is actually fair.”

Worked example

Type I = the critical region's probability (real paper)

Back to the cube test, $X\sim B(10,0.5)$ , $H_1{:}\ p>0.5$ at 5%. We found $P(X\ge8)=5.47\%>5\%$ , so 8 is not in the region; $P(X\ge9)=1.07\%<5\%$ , so the region is $X\ge9$ .

$P(\text{Type I})=P(X\ge9\mid p=0.5)=\tbinom{10}{9}(0.5)^{10}+(0.5)^{10}=\tfrac{11}{1024}$ .

Answer

P(\text{Type I})=\tfrac{11}{1024}=0.0107

(9709/62/O/N/23 Q3) — once the rejection region is fixed, P(Type I) is just its total probability under H₀, never the nominal 5%.

Type II needs the true model

A Type II probability only exists once you are told what is actually true. Swap to that distribution and find the chance the test still lands in the acceptance region.

Worked example

Type I AND Type II from two normal models

A protein level $X$ screens for a condition. Healthy people have $X\sim N(10,3^2)$ ; those with it have $X\sim N(22,7^2)$ . The doctor flags anyone with $X>17$ .

Type I (healthy, flagged): use $N(10,9)$ . $P(X>17)=1-\Phi(\tfrac{17-10}{3})=1-\Phi(2.333)=0.0098$ .
Type II (has it, missed): use $N(22,49)$ . $P(X<17)=\Phi(\tfrac{17-22}{7})=\Phi(-0.714)=0.238$ .

Answer

P(\text{Type I})=0.0098,\quad P(\text{Type II})=0.238

A continuous variable (protein level) needs NO continuity correction. The two probabilities use different curves — switching models is the whole skill.

Worked example

Reverse Type II: how good must the test be?

Same condition model $X\sim N(22,7^2)$ . The doctor wants a cut-off $a$ (flag when $X>a$ ) so the test misses a true case with probability under 0.03. Find the range of $a$ .

A miss is $X<a$ under the true model, so require $P(X<a)<0.03$ .
Standardise: $\Phi\!\left(\tfrac{a-22}{7}\right)<0.03$ , and $\Phi^{-1}(0.03)=-1.881$ .
$\tfrac{a-22}{7}<-1.881\Rightarrow a<22-1.881\times7=8.83$ .

Answer

a<8.83

A lower cut-off catches more true cases (smaller Type II) — but that same move lets more healthy people past the line, so Type I climbs. You are choosing where on the trade-off to sit.

Worked example

The balanced cut-off: equal Type I and Type II

Both models again, $N(10,3^2)$ healthy and $N(22,7^2)$ with the condition. Find the cut-off $x$ where $P(\text{Type I})=P(\text{Type II})$ .

Type I (healthy past $x$ ): $1-\Phi\!\left(\tfrac{x-10}{3}\right)$ . Type II (condition below $x$ ): $\Phi\!\left(\tfrac{x-22}{7}\right)$ .
Set equal: $1-\Phi\!\left(\tfrac{x-10}{3}\right)=\Phi\!\left(\tfrac{x-22}{7}\right)$ . By symmetry of $\Phi$ , the arguments are negatives: $\tfrac{10-x}{3}=\tfrac{x-22}{7}$ .
Cross-multiply: $7(10-x)=3(x-22)\Rightarrow 70-7x=3x-66\Rightarrow x=13.6$ .

Answer

x=13.6\ (\text{each error then }\approx0.115)

This single value is the balance point the slider below has to land on — push the line either side and one error grows while the other shrinks.

Drag the cut-off below: push it right and false alarms (Type I) shrink while misses (Type II) balloon. You trade them — you cannot kill both at once.

cutoff

P(Type I) = 0.010·P(Type II) = 0.238·raising the bar misses more

Why cutting one error grows the other

Examiner note

P(\text{I})=P(\text{reject }H_0\mid H_0)

= the achieved tail under

H_0

;

P(\text{II})=P(\text{accept }H_0\mid H_1)

, so switch to the true model. A discrete test rarely hits the level exactly. (9709/62/M/J/23 Q6) (9709/62/O/N/24 Q3)

Common mistake

Computing

P(\text{Type II})

under

H_0

instead of the true

H_1

model (you must switch distributions); quoting the nominal level as

P(\text{Type I})

when the discrete region is smaller; mixing up the two errors (Type I rejects a true

H_0

); and adding a continuity correction to a continuous variable.

Your turn— tap to reveal the worked answer (Type II under the true alternative)

Use the true model $N(2.4,0.32^2)$ : $P(X<2.25)=\Phi(\tfrac{2.25-2.4}{0.32})=\Phi(-0.469)$ .

Answer

P(\text{Type II})=\Phi(-0.469)=0.320