Type I and Type II errors | CAIE 9709 Probability & Statistics 2 Notes

Chapters

Probability & Statistics 2

Mathematics9709

Name the truth first, then the wrong decision

Truth first, then decision

Reality: H₀ true

reject H₀
Type I error

fail to reject
correct

Reality: H₁ true

reject H₀
correct

fail to reject
Type II error

Type II probability needs a specific true alternative value.

Name the truth first, then the decision. That prevents Type I and Type II from being swapped.

Definition

1 mark

What is meant by a Type I error?

Model answer: Rejecting H₀ when H₀ is true.

Definition

1 mark

What is meant by a Type II error?

Model answer: Failing to reject H₀ when H₁ is true.

Type I is a false alarm: the test reports a change that is not really present.
Type II is a missed effect: a real change is not detected by this sample.
In context, describe the incorrect conclusion without only writing “reject H₀” or “fail to reject H₀”.

Use the null model for Type I and the alternative model for Type II

Worked example

One cutoff, two different binomial models

A test of $H_0:p=\tfrac16$ against $H_1:p<\tfrac16$ uses 30 trials and rejects $H_0$ when $X\le1$ .

For Type I, assume the null value $p=\tfrac16$ is true and enter the rejection region:

X\sim\mathrm{B}\left(30,\frac16\right)

P(X\le1)=0.02949

The actual Type I probability is below 0.05 because a discrete boundary cannot usually land exactly on the stated significance level.

For Type II at the specific true value $p=0.02$ , use the non-rejection region $X\ge2$ :

X\sim\mathrm{B}(30,0.02)

P(X\ge2)=0.1205

(9709/62/M/J/25 Q8(b)–(c))

Key idea

A statement such as

H_1:p<\tfrac16

contains many possible values. Type II probability cannot be calculated until one actual alternative value, such as

p=0.02

, is given.

For normal models, integrate on the correct side of the cutoff

Worked example

A sensor threshold

A sensor reading is $N(10,3^2)$ when there is no fault and $N(22,7^2)$ when there is a fault. The system reports a fault when $X>17$ .

Type I: no fault is true, but the reading crosses the cutoff.

X\sim\mathrm{N}(10,3^2)

P(X>17)=0.0098

Type II: a fault is true, but the reading stays at or below 17.

X\sim\mathrm{N}(22,7^2)

P(X\le17)=0.2375

Lowering $\alpha$ makes the rejection region smaller. For a fixed sample size and rule, Type I decreases but Type II usually increases.
Increasing the sample size usually reduces Type II probability while the chosen Type I level can be kept unchanged.
Do not change the significance level after seeing the sample. That would make the stated error control meaningless.

Your turnType I and Type II from a rejection region [5 marks]

A test of $H_0:p=0.25$ against $H_1:p<0.25$ uses 35 trials and rejects H₀ when $X\le3$ . Find the Type I probability. If the true value is $p=0.05$ , find the Type II probability.

Show worked answer

Under $p=0.25$ ,

P(X\le3)=0.0136

Under $p=0.05$ ,

P(X\ge4)=0.0986

Answers: Type I probability 0.0136; Type II probability 0.0986.

(9709/63/M/J/25 Q7)

Common mistake

Do not calculate both errors from the null distribution. Type I uses the null model; Type II uses the stated true alternative model.

Name the truth first, then the wrong decision

Truth first, then decision

Reality: H₀ true

reject H₀
Type I error

fail to reject
correct

Reality: H₁ true

reject H₀
correct

fail to reject
Type II error

Type II probability needs a specific true alternative value.

Name the truth first, then the decision. That prevents Type I and Type II from being swapped.

Definition

1 mark

What is meant by a Type I error?

Model answer: Rejecting H₀ when H₀ is true.

Definition

1 mark

What is meant by a Type II error?

Model answer: Failing to reject H₀ when H₁ is true.

Type I is a false alarm: the test reports a change that is not really present.
Type II is a missed effect: a real change is not detected by this sample.
In context, describe the incorrect conclusion without only writing “reject H₀” or “fail to reject H₀”.

Use the null model for Type I and the alternative model for Type II

Worked example

One cutoff, two different binomial models

A test of $H_0:p=\tfrac16$ against $H_1:p<\tfrac16$ uses 30 trials and rejects $H_0$ when $X\le1$ .

For Type I, assume the null value $p=\tfrac16$ is true and enter the rejection region:

X\sim\mathrm{B}\left(30,\frac16\right)

P(X\le1)=0.02949

The actual Type I probability is below 0.05 because a discrete boundary cannot usually land exactly on the stated significance level.

For Type II at the specific true value $p=0.02$ , use the non-rejection region $X\ge2$ :

X\sim\mathrm{B}(30,0.02)

P(X\ge2)=0.1205

(9709/62/M/J/25 Q8(b)–(c))

Key idea

A statement such as

H_1:p<\tfrac16

contains many possible values. Type II probability cannot be calculated until one actual alternative value, such as

p=0.02

, is given.

For normal models, integrate on the correct side of the cutoff

Worked example

A sensor threshold

A sensor reading is $N(10,3^2)$ when there is no fault and $N(22,7^2)$ when there is a fault. The system reports a fault when $X>17$ .

Type I: no fault is true, but the reading crosses the cutoff.

X\sim\mathrm{N}(10,3^2)

P(X>17)=0.0098

Type II: a fault is true, but the reading stays at or below 17.

X\sim\mathrm{N}(22,7^2)

P(X\le17)=0.2375

Lowering $\alpha$ makes the rejection region smaller. For a fixed sample size and rule, Type I decreases but Type II usually increases.
Increasing the sample size usually reduces Type II probability while the chosen Type I level can be kept unchanged.
Do not change the significance level after seeing the sample. That would make the stated error control meaningless.

Your turnType I and Type II from a rejection region [5 marks]

A test of $H_0:p=0.25$ against $H_1:p<0.25$ uses 35 trials and rejects H₀ when $X\le3$ . Find the Type I probability. If the true value is $p=0.05$ , find the Type II probability.

Show worked answer

Under $p=0.25$ ,

P(X\le3)=0.0136

Under $p=0.05$ ,

P(X\ge4)=0.0986

Answers: Type I probability 0.0136; Type II probability 0.0986.

(9709/63/M/J/25 Q7)

Common mistake

Do not calculate both errors from the null distribution. Type I uses the null model; Type II uses the stated true alternative model.