Hypotheses, significance and exact tests

We write $H_0$ for the claim being tested and $H_1$ for the possible change. A tail is an extreme end of a probability distribution.

Imagine that a die is claimed to be fair, but the number 6 appears unusually often. First pretend the claim is true. Then calculate how likely it would be to obtain this many sixes, or even more, by chance.

• $H_0$ is the starting model. It contains the equality, such as $H_0:p=\tfrac16$.
• $H_1$ is the change being investigated. Its sign tells you which extreme results count as evidence.
• The test statistic is a quantity calculated from the sample and used in the decision, such as an observed count, sample mean or z-value.
• The tail probability is calculated as if H₀ were true. It is not the probability that H₀ itself is true.

• The significance level $\alpha$ is chosen before the data are inspected. Common values are 5%, 2.5% and 1%.
• If the relevant tail probability is at most $\alpha$, reject $H_0$. If it is larger, fail to reject $H_0$.
• Results inside the critical region lead to rejection of $H_0$. Its boundary is a critical value.
• If the result is not significant, write “fail to reject H₀” or “there is insufficient evidence”. Do not claim that H₀ has been proved.
• Two decision routes are equivalent: compare the tail probability with $\alpha$, or check whether the observed test statistic lies in the critical region.

• Calculator lower tail: use cumulative probability $P(X\le r)$.
• Calculator upper tail: use $1-P(X\le r-1)$, not $1-P(X\le r)$.
• For a Poisson rate, first scale the null mean to the full observation interval. The same tail rule then applies.

Poisson bridge: if the claimed rate is 4 per hour and 20 events are observed in 3 hours, then under H₀ $X\sim\mathrm{Po}(12)$ and $P(X\ge20)=0.0213$. The upper-tail result is significant at 5%.