Hypothesis Test

Hypothesis test is a technique where sample data is used to determine if the confidence interval supports a particular claim. Hypothesis tests quantify how likely our data is given a particular claim.

This page will focus on usage of hypothesis tests in the context of mean comparison.

Procedure (Mean Comparison)

1. Null and Alternative Hypothesis

To perform hypothesis test with mean comparison, we need two things:

• The null hypothesis $H_{0}$ is the statement which we assume to be true
• The alternative hypothesis $H_{A}$ is the complement of the null hypothesis.

Mean comparison work with the difference in means

$\mu _{1}-\mu _{2}$

As such, there are three sets of hypotheses:

• $H_{0}:\mu _{1}-\mu _{2}=0$ vs $H_{A}:\mu _{1}-\mu _{2}\neq 0$
• $H_{0}:\mu _{1}-\mu _{2}\geq 0$ vs $H_{A}:\mu _{1}-\mu _{2}<0$
• $H_{0}:\mu _{1}-\mu _{2}\leq 0$ vs $H_{A}:\mu _{1}-\mu _{2}>0$

2. Test-Statistic

Next, we need to calculate a test-statistic $t_{s}$. This measures how much our sample data differ from $H_{0}$. For mean comparison, this is

$t_{s}={\frac {{\bar {y_{1}}}-{\bar {y_{2}}}-0}{\sqrt {{\frac {s_{1}^{2}}{n_{1}}}+{\frac {s_{2}^{2}}{n_{2}}}}}}$

$t_{s}$ follows t-distribution with $df=\upsilon$. The larger the $t_{s}$, the more our data differs from $H_{0}$. Notice that it increases with sample mean difference and decreases with variance.

Distribution

Because sample mean has normal distribution, by RV linear combination, the sampling distribution of ${\bar {Y}}_{1}-{\bar {Y}}_{2}$ is

$({\bar {Y_{1}}}-{\bar {Y_{2}}})\sim N(\mu _{1}-\mu _{2},{\frac {\sigma _{1}^{2}}{n_{1}}}+{\frac {\sigma _{2}^{2}}{n_{2}}})$

3. Find P-value

The p-value is the probability of observing our data or more extreme if $H_{0}$ is in fact true.

In the case of mean comparison, we have the following p-values:

• For $H_{A}:\mu _{1}-\mu _{2}\neq 0$, the p-value is $2P(t>|t_{s}|)$
  • Two tails
• For $H_{A}:\mu _{1}-\mu _{2}>0$, the p-value is $P(t>t_{s})$
  • Upper tail
• For $H_{A}:\mu _{1}-\mu _{2}<0$, the p-value is $P(t<t_{s})$
  • Lower tail

The smaller the p-value, the less likely it is to observe our data or more extreme if $H_{0}$ is true.

4. Conclusion

We decide a cutoff point for our p-values, typically at $\alpha =0.1,0.05,0.01$, called the level of significance.

If $p<\alpha$, our data supports $H_{A}$, therefore $H_{0}$ is rejected. Otherwise, we failed to reject $H_{0}$.

We are not going to derive it, but the degree of freedom for this particular combination is

$\upsilon ={\frac {({\frac {s_{1}^{2}}{n_{1}}}+{\frac {s_{2}^{2}}{n_{2}}})^{2}}{{\frac {(s_{1}^{2}/n_{1})^{2}}{n_{1}-1}}+{\frac {(s_{2}^{2}/n_{2})^{2}}{n_{2}-1}}}}$

Round down the value to use t-table.

A CI that covers $0$ implies that there is no significant difference, as it is plausible for the population means to be equal.