Hypothesis Test

Hypothesis test is a technique where sample data is used to determine if the confidence interval supports a particular claim. Hypothesis tests quantify how likely our data is given a particular claim.

This page will focus on usage of hypothesis tests in the context of mean comparison.

Procedure (Mean Comparison)

1. Null and Alternative Hypothesis

To perform hypothesis test with mean comparison, we need two things:

• The null hypothesis $H_{0}$ is the statement which we assume to be true
• The alternative hypothesis Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle H_{A}} is the complement of the null hypothesis.

Mean comparison work with the difference in means

Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle \mu _{1}-\mu _{2}}

As such, there are three sets of hypotheses:

• $H_{0}:\mu _{1}-\mu _{2}=0$ vs Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle H_{A}:\mu _{1}-\mu _{2}\neq 0}
• Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle H_{0}:\mu _{1}-\mu _{2}\geq 0} vs Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle H_{A}:\mu _{1}-\mu _{2}<0}
• Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle H_{0}:\mu _{1}-\mu _{2}\leq 0} vs Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle H_{A}:\mu _{1}-\mu _{2}>0}

2. Test-Statistic

Next, we need to calculate a test-statistic $t_{s}$. This measures how much our sample data differ from $H_{0}$. It summarizes our data to one number to perform hypothesis test on.

For mean comparison, the hypothesized difference is 0 (i.e. the means are the same). Therefore, the test-statistic is calculated as follows:

Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle t_{s}={\frac {{\bar {y_{1}}}-{\bar {y_{2}}}-0}{\sqrt {{\frac {s_{1}^{2}}{n_{1}}}+{\frac {s_{2}^{2}}{n_{2}}}}}}}

where the difference mean is subtracted by 0 to since that is the comparison point; all three sets of hypotheses in mean comparison compares against 0.

$t_{s}$, the more our data differs from $H_{0}$. Notice that it increases with sample mean difference and decreases with variance.

3. Find P-value

The p-value is the probability of observing our data or more extreme if $H_{0}$ is in fact true. To find this, we first need to know the sampling distribution of our random variable.

Distribution

In the case of mean comparison, because sample mean has normal distribution, by RV linear combination, the sampling distribution of Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle {\bar {Y}}_{1}-{\bar {Y}}_{2}} is

$({\bar {Y_{1}}}-{\bar {Y_{2}}})\sim N(\mu _{1}-\mu _{2},{\frac {\sigma _{1}^{2}}{n_{1}}}+{\frac {\sigma _{2}^{2}}{n_{2}}})$

Both means follow the t-distribution, therefore the difference also follows t-distribution.

We are not going to derive it, but the degree of freedom in this case is

Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle df=\upsilon ={\frac {({\frac {s_{1}^{2}}{n_{1}}}+{\frac {s_{2}^{2}}{n_{2}}})^{2}}{{\frac {(s_{1}^{2}/n_{1})^{2}}{n_{1}-1}}+{\frac {(s_{2}^{2}/n_{2})^{2}}{n_{2}-1}}}}}

where Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle \upsilon } is rounded down when using the t-table.

Now that we know the degrees of freedom and the test-statistic to compare against, we can calculate the p-value.

P-value

In the case of mean comparison, we have the following p-values:

• For Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle H_{A}:\mu _{1}-\mu _{2}\neq 0} , the p-value is Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle 2P(t>|t_{s}|)}
  • Two tails
• For Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A: \mu_1 - \mu_2 > 0} , the p-value is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(t > t_s)}
  • Upper tail
• For Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A: \mu_1 - \mu_2 < 0} , the p-value is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(t < t_s)}
  • Lower tail

The smaller the p-value, the less likely it is to observe our data or more extreme if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} is true, meaning that our data is unlikely if our claim is true.

4. Conclusion

We decide a cutoff point for our p-values, typically at Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha = 0.1, 0.05, 0.01} , called the level of significance.

If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p < \alpha} , our data supports Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A} , therefore Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} is rejected. Otherwise, we failed to reject Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} .

A CI that covers Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 0} implies that there is no significant difference, as it is plausible for the population means to be equal.