Hypothesis Test: Difference between revisions

From Rice Wiki
VisualWikitext
(Created page with "= Comparing Means: Hypothesis Test = When comparing means, a technique called '''hypothesis test''' is used. This applies to other things, but in this context, we use two hypotheses. Hypothesis test is a technique where sample data is used to determine if the confidence interval supports a particular claim. Hypothesis tests quantify how likely our data is given a particular claim. == Procedure == First, we need two things: * The '''null hypothesis <math>H_0</math>''...")
 
No edit summary
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Comparing Means: Hypothesis Test =
'''Hypothesis test''' is a technique where sample data is used to
determine if the confidence interval supports a particular claim.
Hypothesis tests quantify how likely our data is given a particular
claim.


When comparing means, a technique called '''hypothesis test''' is used.
This page will focus on ''usage'' of hypothesis tests ''in the context
This applies to other things, but in this context, we use two
of mean comparison''.
hypotheses.


Hypothesis test is a technique where sample data is used to determine if
= Assumptions for HT =
the confidence interval supports a particular claim. Hypothesis tests
quantify how likely our data is given a particular claim.


== Procedure ==
We use the same assumptions used for a confidence interval for the
difference in means


First, we need two things:
# A random sample was taken
# Population 1 and 2 are independent
# Sample means are normally distributed
** Sample size is large enough (> 30)
** Population is normally distributed
 
= Procedure (Mean Comparison) =
 
== 1. Null and Alternative Hypothesis ==
 
To perform hypothesis test with mean comparison, we need two things:


* The '''null hypothesis <math>H_0</math>''' is the statement which we assume to be ''true''
* The '''null hypothesis <math>H_0</math>''' is the statement which we assume to be ''true''
* The '''alternative hypothesis <math>H_A</math>''' is the complement of the null hypothesis.
* The '''alternative hypothesis <math>H_A</math>''' is the complement of the null hypothesis.


Because sample mean has normal distribution, by RV linear combination,
Mean comparison work with the difference in means
the sampling distribution of <math>\bar{Y}_1 - \bar{Y}_2</math> is
 
<math>
\mu_1 - \mu_2
</math>
 
As such, there are three sets of hypotheses:
* <math>H_0: \mu_1 - \mu_2 = 0</math> vs <math>H_A: \mu_1 - \mu_2 \neq 0</math>
* <math>H_0: \mu_1 - \mu_2 \geq 0</math> vs <math>H_A: \mu_1 - \mu_2 < 0</math>
* <math>H_0: \mu_1 - \mu_2 \leq 0</math> vs <math>H_A: \mu_1 - \mu_2 > 0</math>
 
== 2. Test-Statistic ==
 
Next, we need to calculate a '''test-statistic <math>t_s</math>'''. This
measures how much our sample data differ from <math>H_0</math>. It
summarizes our data to one number to perform hypothesis test on.
 
For mean comparison, the hypothesized difference is 0 (i.e. the means
are the same). Therefore, the test-statistic is calculated as follows:
 
<math>
t_s = \frac{\bar{y_1} - \bar{y_2} - 0 }{ \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}
</math>
 
On the numerator, the difference mean is subtracted by 0 to since that is the
comparison point; all three sets of hypotheses in mean comparison
compares against 0.
 
On the denominator, the value is divided by the sample standard
deviation. This is a surprise tool that will help us later (bottom of
3.)
 
<math>t_s</math>, the more our data differs from <math>H_0</math>.
Notice that it increases with sample mean difference and decreases with
variance.
 
 
== 3. Find P-value ==
 
The '''p-value''' is the probability of observing our data or more
extreme if <math>H_0</math> is in fact true. To find this, we first need
to know the sampling distribution of our random variable.
 
=== Distribution ===
 
In the case of mean comparison, because sample mean has normal
distribution, by RV linear combination, the sampling distribution of
<math>\bar{Y}_1 - \bar{Y}_2</math> is


<math>
<math>
Line 23: Line 80:
</math>
</math>


We are not going to derive it, but the degree of freedom for this particular combination is
Both means follow the t-distribution, therefore the difference also
follows t-distribution.
 
We are not going to derive it, but the degree of freedom in this case is


<math>
<math>
v = \frac{ (\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2} )^2 }
df = \upsilon = \frac{ (\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2} )^2 }
{ \frac{(s_1^2 / n_1)^2}{ n_1 - 1} + \frac{(s_2^2 / n_2)^2}{ n_2 - 1} }
{ \frac{(s_1^2 / n_1)^2}{ n_1 - 1} + \frac{(s_2^2 / n_2)^2}{ n_2 - 1} }
</math>
</math>


'''Round down''' the value to use t-table.
where <math>\upsilon</math> is '''rounded down''' when using the t-table.
 
Remember how the test-statistic has the sample deviation on the
denominator? This is so that we can use the t-distribution to calculate
the probability! Now that we know the ''degrees of freedom'' and the
''test-statistic'' to compare against, we can calculate the p-value.
 
=== P-value ===
 
In the case of mean comparison, we have the following p-values:
 
* For <math>H_A: \mu_1 - \mu_2 \neq 0</math>, the p-value is <math>2P(t > |t_s|)</math>
** Two tails
* For <math>H_A: \mu_1 - \mu_2 > 0</math>, the p-value is <math>P(t > t_s)</math>
** Upper tail
* For <math>H_A: \mu_1 - \mu_2 < 0</math>, the p-value is <math>P(t < t_s)</math>
** Lower tail
 
The smaller the p-value, the less likely it is to observe our data or
more extreme if <math>H_0</math> is true, meaning that our data is
unlikely if our claim is true.
 
If there are no calculators, we can use a ''range'' of t-values that
contain the test-statistic. The result would be a lower and upper bound
p-value.
 
== 4. Conclusion ==
 
We decide a cutoff point for our p-values, typically at <math>\alpha =
0.1, 0.05, 0.01</math>, called the '''level of significance'''.
 
If <math>p < \alpha</math>, our data supports <math>H_A</math>,
therefore <math>H_0</math> is rejected. Otherwise, we failed to reject
<math>H_0</math>.
 


A CI that covers <math>0</math> implies that there is no significant
A CI that covers <math>0</math> implies that there is no significant
difference, as it is plausible for the population means to be equal.
difference, as it is plausible for the population means to be equal.
= Errors =
There are two types of possible errors for our conclusion.
* '''Type I error:''' Null is true but we rejected it
* '''Type II error:''' Null is false but we failed to reject it
Remeber the ''level of significance?'' Well it is actually
<math>
\alpha = P(Type I Error) = P(Reject H_0 | H_0 True)
</math>
So when setting the level of significance, we are actually controlling
type I error. The reason is that the distribution of <math>t_s</math> is
''non-specific:'' there are lots of possibilities that <math>H_0</math>
is false.
We also have
<math>
\beta = P(Type II Error) = P(FTR H_0 | H_0 False)
</math>
As <math>\alpha</math> increase, <math>\beta</math> decreases.
Therefore, we can choose what <math>\alpha</math> to use depending on
which error is probably worse. We ''cannot'' control <math>\beta</math>.
I don't really get why.
If we think Type I error is use, choose smaller <math>\alpha =
0.01</math>. Otherwise, use <math>\alpha = 0.10</math>.


[[Category:Sample Statistics]]
[[Category:Sample Statistics]]

Latest revision as of 01:55, 16 March 2024

Hypothesis test is a technique where sample data is used to determine if the confidence interval supports a particular claim. Hypothesis tests quantify how likely our data is given a particular claim.

This page will focus on usage of hypothesis tests in the context of mean comparison.

Assumptions for HT

We use the same assumptions used for a confidence interval for the difference in means

  1. A random sample was taken
  2. Population 1 and 2 are independent
  3. Sample means are normally distributed
    • Sample size is large enough (> 30)
    • Population is normally distributed

Procedure (Mean Comparison)

1. Null and Alternative Hypothesis

To perform hypothesis test with mean comparison, we need two things:

  • The null hypothesis is the statement which we assume to be true
  • The alternative hypothesis Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A} is the complement of the null hypothesis.

Mean comparison work with the difference in means

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mu_1 - \mu_2 }

As such, there are three sets of hypotheses:

  • Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0: \mu_1 - \mu_2 = 0} vs Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A: \mu_1 - \mu_2 \neq 0}
  • Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0: \mu_1 - \mu_2 \geq 0} vs Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A: \mu_1 - \mu_2 < 0}
  • Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0: \mu_1 - \mu_2 \leq 0} vs Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A: \mu_1 - \mu_2 > 0}

2. Test-Statistic

Next, we need to calculate a test-statistic Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle t_s} . This measures how much our sample data differ from Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} . It summarizes our data to one number to perform hypothesis test on.

For mean comparison, the hypothesized difference is 0 (i.e. the means are the same). Therefore, the test-statistic is calculated as follows:

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle t_s = \frac{\bar{y_1} - \bar{y_2} - 0 }{ \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}} }

On the numerator, the difference mean is subtracted by 0 to since that is the comparison point; all three sets of hypotheses in mean comparison compares against 0.

On the denominator, the value is divided by the sample standard deviation. This is a surprise tool that will help us later (bottom of 3.)

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle t_s} , the more our data differs from Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} . Notice that it increases with sample mean difference and decreases with variance.


3. Find P-value

The p-value is the probability of observing our data or more extreme if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} is in fact true. To find this, we first need to know the sampling distribution of our random variable.

Distribution

In the case of mean comparison, because sample mean has normal distribution, by RV linear combination, the sampling distribution of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \bar{Y}_1 - \bar{Y}_2} is

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle ( \bar{Y_1} - \bar{Y_2} ) \sim N(\mu_1 - \mu_2, \frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}) }

Both means follow the t-distribution, therefore the difference also follows t-distribution.

We are not going to derive it, but the degree of freedom in this case is

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle df = \upsilon = \frac{ (\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2} )^2 } { \frac{(s_1^2 / n_1)^2}{ n_1 - 1} + \frac{(s_2^2 / n_2)^2}{ n_2 - 1} } }

where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \upsilon} is rounded down when using the t-table.

Remember how the test-statistic has the sample deviation on the denominator? This is so that we can use the t-distribution to calculate the probability! Now that we know the degrees of freedom and the test-statistic to compare against, we can calculate the p-value.

P-value

In the case of mean comparison, we have the following p-values:

  • For Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A: \mu_1 - \mu_2 \neq 0} , the p-value is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 2P(t > |t_s|)}
    • Two tails
  • For Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A: \mu_1 - \mu_2 > 0} , the p-value is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(t > t_s)}
    • Upper tail
  • For Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A: \mu_1 - \mu_2 < 0} , the p-value is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(t < t_s)}
    • Lower tail

The smaller the p-value, the less likely it is to observe our data or more extreme if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} is true, meaning that our data is unlikely if our claim is true.

If there are no calculators, we can use a range of t-values that contain the test-statistic. The result would be a lower and upper bound p-value.

4. Conclusion

We decide a cutoff point for our p-values, typically at Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha = 0.1, 0.05, 0.01} , called the level of significance.

If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p < \alpha} , our data supports Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_A} , therefore Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} is rejected. Otherwise, we failed to reject Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} .


A CI that covers Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 0} implies that there is no significant difference, as it is plausible for the population means to be equal.

Errors

There are two types of possible errors for our conclusion.

  • Type I error: Null is true but we rejected it
  • Type II error: Null is false but we failed to reject it

Remeber the level of significance? Well it is actually

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha = P(Type I Error) = P(Reject H_0 | H_0 True) }

So when setting the level of significance, we are actually controlling type I error. The reason is that the distribution of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle t_s} is non-specific: there are lots of possibilities that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H_0} is false.

We also have

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta = P(Type II Error) = P(FTR H_0 | H_0 False) }

As Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha} increase, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta} decreases. Therefore, we can choose what Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha} to use depending on which error is probably worse. We cannot control Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta} . I don't really get why.

If we think Type I error is use, choose smaller Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha = 0.01} . Otherwise, use Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha = 0.10} .

Retrieved from "http://ricefriedegg.com:80/mediawiki/index.php?title=Hypothesis_Test&oldid=331"