Hypothesis Test: Difference between revisions
No edit summary |
No edit summary |
||
Line 143: | Line 143: | ||
So when setting the level of significance, we are actually controlling | So when setting the level of significance, we are actually controlling | ||
type I error. | type I error. The reason is that the distribution of <math>t_s</math> is | ||
''non-specific:'' there are lots of possibilities that <math>H_0</math> | |||
is false. | |||
We also have | We also have | ||
Line 150: | Line 152: | ||
\beta = P(Type II Error) = P(FTR H_0 | H_0 False) | \beta = P(Type II Error) = P(FTR H_0 | H_0 False) | ||
</math> | </math> | ||
As <math>\alpha</math> increase, <math>\beta</math> decreases. | |||
Therefore, we can choose what <math>\alpha</math> to use depending on | |||
which error is probably worse. We ''cannot'' control <math>\beta</math>. | |||
I don't really get why. | |||
If we think Type I error is use, choose smaller <math>\alpha = | |||
0.01</math>. Otherwise, use <math>\alpha = 0.10</math>. | |||
[[Category:Sample Statistics]] | [[Category:Sample Statistics]] |
Latest revision as of 01:55, 16 March 2024
Hypothesis test is a technique where sample data is used to determine if the confidence interval supports a particular claim. Hypothesis tests quantify how likely our data is given a particular claim.
This page will focus on usage of hypothesis tests in the context of mean comparison.
Assumptions for HT
We use the same assumptions used for a confidence interval for the difference in means
- A random sample was taken
- Population 1 and 2 are independent
- Sample means are normally distributed
- Sample size is large enough (> 30)
- Population is normally distributed
Procedure (Mean Comparison)
1. Null and Alternative Hypothesis
To perform hypothesis test with mean comparison, we need two things:
- The null hypothesis is the statement which we assume to be true
- The alternative hypothesis is the complement of the null hypothesis.
Mean comparison work with the difference in means
As such, there are three sets of hypotheses:
- vs
- vs
- vs
2. Test-Statistic
Next, we need to calculate a test-statistic . This measures how much our sample data differ from . It summarizes our data to one number to perform hypothesis test on.
For mean comparison, the hypothesized difference is 0 (i.e. the means are the same). Therefore, the test-statistic is calculated as follows:
On the numerator, the difference mean is subtracted by 0 to since that is the comparison point; all three sets of hypotheses in mean comparison compares against 0.
On the denominator, the value is divided by the sample standard deviation. This is a surprise tool that will help us later (bottom of 3.)
, the more our data differs from . Notice that it increases with sample mean difference and decreases with variance.
3. Find P-value
The p-value is the probability of observing our data or more extreme if is in fact true. To find this, we first need to know the sampling distribution of our random variable.
Distribution
In the case of mean comparison, because sample mean has normal distribution, by RV linear combination, the sampling distribution of is
Both means follow the t-distribution, therefore the difference also follows t-distribution.
We are not going to derive it, but the degree of freedom in this case is
where is rounded down when using the t-table.
Remember how the test-statistic has the sample deviation on the denominator? This is so that we can use the t-distribution to calculate the probability! Now that we know the degrees of freedom and the test-statistic to compare against, we can calculate the p-value.
P-value
In the case of mean comparison, we have the following p-values:
- For , the p-value is
- Two tails
- For , the p-value is
- Upper tail
- For , the p-value is
- Lower tail
The smaller the p-value, the less likely it is to observe our data or more extreme if is true, meaning that our data is unlikely if our claim is true.
If there are no calculators, we can use a range of t-values that contain the test-statistic. The result would be a lower and upper bound p-value.
4. Conclusion
We decide a cutoff point for our p-values, typically at , called the level of significance.
If , our data supports , therefore is rejected. Otherwise, we failed to reject .
A CI that covers implies that there is no significant
difference, as it is plausible for the population means to be equal.
Errors
There are two types of possible errors for our conclusion.
- Type I error: Null is true but we rejected it
- Type II error: Null is false but we failed to reject it
Remeber the level of significance? Well it is actually
So when setting the level of significance, we are actually controlling type I error. The reason is that the distribution of is non-specific: there are lots of possibilities that is false.
We also have
As increase, decreases. Therefore, we can choose what to use depending on which error is probably worse. We cannot control . I don't really get why.
If we think Type I error is use, choose smaller . Otherwise, use .