Math 143 C/E, Spring 2001
IPS Reading Questions
Chapter 6, Section 2
Both are used after a sample has been collected and measured, and one has a sample statistic in hand. A confidence interval indicates the most likely values for the population parameter, based on the value of our sample statistic and the type of sampling distribution this sample statistic has. A test of significance is also based upon this sampling distribution, but it is used to evaluate how much evidence the sample provides against the belief that the population parameter is in a certain, hypothesized place. (This hypothesis about the placement of the population parameter is called the null hypothesis.) Using the words of the authors on the bottom of p. 457, a test of significance finds the probability of getting an outcome as extreme or more extreme than the actually observed outcome assuming that the population parameter is as hypothesized in the null hypothesis.
One usually decides in advance of the sampling process what P-value will serve as a cutoff between probabilities that are too small and those that are not. This cutoff is called a significance level, and common choices are 10%, 5% and 1%, depending upon how sure you want to be in a given situation. (Note: It is no coincidence that the three values are closely linked to the three most common confidence levels, 90%, 95% and 99%; this relationship is made clearer in the blue box on p. 466.)
One decides based upon whether or not, prior to the collection and analysis of sample data, one has a predisposition to thinking that the population parameter is higher (or lower) than hypothesized.
You never say things like I accept the null hypothesis, nor I reject the alternative hypothesis. Rather, you say that the sample does not provide sufficient evidence to reject the null hypothesis (which very well may mean that the null hypothesis is true, but doesn't establish it as fact), or that the sample data is consistent with the null hypothesis.
The latter statement doesn't tell the reader how close of a call it was to reject the null hypothesis. If the P-value was 0.00057 and all you write is that you reject the null hypothesis at the 1% level, the reader doesn't realize that the null hypothesis would also have been rejected at the 0.1% level, information that makes the null hypothesis appear even less likely to be true.