Confidence Intervals#
Important Readings
[FPP07], Chapters 21, 23
The Accuracy of Percentages#
Let’s briefly review the Central Limit Theorem.
Fig. 49 and Fig. 50 illustrate the central limit theorem for the case of flipping a weighted coin that comes up heads 20% of the time. For \(n\) flips of the coin, we can compute standard errors:
If we repeatedly flip a coin 100 times, we can expect the sample proportion to be in the interval \(0.2\pm 2\times 0.04\) for 95% of the repetitions, where 0.04 is the SE for the proportion.
Fig. 49 Each miniature histogram reflects \(n=100\) coin flips where \(p=0.2\). Miniature histograms are arranged into a larger histogram according to their sample average.#
If we repeatedly flip a coin 400 times, we can expect the sample proportion to be in the interval \(0.2\pm 2\times 0.02\) for 95% of the repetitions, where 0.02 is the SE for the proportion.
Fig. 50 Each miniature histogram reflects \(n=400\) coin flips where \(p=0.2\). Miniature histograms are arranged into a larger histogram according to their sample average.#
This is all quite nice except it answers a question that isn’t very interesting because we start by knowing the true parameter is \(p=0.2\) and then we imagine repeatedly drawing samples.
Consider again the example from Finding the right SE, where a researcher at Peloton wants to find the proportion of members who own an Apple Watch. In practice, only one sample will be collected and only one sample proportion \(\hat{p}\) is observed. Interesting questions relate the unknown \(p\) to the observed \(\hat{p}\). That is, we are trying to make an inference about the population from the sample. To get some sense of the margin of error for the sample proportion \(\hat{p}\), we also have to estimate the SE.
To estimate the SE, we must first estimate the SD for the 0-1 box. According to the bootstrap procedure, the SD of the box can be estimated by substituting the fraction of 0’s and 1’s in the sample for the unknown fractions in the box. This provides a good estimate for a large enough sample.
That is, we estimate the SE based on the population parameters, \(\sqrt{\frac{p(1-p)}{n}}\), by substituting the sample proportion: \(\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}\).
Example: The Pew Research Center conducted a study on tipping. They surveyed 11,945 adults in the US. 59% reported that they always tip when having food delivered. Among all adults in the US, a proportion \(p\) must always tip. The survey estimates this as \(\hat{p} = 0.59\). The estimated SD as \(\sqrt{.59\times.41} \approx 0.49\). The SE for the sample proportion is SE = \(\frac{0.49}{\sqrt{11945}} \approx 0.004\). Converting to percentages, we expect that about 59% of adults always tip, give or take 0.4% as a typical chance error.
Confidence Intervals#
A confidence interval for a sample proportion can be found
And more generally, an arbitrary \(a\%\) confidence interval can be found as \(\hat{p} \pm z^{\star} \text{SE}\), where \(z^\star\) is the critical value such that \(a\%\) of the area under a normal curve is between \(-z^\star\) and \(z^\star\).
Example: Use a \(z\)-table to find the appropriate critical value for a 90% confidence interval.
90% confidence interval
The critical value is 1.645. The interval takes the form \( \hat{p} \pm 1.645 \times \text{SE}\).
Interpretation of a Confidence Interval#
The correct interpretation of a confidence interval is often misunderstood. The interpretation is based on our frequency theory of chance. First, there is nothing random about the parameter \(p\). Our sample is random and thus randomness is manifested in \(\hat{p}\) instead. With \(p\) nonrandom, we can’t make probabilistic statement like \(p\) is in a 95% confidence interval with 95% probability. Relying on probabilities being long-run frequencies instead, we say that if we were to construct many samples and thus many 95% confidence intervals, about 95% of the intervals would contain the true parameter.
Let’s return to the simulated data with \(p=0.2\), in Fig. 49 and Fig. 50. Each miniature histogram is a sample and most of the sample means are not equal to 0.2. However, 95% confidence intervals can be found for each. Indeed, for 95 of 100 of the simulated samples, the confidence interval would cover the true parameter.
Fig. 51 Each miniature histogram reflects \(n=100\) coin flips where \(p=0.2\). Grayed out histograms are those where the confidence interval misses the true parameter.#
Fig. 52 Each miniature histogram reflects \(n=400\) coin flips where \(p=0.2\). Grayed out histograms are those where the confidence interval misses the true parameter.#
Here are a few more wrong interpretations for a 95% confidence interval.
95% of all of the data is in the interval.
95% of randomly selected data points will be in the interval.
The sample average is contained in the interval for 95% of samples.
The Accuracy of Averages#
For averages, the central limit theorem again holds. When drawing at random from a box, the sample average will follow a normal distribution even if the contents of the box do not.
Proportions are a special kind of average. The sample average \(\bar{x}\) estimates the population average \(\mu\). For general averages, there is no formula analogous to SD=\(\sqrt{p(1-p)}\). The standard deviation must be estimated from the data. With a large simple random sample, the SD of the sample is a good estimate of the SD of the box.[1] Then, the SE for the average can be calculated
Once the SE is found, a confidence interval is constructed like in the case of proportions. For example, a 95% confidence interval is \(\bar{x} \pm 2\text{SE}\).
Example: Use this American Time Use Survey data to find a 95% confidence interval for the hours spent alone among 2020 respondents.
95% confidence interval
The sample size is \(n\) = 8782, the sample average is 6.03 and the SE is 0.05.
The interval is \(6.03 \pm 0.1\).
There are important caveats related to this number that would require a look at the ATUS documentation. Notably, no data was collected at the peak of the pandemic.
Exercises#
Exercise 45
In a survey of 96 randomly selected Americans, \(\hat{p} = 0.60\) said they think ice cream should be banned. Find the 95% confidence interval for the proportion.
Exercise 46
You are interested in the 95% confidence interval of a sample mean. Which of the following makes this interval more narrow?
More observations.
Fewer observations.
Higher value of the average.
Lower value of the average.
Both 1 and 4
Exercise 47
Consider a multiple choice exam with choices A, B, C, D, and E for each question. Assume that the probability that a particular question has the answer \(C\) is \(p = 0.2\) and that all letter answers are determined independently. Find a 90% confidence interval for the proportion of questions that will have a correct answer of C on a 1600-question exam.