Quick Summary
This guide will equip you to understand and interpret the potential errors that can occur in hypothesis testing. You will learn to define Type I and Type II errors within the specific context of a problem, analyze the consequences of each error, and understand the relationship between the significance level (α), sample size, and the probabilities of committing these errors. Mastering this topic is crucial for evaluating the real-world implications and limitations of statistical inference.
Key Concepts
In hypothesis testing, our goal is to use sample data to make a decision about a population parameter. We decide whether to reject or fail to reject the null hypothesis (H₀). However, since our decision is based on a sample, not the entire population, we can never be 100% certain. Our conclusion might be incorrect. There are two types of errors we can make.
The context of these errors is best understood with a table:
| Truth About the Population | ||
|---|---|---|
| Our Decision | H₀ is True | H₀ is False (Hₐ is True) |
| Reject H₀ | Type I Error (Incorrect) | Correct Decision |
| Fail to Reject H₀ | Correct Decision | Type II Error (Incorrect) |
[Image: A 2x2 grid showing the four outcomes of a hypothesis test. The columns are "H₀ is True" and "H₀ is False". The rows are "Reject H₀" and "Fail to Reject H₀". The cells contain "Correct Decision", "Type I Error", "Type II Error", and "Correct Decision" in the appropriate locations.]
Type I Error
Definition: A Type I error occurs when we reject the null hypothesis (H₀) when it is actually true.
We conclude there is convincing evidence for the alternative hypothesis (Hₐ) when, in reality, there isn't.
Think of this as a "false positive" or a "false alarm."
Probability of a Type I Error: The probability of making a Type I error is equal to the significance level, denoted by α (alpha).
Formula:
When we set a significance level of α = 0.05, we are accepting a 5% chance of making a Type I error. We are saying that if the null hypothesis is true, we are willing to incorrectly reject it 5% of the time in the long run.
Type II Error
Definition: A Type II error occurs when we fail to reject the null hypothesis (H₀) when it is actually false.
We conclude there is not convincing evidence for the alternative hypothesis (Hₐ) when, in reality, there is.
Think of this as a "false negative" or a "missed opportunity."
Probability of a Type II Error: The probability of making a Type II error is denoted by β (beta).
Formula:
The value of β is generally unknown but is influenced by several factors.
The Power of a Test
Definition: The power of a test is the probability that the test will correctly reject a false null hypothesis. It's the probability of not making a Type II error.
Power represents the ability of a test to detect a real effect or difference.
Formula:
A high power (close to 1) is desirable. It means our test is good at finding convincing evidence when it actually exists.
Factors Influencing Error Probabilities
Understanding the trade-offs between these errors is a critical skill.
Significance Level (α):
There is an inverse relationship between α and β.
Increasing α (e.g., from 0.05 to 0.10) makes it easier to reject H₀. This decreases the probability of a Type II error (β) but increases the probability of a Type I error (α).
Decreasing α (e.g., from 0.05 to 0.01) makes it harder to reject H₀. This decreases the probability of a Type I error (α) but increases the probability of a Type II error (β).
[Image: Two overlapping bell curves representing the sampling distributions under the null and alternative hypotheses. The first graph shows a small alpha (rejection region), which results in a large beta (area of the alternative curve that is not in the rejection region). The second graph shows a larger alpha, which results in a smaller beta.]
Sample Size (n):
Increasing the sample size (n) is the best way to improve a test. It provides more information about the population, reducing the variability of the sampling distribution.
A larger decreases the probability of a Type II error (β) without increasing the probability of a Type I error (α). Therefore, increasing sample size increases the power of the test.
Effect Size (Distance between H₀ and the true parameter value):
The "effect size" is the magnitude of the difference between the value of the parameter specified in the null hypothesis and its true value.
When the true parameter value is far from the null hypothesis value, the effect is easier to detect. This results in a lower probability of a Type II error (β) and higher power.
When the true parameter value is close to the null hypothesis value, the effect is harder to detect. This results in a higher probability of a Type II error (β) and lower power.
Key Vocabulary
Type I Error: The error of rejecting a true null hypothesis (H₀). Concluding there is an effect when there isn't one.
Type II Error: The error of failing to reject a false null hypothesis (H₀). Concluding there is no effect when there is one.
Significance Level (α): The probability of making a Type I error, set by the researcher before the test. It is the threshold for determining if a p-value is small enough to be statistically significant.
Probability of a Type II Error (β): The probability of failing to detect an effect that is actually present.
Power of a Test: The probability of correctly rejecting a false null hypothesis (Power = 1 - β). It measures a test's ability to detect a true effect.
Calculator Tech (TI-84)
No major calculator functions are required for this topic. The concepts of Type I and Type II errors are definitional and contextual, not computational on the TI-84.
How to Show Work on the FRQ
Free Response Questions on this topic require you to define errors and their consequences in the context of the problem. Never give generic, textbook definitions. Use the following templates to ensure you earn full credit.
Template for Defining a Type I Error and its Consequence:
Define the Error in Context: "A Type I error in this context would be to conclude [state the alternative hypothesis, Hₐ, in words] when, in reality, [state the null hypothesis, H₀, in words] is true."
State the Consequence: "A potential consequence of this error is that [describe a specific negative outcome that results from taking action based on the incorrect conclusion]."
Template for Defining a Type II Error and its Consequence:
Define the Error in Context: "A Type II error in this context would be to conclude [state the null hypothesis, H₀, in words] when, in reality, [state the alternative hypothesis, Hₐ, in words] is true."
State the Consequence: "A potential consequence of this error is that [describe a specific negative outcome or missed opportunity that results from failing to take action]."
Practice Problems
Problem 1:
A pharmaceutical company has developed a new drug designed to reduce the average recovery time from a certain illness. The current average recovery time with standard treatment is 18 days. The company will conduct a clinical trial to test if the new drug reduces this time. They will test the hypotheses H₀: μ = 18 days versus Hₐ: μ < 18 days, where μ is the true mean recovery time for patients taking the new drug.
(a) Describe a Type I error in this context.
(b) Describe a Type II error in this context.
(c) From a patient's perspective, which error would be more dangerous? Explain.
Solution:
(a) Type I Error:
Definition: A Type I error would be to conclude that the new drug reduces the average recovery time (μ < 18) when, in reality, the drug has no effect and the average recovery time is still 18 days (μ = 18).
Consequence: The company would spend millions of dollars marketing and producing a drug that is not effective. Patients might be prescribed this new, likely more expensive, drug instead of the standard treatment, with no actual benefit.
(b) Type II Error:
Definition: A Type II error would be to conclude that the new drug does not reduce the average recovery time (μ = 18) when, in reality, the drug is effective and does reduce the average recovery time (μ < 18).
Consequence: An effective drug that could help people recover faster would be abandoned. The company would miss a major opportunity, and patients would be denied access to a superior treatment.
(c) More Dangerous Error:
From a patient's perspective, a Type I error is likely more dangerous. A patient might switch to the new, ineffective drug, potentially paying more and expecting a faster recovery that never comes. This could lead to prolonged illness and wasted resources. While a Type II error represents a missed opportunity for a better treatment, a Type I error involves actively using an ineffective one.
Problem 2:
A factory that produces cereal claims that the average weight of its "16-ounce" boxes is at least 16 ounces. A quality control inspector is concerned that the machine is under-filling the boxes. The inspector takes a random sample of boxes and performs a hypothesis test with the following hypotheses, using a significance level of α = 0.05:
H₀: μ = 16
Hₐ: μ < 16
(a) Describe a Type II error in this context and state a potential consequence.
(b) The inspector is considering increasing the sample size for the next test. What effect would this have on the probability of a Type II error? Explain.
Solution:
(a) Type II Error:
Definition: A Type II error would be to conclude that the average box weight is 16 ounces (failing to find evidence of under-filling) when, in reality, the machine is under-filling the boxes and the true average weight is less than 16 ounces (μ < 16).
Consequence: The company would continue to use a faulty machine, systematically cheating customers by selling them less cereal than advertised. This could lead to customer complaints, loss of brand reputation, and potential legal action if discovered.
(b) Effect of Increasing Sample Size:
Increasing the sample size would decrease the probability of a Type II error (β). A larger sample provides more information and a more precise estimate of the true mean weight. This makes the test more powerful, meaning it is more likely to detect that the machine is under-filling the boxes if that is truly the case.
Common Mistakes to Avoid
Generic Definitions: The most common mistake on the AP exam is defining Type I and Type II errors without using the specific context of the problem. Always refer to the parameters and hypotheses of the given scenario. Use the templates in the "How to Show Work on the FRQ" section.
Confusing the Errors: A Type I error is when you Incorrectly reject a true H₀. A Type II error is when you make a mistake with two hypotheses (failing to reject a false H₀ when Hₐ is true). Another mnemonic: A Type I error is a "false alarm," while a Type II error is a "missed detection."
Confusing α and p-value: The significance level α is the pre-determined risk of a Type I error you are willing to accept (e.g., 5%). The p-value is a probability calculated from your sample data. You compare the p-value to α to make a decision. They are not the same thing.
Thinking α + β = 1: The probabilities of Type I and Type II errors are not complements. They have an inverse relationship (decreasing one tends to increase the other, all else being equal), but they do not sum to 1. Power and β are complements (Power = 1 - β).
Stating Consequences Vaguely: Be specific about the consequences. Don't just say "the company loses money." Explain how: "The company loses money by marketing and producing an ineffective drug." Don't just say "it's bad for customers." Explain why: "Customers will be sold underweight boxes of cereal, leading to dissatisfaction and complaints."