Introducing Statistics: Should I Worry About Error? Chapter Notes | AP Statistics - Grade 9 PDF Download

No statistical study is perfect, and there is always a chance of error occurring. There are several sources of error that can affect the results of a statistical study, including sampling error, measurement error, and bias.

• Sampling Error: Sampling error occurs when the sample used in the study is not representative of the population being studied. This can lead to incorrect conclusions being drawn about the population based on the sample data.
• Measurement Error: Measurement error occurs when there is an error in the measurement of the variables being studied due to the presence of confounding variables. This can also lead to incorrect conclusions being drawn.
• Bias: Bias is another source of error that can occur in statistical studies. Bias can occur in the sampling process, the measurement process, or in the analysis of the data. Bias can lead to incorrect conclusions being drawn about the population being studied.

Type I Errors

A Type I error, also known as a false positive, is an error that occurs when the null hypothesis is rejected when it should have been accepted. The probability of a Type I error occurring is equal to the alpha level, which is the level of significance that is chosen for the study. A common alpha level is 0.05, which means that there is a 5% chance of making a Type I error.
It's important to choose an appropriate alpha level for a study, as a lower alpha level (e.g., 0.01) will result in a higher probability of making a Type I error, while a higher alpha level (e.g., 0.1) will result in a lower probability of making a Type I error. It's also crucial to consider the consequences of making a Type I error, as rejecting the null hypothesis when it is true can lead to incorrect conclusions being drawn about the population being studied.

Example

Let's say an author claims that the mean income for a given area is $45,000.
We sample a group of 50 families and find that the mean income of our sample is $42,500. In performing a statistical test, we would reject the author's claim. If we made an error in our study (either due to sampling or random chance), this would be a Type I error

Type II Errors

A Type II error, also known as a false negative, is an error that occurs when the null hypothesis is not rejected when it should have been. This means that the null hypothesis is accepted when it is actually false.
Type II errors are more likely to occur when the sample size is small, as there is less power in the statistical test to detect a true difference between the population and the sample. Recall from the previous unit that the power of a statistical test is the probability of correctly rejecting the null hypothesis when it is false.
Like the probability of a Type I error, the probability of a Type II error is influenced by the alpha level and the sample size. A higher alpha level and a larger sample size will result in a lower probability of making a Type II error. It is important to consider the consequences of making a Type II error, as failing to reject the null hypothesis when it is false can lead to incorrect conclusions being drawn about the population being studied.

Example

Let's say an author claims that the mean income for a given area is $45,000.
We sample a group of 50 families and find that the mean income of our sample is $38,000. In performing a statistical test, we would fail to reject the author's claim. If we made an error in our study (either due to sampling or by random chance), this would be a Type II error.

How to Minimize Error in Statistical Studies

(1) Minimizing Error Due to Bias in Sampling

• Select a random sample using a method such as simple random sampling.
• Example: A band director wants to survey the school on their opinions of this year's half-time show. To perform the survey, he numbers each student in the school with a number and uses a random number generator to select 20 students. This is a GOOD example of how to select a random sample.
• Avoid volunteer samples, convenience samples, and other sampling methods that may heavily influence your data in one direction.
• Example: A band director wants to survey the school on their opinions of this year's half-time show. To perform the survey, he chooses the first 20 students who arrive at the Fall Band Concert and asks them if the band's show is satisfactory. This is a BAD example of how to select a random sample since he is only using students who were coming to the band concert (convenience sample).

(2) Minimizing Error Due to Bias in Questioning

• Avoid asking questions in a way that will prompt a certain response.
• Example: If the band director wants to know how the half-time show was, he should ask the question in a way such as "Rate the band's half-time show on a scale of 1-10" as opposed to saying, "Was the band's half-time show good?" The latter way of asking the question will likely influence the students to say "yes." This is known as response bias.
• Avoid having someone ask questions that may influence the response (for example, don't have a police officer ask someone if they have ever broken the speed limit).
• Example: If the band director wants to know how the half-time show was, an anonymous survey would be the best way. If the band director asked the students directly, they may feel more inclined to give it a higher rating (especially if their grade in class could be influenced by the way they feel).

(3) Minimizing Error Due to Confounding Variables

• Use blocking in your experiment to account for any known or suspected confounding variables.
• Example: If the band director wants to know how the student body feels about the half-time show, he may consider blocking by grade. That would ensure an equal amount of responses from each class and make sure that the age of students was not a confounding variable.

Key Terms to Review

• Bias: Bias refers to a systematic error that leads to an incorrect or misleading representation of a population or phenomenon.
• Blocking: Blocking is a technique used in experimental design to reduce the impact of variability by grouping similar experimental units together.
• Confounding Variables: Confounding variables are factors that can obscure or distort the relationship between the independent and dependent variables in a study.
• Measurement Error: Measurement error refers to the difference between the actual value of a quantity and the value obtained through measurement.
• Random Sample: A random sample is a subset of individuals chosen from a larger population, where each individual has an equal chance of being selected.
• Sampling Error: Sampling error refers to the difference between the results obtained from a sample and the actual values that would be obtained if the entire population were measured.
• Simple Random Sampling: Simple random sampling is a method of selecting a subset of individuals from a larger population, where each individual has an equal chance of being chosen.
• Type II Error: A Type II error occurs when a statistical test fails to reject a false null hypothesis.
• Type I Error: A Type I Error occurs when a true null hypothesis is incorrectly rejected.
• Volunteer Samples: Volunteer samples are a type of non-probability sampling method where participants self-select to be part of a study.