Outliers/influential points - Statistics AP Study Notes

Outliers are data points that deviate significantly from the overall pattern in a scatterplot. In regression analysis, we distinguish between two types: residual outliers (points with unusually large residuals—vertical distance from the regression line) and influential points (points whose removal substantially changes the regression equation).

Leverage measures how far an observation's x-value is from the mean of all x-values. High-leverage points have extreme x-values and potentially influence the regression line. The key formula: residual = observed y − predicted ŷ.

Influential points combine high leverage with being an outlier. The DFBETA and Cook's distance quantify influence: Cook's D > 1 suggests high influence. An influential point pulls the regression line toward itself, dramatically affecting slope and intercept.

Memory Aid (LORIS): Leverage (x-distance), Outlier (y-deviation), Residual (vertical gap), Influential (changes line), Slope affected.

Three scenarios to master:

1. High leverage + follows pattern = NOT influential (reinforces trend)
2. High leverage + doesn't follow pattern = HIGHLY influential (pulls line)
3. Low leverage outlier = minimal influence (line barely moves)

The studentized residual (residual divided by its standard error) helps identify outliers: |studentized residual| > 2 warrants investigation; > 3 is extreme.

Key distinction: All influential points are outliers, but not all outliers are influential. Context matters—always examine scatterplots before removing points. Cambridge examiners expect you to justify whether removal is appropriate, considering both statistical and contextual reasoning.

Consider house price prediction based on square footage. Imagine a dataset of typical suburban homes (1,200–2,500 sq ft, priced £150,000–£400,000). Now add Bill Gates' mansion: 66,000 sq ft, valued at £100 million.

This mansion has extreme leverage (x-value far from x̄) and doesn't follow the pattern (disproportionately expensive per square foot). Including it would tilt the regression line upward, making predictions terrible for normal homes. It's highly influential—removal changes everything.

Analogy: Think of a seesaw (teeter-totter). The regression line is the balance point. Points near the center (low leverage) barely affect balance. A child sitting at the very end (high leverage) can dramatically shift the seesaw—if they're heavier or lighter than expected (outlier status). An influential point is that unexpected heavy child at the seesaw's edge.

Medical research example: Studying blood pressure vs. age in adults (30–65 years). One participant is 85 years old with normal blood pressure for their age. High leverage (x = 85 is extreme), but follows the overall age-BP relationship—NOT influential. However, if that 85-year-old has blood pressure of a 40-year-old (outlier), they become influential.

Agricultural yields: Crop yield vs. fertilizer amount. One farm uses 10× normal fertilizer but gets average yield (equipment malfunction). High leverage + outlier = influential. Removal reveals the true fertilizer-yield relationship for practical farmers.

Real insight: Influential points often represent measurement errors, data entry mistakes, or genuinely different populations. Always investigate before deciding!