Bias/variability trade-offs - Statistics AP Study Notes

Bias refers to the systematic tendency of a sampling method to over- or underestimate a population parameter. A biased estimator consistently misses the true parameter value in one direction. Mathematically, an estimator θ̂ is unbiased if E(θ̂) = θ, where θ is the true parameter.

Variability (or precision) measures how spread out the sampling distribution is. High variability means estimates fluctuate widely between samples; low variability means consistent estimates. We quantify this using standard error: SE = σ/√n for sample means.

The Bias-Variability Trade-off is the fundamental tension in sampling: reducing bias often increases variability, and vice versa. An ideal estimator has both low bias (accurate on average) and low variability (consistent across samples).

Mean Squared Error (MSE) combines both concepts: MSE = Bias² + Variance. This formula shows that total error comes from two sources. A slightly biased estimator with very low variance might have lower MSE than an unbiased but highly variable one.

Key Formula: For sample mean x̄, we have:

• Bias = E(x̄) - μ
• Variance = σ²/n
• Standard Error = σ/√n

Cambridge Key Point: Understanding when to accept small bias for substantially reduced variability is crucial for A-Level questions involving sampling design.

Sample size affects variability (larger n → smaller SE) but doesn't necessarily reduce bias. A biased method remains biased regardless of sample size—a critical distinction for exam questions.

Consider political polling during elections. A pollster could survey 10,000 people using only landline phones (large sample, high precision) or 500 people using random digit dialing including mobiles (smaller sample, more representative). The landline-only approach has low variability (consistent results across repeated polls) but high bias (systematically underrepresents younger voters). The mixed approach has higher variability (more fluctuation between polls) but lower bias (better represents actual electorate).

Medical dosing studies illustrate this trade-off beautifully. Testing a drug on 20 perfectly matched individuals (same age, weight, genetics) produces low variability—very consistent results. However, this creates high bias because the results won't generalize to the diverse patient population. Testing on 20 diverse patients increases variability but reduces bias, making results more applicable to real-world use.

Think of archery: bias is like a sight that's systematically off-target (all arrows cluster away from bullseye), while variability is scatter (arrows spread widely). A skilled archer with a broken sight has low variability but high bias. A novice with a perfect sight has low bias but high variability. The expert with calibrated equipment achieves both—the ideal.

Convenience sampling at university campuses exemplifies extreme bias-variability issues. Surveying students entering one library creates low variability (similar responses from similar people) but massive bias (excludes non-library users, off-campus students, etc.). The method is precisely wrong rather than approximately right.