What are Regression Assumptions?
Regression assumptions, told as a story about four conditions a weight-to-price scale must meet to hang straight and be trusted.
The teacher set up a scale to predict price from weight.Can this scale's (regression model's) prediction be trusted no matter what?
The scale's (model's) prediction error swings much wider in the heavy range.The error's size changes by weight range — the scale must meet certain conditions.
Regression assumptions are four conditions that keep the scale hanging straight.Only when all four hold can the prediction and its confidence interval be trusted.
When a condition breaks, the data and errors show these patterns.A curved relation, entangled errors, a funnel shape, lopsided errors — each is a sign an assumption has broken.
Break the assumptions and a predicted number still comes out — but it shouldn't be taken at face value.The error range and significance tests are only trustworthy when the assumptions hold.
Regression assumptions are the four conditions — linearity, independence, equal variance (homoscedasticity), and normality — a prediction scale must meet to be trusted. Break them, and a number still comes out, but it shouldn't be taken at face value.
Linear regression needs ① linearity (the relationship between predictors and outcome is linear), ② independence of errors (no autocorrelation), ③ homoscedasticity (constant error variance), and ④ normality of errors. If these break, the coefficient estimates still compute, but broken linearity can bias the coefficients themselves, and broken independence or homoscedasticity can distort standard errors, so confidence intervals and p-values can lead to wrong conclusions too. Broken normality makes confidence intervals and p-values hard to trust in small samples; with large samples, that effect shrinks. Check with residual plots, and if assumptions fail, consider variable transformations, robust standard errors, or a different model.
When grown-ups say it
- Linearity
- The assumption that the relation is straight, not curved. If it breaks, use a nonlinear model or transform the variables.
- Homoscedasticity
- The assumption that the wobble stays the same size across ranges. Breaking it shows up as a funnel-shaped residual plot.
- Autocorrelation
- Yesterday's error entangling with today's. A classic way independence breaks in time-series data. → see it in full
- Residual
- The gap between what the scale predicts and the real value. Its pattern reveals whether an assumption has broken.
- Multicollinearity
- Two variables effectively saying the same thing. A separate problem that muddies how coefficients are read.
- Unbiased Estimator
- A scale dial with no systematic tilt. Some assumptions, like linearity and exogeneity, underpin unbiasedness (normality and equal variance aren't required for it). → see it in full
- Ensemble Methods
- Several operators judging together. Sometimes used as an alternative when assumptions break. → see it in full
- Cross Validation
- Splitting into groups and testing in rotation. Separate from assumption checks, this verifies model performance. → see it in full