Not sure which lesson to do first? Find your level and get a plan →

Statistics · Grade 9-12

Residuals

A residual is the difference between an observed data value and the value predicted by a statistical model, calculated as residual=yobserved−ypredicted.


Quick Answer

A residual is the difference between an observed data value and the value predicted by a statistical model, calculated as residual=yobserved−ypredicted. Positive residuals mean the model underestimated; negative residuals mean it overestimated. You compute residuals after fitting a line, to see how far off each prediction was and, by plotting them against x, to check whether a straight line was the right model at all. A good fit leaves residuals that scatter randomly around zero with no shape.

Why This Matters

Residuals are the evidence a model leaves behind. A residual plot can expose a curve that a high correlation hid, a spread that grows with x, or one point the line badly misses. They also answer practical questions, such as whether a house sold above or below its predicted price. Without them, a straight line can look convincing while being the wrong model.

Intuitive Explanation

A weather app predicted a high of 25 °C, and the thermometer reached 23 °C. The residual is observed minus predicted: 23−25=−2. Negative means the prediction was too high; positive would mean it was too low. In a regression, every point has one residual, its vertical gap from the line: points above the line have positive residuals and points below have negative ones. For a least-squares line, the residuals always add up to zero.

Residuals matter most as a group. Plot each residual against x, with a horizontal line at zero. If the line was a good model, the dots scatter randomly above and below zero like static on a screen. If they form a U or an arch, the data curve and the line cuts across the bend. If they fan out as x grows, predictions get less reliable for larger x. Any shape in the residuals is a pattern the line failed to capture.

Small residuals do not prove a good model. The values y=x2 for x=1 to 6 have a correlation of about 0.98 with x, and the least-squares line misses each point by at most about 3.3, yet the residuals run plus, minus, minus, minus, minus, plus: a clear U. A residual also differs from the true error: the error is a point's gap from the unknown population line, while the residual is its gap from the line fitted to this sample.

Think of it like this. When a tailor tries a suit on a customer, the wrinkles that remain are its residuals: where the fabric bunches or pulls shows where the pattern does not match the body. Small random wrinkles are fine, but the same pull along one seam every time means the pattern itself needs changing.

Core idea. A residual is what the model got wrong for one point, y−y^; residuals that scatter randomly around zero mean the model captured the pattern, while any shape in them means it missed something.


When to Use

Compute residuals after fitting a model: for one point when a question asks whether an observation was above or below the prediction and by how much, and for all points when you must decide whether a straight line was appropriate. A residual plot with a curve or a fan shape means the model should change. To summarize overall fit in a single number, use R2 instead.

Tip. Ask: Do you need how far one actual value sits above or below the value a fitted model predicted for it?

How to Recognize It

Before calling a gap a residual, check that a model made a prediction for this point and that you are comparing it with what really happened.

  1. Is there a fitted line or model that gave a predicted value for this individual?

    Yes means a residual can be found. If you are only comparing a value with the average of all the y-values, that gap is a deviation from the mean, the building block of Standard Deviation.

  2. Does the question ask about one point's miss rather than how well the line fits overall?

    Yes is a residual. One summary number for the whole fit, such as the share of variation explained, is R-Squared, which is built from all the squared residuals together.

  3. Did you subtract the prediction from the observed value, in that order?

    Yes gives the right sign: positive when the point sits above the line, negative when below. Reversing the order keeps the size but flips every sign and every conclusion about over- and under-prediction.

  4. Is the unusual point far above or below the line, rather than far out along the x-axis?

    Yes means its residual will be large. A point far out in x can pull the line toward itself and end up with a small residual, so Outlier Detection needs more than residuals alone.

Residuals vs R-Squared (Coefficient of Determination) vs Outlier Detection vs Standard Deviation

Residuals, R-squared, outliers and standard deviation all measure gaps, so they blur together. Ask what the gap is measured from: the fitted line, the whole model's fit, the rest of the data, or the mean of y.

Residuals

Meaning
A residual is one data point's observed y minus the y the model predicted for it.
Key test
One per point, measured vertically from the line, with a sign.
Formula
ei=yi−y^i
Example
Predicted 158 cm, actual 151 cm: the residual is −7 cm, so the line over-predicted.

R-Squared (Coefficient of Determination)

Meaning
R-squared compares the total of the squared residuals with the total squared spread of y around its mean.
Key test
One number for the whole model, built from every residual at once.
Formula
R2=1−∑ei2∑(yi−yˉ)2
Example
Squared residuals total 40 and squared deviations from the mean total 200, so R2=0.80.

Outlier Detection

Meaning
Outlier detection is the process of flagging data points that lie unusually far from the rest of the data.
Key test
Asks whether a point is unusual; a big residual is one warning sign.
Example
A student who studied 1 hour but scored 98 sits far above the line, with a large positive residual.

Standard Deviation

Meaning
Standard deviation measures the typical distance of values from their own mean, without using any line or second variable.
Key test
Uses y−yˉ, not y−y^; no model is involved.
Formula
s=∑(yi−yˉ)2n−1
Example
Heights with mean 150 cm and standard deviation 8 cm, before any line predicts them from age.

Formula & Notation

How to read it: Residuals are denoted ei or ε^i. The observed value is yi, the predicted value is y^i, and ei=yi−y^i.

Worked Examples

Example 1 — Residuals for a training log

Easy

The least-squares line for weeks of training x and push-ups y is y^=10.9+1.7x, fitted to (1, 13), (2, 15), (3, 14), (4, 18), (5, 20). Find every residual and say which week the line fits worst.
Show the steps and the answer
  1. Predictions: y^=12.6,14.3,16.0,17.7,19.4 for weeks 1 to 5.

    Each residual needs the line's value at that point's own x.

  2. Residuals, observed minus predicted: 13−12.6=0.4, 15−14.3=0.7, 14−16.0=−2.0, 18−17.7=0.3, 20−19.4=0.6.

    The order matters: actual first, so a positive residual means the point lies above the line.

  3. Check: 0.4+0.7−2.0+0.3+0.6=0.

    Residuals from a least-squares line always sum to zero, so any other total signals an arithmetic slip.

  4. Find the largest in size: week 3, at −2.0.

    Week 3's 14 push-ups were 2 below what the trend predicted, perhaps an off day; the other weeks miss by under 1.

Answer: Residuals 0.4, 0.7, −2.0, 0.3, 0.6. The line fits week 3 worst: the student did 2 fewer push-ups than predicted.

Takeaway: Compute actual minus predicted for each point, and use the zero sum as a check.

Example 2 — High correlation, wrong model

Standard

A ball rolls down a ramp. Its distance in cm after t=1,2,3,4,5,6 seconds is 1, 4, 9, 16, 25, 36. The least-squares line is y^=7t−9.33 and r≈0.98. Use the residuals to decide whether the line is a good model.
Show the steps and the answer
  1. Predictions: −2.33,4.67,11.67,18.67,25.67,32.67.

    Substitute each t into 7t−9.33.

  2. Residuals: 3.33,−0.67,−2.67,−2.67,−0.67,3.33.

    The largest miss is only about 3.3 cm, so judging by size alone, the line seems close.

  3. Read the signs in order of t: plus, minus, minus, minus, minus, plus.

    Random residuals would mix signs; a U shape means the data bend upward and the line cuts across the bend.

  4. Notice the line's prediction at t=1 is a negative distance, −2.33 cm.

    An impossible value inside the data range confirms the straight line is the wrong form; distance here grows like t2.

Answer: No. Despite r≈0.98, the residuals form a U, so the relationship is curved; the data fit y=t2 exactly.

Takeaway: Judge a line by the pattern of its residuals, not only by their size or by r.

Example 3 — Find the mistake: the sign of a residual

Application

A model predicts that a house will sell for $310,000. It sells for $295,000. Kim writes: "Residual =310,000−295,000=+15,000, so the house sold for more than the model predicted." Find the mistake and correct it.
Show the steps and the answer
  1. Subtract with the actual price first: 295,000−310,000=−15,000.

    Kim subtracted in the wrong order, which flips the sign of the residual.

  2. Interpret the negative sign: the actual price is below the prediction.

    A negative residual means the model over-predicted; this point lies below the regression line.

  3. Compare with Kim's claim: the house sold for $15,000 less than predicted, not more.

    The size of the gap was right; with the wrong sign, a buyer or seller would misread whether the price was a bargain.

Answer: The residual is −15,000 dollars: the house sold $15,000 below its predicted price, so the model over-predicted.

Takeaway: Always compute actual minus predicted; negative means the model guessed too high.

Where You'll See It

  • Predicting video game scores

    You build a rule that predicts a player's score from hours practiced. If the rule predicts 8,000 points but the player actually scores 9,500, the residual is +1,500 — the rule under-predicted that player by 1,500 points.

  • Guessing height from age

    A line predicts a 12-year-old should be 150 cm tall. A specific 12-year-old is 142 cm, so the residual is −8 cm — the line over-predicted, and that leftover gap is the residual.

  • Checking a weather forecast

    The forecast model predicts 25 C, the actual high is 23 C, so the residual is −2 C. Looking at many days of residuals shows whether the forecast tends to run hot or cold.

Common Mistakes

  1. The slip: A residual is the same as predicted minus observed.

    Why it's tempting: Both subtractions give the same size number, so students think the order doesn't matter — but it flips every sign.

    Instead: A residual is always observed minus predicted: residual=yobserved−ypredicted. A positive residual means the actual value is above the prediction.

  2. The slip: If a residual is positive, the model over-predicted.

    Why it's tempting: Positive sounds like 'too much,' so students assume positive means the model went too high.

    Instead: A positive residual means the observed value is higher than the predicted value, so the model UNDER-predicted (the prediction was too low).

  3. The slip: A residual plot that shows a clear curve or fan shape is fine as long as the points are close to the line.

    Why it's tempting: Small residuals feel like a good fit, so students stop looking for patterns.

    Instead: Even small residuals signal trouble if they form a pattern (curve, fan, or trend). Good residuals should look like random scatter around zero with no shape.


Mini Practice

Try these on your own. Tap Reveal when you want to check.

  1. A model predicts y^=3x+5. For the data point (4,20), find the residual. Did the model overestimate or underestimate?

    Hint: Find the predicted value first, then subtract it from the observed value.

  2. Find the residuals for y^=2x+1 at the points (1,4), (2,4), (3,8) and (4,8).

    Hint: Compute each prediction, then do observed minus predicted.

  3. A student's actual score was 72, and the residual for that student is −6. What score did the model predict?

    Hint: Write residual = observed minus predicted and solve.

  4. A linear model gives residuals of 4, 1, −2, −3, −2, 0, 2 for x=1 to 7. What does this pattern say about the model?

    Hint: Picture these residuals plotted against x. Are they scattered randomly?

  5. A model predicts a car's stopping distance as 42 m, but it actually took 47 m. You want to describe how far off the model was for this one car. Should you use the residual or R2?

    Hint: One measure is about a single point; the other is about all points together.

  6. For a point with observed y=15 and predicted y^=19, Noor writes, "The residual is 19−15=4, so the model underestimated." Correct her work.

    Hint: Check the order of the subtraction.

Want the full set?

50 practice questions for this concept — free to try, every one with a complete worked solution showing the why, not just the answer.

Frequently Asked Questions

What is the standard deviation of the residuals, and why do people report it?

It is the typical size of a prediction miss, in the units of yy. For a straight-line regression it is se=∑ei2/(n−2)s_e = \sqrt{\sum e_i^2 / (n - 2)}, dividing by n−2n - 2 because two numbers, the slope and the intercept, were estimated from the data. If a line predicting test scores has se=5s_e = 5 points, its predictions are usually off by around 5 points, which is easier to picture than R2R^2.

What does a fan shape in a residual plot tell me in real life?

It means the misses get bigger as xx grows, so the line's predictions are more precise for small xx than for large xx. Predicting a family's yearly spending from its income often looks like this: low-income families spend close to the prediction, while high-income families vary far more. The line may still show the average trend, but one error range quoted for every income would mislead.

Is a large residual a mistake in the data?

Not necessarily. Most residuals are ordinary variation that the model does not capture: two students who study the same hours rarely get the same score. A large residual is worth checking, because it can come from a typing error, such as a height of 1.5 m entered as 15 m. But if the value is real, it belongs in the data, and it may be telling you something the model misses.

Why plot the residuals when I can already see the scatter plot?

Because the residual plot removes the trend and enlarges what is left. On a scatter plot, a slight bend can hide inside a strong upward trend and look straight. Once the line's predictions are subtracted, the residuals sit around zero, and the same bend shows up plainly as positive residuals at both ends with negative ones in the middle, or the reverse.

Learning Path

← Before

Linear Regression
Residuals

You are here

Next →

Nothing after this yet
You need Linear Regression first, since residuals are measured from its fitted line y^. This concept turns those misses into a diagnostic that tells you whether a line suits the data and which points it fits worst. With residuals you can judge any model, from curved fits to regressions with several predictors, and they feed directly into R2, which compares their total size with the total variation in y.

See Also