Residuals Formula

The difference between an observed value and its predicted value from a regression model: residual = y - ŷ (observed minus predicted).

The Formula

ei=yi−y^i

Ask yourself: Am I taking one point's actual value minus the line's predicted value to measure its individual miss?

Quick example

Regression predicts a student who studies 5 hours will score y^=76. Actual score is y=82. Residual=82−76=+6 The model underpredicted by 6 points.

Notation

ei is the residual for the i-th observation. The sum of all residuals from a LSRL is always zero: ∑ei=0.

What this formula means

The difference between an observed value and its predicted value from a regression model: residual=y−y^ (observed minus predicted).

A residual is like the gap between a weather forecast and the actual temperature. Forecast 20 °C, actual 23 °C: the forecast was 3 degrees too low, a residual of +3. One gap says little, but a forecast that is too low every afternoon reveals a flaw in the model, just as a pattern in a residual plot does.

Formal view

ei=yi−y^i where y^i=a+bxi; for LSRL, ∑i=1nei=0 and ∑i=1nxiei=0

Worked examples

Example 1

easy
Given y^=2+3x, and observed point (4,15): calculate the residual and interpret whether the model over- or under-predicts.
Show solution

Answer

e=15−14=1 (positive). The model under-predicts by 1 unit.

First step

1.
Calculate predicted value: y^=2+3(4)=2+12=14

Full solution

  1. 2.
    Calculate residual: e=y−y^=15−14=1
  2. 3.
    Positive residual: actual value (15) is ABOVE the predicted value (14)
  3. 4.
    Interpretation: the model under-predicts by 1 unit for this observation
A residual e=y−y^ measures the vertical distance between observed and predicted. Positive residual = point above the line (model under-predicts); negative residual = point below the line (model over-predicts). Residuals should average to zero for a good model.

Example 2

medium
Five observed and predicted values: (y,y^): (10,8),(15,14),(12,13),(20,19),(8,11). Calculate all residuals, verify they sum to 0, and compute ∑ei2.

Example 3

medium
Data: (1,3),(2,5),(3,4),(4,8). Fitted line: y^=1.5x+1.5. Compute all residuals.

Common mistakes

  • Computing predicted minus observed - the standard is observed minus predicted, y−y^.
  • Expecting nonzero residuals to sum to something meaningful - for an LSRL the residuals always sum to zero, so use squared residuals to measure total error.
  • Ignoring a curved pattern in the residual plot - a clear curve means a line is the wrong model, even if individual residuals are small.

Why this formula matters

A regression line is only useful if its misses are small and patternless. Residuals measure each miss in real units, such as "4 kg heavier than predicted," which flags unusual cases like an underpriced house or a player outperforming his minutes. A residual plot also exposes a line of the wrong shape: a curved pattern means the predictions are off in a predictable, fixable way.

Frequently asked questions

What is the Residuals formula?

The difference between an observed value and its predicted value from a regression model: residual=y−y^ (observed minus predicted).

How do you use the Residuals formula?

Use residuals when a question asks how far a specific point is from its prediction, whether the model overpredicted or underpredicted, or whether a line is a good model, judged from a residual plot. Compute y−y^, observed minus predicted. If the question asks how far a value is from the overall average, that is a deviation from the mean, y−yˉ. If it asks for one number summarizing all the misses, use r2 or the standard deviation of the residuals.

What do the symbols mean in the Residuals formula?

ei is the residual for the i-th observation. The sum of all residuals from a LSRL is always zero: ∑ei=0.

Why is the Residuals formula important in Math?

A regression line is only useful if its misses are small and patternless. Residuals measure each miss in real units, such as "4 kg heavier than predicted," which flags unusual cases like an underpriced house or a player outperforming his minutes. A residual plot also exposes a line of the wrong shape: a curved pattern means the predictions are off in a predictable, fixable way.

What do students get wrong about Residuals?

The procedure for residuals is the easy part; the trap is computing predicted minus observed. Asking "Am I taking one point's actual value minus the line's predicted value to measure its individual miss?" first is what keeps a correct-looking calculation from being attached to the wrong concept.

What should I learn before the Residuals formula?

Before studying the Residuals formula, you should understand: linear regression lsrl.