Residuals Formula

The Formula

e_i = y_i - \hat{y}_i

When to use: A residual is how much the model got wrong for a specific data point. Positive residual means the actual value was higher than predicted; negative means it was lower. If you plot all residuals, the pattern (or lack thereof) tells you whether the model is appropriate.

Quick Example

Regression predicts a student who studies 5 hours will score \hat{y} = 76. Actual score is y = 82. \text{Residual} = 82 - 76 = +6 The model underpredicted by 6 points.

Notation

e_i is the residual for the i-th observation. The sum of all residuals from a LSRL is always zero: \sum e_i = 0.

What This Formula Means

The difference between an observed value and its predicted value from a regression model: \text{residual} = y - \hat{y} (observed minus predicted).

A residual is how much the model got wrong for a specific data point. Positive residual means the actual value was higher than predicted; negative means it was lower. If you plot all residuals, the pattern (or lack thereof) tells you whether the model is appropriate.

Formal View

e_i = y_i - \hat{y}_i where \hat{y}_i = a + bx_i; for LSRL, \sum_{i=1}^{n} e_i = 0 and \sum_{i=1}^{n} x_i e_i = 0

Worked Examples

Example 1

easy
Given \hat{y} = 2 + 3x, and observed point (4, 15): calculate the residual and interpret whether the model over- or under-predicts.

Solution

  1. 1
    Calculate predicted value: \hat{y} = 2 + 3(4) = 2 + 12 = 14
  2. 2
    Calculate residual: e = y - \hat{y} = 15 - 14 = 1
  3. 3
    Positive residual: actual value (15) is ABOVE the predicted value (14)
  4. 4
    Interpretation: the model under-predicts by 1 unit for this observation

Answer

e = 15 - 14 = 1 (positive). The model under-predicts by 1 unit.
A residual e = y - \hat{y} measures the vertical distance between observed and predicted. Positive residual = point above the line (model under-predicts); negative residual = point below the line (model over-predicts). Residuals should average to zero for a good model.

Example 2

medium
Five observed and predicted values: (y, \hat{y}): (10,8), (15,14), (12,13), (20,19), (8,11). Calculate all residuals, verify they sum to 0, and compute \sum e_i^2.

Common Mistakes

  • Computing residuals as \hat{y} - y instead of y - \hat{y}—the convention is observed minus predicted.
  • Ignoring the residual plot and only looking at r^2—a high r^2 can still come with a terrible model if the relationship is curved.
  • Expecting residuals to all be close to zero—some large residuals are normal; look for patterns, not individual values.

Why This Formula Matters

Residuals are how you check whether your model is appropriate. The regression equation alone doesn't tell you if the model fits well—the residual plot does.

Frequently Asked Questions

What is the Residuals formula?

The difference between an observed value and its predicted value from a regression model: \text{residual} = y - \hat{y} (observed minus predicted).

How do you use the Residuals formula?

A residual is how much the model got wrong for a specific data point. Positive residual means the actual value was higher than predicted; negative means it was lower. If you plot all residuals, the pattern (or lack thereof) tells you whether the model is appropriate.

What do the symbols mean in the Residuals formula?

e_i is the residual for the i-th observation. The sum of all residuals from a LSRL is always zero: \sum e_i = 0.

Why is the Residuals formula important in Math?

Residuals are how you check whether your model is appropriate. The regression equation alone doesn't tell you if the model fits well—the residual plot does.

What do students get wrong about Residuals?

Students compute residuals correctly but don't know how to read residual plots. The key: look for patterns. No pattern = good. Any systematic pattern = problem.

What should I learn before the Residuals formula?

Before studying the Residuals formula, you should understand: linear regression lsrl.

Want the Full Guide?

This formula is covered in depth in our complete guide:

Data Representation, Variability, and Sampling Guide →
← Back to Residuals See All Examples Practice Problems