On this concept: Formula

Not sure which lesson to do first? Find your level and get a plan →

Math · Statistics & Probability · Grade 9-12

Least Squares Regression Line

The least-squares regression line (LSRL) is the unique line y^=a+bx that minimizes the sum of squared vertical residuals between the data points and the line.

y^=a+bxwhereb=r⋅sysx,a=yˉ−bxˉ

Quick Answer

The least-squares regression line (LSRL) is the unique line y^=a+bx that minimizes the sum of squared vertical residuals between the data points and the line. Use it when a scatter plot of two numeric variables shows a roughly straight-line trend and you want to describe the trend or predict y from x. Compute the slope b=rsysx and the intercept a=yˉ−bxˉ; the line always passes through the point of averages (xˉ,yˉ).

Why This Matters

A scatter plot shows a trend; the LSRL turns it into a usable rule, such as "each extra hour of study goes with 4.8 more points." Its slope gives the rate of change in real units, and the equation gives predictions for new cases. Used carelessly, it extrapolates far beyond the data or is forced onto a curved pattern, and then its predictions are confidently wrong.

Intuitive Explanation

Plot study hours against test scores for a class and the points drift upward without lining up perfectly. Any line you draw misses each point by a vertical distance, the residual y−y^. Least squares chooses the line that makes the sum of the squared misses as small as possible. Squaring makes misses above and below the line count alike, and it punishes one big miss more than several small ones.

The best line has a short recipe. Its slope is b=rsysx: the correlation r sets the direction and how closely the points follow the trend, and the ratio of standard deviations converts that to units of y per unit of x. Its intercept a=yˉ−bxˉ forces the line through the point of averages. If heights average 160 cm with sx=10, weights average 55 kg with sy=8, and r=0.75, then b=0.75×810=0.6 kg per cm.

The line measures only vertical misses because it is built to predict y from x. Predict height from weight instead and you get a different line, not the same one solved for x. The equation is also trustworthy only inside the range of x values in the data: a line fitted to children aged 5 to 12 can't predict an adult's height.

Not every line needs least squares. If two points are known exactly, such as (1,2) and (4,11), the slope 11−24−1=3 fixes the line and there is nothing to minimize. Least squares is for clouds of points that no single line passes through.

Think of it like this. Imagine each data point attached to a stiff rod by a vertical spring whose pull grows with its stretch. Let go, and the rod settles where all the pulls balance: close to most points, tugged hardest by the farthest. That resting position is the least-squares line. Like any rod, it stays straight, so it can't follow a curved pattern.

Core idea. The least-squares regression line is the unique line y^=a+bx minimizing the total squared vertical distance to the data.


When to Use

Use the LSRL when you have paired numeric data, a scatter plot with a roughly linear trend, and a need to predict y from x or state a rate of change. Check the scatter plot first: if the pattern curves, a straight line is the wrong model. If you need only the strength of the relationship, report r or r2. If two points lie exactly on a line with no scatter, the ordinary two-point slope is enough.

Tip. Ask: Am I fitting a single straight line to two-variable numeric data by minimizing squared vertical distances?

How to Recognize It

Before fitting a least-squares line, look at the shape of the scatter plot, which variable you want to predict, and the range of x values the data cover.

  1. Does the scatter plot show a roughly straight band of points, without a clear bend?

    If yes, a straight line is a reasonable model. If the points curve, as with growth that speeds up, a line will miss in a systematic way; the residual plot will show the bend, and a curved or transformed model fits better.

  2. Is one variable the one you want to predict, the response y, with the other as the explanatory variable x?

    If yes, regress y on x. Swapping the roles gives a different line, because the method minimizes vertical misses only. If nothing is being predicted and you only want the strength of the link, report the correlation r.

  3. Is the x value you are predicting for inside the range of the data?

    If yes, the prediction is an interpolation and reasonably trustworthy. If it lies far outside, such as predicting a 40-year-old's height from data on children aged 2 to 12, that is extrapolation, and the line can give nonsense.

  4. Do the points lie exactly on a line, with no scatter at all?

    If yes, you need only the ordinary slope through two points, y2−y1x2−x1. Least squares earns its keep when the points scatter and no single line passes through all of them.

Least Squares Regression Line compared with Correlation r, Slope (algebra), and Residuals

All four rows come from the same scatter of paired data, so they get swapped. Ask what you need: a prediction equation (the LSRL), a number for the strength of the link (correlation), an exact line through two points (slope), or one point's miss from the line (a residual).

Least Squares Regression Line

Meaning
Use this when you have paired numeric data with a roughly linear trend and need a best-fit line to summarize it or predict y.
Key test
Am I fitting a single straight line to two-variable numeric data by minimizing squared vertical distances?
Formula
y^=a+bxwhereb=r⋅sysx,a=yˉ−bxˉ
Example
For a sample, xˉ=160 cm, yˉ=55 kg, sx=10, sy=8, r=0.75. Find the LSRL.

Correlation r

Meaning
Measures the strength and direction of linear association, unitless from −1 to 1; it's not the line itself.
Key test
Use when you want association strength, not a prediction equation.
Formula
b=rsysx
Example
r=0.9 means strong positive, but isn't a slope

Slope (algebra)

Meaning
A line through two exact points; LSRL fits MANY scattered points by minimizing error.
Key test
Use when two points lie exactly on the line.
Formula
m=y2−y1x2−x1
Example
Slope between (1,2) and (4,11)

Residuals

Meaning
The leftover gaps y−y^ the LSRL minimizes, not the line.
Key test
Use when checking how far individual points miss the fitted line.
Formula
ei=yi−y^i
Example
A point 3 above its predicted value

Formula & Notation

y^=a+bxwhereb=r⋅sysx,a=yˉ−bxˉ
y^=a+bx where b=r⋅sysx and a=yˉ−bxˉ; equivalently, b=∑(xi−xˉ)(yi−yˉ)∑(xi−xˉ)2

How to read it: y^ is the predicted value. b is the slope. a is the y-intercept. r is the correlation coefficient. sx,sy are the standard deviations of x and y.

Worked Examples

Example 1 — Predicting weight from height

Easy

For a sample, xˉ=160 cm, yˉ=55 kg, sx=10, sy=8, r=0.75. Find the LSRL.
Show the steps and the answer
  1. Slope: b=rsysx=0.75×810=0.6.

    The ratio sysx turns the unitless r into kilograms per centimeter.

  2. Intercept: a=yˉ−bxˉ=55−0.6(160)=55−96=−41.

    The LSRL always passes through (xˉ,yˉ)=(160,55), which fixes a.

  3. Write the line: y^=−41+0.6x.

    Each extra centimeter of height predicts 0.6 kg more weight.

  4. Check at the mean height: −41+0.6(160)=55.

    The prediction at xˉ equals yˉ, as it must. The intercept of −41 kg has no meaning by itself, since no one is 0 cm tall.

Answer: y^=−41+0.6x

Takeaway: Take the slope from r and the two standard deviations, and the intercept from the point of averages.

Example 2 — Exact two-point line

Standard

You're told a line passes exactly through (1,2) and (4,11) with no scatter. Do you need least squares?
Show the steps and the answer
  1. Notice that both points lie exactly on the line.

    There are no misses to minimize, so least squares has nothing to do.

  2. Slope from the two points: 11−24−1=93=3.

    With exact points, rise over run gives the slope directly.

  3. Intercept: 2=3(1)+c, so c=−1 and the line is y=3x−1.

    Check with the other point: 3(4)−1=11.

Answer: No — just compute slope 11−24−1=3

Takeaway: Least squares is for scattered data; exact points need only the two-point slope.

Example 3 — Find the mistake: using r as the slope

Application

For 30 used cars, age x (years) and price y (thousands of dollars) have xˉ=5, yˉ=14, sx=2, sy=4 and r=−0.8. Noah writes the LSRL as y^=18−0.8x, saying "the slope is just r." Find the mistake and correct it.
Show the steps and the answer
  1. Compute the slope properly: b=rsysx=−0.8×42=−1.6.

    r has no units; multiplying by sysx converts it to thousands of dollars per year.

  2. Recompute the intercept: a=yˉ−bxˉ=14−(−1.6)(5)=14+8=22.

    The line must pass through (xˉ,yˉ)=(5,14).

  3. Write the corrected line: y^=22−1.6x.

    Each extra year of age predicts a price $1,600 lower, twice the drop in Noah's line.

  4. Compare the lines for an 8-year-old car: Noah's predicts 18−0.8(8)=11.6, the correct one 22−1.6(8)=9.2.

    Both lines pass through (5,14), which hides Noah's error near the center; it shows up away from the mean age.

Answer: y^=22−1.6x (price in thousands of dollars)

Takeaway: The slope is rsysx, not r; the two match only when sx=sy.

Where You'll See It

  • Estimating a home's price from its size

    Real-estate sites fit lines such as y^=50+0.2x (price in thousands of dollars, size in square feet) to recent sales nearby. The slope says each extra square foot adds about $200 to the predicted price, and the line gives a first estimate for a newly listed house.

  • Budgeting fuel for delivery routes

    A delivery company fits fuel used against distance driven for its vans. If the LSRL is y^=1.5+0.09x liters, a 200 km route is predicted to use 1.5+0.09(200)=19.5 liters. The prediction is safe for new routes only if they fall within the range of distances in the data.

  • Reporting a warming rate

    Climate scientists fit least-squares lines to yearly global temperature records to report a warming rate. In recent decades the fitted slope is roughly 0.2 °C per decade; that single number is the slope of the regression line through the yearly points.

Common Mistakes

  1. The slip: Minimizing perpendicular or horizontal distances

    Instead: LSRL minimizes squared VERTICAL distances (y residuals) only.

  2. The slip: Confusing the slope b with the correlation r

    Instead: they relate by b=rsysx; b has units, r does not.

  3. The slip: Extrapolating far outside the data's x-range

    Instead: the line is only trustworthy across the observed x values.


Mini Practice

Try these on your own. Tap Reveal when you want to check.

  1. The LSRL for test score y^ and hours studied x is y^=50+6x. Predict the score for 4 hours of study and explain what the slope means.

    Hint: Substitute x=4. The slope is the change in y^ per hour.

  2. For a sample, xˉ=10, yˉ=40, sx=2, sy=5 and r=0.8. Find the LSRL.

    Hint: Use b=rsysx, then make the line pass through (xˉ,yˉ).

  3. Find the least squares regression line for the points (1,2), (2,4), (3,5), (4,7).

    Hint: Use b=∑(x−xˉ)(y−yˉ)∑(x−xˉ)2 with xˉ=2.5 and yˉ=4.5.

  4. The LSRL for a data set is y^=12+3x, and the mean of the x values is xˉ=4. Find yˉ without seeing the data.

    Hint: Every least squares line passes through one special point.

  5. You want one number between −1 and 1 that describes how strongly study time and score move together in a straight line. You do not need predictions. LSRL or correlation r?

    Hint: Which tool gives a unitless strength, and which gives an equation?

  6. Using y^=50+6x, fit from students who studied 0 to 5 hours, Ravi predicts a student who studies 15 hours will score 140 on a 100-point test. Correct Ravi.

    Hint: Check the range of x values the line was built from.

Want the full set?

50 practice questions for this concept — free to try, every one with a complete worked solution showing the why, not just the answer.

Frequently Asked Questions

Why square the residuals instead of using their absolute values?

Squaring gives a smooth function whose minimum can be found with algebra, which leads to the neat formulas b=rsysxb = r\frac{s_y}{s_x} and a=yˉ−bxˉa = \bar{y} - b\bar{x} and always a single best line. Minimizing absolute values also works and resists outliers better, but it can have several equally good lines and needs a computer search. The price of squaring is that one far-off point pulls the line strongly.

Why does the line always pass through (xˉ,yˉ)?

It follows from the intercept formula. Put x=xˉx = \bar{x} into y^=a+bx\hat{y} = a + bx with a=yˉ−bxˉa = \bar{y} - b\bar{x}, and you get y^=yˉ−bxˉ+bxˉ=yˉ\hat{y} = \bar{y} - b\bar{x} + b\bar{x} = \bar{y}. In words, a case with an average xx is predicted to have an average yy. This gives a quick check on any line you compute: plug in xˉ\bar{x}, and you must get yˉ\bar{y} back.

Is the line for predicting x from y just the same line solved for x?

No, unless the points lie perfectly on a line. Regressing yy on xx minimizes vertical misses; regressing xx on yy minimizes horizontal ones, so the two lines differ. Both pass through (xˉ,yˉ)(\bar{x}, \bar{y}), but drawn on the same axes the first has slope rsysxr\frac{s_y}{s_x} and the second the steeper slope syr sx\frac{s_y}{r\,s_x}. They match only when r=±1r = \pm 1.

What does the word regression have to do with going backwards?

The name comes from Francis Galton, who in the 1880s compared the heights of parents with those of their adult children. Very tall parents tended to have children who were tall but closer to average, and very short parents had children closer to average too. He called this regression toward the mean. The line he used to show it kept the name, although today regression means any line or curve fitted for prediction.

Learning Path

Least Squares Regression Line

You are here

You need Correlation, Scatter Plots, the Mean and Standard Deviation first, because the slope is rsysx and the line passes through (xˉ,yˉ). The LSRL adds one best line for prediction and an interpretable slope. Next, Residuals show how far each point misses the line, the Coefficient of Determination measures how much variation it explains, and Inference for Regression asks whether the slope is real or chance.

See Also