Not sure which lesson to do first? Find your level and get a plan →

Statistics · Grade 9-12

Linear Regression

Linear regression is a statistical method for modeling the relationship between a dependent variable and one or more independent variables by fitting a straight line that minimizes the sum of squared vertical distances from data points to the line (least squares method).


Quick Answer

Linear regression is a statistical method for modeling the relationship between a dependent variable and one or more independent variables by fitting a straight line that minimizes the sum of squared vertical distances from data points to the line (least squares method). Use it when paired numerical data follow a roughly straight trend and you want an exact equation, y^=b0+b1x, for predicting one variable from the other. The slope b1 is the predicted change in y for each one-unit increase in x, and the intercept b0 is the predicted y at x=0. Each point's miss from the line is its residual.

Why This Matters

Regression gives everyone who uses the same data the same line, so a prediction such as "a 70-inch adult weighs about 135 pounds on average" can be reproduced and checked. It is the basic tool behind price estimates, demand forecasts and growth charts. Without it, trend lines are drawn by eye and differ from person to person, with no agreed way to measure how badly a line misses.

Intuitive Explanation

Imagine each data point tied to a straight rod by a vertical spring, where each spring stores energy that grows with the square of its stretch. Let go, and the rod settles where the total stored energy is smallest, which is exactly the position that makes the sum of squared vertical gaps as small as possible. That resting position is the least-squares regression line. Far-off points stretch their springs most and pull hardest, which is why one outlier can drag the line.

You do not need springs to find it. With means xˉ and yˉ, the slope is b1=∑(xi−xˉ)(yi−yˉ)∑(xi−xˉ)2 and the intercept is b0=yˉ−b1xˉ, so the line always passes through the point of averages (xˉ,yˉ). For five weeks of training with push-up counts 13, 15, 14, 18 and 20, this gives y^=10.9+1.7x: about 1.7 more push-ups per week. The gap between an actual value and the line, ei=yi−y^i, is a residual.

Regression describes the data you have; it is not a law. The same arithmetic that predicts a 7-year-old's height well predicts a 40-year-old to be over 3 metres tall, because data on children's growth say nothing about adults. A steep slope from observational data does not show cause either: neighbourhoods with more coffee shops may have higher incomes because wealthy areas attract coffee shops. Regression delivers prediction inside the data and a description of association; extrapolation and causation need more evidence.

Think of it like this. A regression line behaves like a balanced seesaw for the data: it pivots through the point of averages, and its tilt settles where the pushes from points above and below cancel, with points far from the middle having the most leverage. Unlike a seesaw, it weighs only vertical gaps, so swapping x and y gives a different line.

Core idea. Least squares picks the one straight line with the smallest total of squared vertical misses; its slope and intercept summarize how y changes with x in the observed data, for prediction inside that range, not as proof of cause.


When to Use

Use linear regression when both variables are numerical, the scatter plot is roughly straight, and you need an exact equation to predict y from x or to state a rate of change with its units. Check the residuals afterward. If the plot curves or the residuals show a pattern, try a curved model; if you only need the strength of the link, use r; if the question is whether x causes y, you need an experiment, not a better fit.

Tip. Ask: Must you compute the least-squares slope and intercept from the data or summary statistics, then interpret them in context?

How to Recognize It

Before running a regression, decide which variable you are predicting and check that a straight-line model suits the data.

  1. Is one variable clearly the response you want to predict from the other?

    Yes: regress that response on the explanatory variable, because swapping the roles gives a different line. If neither is being predicted and you only want how closely they move together, use the Correlation Coefficient.

  2. Are you asked to find the equation from the data or from summaries like xˉ, sx and r?

    Yes is Linear Regression: b1=r sy/sx and b0=yˉ−b1xˉ. If the line is already drawn and you only read a value off the graph, that is Line of Best Fit work.

  3. Does the question ask what the slope or intercept means in context, with units?

    Yes is regression interpretation, such as each extra week of training going with about 1.7 more push-ups. If it asks what share of the variation the line explains, report R-Squared instead.

  4. After fitting, do the residuals scatter randomly around zero with no curve or fan?

    Yes means a straight-line model is reasonable. A curved or fanning residual plot means a line is the wrong model even when r is high; Residuals shows how to read one.

Linear Regression vs Line of Best Fit vs Correlation Coefficient vs R-Squared (Coefficient of Determination)

These four come out of the same calculator screen, so they blur together. Ask whether you need an equation for predicting, a drawn trend line, a signed strength, or a share of variation explained.

Linear Regression

Meaning
Linear regression calculates the line that makes the squared vertical misses as small as possible and uses its slope and intercept to describe and predict.
Key test
A computation from the data, ending in an equation with interpreted coefficients.
Formula
y^=b0+b1x,b1=rsysx
Example
Car price on age: y^=21,000−1200x, so each year of age goes with about $1,200 less.

Line of Best Fit

Meaning
A line of best fit is any trend line drawn through a scatter plot to follow the dots, whether placed by eye or taken from a calculation.
Key test
A line on the graph, used to read off estimates.
Formula
y^=mx+b
Example
Laying a ruler along the cloud of car ages and prices to estimate a 4-year-old car's price.

Correlation Coefficient

Meaning
The correlation coefficient measures how closely paired data follow a straight line and whether it rises or falls, with no units.
Key test
Unchanged when x and y swap roles; the regression slope changes.
Formula
r=b1sxsy
Example
Car age and price: r=−0.9, a strong downward trend in any currency.

R-Squared (Coefficient of Determination)

Meaning
R-squared is the fraction of the variation in y that the fitted line accounts for, a value from 0 to 1.
Key test
Rates the fit; gives no direction and no prediction.
Formula
R2=r2
Example
r=−0.9 gives R2=0.81: age accounts for 81% of the variation in price.

Formula & Notation

How to read it: y^ is the predicted value, b0 is the y-intercept, b1 is the slope, x is the independent variable, and the residual is ei=yi−y^i.

Worked Examples

Example 1 — Fit the least-squares line

Easy

A student records weeks of training x and push-ups in one set y: (1, 13), (2, 15), (3, 14), (4, 18), (5, 20). Find the least-squares regression line and interpret its slope and intercept.
Show the steps and the answer
  1. Means: xˉ=155=3 and yˉ=805=16.

    The slope formula measures every point's gap from the means, and the line must pass through (xˉ,yˉ).

  2. Gaps: x−xˉ=−2,−1,0,1,2 and y−yˉ=−3,−1,−2,2,4. Products: 6,1,0,2,8, sum 17. Squared x gaps: 4,1,0,1,4, sum 10.

    A product is positive when a point is above average on both variables or below on both, so the sum carries the direction and size of the trend.

  3. Slope b1=1710=1.7; intercept b0=16−1.7(3)=10.9.

    The intercept formula b0=yˉ−b1xˉ is what puts the line through the point of averages.

  4. Interpret y^=10.9+1.7x: about 1.7 more push-ups per week of training, from a predicted 10.9 at week 0.

    The slope's units are push-ups per week; the intercept is meaningful here because week 0, the start, sits just below the data range.

Answer: y^=10.9+1.7x: about 1.7 extra push-ups per week, starting from about 11.

Takeaway: Find the means first; the slope is the sum of deviation products over the sum of squared x deviations, and the intercept puts the line through (xˉ,yˉ).

Example 2 — Same equation, unsafe prediction

Standard

Heights of children at ages 2, 4, 6, 8 and 10 give the regression line y^=75.1+6.45x, with height in cm and age in years. Predict the height of a 7-year-old and of a 40-year-old. Which prediction can you trust?
Show the steps and the answer
  1. Age 7: y^=75.1+6.45(7)=75.1+45.15≈120 cm.

    Age 7 lies inside the 2 to 10 range the line was fitted on, so this is interpolation.

  2. Age 40: y^=75.1+6.45(40)=75.1+258=333.1 cm.

    The arithmetic is identical, but 40 is far outside the data, so this is extrapolation.

  3. Reality check: 333 cm is over 3 metres, and people stop growing in their late teens.

    The data contain no ages past 10, so they cannot tell the line that growth slows and stops.

Answer: About 120 cm at age 7, which is reasonable; the 333 cm prediction for age 40 is meaningless extrapolation.

Takeaway: A regression line speaks only for the x-range it was fitted on.

Example 3 — Find the mistake: slope read as cause

Application

For 50 neighbourhoods, a regression of median household income y (in $10,000s) on the number of coffee shops x gives y^=2.1+0.8x. Alex writes: "Each new coffee shop raises a neighbourhood's median income by $8,000." Find the mistake and correct it.
Show the steps and the answer
  1. Check the size: y^(3)=2.1+2.4=4.5 and y^(4)=2.1+3.2=5.3, a gap of 0.8, or $8,000.

    Alex read the slope's size and units correctly, so the error lies in what he says it means.

  2. Identify the data: existing neighbourhoods were observed; nobody opened coffee shops at random to see what happened.

    A regression on observational data describes how neighbourhoods differ, not what happens when one of them changes.

  3. Name alternatives: chains open shops where people already earn more, and a central location could raise both counts.

    Reverse causation or a confounder would produce the same positive slope even if coffee shops had no effect on income.

  4. Rewrite as a comparison: neighbourhoods with one more coffee shop have, on average, a median income about $8,000 higher.

    This keeps the slope's meaning and drops the causal claim the data cannot support.

Answer: Neighbourhoods with one more coffee shop have a median income about $8,000 higher on average; the data do not show that adding a shop raises income.

Takeaway: Read a slope from observational data as a comparison between individuals, not as the effect of changing x.

Where You'll See It

  • Pricing a home by its size

    An agent fits sale price against floor area for 40 recent sales and gets roughly y^=50,000+180x dollars for x square feet. A 1,500 square foot listing is then predicted near $320,000, and the slope says each extra square foot goes with about $180 more.

  • Homework completion and final grades

    Regressing final grade (%) on the percent of homework completed for 120 students might give a slope of 0.3: every extra 10 percentage points of homework goes with a final grade about 3 points higher. That is an association, since motivated students may both finish homework and study more.

  • Forecasting electricity demand

    A utility regresses daily electricity use on outdoor temperature for summer days. If the slope is 2.5 megawatt-hours per °C, a forecast of 35 °C instead of 30 °C tells operators to plan for about 12.5 MWh more demand.

Common Mistakes

  1. The slip: A significant regression slope proves the predictor causes the response.

    Why it's tempting: A fitted model that 'works' feels like it explains why.

    Instead: Regression on observational data shows association and enables prediction, not causation. A confounder can drive the relationship; only a controlled experiment supports causal claims.

  2. The slip: The regression equation is valid for any x-value you plug in.

    Why it's tempting: The equation looks like it works everywhere a number fits.

    Instead: Predictions outside the observed range (extrapolation) are unreliable; the linear pattern may break down beyond the data you actually have.

  3. The slip: A high correlation means the straight line is the right model.

    Why it's tempting: A strong r looks like confirmation that a line fits well.

    Instead: Always check the residual plot. If residuals show a pattern (curve, fan shape), a line is the wrong model even with a high correlation; the relationship may be nonlinear or have non-constant spread.


Mini Practice

Try these on your own. Tap Reveal when you want to check.

  1. A regression of house price y (in $1,000s) on size x (in 100s of square feet) gives y^=12x+50. Predict the price of an 1,800 square foot house.

    Hint: Check the units of x before substituting.

  2. For a data set, xˉ=10, yˉ=50, r=0.8, sx=2 and sy=5. Find the least-squares regression line.

    Hint: The slope is rsysx, and the line passes through (xˉ,yˉ).

  3. The regression of fuel used y (litres) on distance x (km) is y^=0.07x+0.5. A 300 km trip used 22 litres. Find the predicted fuel use and the residual.

    Hint: Residual is observed minus predicted.

  4. For students who play 0 to 20 hours of video games a week, the regression of test score on hours is y^=88−1.5x. Explain the slope, and predict the score for 12 hours.

    Hint: The slope tells you the change in predicted score for one more hour.

  5. A coach wants to predict a sprinter's 100 m time from her stride length. Should she use the correlation coefficient or a regression line?

    Hint: Which one gives an equation you can plug a value into?

  6. The model y^=88−1.5x was fit to students playing 0 to 20 hours of games. Lin predicts that a student who plays 70 hours will score 88−1.5(70)=−17. Correct her.

    Hint: Compare 70 hours with the range of the data.

Want the full set?

50 practice questions for this concept — free to try, every one with a complete worked solution showing the why, not just the answer.

Frequently Asked Questions

Why is it called regression?

The name comes from Francis Galton, who studied the heights of parents and their grown children in the 1880s. Very tall parents tended to have tall children, but not quite as tall as themselves, and very short parents had children closer to average too. He called this regression toward mediocrity, now called regression toward the mean, and the name stuck to the line he used for predicting and then to the whole method.

What does the intercept mean when x=0 makes no sense?

Often nothing. The equation in this lesson, weight=4.5×height−180\text{weight} = 4.5 \times \text{height} - 180, predicts −180-180 pounds for a person 0 inches tall, which is impossible. The intercept is just where the line crosses the vertical axis, needed to place the line correctly where the data are. Interpret it in context only when x=0x = 0 is a sensible value close to the data, such as 0 weeks of training.

Can linear regression handle data that curve?

Sometimes, after a change of variable. 'Linear' refers to the coefficients: the equation adds up terms that are each multiplied by a coefficient, so you can regress yy on x2x^2 or on log⁡x\log x instead of on xx. Data that grow by a steady percentage, like a savings balance or a bacteria count, curve upward, but the logarithms of their yy-values fall close to a straight line, which ordinary regression can fit. The residual plot tells you whether the change worked.

How do I get a regression line from a calculator or spreadsheet?

Put the xx-values in one list or column and the yy-values in another. On a TI-84, choose STAT, then CALC, then LinReg(ax+b); it reports aa, the slope, and bb, the intercept, plus rr if diagnostics are turned on. In Excel or Google Sheets, =SLOPE(y-range, x-range) and =INTERCEPT(y-range, x-range) give the two numbers, or you can add a linear trendline to a scatter chart.

Learning Path

You need Scatter Plot (seeing the cloud), Correlation (direction and strength) and Line of Best Fit (the idea of a trend line). Linear regression makes that line exact with the least-squares formulas and gives a residual for every point. Next, Residuals checks whether a line was the right model, and R-Squared measures what fraction of the variation in y the line accounts for.

See Also