On this concept: Formula

Not sure which lesson to do first? Find your level and get a plan →

Math · Statistics & Probability · Grade 9-12

Inference for Regression

Inference for regression uses a t-test (or confidence interval) on the sample slope to decide whether the TRUE population slope β1 is really different from zero.

t=b−β1,0SEbwhereSEb=s∑(xi−xˉ)2

Quick Answer

Inference for regression uses a t-test (or confidence interval) on the sample slope to decide whether the TRUE population slope β1 is really different from zero. Use it when your regression line comes from a sample and you must judge whether the relationship would hold in the population, or give a range for the true slope. Compute t=b−0SEb with df=n−2, or the interval b±t∗ SEb, after checking that the residuals look linear, independent, evenly spread and roughly normal.

Why This Matters

Any sample of 20 points will show some nonzero slope, even when the two variables are unrelated. Inference for regression tells you whether your slope is too steep to blame on that random tilt. It is how researchers back claims like "more sleep goes with higher grades" and how they put an honest range on the size of the effect, instead of reporting one sample's slope as fact.

Intuitive Explanation

Suppose that in the whole population there is no relationship: the true slope is β1=0. Draw a sample of 22 points and fit a line anyway. Random scatter will tilt it a little up or down. Repeat the sampling many times and the sample slopes form a distribution centered at 0, with spread SEb. The test asks whether your actual slope sits far out in that distribution.

Measure the distance in standard errors: t=b−0SEb. For b=2.3 and SEb=0.8, that gives t=2.875. Because SEb is estimated from the residuals, use the t distribution with df=n−2; two degrees of freedom are used up because the line estimated both a slope and an intercept. For n=22, df=20 and the two-sided p-value is about 0.009, so a slope this steep would be rare if β1 were 0.

The same pieces build a confidence interval, b±t∗ SEb, with t∗=2.086 for 95% and df=20. That gives 2.3±1.67, about 0.63 to 3.97: a range of plausible true slopes that excludes 0, matching the test. All of this relies on the regression conditions: a straight-line pattern, independent observations, residuals with similar spread at every x, and roughly normal residuals.

Writing y^=a+bx for the sample is description, not inference; no test is needed to report the sample's own line. Inference begins when you claim something about the population the sample came from.

Think of it like this. Testing a slope is like checking whether a table is level by rolling a marble across it. A small drift might come from bumps in the wood, the random scatter; a steady roll toward one edge is a real tilt. The t-statistic compares the drift you saw with the drift that bumps alone usually produce.

Core idea. Inference for regression tests whether the population slope β1 differs from zero using a t-test on the sample slope.


When to Use

Use inference for regression when a question asks whether there is evidence of a linear relationship in the population, whether the true slope is nonzero, or for a confidence interval for the slope, often from computer output listing b, SEb, t and a p-value. Use df=n−2. If you only need the sample's equation or a prediction, the LSRL alone is enough. If the question compares two group means rather than a slope, use a two-sample t-test.

Tip. Ask: Am I testing whether the underlying population slope is nonzero (rather than just computing or describing the sample slope)?

How to Recognize It

Before testing a slope, check where the data came from, what the computer output gives you, and whether the residuals meet the conditions for a t procedure.

  1. Are the data a random sample or a randomized experiment, standing in for a larger population?

    If yes, inference makes sense: the sample slope b estimates the population slope β1. If the data already cover every case you care about, such as every school in a small district, the LSRL describes them and a test adds little.

  2. Does the question want a range of plausible values for the true slope rather than a yes-or-no decision?

    If yes, build b±t∗SEb with df=n−2. If it asks whether any linear link exists, test H0:β1=0 instead; a 95% interval that misses 0 matches a rejection at the 5% level.

  3. Does the residual plot show random scatter with even spread, and do the residuals look roughly normal?

    If yes, the t procedures are trustworthy. A curved residual plot means a line is the wrong model, and a fan shape makes the standard error unreliable, so fix the model before testing.

  4. Are you comparing the means of two separate groups rather than a slope?

    If yes, use a two-sample t-test. Regression inference needs a quantitative x and y measured on each case, with the slope as the parameter.

Inference for Regression compared with LSRL, Correlation test, and Two-sample t-test

All four rows use a t statistic or regression output, which makes them easy to blur. Ask which quantity is in question: the population slope (regression inference), only the sample's line (LSRL), the population correlation ρ (correlation test), or two group means (two-sample test).

Inference for Regression

Meaning
Use this when you have a sample regression line and need to test whether the population slope differs from zero (not just describe the sample).
Key test
Am I testing whether the underlying population slope is nonzero (rather than just computing or describing the sample slope)?
Formula
t=b−β1,0SEbwhereSEb=s∑(xi−xˉ)2
Example
From n=22 data points, the sample slope is b=2.3 with standard error SEb=0.8. Test H0:β1=0 at α=0.05.

LSRL

Meaning
Computes the sample slope and line descriptively; inference asks if that slope generalizes.
Key test
Use when you only need the best-fit line for the data in hand.
Formula
b=rsysx
Example
Fitting y^=a+bx to the sample

Correlation test

Meaning
Tests whether ρ=0 (association exists), mathematically equivalent here but framed about r not the slope.
Key test
Use when the question is about correlation rather than the rate of change.
Formula
t=rn−21−r2
Example
Testing if r differs from 0

Two-sample t-test

Meaning
Compares two group MEANS, not a slope; uses df=n1+n2−2 (pooled) vs regression's df=n−2.
Key test
Use when comparing averages of two groups, not a linear trend.
Formula
t=xˉ1−xˉ2SE
Example
Comparing two teaching methods' means

Formula & Notation

t=b−β1,0SEbwhereSEb=s∑(xi−xˉ)2
t=b−β1,0SEb with df=n−2 where SEb=s∑(xi−xˉ)2; CI: b±t∗⋅SEb

How to read it: b = sample slope, β1 = population slope, SEb = standard error of the slope, s = standard deviation of residuals, df=n−2.

Worked Examples

Example 1 — Is the slope significant?

Easy

From n=22 data points, the sample slope is b=2.3 with standard error SEb=0.8. Test H0:β1=0 at α=0.05.
Show the steps and the answer
  1. State H0:β1=0 and Ha:β1≠0.

    The null says there is no linear relationship in the population; the alternative allows a slope in either direction.

  2. Measure the slope in standard errors: t=2.3−00.8=2.875.

    The sample slope sits 2.875 standard errors from the null value of 0.

  3. Use df=n−2=22−2=20, for which the two-sided 5% critical value is t∗=2.086.

    Fitting a line estimates two parameters, the slope and the intercept, which uses up two degrees of freedom.

  4. Compare: 2.875>2.086, and the two-sided p-value is about 0.009, below 0.05, so reject H0.

    A slope this steep would rarely come from random scatter if the true slope were 0.

Answer: Reject H0 — the slope is significantly nonzero

Takeaway: Use t=bSEb with df=n−2, and reject when the slope is many standard errors from 0.

Example 2 — Just describing the line

Standard

You're only asked to write the equation of the best-fit line from the sample data. Do you need a t-test?
Show the steps and the answer
  1. Notice that the question asks only to describe the sample.

    Writing the sample's line makes no claim about a population, so no test is involved.

  2. Build the line from summary statistics: for instance, with b=2.3, xˉ=10 and yˉ=40, a=40−2.3(10)=17, so y^=17+2.3x.

    The slope and the point of averages fix the least-squares line; no standard error is needed.

  3. Keep the t-test for population questions.

    Only a question like 'is there evidence of a relationship?' calls for comparing b with SEb.

Answer: No — just report y^=a+bx

Takeaway: Describing the sample line needs no inference; a claim about β1 does.

Example 3 — Find the mistake: a nonzero slope is not proof

Application

For a sample of n=15 students, a regression of quiz score on minutes of music played while studying gives slope b=0.9 with SEb=0.6. Ivan writes: "The slope is 0.9, not 0, so music and quiz scores are related in the population." Find the mistake and correct it.
Show the steps and the answer
  1. Notice that Ivan judged the slope by its size alone.

    Even if β1=0, a sample slope is almost never exactly 0; what matters is how far 0.9 is from 0 relative to its standard error.

  2. Standardize the slope: t=0.9−00.6=1.5, with df=15−2=13.

    The slope is only 1.5 standard errors from 0.

  3. Compare with t∗=2.160 for a two-sided test at α=0.05: 1.5<2.160, p-value about 0.16.

    Random scatter alone would produce a slope at least this far from 0 about 16% of the time.

  4. Build the 95% interval: 0.9±2.160(0.6)=0.9±1.296, about −0.40 to 2.20.

    The plausible true slopes include 0 and even negative values, so the data can't establish a relationship.

Answer: t=1.5 with df=13 and a p-value of about 0.16: fail to reject H0:β1=0. The sample gives no convincing evidence of a relationship.

Takeaway: A nonzero sample slope is expected by chance; test it against SEb before claiming a population relationship.

Where You'll See It

  • Reading a salt and blood pressure study

    A study of 200 adults reports a slope of 1.8 mmHg of blood pressure per 10 g of daily salt, with SEb=0.5. Then t=3.6 with df=198, the p-value is below 0.001, and the 95% interval is about 0.8 to 2.8 mmHg. Readers learn both that the link is real and roughly how large it is.

  • Does advertising move sales

    A shop with 12 months of data finds sales rise by $3.20 for each extra $1 of advertising, with SEb=1.10. With df=10, t≈2.91 and p≈0.016, so the owner has evidence that the relationship is more than a coincidence of those 12 months.

Common Mistakes

  1. The slip: Treating a nonzero sample slope as proof of a population relationship

    Instead: test b against its standard error before concluding β1≠0.

  2. The slip: Using the wrong degrees of freedom

    Instead: regression inference uses df=n−2, not n−1.

  3. The slip: Forgetting the conditions (linearity, independence, equal spread, normal residuals)

    Instead: the t-test is only valid when the regression assumptions hold.


Mini Practice

Try these on your own. Tap Reveal when you want to check.

  1. From n=20 points, the sample slope is b=1.8 with SEb=0.6. Find the t statistic for H0:β1=0 and its degrees of freedom.

    Hint: Divide the slope by its standard error. Regression uses two fewer than n.

  2. From n=30 points, b=0.5 and SEb=0.4. The two-sided critical value at α=0.05 is 2.048. Is the slope significant?

    Hint: Find t and compare its size with 2.048.

  3. From n=20 points, b=2.5 and SEb=0.5. Using t∗=2.101 (df=18), build a 95% confidence interval for β1. Does it suggest a real relationship?

    Hint: Use b±t∗⋅SEb, then check whether 0 is inside.

  4. A study of 25 points reports t=2.6 for the slope. Which degrees of freedom should be used to find the p-value?

    Hint: Two quantities, slope and intercept, are estimated from the data.

  5. You have exam scores for two separate classes and want to know whether their mean scores differ. Inference for regression or a two-sample t-test?

    Hint: Is there an x variable predicting y, or two groups being compared?

  6. From n=12 points, Ken tests the slope and uses df=11. Correct his degrees of freedom.

    Hint: How many parameters does the regression line estimate?

Want the full set?

50 practice questions for this concept — free to try, every one with a complete worked solution showing the why, not just the answer.

Frequently Asked Questions

Why is it df=n−2 and not n−1?

Degrees of freedom count the independent pieces of information left after estimating what the formula needs. A one-sample tt-test estimates one thing, the mean, so it loses one: n−1n - 1. Regression estimates two things, the intercept and the slope, before it can measure the scatter of the residuals, so it loses two. With only 2 points the line fits perfectly and nothing is left to measure the noise.

What does SEb actually measure?

It estimates how much the sample slope would vary from sample to sample. If you took many random samples of the same size and fitted a line to each, the slopes would differ, and SEb\text{SE}_b is the typical size of that wobble. Its formula, s∑(x−xˉ)2\frac{s}{\sqrt{\sum(x - \bar{x})^2}}, shows how to shrink it: less scatter around the line, more data, or xx values spread over a wider range.

If the test rejects β1=0, does that mean x causes y?

No. The test only says the linear link is unlikely to be a fluke of sampling. Whether it is causal depends on how the data were collected. With observational data, such as ice cream sales and drownings over a summer, a confounder like hot weather can produce a real, significant slope with no cause between the two. Only a randomized experiment, where the researcher sets xx, lets a significant slope support a cause.

Can a slope be statistically significant but too small to matter?

Yes. With a huge sample, SEb\text{SE}_b becomes tiny, so even a slope near zero can give a large tt. A study of 100,000 people might find that each extra hour of weekly screen time goes with 30 seconds less sleep a night, with a p-value below 0.001. The relationship is real but almost irrelevant in daily life. Report the slope with a confidence interval, not just the p-value.

Learning Path

Inference for Regression

You are here

Next →

Nothing after this yet
You need the LSRL, Residuals, the Coefficient of Determination, Hypothesis Testing and Confidence Intervals first. Inference for regression brings them together: the residuals give SEb, and a test and an interval judge the population slope β1. With it you can read regression output in research papers and software, decide whether a trend is real, and move on to multiple regression, where several slopes are tested at once.

See Also