Imagine each data point tied to a straight rod by a vertical spring, where each spring stores energy that grows with the square of its stretch. Let go, and the rod settles where the total stored energy is smallest, which is exactly the position that makes the sum of squared vertical gaps as small as possible. That resting position is the least-squares regression line. Far-off points stretch their springs most and pull hardest, which is why one outlier can drag the line.
You do not need springs to find it. With means xˉ and yˉ, the slope is b1=∑(xi−xˉ)2∑(xi−xˉ)(yi−yˉ) and the intercept is b0=yˉ−b1xˉ, so the line always passes through the point of averages (xˉ,yˉ). For five weeks of training with push-up counts 13, 15, 14, 18 and 20, this gives y^=10.9+1.7x: about 1.7 more push-ups per week. The gap between an actual value and the line, ei=yi−y^i, is a residual.
Regression describes the data you have; it is not a law. The same arithmetic that predicts a 7-year-old's height well predicts a 40-year-old to be over 3 metres tall, because data on children's growth say nothing about adults. A steep slope from observational data does not show cause either: neighbourhoods with more coffee shops may have higher incomes because wealthy areas attract coffee shops. Regression delivers prediction inside the data and a description of association; extrapolation and causation need more evidence.
Think of it like this. A regression line behaves like a balanced seesaw for the data: it pivots through the point of averages, and its tilt settles where the pushes from points above and below cancel, with points far from the middle having the most leverage. Unlike a seesaw, it weighs only vertical gaps, so swapping x and y gives a different line.
Core idea. Least squares picks the one straight line with the smallest total of squared vertical misses; its slope and intercept summarize how y changes with x in the observed data, for prediction inside that range, not as proof of cause.