Measured data never sit exactly on a line. Write one equation per data point, accept that the system has no exact solution, and let least squares choose the coefficients that make the total squared error smallest.
Recognize the problem
Trigger words: line of best fit, regression line, least-squares line, best-fitting parabola, fit a model, predict. A table of (t,y) points and a model with unknown coefficients: y=c0+c1t, y=c0+c1t+c2t2, y=kt.
Set it up
1. ModelWrite the model with its unknown coefficients: y=c0+c1t for a line, add c2t2 for a parabola, drop c0 for a line through the origin.
2. EquationsSubstitute each data point. Each point gives one equation that is linear in the coefficients, even when the model is curved in t.
3. MatrixStack the equations as Ac=y: one row per point, one column per coefficient (a column of 1s, a column of t values, a column of t2 values). y holds the measured values.
4. Normal equationsForm ATA and ATy (normal equations). For a line, ATA=(n∑t∑t∑t2) and ATy=(∑y,∑ty).
5. CheckYou need more points than coefficients, and at least two different t values for a line; otherwise ATA is singular.
Interpret the result. Write the fitted model with the coefficients in place and units on t and y. The slope is the change in y per unit of t. Predict by substituting a new t, with care far outside the data range. The residuals yi−y^i should be small and show no pattern; the sum of their squares is the minimum error. This is regression as least squares.
Worked examples
Example 1 · weekly sales
line
sales in weeks 1 to 4: 2,3,5,6 hundred units; predict week 6
model y=c0+c1t
ATA=(4101030), ATy=(16,47)
∑t=10, ∑t2=30, ∑ty=47
c^0=0.5, c^1=1.4
determinant 120−100=20
y=0.5+1.4t; week 6: about 890 units
0.5+1.4⋅6=8.9 hundred
Example 2 · spring stretch
through the origin
forces 1,2,3 N stretch a spring 2,6,7 cm; model x=kF
no intercept in this model
A=(1,2,3)T: ATA=14, ATx=2+12+21=35
one column, no column of 1s
k^=1435=2.5 cm per N
residuals −0.5,1,−0.5
Example 3 · height of a thrown ball
parabola
heights 1,4,4,0 m at t=0,1,2,3 s; model h=c0+c1t+c2t2
three coefficients, four points
ATA=461461436143698, ATh=(9,12,20)
sums of tk and tkh
c^=(0.95,4.95,−1.75)
solve the 3×3 system
h=0.95+4.95t−1.75t2, squared error 0.05
residuals 0.05,−0.15,0.15,−0.05
See it happen
Picture each data point tied to the line by a vertical spring whose stretch is its residual. Least squares finds the position where the total squared stretch is smallest; there the residuals add to zero and their products with t add to zero, which is what the two normal equations say. Drag the points in the least squares visualizer.
Common mistakes
Measuring the wrong distances
✗measuring residuals horizontally or perpendicular to the line
✓least squares measures vertical errors yi−y^i
Forgetting the column of ones
✗A=(1,2,3,4)T for the model y=c0+c1t
✓A has a column of 1s for c0 and a column of t values for c1
Calling a curved model nonlinear
✗y=c0+c1t+c2t2 needs a nonlinear method
✓it is linear in c0,c1,c2; use one column per power of t
Predicting far outside the data
✗week 50: 0.5+1.4⋅50=70.5 hundred units, stated as fact
✓trust a fit near the data; flag long-range predictions as rough