Learn
A4.3.1 | LINEAR REGRESSION (HL ONLY)
01 | PREDICT A NUMBER, NOT A CATEGORY
A delivery service wants to estimate how many minutes a journey will take before a van leaves. Predicting “late” or “on time” would choose a category. Predicting 14.6 minutes estimates a numerical quantity.
Linear regression is a supervised learning approach that fits a linear relationship between input variables and a numerical response. It can then use that relationship to predict an outcome for a new input.
This lesson begins with one input: journey distance. You will build the meaning of the equation, see how a line is fitted and judge its fit without assuming that every prediction is correct. All delivery values are fictional teaching data.
02 | WHAT DOES CONTINUOUS MEAN?
A continuous outcome represents a measured quantity that can, in principle, take values between other values. Duration, temperature and mass are examples. A journey could take 11 minutes, 11.5 minutes or 11.53 minutes, even if the recording system rounds it.
The display format does not determine the task. A duration recorded as an integer number of minutes is still a rounded measurement. A numeric postcode or category code is not automatically a meaningful continuous target.
Regression estimates quantities; classification predicts discrete categories. Linear regression produces numeric predictions and does not automatically constrain them to physically valid values such as positive durations. Check predictions in context.
03 | NAME THE INPUT AND THE RESPONSE
The independent variable, also called the predictor, is the input used to make a prediction. Here it is distance x, measured in kilometres. The dependent variable, or response, is the outcome being estimated: duration y, measured in minutes.
These names describe roles in the model. Calling distance independent does not prove that we experimentally controlled it or that distance alone causes every change in duration.
During supervised training, historical examples contain both known inputs and observed outcomes. At prediction time, provide the new distance; the model calculates an estimated duration without seeing that journey’s eventual outcome.
04 | BEGIN WITH PAIRED OBSERVATIONS
| Journey | Distance x (km) | Observed duration y (min) |
|---|---|---|
| A | 1 | 7 |
| B | 2 | 8 |
| C | 3 | 11 |
| D | 4 | 12 |
| E | 5 | 17 |
Each row pairs the distance of one journey with its actual recorded duration. Longer journeys generally take longer here, although the increases are not identical. Stops, traffic and other factors can affect duration too.
Plot distance on the horizontal x-axis and duration on the vertical y-axis. Each row becomes one observed point. A fitted line summarises a pattern in those points; it does not join every observation in sequence.
This tiny dataset is for transparent calculations, not a sufficient operational evaluation of a delivery service.
05 | READ THE EQUATION ONE PART AT A TIME
For simple linear regression with one input, write ŷ = mx + b. The symbol ŷ, read “y-hat”, is the predicted response. The actual observed response is y.
The slope m controls how steeply the prediction changes as x increases. The intercept b sets the prediction at x = 0. These learned quantities are the model’s parameters.
You may also see ŷ = β₀ + β₁x. Here β₀ is the intercept and β₁ is the slope. The notation differs but the roles are the same. Keep the input, actual outcome and predicted outcome distinct.
06 | THE SLOPE IS A CHANGE PER INPUT UNIT
Consider the candidate rule ŷ = 2x + 5. Its slope is 2 minutes per kilometre. When distance increases by 1 km, the predicted duration increases by 2 minutes.
For a change of 3 km, the predicted increase is 2 × 3 = 6 minutes. The slope describes a change, not the entire prediction. At x = 3, you must still include the intercept.
The sign also matters. A positive slope means the fitted prediction increases with x; a negative slope means it decreases; a zero slope means the prediction does not vary with that input. A negative slope is possible in other tasks, such as a fitted relationship between altitude and temperature over an appropriate range.
07 | THE INTERCEPT LOCATES THE LINE
In ŷ = 2x + 5, the intercept is 5 minutes. Substituting x = 0 gives ŷ = 5. Geometrically, it is where the line crosses the vertical axis.
One possible interpretation is a fixed component of journey time, but this dataset does not prove that interpretation. Our observed distances range from 1 to 5 km; a zero-distance delivery is not represented.
An intercept can be useful for positioning the line even when x = 0 is outside the meaningful or observed range. Do not claim it is a real measured waiting time merely because the fitted value is positive.
08 | MAKE A PREDICTION BY SUBSTITUTION
| Step | Working for a 4.5 km journey |
|---|---|
| Write the rule | ŷ = 2x + 5 |
| Insert the known input | ŷ = 2 × 4.5 + 5 |
| Multiply | ŷ = 9 + 5 |
| Add and interpret | ŷ = 14 minutes, a prediction rather than a known outcome |
The model takes a known input and produces an estimated numerical response. It does not look up an identical journey or require the new outcome to be supplied.
Multiplication happens before addition. Omitting the intercept would give 9 minutes, which belongs to a different model. Include units so that the result has a clear meaning.
09 | PREDICTIONS DO NOT NEED TO LAND ON THE POINTS
At distance 2 km, the candidate rule predicts 2 × 2 + 5 = 9 minutes. Journey B actually took 8 minutes. That difference is normal evidence to examine, not a reason to replace the observed duration with the prediction.
A single straight line usually cannot pass through every noisy observation. The aim is to capture a useful overall relationship while allowing unexplained variation.
A line that passes through every training example is not automatically reliable on new examples. Judge performance with independent data and context, rather than demanding perfect training fit.
10 | RESIDUALS SHOW WHICH WAY A PREDICTION MISSES
A residual is observed minus predicted: e = y − ŷ. At journey B, e = 8 − 9 = −1 minute. The prediction is 1 minute too high, so the residual is negative.
At journey E, the rule predicts 15 minutes but the actual duration is 17. Its residual is 17 − 15 = +2 minutes: the prediction is too low.
On a scatter plot, the residual is the vertical separation between an observed point and the line at the same x-coordinate. It is not the shortest perpendicular distance to the line.
11 | SQUARE THE RESIDUALS TO COMPARE LINES
| x | Observed y | Predicted 2x + 5 | Residual y − ŷ | Squared residual |
|---|---|---|---|---|
| 1 | 7 | 7 | 0 | 0 |
| 2 | 8 | 9 | -1 | 1 |
| 3 | 11 | 11 | 0 | 0 |
| 4 | 12 | 13 | -1 | 1 |
| 5 | 17 | 15 | 2 | 4 |
The signed residuals add to zero here: 0 − 1 + 0 − 1 + 2 = 0. That does not mean there is no error, because positive and negative misses cancel.
Squaring makes each contribution nonnegative and penalises larger misses more strongly. The sum of squared errors, SSE, is 0 + 1 + 0 + 1 + 4 = 6 min². It is also called the residual sum of squares.
SSE is a combined measure for these particular observations. It is not a percentage and is not expressed in ordinary minutes. Comparing SSE between different-sized datasets needs care.
12 | WHAT DOES TRAINING THE LINE MEAN?
Ordinary least squares selects the slope and intercept that minimise the sum of squared residuals on the training data. The software adjusts the line’s parameters to reduce that objective.
For the five supplied observations, the least-squares line with an intercept is ŷ = 2.4x + 3.8. Its SSE is 4.4, lower than 6 for the candidate 2x + 5 line.
Training learns the parameters from paired examples. Inference then applies the fitted equation to new inputs. Some implementations solve the least-squares problem directly; others use iterative optimisation. You do not need to assume the model repeatedly guesses random lines.
13 | FOLLOW THE FITTED LINE THROUGH THE DATA
| x | Observed y | Predicted 2.4x + 3.8 | Residual | Squared residual |
|---|---|---|---|---|
| 1 | 7 | 6.2 | 0.8 | 0.64 |
| 2 | 8 | 8.6 | -0.6 | 0.36 |
| 3 | 11 | 11 | 0 | 0 |
| 4 | 12 | 13.4 | -1.4 | 1.96 |
| 5 | 17 | 15.8 | 1.2 | 1.44 |
The squared residuals sum to 0.64 + 0.36 + 0 + 1.96 + 1.44 = 4.4. The best line under this objective still misses individual observations.
At x = 4.5, the fitted prediction is 2.4 × 4.5 + 3.8 = 14.6 minutes. Its slope indicates an extra 2.4 predicted minutes for each additional kilometre within this fitted relationship.
The coefficients are supported by this small training set. They are not a guarantee that all future journeys take exactly that time.
14 | ADJUST THE LINE AND WATCH THE RESIDUALS
Change the slope and intercept, or load the least-squares fit and the mean baseline. The green line shows predictions, pink dots show observed durations and dashed pink segments show vertical residuals. The outlined marker is your new prediction.
The control ranges are chosen to keep this graph readable; linear regression itself can use negative slopes and intercepts.
Try the initial 2x + 5 line, the least-squares fit, and then the mean baseline. The displayed SSE and R² are calculated on these five teaching observations, not an independent test set. Try predicting at 7 km and notice the extrapolation warning.
15 | COMPARE WITH A SIMPLE BASELINE
Before judging how useful a fitted model is, compare it with an uninformative reference. The mean observed duration in this dataset is (7 + 8 + 11 + 12 + 17) / 5 = 11 minutes.
A constant line ŷ = 11 ignores distance and always predicts the mean. The squared deviations from this mean are 16, 9, 0, 1 and 36, summing to 62. This is the total sum of squares, SST.
R² uses this mean reference for the same evaluated outcomes. In a real held-out comparison, a deployable mean-only baseline would be learned from training data; the evaluation-set mean used in the R² denominator is a mathematical reference, not permission to learn predictions from test outcomes.
16 | R²: HOW MUCH BETTER THAN THE MEAN REFERENCE?
The coefficient of determination, R², compares residual squared error with variation around the evaluated outcomes’ mean: R² = 1 − SSE / SST, provided SST is nonzero.
For the fitted line, R² = 1 − 4.4 / 62 ≈ 0.929. On these five training observations, the line accounts for approximately 92.9% of their squared variation relative to the mean reference.
For the candidate 2x + 5 line, R² = 1 − 6 / 62 ≈ 0.903. The mean line has R² = 1 − 62 / 62 = 0. A perfect prediction of every evaluated value would have SSE = 0 and R² = 1.
17 | WHAT R² DOES NOT TELL YOU
R² = 0.929 does not mean 92.9% of predictions are exactly correct, or that every prediction is within 7.1% of the answer. It is not classification accuracy and does not establish causation.
An evaluated model can have a negative R² if its SSE exceeds SST: it is worse under this comparison than the evaluated mean reference. A negative slope does not automatically produce negative R²; slope direction and goodness of fit are different ideas.
If every evaluated outcome is identical, SST = 0 and this formula has a zero denominator. The ordinary formula is not defined; libraries may apply a special convention. R² also does not give an individual prediction’s uncertainty or error in minutes.
18 | LOOK AT ERRORS IN USEFUL UNITS TOO
For the fitted line, absolute residuals are 0.8, 0.6, 0, 1.4 and 1.2 minutes. Their average is 0.8 minutes, the mean absolute error, MAE, on these observations.
MAE answers a different question from R². It describes average absolute miss in the target’s units, while R² compares squared error with the evaluated mean reference. Both need the evaluation context to be meaningful.
A high R² can coexist with errors too large for a particular application. Inspect individual errors, relevant groups and rare cases rather than relying on a single headline number.
19 | INTERPOLATION AND EXTRAPOLATION
The observed distances span 1 to 5 km. Predicting at 4.5 km is interpolation, because it is inside that observed range. Predicting at 7 km is extrapolation, because it is outside it.
The fitted equation can calculate 2.4 × 7 + 3.8 = 20.6 minutes, but calculating a number is not evidence that the relationship continues there. Longer journeys might involve a motorway, different stops or different operating conditions.
Interpolation is not guaranteed correct either. A new journey can differ in important unrecorded ways. Treat both as estimates, with particular caution when the model is applied beyond the training range or population.
20 | CHECK WHETHER A STRAIGHT LINE IS REASONABLE
A straight line represents a constant predicted change per unit increase in the input. If duration grows rapidly after a congestion threshold, one line may miss a systematic pattern.
Inspect a scatter plot and residuals. Curved residual patterns can suggest that a straight-line relationship is inadequate. A growing spread of residuals can indicate that errors are less stable for some input ranges. Neither pattern is resolved simply by quoting a high training R².
Least squares is sensitive to large squared residuals, and unusual high-leverage inputs can influence a fitted line. Investigate context before removing observations: a genuine long journey is not an error merely because it changes the fit.
21 | MORE THAN ONE PREDICTOR
Simple linear regression uses one predictor. Multiple linear regression can use several, for example ŷ = b₀ + b₁ × distance + b₂ × stops. The output remains a numerical response.
In such a model, the distance coefficient describes the change in predicted duration for an extra kilometre while holding the other included predictor values fixed. It is not automatically the same as the slope fitted using distance alone.
Additional inputs can capture useful differences but also introduce redundancy, leakage or overfitting. Apply the earlier lessons on cleaning, selection and validation. A more complicated equation is not automatically a better deployed predictor.
22 | TRAIN, EVALUATE, THEN PREDICT
Collect representative paired examples and define when the prediction is needed. Split data appropriately, fitting any learned cleaning or scaling on training data only. Fit the regression parameters on training examples.
Use validation procedures for model and pipeline choices, then evaluate the chosen pipeline on independent test examples. Report which set each metric describes. Training R² alone does not demonstrate future performance.
At deployment, supply the same input meanings, units and preprocessing used by the fitted pipeline. Monitor errors and changed conditions. A 4.5-mile input supplied as 4.5 kilometres silently changes the prediction’s meaning.
23 | CHECK YOUR INTERPRETATION
Choose the concept that best explains each statement. Focus on the meanings of the equation and evaluation results rather than memorising isolated definitions.
24 | EXPLAIN THE PROCESS WITH CONNECTED REASONS
A complete explanation can follow the chain: paired input and response examples reveal a relationship; least-squares fitting chooses a line that minimises training squared residuals; substituting a new input produces a continuous numerical prediction; independent evaluation checks how useful that fitted relationship is.
Use connectives to make the reasoning clear: “The slope is 2.4 minutes per kilometre, therefore increasing distance by 1 km raises the predicted duration by 2.4 minutes. Actual journeys vary because distance does not capture every influence.”
Link fit and limits: “R² compares squared residuals with variation around the mean, so a value near 1 indicates a close fit on the evaluated data. However, training fit alone does not establish accurate predictions for new journeys.”
Further reading: ordinary least-squares fitting and R² interpretation and edge cases.