This note covers fitting non-linear models to bivariate data (quadratic, cubic, exponential, power and sine models), measuring how well a model fits using the sum of squared residuals and the coefficient of determination \(R^2\), and using a fitted model to make predictions. Many real relationships, such as growth, braking distance and seasonal cycles, are curved, so a straight line is not enough.
Key ideas
Start with a scatter diagram. If the points curve, a straight-line model will be poor. Choose a model from the shape and the context, then fit it using the regression feature on your GDC.
- Quadratic \(y=ax^2+bx+c\): one turning point (for example, the height of a thrown ball over time).
- Cubic \(y=ax^3+bx^2+cx+d\): up to two turning points, or an S-shaped curve.
- Exponential \(y=a\times b^{x}\) (or \(y=ae^{kx}\)): constant percentage growth or decay (populations, cooling, interest).
- Power \(y=ax^{b}\): for example, the period of a pendulum against its length, or area against side length.
- Sine \(y=a\sin(bx+c)+d\): repeating patterns such as tides, daylight hours or temperatures through the year.
Residuals. For each data point, the residual is \(y_i-\hat{y}_i\), the observed value minus the value predicted by the model. A good model has small residuals with no pattern. The regression process chooses the parameters that make the sum of squared residuals, \(SS_{res}=\sum(y_i-\hat{y}_i)^2\), as small as possible. This is called the least-squares method.
Coefficient of determination \(R^2\). Compare the model with the simplest possible model, which always predicts the mean \(\bar{y}\). The total variation in \(y\) is \(SS_{tot}=\sum(y_i-\bar{y})^2\). Then
\[R^2=1-\frac{SS_{res}}{SS_{tot}}\]
- \(R^2\) lies between 0 and 1 for a least-squares fit with a constant term. A value close to 1 means the model accounts for most of the variation in \(y\). For example, \(R^2=0.95\) means that about 95% of the variation in \(y\) is explained by the model, and about 5% is not.
- A larger \(R^2\) means a better fit for the same data. Use it to compare models fitted to the same set of points.
- For a straight-line fit, \(R^2=r^2\), where \(r\) is Pearson's correlation coefficient.
Choosing and using a model. A high \(R^2\) does not prove that the model is right. A linear model can have a high \(R^2\) on data that are clearly curved. Also, adding more parameters (for example a cubic instead of a quadratic) can only keep \(R^2\) the same or increase it, so prefer the simplest model that suits the shape and context. Check the residuals, or the plotted curve, as well as \(R^2\).
Predictions. Substitute \(x\) into the fitted equation. Predicting inside the range of the data (interpolation) is generally reliable. Predicting outside the range (extrapolation) is risky, because the pattern may not continue. Some calculators report \(R^2\) for exponential and power models using transformed (logarithm) values, so compare values from the same type of calculation.
Definitions and formulae
| Term or formula | Meaning / when to use |
|---|---|
| Residual \(y_i-\hat{y}_i\) | Observed value minus the model's predicted value. |
| \(SS_{res}=\sum(y_i-\hat{y}_i)^2\) | Sum of squared residuals; the quantity that least-squares regression minimises. |
| \(SS_{tot}=\sum(y_i-\bar{y})^2\) | Total variation of the \(y\)-values about their mean. |
| \(R^2=1-\dfrac{SS_{res}}{SS_{tot}}\) | Coefficient of determination: proportion of the variation in \(y\) explained by the model. |
| \(y=a\times b^{x}\) | Exponential model; \(b\gt 1\) is growth and \(0\lt b\lt 1\) is decay. |
| \(y=ax^{b}\) | Power model. |
| \(y=a\sin(bx+c)+d\) | Sine model with period \(\frac{2\pi}{b}\) (radians). |
Worked examples
Example 1 The weekly downloads (in thousands) of a new app in weeks 1 to 8 are: 3.1, 4.6, 5.2, 5.0, 5.9, 8.0, 11.9, 18.2. Compare linear, quadratic and cubic models using \(R^2\).
Enter the data in two lists and use each regression feature on the GDC.
Linear: \(y\approx 1.80x-0.379\), with \(R^2\approx 0.778\).
Quadratic: \(y\approx 0.426x^2-2.03x+6.01\), with \(R^2\approx 0.952\).
Cubic: \(y\approx 0.118x^3-1.17x^2+4.07x+0.143\), with \(R^2\approx 0.999\).
The cubic explains about 99.9% of the variation, compared with 77.8% for the line. The data rise, flatten briefly and then rise steeply, which suits a cubic shape better than a parabola.
Answer: cubic is best (\(R^2\approx 0.999\)); quadratic 0.952; linear 0.778.
Example 2 A culture of bacteria is counted each hour, starting at \(t=0\): 52, 71, 98, 131, 182, 247. An exponential model is fitted. (a) Write the model and state \(R^2\). (b) Predict the count at \(t=5.5\) and at \(t=8\), and comment.
(a) The counts grow by a roughly constant factor each hour (\(\frac{71}{52}\approx 1.37\), \(\frac{98}{71}\approx 1.38\)), which suggests exponential growth. The exponential regression feature gives \(N\approx 52.0\times 1.366^{t}\), with \(R^2\approx 0.9998\).
(For comparison, a straight line gives \(R^2\approx 0.952\), which is high but the curve is clearly not straight.)
(b) At \(t=5.5\): \(N=52.05\times 1.3656^{5.5}\approx 289\). This lies inside the data range (0 to 5), so it is interpolation and is fairly reliable.
At \(t=8\): \(N=52.05\times 1.3656^{8}\approx 630\). This is extrapolation. Real cultures run out of food and space, so the growth may not continue at the same rate.
Answer: \(N\approx 52.0\times 1.366^t\), \(R^2\approx 0.9998\). Predictions: about 289 at \(t=5.5\) (reliable) and about 630 at \(t=8\) (uncertain).
Example 3 Two models are fitted to the same 10 data points. Model A has \(SS_{res}=12.4\) and model B has \(SS_{res}=41.9\). The total variation is \(SS_{tot}=310.0\). Find \(R^2\) for each and interpret.
Model A: \(R^2=1-\frac{12.4}{310.0}=1-0.04=0.960\).
Model B: \(R^2=1-\frac{41.9}{310.0}=1-0.1352=0.865\).
Model A explains 96.0% of the variation in \(y\); model B explains 86.5%. Model A fits these data better.
Answer: \(R^2_A=0.960\), \(R^2_B=0.865\); model A fits better.
Common errors
- Saying that a high \(R^2\) proves the model is correct or that one variable causes the other. It only measures fit to these data.
- Choosing the most complicated model just because it has the largest \(R^2\), without thinking about the shape of the data and the context.
- Using a model far outside the data range and treating the answer as reliable.
- Interpreting \(R^2=0.72\) as "72% of the points lie on the curve". It means 72% of the variation in \(y\) is accounted for by the model.
- Forgetting units, or giving the equation without the variables defined (for example, writing \(y=\dots\) when the context uses \(N\) and \(t\)).
Exam tip: When a question asks you to justify a model, refer both to the shape of the scatter diagram or context and to the value of \(R^2\). When it asks for a prediction, state whether it is interpolation or extrapolation.
Check yourself
- A model has \(R^2=0.72\). Explain what this means.
Answer
About 72% of the variation in \(y\) is explained by the model, and about 28% is due to other factors or random variation. - A fit has \(SS_{res}=18\) and \(SS_{tot}=240\). Find \(R^2\).
Answer
\(R^2=1-\frac{18}{240}=1-0.075=0.925\). - Using the model \(N=52.05\times 1.3656^{t}\) from Example 2, predict \(N\) at \(t=7\). Is this interpolation or extrapolation?
Answer
\(N=52.05\times 1.3656^{7}\approx 461\). The data cover \(t=0\) to 5, so \(t=7\) is extrapolation and less reliable. - Model A has \(R^2=0.91\) and model B has \(R^2=0.97\) for the same data. Does B have to be the better model?
Answer
B fits the data more closely, but the best model also needs to suit the shape of the scatter diagram and the context (for example, not predict impossible values). A simpler model with a slightly lower \(R^2\) can be preferable.