S1.4 Correlation and regression
⤓ Presentation⤓ PowerPointPresentation: opens offline in any browser, simulations live. PowerPoint: editable.
Statistics 1 (AS). Hypothesis tests on a correlation coefficient and rank correlation are not part of S1; means and standard deviations of a single variable are on the S1.2 page.
What this page covers
7 points
- Scatter diagrams and the language of correlation: positive, negative, strong, weak, none
- Calculating \(S_{xx}\), \(S_{yy}\) and \(S_{xy}\) from raw data or from summary statistics
- The product moment correlation coefficient \(r\): calculating it, interpreting it in context, and its limitations
- The least-squares regression line \(y=a+bx\): what "least squares" means and how to find \(a\) and \(b\)
- Explanatory and response variables, and interpreting \(a\) and \(b\) in context
- Interpolation and extrapolation, and how reliable an estimate is
- Coding: the effect on \(r\), on the summary statistics and on the regression line
In one minute
6 points
- Correlation and regression measure and use a straight-line relationship between two variables, and three summary numbers, \(S_{xx}\), \(S_{yy}\) and \(S_{xy}\), carry the whole topic.
- \(S_{xx}=\sum x^2-\dfrac{(\sum x)^2}{n}\), \(S_{yy}=\sum y^2-\dfrac{(\sum y)^2}{n}\), \(S_{xy}=\sum xy-\dfrac{(\sum x)(\sum y)}{n}\).
- \(r=\dfrac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}\), always between −1 and 1. It measures linear association only, and correlation does not prove cause.
- Regression line of \(y\) on \(x\): \(b=\dfrac{S_{xy}}{S_{xx}}\), \(a=\bar y-b\bar x\). It passes through \((\bar x,\bar y)\) and minimises the sum of the squared vertical distances.
- \(b\) is the change in \(y\) for each extra unit of \(x\); \(a\) is the value of \(y\) when \(x=0\) (often outside the data, so treat it with care).
- Interpolation is reliable when \(r\) is close to ±1; extrapolation is not. Coding \(u=(x-p)/q\) does not change \(r\) (for \(q>0\)).
Contents
15 sections
- What does a scatter diagram show?
- How do you calculate Sxx, Syy and Sxy?
- What does the product moment correlation coefficient measure?
- How do you find the least-squares regression line?
- Which variable is which, and what do a and b mean?
- When can you trust an estimate from the line?
- How does coding affect correlation and regression?
- What are the limitations of the correlation coefficient?
- How do the pieces fit together in a full question?
- Exam tips for S1.4
- Common misconceptions
- How this topic connects
- Key formulae
- Check your understanding
- Specification checklist
What does a scatter diagram show?
With positive correlation, as \(x\) increases \(y\) tends to increase: the points rise from left to right. With negative correlation they fall from left to right. With no correlation there is no straight-line pattern. The closer the points lie to a straight line, the stronger the (linear) correlation. In context, say what happens to one variable as the other increases: "days with more sunshine tend to generate more energy".
Writing just "positive correlation" when asked to interpret a diagram in context. Name the variables: "as the hours of sunshine increase, the energy generated tends to increase".
How do you calculate Sxx, Syy and Sxy?
\(S_{xx}\) and \(S_{yy}\) are never negative. \(S_{xy}\) is positive for positive correlation and negative for negative correlation; it carries the sign of everything that follows. Why? For a point above-right of the mean point, both \((x-\bar x)\) and \((y-\bar y)\) are positive, so its product is positive; points below-left also give positive products. A rising pattern therefore makes \(S_{xy}\) positive.
In most questions you are given \(\sum x\), \(\sum y\), \(\sum x^2\), \(\sum y^2\) and \(\sum xy\): substitute into the right-hand forms. From raw data, use your calculator's statistics mode, but still write the formula with the numbers in it, and keep full calculator values of \(S_{xx}\), \(S_{yy}\) and \(S_{xy}\): rounding early can change the third significant figure of \(r\).
The solar-panel data used in the next three sections: hours of sunshine on 8 days and the energy a domestic solar panel generated on each of those days.
| hours of sunshine, \(x\) | 2.5 | 3.0 | 4.5 | 5.0 | 6.5 | 7.0 | 8.5 | 9.5 |
|---|---|---|---|---|---|---|---|---|
| energy, \(y\) (kWh) | 3.1 | 4.4 | 5.2 | 6.9 | 7.8 | 9.6 | 10.3 | 12.1 |
For the solar-panel data, \(\sum x=46.5\), \(\sum y=59.4\), \(\sum x^2=314.25\), \(\sum y^2=509.12\) and \(\sum xy=399.25\). Calculate \(S_{xx}\), \(S_{yy}\) and \(S_{xy}\).
\(S_{xx}=314.25-\dfrac{46.5^2}{8}=314.25-270.28125=\) 43.96875
\(S_{yy}=509.12-\dfrac{59.4^2}{8}=509.12-441.045=\) 68.075
\(S_{xy}=399.25-\dfrac{46.5\times59.4}{8}=399.25-345.2625=\) 53.9875, positive, matching the rising pattern of the points.
What does the product moment correlation coefficient measure?
| Value of \(r\) | Meaning |
|---|---|
| \(r=1\) | perfect positive linear correlation: every point on a rising straight line |
| close to 1 (e.g. 0.9) | strong positive linear correlation |
| around 0.4 to 0.6 | moderate / weak positive linear correlation |
| close to 0 | little or no linear correlation (there may still be a curved relationship) |
| close to −1 | strong negative linear correlation |
| \(r=-1\) | perfect negative linear correlation: every point on a falling straight line |
Using the results of Worked example 1, calculate the product moment correlation coefficient between hours of sunshine and energy generated, and interpret it in context.
\(r=\dfrac{53.9875}{\sqrt{43.96875\times68.075}}=0.98680\ldots=\) 0.987
Strong positive linear correlation: days with more hours of sunshine tend to generate more energy.
An interpretation of \(r\) needs the direction (positive/negative), the strength (strong/weak) and the context (both variables named). "Strong" on its own, or a bare number, is not an interpretation.
How do you find the least-squares regression line?
For each point, the residual is its vertical distance from the line: observed \(y\) − value of \(y\) on the line. The least-squares regression line of \(y\) on \(x\) is the straight line that makes the sum of the squares of the residuals as small as possible. The distances are measured parallel to the \(y\)-axis because \(y\) is the variable being predicted.
Using the results of Worked example 1, find the equation of the regression line of \(y\) on \(x\) for the solar-panel data, giving \(a\) and \(b\) to 3 significant figures.
Gradient: \(b=\dfrac{53.9875}{43.96875}=1.2279\ldots\)
Means: \(\bar x=\dfrac{46.5}{8}=5.8125\), \(\bar y=\dfrac{59.4}{8}=7.425\).
Intercept, using the unrounded \(b\): \(a=7.425-1.2279\ldots\times5.8125=0.2881\)
\(y=0.288+1.23x\)
Rounding \(b\) to 1.23 before finding \(a\): \(7.425-1.23\times5.8125=0.276\), not 0.288. Keep the full value of \(b\) in your calculator for the next step.
Which variable is which, and what do a and b mean?
Ask which variable could affect the other: hours of sunshine can affect the energy generated, not the other way round. The regression line of \(y\) on \(x\) is used to estimate \(y\) for a given \(x\); it is not the right line for estimating \(x\) from \(y\).
- Interpreting \(b\): "for each extra (unit of \(x\)), (the response) increases/decreases by about \(b\) (units of \(y\))".
- Interpreting \(a\): the value of \(y\) when \(x=0\), meaningful only if \(x=0\) makes sense and is not far outside the data.
The phone data: hours of screen-on time \(x\) since a full charge, and battery left \(y\) (%), for one phone on 8 occasions.
| \(x\) (hours) | 1 | 1.5 | 2.5 | 3 | 4 | 4.5 | 5.5 | 6 |
|---|---|---|---|---|---|---|---|---|
| \(y\) (%) | 89 | 82 | 74 | 63 | 58 | 49 | 41 | 32 |
For the phone data, \(n=8\), \(\sum x=28\), \(\sum y=488\), \(\sum x^2=121\), \(\sum y^2=32\,560\) and \(\sum xy=1456\). (a) State, with a reason, which is the explanatory variable. (b) Find the regression line of \(y\) on \(x\). (c) Interpret the gradient and the intercept in context.
(a) Screen-on time: using the screen drains the battery, so the battery level depends on the time, not the reverse.
(b) \(S_{xx}=121-\dfrac{28^2}{8}=23\), \(S_{xy}=1456-\dfrac{28\times488}{8}=-252\). \(b=\dfrac{-252}{23}=-10.957\ldots\), \(a=61-(-10.957\ldots)(3.5)=99.35\): \(y=99.3-11.0x\).
(c) \(b\): the battery level falls by about 11 percentage points for each hour of screen-on time. \(a\): with no screen-on time the battery is at about 99%, close to the full charge it starts from.
When can you trust an estimate from the line?
An interpolated estimate is reliable when the correlation is strong (\(r\) close to ±1). An extrapolated estimate is unreliable: nothing in the data shows that the straight-line pattern carries on beyond it. A weak correlation makes every estimate unreliable, inside the range or not.
Use the regression line from Worked example 4 to estimate the battery left after (a) 2 hours and (b) 10 hours of screen-on time. Comment on the reliability of each estimate.
(a) \(y=99.348-10.957\times2=\) 77.4%. Reliable: 2 hours is inside the data (interpolation), and \(r=-0.994\) (using \(S_{yy}=2792\) from the sums in Worked example 4) shows strong negative linear correlation.
(b) \(y=99.348-10.957\times10=\) −10.2%. Unreliable: 10 hours is outside the data (extrapolation), and a negative battery level is impossible.
Rearranging \(y=a+bx\) to estimate \(x\) from a given \(y\). The line of \(y\) on \(x\) minimises vertical distances only; to estimate \(x\) you would need the regression line of \(x\) on \(y\), a different line unless \(r=\pm1\).
How does coding affect correlation and regression?
Data are often coded to make the numbers smaller: \(u=\dfrac{x-p}{q}\) and \(v=\dfrac{y-s}{t}\). Adding or subtracting a constant shifts the points; dividing by a positive constant rescales an axis. Neither changes the shape of the scatter diagram, so \(r\) is unchanged. Dividing by a negative constant reflects the diagram: \(r\) changes sign but keeps its size. To turn a coded regression line into one for \(x\) and \(y\), substitute the coding into it and rearrange.
A batch of 8 steel bolts is heat-treated at different furnace temperatures \(x\) °C and the hardness \(y\) of each is measured. The data are coded using \(u=\dfrac{x-800}{5}\) and \(v=y-200\), giving \(\sum u=93\), \(\sum v=236\), \(\sum u^2=1383\), \(\sum v^2=7964\), \(\sum uv=3290\). (a) Find the product moment correlation coefficient between \(x\) and \(y\). (b) Find the regression line of \(v\) on \(u\). (c) Hence find the regression line of \(y\) on \(x\). (d) Estimate the hardness of a bolt treated at 860 °C.
(a) \(S_{uu}=1383-\dfrac{93^2}{8}=301.875\), \(S_{vv}=7964-\dfrac{236^2}{8}=1002\), \(S_{uv}=3290-\dfrac{93\times236}{8}=546.5\). \(r=\dfrac{546.5}{\sqrt{301.875\times1002}}=0.994\); coding does not change \(r\), so \(r=0.994\).
(b) Gradient \(=\dfrac{546.5}{301.875}=1.8104\ldots\); \(\bar u=11.625\), \(\bar v=29.5\); intercept \(=29.5-1.8104\ldots\times11.625=8.4547\): \(v=8.45+1.81u\).
(c) Substitute: \(y-200=8.4547+1.8104\times\dfrac{x-800}{5}\), so \(y=208.45+0.36207x-289.66\): \(y=-81.2+0.362x\).
(d) \(y=-81.202+0.36207\times860=\) 230.
Check: \(S_{xx}=5^2\times S_{uu}=7546.875\) and \(S_{xy}=5\times S_{uv}=2732.5\), so \(b=2732.5\div7546.875=0.362\). The intercept −81.2 would be the hardness at 0 °C, far outside the data, so it has no practical meaning.
What are the limitations of the correlation coefficient?
- Linear only. \(r\) measures how close the points are to a straight line. A strong curved relationship can have \(r\) near 0, so always look at the scatter diagram.
- Correlation is not causation. Both variables may depend on a third: ice-cream sales and sunburn cases rise together because both increase in hot, sunny weather.
- Outliers. One unusual point, often a recording error, can change \(r\) a lot, in either direction.
- Small samples. With only a few points, a large \(r\) can happen by chance (testing for this is beyond S1).
- The range of the data. \(r\) describes the relationship only within the values measured.
The arm span \(x\) cm and height \(y\) cm of 10 students give \(\sum x=1708\), \(\sum y=1654\), \(\sum x^2=292\,876\), \(\sum y^2=277\,202\), \(\sum xy=283\,512\), and \(r=0.494\). One student's pair was recorded as (172, 117); it should be removed. (a) Find the new summary statistics for the remaining 9 students. (b) Find the new value of \(r\) and comment.
(a) Subtract the incorrect pair from every sum: \(\sum x=1536\), \(\sum y=1537\), \(\sum x^2=292\,876-172^2=263\,292\), \(\sum y^2=277\,202-117^2=263\,513\), \(\sum xy=283\,512-172\times117=263\,388\).
(b) With \(n=9\): \(S_{xx}=263\,292-\dfrac{1536^2}{9}=1148\), \(S_{yy}=263\,513-\dfrac{1537^2}{9}=1027.56\), \(S_{xy}=263\,388-\dfrac{1536\times1537}{9}=1073.33\). \(r=\dfrac{1073.33}{\sqrt{1148\times1027.56}}=\) 0.988. The correlation is now strong and positive: taller students tend to have longer arm spans; the single error had hidden this.
Removing a point by subtracting its values from \(\sum x\) and \(\sum y\) but leaving \(\sum x^2\), \(\sum y^2\), \(\sum xy\) and \(n\) unchanged. Every sum loses that point's contribution, and \(n\) goes down by 1.
How do the pieces fit together in a full question?
A full question usually runs: summary statistics → \(r\) and its meaning → regression line → interpret the gradient → estimate and judge reliability → coding or a change of units. Keep full calculator values throughout.
The table gives the altitude \(x\) (hundreds of metres) and the mean July temperature \(y\) (°C) for 8 weather stations in one mountain region.
| \(x\) | 2 | 5 | 7 | 9 | 12 | 14 | 16 | 19 |
|---|---|---|---|---|---|---|---|---|
| \(y\) | 24.6 | 23.5 | 21.9 | 21.4 | 19.1 | 18.6 | 16.8 | 15.7 |
You may use \(\sum x=84\), \(\sum y=161.6\), \(\sum x^2=1116\), \(\sum y^2=3334.48\), \(\sum xy=1569.3\). (a) Find \(S_{xx}\), \(S_{yy}\) and \(S_{xy}\). (b) Calculate \(r\) and interpret it in context. (c) Find the regression line of \(y\) on \(x\). (d) Interpret the gradient in context. (e) Estimate the mean July temperature at a station 600 m above sea level, and comment on its reliability. (f) Explain why the line should not be used for a summit at 3500 m. (g) The altitudes are re-recorded in metres, \(w=100x\). State the correlation coefficient between \(w\) and \(y\), and the gradient of the regression line of \(y\) on \(w\).
(a) \(S_{xx}=1116-\dfrac{84^2}{8}=\) 234; \(S_{yy}=3334.48-\dfrac{161.6^2}{8}=\) 70.16; \(S_{xy}=1569.3-\dfrac{84\times161.6}{8}=\) −127.5.
(b) \(r=\dfrac{-127.5}{\sqrt{234\times70.16}}=\) −0.995: strong negative linear correlation; higher stations tend to have lower mean July temperatures.
(c) \(b=\dfrac{-127.5}{234}=-0.54487\ldots\); \(\bar x=10.5\), \(\bar y=20.2\); \(a=20.2-(-0.54487\ldots)(10.5)=25.921\): \(y=25.9-0.545x\).
(d) For each extra 100 m of altitude, the mean July temperature falls by about 0.545 °C.
(e) 600 m means \(x=6\): \(y=25.921-0.54487\times6=\) 22.7 °C; reliable, because \(x=6\) is inside the data (interpolation) and the correlation is strong.
(f) 3500 m is \(x=35\), far outside the range 2 to 19: extrapolation, so the linear pattern may not continue.
(g) Multiplying \(x\) by a positive constant does not change \(r\); it divides the gradient by 100: \(r=-0.995\), gradient −0.00545 °C per metre.
Exam tips for S1.4
Show \(S_{xy}=1569.3-\dfrac{84\times161.6}{8}\) before the answer. If a calculator slip gives a wrong value, the method is still visible and later parts can follow through from it.
Store \(S_{xx}\), \(S_{yy}\), \(S_{xy}\) and \(b\) in your calculator memory. Give \(r\), \(a\) and \(b\) to 3 significant figures at the end, unless the question asks otherwise.
"For each extra hour of sunshine, about 1.23 kWh more energy is generated" is an interpretation; "\(b\) is the gradient" is not. Use the names and units of both variables.
\(r\), \(S_{xy}\) and \(b\) must all have the same sign as the trend in the scatter diagram, and \(-1\le r\le1\). The line must pass through \((\bar x,\bar y)\).
To judge an estimate, say whether it is interpolation or extrapolation and what the value of \(r\) shows about the strength of the correlation.
Common misconceptions
| Misconception | Why it is wrong | Correct idea |
|---|---|---|
| \(r=0\) means the variables are unrelated. | \(r\) only measures linear association; a curve can give \(r=0\). | \(r=0\) means no linear correlation; look at the scatter diagram. |
| A strong correlation proves that \(x\) causes \(y\). | Both may depend on a third variable, or it may be coincidence. | Correlation describes association, not cause. |
| \(r=-0.9\) is weaker than \(r=0.5\). | The sign gives the direction, the size gives the strength. | −0.9 is a strong (negative) correlation. |
| Coding changes the correlation coefficient. | Shifting and positive rescaling do not change the shape of the scatter diagram. | \(r\) is unchanged by \(u=(x-p)/q\) with \(q>0\). |
| The regression line of \(y\) on \(x\) can be rearranged to predict \(x\). | It minimises vertical distances only. | Use it only to estimate \(y\) from \(x\). |
| An estimate is reliable as long as \(r\) is close to 1. | The pattern is only known inside the data. | Extrapolated estimates are unreliable however strong \(r\) is. |
How this topic connects
S1.2 Measures of location and dispersion
\(S_{xx}/n\) is the variance of \(x\), so the coding rules here are the same ones used for means and standard deviations.
P1 Straight-line graphs
The regression line \(y=a+bx\) is a straight line: gradient and intercept are read and interpreted exactly as for \(y=mx+c\).
P2/P3 Exponentials and logarithms
A curved relationship such as \(y=kx^n\) becomes linear when you plot \(\log y\) against \(\log x\); correlation and regression then apply to the logged data.
Key formulae
| Formula | Use | In the formulae list? |
|---|---|---|
| \(S_{xx}=\sum x^2-(\sum x)^2/n\) | spread of \(x\); also \(S_{yy}\) for \(y\) | Yes |
| \(S_{xy}=\sum xy-(\sum x)(\sum y)/n\) | how \(x\) and \(y\) vary together | Yes |
| \(r=S_{xy}/\sqrt{S_{xx}S_{yy}}\) | product moment correlation coefficient | Yes |
| \(b=S_{xy}/S_{xx}\), \(a=\bar y-b\bar x\) | regression line of \(y\) on \(x\): \(y=a+bx\) | Yes |
| \(\bar y=a+b\bar x\) | the line passes through the mean point | No – derive it |
| \(S_{uu}=S_{xx}/q^2\), \(S_{uv}=S_{xy}/(qt)\), \(r_{uv}=r_{xy}\) | coding \(u=(x-p)/q\), \(v=(y-s)/t\) | No – learn it |
Check your understanding
1. For a set of data, \(S_{xx}=40\), \(S_{yy}=90\) and \(S_{xy}=-48\). Find \(r\).
\(r=-48\div\sqrt{40\times90}=-48\div60=-0.8\): fairly strong negative linear correlation.
2. The regression line of \(y\) on \(x\) is \(y=12.4+0.75x\), and \(\bar x=20\). Find \(\bar y\).
The line passes through \((\bar x,\bar y)\), so \(\bar y=12.4+0.75\times20=27.4\).
3. Two variables have \(r=0.03\). Does this show they are not related?
No. It shows there is almost no linear correlation. The variables could still have a strong curved relationship, which a scatter diagram would show.
4. Data are coded with \(u=(x-50)/10\) and \(v=(y-3)/2\). The coded data have \(r=0.62\). Find \(r\) for \(x\) and \(y\).
0.62. Subtracting constants and dividing by positive constants does not change the correlation coefficient.
5. Phone sales in a country and the number of people over 80 have both risen steadily, with \(r=0.97\). Does selling phones make people live longer?
No. Correlation does not imply causation: both quantities increase over time (time, wealth and population growth are third variables), so they are correlated without one causing the other.
Specification checklist
- (S1 4.1) I can draw and describe a scatter diagram, using positive, negative or no correlation and saying how strong it is.
- (S1 4.1) I can calculate \(S_{xx}\), \(S_{yy}\) and \(S_{xy}\) from raw data or from given sums.
- (S1 4.1) I can explain that the least-squares regression line of \(y\) on \(x\) minimises the sum of the squared vertical distances, and find it as \(y=a+bx\) with \(b=S_{xy}/S_{xx}\) and \(a=\bar y-b\bar x\).
- (S1 4.2) I can identify the explanatory and the response variable and use the regression line to estimate the response.
- (S1 4.2) I can interpret \(a\) and \(b\) in context, and explain when the intercept has no practical meaning.
- (S1 4.2) I can tell interpolation from extrapolation and comment on the reliability of an estimate.
- (S1 4.2) I can find a regression line from coded data and convert it back to the original variables.
- (S1 4.3) I can calculate the product moment correlation coefficient, interpret it in context and know that coding does not change it.
- (S1 4.3) I can explain the limitations of \(r\): linear relationships only, the effect of outliers, and that correlation does not imply causation.