This note covers the mean and variance of linear combinations of random variables, such as \(aX+b\), \(aX+bY\) and sums of independent copies of the same variable. It also covers combinations of normal variables and the distribution of a sample mean. These ideas let you model totals, differences and averages, for example the total mass in a lift or the average of a sample.
Key ideas
One variable. If every value of \(X\) is multiplied by \(a\) and then \(b\) is added, the mean is transformed in the same way, but the variance only responds to the scaling:
- \(\mathrm{E}(aX+b)=a\,\mathrm{E}(X)+b\)
- \(\mathrm{Var}(aX+b)=a^2\,\mathrm{Var}(X)\)
Adding a constant shifts the distribution but does not change its spread. Multiplying by \(a\) multiplies the standard deviation by \(|a|\), so the variance is multiplied by \(a^2\).
Two or more variables. For any random variables, means always add:
- \(\mathrm{E}(aX+bY)=a\,\mathrm{E}(X)+b\,\mathrm{E}(Y)\)
For independent variables, variances add, and they add even for a difference:
- \(\mathrm{Var}(aX+bY)=a^2\,\mathrm{Var}(X)+b^2\,\mathrm{Var}(Y)\)
- \(\mathrm{Var}(X-Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)\). Subtracting a variable does not reduce the spread. It makes the result less predictable.
Sums of copies versus multiples. Let \(X_1,X_2,\dots,X_n\) be independent observations of \(X\), each with mean \(\mu\) and variance \(\sigma^2\). Then \(X_1+\dots+X_n\) has mean \(n\mu\) and variance \(n\sigma^2\). This is different from \(nX\), which has mean \(n\mu\) but variance \(n^2\sigma^2\). The first describes \(n\) separate items (four different people). The second describes one item copied \(n\) times (one person counted four times).
Normal variables. A linear combination of independent normal variables is also normal. Find its mean and variance using the rules above, then use the normal distribution on your GDC. Poisson variables behave differently: a sum of independent Poisson variables is Poisson, but a multiple such as \(2X\) is not Poisson.
Sample mean. For a random sample of size \(n\) with mean \(\bar{X}=\frac{X_1+\dots+X_n}{n}\):
- \(\mathrm{E}(\bar{X})=\mu\) and \(\mathrm{Var}(\bar{X})=\dfrac{\sigma^2}{n}\).
- If \(X\sim N(\mu,\sigma^2)\), then \(\bar{X}\sim N\!\left(\mu,\dfrac{\sigma^2}{n}\right)\) exactly.
- If \(X\) is not normal but \(n\) is large, the central limit theorem says \(\bar{X}\) is approximately normal with the same mean and variance.
Larger samples give sample means that are closer to \(\mu\), because \(\mathrm{Var}(\bar{X})\) shrinks as \(n\) grows. An unbiased estimator has an expected value equal to the quantity it estimates. So \(\bar{X}\) is an unbiased estimator of \(\mu\), and the sample variance with divisor \(n-1\) is an unbiased estimator of \(\sigma^2\).
A quick context example. Daily temperature in a city has mean 20 and standard deviation 4 in degrees Celsius. The Fahrenheit value is \(F=1.8C+32\), so its mean is \(1.8\times 20+32=68\) and its variance is \(1.8^2\times 16=51.84\). The standard deviation is \(1.8\times 4=7.2\). The \(+32\) shifts the mean only. This is the same rule with \(a=1.8\) and \(b=32\).
When to use each result. Use the \(\bar{X}\) result when the question talks about an average of a sample, and the sum result when it talks about a total. If the question asks about the difference between two quantities, form \(X-Y\) first, then find its mean and variance.
Definitions and formulae
| Term or formula | Meaning / when to use |
|---|---|
| \(\mathrm{E}(aX+b)=a\mathrm{E}(X)+b\) | Mean of a linear transformation. |
| \(\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)\) | Variance of a linear transformation; \(b\) has no effect. |
| \(\mathrm{E}(aX\pm bY)=a\mathrm{E}(X)\pm b\mathrm{E}(Y)\) | Always true, independent or not. |
| \(\mathrm{Var}(aX\pm bY)=a^2\mathrm{Var}(X)+b^2\mathrm{Var}(Y)\) | Independent variables only. Variances add for both \(+\) and \(-\). |
| \(X_1+\dots+X_n\) | Mean \(n\mu\), variance \(n\sigma^2\) (independent copies). |
| \(\bar{X}\) | Mean \(\mu\), variance \(\sigma^2/n\). |
| \(\bar{X}\sim N(\mu,\sigma^2/n)\) | Exact if \(X\) is normal; approximate for large \(n\) otherwise. |
| Unbiased estimator | An estimator whose expected value equals the true parameter. |
Worked examples
Example 1 \(X\) and \(Y\) are independent random variables with \(\mathrm{E}(X)=12\), \(\mathrm{Var}(X)=4\), \(\mathrm{E}(Y)=20\) and \(\mathrm{Var}(Y)=9\). Find the mean, variance and standard deviation of \(W=3X-2Y\).
Mean: \(\mathrm{E}(W)=3\times 12-2\times 20=36-40=-4\).
Variance: the variables are independent, so \(\mathrm{Var}(W)=3^2\times 4+(-2)^2\times 9=36+36=72\).
Standard deviation: \(\sqrt{72}\approx 8.49\).
Answer: mean \(-4\), variance 72, standard deviation 8.49.
Example 2 The mass of an adult passenger is \(W\sim N(75,\,12^2)\) kg. Four independent passengers enter a lift. (a) Find the probability that their total mass exceeds 330 kg. (b) Find the probability that \(4W\) exceeds 330 kg, and explain why it differs.
(a) Let \(T=W_1+W_2+W_3+W_4\). Then \(T\) is normal with mean \(4\times 75=300\) and variance \(4\times 144=576\), so \(T\sim N(300,\,24^2)\).
\(P(T\gt 330)\): \(z=\frac{330-300}{24}=1.25\). Using the normal cumulative distribution function on the GDC gives \(P(T\gt 330)\approx 0.106\).
(b) \(4W\) is normal with mean 300 and variance \(4^2\times 144=2304\), so the standard deviation is 48. Then \(z=\frac{30}{48}=0.625\) and \(P(4W\gt 330)\approx 0.266\).
The probabilities differ because four different passengers partly cancel each other (some heavy, some light), so the total varies less than four times one person's mass.
Answer: (a) 0.106 (b) 0.266
Example 3 The scores in a test are \(X\sim N(100,\,15^2)\). A random sample of 25 people is taken. Find the probability that the sample mean is greater than 104.
\(\mathrm{E}(\bar{X})=100\) and \(\mathrm{Var}(\bar{X})=\frac{15^2}{25}=9\), so \(\bar{X}\sim N(100,\,3^2)\).
\(z=\frac{104-100}{3}\approx 1.333\). The normal cumulative distribution function on the GDC with lower bound 104, mean 100 and standard deviation 3 gives 0.0912.
Answer: \(P(\bar{X}\gt 104)\approx 0.0912\).
Common errors
- Subtracting variances for \(X-Y\). The variance of a difference of independent variables is the sum of the variances.
- Forgetting to square the coefficient: \(\mathrm{Var}(3X)=9\mathrm{Var}(X)\), not \(3\mathrm{Var}(X)\).
- Confusing \(X_1+X_2\) (two independent observations, variance \(2\sigma^2\)) with \(2X\) (variance \(4\sigma^2\)).
- Using the standard deviation as if it were the variance when combining, or entering the variance into the GDC where it needs the standard deviation.
- Using the rule for the variance of a sum when the variables are not independent. State the independence assumption.
Exam tip: Write down the new distribution, with both its mean and its variance worked out, before you use the GDC. Then convert the variance to a standard deviation for the normal function.
Check yourself
- \(X\) and \(Y\) are independent with \(\mathrm{E}(X)=6\), \(\mathrm{Var}(X)=5\), \(\mathrm{E}(Y)=4\), \(\mathrm{Var}(Y)=3\). Find \(\mathrm{E}(X-2Y)\) and \(\mathrm{Var}(X-2Y)\).
Answer
\(\mathrm{E}=6-8=-2\). \(\mathrm{Var}=5+(-2)^2\times 3=5+12=17\). - Three independent variables are each \(N(10,\,2^2)\). Find the probability that their sum is less than 26.
Answer
The sum is \(N(30,\,12)\), with standard deviation \(\sqrt{12}\approx 3.464\). \(P(S\lt 26)\approx 0.124\). - A sample of 16 is taken from \(N(80,\,10^2)\). Find \(P(\bar{X}\lt 77)\).
Answer
\(\bar{X}\sim N(80,\,2.5^2)\). \(z=\frac{77-80}{2.5}=-1.2\), so \(P\approx 0.115\). - \(A\sim N(20,\,3^2)\) and \(B\sim N(18,\,2^2)\) are independent. Find \(P(A\gt B)\).
Answer
\(D=A-B\sim N(2,\,9+4)=N(2,\,13)\). \(P(D\gt 0)\): \(z=\frac{0-2}{\sqrt{13}}\approx -0.555\), so \(P\approx 0.710\).