4 · IB Maths
Markov chains
A system hops between two states with fixed transition probabilities. Change where it starts and how many steps it takes, and watch the distribution settle to the same steady state. A random walker hops alongside as a check.
Predict first
Start from the defaults (p = 0.30, q = 0.20), where P(A) settles at 0.40. You raise q to 0.30. What is the new steady-state P(A)?
The steady state is (q, p) ÷ (p + q). With p = 0.30 and q = 0.30, P(A) = 0.30 ÷ 0.60 = 0.50. A larger chance of returning from B to A means A is occupied more often.
Check it in the model: Move the P(B → A) = q slider from 0.20 to 0.30 and read the Steady state line.
state A state B random walker (share of hops spent in A)
steady state: Tx = x, so x = (q, p) ÷ (p + q)
second eigenvalue: λ = 1 − p − q
Try this
- Use the defaults (p = 0.30, q = 0.20) and drag n from 0 to 30. How many steps until P(A) is within 0.01 of 0.4?
- Change 'Start in A' from 100% to 0%. Does the steady state change, and does the time to reach it?
- Set p = q = 1.00 with a 100% start. Why does the distribution never settle, and what does λ = −1 tell you?
Common error: each column of the transition matrix must sum to 1 when the state vector is a column; writing the probabilities in the wrong orientation gives the wrong steady state.
More on 4: Venn diagrams and conditional probability · Discrete random variables · Continuous random variables · Bayes' theorem: medical test · Standard deviation and outliers · Binomial distribution · The normal distribution · Poisson distribution · Scatter and regression · Tree diagrams and Bayes’ theorem · Box plots and histograms