Markov chains

يعتمد على

عدد المتطلبات السابقة الإلزامية (حتى الأساس): 42

ستتعلّم

• Model processes with transition matrices and compute n-step transition probabilities.
• Classify states as recurrent or transient and find stationary distributions.
• Prove convergence to equilibrium for irreducible aperiodic finite chains.

تعريف

• A Markov chain is a sequence of random variables in which the conditional distribution of the next state, given the past, depends only on the current state.
• A probability distribution π (as a row vector) is stationary for transition matrix P if πP = π.
• A chain is irreducible if every state can be reached from every other state.

مبرهنة

• An irreducible Markov chain on a finite state space has a unique stationary distribution.
• If it is also aperiodic, the distribution at time n converges to the stationary distribution from any starting distribution.
• Pólya: simple random walk is recurrent on ℤ and ℤ² and transient on ℤ³.

مفاهيم خاطئة شائعة

• Reading 'memoryless' as 'forgets everything'; it means the future depends on the past only through the present.

يقود إلى

اكتشافات اختيارية

• Mathematical biology Stochastic models of populations and of genetic drift, such as the Wright–Fisher model, are Markov chains.

قراءات إضافية

• J. R. Norris, Markov Chains (1997)