Information AboutMarkov Chain |
| CATEGORIES ABOUT MARKOV CHAIN | |
| probability theory | |
| stochastic processes | |
|
A Markov chain describes at successive times the states of a system. At these times the system may have changed from the state it was in the moment before to another or stayed in the same state. The changes of state are called transitions. The Markov property means the system is ''memoryless'', i.e. It does not "remember" the states it was in before, just "knows" its present state, and hence bases its "decision" to which future state it will transit purely on the present, not considering the past. DEFINITION A Markov chain is a sequence ''X''1, ''X''2, ''X''3, ... of distribution of the next future state ''X''''n''+1 given the present and past states is a function of the present state ''X''''n'' alone, i.e.: | ||
| Finite Markov Chain Can Be Characterized By A Matrix Of Probabilities Whose ''x'', ''y'' Element Is Given By <math> \Pr(X {n+1} | xX_n=y) \, </math> and is independent |
|
| : <math> \Pr(X {n+1} | x X_n)\, </math> |
|
| : <math> \Pr(X {n+2} | xX_n) = \int \Pr(X_{n+2}=x,X_{n+1}=yX_n)\,dy |
|
| \int \Pr(X_{n+2}=xX_{n+1}=y) \, \Pr(X_{n+1}=yX_n) \, dy</math> |
||
| : <math> \Pr(X {n+3} | xX_n) = \int \int \Pr(X_{n+3}=xX_{n+2}=y)\, \Pr(X_{n+2}=yX_{n+1}=z) \, \Pr(X_{n+1}=zX_n) \, dz \, dy</math> |
|
| : <math> \Pr(X {n+1} | x) = \int P(X_{n+1}=xX_n=y)\,P(X_n=y)\,dy </math> |
|
| :<math> \pi(x) | \int \Pr(X=xY=y)\,\pi(y)\,dy</math> |
|
|