Definition: A stochastic process X1, X2, ... is a Markov process if for any discrete time index n = 1, 2, ... Pr(Xn+1 = xn+1 | Xn = xn, ... , X1 = x1) = Pr(Xn+1 = xn+1 | Xn = xn)
In a Markov process, the probability of a random variable X being equal to some value x at time n + 1, given all the x values that came before it in the sequence, is equal to the probability of X being equal to some value x at time n + 1 given just the value of x that came before it. In other words, X at time n + 1 is only dependent on x at time n, not any other value of x. That is, X at time n + 1 is independent of all other x except X at time n.
This code repository covers two examples - weather and epidemic - on how to:
- define a transition probability matrix
- create a discrete-time Markov chain
- access a transition probability from one to the other states
- simulate the state probability after n steps, given an initial state
- find transient states and absorbing state
- find the state probability at steady state
- draw the state transition diagram
In the third case study, the epidemic is modeled and solved with a system of ordinary differential equations as well as with Jacobian matrix.
For simplicity, this study considers the states of "susceptible", "infected" and "recovered" (SIR). Often times, such model can also be viewed as "susceptible", "exposed", "infected" and "recovered" (SEIR).
In other cases, such as COVID-19, there could also be the states of "hospitalized", "ICU" and "dead".
One interesting scientific article even broke down the states into "infected undetected", "infected detected", "recovered undetected" and "recovered undetected" to find a balance among interventions: testing/tracing/isolating, quarantine/lockdown, and raising ICU capacity.
For the weather and epidemic examples, run the following command on terminal:
sudo apt-get install r-cran-igraph
sudo apt-get install r-cran-markovchain
Then in R script, call markovchain and diagram libraries.
> library(markovchain)
Package: markovchain
Version: 0.8.5-4
Date: 2021-01-07
BugReport: https://github.com/spedygiorgio/markovchain/issues
> library(diagram)
Loading required package: shape
For the case study on epidemic modeling with system dynamics, install deSolve library.
Weather forecast: weather.R, default transition (state/probability) diagram, more beautiful diagram
Epidemic simulation (susceptible-infected-recovered, SIR): epidemic.R
Solving SIR model with ordinary differential equations, i.e. system dynamics approach: epi_ode.R
More detailed instructions and reading resources for the weather example can be found in RPubs.
Markov chain applied to epidemic (Susceptible-Infected-Recovered) models can be found in R-bloggers.
Explanation on modeling the pandemics were previously written in the series of blog post: part1, part2, part3
Other resources include R Package: General Solver for Ordinary Differential Equations and Visualization with Infectious Disease Models with shinySIR.