Econometrics Toolbox™ supports modeling and analyzing discrete-time Markov models. These models describe stochastic processes that assume states xt in a state space X, subject to the Markov property, which requires the distribution of xt+1 to be independent of the process history before reaching state xt.
A discrete state-space Markov process, or Markov chain, is represented by a directed graph and described by a right-stochastic transition matrix P. The distribution of states at time t+1 is the distribution of states at time t multiplied by P. The structure of P determines the evolutionary trajectory of the chain, including asymptotics.
A Markov-switching dynamic regression model describes the dynamic behavior of time series variables in the presence of structural breaks or regime changes. A discrete-time Markov chain represents the discrete state space of the regimes, and specifies the probabilistic switching mechanism among the regimes. A collection of dynamic regression (ARX or VARX) submodels describes the dynamic behavior of the time series within the regimes.
A continuous state-space Markov process, or state-space model, allows for trajectories through a continuous state space. The underlying Markov process is typically unobserved. A supplemental observation equation describes the evolution of measurable characteristics of the system, dependent on the Markov process.