Consider this theoretical, right-stochastic transition matrix of a stochastic process.

Create the Markov chain that is characterized by the transition matrix P.

P = [0.5 0.5 0 0; 0.5 0.4 0.1 0; 0 0 0 1; 0 0 1 0];
mc = dtmc(P);

Plot a directed graph of the Markov chain. Visually identify the communicating class to which each state belongs by using node colors.

figure;
graphplot(mc,'ColorNodes',true);

States 3 and 4 belong to a communicating class with period 2. States 1 and 2 are transient.

Programmatically identify to which communicating classes the states belong.

bins = classify(mc)

bins = 1×4

     1     1     2     2

bins is a 1-by-4 vector of communicating class labels. Elements of bins correspond to the states in mc.StateNames. For example, bins(3) = 2 indicates that state 3 is in communicating class 2.

Identify the communicating classes of a Markov chain. Then, determine whether the classes are recurrent and their periodicity.

Generate a random seven-state Markov chain. Specify that 40 random elements in the transition matrix should be zero.

rng(1); % For reproducibility
mc = mcmix(7,'Zeros',40);

Plot a directed graph of the Markov chain. Visually identify the communicating class to which each state belongs by using node colors.

figure;
graphplot(mc,'ColorNodes',true)

Identify the communicating classes in mc, and then determine:

[bins,ClassStates,ClassRecurrence,ClassPeriod] = classify(mc)

bins = 1×7

     6     4     6     3     2     5     1

ClassStates=1×6 cell array
    {["7"]}    {["5"]}    {["4"]}    {["2"]}    {["6"]}    {1x2 string}

ClassRecurrence = 1x6 logical array

   0   0   0   0   0   1

ClassPeriod = 1×6

     1     1     1     1     1     2

mc has seven classes. Each state is its own communicating class, except states 1 and 3, which together compose class 6. Class 6 is the only recurrent class; classes 1 through 5 are transient. Class 6 has period 2; all other classes are aperiodic.

Identify the communicating classes of a Markov chain. Then, extract any recurrent subchains from the Markov chain.

Generate a random seven-state Markov chain. Specify that 40 random elements in the transition matrix should be zero.

rng(1); % For reproducibility
mc = mcmix(7,'Zeros',40);

Identify all communicating classes in the Markov chain and whether the classes are recurrent.

[bins,~,ClassRecurrence] = classify(mc);
recurrentClass = find(ClassRecurrence,1) 

recurrentClass = 6

recurrentState = find((bins == recurrentClass))

recurrentState = 1×2

     1     3

Class 6, composed of states 1 and 3, is the only recurrent class in mc.

Create a subchain from class 6 by specifying at least one state in the subchain. Plot a digraph of the subchain.

sc = subchain(mc,recurrentState);

figure;
graphplot(sc,'ColorNodes',true);