Matrix analytic method

In probability theory, the matrix analytic method is a technique to compute the stationary probability distribution of a Markov chain which has a repeating structure (after some point) and a state space which grows unboundedly in no more than one dimension. Such models are often described as M/G/1 type Markov chains because they can describe transitions in an M/G/1 queue. The method is a more complicated version of the matrix geometric method and is the classical solution method for M/G/1 chains.

Method description
An M/G/1-type stochastic matrix is one of the form


 * $$P = \begin{pmatrix}

B_0   & B_1    & B_2    & B_3    & \cdots \\ A_0   & A_1    & A_2    & A_3    & \cdots \\ & A_0   & A_1    & A_2    & \cdots \\ &       & A_0    & A_1    & \cdots \\ \vdots & \vdots & \vdots & \vdots & \ddots \end{pmatrix}$$

where Bi and Ai are k × k matrices. (Note that unmarked matrix entries represent zeroes.) Such a matrix describes the embedded Markov chain in an M/G/1 queue. If P is irreducible and positive recurrent then the stationary distribution is given by the solution to the equations


 * $$P \pi = \pi \quad \text{ and } \quad \mathbf e^\text{T}\pi = 1$$

where e represents a vector of suitable dimension with all values equal to 1. Matching the structure of P, π is partitioned to π1, π2, π3, …. To compute these probabilities the column stochastic matrix G is computed such that


 * $$ G = \sum_{i=0}^\infty G^i A_i.$$

G is called the auxiliary matrix. Matrices are defined


 * $$\begin{align}

\overline{A}_{i+1} &= \sum_{j=i+1}^\infty G^{j-i-1}A_j \\ \overline{B}_i &= \sum_{j=i}^\infty G^{j-i}B_j \end{align}$$

then π0 is found by solving


 * $$\begin{align}

\overline{B}_0 \pi_0 &= \pi_0\\ \quad \left(\mathbf e^{\text{T}} + \mathbf e^{\text{T}}\left(I - \sum_{i=1}^\infty \overline{A}_i\right)^{-1}\sum_{i=1}^\infty \overline{B}_i\right) \pi_0 &= 1 \end{align}$$

and the πi are given by Ramaswami's formula, a numerically stable relationship first published by Vaidyanathan Ramaswami in 1988.


 * $$\pi_i = (I-\overline{A}_1)^{-1} \left[ \overline{B}_{i+1} \pi_0 + \sum_{j=1}^{i-1} \overline{A}_{i+1-j}\pi_j \right], i \geq 1.$$

Computation of G
There are two popular iterative methods for computing G,
 * functional iterations
 * cyclic reduction.

Tools

 * MAMSolver