The application of block upper Hessenberg iterative methods in the structured Markov chains for stationary distribution

Abstract

The solutions of stationary distributions vector of Markov chains that are characterized by transition matrices, which possess a specific block structure are presented in this paper using a numerical method. These methods begin with an initial estimate of solution vector, and then update it in such a way that the sequence converges to the expected solution while leaving the transition matrices unaltered. Our aim is to compute the solution on the structured Markov chain using block upper Hessenberg numerical iterative method. This is carried out with the help of some boundary conditions, laws and formulae of Markov chain with matrix operations and normalization constant to derive the stationary distribution vector's π = (πο, πι, π2. ..), and for the Markov chain is in any level i, for all i = 0, 1, 2, 3,4, the probability||7;||1. This was demonstrated on the illustrative example with the following parameters α₁ = 1, α2 = 0.5, µ = 4,81 = 5,82 = 3, and additional transition parameters 1 = 0.25 and 2 = 0.75.