markov chain decomposition (nearly completely decomposable system)

From: Daniel Sadoc (sadoc_at_rio.com.br)
Date: 11/12/04 

Date: 11 Nov 2004 18:48:14 -0800

Hi,

Please, I would like to know if it would be possible to generalize any
results about the steady state solution of an ERGODIC Markov Chain
with a probability transition matrix P having the following structure:

P11 P12 P13 ... P1k
P21 P22 P23 ... P2k
... ...
Pk1 Pk2 Pk3 ... Pkk

Where the magnitude of the elements of the non-diagonal blocks Pij are
very small relative to 1 (they are all equal to epsilon or zero) -
except the elements of the last row (the elements of the blocks Pkj).
So, the system is nearly completely decomposable, except for the last
row.

Besides that, I know one more very important property: no proper
subset of states of Pkk have all its output probabilities very small
(equal to epsilon or zero).

My intuition says that when epsilon goes to zero, the states
characterized by the block Pkk will not be on the support of the
steady state probability - they will have probability near zero. But
I'm not sure about this! Any help in order to show it, or find a
counterexample, is very welcome!

Thanks a lot.

Regards,
Daniel Sadoc

Relevant Pages