Re: markov chain decomposition (nearly completely decomposable system)
From: Dan Heyman (danheyman_at_yahoo.com)
Date: 11/12/04
- Next message: Dave Seaman: "Re: generalized birthday problem"
- Previous message: George Cox: "Re: Root Finder v."
- In reply to: Daniel Sadoc: "markov chain decomposition (nearly completely decomposable system)"
- Messages sorted by: [ date ] [ thread ]
Date: 12 Nov 2004 13:30:58 -0800
I think your intuition is correct. It's certainly true that when all
the epsilons are actually zero, the chain has k-1 communicating
classes and the states in group k are all transient. For epsilons
small and positive (and well placed so the chain is irreducible), a
paper by Alan Karr from 1975 (Weak convergence of a sequence of Markov
chains, Z. Wahr... und Verw. Gebiete 33) showed that the stationary
distribution of an ergodic chain was a continuous function of the
tranistion probabilities, so what's true for epsilon equal 0 is
approximately true for small epsilon. Since your limiting chain (as
epsilons ->0) is not ergodic, this result doesn't hold, but maybe you
can build on Karr's analysis to show that when the limiting process
makes a state transient, the stationary probability of that state
converges to zero.
The basic difficulty in proving results of this type is that the
stationary distiribution is a limit, so what really happens is that
epsilon goes to zero first, and then the number of transitions goes to
00. Looking at the limit of the stationary distribution as epsilon ->0
has the number of tranistions going to 00 first and then the epsilons
go to 0, so the order of taking limits has been reversed. Some
conditions need to be imposed to get both double limits to be the
same. Ward Whitt and I have some results in this vein in "Limits for
queues as the waiting room grows", Queueing Systems 5, 1989.
Dan Heyman
sadoc@rio.com.br (Daniel Sadoc) wrote in message news:<d016349.0411111848.5cfb7d4b@posting.google.com>...
> Hi,
>
> Please, I would like to know if it would be possible to generalize any
> results about the steady state solution of an ERGODIC Markov Chain
> with a probability transition matrix P having the following structure:
>
> P11 P12 P13 ... P1k
> P21 P22 P23 ... P2k
> ... ...
> Pk1 Pk2 Pk3 ... Pkk
>
>
> Where the magnitude of the elements of the non-diagonal blocks Pij are
> very small relative to 1 (they are all equal to epsilon or zero) -
> except the elements of the last row (the elements of the blocks Pkj).
> So, the system is nearly completely decomposable, except for the last
> row.
>
> Besides that, I know one more very important property: no proper
> subset of states of Pkk have all its output probabilities very small
> (equal to epsilon or zero).
>
> My intuition says that when epsilon goes to zero, the states
> characterized by the block Pkk will not be on the support of the
> steady state probability - they will have probability near zero. But
> I'm not sure about this! Any help in order to show it, or find a
> counterexample, is very welcome!
>
> Thanks a lot.
>
> Regards,
> Daniel Sadoc
- Next message: Dave Seaman: "Re: generalized birthday problem"
- Previous message: George Cox: "Re: Root Finder v."
- In reply to: Daniel Sadoc: "markov chain decomposition (nearly completely decomposable system)"
- Messages sorted by: [ date ] [ thread ]
Relevant Pages
|
