Re: MatLab randn and Simulation Step Numbers

On 26 Apr 2005 09:45:10 -0700, "Brenneman" <brennemt@xxxxxxxxxx>
wrote:

> OK, first I thought it was because the number of runs was too low. This
> could have been a problem because on the tail of the distributions
> n_expected(bin i) a fraction when it divides the squared difference
> might inflate the values.
>
> No such luck.

The gross chi-squared goodness of fit test is going to
lack power when the N is suitably small. I don't know
*how* small that would be for a test with 719 d.f.

As I read this thread, I've wondered whether you are
referring to test values that are indeed, highly significant
for showing non-fit, or if you were concerned about
smaller values that were actually, merely, approaching
the proper asymptotic sums.

>
> A statistician friend of mine says when the number of simulation runs
> is very high (>10^6), the chi square test is so sensitive that it can
> be thrown off by deviations from ideality.
>
> All of which is totally counter-intuitive to my notion that more
> sampling should yield more accurate/better results.

Are you assuming that "more accurate/better" would not
detect any deviation because the method is perfect?

"deviations from ideality" include the computer round-off
error that you were concerned about, accuracy of the
CDF (which is probably good), accuracy of your programming,
and accuracy of the uniformity of the generator.

Then, if it is a really good *random* generator, it should
fail a 5% test for approximately 5% of the starting seeds.

--
Rich Ulrich, wpilib@xxxxxxxx
http://www.pitt.edu/~wpilib/index.html
.