Re: Help a nonMath figure out optimum values

On Mon, 27 Feb 2006 19:22:29 -0800, Robert A. Macy wrote:

There may not be an answer to this question, but here goes...

I have a variable, containing white noise (Guassian distribution?), and
I can select the number of samples I use. I have a choice of finding
the mean of this variable by taking some quantity of samples, summing,
and finding the mean, which requires knowing the number of samples.

Or, I can do a recursive "average" where the new average value is equal
to (1-k) times the old value plus k times the new value, where 0.001 <
k < 0.5, and simply go along for some number of samples until there is
some assurance that the average has been calculated.

This 'open' recursive form is preferred, since it allows some
observations to occur before stopping the accumulation process. Plus,
recursion slowly kills old data, in case a transient noise spike sneaks
in.

I tried to solve this empirically, by using 100, 200, and 1000 samples,
combined with range of k. Plotting the difference between the mean and
the running average versus k showed some very interesting, but
non-conclusive, observations. For exmaple, using 100 samples the error
was minimized using k equal to 0.0075 (which seemed really strange) and
0.1

I expected k waould be best if equal to something like 2 times 1 over
the number of samples, or such. Doesn't seem to be the best value for
k.

I wrote an octave script to calculate a random set of variables, find
the mean, do recursive average, compare the two. plot 'error' versus k
factor.

There was no obvious minimum in the accuracy. The accuracy was greatly
a function of the pattern of the noise in the samples. Although
random, the number of samples was often too small to really insure a
random set of variables.

Very unexpectedly, there seemed to be a relationship between the number
of samples and the number of minimum nulls.

Has anybody done work like this? Has an answer, or insight into the
answer?

Or, should I just blow the whole thing off and fix the number of
samples and take the mean?

It's just that using a fixed number of samples after a bad transient
hits I have to wait a fixed amount of time to accumulate all the
necessary samples. Where with recursion I can kind of "see" the
transient, estimate how much longer to run the recursion and then cut
off the calculation and be fairly certain that the mean I've just
calculated is good.

I'm no a mathematician, so please bear with me here.

- Robert -

Would it be possible to use a different recursion, viz
if your samples are X[1],X[2]...
M[1] = X[1]
M[n+1] = M[n] + (X[n+1]-M[n])/(n+1)
Then M[n] is the average of X[1]..X[n]

Duncan



.

Relevant Pages

  • Help a nonMath figure out optimum values
    ... in case a transient noise spike sneaks ... I wrote an octave script to calculate a random set of variables, ... It's just that using a fixed number of samples after a bad transient ... Where with recursion I can kind of "see" the ...
    (sci.math)
  • Re: Help a nonMath figure out optimum values
    ... in case a transient noise spike sneaks ... It's just that using a fixed number of samples after a bad transient ... Where with recursion I can kind of "see" the ... off the calculation and be fairly certain that the mean I've just ...
    (sci.math)