Re: Roberts versus Lazio on "Overaveraging"

From: Bill Rowe (readnewscix_at_earthlink.net.invalid)
Date: 01/28/05 

Date: Fri, 28 Jan 2005 06:02:07 GMT

In article <LnzJd.6297$VA5.1171@fe07.usenetserver.com>,
 "greywolf42" <mingstb@marssim-ss.com> wrote:

> Bill Rowe <readnewscix@earthlink.net.invalid> wrote in message
> news:readnewscix-871AD6.23104719012005@news1.west.earthlink.net...

<snip>

> > > > If I make a series of such measurements that are
> > > > STATISTICALLY INDEPENDENT I can improve that accuracy to the limit of
> > > > the systematic errors involved, by averaging multiple measurements.

> > > 1) Can you support this claim, instead of simply assert it?

> > Do you have any familiarity at all with basic statistics? In particular
> > the central limit theorem? If so, it should be immediately apparent
> > Tom's claim above is a direct consequence of the central limit theorem.
> > And if you are not familiar with it pick up any reasonable basic text on
> > statistics, go to the index or table of contents and find central limit
> > theorem and turn to the referenced page.

> So sorry, but the central limit thoerem has nothing to say about taking data
> beyond the physical capabilities of the apparatus.

And my comments in no way imply the central limit theorem or any
mathematical theorem has anything to say about the methodology of taking
data. Such an assertion would be totally inane.

But the central limit theorem does apply to analysis of data. In
particular, it tells you what you should expect when you average
statistically independent samples take from a distribution that has a
finite mean and finite variance. And that is what is applicable here.

> > > Systematic errors do not affect the error bars on the statistical
> > > results. If you know that there is a systematic error, then you
> > > redo the experiment.

> > True, and this has nothing at all to do with the comment about averaging
> > independent observations.
 
> I agree that Tom's approach is flawed.

My comment doesn't say Tom's approach is flawed (it isn't). It merely
states your response to Tom's comment isn't relevant to that comment.
 
> > > > To make them statistically independent, in this case I must
> > > > re-apply the meter stick to the desk for each measurement (merely
> > > > re-reading the scale without repositioning the stick would not give
> > > > independent measurements).

> > > Yes, one must actually perform each measurement... not simply
> > > count the same measurement 'n' times.

> > True, but meaningless as human beings are unable to achieve what is
> > required.

> No science was done before computers existed?

The existence of science before the existence of computers has nothing
whatever to do with my comment. Humans have certain built in biases that
are inescapable. They are quite incapable of achieving statistical
independence when making repeated measurements. But that incapacity does
not mean an incapacity to do science.

> LOL! Didn't you ever do experiments, Bill?

I do experiments quite often and I am well aware of sources of
uncertainty in my experiments. Your comments lead me to doubt you have
similar experience.
 
> > They cannot help but remember what they did moments before and
> > repeat the measurement in essentially the same way. Hence, repeated
> > measurements made by humans one after another never really achieve
> > statistical independence.

> LOL! While computers always do things exactly the same way. So they never
> achieve statistical independence, either?

>From this, I assume you've no knowledge of quantization uncertainty,
random noise etc that cause the least significant digits of a digital
meter to vary. I also assume you've not familiarity with chaos theory
where one gets unpredictable results using a deterministic algorithm on
a computer. In short, if you really believe what you wrote and imply,
you clearly have little knowledge regarding what you are writing about.

<snip>

> > Certainly and observer can move his viewpoint. But this isn't much of a
> > solution in practice. Basically, what one would do is move your
> > viewpoint until you got what you thought was the best reading.

> Huh? Didn't anyone train you how to avoid optical parallax when taking
> readings? You don't look for the "best" reading, but the middle of the two
> extremes. Net result is no optical parallax error.

And you apparently do not realize when someone does "the middle of the
two extremes" they do this in a manner that is not independent of the
observer and has certain bias built in that is unavoidable.

> > But since we tend to do things the same way over and over again,
> > you simply trade one bias for another.
 
> If we follow the correct procedure, we eliminate optical parallax as a
> systematic error.

When things are done using best experimental technique, systematics
errors can be eliminated. But this has nothing to do with the bias I and
Tom have mentioned.
 
> > > > temperature difference in the meter stick between its calibration and
> > > > use

> > > This is not systematic error, for it can be controlled. Unless the
> > > experimenter is not competent.

> > No matter how competent and experimenter is there are limits to how well
> > any environmental factor can be controlled and measured. In the case of
> > temperature, it is impossible to make buffer against the environment
> > temperature and have 0 temperature gradient (so that the point at which
> > you measure temperature is the temperature that is important) at the
> > same time.

> But there is no need for perfection. The point is that a competent
> experimenter can control any systematic errors that might arise from
> temperature differences. All (s)he needs to do is to limit any effect to
> below the resolution required for the experiment.

Do you not understand the difference between controlling uncertainty to
some level and eliminating it?

> {snip exchange uncommented by Bill}
 
> > > > That's why averaging many readings is highly suspect when someone
> > > > claims an improvement of an order of magnitude over the intrinsic
> > > > resolution of the instrument.

> > > So, I presume you would agree that claims to 1 part in 100,000 are
> > > "highly suspect", when the intrinsic resolution of the instrument
> > > is 1 part in 10,000?

> > Do you not understand the difference between saying a measurement is
> > accurate to 10 ppm because your instrument as an accuracy of 10 ppm and
> > saying to can average 100 readings to improve the resolution of the of
> > the instrument by a factor of 10 over the specified resolution of the
> > instrument?
 
> Of course I understand the difference. The first is real. The second is
> wishful thinking. The resolution of the instrument does not depend on how
> many readings we take.

Exactly. And this is one of the main points made by Tom Roberts

> > These are two separate and distinct things. A claim of
> > accuracy of 10 ppm using an instrument specified to have that accuracy
> > is not suspect. A claim that resolution was improved by a factor of 10
> > over the specified resolution of the instrument by averaging is "highly
> > suspect". So suspect as to be considered invalid.
 
> Yet I don't see you complaining about the COBE "variations" (a factor of 10
> below the physical resolution of the instrument). Or the Hipparcos claims
> about light bending (a factor of 1000 below the physical resolution of the
> instrument).

And your point is...? Are you suggesting I have time to review and
comment every conceivable experiment? I assure you I don't.
 
> > Averaging only improves resolution when measurements are statistically
> > independent. Repeated measurements by humans don't achieve this.

> And how do "non-humans" achieve this? They are even more prone to doing
> things the same way, over and over.

Perhaps you should do some research on how a given instrument works. Pay
particular attention to discussions of noise, quantization errors,
non-linearities etc. In short, computerized measurements do not result
in the exact same measurement over and over again.

> > And statistical independence won't always be enough even if it could be
> > achieved by eliminating all human bias and systematic error.

> My point is that it will *never* be enough.

This certainly was not clear in your early posts.

> > For averaging to work its magic, the central limit has to apply.
> > And the central limit theorem does not apply to all distributions.
 
> How do you know what the "distribution" will be, before you do the
> experiment?

You might try understanding thoroughly how the measurements are to be
done, doing a bit of characterization of the measurement instruments and
some appropriate statistical analysis. But beyond this it isn't
necessary to know the distribution *before* the experiment is done. But
it is very important to know what assumptions you are making when you
analyze the data resulting from the experiment. And any time you do
things like computing an average, you are implicitly making assumptions
whether you realize it or not. 

-- 
To reply via email subtract one hundred nine

Relevant Pages

  • Re: Roberts versus Lazio on "Overaveraging"
    ... >> Tom's claim above is a direct consequence of the central limit theorem. ... >> statistics, go to the index or table of contents and find central limit ... independence when making repeated measurements. ... The resolution of the instrument does not depend on how ...
    (sci.physics.relativity)
  • Re: Roberts versus Lazio on "Overaveraging"
    ... >> Tom's claim above is a direct consequence of the central limit theorem. ... >> statistics, go to the index or table of contents and find central limit ... independence when making repeated measurements. ... The resolution of the instrument does not depend on how ...
    (sci.physics)
  • Re: Questions on Central Limit Theorem
    ... unknown distribution. ... independent x_bar statistics as I want. ... "fairly safe" assumption that the central limit theorem applies? ...
    (sci.stat.math)