Re: Roberts versus Lazio on "Overaveraging"

From: greywolf42 (mingstb_at_marssim-ss.com)
Date: 01/19/05 

Date: Wed, 19 Jan 2005 18:45:55 GMT

Tom Roberts <tjroberts@lucent.com> wrote in message
news:RinHd.10855$Vj3.10568@newssvr17.news.prodigy.com...
> greywolf42 wrote:
> >>Joseph Lazio <jlazio@adams.patriot.net> wrote in message
> >>news:llis6aghwh.fsf@adams.patriot.net...

> >>>This is Data Analysis 101. Let your detector be anything you want it
> >>>to be. Let it measure temperature on the sky, volts out of a
> >>>voltmeter, whatever. If you take a long data stream from it, you can
> >>>easily measure well below the "resolution" of the detector.

> >>> [later]

> >>>it is well known that one can make specific kinds of measurements
> >>>below the resolution limit of an instrument,
> >
> > Joseph, *why* do you keep repeating this silly statement? Many people
> > make such claims, but it is not valid science or statistics. You can
easily
> > show me wrong, by directing me to a statistics treatise on how to
> > perform measurements below the resolution of the instrument used.
>
> N.C.Barford, _Experimental_Measurements:_Precision,_Error,_and_Truth_.
> This is old and elementary, but it's what we used in the version of
> "Data Analysis 101" I took 30-some years ago.

Great! Now please provide the reference properly. Page number (or section)
where the text explains how this ability is derived. In other words, where
Barford *explicitly* explains how such methods can be used to go below the
physical resolution of the instrument.

An excerpt would be nice.

> I do not disagree with what Joseph Lazio wrote above.

I can't wait for the explanation! ;)

> But greywolf42's
> lack of knowledge and inability to read have apparently caused him to
> think otherwise. This is all well known, and is indeed "Data Analysis
> 101" -- greywolf42 explicitly displays his ignorance here.

The standard special plead, ad hominem.

> > Tom Roberts (and Bill Rowe), on the other hand, have many times called
> > such processes "overaveraging" (at least when it is applied to
experiments
> > that would otherwise disprove SR). i.e.:
> > http://www.google.com/groups?selm=vrd5jhksdt685b%40corp.supernews.com
> >
> > "And results reported implying an order of magnitude improvement in
> > resolution over the best the instrument can achieve are very dubious."
>
> Yes.

Well, this certainly looks different than your claim, above. In the link
above, you were complaining that Miller was providing a measured value of
0.24 fringe, when you agreed that the physical resolution of the device was
0.1 fringe. You were upset about the implication of the second digit.

In the above case, the intensity resolution of the COBE is 1 part in 10,000.
Yet the "variations" are given with an absolute value that is 10 times below
the resolution of the instrument. Which is equivalent to Miller declaring
that he had found a value of 0.024 fringe.

> A discussion:
>
> For a basic measurement like that of the width of my desk, a given
> technique has a given resolution.

And for a basic measurement like the width of a fringe, or the position of a
star image, a given technique has a given resolution. OK.

> For example this meter stick is marked
> in millimeters, and I can read it to about 0.2 mm resolution. So using
> it to make a single measurement of the desk, I obtain an answer accurate
> to ~0.2 mm.

For example, this interferometer is marked in fringes, and I can read it to
about 0.1 fringe resolution. Fore example, this astrometrical CCD is marked
in arc-seconds, and I can read it to about 3 milliarc second resolution.

> If I make a series of such measurements that are
> STATISTICALLY INDEPENDENT I can improve that accuracy to the limit of
> the systematic errors involved, by averaging multiple measurements.

1) Can you support this claim, instead of simply assert it?

Systematic errors do not affect the error bars on the statistical results.
If you know that there is a systematic error, then you redo the experiment.

> To
> make them statistically independent, in this case I must re-apply the
> meter stick to the desk for each measurement (merely re-reading the
> scale without repositioning the stick would not give independent
> measurements).

Yes, one must actually perform each measurement... not simply count the same
measurement 'n' times.

> As is well known, under these conditions, the mean of the
> multiple measurements approaches the actual value to within an error
> determined by the systematic errors combined with the intrinsic error of
> the meter stick (~0.2 mm) divided by the square root of the number of
> measurements contributing to the mean.

I don't care if you think that it is "well known." I'm looking for an
actual reference that this is part of physical, statistical theory.

And in Joseph's case, he would be measuring the width of paramecia to be
0.01 mm, using a meter stick. Do you think that this is valid? In the case
of the Hipparcos-light-bending crew, this would be claiming a result of
0.000013 +- .000002 mm (using the meter stick with resolution of 0.2 mm).
Is this valid, Tom?

> In this case, some of the systematic errors are:
> errors in scribing the marks on the meter stick

This isn't "systematic" error. This can be avoided by using a different
meter stick for each measurement, or measuring over different intervals.

> optical parallax

This isn't systematic error (the observer can move his eyes around).

> temperature difference in the meter stick between its calibration and use

This is not systematic error, for it can be controlled. Unless the
experimenter is not competent.

> It should be clear that none of these error sources are affected by
> averaging, and they are related to the meter stick's construction and
> manner of use.

Yes. And real systematic errors can't be quantified within the process of
the specific experiment.

> Now the manufacturer of the meter stick knows about these
> systematic errors, and does not make heroic efforts to reduce them below
> a human's ability to read and use it, so they are not enormously smaller
> than ~0.2 mm. That applies to essentially any instrument.

Yes. So your entire digression into systematic errors was a red herring.

> That's why
> averaging many readings is highly suspect when someone claims an
> improvement of an order of magnitude over the intrinsic resolution of
> the instrument.

So, I presume you would agree that claims to 1 part in 100,000 are "highly
suspect", when the intrinsic resolution of the instrument is 1 part in
10,000?

> [For instance, wear on the end of the stick can be comparable
> to that accuracy. That's why the 0 mark is not at the end.]
>
>
> In the measurments greywolf42 references above, on which I commented
> that they involved overaveraging, the experimenters claimed an
> improvement of more than an order of magnitude by averaging.

Which is fine by your method, above, so long as "systematic" errors are less
than the resolution of the instrument.

> None of
> them could claim their systematic errors were samll enough to justify
> that smaller resolution.

Why not, Tom? They didn't have "marking errors", "parallax errors", or
"temperature errors."

> Moreover, most of them had a clear human bias
> in roundoff, which makes multiple measurements be statistically
> correlated,

Please provide a sample of the data that supports your claim. (For example,
evidence of the "sawtooth" bias.) And a measurement of the statistical
bias..

> which means that averaging does not improve the actual
> resolution of the mean below the amount of roundoff.

No, systematic errors will not change the resolution of the instrument. Nor
will they change the resolution (precision) of the result. Systematic
errors will change the *accuracy* of the result. But this is simply bad
experimental design, and has nothing to do with the statistical "averaging"
process.

> For instance, if when reading that meter stick I always
> rounded up to the next millimeter,

Then you wouldn't have a theoretical resolution of 0.2 mm -- but only of 1
mm.

> it should be clear that
> the value I obtain will be larger than the actual value,
> and no amount of averaging multiple measurements will
> improve the accuracy of the measurement below ~0.5 mm.

But that would simply be a biased experimenter, Tom. Which has nothing to
do with averaging.

And you have nicely avoided the issue. When you use a meter stick that is
(theoretically) precise to 0.2mm, you don't select that instrument to
measure paramecia who's absolute diameter is on the order of 0.01mm. You
use a meter stick to measure objects with characteristic dimensions on the
order of several mm to 1 meter. You want 2 or possibly 3 significant
figures. In Miller's case, you claim his results were only 1 significant
figure, but he claimed two significant figures.

Now, in the Joseph's case, above, we are talking about effects similar to
measuring paramecia with a meter stick. The COBE resolution is 1 part in
10,000 at any given intensity. But the absolute value of the reported
results are 1 part in 100,000 from the background blackbody curve. 

--
greywolf42
ubi dubium ibi libertas
{remove planet for return e-mail}

Relevant Pages

  • Re: Roberts versus Lazio on "Overaveraging"
    ... but it is not valid science or statistics. ... >> perform measurements below the resolution of the instrument used. ... > For a basic measurement like that of the width of my desk, ... Systematic errors do not affect the error bars on the statistical results. ...
    (sci.physics)
  • Re: Roberts versus Lazio on "Overaveraging"
    ... but it is not valid science or statistics. ... >> perform measurements below the resolution of the instrument used. ... > For a basic measurement like that of the width of my desk, ... Systematic errors do not affect the error bars on the statistical results. ...
    (sci.physics.relativity)
  • Re: Roberts versus Lazio on "Overaveraging"
    ... but it is not valid science or statistics. ... > measurements below the resolution of the instrument used. ... the systematic errors involved, by averaging multiple measurements. ...
    (sci.astro)
  • Re: Roberts versus Lazio on "Overaveraging"
    ... but it is not valid science or statistics. ... > measurements below the resolution of the instrument used. ... the systematic errors involved, by averaging multiple measurements. ...
    (sci.physics.relativity)
  • Re: Roberts versus Lazio on "Overaveraging"
    ... but it is not valid science or statistics. ... > measurements below the resolution of the instrument used. ... the systematic errors involved, by averaging multiple measurements. ...
    (sci.physics)