Re: How many data points for statistical significant results?
- From: Paige Miller <paige.miller@xxxxxxxxx>
- Date: Mon, 8 Sep 2008 12:43:25 -0700 (PDT)
On Sep 8, 2:59 pm, Dave <f...@xxxxxxx> wrote:
I need to make some measurements of a signal and would expect around a
30 dB variation between maximum and minimum. I've been asked to provide
the minimum, maximum, mean of the measurements. Also the 5th percentile.
The results have to be "statistically significant", but what is
statistically significant has not been defined.
My guess is to to provide the 5th percentile, I'll need an absolute
minimum of 100/5 = 20 measurements. Does that seem reasonable? Trying to
interpolate with say 10 measurements would I suspect be a bit dubious.
But what can be said about the significance of the 5th percentile if I
make n measurements?
Would it be sensible to do a sufficiently large number of measurements
so I could say something like "We can be 90% sure the 5th percentile
will be in the range X to Y"?
Maths is not my strongest subject, so I am looking for some guidance on
what would be a sensible number of measurements to take to provide
someone with a meaningful value for the 5th percentile.
If I took say 200 measurements, what could I say about the calculation
of the 5th percentile?
I'm aware that the definition of percentile is not well defined.
I don't believe the person asking for this information is an expert on
statistics, so I think they would be looking for some guidance from me
on how many measurements I think is reasonable.
It goes without saying that the time to take the measurements is not
short, so I don't want to do 20,000 if 100 would be reasonable.
Dave
First of all, you don't have good definitions. I think you probably
won't ever get very far unless you have good definitions. And the
problem is not that percentile isn't well defined, the problem is that
"statistically significant" isn't well defined.
In particular, you need to know what the words "statistically
significant" mean to the person making the request. In statistics,
they refer to an hypothesis test, and I don't see that you have such a
thing here. To define an hypothesis test, you want to test that the
mean (or median or standard deviation or 5th-percentile or whatever)
is equal to a specific value. I don't see that here. Alternatively,
you might want the width of an X% confidence interval for the mean (or
median or standard deviation or 5th-percentile or whatever) to be
smaller than a given quantity. I don't see that here either.
If someone makes a request of you in the language of statistics, you
and your requester have to use the language and mechanics of
statistics throughout the problem definition phase and throughout the
problem execution phase, or you have to translate some words that are
not statistical into words that mean something statistically. Asking
for something to be "statistically significant" without additional
information that statistics requires, is an undefined problem and
cannot be solved. To perform statistics properly, you will need to
consult with the person who made the request to obtain a much better
idea about what he is looking for and why, so that his request will
make sense and be defined in a statistical way.
--
Paige Miller
paige\dot\miller \at\ kodak\dot\com
.
- Follow-Ups:
- References:
- Prev by Date: How many data points for statistical significant results?
- Next by Date: Re: Improving Lilliefors Test
- Previous by thread: How many data points for statistical significant results?
- Next by thread: Re: How many data points for statistical significant results?
- Index(es):
Relevant Pages
|
