Re: Explaining significance to laypersons: why the tails?

In <1116347367.023147.263230@xxxxxxxxxxxxxxxxxxxxxxxxxxxx> clemenr@xxxxxxxxxx writes:


>bill wrote:
>> I am having a hard time coming up with a layperson-friendly and at
>> the same time sufficiently convincing rationale for why statisticians
>> use the tails of distributions (especially in the case of discrete
>> distributions) to assess statistical significance.
>>
>> I can explain the idea of wanting to distinguish those observations
>> that are likely the result of chance fluctuations from those that
>> aren't. I can also explain the basic notion of a null hypothesis,
>> and its key role in the process of assessing significance.
>>
>> But I get tongue-tied when I try to explain why we rule out the
>> null hypothesis when we determine that, if it were true, then the
>> observed data ***and anything more "extreme"*** would be very
>> unlikely to occur.

>Disclaimer: I'm no expert statisticial but that rarely shuts me up.

>A good example is looking at the binomial distribution with large N and
>moderate p. For example, N=2000 and p=0.5. The most likely result of a
>random sampling from this distribution is x=1000, which has a
>probability of 0.01783901. Assume that this distribution is our null
>hypothesis and that it is true. I.e. that we are sampling from this
>distribution. Hence, if we took a stereotypical confidence interval of
>95% for rejecting the null hypothesis, and didn't sum the probabilities
>for the result of our experiment plus anything more extreme, then no
>matter what result we got, we would reject the null hypothesis.

>But, to address your question, it needs to be put into layman's
>language. Perhaps you could use the flipping of a fair coin 2000 times.
>No possible number of heads has a chance of more than 0.01783901 of
>coming up, so if we just the probability of that result and compared it
>to our significance level of 0.05, we could never ever conclude that
>the coin was fair.

>Is that somewhere in the ballpark?

Halfway there! I was about to reply that this argument would not
do because it relies on an arbitrary significance level of 0.05,
which does not mean much to the layperson; he/she would simply
counter "just redefine your cutoff for significance to something
well below 0.01783901".

But as I was writing my reply to you it hit me: a reasonable (perhaps
*the* most reasonable) criterion for choosing such a cutoff would
be based on the *fraction* of all possible outcomes that it will
classify as "too unlikely," which immediately leads one to consider
the tails of the distribution.

In other words, the tails enter the picture when we confront the
problem of coming up with a rational basis for *choosing* a specific
cutoff for significance!

I think I can work with this.

Thanks!

bill

.

Relevant Pages

  • Re: Testing for product of Gaussians
    ... but rejected it out of hand based on my limited knowledge ... distribution you are expecting, and sounds sufficient for my needs. ... And when the statisticians of old were devising out ...
    (sci.math)
  • Re: Explaining significance to laypersons: why the tails?
    ... > the same time sufficiently convincing rationale for why statisticians ... A good example is looking at the binomial distribution with large N and ... Perhaps you could use the flipping of a fair coin 2000 times. ... so if we just the probability of that result and compared it ...
    (sci.stat.math)
  • Re: Dynamical Systems and Expansion-Contraction
    ... >>From Osher Doctorow ... academic statisticians) point out in one of their early 1970s volumes ... bivariate normal/Gaussian distribution with correlation coefficient not ...
    (sci.physics)
Quantcast