Re: Two nit-picks re definition of p-value (Was: goodness of fit ?)


David Winsemius wrote:
"Kevin E. Thorpe" <kevin.thorpe@xxxxxxxxxxx> wrote in
news:1157196518.099870.57280@xxxxxxxxxxxxxxxxxxxxxxxxxxx:


David Winsemius wrote:
--------definitons from math stats texts------------
Cox and Hinkley says a "level of significance" p_obs is defined as:
p_obs= Pr(T >= t_obs;H0)

Kalbfleisch uses significance level as
SL == Pr(D>= D_obs|H0)

DeGroot says p-value is sup(Pr(T >= t|theta))

As I read the three authorities above, they agree with RFB's
disputants who would use the language "equal to or more extreme
than".

Freund says that the [left] critical region of size alpha/2 is
X <= K_alpha ; where K_alpha is the largest integer for which

sum(Pr(Bin(y;n,theta)) <= alpha/2

As I read Freund, he also disagrees with RFB, because the first
integer for which sum Pr(...) is greater than 0.025 in the series
above is X = 3, so X=2 is in the critical region (where p-value
<0.025).

RFB> You learn it from the correct definition or you don't.

But first we need to settle on an agreed definition. As I read them,
all of the mathematical statistics texts I had on my bookshelf
disagree with your definition which disallows the equality in looking
at the exclusion of the null. I am wondering your focus on the value
X=2 is disturbing your concentration.

Thanks for posting these definitions David. I was looking at some
of my books last night as well (this is one of the good parts about
these discussions).

I think that what we are seeing here is the difference between
Fisher's approach to SIGNIFICANCE TESTING (as Cox and Hinkley
call it) and NP HYPOTHESIS TESTING.

Reef Fish has stated that his training is in the NP theory.
One of my books that is also NP based clearly uses strict
inequality in it's comparison between critical value and test
statistic.

In the Fisher significance testing approach, the SL is indeed
"as extreme or more extreme." What we have today often is
a blending of, at times, incompatible concepts. If you look
at the chapter of Cox and Hinkley called "Pure Significance
Tests" you will see that the Ha does not enter the
discussion. It does however in the next chapter which is
about hypothesis testing. I should note that Cox and
Hinkley do not champion one approach over the other,
they are simply giving a complete account.

To the pure NP hypothesis tester, the Fisher approach is
WRONG and vice versa. It is the same battle Fisher and
Neyman waged for years and one not likely to be
solved in sci.stat.math.

RFB isn't talking to me any more.

Thank you for the curtesy of accepting my wishes.

This is merely a correction of your MISATTRIBUTION of what I had
posted, for the benefit of Kevin Thorpe and other readers.

Since he has banished me from his
lecture hall and sent me over here, would it be OK to continue a
discussion on these points? My understanding was that there was really no
mathematical contradiction between N-P hypothesis testing and
formulations that used a significance level or Fisherian p-values. Neyman
used the term significance level, himself. My understanding was that
either would arrive at the same decision. When the p-value for a test
statistic was less than alpha in (or alpha/2) then a hypothesis test
would give the same "verdict" as would Fisherian advice about a test of
the null hypothesis. I _thought_ this had been established on reasonably
firm mathematical grounds and that there was general acceptance that the
Fisher.v.N-P wars were mathematically moot. I understand that (sometimes
intense) philosophical rangling persists. I was not attempting to start
that war again.

The problem that started this was posed by Weaver as:
---------Weaver's original nit #2---------
2. Isn't it really "EQUAL TO or greater than" rather than "greater
than"? I know that for test statistics that have continuous sampling
distributions, the difference is trivial. But not so for those with
discrete sampling distributions. Here's a binomial problem, for
example, with a non-directional alternative hypothesis:

X = number of successes in N = 13 trials
p = p(success)
q = 1-p = p(failure)

H0: p EQ 0.5
H1: p NE 0.5

Observed X = 2

X p(X|H0)
---------------
0 .0001
1 .0016
2 .0095 <-- observed X
3 .0349
4 .0873
5 .1571
6 .2095
7 .2095
8 .1571
9 .0873
10 .0349
11 .0095
12 .0016
13 .0001
---------------

I (Weaver) was taught to include the 0.0095 when computing the p-value:

p = (0.0095 + 0.0016 + 0.0001)*2 = 0.0224

Do you (RFB) agree? Thanks for clarifying.
---------end original Weaver problem----------

So that was posed as a standard two-sided test. Nothing particularly
fancy. Evidence was offered that three standard statistical packages
tested arrived at Weaver's conclusion. Evidence was offered that
mathematical statistics texts dealing with discrete RVs would include the
0.0095. No evidence (other than RFB's assertions) has been provided
against it. RFB has claimed that some sort of ambiguity in the
construction of the alternative hypothesis allows him (and apparently him
alone) to exclude the contribution of the probability of an observed
value from calculating a p-value.

RFB argued that the 0.0095 needs to be excluded because of a strict
equality in (his) definition of the problem. I do not think he actually
offered an answer to _what_ the correct p-value would be.

I corrected that 0.0095 should be excluded because "2" was not "more
extreme" than "2".

I had also given the obvious arithmetic of 2(.0001+0.0016) = 0.0034 as
the correct p-value.

-- Reef Fish Bob.


Would he
include the pdf contributions of the X=11, since 11 was not observed, but
would be in the right tail of the distribution and contributing to
evidence against H0: P= 0.5? Would his answer have been:

<possible RFB answer> p-value= 0.0095 + (0.0016 + 0.0001)*2 = 0.0127

It was pointed out that if X were observed to be zero, then RFB would be
forced to return a p-value of ZERO. Even this absurdity will not move him
of his position. He still persisted, despite being chided for being in
the ZERO Probability Zone.

Do you understand how he can do this with a straight face? Would Fisher
have understood it? Would Neyman have understood it? Does anybody else
understand it?

--
David Winsemius

.

Relevant Pages
Loading