Re: Two nit-picks re definition of p-value (Was: goodness of fit ?)


David Winsemius wrote:
--------definitons from math stats texts------------
Cox and Hinkley says a "level of significance" p_obs is defined as:
p_obs= Pr(T >= t_obs;H0)

Kalbfleisch uses significance level as
SL == Pr(D>= D_obs|H0)

DeGroot says p-value is sup(Pr(T >= t|theta))

As I read the three authorities above, they agree with RFB's disputants
who would use the language "equal to or more extreme than".

Freund says that the [left] critical region of size alpha/2 is
X <= K_alpha ; where K_alpha is the largest integer for which

sum(Pr(Bin(y;n,theta)) <= alpha/2

As I read Freund, he also disagrees with RFB, because the first integer
for which sum Pr(...) is greater than 0.025 in the series above is X = 3,
so X=2 is in the critical region (where p-value <0.025).

RFB> You learn it from the correct definition or you don't.

But first we need to settle on an agreed definition. As I read them, all
of the mathematical statistics texts I had on my bookshelf disagree with
your definition which disallows the equality in looking at the exclusion
of the null. I am wondering your focus on the value X=2 is disturbing
your concentration.

Thanks for posting these definitions David. I was looking at some
of my books last night as well (this is one of the good parts about
these discussions).

I think that what we are seeing here is the difference between
Fisher's approach to SIGNIFICANCE TESTING (as Cox and Hinkley
call it) and NP HYPOTHESIS TESTING.

Reef Fish has stated that his training is in the NP theory.
One of my books that is also NP based clearly uses strict
inequality in it's comparison between critical value and test
statistic.

In the Fisher significance testing approach, the SL is indeed
"as extreme or more extreme." What we have today often is
a blending of, at times, incompatible concepts. If you look
at the chapter of Cox and Hinkley called "Pure Significance
Tests" you will see that the Ha does not enter the
discussion. It does however in the next chapter which is
about hypothesis testing. I should note that Cox and
Hinkley do not champion one approach over the other,
they are simply giving a complete account.

To the pure NP hypothesis tester, the Fisher approach is
WRONG and vice versa. It is the same battle Fisher and
Neyman waged for years and one not likely to be
solved in sci.stat.math.

--
Kevin E. Thorpe
Assistant Professor, Department of Public Health Sciences
Faculty of Medicine, University of Toronto

.

Relevant Pages