Re: Covariate in ANCOVA question

Ray Koopman <koopman@xxxxxx> wrote in
news:1193386099.129740.257200@xxxxxxxxxxxxxxxxxxxxxxxxxxxx:

On Oct 25, 7:31 pm, David Winsemius <doe_s...@xxxxxxxxxxx> wrote:
Bruce Weaver <bwea...@xxxxxxxxxxxx> wrote in
news:1193318943.008583.179600@xxxxxxxxxxxxxxxxxxxxxxxxxxx:

On Oct 24, 8:57 pm, David Winsemius <doe_s...@xxxxxxxxxxx> wrote:
Bruce Weaver <bwea...@xxxxxxxxxxxx> wrote
innews:1193226596.924562.47100@xxxxxxxxxxxxxxxxxxxxxxxxxxx:

--- snip ---

So because there are 3 groups we should not think about type I
erors in post-hoc testing? Why, then, do the texts I have talk
about controlling for alpha and use three group examples?

--
David Winsemius

Good question. I can only speculate about the reasons. Fisher's
LSD fell out of favour generally because it does not control the
family- wise alpha well when there are 4 or more groups. I think
that over time, we (collectively) have forgotten that it *does*
control family- wise alpha when there are 3 groups. Howell's book
is the only one I'm aware of that says this.

The only one, eh. Could that be because the logic is questionable?

I think you should go back to the argument you offered from Howell
and think more deeply. When you say he says "assume none of the mu's
are equal" and as a result says type I error is not possible...that
is a tautology rather than a statistical inference. When you _know_
there is a difference then the question is what is power of the test
to support that. The risk is a type II error. Type I errors occur
when there is no difference and one risks (falsely) concluding that
there is a difference.



Whatever its faults may be when there are more than 3 groups, the
Fisher LSD procedure does keep the actual familywise type I error
rate at its nominal level when there are 3 groups.


I offer this material as being clear and more convincing than earlier
efforts ...from:
<http://www.ppsw.rug.nl/~sda/siv/Posthoc_print.pdf>

---------begin quote----
One of the oldest methods for making post hoc pairwise comparisons is the
LSD procedure of Fisher. It is also known as Fisher's protected t. The
procedures consists of two steps:
1. Perform an ordinary ANOVA F test using significance level alpha_FW
(the so-called omnibus F test). If the test is not significant, none of
the pairwise comparisons is significant.
2. If the omnibus F test gives a significant result, all pairwise
comparisons are made using ordinary t tests (for equal variances) at
significance level alpha_C=alpha_FW.

This procedure very simple and therefore attractive. Moreover, in step
two there is no correction of the level of alpha_C, which means that the
critical values of the second stage tests are smaller than those of other
procedures. However, the protection by the omnibus test in step one only
is complete if the null hypothesis is true, that is, if all group means
are equal. In situations where part of the null hypothesis is true the
omnibus test fails to protect that part of null hypothesis that is true.
For example, in the situation with k = 4 and mu1 = mu2 < mu3 < mu4. In
this situation, the F test will reject the null hypothesis of equal group
means with a large probability, even approaching 1 (especially for large
group sizes), whereas you do not want to conclude that mu1 and mu2 are
different.
Only if k = 3 does the procedure give full protection. This is obvious
in the situation where the null hypothesis is true (as argued above). If
the complete null hypothesis is not true but a more limited one is, there
can only be one null difference among three means and, therefore, only
one chance of making a Type 1 error (with probability alpha). Therefore,
the LSD procedure should not be used in experiments with k >= 4.
------end quotation--------

--
David Winsemius
.

Relevant Pages