Re: T-test after ANOVA results

On Tue, 4 Mar 2008 06:31:17 -0800 (PST), Bruce Weaver
<bweaver@xxxxxxxxxxxx> wrote:

On Mar 4, 4:00 am, PaulP <folke...@xxxxxxxxx> wrote:
I think this question somehow have been answered earlier but this is with a comment:
After running an ANOVA on five different groups i did not find any significant difference between them.
However using a T-test on two of the groups this gave a significant difference.
Doing this often give false positives i have read/been told.
But! What if your groups have a gradual change in experimental conditions?
To the OP -

THEN - it would be "wrong" to do your important test
in at way that ignores the ordering - if you have any
need for statistical power to detect the difference.
(If your N is huge, so ANY test will show an effect, then
who cares which test? The effect size might be the important
question when the N is huge.)

PP > > Example:
You have 5 groups of plants that you give different concentrations of a fertilizer.
Group 1 no fertilizer, group 2 10%, group 3 20%... group 5 40%. And then you do an anova on the growth of these five groups. Giving no significant difference. When in fact group 1 and five are different you wont be able to detect it with an ANOVA. Maybe in this example a regression would be the best but in my case the experimental conditions can not be used in a linear regression.
So to conclude with a question:
Is it completely wrong to run a T-test on two groups that have recived the most different treatment after ANOVA have failed to show significant differences?

Doing "every possible test" is a way to fool yourself, but
it is also the practical way to learn how the tests behave.

If you have a 1-degree of freedom hypothesis, it is a waste
of power to rely on a 4 d.f. test.

Look at it this way:
Your ANOVA F test has a Sum-of-squares divided by 4 d.f.
If the effect is, indeed, linear, then the Linear SS will be 90%
or 95% as large as the 4-group SS -- but it will be divided
by 1 d.f. instead of 4 d.f. and the resulting F will be about
3 times as large as the other. That should overwhelm the
fact that there are different cutoff levels for the different df's.



BW >
For the situation you describe, polynomial trend analysis would be a
good approach.

And, the test that you are interested in is the test on the
1 d.f. for the Linear Trend component.

--
RIch Ulrich

http://www.pitt.edu/~wpilib/index.html
.

Relevant Pages

  • Re: ANOVA question
    ... p < 0.05 in ANOVA just says that at least one those ... LSD (Least Significant Difference) using the pooled error term. ... these careful tests to fail to show a difference, in follow-up, ...
    (sci.stat.math)
  • T-test after ANOVA results
    ... I think this question somehow have been answered earlier but this is with a comment: ... After running an ANOVA on five different groups i did not find any significant difference between them. ... Is it completely wrong to run a T-test on two groups that have recived the most different treatment after ANOVA have failed to show significant differences? ...
    (sci.stat.math)
  • Re: T-test after ANOVA results
    ... After running an ANOVA on five different groups i did not find any significant difference between them. ... Is it completely wrong to run a T-test on two groups that have recived the most different treatment after ANOVA have failed to show significant differences? ...
    (sci.stat.math)
  • TWo way ANOVA
    ... I did a Two way ANOVA for comparing 2 treatments with 3 stages. ... that i got a significant difference in the treatment. ...
    (sci.stat.math)
  • Re: Doesnt a t-test work here?
    ... significant difference somewhere when in fact there are none at all is ... t-test would be okay. ... different treatments all being compared to a control treatment. ... I know you guys are all statistics people so you will probably hate me ...
    (sci.stat.edu)