Re: Pairwise group comparisons in SPSS

From: Jo Madden (jrm54_at_cam.ac.uk)
Date: 02/02/05 

Date: Wed, 2 Feb 2005 12:48:55 +0000 (UTC)

Thanks for those ideas Richard. I'm afraid that I'm a zoologist, not a
statistician, hence some inexact terminology. I'll try to clarify
things below in response to your comments.

On Tue, 01 Feb 2005 19:24:39 -0500, Richard Ulrich wrote:
>On Tue, 1 Feb 2005 12:42:48 +0000 (UTC), jrm54@cam.ac.uk (Jo Madden)
>wrote:
>
>> I aim to conduct a single stepwise DFA (neccessary for consistent
>> interpretation of results) across a large (n = 13) number of
groups. I
>
>I hope you mean, "necessary for being consistent with
>a previous *method* of interpretation of results...."

If I conduct a series of DFA between specific groups, I generate
different models, relying on different variables. This makes
biological interpretation difficult e.g. syllable duration is
important in one case but peak frequency is important in another.
Instead I should like to use a single model of discrimination to
separate all groups and place them in a 3d space. My idea is then to
use post hoc tests - apparently supplied by this pairwise distances
measure - to test if specific pairs of groups differ.

>
>"Stepwise" is famous for giving inconsistent results
>in simple regression; if you have 13 groups with
>varying Ns to screw up the testing, I would expect
>extraordinary inconsistency would be possible.
>

As ever, in biological field science, my sample sizes differ due to
mortality etc!

>If you want to compare across bootstrapping, you should
>probably mention that and ask for direct alternatives.
>

Could you briefly explain how bootstrapping in DFA works? I'm not
aware of this.

>
>> then wish to test for significant differences between certain pairs
of
>> groups i.e. does A differ from B in the DF model generated? In SPSS
>> v11 I find that there is a checkable option in the <<Stepwise
Method>>
>> menu for <<F for pairwise distances>>. This produces a grid of what
>> appear to be pairwise comparisons betweeen each group for each of
the
>> three steps in building the model. The statistics given are an F
and P
>> value.
>
>I think you are saying, above, that you stop with 3 variables
>in the model. Yes? And you are looking at F-tests that are
>described for each variable?
>

You're correct that 3 variables can enter the model and prevent any
others from making a significant improvement. What then follows is a
series of pairwise distances between each group at each step.
Therefore, for step 3 I am presuming that all three variables are
entered and so considered in the analysis. The df given for step 1 are
F 1,95; for step 2 F 2,94 and step 3 F 3,93, suggesting that in the
final step all three variables are considered.

>>
>> My questions: a) Is this an appropriate method to test for pairwise
>> differences between groups accounting for all three dimensions
>> (DF1-3); b)Am I correct in thinking that I pay attention to the
>> results from step 3, relating to the final model; c) What
assumptions
>> (if any) are made by this test;
>
>The assumptions made by the ordinary p-values for the
>F-test include the strong one, that *this* test is a single,
>important hypothesis (and not that this is one test of a
>dozen, or selected as 'best' of a set).
>

Each pairwise comparison is specifically testing a single hypothesis,
so the issue of multiple testing is not raised.

>
>> d) Are there any references or
>> published details of this test - I cannot find any on the SPSS help
or
>> online help.
>
>If you aren't talking about an ordinary F, I don't know
>what you are talking about. You could check my stats-FAQ
>for comments on Stepwise, which I gleaned several years
>ago from posts made to the stats-groups. Or you could
>google groups for < stepwise regression >.
>

Thanks for those ideas. I've tried the googling. Could you send me the
url of your stats FAQ which sounds very helpful. Thanks again for your
prompt reply and all your help.

Jo

>--
>Rich Ulrich, wpilib@pitt.edu
><a
href="http://www.pitt.edu/~wpilib/index.html">http://www.pitt.edu/~wpilib/index.html>

Quantcast