Re: SPSS power calculation with MANOVA (was: sample required for 2x2 factorial design...)

From: Bruce Weaver (bweaver_at_lakeheadu.ca)
Date: 11/16/04 

Date: Tue, 16 Nov 2004 15:32:13 -0500

[ cross-posted to comp.soft-sys.stat.spss ]

Richard Ulrich wrote:

> On Mon, 15 Nov 2004 15:31:12 -0500, Bruce Weaver
> <bweaver@lakeheadu.ca> wrote:
>
>
>>Richard Ulrich wrote:
>>
>>-- snip --
>>
>>> - A tool like the SPSS estimator of power can be very
>>>misleading about power when the d.f. in the example is
>>>small -- Unless that has been changed in recent years.
>>>I posted about this a few years ago in a discussion in
>>>the comp.stat-sys.spss group, with examples.
>>>
>>
>>Is this the post you're referring to, Rich (mind the word-wrap)?
>>
>>http://groups.google.ca/groups?q=Richard+Ulrich+power+SPSS&hl=en&lr=&selm=5imb4d%24qhe%40usenet.srv.cis.pitt.edu&rnum=1
>>http://groups.google.ca/groups?selm=5imb4d%24qhe%40usenet.srv.cis.pitt.edu&output=gplain
>
>
> Yes, that's one. The shorter reference gets article by itself (How do
> you do that?) and the longer one links to the original thread.

I got the shorter URL by clicking on "View original format" in Google
Groups.

> The SPSS code was generating random data as if it were 'pilot data',
> and then showing that the SPSS procedure reports very high power
> for a very-non-significant result.
>
> This seeming-paradox is a consequence of how the 'effect sizes'
> do not take chance into account. You will *observe* a large
> R-squared with many variables and few cases; an *underlying*
> R-squared that is large will have pretty good power -- so that
> is what SPSS reported. David Nichols said then, other programs do
> the same, which is a reason to be careful about using any of them.
>
> That was 1997. I haven't looked at what SPSS does now.
> Nobody mentioned any changes in the basis, that I recall.
>

Interesting. Looks like we should be having this (re-)discussion in
comp.soft-sys.stat.spss. Maybe we can get some commentary from the
current SPSS tech support folks. Are you there, Jon Fry?

By the way, I had to tinker with the original syntax a bit to make it
run in version 11 under XP Professional (and I added a call to MEANS in
the macro to display the cell means). Here is the revision:

* Rich Ulrich's original syntax with modifications to run in SPSS 11
under WinDoze XP .

TITLE random data, N=50
set seed=7000111
/* the Macro is defined once, so that one line can invoke the whole set.
define get50()
new file.
input program.
loop id=101 to 150.
+ do repeat dum=a1 to a6.
compute dum=normal(10).
end repeat.
end case.
end loop.
end file.
end input program.
formats id(f5) a1 to a6(f5.2).
rank variables= a6 /percent into grp.
compute grp= trunc((grp-.1)/10). /* make GRP= (0,9) .

manova a1 to a5 by grp(0,9)
  /print=signif(hypoth)
  /power=F(.05) .
means a1 to a5 by grp /cells = mean.
!enddefine.

get50.
get50.
get50.

* ----- End of syntax ------ . 

-- 
Bruce Weaver
bweaver@lakeheadu.ca
www.angelfire.com/wv/bwhomedir