Re: power analysis: within-subjects, 2x2, random factors: How ?

From: Bruce Weaver (weaverb_at_mcmaster.ca)
Date: 06/28/04 

Date: Mon, 28 Jun 2004 11:11:57 -0400

Martin O'Hare wrote:
> Dear all,
>
> One of the reviewers of an article I submitted to a journal asked about some
> results of an experiment I reported as not reliable whether the design had
> enough power to detect such a difference, if one existed, and asked for a
> power test. I'm quite confused because I've read (and been convinced) that
> post-hoc power analysis is bogus.

Quite right. Here are a couple of articles that comment on
this (mind the line wrap):

   http://www.stat.uiowa.edu/~rlenth/Power/2badHabits.pdf
 
http://www.learnlink.mcmaster.ca/OpenForums/00031830-80000001/00040559-80000001/004E5CB8-00977198-005B1B1B

> On the other hand it seems that I have to
> do it (proposed interval analysis but the editorial board didn't get it/like
> it). I am not statistician and my knowledge of statistics is limited to the
> basics. I'm not sure how to do a power analysis with the design I have.
>
> I had about ~25 subjects (same number to other studies in the field
> examining similar stuff), all of them examined 100 items, DV was continuous,
> items were balanced across two binary 2-level IVs. Here are some more
> details:
>
> Factor 1 and Factor 2 were crossed in a 2x2 within-subjects design. The two
> within-subjects factors were Factor 1 (level 1 or level 2) and Factor 2
> (level 1 or level 2) and the analysis was carried out by-subjects (F1) and
> by-items (F2). For the by-subjects analysis Factor 1 and Factor 2 were
> treated as fixed factors whereas subjects was the random factor. For the
> by-items analysis, items nested within the compound type and familiarity
> conditions were the random factor.
> All terms and possible interactions between them were included in the model.
> General Linear Model's regression was used for the analysis of variance of
> means.
>
> What I should do ? I looked high and low for software and formulae. I found
> formulae for different designs but I find it difficult to adapt them to my
> problem and do not want to do something wrong or based on "intuition".
>
> Any help will be greatly appreciated !
>
> -- Martin

There's a paper by D'Amico, Neilands, and Zambarano (in
Behavior Research Methods, Instruments, & Computers,
2001, 33(4), 479-484) that shows how to do power analysis
using the MANOVA procedure in SPSS. I have a syntax file
with their examples here:

   http://www.angelfire.com/wv/bwhomedir/spss/power_analysis.txt

None of the examples match your scenario exactly, but I'm
sure you could make it work. 

-- 
Bruce Weaver
weaverb@mcmaster.ca
www.angelfire.com/wv/bwhomedir/

Relevant Pages