Re: Effect Size and Sample Size

On Tue, 8 Jan 2008 23:16:26 -0800 (PST), beginner1.mat@xxxxxxxxxxx
wrote:

I would like to compare 2 groups (treatment and control) using a t-
test. Due to expense, I will only be able to perform a small test on
a total of 4 (2 treatment and 2 control). Would calculating sample
size based off of the effect size calculated from this small test be
valid (difference between groups/pool sd)? Is there a rule for the n
needed to be able to calculate the effect size for sample size
estimates?

Requirements on Ns are generally "rules of thumb" that
apply in a particular area, and not hard-and-fast rules;
unless you are talking about terms that become undefined
(see below).

Doing a t-test of N=2 vs. 2 is something that I've thought
about a few times, but I don't think I've ever done it, or
even seen it done. However, I don't read papers on the
preliminary sort of laboratory experiments where that would
arise. Years ago, my lab friends who used tiny samples, and
whom I asked about it, said that they did not bother about
t-tests, like, "a huge effect either exists or it doesn't, and
the replication is to make sure than you didn't screw up
the lab procedures." I don't know if that is still the same.

So - if you are working in an area when they would accept
or expect a t-test on Ns of 2, maybe folks would be happy
to see an "effect size", too. You do want to work with
scores that are figured and written out with enough precision
that you are not screwed up by round-off error, for both the
t-test and the effect size.

"Valid" is an interesting word to use here -- The estimate
would not be as reliable a number (precise estimate),
compared to those with much larger Ns. With 2 d.f., I
*think* there is nothing to say that it is not a "valid estimate"
in the technical sense; though I believe that with 1 d.f., the
variance of the estimate itself would be undefined. That
1 d.f. test would allow for a computable "effect size" that
would arguably be "not-valid".

I don't recall seeing these issues discussed, so I won't feel
bad if someone else chimes in.


--
Rich Ulrich, wpilib@xxxxxxxx
http://www.pitt.edu/~wpilib/index.html
.