Re: Bootstrapping for confidence interval of effect size

Thanks for those points, Rich. I see what you mean about the ceiling
effects and I will think about that.

Cheers,

Ben

On Jul 31, 2:33 am, Richard Ulrich <Rich.Ulr...@xxxxxxxxxxx> wrote:
On Mon, 30 Jul 2007 10:05:58 -0000, amorphia <spam.onto...@xxxxxxxxx>
wrote:

Hi all,

Thanks for your reply Rich. I take your point about how you can't
compare a confidence interval to a non-constant mean. However, I have
seen it written in many places that comparing a confidence interval to
a constant is equivalent to a hypothesis test.

- Right. You can create a CI by inverting the test.

So it seems to me that
logically there ought to be some way to compare two confidence
distributions (which is what you get with a bootstrap).

Well, yes, but not efficiently, and only if they are independent.



For example, you could compare every point (Ai) in bootstrap
distribution A with every point (Bi) in bootstrap distribution B, and
count how often Ai > Bi. That would give your your confidence that A
is greater than B, right? It seems to me that it is exactly the same
logic as comparing against a fixed constant.

Do it that way. That gives a better limit. Complications that
arise are most likely with highly unequal Ns and variances,
and I really don't know what is assured for that sort of case.



Your point about how the control groups are not equal doesn't apply -
I think you missed a point in my original point - I am comparing the
effect sizes of the two comparisons, not the means. There may be
something wrong with that aspect of what I did but it isn't what you
said! ;)

Sorry, but I still think you are missing a basic point of experimental
design. There are three potential criteria for what group ends
up better -- which has the better final score, which has the better
change score, or which has the better *regressed* change score.
These are not assured to be in agreement, if the groups are not
comparable at their starting scores,

These especially can arise when there are scaling problems,
such as 'basement' and 'ceiling' effects. It is nice to assure
the audience that such problems won't effect the analysis or
conclusions.

There are separate problems that can arise in deciding what is
the appropriate regression for the change, in special cases.



By the way, I have always in the past transformed my data too, but
this data is just untranformable - you would see what I mean if you
saw it.

[snip, previous]

--
Rich Ulrich, wpi...@xxxxxxxxxxxx://www.pitt.edu/~wpilib/index.html


.

Loading