Re: Trying to find significant factors in experimental results

On Thu, 10 Apr 2008 14:27:21 -0700 (PDT), Rob <rtshilston@xxxxxxxxx>
wrote:

Hi,

I've asked a 100 subjects to assess a quantity. For simplicity,
assume I've asked them to estimate the length of a piece of wood.
I've also asked each subject a number of questions (sex, age, whether
they're short sighted or long sighted, if they were glasses, contact
lenses or nothing).

My hypothesis is that the results are independant of all question
answers.

Sometimes, these examples "for simplicity" turn
out to be misleading. The standard hypothesis
has to do with an interesting *difference* or effect,
so there is an extra problem of trying to figure if
you really want to reverse things, or if you just asked
the question in a clumsy way.

But I will take the question as it is asked.

Entertaining the hypothesis of no-difference usually
requires much larger samples than other experiments
with the same factors and outcomes. That is because
you can *never* establish that there is no difference
at all - which is almost impossible, philosophically,
for observational data - You can only establish that the
difference is too small to be interesting.

This is what is done in bio-equivalence studies, which
you might want to look up, for comparison.

With multiple factors, you are stuck with providing
confidence limits for each of the observed differences,
and then arguing that they are each small. If there is
any suggestion that these might combine ("bad vision,
no glasses"), you also need to discredit those things
that anyone else might argue for.


How can I prove this? As a first attempt, I've partitioned the
subjects into every possible partition, based on the question answers,
where I've still got a minimum of ten subjects in the smallest
population. I then ran ANOVA for the two formed populations, and
repeated for every possible partition.

This strikes me as being a bit clumsy, but I'm not sure how else I can
do this. My reading of Factorial ANOVA suggests that every population
needs to be of the same size, and so this isn't possible with my data.

That is surely not the case. The most powerful
tests have equal group sizes; and cross-classifications
need cell sizes that are proportionate if the tests are
to remain independent and "unconfounded" with each
other. There still can be tests.


Can anyone point me in the direction of a technique that could be used
to analyse my data?

If you don't want separate confidence intervals for
every difference that anyone might argue for, then
you may need to be more explicit about the variables
and the problem.

--
Rich Ulrich

http://www.pitt.edu/~wpilib/index.html
.

Relevant Pages