Re: Simulation Process and Independent Vs Paired Samples

On Sat, 26 Apr 2008 16:49:47 -0400, Paul Rubin <rubin@xxxxxxx> wrote:

Hanspeter wrote:
I have a question about Simulation Process and Independent Vs Paired
Samples.

Given two related samples, in order to enhance the sensitivity of t-
test,
1 sample t-test should be applied to the two samples differences (null
hypothesis = the differences average is statistically zero), instead
of
applying 2 sample t-test to both the samples separately (null
hypothesis =
both the samples come from the same population).
Simulations made to decide whether to modify or not a certain device,
yield paired (=related) samples (i.e.: before- and after-modification
measurements) spontaneously, however.
Sometimes it happens that the two samples differences result
significantly different from zero (i.e.: 1 sample t-test is
significant),
while both samples seem coming from the same population (i.e.: 2
sample
t-test is not significant).
In these cases, I do not want to modify the device under test because
I am
not able to reject the null hypothesis of the 2 sample t-test. The
"paired
t-test" (i.e.: 1 sample t-test applied to the two samples differences)
suggests to modifying the device under test, however.

What is the proper treatment of paired samples resulting spontaneously
from simulation?


If the simulation generates pairs of observations (one from each
population), and if the paired observations are correlated to each
other, then the paired-difference t-test should be more accurate. The

I would go further, and state that if the observations
are correlated, the grouped test is *wrong* -- the
assumption of independence was not met, and the
question then is how much it matters. If you get
different results, then it apparently matters.


two sample t-test calculates the variance of the difference in means
under the assumption that the samples are mutually independent, and will
overestimate the variance of the difference in means (reducing power) if
there is a positive correlation between pairs of observations.

Similarly, a negative correlation will cause the Student's
test to underestimate the variance, and yield too much
power. (Someone once posted a related question concerning
a forced-choice design of this sort, where Left+Right < or = 10.)


--
Rich Ulrich

http://www.pitt.edu/~wpilib/index.html
.

Relevant Pages

  • Simulation Process and Independent Vs Paired Samples
    ... I have a question about Simulation Process and Independent Vs Paired ... sample t-test should be applied to the two samples differences (null ... Simulations made to decide whether to modify or not a certain device, ... Sometimes it happens that the two samples differences result ...
    (sci.stat.math)
  • Simulation Process and Independent Vs Paired
    ... I have a question about Simulation Process and Independent Vs Paired ... sample t-test should be applied to the two samples differences (null ... Simulations made to decide whether to modify or not a certain device, ... Sometimes it happens that the two samples differences result ...
    (sci.stat.edu)
  • Re: Simulation Process and Independent Vs Paired Samples
    ... sample t-test should be applied to the two samples differences (null ... Simulations made to decide whether to modify or not a certain device, ... If the simulation generates pairs of observations, and if the paired observations are correlated to each other, then the paired-difference t-test should be more accurate. ...
    (sci.stat.math)