Re: Bootstrapping Hypothesis Test

From: Lisa Bloomer (lbloomer_at_mtsu.edu)
Date: 09/23/04 

Date: Thu, 23 Sep 2004 16:53:44 +0000 (UTC)

No, I really am talking about bootstrapping.

Here's the situation. A colleague and I have developed a test
statistic to use in a resampling test to test for the equality of
certain data sets. On the actual data we have, the test seems to work
well compared to tests currently in use. We also ran simulations
using simulated data that satisfied the null hypothesis and data that
did not. The histogram of our p-values (under the null) was close to,
but not quite equal to the uniform distribution.

I think the problem is that this is an asymptotic property, and I'd
like to prove that if we had a larger data set, we would be getting
uniform values. But I want to make sure that our test statistic is
not causing the problem. It's a quadratic form, and so I should be
able to prove that it has nice properties. But I don't know which
properties I need, exactly.

Thanks,
Lisa Bloomer

On Thu, 23 Sep 2004 01:00:55 -0400, Richard Ulrich wrote:
>
> - asking for clarification -
>
>On Wed, 22 Sep 2004 18:51:34 +0000 (UTC), lbloomer@mtsu.edu (Lisa
>Bloomer) wrote:
>
>> Hi,
>>
>> I need to know what characteristics a test statistic has to satisfy
>> for the distribution of the bootstrapped estimate of the p-value to
>> converge to the uniform distribution.
>>
>> Does anyone have a reference for me to an article that might deal
with
>> this?
>
>Bootstrapped? Are you sure? Or, could you be confusing
>two of the computer-intensive methods....
>
>What you get from *bootstrapping* depends largely on
>the fixed sample that you start with, and not on the test
>statistic. Looking at their p-values also seems odd.
>
>It is a lot easier for me to make sense of the question
>if you are asking about the results of Monte Carlo
>estimation. For example,
>
> - What allows a Monte Carlo estimator of a statistic
>to result in p-values that are uniform?
>
>
>--
>Rich Ulrich, wpilib@pitt.edu
><a
href="http://www.pitt.edu/~wpilib/index.html">http://www.pitt.edu/~wpilib/index.html>

Relevant Pages

  • Re: Interpreting the test result on a RNG
    ... To check whether the KS p-values are good I ... I expect that real RNGs will tend to fail one particular ... a couple single-test KS scores are quite small. ... smallest few values from a uniform distribution?" ...
    (sci.crypt)
  • Re: Bootstrapping Hypothesis Test
    ... > converge to the uniform distribution. ... What you get from *bootstrapping* depends largely on ... Looking at their p-values also seems odd. ... if you are asking about the results of Monte Carlo ...
    (sci.stat.math)
  • Re: random data
    ... Bryan Olson wrote: ... > The Kolmogorov-Smirnov test is often a good way to combine the ... all p-values from all kinds of tests in it for doing one ... the uniform distribution)? ...
    (sci.crypt)