Re: Estimation of variance of proportions

David Winsemius wrote:
=?UTF-8?Q?Jean-No=C3=ABl?= <aubertot@xxxxxxxxxxxxxxx> wrote in
news:11931925.1151133920157.JavaMail.jakarta@xxxxxxxxxxxxxxxxxxxxxx:

Hi everyone,
I would like to estimate the variance of estimated proportions. I
think that it is generally correct to estimate the variance by
VAR^=np^(1-p^), where p^ is the estimated proportion and n the sample
size. However, when p^=0, this leads to an estimated variance VAR^=0,
which puzzles me because the variance does not depend anymore on n. It
is quite different to obtain p^=0 with n=2, or with n=100000 for
instance. Does anyone know how to estimate the variance of a
proportion when the estimated proportion is null ?

Thank you in advance for your help.

JN


If you _know_ that p or (1-p) is zero, then the variance _is_ zero. If you do not know that p=0, then why would you torture the formula into giving you a meaningless estimate?

If you think that the proportion of successes, hits or events is not necessarily zero, then you should not use zero, but rather a small non-zero estimate. In that instance, the Poisson distribution and associated methods might be mathematically convenient. The variance of the Poisson equals the expected value which is np. The reasonableness of the Poisson approximation (with large n) to the binomial is easy to see, since np(1-p) will approach np because (1-p) is near 1.

And then you plug in p=0... :-)

One can calculate the 95% confidence interval for such a proportion by plotting the likelihood against p, and then finding the 95% quantile (the lower limit will be zero: you can calculate a symmetric CI, but it doesn't make sense as it doesn't include the point estimate). If the proportion comes from a series of Bernoulli trials, then the likelihood is a Beta distribution, so a good stats package will be able to calculate it.

Bob

--
Bob O'Hara
Department of Mathematics and Statistics
P.O. Box 68 (Gustaf Hällströmin katu 2b)
FIN-00014 University of Helsinki
Finland

Telephone: +358-9-191 51479
Mobile: +358 50 599 0540
Fax: +358-9-191 51400
WWW: http://www.RNI.Helsinki.FI/~boh/
Journal of Negative Results - EEB: www.jnr-eeb.org
.

Relevant Pages

  • Re: Estimation of variance of proportions
    ... I would like to estimate the variance of estimated proportions. ... of a proportion when the estimated proportion is null? ... necessarily zero, then you should not use zero, but rather a small ... One fairly widely known quick approximation for the upper 95% confidence ...
    (sci.stat.math)
  • Re: Unusual formulae for confidence intervals
    ... Ray Koopman wrote: ... For a population proportion, use as the standard error 0.5 / ... estimate of the variance. ...
    (sci.stat.math)
  • Variance of an index of dispersion
    ... I wish to test a hypothesis that neighborhood organizations are more ... the squared proportion of each racial group and substracts that figure ... variance, ... and calculates the proportion of ties ...
    (sci.stat.consult)
  • Re: go directly to jail
    ... fourth or fifth power of the variance in axle loading, ... vehicles are responsible for a much larger proportion of the road ...
    (rec.arts.sf.fandom)
  • Re: I can now run EMC2 and it kind of "works"
    ... Proportion is proportional to the error. ... Derivative is the rate of change, If your cruise control is set for 60MPH ... Derivative gain, Kd, ... hit the cruise control, Proportional error is zero, Derivative is zero, ...
    (rec.crafts.metalworking)