Re: interpreting poll margin of error?

From: Art Kendall (Art_at_DrKendall.org)
Date: 10/23/04

  • Next message: shanky: "Need help with deciding sample n sub sample. size for nested design(DOE)" 
    Date: Sat, 23 Oct 2004 14:53:30 GMT

    Stan makes a vital point about extra-statistical considerations.

    The total uncertainty about the meaning of a statistical result is made
    up of uncertainty due to sampling, and the "big sources" of uncertainty.

    The perceived or actual polemic position (often misnamed rhetorical
    position) of the sponsor is one, but since pollsters live in glass
    houses, and because many sponsors want as objective and impartial
    information so they can get to design strategy effectively, often a
    great deal of effort is made to minimize this.

    But here are always the questions about the non-statistical uncertainty,
    like those that follow.
    Is the population intended identical to the population about which we
    wish to make rhetorical (in the old sense of the word as principled
    argument) statements? (frame errors -- respondents not identical to
    voters, non response not truly random due to social desirability,
    non-attitudes, etc.) Is there likely to be some change over time?
    (history, maturation, cohort effects, vote intention formation or
    change, perception of what is likely to happen, wanting to be on winning
    side, etc.)? How reliable is the measurement process: wording,
    understanding, sequence of questions, interviewer or instrument by
    respondent interaction, etc.?

    Hope this casts some light.

    Art
    Political Psychologist and retired government Mathematical Statistician.
    Art@DrKendall.org
    Social Research Consultants
    University Park, MD USA
    (301) 864-5570

    Stan Brown wrote:
    > "George Kahrimanis" <anakreon@hol.gr> wrote in sci.stat.edu:
    >
    >>I read with great interest the responses by Rich Ulrich and
    >>Stan Brown. They both make good sense to me.
    >>
    >>If the meaning of "3% margin of error" is that the number of
    >>valid and pertinent responses is such that a fair coin would
    >>tally between 47% and 53%, with confidence 95% ("two sigma"),
    >>then we are talking about some number a little below 9000.
    >>Not too high.
    >
    >
    > Where do you get 9000?
    >
    > If the true proportion p is close to .5, then the margin of error
    > for 95% confidence with sample size n is
    > z(.025)*sqrt(p(1-p)/n) =
    > 1.96*sqrt(.5*.5/n) =
    > 0.98/sqrt(n)
    >
    > Setting that equal to .03 gives n = 1067.
    >
    >
    >>Wrt the tacit 95% confidence level, I admit that due to my
    >>background I had supposed a 68% CL. Moreover, I had unconsciously
    >>presumed that the behavour of voters is too capricious to merit
    >>a higher CL.
    >
    >
    > Why? The confidence interval is about an estimate of voter opinion
    > _now_, not voter opinion at some other time. If the sample of 1067
    > was a good random sample, then indeed we can be 95% confident that
    > it's not off by more than 3% from the true figure, as of the date
    > the poll was taken.
    >
    > There are many extra-statistical reasons, however, why the poll
    > results may not predict the election. As you mentioned, some voters
    > do change their minds. In addition, in the US we have a system where
    > people's votes don't all count equally, because of the complication
    > of electoral votes.
    >

  • Next message: shanky: "Need help with deciding sample n sub sample. size for nested design(DOE)"