Re: hypothesis test question on two proportions

prof wrote:
m00es wrote:
prof wrote:
In the two population proportion test of H0: p1 = p2 vs Ha: p1 not
equal p2 with a los of 0.05, one way to conduct the test is to
construct a 95% CI on p1 - p2. Then, if the interval contains "0",
accept H0; otherwise reject H0. An alternate way is to calculate the z
statistic under the assumption that H0 is true and compare to the 5%
critical region. If the statistic is in the critical region, reject H0;
otherwise accept H0. The question is: do the two methods give identical
conclusions? The answer is not obvious to me since in the CI approach
an assumption of H0 being true is not used; thus there are separate
estimates for the two sample proportions which appear in the CI
formula. whereas, in the test statistic approach, H0 being true is
assumed; thus giving a combined estimate for the common proportion p
which is used in the test statistic.. It is not obvious that identical
conclusions would always be reached. Help please!!!

Let's make this question more concrete.

Assume you have two independent samples of sizes n1 and n2 and the
number of people with some outcome are observed in each group. Let x1
and x2 denote these numbers. Then p1 = x1/n1 and p2 = x2/n2 are the
observed proportions in each group. We now want to test whether H0: pi1
= pi2, where pi1 and pi2 are the true proportions in each group.

For an approximate 95% confidence interval for pi1 - pi2, we calculate:

(p1 - p2) +- 1.96 sqrt( p1(1-p1)/n1 + p2(1-p2)/n2 )

and if 0 is not included in the confidence interval, we reject H0: pi1
= pi2.

The hypothesis test assumes that H0: pi1 = pi2 is true. Therefore, p1
and p2 are estimates of the same proportion. These estimates are then
commonly pooled together into p = (x1 + x2)/(n1 + n2), yielding a
single estimate for pi = pi1 = pi2. Then:

z = (p1 - p2) / sqrt( p(1-p) (1/n1 + 1/n2) )

can be compared against +-1.96, the critical values for a normal
distribution with alpha = .05.

It is now possible to construct an example where the confidence
interval and the hypothesis test will yield a different conclusion.

Example:

x1 = 40, n1 = 50, p1 = .8
x2 = 340, n2 = 500, p2 = .68
p = .69

95% CI = (.002, .238) and therefore we conclude pi1 not equal to pi2.
z = 1.75 and therefore we do not reject H0: pi1 = pi2.

Those "pathological" cases are rare though and the group sizes have to
be substantially different for this to happen.

Also note that the CI and the hypothesis test as given here are
approximations anyway.

m00es

Thank you very much, Your explanation was very helpful. One more
question. Which method is usually preferred???

.... with respect to deciding whether to reject H0: pi1 = pi2? One would
have to compare the Type I error rates and power of the two approaches
to see if one does better than the other. I doubt that there is much of
a difference between the two approaches (at least under realistic
conditions), but now I am just doing a bit of hand-waving.

In general, I would personally go with a confidence interval. It does
not make any assumption about pi1 and pi2 being equal, it still let's
you decide whether to reject H0: pi1 = pi2, and you get to see how well
you are estimating pi1 - pi2.

But again, whether the "confidence interval approach" actually does
better (in terms of the Type I error rate and power) when trying to
decide whether to reject H0: pi1 = pi2 would require further analysis.

m00es

.

Relevant Pages

  • Re: hypothesis test question on two proportions
    ... m00es wrote: ... If the statistic is in the critical region, ... estimates for the two sample proportions which appear in the CI ... We now want to test whether H0: pi1 ...
    (sci.stat.edu)
  • Re: hypothesis test question on two proportions
    ... If the statistic is in the critical region, ... estimates for the two sample proportions which appear in the CI ... We now want to test whether H0: pi1 ... interval and the hypothesis test will yield a different conclusion. ...
    (sci.stat.edu)
  • =?iso-8859-1?q?Re:_Bob_Ling=B4s_ignorance_is_clear?=
    ... can do in order to compare two INDEPENDENT proportions." ... Fisher's Exact Test is most appropriate for comparing two independent ... be used to compare two independent proportions. ... Not once have I suggested using the Normal approximation to a binomial. ...
    (sci.stat.math)
  • Re: Is this sample representative of the population?
    ... > The question asked by the client is: "Is this sample representative of the ... The table wants to compare the two parts of a total, ... The standardized cell deviations can be used as a measure ... The raw proportions might also be interesting. ...
    (sci.stat.consult)
  • Re: hypothesis test question on two proportions
    ... m00es wrote: ... If the statistic is in the critical region, ... estimates for the two sample proportions which appear in the CI ... interval and the hypothesis test will yield a different conclusion. ...
    (sci.stat.edu)