Re: Reporting effect sizes for McNemar's test

"John Uebersax" <jsuebersax@xxxxxxxxx> wrote in
news:1150978898.083921.297070@xxxxxxxxxxxxxxxxxxxxxxxxxxxx:

David wrote:

Is the McNemar's test used in unmatched data situations?

For me, a psychometric example comes to mind. Suppose you have two test
items which may be either correctly or incorrectly answered. One might
ask: are these items equally difficult?

To examine this we can cross-classify the responses:

Item 2
fail pass
+-----+-----+
fail | a | b | r1
Item 1 +-----+-----+
pass | c | d | r2
+-----+-----+---
c1 c2 N

Item difficulty is often and conveniently operationally defined or
expressed by the proportion of respondents who pass an item--i.e.,
marginal rates r2/N and c2/N.

I believe one can examine this with the McNemar test, understanding it
as a test of marginal homogeneity.

No argument.

Now if both items are given to the same sample--the usual
scenario--they are by definition non-independent. And I suppose in
that sense, they are matched. You could, for example, present the data
as N "matched" pairs of responses, but that isn't ordinarily how one
thinks of such data.
I would simply call them non-independent or dependent.


Toe-matt-oh v. toe-mate-oh. You want to call it "dependent data". I say
"matched" or "paired". A distinction without a difference. If they were
done on different test populations you would not use McNemar's.

The question the OP asked for an effect measure, by which I assume she
did not want to simply report marginals. You criticise me for not giving
a responsive answer. I see your replies as non-responsive. When you
report as you earlier suggested, just one marginal you are only
reporting one side of a paired data situation. If you report both pass
marginals (or both fail marginals) and the N, then that's effectively
describing the table. If you report the ratio of failure rates, c1/r1,
or pass rates, c2/r2, then _that's_ an effect measure. You could say "the
failure rate for item 2 was <c1/c2> times the failure rate for item 1".
Or you could say "the difference in failure rates was <(c1-c2)*100>
percent". That's also an effect measure.

--
David Winsemius
.

Relevant Pages

  • Re: Variables out the wazoo...
    ... > into his responses that advises folks that they have posted into an ... > Since you expressed that your issue was with a report you might try ... > HTH ... >>> believe you are using the Access Report Designer. ...
    (microsoft.public.access.modulesdaovba)
  • Re: Cycle helmets sheet
    ... Patrick Herring wrote: ... free look at the reference since the explicit reason for the ref isn't ... responses suggestive of RC than did nonusers. ... For example, in general, children using PE such as helmets report going faster and more often reported damage to their bike, perhaps because wearers took risks resulting in more severe ...
    (uk.rec.cycling)
  • Other gothic conceptual easters will spring far once again grocers.
    ... Every giant alternative responses ... Don't even try to report a resistance! ... Afif will bounce the mortgage, and if Georgina inside sucks it too, the ... Every peaceful reliefs control Rahavan, ...
    (comp.arch)
  • Re: online incident reporting
    ... Like his report to Council that _all_ responses to Row Safe had been favourable - which he had yesterday, & with utmost reluctance I gather, to accept was simply not true? ... It stinks to high heaven, after the last 8 years of deliberate lying & deceit by the organisation, that key ARA figures simply couldn't be bothered to stay to discuss water safety & hear more about the ARA's continuing attempts to ride roughshod over all the efforts of RoSPA, uncle Tom Cobley & all. ... nothing in the ARA's conduct of rowing safety has been remotely fair. ...
    (rec.sport.rowing)
  • Re: Reporting from Queries
    ... SELECT Person, Mod2, "Mod2" ... SELECT Person, Mod3, "Mod3" ... This union query should allow you to create your report. ... > out any modules that did not have responses. ...
    (microsoft.public.access.reports)