Re: comparing Kappa Statistics in case of dependence

On Jan 15, 2:36 am, Bart Hamers <bart.ham...@xxxxxxxxx> wrote:
In order to compare different raters, we often use the nonparametric
Kappa statistic as a measure of agreement.
However, we would like to compare different kappa statistics. Moreover
do we need a test to compare the kappa's in case of dependence.
Is there a test to compare Kappa(X,Y) and Kappa(Y,Z) taking into
account the dependence introduced by the mutual rater Y?

Googling for <correlated kappa> turned up
Weighted Least-Squares Approach for Comparing Correlated Kappa
Barnhart, Huiman X., & Williamson, John M.
Biometrics, Volume 58, Number 4, December 2002 , pp. 1012-1019

But note that the correlatedness is due to using the same N cases
for both kappas. It would still exist even if you were comparing
kappa(x,y) to kappa(w,z).
.

Relevant Pages