Re: definition of measurement of agreement

Jacqueline,

It's up to you as the investigator to define what you understand by the
word, "agreement."
Ultimately, "agreement" is measured by tabulating how often raters
agree, or the proportion of times they do--i.e., a raw index of
agreement. The arguments that have been made for rejecting raw
agreement indices are very weak, in my own opinion, and many of the
same criticism that apply to raw agreement indices also apply to
various alternatives like kappa.

If you use something else, like ICC or kappa, then it seems like you
are not measuring agreement, but rather some indirect construct (e.g.,
reliability) that you suppose to be related to agreement in some way.

In any case, it's just a word. What matters is why you're collecting
and analyzing the data. Why do you want an index of reliability or
agreement?

http://ourworld.compuserve.com/homepages/jsuebersax/agree.htm

Jacqueline wrote:


Hello group, wish all of u a Happy New Year in advance.

Do you know the definition of measurement of agreement (related to ICC
and kappa)? any literature is helpful.

Thank u.

Jacqueline

.

Relevant Pages

  • Re: sample size for kappa?
    ... Kappa is not a very good 'absolute' statistic beyond the 2x2 case. ... raters and a audit tool/questionnaire that has 250 ... know if there is inter rater agreement between the 2 raters on the 250 ... agreement in measuring a single 'dimension' that is shared ...
    (sci.stat.consult)
  • definition of measurement of agreement
    ... Do you know the definition of measurement of agreement (related to ICC ... and kappa)? ...
    (sci.stat.math)
  • Last nights RAW
    ... Are we all in agreement that last night's RAW should be labeled by the ... EPA as a Level 2 Biohazard? ...
    (rec.sport.pro-wrestling)