Re: Can I use Ordinal Regression for Rank and Nominal data ?

On Jul 22, 9:34 pm, <geetha.shree@xxxxxxxxx> wrote:
On Jul 22, 3:53 am, Ray Koopman <koopman@xxxxxx> wrote:
On Jul 21, 6:06 am, <geetha.shree@xxxxxxxxx> wrote:
[...]
I did calculate the Kendall Coefficient (W = 0.102).

Kendall's W is more easily interpreted if it is changed to
an average Spearman r among the n(n-1)/2 pairs of raters,
where n = the # of raters: average r = W - (1-W)/(n-1).
You don't say what your n is, but whatever it is, the average r
is small, less than .102. Evidently there is little agreement
among your raters.

There is a chi-square test of the hypothesis that the average
correlation among the raters is 0; i.e., that every panel's
true average rank equals (k+1)/2, where k = the # of panels:
chi-square(k-1) = n(k-1)W, with large values of chi-square
leading to rejection of the hypothesis. Did you do this test
(which is also known as Friedman's two-way anova on ranks)?

Hi Mr. Ray Koopman,
Thanks for giving me a lead in solving this
"Interesting" problem. I earlier had a sample size of 60 People
to evaluate 14 panels. Then I had to reduce the sample size to 30
People to evaluate the 14 panels for the rest of the tests. I did
the Wilcoxon Rank Sum Test to verify if I could use just 30 people
instead of 60. The results were positive.

What did you do the Wilcoxon test on? Why do you think the results
justify using only 30 people instead of 60?


SO the final results I do have is the Rank from 30 people
evaluating 15 panels.

How many panels? First you said 18, then you said 14, now you say 15.


I did the Chi - Square test, I got a value of 47.6. I also tested
W for significance using Snedecors Distribution for F (F = 3.4).

Now F value is greater than the Critical ( tabulated) F value. So
the null hypothesis ( which states there is no correlation among
the judges/ people) is rejected.

A chi-square of 47.6 with 14 degrees of freedom will let you reject
the null hypothesis. Why and how did you go from chi-square to F?
What degrees of freedom did you use for F?


The value of W must be close to 1 to indicate there is an
agreement between the evaluation.
I got a value less than 1. which indicated disagreement.
But on the other hand the F value contradicts this statement.
Could you please let me know who should I go about this.

Rejecting the null leads only to the conclusion that there is some
degree of agreement. In this case, there is very little agreement
(average r = .071). Whether that is sufficient is an open question.


I am not using pairs so I didnt use the Spearmans R Calculation.

The average r is a way of characterizing the degree to which the
raters agree with one another. If you pick two raters at random,
you would expect their rankings to correlate only .071.


P.S. I am very naive about statistics, your help would be much
appreciated.

I strongly urge you to get local assistance. You need closer
guidance than can be provided in a forum such as this.


Thanks,
Geetha.
.

Relevant Pages

  • Re: Can I use Ordinal Regression for Rank and Nominal data ?
    ... I did calculate the Kendall Coefficient. ... Evidently there is little agreement ... among your raters. ... People to evaluate the 14 panels for the rest of the tests. ...
    (sci.stat.math)
  • Re: sample size for kappa?
    ... Kappa is not a very good 'absolute' statistic beyond the 2x2 case. ... raters and a audit tool/questionnaire that has 250 ... know if there is inter rater agreement between the 2 raters on the 250 ... agreement in measuring a single 'dimension' that is shared ...
    (sci.stat.consult)
  • Re: Interobserver agreement question
    ... Two raters. ... Report these raw agreement indices: the proportion ...
    (sci.stat.consult)
  • Re: Can I use Ordinal Regression for Rank and Nominal data ?
    ... I did calculate the Kendall Coefficient. ... an average Spearman r among the n/2 pairs of raters, ... among your raters. ... There is a chi-square test of the hypothesis that the average ...
    (sci.stat.math)