Re: Intra- and interrater reliability - examples

Thanks for your answer. I have some problems in understanding...


Ray Koopman schrieb:

[snip]

Intra- and inter-rater reliability are ideas more than particular
formulas. Here is one way to implement them.

For intra-rater reliability, you get a value for each rater. Do a
two-way anova each rater's 19x3 table of ratings, and get the ratio
of the estimated patient variance over the sum of the estimates of
all components of variance.
"you get a value for each rater" - does that mean I have to average
over the raters? If yes, which repetitions will be averaged together?
Or over the repetitions (if so, my table will be 19x5) - that makes no
sense.
Two-way anova - with factors patient and repetitions?
With the "estimated patient variance" you mean the mean squares?
Is the error term in the "sum of the estimates of all components of
variance" included?


For inter-rater reliability, do a three-way anova on the 19x5x3 table
of all the ratings; again, use the ratio of the estimated patient
variance over the sum of the estimates of all components of variance.
As before: Is the error term in the "sum of the estimates of all
components of variance" included?
Three-way anova - with factors patient, raters and repetitions?

Do you know a *detailed* example in the net or in books/papers? Even
with missing data?

Thanks!
Karl.

.

Relevant Pages

  • Re: variance across 2 independent experiments
    ... > The variance from the ANOVA table gives the variance WITHIN the groups ... > mind that a, b, c etc are not replicates but unique experiments within ...
    (sci.stat.edu)
  • Re: Datasets of unequal size and tiny variances
    ... Sean Richards wrote: ... has 5 levels (the diffferent values of 'a' mean that the datasets are of ... So I can do a one way Anova to test this? ... After the one way Anova I should test for homogeneity of variance ...
    (sci.stat.math)
  • Re: Datasets of unequal size and tiny variances
    ... > Sean Richards wrote: ... What effect does the huge difference in group sizes have? ... In the sense that you can still do a one-way ANOVA. ... After the one way Anova I should test for homogeneity of variance ...
    (sci.stat.math)
  • Datasets of unequal size and tiny variances
    ... Some basic stats on the datasets: ... So I can do a one way Anova to test this? ... What effect does the huge difference in group sizes have? ... After the one way Anova I should test for homogeneity of variance ...
    (sci.stat.math)