Re: Likert Scale Analysis

On 20 May 2005 09:41:01 -0700, wowen@xxxxxxxxxxxx wrote:

> OK. Can you provide a reference (preferably written by a
> statistician... yes, I know that Jamieson probably isn't) where it is
> stated that Likert scale data is fine for ANOVA and other
> interval-valued procedures? I'm also interested in a discussion on 5,
> 7, and 9-point scales. I could never be convinced that a 3-point scale
> could be considered interval valued, but what is the difference between
> 7 and 9? At what point does the number become become too large so that
> it confuses the subject (and isn't that subjective)?

Roderick McDonald's 1999 text, "Test Theory, a unified treatment,"
says, after describing Likert and other item-types --
"Quantitative item scores virtually never take a sufficiently wide
range of values to allow them to be modeled as continuous, possibly
normally distributed variables. However, in a number of methods of
analysis treated in this book, we choose to regard the normal
distribution assumption as an acceptable approximation."

If your "3-point scale" is not considered interval, consider
what the "non-parametric" solution consists of. First, there
are *ties*, which violates the rank-test assumption of
continuous measures (no ties). The universal solution
for that is to assign to all cases the average of the ties at
a score. Conover et al. demonstrated that the book-version
of tests with ties are not necessarily better -- but they are
hardly ever much different -- from computing ANOVA on
the assigned scores, even for sample Ns that are fairly small.

Okay, this gets rocky if you are not comfortable with the idea
that ANOVA only cares about relative spacing on a scale.
Linear transformations don't matter, such as, dividing by N
or anything else, or adding/subtracting a constant - as long as
you do it to all scores.

So, the alternative is to use (1,2,3), or (0,10,20) or any similar
equal spacing, versus using whatever-spacing-emerges from
taking the *ranks* for the sample in hand. If Ns for the
categories 1 and 3 are equal, the "nonparametric" result is
equal-intervals, and the two ANOVAs will be identical.
If Ns are unequal, intervals are unequal.

My advice remains what it has been: consider whether the
inconvenience of "doing a rank-transform" is justified by
an improvement in the size of intervals. Percentages of
(60, 30, 10) yield average percentiles, which work the same
as ranks for an ANOVA, of (30,75,95). By simple linear
transformation of the range, preserving the relative spacing,
this is a scoring of (1,2.38, 3); or, equivalently, (1,2, 2.44).
Is that an improvement over (1,2,3)? If so, you will be
happier using the rank, or by assigning the subjective score
in the first place.

In a couple of his textbooks, Agresti has the example of
observed fetal deaths or birth defects, versus mothers'
drinking. This can be analyzed with ANOVA by using
as numbers the categories of alcohol consumption, 1-5;
as ranks, which compresses the top of the scale as my
example does, above; or as mid-scores for the "drinking"
variable ["1-2 drinks" becomes 1.5] , which stretches the
top of the scale.
The "nonparametric" answer, using ranks, has the least
power for that example.

--
Rich Ulrich, wpilib@xxxxxxxx
http://www.pitt.edu/~wpilib/index.html
.

Relevant Pages

  • Re: when is data considered "continuous" for parametric testing?
    ... a t-test comparing these two groups, or is that not continous data. ... not directly related to ANOVA. ... One assumption for *non-parametric* testing based on ranks ... When there are a lot of ties, the ANOVA ...
    (sci.stat.edu)
  • Re: ANOVA on ordinal data
    ... me [concerning doing ANOVA, but not reporting means] ... Just to be more complete -- the scale is not wholly determined ...
    (sci.stat.math)
  • Re: computing scales and reliability
    ... Art Kendall wrote: ... It is congenital to treat a summative score as continuous, i.e., with ANOVA. ... If one has a number of items, say 7, measured on a 1-5 likert scale ... Then I could use the sum scale in a One Way Anova or Kruskal-Wallis to ...
    (sci.stat.consult)
  • Re: computing scales and reliability
    ... With a moderate alpha in the .7 to .8 range it is acceptable in most circumstances to treat the scale as ok. ... It is congenital to treat a summative score as continuous, i.e., with ANOVA. ... Then I could use the sum scale in a One Way Anova or Kruskal-Wallis to ...
    (sci.stat.consult)