Re: Kendall/Spearman rank correlation

On Sat, 11 Aug 2007 12:03:31 EDT, Cathy Laporte
<cathy@xxxxxxxxxxxxx>
wrote:

Thank you for the suggestions. I will definitely
look up Conover's textbook.

I want to use one of the two measures to evaluate
the accuracy of an approximate sorting algorithm. By
approximate, I mean that it is an algorithm that
sorts data based on redundant, noisy measurements of
similarity between the different data points. I want
to compare the ordering output by the algorithm with
the ground truth order. I think that either
correlation (Spearman's or Kendall's) between the
true and inferred ranks should give me a good idea of
the algorithm's robustness.

Either one would allow you to compare several
algorithms.

That is exactly what I need to do. I am comparing my "robust" algorithm against a base-line algorithm which works perfectly in the noiseless case but whose performance degrades nastily with noise.


You have only one algorithm? You are left with an
unknown standard -- Correlations best measure
'association',
which you surely should have plenty of, and not
'deviations.'
What difference for 1.0 is going to be important?
(And who
are you trying to convince of it?)

Why not get a frequency distribution of the
deviations (plus and minus rank errors, or absolute
errors)
for your multiple trials? I think that will give you

a more persuasive argument about robustness.

Good point, though. I will give that some thought.

Thanks!


--
Rich Ulrich, wpilib@xxxxxxxx
http://www.pitt.edu/~wpilib/index.html
.

Relevant Pages

  • Re: Parallel application ran more slowly than the sequential one?
    ... > Could you tell me what are the reasons to make an parallel application ... Your parallel algorithm may be less efficient than your serial algorithm. ... loading/unloading MPI data structures, sharing files over NFS, process ... Compare the runtime of your parallel program to ...
    (comp.parallel)
  • Re: count of each word occurred
    ... > take each word in order and compare it with the rest of the list ... Another version of this algorithm takes advantage of the fact that it does ... loop over all words in the list ... a.c.l.l.c-c++ FAQ: http://www.comeaucomputing.com/learn/faq ...
    (alt.comp.lang.learn.c-cpp)
  • Re: Reading and writing a big file in Ada (GNAT) on Windows XP
    ... common cases (no mapping, single character patterns, and so on). ... block move and compare instructions because they fouled up the pipeline and ... So in this case a better algorithm is probably the way to go (and I ... the source length is>M and the pattern length is>N, ...
    (comp.lang.ada)
  • Re: ArrayList BinarySearch vs Contains
    ... whether a required sorting operation will actually reduce the overall ... performance of an algorithm so that it becomes worse than a linear ... A Linear search does a Fast Compare and a fast iteration to the next ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: how to compare between different training algorithms, to find the best algorithm???
    ... how to compare between different algorithms (such as ... discuss algorithm comparisons. ... If the training set is too large for LM, ... minimize an objective function (say MSE) over the population ...
    (comp.soft-sys.matlab)