Re: t test for outlier detection

t test for detecting outliers.

Hi, it is possible to use a test for outlier
detection in the
following way?

Consider the case we want to detect an outlier in
a set of n data points:
(p1,...,pN).

Consider x as outlier if

x- mean(x)
------------ > critical value
sqrt(var(x))

x- mean(x)
------------ < critical value
sqrt(var(x))

critical_value=t(1-alpha/2,N-1)

Most standard outlier test cannot be used to remove
more than a single
point or cannot be used for small data sets. However
a t-test is usually
unproblematic. Since if the degrees of freedom are
small, a higher
value is needed to become an outlier which is quite
what is expected.

Thanks for any remarks,

Tim.


What you've described is similar to the Grubbs Outlier Test, also known as the ESD (extreme studentized deviate) method.

It is given by

z = max|Xi - Xbar|/s

The critical values Cn are determined so that the probability that z > Cn is 0.05 when in fact all the Xi come from the same normal distribution. So the test has the property that if there are no actual outliers in the sample, the probability of wrongly declaring an outlier is 5%.

Jack
.

Relevant Pages

  • Re: t test for outlier detection
    ... Consider the case we want to detect an outlier in ... Most standard outlier test cannot be used to ... Tim, although the Grubbs test was not intended to be used for detecting more than a single outlier, I have modified it as a stepwise forward-elimination Grubbs test for detecting one or more outliers. ...
    (sci.stat.math)
  • Re: t test for outlier detection
    ... Consider the case we want to detect an outlier in ... What you've described is similar to the Grubbs Outlier Test, also known as the ESD method. ... Or do you know maybe a variation to apply it on a small amount of data as well? ...
    (sci.stat.math)
  • Selecting Representative Data
    ... classifiers. ... these are the text of books for which I ... I note that there is quite a bit of work on automatic outlier ... "Robust automatic methods for outlier and error detection", ...
    (sci.stat.math)