Re: an outlier problem
From: Graham Jones (graham_at_visiv.co.uk)
Date: 03/25/05
- Next message: PKS: "Re: Polynomial Regression"
- Previous message: robert_dodier_at_yahoo.com: "Re: Naive Bayes Classifiers"
- In reply to: Leslaw Bieniasz: "an outlier problem"
- Next in thread: dave_at_autobox.com: "Re: an outlier problem"
- Reply: dave_at_autobox.com: "Re: an outlier problem"
- Messages sorted by: [ date ] [ thread ]
Date: Fri, 25 Mar 2005 11:17:02 +0000
To implement a statistical approach to this sort of problem you need a
model for the signal, and a model for the noise. The better you can
characterise what the signal is like and what the noise is like, the
more reliably you will be able to distinguish them.
In the example data, it seems that: (1) The signal varies slowly, while
the noise varies quickly. (2) There are quite long gaps between noise
spikes (can you put a minimum on that?). (3) The spikes are always in
one direction (up or down, not both). (4) The data, noise and signal, is
anti-symmetric, which might help with speed at least. But I don't know
if these assumptions are valid for all your data.
Your models are probably going to be local. Ross has suggested looking
at adjacent pairs of points, and he has an implicit model ("signal
changes slowly, noise jumps"). You will probably get better results
looking at sequences of 3 or more. Eg "signal is approximately linear,
noise isn't".
BTW, I don't know what "4253H-twice smoothing" means.
-- Graham Jones http://www.visiv.co.uk Emails to graham@visiv.co.uk may be deleted as spam Please add a j just before the @ to ensure delivery
- Next message: PKS: "Re: Polynomial Regression"
- Previous message: robert_dodier_at_yahoo.com: "Re: Naive Bayes Classifiers"
- In reply to: Leslaw Bieniasz: "an outlier problem"
- Next in thread: dave_at_autobox.com: "Re: an outlier problem"
- Reply: dave_at_autobox.com: "Re: an outlier problem"
- Messages sorted by: [ date ] [ thread ]
Relevant Pages
|
|
