Re: a smoothing problem

From: Leslaw Bieniasz (nbbienia_at_cyf-kr.edu.pl)
Date: 03/22/05 

Date: Tue, 22 Mar 2005 10:21:57 +0100
To: Ray Koopman <koopman@sfu.ca>

On Thu, 16 Mar 2005, Ray Koopman wrote:

> Leslaw Bieniasz wrote:
> > On Mon, 14 Mar 2005, Ray Koopman wrote:
> >> Try Tukey's 4253H-twice smoother.
> >

                                Cracow, 22.03.2005

Hello,

I have tried the 4253H-twice, and I see that it smoothes nicely, but the
effect is not exactly as I would like to have. As I explained previously,
I want the spikes to be removed, or filtered out completely, and the
"correct" data to be left unchanged. However, 4253H-twice only partially
suppresses the spikes, and broadens them, at the same time distorting the
"correct" data. Are there no better alternatives? Perhaps some sort of
filtering out higher frequencies would work? The spikes usually do not
last for more than about 3 adjacent data points, which might be helpful in
determining the cut-off frequency.

I am attaching a typical data set below. Here "y" presents the original
data, "ysmooth" is the result of 4253H-twice smoothing, and "desired"
is what I would like to have as a result of the filtering. I have created
the last column "by hand", simply by cutting off the spikes, and replacing
them by linear interpolation/extrapolation of the remaining data points
(the modified numbers are shifted to the left).

If anybody knows how to solve the problem, please help me.
It would be nice if the algorithm is not only successful, but also fast.

L.B.

==================================================
i y ysmooth(4253h-twice) desired
==================================================
1 0 -0.0007775852 0.00083025
2 -0.00161438 -0.0007775852 0.00087227
3 -0.00141655 -0.0007775852 0.00091429
4 -0.00048605 -0.0004308469 0.00095631
5 0.00099833 0.0003704646 0.00099833
6 0.00104035 0.000932873 0.00104035
7 0.0010507 0.001042737 0.0010507
8 0.00102727 0.001033648 0.00102727
9 0.00097106 0.0009865582 0.00097106
10 0.0008862 0.0009003709 0.0008862
11 0.00077941 0.0007075751 0.00077941
12 0.00065899 0.0001708214 0.00065899
13 -0.00057051 -0.0005023365 0.0005122
14 -0.0023549 -0.0007813211 0.0003664
15 -0.00119831 -0.0006883201 0.0002206
16 7.481e-05 -0.0003899586 7.481e-05
17 6.111e-05 -7.223859e-05 6.111e-05
18 4.801e-05 4.012094e-05 4.801e-05
19 3.546e-05 3.6125e-05 3.546e-05
20 2.337e-05 2.501992e-05 2.337e-05
21 1.16e-05 1.187648e-05 1.16e-05
22 0 0 0
23 -1.16e-05 -1.187648e-05 -1.16e-05
24 -2.337e-05 -2.501992e-05 -2.337e-05
25 -3.546e-05 -3.6125e-05 -3.546e-05
26 -4.801e-05 -4.012094e-05 -4.801e-05
27 -6.111e-05 7.223859e-05 -6.111e-05
28 -7.481e-05 0.0003899586 -7.481e-05
29 0.00119831 0.0006883201 -0.0002206
30 0.0023549 0.0007813211 -0.0003664
31 0.00057051 0.0005023365 -0.0005122
32 -0.00065899 -0.0001708214 -0.00065899
33 -0.00077941 -0.0007075751 -0.00077941
34 -0.0008862 -0.0009003709 -0.0008862
35 -0.00097106 -0.0009865582 -0.00097106
36 -0.00102727 -0.001033648 -0.00102727
37 -0.0010507 -0.001042737 -0.0010507
38 -0.00104035 -0.000932873 -0.00104035
39 -0.00099833 -0.0003704646 -0.00099833
40 0.00048605 0.0004308469 -0.00095631
41 0.00141655 0.0007775852 -0.00091429
42 0.00161438 0.0007775852 -0.00087227
43 0 0.0007775852 -0.00083025
==================================================

*-------------------------------------------------------------------*
| Dr. Leslaw Bieniasz, |
| Institute of Physical Chemistry of the Polish Academy of Sciences,|
| Department of Electrochemical Oxidation of Gaseous Fuels, |
| ul. Zagrody 13, 30-318 Cracow, Poland. |
| tel./fax: +48 (12) 266-03-41 |
| E-mail: nbbienia@cyf-kr.edu.pl |
*-------------------------------------------------------------------*
| Interested in Computational Electrochemistry? |
| Visit my web site: http://www.cyf-kr.edu.pl/~nbbienia |
*-------------------------------------------------------------------*

Relevant Pages

  • Re: [kde] Fighting line noise with Kmail
    ... signals are expected to be lousy. ... Telecom's line noise, considering I can hear the noise crackling in my phone. ... except that rain does not make spikes, ... pops and crackles could be reduced with filtering, ...
    (KDE)
  • Re: [kde] Fighting line noise with Kmail
    ... where signals are expected to be lousy. ... like what is made by transtector, ... except that rain does not make spikes, ... pops and crackles could be reduced with filtering, ...
    (KDE)
  • Re: a smoothing problem
    ... > suppresses the spikes, and broadens them, at the same time distorting the ... > is what I would like to have as a result of the filtering. ... This all sounds much like the application of wavelets to denoising ... The wavelet denoising is sufficiently ...
    (sci.math.num-analysis)
  • Re: What Size Partitions for XP and Linux?
    ... If the power supply is any good, the spikes, etc., ... After chopping, transforming, rectifying again, filtering ... The power supply, absent a well designed surge protector, will take the ...
    (comp.os.linux.misc)
  • Re: a smoothing problem
    ... What you have to look at is how you detect the spikes you wish to delete. ... The 4253H smoother may be too short for your purposes ... >filtering out higher frequencies would work? ... >is what I would like to have as a result of the filtering. ...
    (sci.math.num-analysis)
Loading