Re: a smoothing problem

From: Gordon Sande (g.sande_at_worldnet.att.net)
Date: 03/22/05 

Date: Tue, 22 Mar 2005 13:22:43 GMT

Leslaw Bieniasz wrote:
>
> On Thu, 16 Mar 2005, Ray Koopman wrote:
>
>
>>Leslaw Bieniasz wrote:
>>
>>>On Mon, 14 Mar 2005, Ray Koopman wrote:
>>>
>>>>Try Tukey's 4253H-twice smoother.
>>>
>
> Cracow, 22.03.2005
>
> Hello,
>
> I have tried the 4253H-twice, and I see that it smoothes nicely, but the
> effect is not exactly as I would like to have. As I explained previously,
> I want the spikes to be removed, or filtered out completely, and the
> "correct" data to be left unchanged. However, 4253H-twice only partially
> suppresses the spikes, and broadens them, at the same time distorting the
> "correct" data. Are there no better alternatives? Perhaps some sort of
> filtering out higher frequencies would work? The spikes usually do not
> last for more than about 3 adjacent data points, which might be helpful in
> determining the cut-off frequency.
>
> I am attaching a typical data set below. Here "y" presents the original
> data, "ysmooth" is the result of 4253H-twice smoothing, and "desired"
> is what I would like to have as a result of the filtering. I have created
> the last column "by hand", simply by cutting off the spikes, and replacing
> them by linear interpolation/extrapolation of the remaining data points
> (the modified numbers are shifted to the left).
>
> If anybody knows how to solve the problem, please help me.
> It would be nice if the algorithm is not only successful, but also fast.

As they say, "good, fast or cheap, chose any TWO!"

This all sounds much like the application of wavelets to denoising
various spectra. The invariance properties of wavelets more closely
match your problem statement. Other than the pointer to wavelets
I have no advice to offer. The wavelet denoising is sufficiently
elaborate a procedure that it hardly qualifies as a smoother or
filter even though it achieves the same end.

There will be those who wonder why expensively obtained data does not
warrant more than just cheap processing.

> L.B.
>
> ==================================================
> i y ysmooth(4253h-twice) desired
> ==================================================
> 1 0 -0.0007775852 0.00083025
> 2 -0.00161438 -0.0007775852 0.00087227
> 3 -0.00141655 -0.0007775852 0.00091429
> 4 -0.00048605 -0.0004308469 0.00095631
> 5 0.00099833 0.0003704646 0.00099833
> 6 0.00104035 0.000932873 0.00104035
> 7 0.0010507 0.001042737 0.0010507
> 8 0.00102727 0.001033648 0.00102727
> 9 0.00097106 0.0009865582 0.00097106
> 10 0.0008862 0.0009003709 0.0008862
> 11 0.00077941 0.0007075751 0.00077941
> 12 0.00065899 0.0001708214 0.00065899
> 13 -0.00057051 -0.0005023365 0.0005122
> 14 -0.0023549 -0.0007813211 0.0003664
> 15 -0.00119831 -0.0006883201 0.0002206
> 16 7.481e-05 -0.0003899586 7.481e-05
> 17 6.111e-05 -7.223859e-05 6.111e-05
> 18 4.801e-05 4.012094e-05 4.801e-05
> 19 3.546e-05 3.6125e-05 3.546e-05
> 20 2.337e-05 2.501992e-05 2.337e-05
> 21 1.16e-05 1.187648e-05 1.16e-05
> 22 0 0 0
> 23 -1.16e-05 -1.187648e-05 -1.16e-05
> 24 -2.337e-05 -2.501992e-05 -2.337e-05
> 25 -3.546e-05 -3.6125e-05 -3.546e-05
> 26 -4.801e-05 -4.012094e-05 -4.801e-05
> 27 -6.111e-05 7.223859e-05 -6.111e-05
> 28 -7.481e-05 0.0003899586 -7.481e-05
> 29 0.00119831 0.0006883201 -0.0002206
> 30 0.0023549 0.0007813211 -0.0003664
> 31 0.00057051 0.0005023365 -0.0005122
> 32 -0.00065899 -0.0001708214 -0.00065899
> 33 -0.00077941 -0.0007075751 -0.00077941
> 34 -0.0008862 -0.0009003709 -0.0008862
> 35 -0.00097106 -0.0009865582 -0.00097106
> 36 -0.00102727 -0.001033648 -0.00102727
> 37 -0.0010507 -0.001042737 -0.0010507
> 38 -0.00104035 -0.000932873 -0.00104035
> 39 -0.00099833 -0.0003704646 -0.00099833
> 40 0.00048605 0.0004308469 -0.00095631
> 41 0.00141655 0.0007775852 -0.00091429
> 42 0.00161438 0.0007775852 -0.00087227
> 43 0 0.0007775852 -0.00083025
> ==================================================
>
>
> *-------------------------------------------------------------------*
> | Dr. Leslaw Bieniasz, |
> | Institute of Physical Chemistry of the Polish Academy of Sciences,|
> | Department of Electrochemical Oxidation of Gaseous Fuels, |
> | ul. Zagrody 13, 30-318 Cracow, Poland. |
> | tel./fax: +48 (12) 266-03-41 |
> | E-mail: nbbienia@cyf-kr.edu.pl |
> *-------------------------------------------------------------------*
> | Interested in Computational Electrochemistry? |
> | Visit my web site: http://www.cyf-kr.edu.pl/~nbbienia |
> *-------------------------------------------------------------------*

Relevant Pages

  • Re: [kde] Fighting line noise with Kmail
    ... signals are expected to be lousy. ... Telecom's line noise, considering I can hear the noise crackling in my phone. ... except that rain does not make spikes, ... pops and crackles could be reduced with filtering, ...
    (KDE)
  • Re: [kde] Fighting line noise with Kmail
    ... where signals are expected to be lousy. ... like what is made by transtector, ... except that rain does not make spikes, ... pops and crackles could be reduced with filtering, ...
    (KDE)
  • Re: What Size Partitions for XP and Linux?
    ... If the power supply is any good, the spikes, etc., ... After chopping, transforming, rectifying again, filtering ... The power supply, absent a well designed surge protector, will take the ...
    (comp.os.linux.misc)
  • Re: a smoothing problem
    ... On Thu, 16 Mar 2005, Ray Koopman wrote: ... I want the spikes to be removed, or filtered out completely, and the ... filtering out higher frequencies would work? ... is what I would like to have as a result of the filtering. ...
    (sci.math.num-analysis)
Loading