Re: Finding Significant Data Points in a Time Series



On 14 nov, 21:35, "erikcw" <erikwickst...@xxxxxxxxx> wrote:
Hi all,

I have a collection of ordered numerical data in a list (time series).
The numbers
when plotted on a line chart make a low-high-low-high-high-low (random)
pattern. I need an algorithm to extract the "significant" high and low
points from this data.

Here is some sample data:
data = [0.10, 0.50, 0.60, 0.40, 0.39, 0.50, 1.00, 0.80, 0.60, 1.20,
1.10, 1.30, 1.40, 1.50, 1.05, 1.20, 0.90, 0.70, 0.80, 0.40, 0.45, 0.35,
0.10]

In this data, some of the significant points include:
data[0]
data[2]
data[4]
data[6]
data[8]
data[9]
data[13]
data[14]
....

How do I sort through this data and pull out these points of
significance?

Thanks for your help!

Erik

Hi,

This is the way I would do to select "significant" points
(just a toy example programmed in Mathematica) :

In[1]:=data ={10,50,60,40,39,50,100,80,60,120,110,
130,140,150,105,120,90,70,80,40,45,35,10};

In[2]:=nextdata=RotateLeft[data]

Out[2]={50,60,40,39,50,100,80,60,120,110,130,140,
150,105,120,90,70,80,40,45,35,10,10}

In[3]:=priordata=RotateRight[data]

Out[3]={10,10,50,60,40,39,50,100,80,60,120,110,130,
140,150,105,120,90,70,80,40,45,35}

In[4]:=nmax=Length[data]-1; nn=Range[0,nmax]

Out[5]={0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,
17,18,19,20,21,22}

In[6]:=data2={nn,priordata,data,nextdata}//Transpose

Out[6]={{0,10,10,50},{1,10,50,60},{2,50,60,40},
{3,60,40,39},{4,40,39,50},{5,39,50,100},{6,50,100,80},
{7,100,80,60},{8,80,60,120},{9,60,120,110},
{10,120,110,130},{11,110,130,140},{12,130,140,150},
{13,140,150,105},{14,150,105,120},{15,105,120,90},
{16,120,90,70},{17,90,70,80},{18,70,80,40},{19,80,40,45},
{20,40,45,35},{21,45,35,10},{22,35,10,10}}

In[7]:=test[{n_,a_,b_,c_}]:=
{n,b,Which[n == 0||n == nmax,"?",
a < b && c < b, "hi",
a > b && c > b, "lo",
a <= b <= c, "up",
a >= b >= c, "down",
True, "?"]};

In[8]:=Map[test,data2]

Out[8]={{0,10,?},{1,50,up},{2,60,hi},{3,40,down},{4,39,lo},
{5,50,up},{6,100,hi},{7,80,down},{8,60,lo},{9,120,hi},
{10,110,lo},{11,130,up},{12,140,up},{13,150,hi},{14,105,lo},
{15,120,hi},{16,90,down},{17,70,lo},{18,80,hi},{19,40,lo},
{20,45,hi},{21,35,down},{22,10,?}}

In[9]:=significants = Select[Map[test,data2],
Last[#] == "hi" || Last[#]== "lo" & ]

Out[9]={{2,60,hi},{4,39,lo},{6,100,hi},{8,60,lo},{9,120,hi},
{10,110,lo},{13,150,hi},{14,105,lo},{15,120,hi},{17,70,lo},
{18,80,hi},{19,40,lo},{20,45,hi}}


hth

V.Astanoff

.

Relevant Pages
Quantcast