Re: spatial autocorelation methods

From: jim (_at_m.sjedging@mwt.net)
Date: 10/13/04 

Date: Wed, 13 Oct 2004 09:43:37 -0500

Ara.T.Howard@noaa.gov wrote:
>
> On Tue, 12 Oct 2004, Fred Marshall wrote:
>
> > Is the data subject to lateral movement? Or, are the positions fixed and
> > you just have new data sets and are trying to find the best match?
>
> hmmm. i'm not positive what you are asking? the data are satelite images
> (essentially masked) designating fires as detected by a particular sensor.
> eg.
>
> 0 0 0 0
> 0 1 1 0
> 0 1 1 1
> 0 0 0 1
>
> where the ones might identify a 'cluster' of fire pixels. the task is to
> identify the 'clusteredness' of the pixels en route to some statistical
> verifaction against another data set (which set of variables is corelated with
> a 'detect' in the other data set). i had initially been looking into
> algorithms like moran's i, which yield a value indicating the level of
> clusteredness, but am leaning towards brute force identification of clusters
> and their sizes and all algorithms looked at so far requir input data to be
> square.
>

Hi Ara
        Your not being very clear in defining the problem. I don't understand
what the problem is with input data needing to be square. It seems to me
your data *is* rectangular and should be easily divisible into squares.
Also, what do you mean by cluster and clusteredness? What exactly are
you trying to measure?
         It sounds like you have data in a rectangular grid where each square
(meter^2, acre, square mile) of the grid is assigned a value of 1 (true)
or zero (false) depending on whether a wildfire is present or not. Also
it appears that you want to in some way quantify when groups of
neighboring squares reach some measure of critical mass.
        Assuming the above is correct, you may want to look at dilation and
erosion filters applied to the data to simplify the problem. Erosion
would involve iterating through the array, so that for each square with
a 1, if some threshold (say more than half) of the 8 neighbors is 0,
than change to 0 otherwise do nothing. For dilation iterate through the
array, and for each square with 0 if some threshold (say more than half)
of the 8 neighbors is 1, than change to 1. These sort of filters are
used in similar problems identifying certain scattered tissue types in
medical imaging.
        Your algorithm might work like this: you grow the areas with 1 with
dilation filter (possibly more than one pass). And then apply erosion
filter (again maybe several passes) to shrink the areas marked as 1.
After sufficient erosion what is left that is still 1's will be the
areas that were of sufficient critical mass to survive.
        The trick of course is to find the appropriate thresholds for dilating
and eroding, and also the appropriate number of iterations to apply them
so that you get the best fit predictive model of what your data
represents and what you are trying to measure.
        
        Also, there is the issue that some sort of metric of clusteredness has
already been applied to the data when at some point it was originally
quantified that each square was either a 1 or a 0. To what extent the
sensitivity of that measurement will affect the outcome is an open
question.

-jim

> > 2-D correlation is the same as 1-D correlation but with a double integral -
> > which you might envision as a raster scan with different degrees of overlap
> > perpendicular to the raster scan direction - if the coordinate system is
> > rectangular. No matter what it's still multiply and integrate (i.e.
> > accumulate) at each value of offset. So, if there's no lateral motion
> > involved then there's no need to look at offset registration and you do a
> > single zero-valued offset multiply and integrate.
>
> at a fundemental level this is, of course, always true since memory is flat.
> i guess what i am trying to say is that my math has degraded to the point
> where i'm unsure i can 'flatten' the existing algorithms out there. ;-)
>
> > As far as missing data is concerned, there would be a couple of issues:
> > - ignoring those coordinates
>
> the easiest part in general, but tricky along edges of ragged shapes...
>
> > - weighting for the loss of inputs to the integration.
> > Assume that there is a grid for NxN data points and that the number of
> > missing data points is [I J K L M .... ]
> >
> > The potential sum of unit samples is N^2. If the real potential sum is
> > N^2-I, N^2-J, ... etc. then I think you might weight the results by
> > N^2/[N^2-m] where m is the number of missing points for each integration -
> > so that a plane of unit samples (with some missing) will always have a
> > weighted integral of N^2. The same applies if you have to shift the data
> > and compute multiple points of the correlation function and you might treat
> > the nonoverlapping areas the same way - as "missing". If you do, then the
> > edges will be noisy!! So, maybe it's better to let the nonoverlap be zeros.
>
> the moran's i impl. i have pads the edges of the deviation array with zero for
> this reason. again, i'm a bit fuzy here. sorry.
>
> > All this is just logical to me - no special knowledge on my part.
>
> more than i have ;-) forgive me if i'm a bit slow on the uptake here - it's
> been a long time since i've done any math except that involving powers of two.
>
> regards.
>
> -a
> --
> ===============================================================================
> | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
> | PHONE :: 303.497.6469
> | When you do something, you should burn yourself completely, like a good
> | bonfire, leaving no trace of yourself. --Shunryu Suzuki
> ===============================================================================

----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= East/West-Coast Server Farms - Total Privacy via Encryption =---

Relevant Pages

  • Re: spatial autocorelation methods
    ... a 'detect' in the other data set). ... clusteredness, but am leaning towards brute force identification of clusters ... > accumulate) at each value of offset. ... > and compute multiple points of the correlation function and you might treat ...
    (sci.image.processing)
  • Re: Square root of a negative rral value
    ... data set that went with this, crashes on trying to take the square ... square root operation before terminating correctly with a final table. ... When the standard deviation is zero, ...
    (comp.lang.fortran)
  • Nonlinear Least Square(Marquardt)
    ... y data set and I want to do nonlinear least square ... fitting of the data. ... % errordlg('Argument length mismatch','Error in function mkety') ...
    (comp.soft-sys.matlab)
  • Weight Squared
    ... I'm working at the office with an old survey data set that I ... didn't create - and no one wrote a methodology for. ... Can anyone explain someone might square a data set's weight variable? ...
    (sci.stat.math)
Quantcast