Re: Weighting regression points?

From: Jonathan Greenberg (greenberg_at_ucdavis.edu)
Date: 07/20/04 

Date: Tue, 20 Jul 2004 23:06:09 GMT

I should probably explain what I'm doing, to understand why I want to weight
certain points stronger than others as opposed to getting rid of those
points -- this is being used for a remote sensing calibration exercise -- we
have two satellite images, with overlapping regions between them. One of
them is a reference image (where the values of the pixels have been
transformed into units of reflectance), and we want to make the second image
have the same brightness values for the same surface -- in other words, if
we are generating a set of pairs of pixels (one from the reference, one from
the uncalibrated image), and generating regression coefficients, which we
will then apply to the uncalibrated image (this is known as "empirical line
calibration" in the literature). The issue comes up that certain pairs of
pixels are better than others, but we can't check all of them and there is
no good way to get rid of the "bad" ones (consider in one image we have a
parking lot and the second image we have a cloud -- this would appear as an
outlier). However, the BULK of the signal will be along the regression
line, but I want to "boost" the results by having a program import, say, a
small list of pixels (small = not sufficient to get to the appropriate
sample size without the other unknown pixels) which we want to weight more
heavily than others.

Since this is generating a database of 1000s of pixel pairs, doing manual
outlier analysis is not really an option (this is being developed to
automate this process).

Thoughts?

--j

On 7/20/04 1:03 PM, in article cdjtpe$177$1@news.kodak.com, "Paige Miller"
<paige.miller@kodak.com> wrote:

> Jonathan Greenberg wrote:
>
>> I was wondering if anyone knew of good techniques for generating a linear
>> regression where a) we assume there IS a linear relationship and b) certain
>> data points are more "truthful" than others and we want to weight them to
>> influence the regression parameters more than other points. Is there a good
>> way to weight certain data more heavily than others (the obvious way is to
>> create duplicates of the "superior" data points). Thoughts?
>
> More truthful?
>
> If you don't believe certain data points, get rid of them. (I often
> do this with outliers, especially when I have an assignable cause
> that makes them outliers).
>
> Well, now, maybe you meant to say that some data points come from a
> distribution that is more or less variable than other data points.
> If that is the case, you can use weighted least squares regression.
> which should be covered in most textbooks on regression.

Relevant Pages

  • Re: formula for linear regression
    ... > posts in this thread how familiar you are with regression. ... you might have data on the weight and height of 50 ... > The TREND function will take account of the numeric relationship between ... This is termed "multiple regression." ...
    (microsoft.public.excel.worksheet.functions)
  • Re: formula for linear regression
    ... "Simple" linear regression seeks to predict the value of one variable from ... you might have data on the weight and height of 50 ... The TREND function will take account of the numeric relationship between ... This is termed "multiple regression." ...
    (microsoft.public.excel.worksheet.functions)
  • Re: weighted regression
    ... regression line though these but some y-values are more uncertain ... than others and I want to put less weight on them, ... constant term as usual, ... matrix terms, the column of 1's in the design matrix would be replaced ...
    (sci.stat.math)
  • Re: maintain the size of an image file (jpeg)
    ... Hmm - *saving* an image, in and of itself, does nothing to reduce the weight ... since you are removing pixels when you do that. ... One of the worst things you can do to a JPG image, by the way, is to JPG it ...
    (microsoft.public.frontpage.programming)
  • Re: A new method for robust regression?
    ... Gottfried Helms wrote: ... (i.e. a more model driven type of robust regression)? ... Why not weight ...
    (sci.stat.math)
Quantcast