Re: Polynomial regression analysis

In article <1134427042.361359.101640@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>,
Reef Fish <Large_Nassau_Grouper@xxxxxxxxx> wrote:

>junoexpress wrote:
>> Hi,

>> I have a simple question about multiple linear regression for a
>> polynomial model.

>That is just another multiple LINEAR regression model, although
>it is sometimes labeled as a polynomial model because most
>people know what a polynomial is, and not many (proven in
>the sci.stat.* groups) know what a LINEAR model is, in the
>multiple regression context/definition.


>> I'm fitting sections of a data set to a polynomial
>> (at most 3rd order), and there are times (due to noise which I can
>> smooth out some with a running avg, but not all of it), that the
>> polynomial will fit a cubic when in fact a line would probbaly work
>> just as well. That is, the coefficients on the cubic are very small in
>> magnitude.

>Here is the key to both the statement and the answer to your
>question -- what is the meaning of "very small".

>What you meant is "very small" in absolute NUMERICAL magnitude
>such as .0001 is small, and 10,000 is big.

>But the opposite could be true when the value is the COEFFICIENT
>of a multiple regression!

>The .0001 in the coefficient of X^3 may be statistically much BIGGER
>(say 10 std err) than the coefficient 10,000 of X (which may be
>statistically indistinquishable from a zero!).


>> I could do a traditional regression analysis considering the effects of
>> having a third order coeff as opposed to having a quadratic, etc using
>> conditional SSE values .

>That's getting warmer, but the SSE only measures the magnitude
>of the SSE, wihtout a statistical yardstick to tell you whether it is
>big or small.


>> OTOH though it occurred to me that may be a very
>> simple way to deal with this problem would be to compute the
>> (1-alpha)100% CI for the cubic and quadratic coeffs and if they
>> contained zero, say that a linear model was good enough.

>That's getting still warmer, but STILL not quite correct. That
>will tell you whether each of the polynomial coefficients is
>statistically different from zero or not, but the ONLY coefficient
>that is relevant for a cubic polynomial is the coefficient of the
>CUBIC term. The constant, linear, or quadratic coefficients
>may or may not be different from zero and the polynomial is
>still CUBIC, if the cubic coefficient is statistically different from
>zero.

>> general,
>> would this approach tend to work most of the time (I'm not interested
>> in considering pathological cases) and do people ever do this?

>That's the ONLY way to test if the polynomial is cubic -- look
>ONLY at the cubic coefficient. fitted to a model with linear and
>quadratic terms -- which measures the effect of the
>cubic term (in the presence of the linear and quadratic terms).
>If it's statistically significant, then the polynomial is cubic.

This is not necessarily the case if the scale of the
independent variable has a natural zero. In this case, it
can be of interest whether the lower degree coefficients
are different from zero.

>If not, then the lower order polynomial model may be quadratic,
>linear, or constant.

See the above. If there is not a natural zero, what you
have said is correct.

For example, we have that the black body radiation
is proportional to the fourth power of the absolute
temperature. This is quite different from following
a fourth degree polynomial.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hrubin@xxxxxxxxxxxxxxx Phone: (765)494-6054 FAX: (765)494-0558
.

Relevant Pages

  • Re: Polynomial regression analysis
    ... That is just another multiple LINEAR regression model, ... But the opposite could be true when the value is the COEFFICIENT ... > contained zero, say that a linear model was good enough. ...
    (sci.stat.edu)
  • Re: Polynomial regression analysis
    ... >>That is just another multiple LINEAR regression model, ... >>> contained zero, say that a linear model was good enough. ... >>will tell you whether each of the polynomial coefficients is ...
    (sci.stat.edu)
  • Re: LinAlg: Vector Spaces - need help understanding an example
    ... > As for the business of setting all linear combinations to the zero vector, ... > guarantees it is a vector space in either case. ... > entries of both vectors add to zero. ...
    (sci.math)
  • Re: Calculating two free axes
    ... You can permuate the coordinates if vn is zero to get ... On paper you solve systems of linear equations easily. ... generate a set of linearly independent vectors huh? ... algorithm to do this while I have. ...
    (comp.graphics.algorithms)
  • Re: LinAlg: Vector Spaces - need help understanding an example
    ... > I'm a hobbyist math student, currently studying linear algebra. ... > both satisfy the condition that their entries add to zero then v + w ... guarantees it is a vector space in either case. ...
    (sci.math)