Re: Linear regression


Jens wrote:
Thanks Bob for your prompt answer.

<skip>
That's the usual assumption about the ERRORS of the conditional
distribution of Y given X. More specifically iid N(0, sigma^2) is
the usual assumption.
<skip>
There's nothing to investigate until AFTER you've fitted some tentative
model. THEN, you should examine the residuals (observed errors) to
validate the iid assumptions.
So my model should be Xb=y+e and e=Xb-y should be "close to" iid N(0,
sigma^2).

It's Y = Xb + e, where e is nearly iid N(0, sigma^2).

Y = Xb is what you try to model or fit. What does NOT fit is lumped
into
the random error term e. e may be the results of hundreds and
thousands
of variables that actually have SOME effect on what you measure, but
you
simply (for simplity sake) assume it comprises ONE single random error.


<skip>
By "couplings" I think you mean either statistical or deterministic
relations some pairs of X's. Short answer, as long as the X's are linearly
independent, there's nothing to worry about.
By couplings I mean some unknown relations between the columns. Unknown
in the sence that no-one have investigated such relations properly, but
based on "intelligent" guess some relations surely exists.

But you're NOT interested in those unknown, uninvestigated between
the "columns" (on the RIGHT hand side -- the independent variables).
If you want to explore its relation with the others, put it on the LEFT
as
the dependent variable.

<skip>
You seem to have the common misconceptions and misunderstanding
between "linear indepdence" (linear algebra) and stochastic
(statistical) independence of the error terms.
There are several threads in sci.stat.math specifically about the
difference of these two concepts.

Yes indeed I need to try to understand this topic a better manner.
Actually my current observations are only a subset of a larger set.
I'll return later with more specific questions.

What you are asking are the most BASIC questions regarding a
Linear (Regression) model. The fact that you wrote you model as

Xb=y+e and e=Xb-y

suggests that you haven't read any book on the subject. Try
Neter, Wasserman, Kutner & Nachsheim's book on "Applied
Linear Models" which is a standard textbook in many universities
and recommended by many (including myself) as a good book
on the subject.

-- Bob.

.

Relevant Pages

  • Re: Linear regression
    ... you should examine the residuals (observed errors) to ... validate the iid assumptions. ... By couplings I mean some unknown relations between the columns. ... between "linear indepdence" and stochastic ...
    (sci.stat.math)
  • Overdetermined mixed linear/quadratic system - least squares problem
    ... I have a system of k linear and m quadratic equations in d unknown ... A least square solution is required in this case. ... If the system were purely linear, ...
    (sci.math)
  • Re: solving 10 equations with 10 unknown
    ... unknown of the form ... Otherwise, you might try adapting FMINSEARCH, ... for instance by minimizing the function ... linear combinations of a^3, b^2, c, and a constant. ...
    (comp.soft-sys.matlab)
  • Re: ? best approx of matrix of a linear sys
    ... If one has a matrix, rather that a linear function, then one has the ... standard basis of column matrix vectors with a single entry of 1 and ... The matrix A is unknown. ...
    (sci.math)
  • Re: CRC reverse engineering
    ... One conceptually simple thing you can try is linear algebra. ... solve the system of linear equations and find the linear function ...
    (sci.crypt)