Re: Degree of Freedom

On Wed, 16 Jul 2008 11:34:08 EDT, Jack Tomsky <jtomsky@xxxxxxxxxxxxx>
wrote:

Hi,
I've two questions. First, what is degree of freedom
(DoF)? Second,
why it is important in the regression analysis.
Suppose in two similar
type of regression models one regression model I've
10 DoF & another
I've 15 DoF. How more DoF affect the results.
Thank you.
sagar.


If you have a (multiple) regression with k factors (possibly including
an intercept) and N observations, the degrees of freedom for
estimating the variance is N-k.
To perform a statistical test that an individual regression
coefficient beta_i is zero, say, you divide the estimate by its
corresponding standard error and you end up with a t test with N-k
degrees of freedom.

If you're testing several regression coefficients simultaneously, you
end up with an F test where the degrees of freedom in the denominator
is N-k and the degrees of freedom in the numerator is the number of
coefficients tested. All things being equal, the higher the degrees
of freedom, the more accurate will be the estimates.

Jack

Also -- More d.f. in the numerator says that you are using
more predictors, and that is why you expect more accuracy -
for the fit, if not for the effective prediction in the future.
R^2 never decreases. Also, more d.f. in the denominator says
that your sample size is larger, providing more accuracy (all things
being equal).

When the d.f. in the denominator is not large, then the
increase in the number of predictors can come at the cost
of "overfitting". The "Adjusted R^2" is an estimate of what
your equation is worth, accounting for the fact that the
expected R^2 for random predictors is equal to the number
of predictors divided by the total d.f.

--
Rich Ulrich
.

Relevant Pages

  • Re: Degree of Freedom
    ... what is degree of freedom ... why it is important in the regression analysis. ... 10 DoF & another ... more predictors, and that is why you expect more accuracy - ...
    (sci.stat.math)
  • Re: DOF :)
    ... Congrats on reaching DOF Steff. ... Freedom! ... > pick up the smoke to have just one... ...
    (alt.support.stop-smoking)
  • Re: Checking in
    ... Robin, freedom is the 'Holy Grail' and never to be taken for granted. ... >> quitters growing in confidence and success and then in turn, supporting ... >> DOF and then some ...
    (alt.support.stop-smoking)
  • Cuckoo Cat!!! Re: *DOF*DOF*DOF*!!!!!!!!!!!!!!!
    ... Belated congratulations ... on your DOF status:) I am so very proud of you. ... for one day you too shall have freedom and this freedom is ... I shall close with my new, flashy, shiny meter and a great repost from our ...
    (alt.support.stop-smoking)
  • Re: *DOF*DOF*DOF*!!!!!!!!!!!!!!!
    ... Belated congratulations ... on your DOF status:) I am so very proud of you. ... for one day you too shall have freedom and this freedom is ... I shall close with my new, flashy, shiny meter and a great repost from our ...
    (alt.support.stop-smoking)