Re: mulitivariate linear regression and Bonferroni


but the Bonferroni adjustment is really harsh. with 10 variables, you
go from 95% to 99.5%. In some settings this may make sense but in many
social science settings, this sounds completely unrealistic.

Bruce Weaver wrote:
David wrote:
This is not the usual practice for regression.

Historically, multiple comparisons developed for experimental designs
to control the overall error rate of inferences about a set of
treatments.

Should this be done with the coefficients of a single regression? I
think the answer is no, unless there is a clear, logically compelling
reason why. While I haven't seen one, I am open. The lack of common
use of multiple comparisons this way shows that few or none are
convinced this is necessary or useful.

The answer to this question also bears on the larger questions of
whether a single report, a single research study, or a single lifetime
should also be subjected to multiple comparisons, such as Bonferroni
bounds, on all the tests performed.

Sounds like an overzealous reviewer.

I don't really disagree with anything you've said. But I'll play
devil's advocate for a moment anyway. ;-)

Suppose I have an ANOVA design with one (pre-specified) control
condition, and 3 treatment groups. Dunnett's test is recommended for
comparing each of k-1 treatments to a control.

But suppose I decide to compute indicator variables for my 3 treatment
groups, and do the analysis using the REGRESSION procedure rather than
the ANOVA/GLM procedure. The F-test (MS_regression/MS_residual) will
be identical to the omnibus F-test from my ANOVA. But the t-tests on
my 3 regression coefficients will not be equivalent to the t-tests I
get using Dunnett's test. Rather, they will be equivalent (I think) to
the t-tests I'd get if I used Fisher's LSD.

So, I sympathize with the OP, and have wondered myself why we make such
a fuss about keeping a handle on familywise alpha in one context
(GLM/ANOVA), but not in the other (regression), even when the analyses
are identical (apart from which procedure you used to do it).

Cheers,
--
Bruce Weaver
bweaver@xxxxxxxxxxxx
www.angelfire.com/wv/bwhomedir

.

Relevant Pages

  • Re: mulitivariate linear regression and Bonferroni
    ... multiple comparisons developed for experimental designs ... Should this be done with the coefficients of a single regression? ... Suppose I have an ANOVA design with one control ... comparing each of k-1 treatments to a control. ...
    (sci.stat.edu)
  • Re: Interpreting results of logistic regression
    ... The first column is indicating different days, there are two treatments ... Parameter values after fitting a logistic regression are the following ones: ... which appears to be strongly different from zero. ... An important oversight may be that you can't actually take a logit ...
    (sci.stat.consult)
  • Re: mulitivariate linear regression and Bonferroni
    ... This is not the usual practice for regression. ... multiple comparisons developed for experimental designs ... to control the overall error rate of inferences about a set of ...
    (sci.stat.edu)
Quantcast