Re: Statins do NOT protect against Azlheimer's

From: David Rind (drind_at_caregroup.harvard.edu)
Date: 02/14/05 

Date: Sun, 13 Feb 2005 23:42:00 -0500

Sbharris[atsign]ix.netcom.com wrote:
> If your point is that the probability of statins influencing TIAs in
> this large randomized blinded controlled well-powered study isn't
> *exactly* the converse of the p value for rejection of the null that
> they don't, but is a bit less due to other very improbable scenarios, I
> can concede your point, and agree. I am properly chastized and
> shouldn't have used the precise 1-p number, but noted it as an
> estimate. Otherwise, I think you're being overly contrary. If you were
> *really* that epistemologically contrary in facing life, you wouldn't
> be able to function at all.
>
> SBH

No, this isn't the point I'm making, and it's a hard discussion to have
via the Web. I'll try to briefly clarify in two different ways.

1) Imagine running 20 beautifully designed randomized trials to see
whether eating bananas decreases the risk of TIAs. In the way we
currently define statistically significant, we would expect one of those
20 studies to "show" that bananas decrease the risk of TIAs purely due
to random chance. Let's say the p-value calculated in that study was
0.045. Given that you had no expectation that bananas would decrease the
risk of TIAs (not to mention 19 other negative studies), you should not
conclude that the probability that bananas decreased the risk of TIAs in
that study was 95.5%. The probability is close to 100% that bananas had
nothing to do with the decrease in TIAs in that study and that you were
seeing a statistical blip. In the other direction, imagine that instead
of bananas you were studying aspirin and got that same p-value of 0.045.
You would be wrong to conclude that based on that study the likelihood
that aspirin reduces the risk of TIAs is only 95.5%. Given all the prior
evidence that aspirin works, the probability that it worked in that
study was nearly 100%, no matter what the p-value was.

2) Think of p-values the way you think of sensitivity. That is,
sensitivity is the thing you are generally able to know about a
diagnostic test (probability that the test will be positive in someone
who has the disease), but positive predictive value is what you really
care about clinically (probability that if someone has a positive test
that they have the disease). In the same way, p-values tell you the
probability in the wrong direction -- they tell you the probability that
if nothing were going on (null hypothesis) that you would have seen the
result found in the trial. They do not tell you what the probability is
given the result found in the trial that something is going on (real
effect).

Again, remember that the way we calculate p-values, we expect a value
below 0.05 in one of twenty studies where nothing is happening. So when
someone studies 20 homeopathic remedies and one of them has finds a
p-value below 0.05, we don't want to conclude that there is better than
a 95% chance that the homeopathic preparation really worked in that
study. On the contrary, we want to recognize that the p-value does not
translate in any direct way into the probability that the homeopathic
remedy really worked. 

-- 
David Rind
drind@caregroup.harvard.edu

Relevant Pages

  • Re: Statins do NOT protect against Azlheimers
    ... 94.9% chance that pravachol really does decrease TIAs. ... and there is no simple way to get from there to the probability you ... As we all know from Hume, inductive conclusions about causality always ... and that's to design good ...
    (sci.med.cardiology)
  • Re: Statins do NOT protect against Azlheimers
    ... >whether eating bananas decreases the risk of TIAs. ... >conclude that the probability that bananas decreased the risk of TIAs in ... >of bananas you were studying aspirin and got that same p-value of 0.045. ...
    (sci.med.cardiology)
  • Re: Statins do NOT protect against Azlheimers
    ... >>whether eating bananas decreases the risk of TIAs. ... >>conclude that the probability that bananas decreased the risk of TIAs in ... > in statistics and experimental design by saying that a hypothesis ...
    (sci.med.cardiology)
  • Re: Statins do NOT protect against Azlheimers
    ... It may well be that pravachol and other statins have ... > decrease TIAs in the trial). ... 94.9% chance that pravachol really does decrease TIAs. ... probability was that pravachol decreased TIAs before the study was done. ...
    (sci.med.cardiology)
  • Re: Comparing fractions (or proportions)
    ... THE exact method provides a p-value which is BY ... result as least as extreme as that obtained by chance ... value i.e. the probability to get a data (x, n1, y, ...
    (sci.stat.math)