Re: Alpha, beta, power, CL, CI
- From: Bjoern <bjoern@xxxxxxxxxxxxxx>
- Date: Fri, 07 Jul 2006 19:16:40 +0200
Tony wrote:
Hi!
Would anyone explain to me in simple terms the relationship between
alpha, beta, power, confidence level (CL), and confidence interval
(CI).
Let's describe this with an example, which makes it much easier to avoid too much technical jargon. Assume we are testing a new medical treatment T against an accepted standard treatment S. We will collect some data about the effectiveness of both treatments and then perform a test to see whether we can conclude that T is better than S (=alternative hypothesis) or whether we do not have enough evidence to conclude that (=null hypothesis).
I.e. once we have our data, we calculate a numerical value (=test statistic) based on the data and if that test statistic exceeds a critical value we reject the null hypothesis, otherwise we do not reject it.
Alpha is the probability that our test would conclude that T is better than S even though this is not the case. Alpha is usually fixed and given. E.g. in this example regulatory authorities would usually demand that alpha<=2.5%.
Beta and power are two ways of talking about the same thing. They are all about how likely it is that, given fixed alpha, the number of patients N we will test the treatments on and assuming a scenario under which T actually is better than S (e.g. T cures 90% of all patients, while S only cures 60%), our test will actually conclude that T is better than S. That probability is called power, while beta is simply 1-power.
Note that alpha and N, different scenarios do lead to different power. E.g. if we assume T only cures 80% of all patients and S 60%, then we have less power than for 90% vs. 60%, basically the clearer the difference between the two treatments is, the easier it will be show this in an experiment. What one thus does is assume a suitably chosen scenario, fix alpha and find an acceptable balance between power and N. Only all these choices together really fix the critical values of the test statistic (assuming of course we actually end up collecting as much data as specified etc.).
Confidence intervals corresponding to a certain confidence level are loosely speaking* based on the following idea: Given outcomes to our experiment, we want to quantify how precise our estimates of e.g. for the difference in response rates between the two treatments are, for this purpose we want to specify a range about which we are confident that the true difference will lie in it. Loosely speaking* that is a confidence interval and the confidence level is how confident we are about it.
Hope this helps,
Björn
* A more precise way of saying this is that we specify a way of calculating an interval based on the data we will collect so that with a certain probability the true value will be contained in it. Once we have observed the data and thus calculated such an interval, the true value of course either lies within that interval or not.
.
- References:
- Alpha, beta, power, CL, CI
- From: Tony
- Alpha, beta, power, CL, CI
- Prev by Date: Re: Mann-Whitney score and AUC
- Next by Date: Hotelling's T^2 test when covariance matrix has a special form
- Previous by thread: Alpha, beta, power, CL, CI
- Next by thread: Hotelling's T^2 test when covariance matrix has a special form
- Index(es):
Relevant Pages
|
