Re: Standard Deviation and "False Alarm" Rate

Richard,

Thank you very much for the assistance. What he is trying to comment
on is a methodology that is currently being utilized to identify high
and low performers. The current methodology simply utilizes standard
deviation differences of 0.5 and 1.0. He was arguing that these
differences may not constitute statistically significant differences
from the overall mean. The percentage he was trying to calculate was
the percentage that would display a difference of 0.5 and 1 standard
deviations but not necessarily be statistically significant. Would the
proper way to calculate this be as follows:

For 0.5 Standard Deviations:
[(31% above 0.5 standard deviations) + (31% below 0.5 Standard
deviations) ] -
[(5% above 1.96 standard deviations) + (5% below 1.96 standard
deviations) ] = ~52%
Therefore, 52% of the time when the student's score is 0.5 or more
standard deviations above the mean the difference will not be
statistically significant.

For 1.0 Standard Deviations:
[(16% above 1 standard deviations) + (16% below 1 Standard deviations)
] -
[(5% above 1.96 standard deviations) + (5% below 1.96 standard
deviations) ] = ~22%
Therefore, 22% of the time when the student's score is 1.0 or more
standard deviations above the mean the difference will not be
statistically significant.

Please let me know if what I have proposed above seems reasonable to
you.

Thank you very much for your time!

L.T.

Richard Ulrich wrote:
On 8 Sep 2006 15:12:57 -0700, bodybuilder@xxxxxxxxxxxxxx wrote:

Hello. I am a stats novice and have a question regarding how the
percentages quoated in the following statement were derived:

"It is important to note that even when there is no significant
difference between a student's score and the class average, tehse two
values will still differ by 0.5 or more standard deviations 35 percent
of the time, and by 1 or more standard deviations 24 percent of the
time. This represents a high percentage of "false alarms," which can
lead to inaccurate conclusions."

No additional information is provided. I am trying to determine how to
derive the 35% and 24% quoted above. Any assistance you can provide
would be greatly appreciated! Thank you!

Usually, such claims make use of the normal distribution.

This one misuses the normal distribution in a unique way -
I think I found where the writer got his number

The proper numbers would be that 31% score more than
0.5 SD above the mean, and another 31% score that much
below it, 62% in all. And 16%+16%, or 32% score, more
than 1 SD from the mean.

A handy table of the normal distribution shows me that the
numbers from the writer, 0.35 and 0.24, happen to be the
y-ordinates for the normal curve, at 0.50 and 1.0 respectively.
It seems that he looked them up, and used the wrong column.


--
Rich Ulrich, wpilib@xxxxxxxx
http://www.pitt.edu/~wpilib/index.html

.

Relevant Pages

  • Re: Standard Deviation and "False Alarm" Rate
    ... The current methodology simply utilizes standard ... deviations but not necessarily be statistically significant. ... For 0.5 Standard Deviations: ... such claims make use of the normal distribution. ...
    (sci.stat.edu)
  • Re: Standard Deviation and "False Alarm" Rate
    ... The current methodology simply utilizes standard ... deviations but not necessarily be statistically significant. ... For 0.5 Standard Deviations: ...
    (sci.stat.edu)
  • Re: Vanilla junk problem. Suggestions? Theories?
    ... It's capped at 4 standard deviations as well, ... Generate a random integer number of NORMAL distribution ... stand = Amount by which one standard deviation differs from the ... s16b Rand_normal ...
    (rec.games.roguelike.angband)
  • Re: Standard Deviation and "False Alarm" Rate
    ... values will still differ by 0.5 or more standard deviations 35 percent ... lead to inaccurate conclusions." ... such claims make use of the normal distribution. ... I think I found where the writer got his number ...
    (sci.stat.edu)