Re: A misapplication of probability theory in exam grading


pauldepstein@xxxxxxx wrote:
In standardized multiple-choice exams, where each question has 5
options, a common (and extremely dumb) rule used is that correct
answers score 4 but wrong answers score -1.

Presumably the inventors of this rule noticed that the ordinary policy
of simply counting the right answers produces a randomness factor, and
they desired to eliminate it by "penalizing random guessing" according
to the above scheme.

The irony is that this penalty clause does absolutely nothing to
penalize random guessing because a random guess scores 0 on average,
the same as an omitted question.

Therefore, from a game theory stance, an optimal way to play the exams
is to attempt all questions regardless of whether or not there is a
penalty clause.

Therefore, when played correctly, the randomness components in both
grading systems is exactly the same.

The inventors of the modified grading system saw a problem, and very
bizarrely managed to "solve" the problem by changing the grading system
to one with exactly the same problem.

If the intent is to "penalize random guessing" then, assuming -1 for a
wrong answer (and 5 choices per question), the amount for a correct
answer needs to be < 4, not exactly 4.

How did such a mathematically dumb grading system become so universally
accepted? And what on earth is the rationale for the (-1, 4) grading
system, as opposed to (for example) (-1, 3)?

Paul Epstein

Hi,

I don't think it was an attempt to remove randomness, but rather to
arrange so that random guessing doesn't score better than not
answering. Hence the choice of (-1, 4): It gives the expected value 0
for random guessing. Although there is still an element of randomness,
the expected score for the whole test becomes 4 x the number of
questions the candidate actually knew the answer to, regardless of
whether he or she made random guesses or chose to leave some questions
unanswered.

---
J K Haugland
http://home.no.net/zamunda

.

Relevant Pages