Re: A misapplication of probability theory in exam grading


Duncan Smith wrote:

matt271829-news@xxxxxxxxxxx wrote:
Robert Israel wrote:


In article <1166205811.228936.105840@xxxxxxxxxxxxxxxxxxxxxxxxxxx>,
<jankrihau@xxxxxxxxxxx> wrote:

pauldepstein@xxxxxxx wrote:

In standardized multiple-choice exams, where each question has 5
options, a common (and extremely dumb) rule used is that correct
answers score 4 but wrong answers score -1.

Presumably the inventors of this rule noticed that the

ordinary policy

of simply counting the right answers produces a randomness

factor, and

they desired to eliminate it by "penalizing random guessing"

according

to the above scheme.

The irony is that this penalty clause does absolutely nothing to
penalize random guessing because a random guess scores 0 on average,
the same as an omitted question.

Therefore, from a game theory stance, an optimal way to play

the exams

is to attempt all questions regardless of whether or not there is a
penalty clause.

Therefore, when played correctly, the randomness components in both
grading systems is exactly the same.

The inventors of the modified grading system saw a problem, and very
bizarrely managed to "solve" the problem by changing the

grading system

to one with exactly the same problem.

If the intent is to "penalize random guessing" then,

assuming -1 for a

wrong answer (and 5 choices per question), the amount for a correct
answer needs to be < 4, not exactly 4.

How did such a mathematically dumb grading system become so

universally

accepted? And what on earth is the rationale for the (-1,

4) grading

system, as opposed to (for example) (-1, 3)?

Paul Epstein

Hi,

I don't think it was an attempt to remove randomness, but rather to
arrange so that random guessing doesn't score better than not
answering. Hence the choice of (-1, 4): It gives the expected value 0
for random guessing. Although there is still an element of randomness,
the expected score for the whole test becomes 4 x the number of
questions the candidate actually knew the answer to, regardless of
whether he or she made random guesses or chose to leave some questions
unanswered.

In addition, you want to don't want to unduly discourage a
student who is not completely sure of the answer but has
enough knowledge or skill to see that some answers are more
likely to be correct than others. A poor student who isn't
certain of the answers has an interesting optimization
problem to solve. Suppose there are n questions, and I have
probability p of getting any question right. How many should
I answer to maximize my probability of passing the test?



Why does n matter? Assuming that p > 1/5 wouldn't you answer all of
them? (And if, somehow, p < 1/5, answer none of them?)


Because maximising the expected value is not the same as maximising the
probability of a pass.


Yes, you're right... I didn't think it through.

.

Relevant Pages

  • Re: A misapplication of probability theory in exam grading
    ... a common (and extremely dumb) rule used is that correct ... Presumably the inventors of this rule noticed that the ordinary policy ... they desired to eliminate it by "penalizing random guessing" according ... The inventors of the modified grading system saw a problem, ...
    (sci.math)
  • Re: A misapplication of probability theory in exam grading
    ... a common (and extremely dumb) rule used is that correct ... The inventors of the modified grading system saw a problem, ... If the intent is to "penalize random guessing" then, ...
    (sci.math)
  • Re: A misapplication of probability theory in exam grading
    ... a common (and extremely dumb) rule used is that correct ... The inventors of the modified grading system saw a problem, ... If the intent is to "penalize random guessing" then, ... I answer to maximize my probability of passing the test? ...
    (sci.math)
  • Re: A misapplication of probability theory in exam grading
    ... a common (and extremely dumb) rule used is that correct ... The inventors of the modified grading system saw a problem, ... If the intent is to "penalize random guessing" then, ... I answer to maximize my probability of passing the test? ...
    (sci.math)
Quantcast