Re: A simple but confusing question

From: George Kahrimanis (anakreon_at_hol.gr)
Date: 10/08/04 

Date: 8 Oct 2004 05:10:23 -0700

Here I am trying to clarify some trifling misunderstandings
(imho) in Ian's message, news:<ck3b9b$cqr$1@news-sop.inria.fr>
(7 Oct 2004 14:02:49 +0200). I postpone discussing some very
interesting issues raised by him (in another message) and other
posters.

>One must take a bit of care in comparing these examples. The
>coin tossing experiment corresponds to sampling with replacement.

It is the other way around, as I reckon this matter. Lewis Caroll
does not assume that the bucket initially had half of its balls
white; he assumes that someone has stuffed that bucket choosing
the color of each ball by tossing a fair coin. Therefore, each
time we pick blindly a ball (without replacement) and then record
its color, we just record the outcome of the related coin toss.
The remaining balls in the bucket might as well have been in
a different bucket; we learn nothing about them.

On the other hand, if we replace this ball in the bucket -- say
its color was white -- then the next drawing comes from a
different population: of the N balls, we know that N-1 of them
have Pr(w)=0.5, plus one ball having Pr(w)=1.0 . Therefore the
new prior for p is again a delta function but is set, instead of
0.5, at (0.5 (N-1) + 1.0)/N .

That is why the coin-tossing example corresponds to sampling
Caroll's bucket *without* replacement. I repeat, our absolute
certainty refers not to half of the balls being white, but to
the fairness of the coin of the wo-/man who has prepared the
bucket.

>We discard the assumption p = 1/2 when the data overwhelm our prior
>information. How is this a paradox? In practice, we never have delta
>function prior knowledge, since this requires infinite precision
>measurements.

We do not disagree here!
That is why I call this example a `paradox'. This not to say that
the method is suspicious, as the term `paradox' is understood often
(e.g., Stone's paradoxes, marginalization "paradox"...).
I only meant that, if one insists that the coin was fair, inspite of
sampling 100 white balls and no black ball, her/his attitude would
be called "audacity" at best.

>From the American Heritage Dictionary, 3rd Ed., e-version, cop. 1992
~~~~ paradox ~~~~
1. A seemingly contradictory statement that may nonetheless be true.
2. One exhibiting inexplicable or contradictory aspects.
3. An assertion that is essentially self-contradictory,
though based on a valid deduction from acceptable premises.
4. A statement contrary to received opinion.

I had in mind meaning [1], and perhaps also [4], if by
`received' we understand `naive but entrenched'.
(Greek "doxa" is an opinion relative to the thinker or his cohort,
rather than an objectively justified conclusion.)
The illustrious paradoxes in statistics correspond to
meaning [2]. Sometimes the paradox is a full-blown contradiction,
but people want to speak softly about it. Then the real paradox
imo is that people want to play down a contradiction and try to
"live" with it. (That is, it does not fit my naive and
entrenched opinions about how people should act.)

~ George Kahrimanis