Re: A simple but confusing question

From: Ian Jermyn (Ian.Jermyn_at_sophia.inria.fr)
Date: 10/08/04 

Date: Fri, 8 Oct 2004 14:46:57 +0200

This is simply a question of the meaning of the word 'corresponds', not a
substantive issue. In the following I will refer to 'heads' and 'tails' as
'white' and 'black'. The following statements are true:

1) Tossing a coin with a delta function prior at 0.5 for its probability of
coming up white, means that the probability of w_{n + 1} given W_{n} is
always 1/2.

2) Sampling balls from a bucket without replacement using Carroll's prior
means that the probability of w_{n + 1} given W_{n} is always 1/2.

In this sense, tossing a coin 'corresponds' to sampling without replacement.

3) The probability of W_{n} given M and N for coin tossing is (M/N)^{n}.

4) The probability of W_{n} given M and N for sampling with replacement is
(M/N)^{n}.

In this sense, coin tossing 'corresponds' to sampling with replacement.

5) The probability of W_{n} given M and N for sampling without replacement
is C(M, n) / C(N, n).

The fact that (1) and (2) are true, given that (3) and (5) are true, is due
to the fact that the prior on M, or equivalently M/N, or equivalently the
probability of coming up heads, is *different* in (3) and (4).

However, the prior on the probablity with which the balls are picked and put
into the bucket is indeed a delta function at 1/2. If you interpret the
experiment in a serial manner (which is essentially what you are doing),
where a ball is placed in the bucket and immediately sampled, then of course
this does not merely 'correspond' to, but is isomorphic to the coin tossing
case, since the probability of picking a colour given the colour in the
bucket is a delta function. Intuition does strongly suggest that the serial
experiment and the experiment in which the balls are picked first and
sampled later are equivalent, but only calculation can prove this.

I have dictionary too thanks. The result is not 'seemingly contradictory' to
me however, and hence is not a paradox in sense (1). 'Unexpected at first
glance' might be a better way of characterizing it.

Ian. 

-- 
--------------------------------------------------
Ian Jermyn
ianjermyn@wanadoo.fr
"George Kahrimanis" <anakreon@hol.gr> a écrit dans le message de
news:3ce8f26b.0410080410.32d89e6c@posting.google.com...
> Here I am trying to clarify some trifling misunderstandings
> (imho) in Ian's message, news:<ck3b9b$cqr$1@news-sop.inria.fr>
> (7 Oct 2004 14:02:49 +0200). I postpone discussing some very
> interesting issues raised by him (in another message) and other
> posters.
>
> >One must take a bit of care in comparing these examples. The
> >coin tossing experiment corresponds to sampling with replacement.
>
> It is the other way around, as I reckon this matter. Lewis Caroll
> does not assume that the bucket initially had half of its balls
> white; he assumes that someone has stuffed that bucket choosing
> the color of each ball by tossing a fair coin. Therefore, each
> time we pick blindly a ball (without replacement) and then record
> its color, we just record the outcome of the related coin toss.
> The remaining balls in the bucket might as well have been in
> a different bucket; we learn nothing about them.
>
> On the other hand, if we replace this ball in the bucket -- say
> its color was white -- then the next drawing comes from a
> different population: of the N balls, we know that N-1 of them
> have Pr(w)=0.5, plus one ball having Pr(w)=1.0 . Therefore the
> new prior for p is again a delta function but is set, instead of
> 0.5, at (0.5 (N-1) + 1.0)/N .
>
> That is why the coin-tossing example corresponds to sampling
> Caroll's bucket *without* replacement. I repeat, our absolute
> certainty refers not to half of the balls being white, but to
> the fairness of the coin of the wo-/man who has prepared the
> bucket.
>
> >We discard the assumption p = 1/2 when the data overwhelm our prior
> >information. How is this a paradox? In practice, we never have delta
> >function prior  knowledge, since this requires infinite precision
> >measurements.
>
> We do not disagree here!
> That is why I call this example a `paradox'. This not to say that
> the method is suspicious, as the term `paradox' is understood often
> (e.g., Stone's paradoxes, marginalization "paradox"...).
> I only meant that, if one insists that the coin was fair, inspite of
> sampling 100 white balls and no black ball, her/his attitude would
> be called "audacity" at best.
>
> From the American Heritage Dictionary, 3rd Ed., e-version, cop. 1992
> ~~~~ paradox ~~~~
> 1. A seemingly contradictory statement that may nonetheless be true.
> 2. One exhibiting inexplicable or contradictory aspects.
> 3. An assertion that is essentially self-contradictory,
> though based on a valid deduction from acceptable premises.
> 4. A statement contrary to received opinion.
>
> I had in mind meaning [1], and perhaps also [4], if by
> `received' we understand `naive but entrenched'.
> (Greek "doxa" is an opinion relative to the thinker or his cohort,
> rather than an objectively justified conclusion.)
> The illustrious paradoxes in statistics correspond to
> meaning [2]. Sometimes the paradox is a full-blown contradiction,
> but people want to speak softly about it. Then the real paradox
> imo is that people want to play down a contradiction and try to
> "live" with it. (That is, it does not fit my naive and
> entrenched opinions about how people should act.)
>
> ~ George Kahrimanis