Re: resampling methods are serious procedure?



Indeed. I guess we are talking about two different things. Most of
your email is about how to teach successfully, which is fine and
important. However, your example with M&M's I would just call
sampling. You can call it re-sampling if you want.

Originally, I thought you were talking about teaching at a higher
level. When I used the word 'classical', I was  referring to
statistical techniques not teaching techniques. Much of re-sampling
theory and practice is dedicated to computing confidence intervals,
classical hypothesis tests, and distributions for estimators, things
that should never be taught at all. Thus any teaching method that
makes them more attractive is a bad thing: the students are better off
if they forget everything.

illywhacker;

OK, let's try again. Recall that the contents of the two shoebox were
different. The fraction of red M&Ms in one box was different from the
fraction of red beads in the other box. The boxes can be viewed as two
different "populations" if you want to put it that way.

When I mentioned "sampling" I meant sampling. That's the concept of
drawing a sample from a source of data... in this context with a
sample that is small compared to the source. So I'd spend a few
minutes doing "sampling" and point out that the fraction of red M&Ms
varied from sample-to-sample. Obviously, this was in preparation for
having students get ready to draw some samples from the boxes.

After you have grasped that much, continue to the next paragraph.

After drawing a sample of M&Ms from each box (blindfolded) we counted
the number of read M&Ms in each sample. The issue now is a matter of
inference. (That's fairly high level stuff for middle-school kids...
and it's even tough for a lot of college undergraduates.) So we are
going to compare the two samples and somehow judge whether the
contents of the two shoeboxes are different... or "the same" (which
for Alfonso's sake, could mean the difference is trivially small.)
This is indeed a tough concept. I've interviewed PhD statisticians (to
work in my group) who could not explain inference in plain English.
Nor could they explain a confidence interval, but that's another
matter.

How shall we do the comparison? Compare the fraction of red beads in
the samples, of course. There are several ways to do this. Do you
want to teach Fisher's Exact Test at this point? I don't think so!!
How about using a normal approximation to the binomial? Hardly!!
So we'll compare the samples (difference of fractions) via resampling.
We are using resampling to compare sample averages expressed as
fractions. Notice that in doing this we do not have to calculate a
"test statistic" like Chi-sq or t or F, etc. We get a "probability"
and we get it without going through a test statistic.

Let's call the difference of those two sample fractions "the observed
difference".

Side note: That's one of the neat things about Fisher's Test. There's
no test statistic involved.

So far so good. Stop me if you get lost.

From a teaching/learning perspective we'll do the resampling with the
physical samples... with the M&Ms in hand. We'll do that several
times. The observed difference of fractions changes every time we
resample. How often does the difference of the fractions in our
samples equal or exceed the difference of fractions we found when we
first calculated that difference... "the observed difference". Keep
records and see that sometimes the resampled differences equal or
exceed the observed difference. Keep a record of how often that
happens.

If this was too much at once, take a break and come back later.

Doing the resampling with physical objects (M&Ms) gets old after
awhile. So show the kids a simple piece of code (yes, the internal
guts and feathers) of a bit of software that simulates the resampling.
How's that for a moving their minds upscale several notches... real
data... resampling done with a simulation... they get to see the
code ... they get a copy of the code... it's simple and they can read
it. The mystique is gone!! Run the software and do about 1000
simulations to get a good approximation to "the probability". Now slow
down and read this very carefully. The simulated resampling is done
on data from those two samples of M&Ms (Remember those? Rhey came from
different boxes).

Then for perspective do this entire experiment again from the
beginning. Open a third box of M&Ms that happen to have the same
fraction of reds as one of the earlier boxes. Proceed to compare
these two boxes. They both have the same fraction of red M&Ms.
(For Alfonso who is probably steaming by now, we are compare two
populations that have the same averages). Set up a team of two
students to do this several times... new samples each time... and let
them run the numbers through the software. Notice "the
probabilities".
Now we have a basis for teaching what some like to call the null
hypothesis (an expression I never ever use in a classroom setting.)

There's just enough time left over to go back to the two original
boxes and do this one more time... two new samples... but do it this
time with larger samples (about 4X larger than in the initial
experiment.) We'll get a "more convincing" probability this time.

Ask some "questions for understanding". Challenge them to think of
some places where they could get data and make some comparisons.
(From comments by their parents, quite a few did this when they got
home.)

Wow!! All in one hour with a promise that they can come back at the
end of the day and eat the M&Ms. Which they did.

If you can show me a faster, more efficient, "sticky" (they'll
remember it the rest of their lives) in one hour I'd like to hear it.

A large fraction of my time at a major corporation (40 years) was
devoted to teaching and convincing. The people I taught were busy and
had short attention spans and no patience for messing around. I've
taught from the bottom (technicians and operators) to the top (boards
of directors). When teaching 'way up the ladder I'd get at most an
hour and sometimes less. They are bright and they are impatient.
Guess what... I often used resampling methods and simulations to teach
them all sorts of interesting things about data.

You wrote...

Much of re-sampling
theory and practice is dedicated to computing confidence intervals,
classical hypothesis tests, and distributions for estimators, things
that should never be taught at all. Thus any teaching method that
makes them more attractive is a bad thing: the students are better < > off if they forget everything.

First of all, I never use the expression "hypothesis test" or variants
on it. Nor the "null hypothesis". Certainly not "alternative
hypothesis"!!! Nor "significant" nor "not significant" nor
"insignificant". (There are solid legal reasons for avoiding those
"significance" expressions like the plague.)

There is a place for teaching the distribution of estimators... that
can be very important... but it's not for everyone.

I take exception to saying they "should never be taught at all".

IMHO, "statistics departments" at universities are most interested in
teaching graduate students (and doing research for money) and breeding
more of their kind. Inspiring undergraduates so they are intrigued
and see the value in applied statistics... and want to learn more...
depends on the personalities of those few who will attempt to teach
that way. Ellis Ott (Rutgers) was one of the very few who could
inspire, enlighten, and intrigue students who were majoring in diverse
fields such as engineering, chemistry, physics, etc.

Yes, I'm talking about teaching methods that inspire people to want to
learn more. I'm claiming that resampling methods can do that.

As I said... let the flaming begin.

To the Original Poster... Let's go back to the beginning. Tell us what
you have in mind. Please. OMU
.

Relevant Pages

  • Re: The American economy
    ... to compare the standard of living in a fair ... >What proportion of males aged 30, then and now, are married, have ... Also a larger fraction of marriages today ... In 1960 people got kids at an earlier age - today it's not at all ...
    (sci.astro.amateur)
  • Re: Gobsmacked
    ... > than 1% of the annual passenger mileage ... Not if you compare like for like, ie compare roads where both cycles ... > only a fraction of the annual RTCs each year have a car driver as the cause. ...
    (uk.rec.cycling)