Re: Effect Size and Sample Size

On Jan 12, 10:28 am, David Winsemius <doe_s...@xxxxxxxxxxx> wrote:
Ryan <Ryan.Andrew.Bl...@xxxxxxxxx> wrote innews:bac8fff5-b978-44a2-b40b-28fbbda4b941@xxxxxxxxxxxxxxxxxxxxxxxxxxxx:





On Jan 10, 7:23 pm, David Winsemius <doe_s...@xxxxxxxxxxx> wrote:
Ryan <Ryan.Andrew.Bl...@xxxxxxxxx> wrote;

My two cents...

The answer to your question depends upon how well the variable
you are analyzing meets the assumptions of the t test. Generally
speaking, as long as (1) the variable is normally distributed in
each group and (2) the variance of the data is reliably the same
between groups, you could have a very small sample size.

Just curious. How do you propose checking any of your assumptions
with "a very small sample size"? And aren't the "assumptions of the
t-test" that you are working with a data situation that is
asymptotically ....large?

--
David Winsemius- Hide quoted text -

- Show quoted text -

I would look at numerical (e.g. skewness, kurtosis) and graphical
(e.g. histogram) descriptive statistics. You can reach a sample size
where these statistics cannot be reasonably interpreted. Even if you
theoretically can run a t test, if you cannot, in some way check the
assumptions, then you shouldn't run it. Moreover, there are serious
limitations to the results of a study with a low sample size, such
as generalizability/reliability of results.

You appear to be dodging the question or perhaps not een understanding
why it was posed. Do you care to discuss what sort of power you expect
from your tests when applied to data with "a very small sample size"?
What value does a statistical test have when its power is effectively
nil? Maybe you should offer some specifics about when you think "very
small" is reached. Then we could see at what point you believe "these
statistics" which were not named, "cannot be reasonably interpreted."  

--
David Winsemius- Hide quoted text -

- Show quoted text -

I wasn't responding to the original question. I was responding to a
completely separate question posted by the person who started the
original post which was "Can I ask, how what is the "smallest" N that
can be used for a t-test (or statistics in general) and why?"

Instead of trying to figure out the "smallest N possible" I
recommended that the person look at the data and see if the
assumptions are tenable. If the assumptions are tenable, then running
a t test may be acceptable, provided the person understand the
limitations of running such a test.

If you'd like me to respond to power when running a t test with small
sample sizes, I can.

Power is compromised when you have a small sample size. Detecting a
"significant difference" is based on power, but power is affected by
several issues - effect size, type I/II error rate, sample size. And
as for the "statistics," I was referring to the "descriptive
statistics" I mentioned. Look at the overall shape and variablility of
the data. If the sample size is too small to get a sense of the
distribution of the data, I'd advise not to run the test. I understand
that running inferential tests with small sample sizes is
controversial due to many issues including generalizability of results
(which I mentioned before), power, etc. Again, running inferential
tests with small sample sizes has serious limitations, however, in my
humble opinion, if you choose to do so then make sure the assumptions
are tenable and understsand that there are serious limitations.

As for the "dodging the question" remark, I would never do such a
thing. If I do not know the answer, I will say I do not know and ask
others. Anybody who has followed my posts knows that I learn/practice/
discuss statistics with great humility, always looking to obtain a
deeper understanding of the subject matter.

.

Relevant Pages

  • Re: Aragorn
    ... They did not voluntary impose or endure any limitations of power. ... their vessel bodies. ... Hobbits. ...
    (rec.arts.books.tolkien)
  • Re: Aragorn
    ... They did not voluntary impose or endure any limitations of power. ... spirit with physical fear, and of being 'killed', though ... the Istari appeared in their incarnate forms ...
    (rec.arts.books.tolkien)
  • Re: Albert Pujols - Greatest first 5 years in history?
    ... would likey come under scrutiny for it's limitations (and ... >>> does have at least two known limitations). ... >> limited statistics, I see little reason why they would scrutinize a far ... the same happening with OPS+ ...
    (rec.sport.baseball)
  • Re: Intelligent Defign as feene through a Microfcope
    ... Right, but given the same limitations of materials, force, and time, ... deliberate processes can outdo non-deliberate processes every time. ... Non-intelligent processes are limited with respect to certain materials ... the degree of power employed. ...
    (talk.origins)
  • Re: Mad idea? Dirt earth, Marconi antennae, and PME
    ... it can come into contact with the "dirt" earth. ... Equally, the power you run ... Are there any statistics about how often such a PME/non-PME fault has ...
    (uk.radio.amateur)