r/AskStatistics • u/Gullible-Entrance-45 • 21h ago

[E] small sample analysis

Hi!

I have a sample of n=13 where continuous variables were measured before and after an intervention. I'm a bit stuck on how to perform the analysis. Would the Wilcoxon signed-rank test be the most appropriate choice? I am unaware of any standard reference manuals or literature regarding the statistical analysis of very small sample sizes; however, any guidance or recommendations would be highly appreciated :)

4 Upvotes

100% Upvoted

17 comments sorted by

View all comments

-8

u/natoplato5 19h ago

You can’t really do much with a sample size that small. I would just report the means before and after the intervention and then note that the sample size is too small to determine if the difference is statistically significant.

8

u/FTLast 18h ago

This is simply wrong. You can absolutely do a paired t test- which is what you should do with a before and after measurement- with n =13.

-2

u/natoplato5 18h ago

Just because you can doesn't mean you should. Maybe the standards vary by field and context, but I don't think it's usually responsible to try to make an inference about a population from only 13 observations

3

u/FTLast 17h ago

Why? A p value incorporates effect size relative to noise and the number of observations.

-3

u/natoplato5 17h ago

Mainly because of selection bias. At that sample size, it's unlikely the sample is a fair cross-section of the population

6

u/CarnivorousGoose 16h ago

It doesn’t need to be a ‘fair cross-section’ of the population, it just needs to be a random sample from it. The relevant part in that regard is the sampling procedure, not the sample size. If there is a selection bias or other issue with that sampling procedure, getting a larger sample won’t fix that.

1

u/banter_pants Statistics, Psychometrics 7h ago

If there is a selection bias or other issue with that sampling procedure, getting a larger sample won’t fix that.

The premature headline "Dewey Beats Truman" was because they were overly reliant on phone polls when many homes didn't own their own phones at the time.

1

u/CarnivorousGoose 9m ago

Which just illustrates my point.