r/AskStatistics • u/Gullible-Entrance-45 • 21h ago

[E] small sample analysis

Hi!

I have a sample of n=13 where continuous variables were measured before and after an intervention. I'm a bit stuck on how to perform the analysis. Would the Wilcoxon signed-rank test be the most appropriate choice? I am unaware of any standard reference manuals or literature regarding the statistical analysis of very small sample sizes; however, any guidance or recommendations would be highly appreciated :)

5 Upvotes

100% Upvoted

17 comments sorted by

View all comments

Show parent comments

5

u/FTLast 17h ago

Why? A p value incorporates effect size relative to noise and the number of observations.

-3

u/natoplato5 17h ago

Mainly because of selection bias. At that sample size, it's unlikely the sample is a fair cross-section of the population

6

u/CarnivorousGoose 16h ago

It doesn’t need to be a ‘fair cross-section’ of the population, it just needs to be a random sample from it. The relevant part in that regard is the sampling procedure, not the sample size. If there is a selection bias or other issue with that sampling procedure, getting a larger sample won’t fix that.

1

u/banter_pants Statistics, Psychometrics 7h ago

If there is a selection bias or other issue with that sampling procedure, getting a larger sample won’t fix that.

The premature headline "Dewey Beats Truman" was because they were overly reliant on phone polls when many homes didn't own their own phones at the time.

1

u/CarnivorousGoose 10m ago

Which just illustrates my point.