At the simplest (“paired tests”) this is testing whether the mean change across two measurements made in a number of participants is large enough that a difference as large as was seen in the data is unlikely to have arisen by chance sampling from a population in which change is zero. (I.e. a typical NHST approach to quantitative statistics.) The big difference is from between group comparisons where means scores in two different groups are compared. Within-participant tests can be more complicated than this, e.g. there may by more than two measurements over time and though the basic idea is located in the ANOVA (ANalysis Of VAriance) parametric model assuming Gaussian distributions, there are non-parametric equivalents for some within-participant tests, the most famous is the Wilcox-Mann-Whitney test. The confidence interval/estimation approach in which the size of the population change is estimated from the data can also be used and is probably preferable (as usual) to the NHST approach.
Also known, very reasonably, as repeated measures tests.
Details #
The fundamental issue in within-participant tests is that no variance from simple differences between people has to be included in the mathematical/statistical model as both measurements are in the same person for each participant. This makes within-participant tests more statistically powerful, i.e. they need fewer participants to have a certain likelihood of detecting a real, non-zero/non-null change in the population model. There was a vogue for designing studies to use matched pairs of participants and then to use a paired/within-participant tests, a popular example was to match participants on gender and age though sometimes the matching is on more other variables. I think this is rarely justified as it’s assuming that all possible differences between the participants in whatever is being measured are accounted for by the matching. I find it hard to think of situations in which this is remotely plausible.
The development of multi-level models has pushed the real importance of the within-participants vs. between groups distinction to the history of statistics as MLMs can handle much more complex models, in our field typically testing for change across multiple change measure scores in different participants taking into account “nesting” e.g. perhaps looking for differences across therapists, or by service: the range of such models is huge. However, understanding the principle of the within-participants vs. between groups distinction is still useful as the MLM models turn on the same issue of where you are modelling random effects and where you are safe to make comparisons within participants.
Try also #
Between groups comparisons
Change
Confidence intervals
Estimation
Gaussian (“Normal”) distribution
Multi-level models (MLMs)
Wilcox test
Mann-Whitney test
Non-parametric tests
Null Hypothesis Significance Testing (NHST)
Parametric tests
Statistical power
Chapters #
Not addressed directly in the book, would have been in Chapter 5!
Online resources #
None currently.
Dates #
First created 9.ii.24, tweaked links 7.x.24.
