Literally “before this” I think. Crucial idea in inferential statistics, “tests” and the Null Hypothesis Significance Test (NHST) paradigm. The key idea is that you have to have planned your test before looking at the data. Opposite of post hoc tests/hypotheses.
Details #
The beautiful, if rather abstract, logic of the NHST depends on you having some equipoise, some of that “blinded justice” with a pair of scales and a blindfold to weigh things without bias or prejudgement. If you look at your data and run tests on everything against everything and just report the findings that were “statistically significant” you have completely misunderstood the equipoise and logic of the method … or, horribly possible, you are deliberately abusing it.
There was a story, somewhere I worked, avowed with some conviction by those older than me, of a visiting professor running down the corridor shouting “I’ve got a p < .001″ result” only to have it pointed out that the p value was for the correlation between date of birth and age. Whether true or not, that sort of illustrates the problem of “p value hunting” and the importance of only using the NHST on specific, pre hoc hypotheses.
This has become a central part of the movement for “reproducibility” and for “pre-registration” and “protocol papers”. These last two should be ways of clarifying what analyses were really pre hoc and following a protocol, including a “Data Analysis Plan” (DAP) created, and registered somewhere public, ideally before data collection started, certainly before the data were seen by the data analysts.
This doesn’t rule out post hoc analyses particularly around what I call emergent features or findings. You might not have planned to look at parenthood as a factor in improvement across therapy but if it turns out, in your purely descriptive/exploratory analyses to show a very strong association with change then there is nothing illegal (!) in describing that association and even giving it a p value, or more logically, a confidence interval. What matters is that it should be clear that that analysis is post hoc.
Try also #
Descriptive statistics
Exploratory analyses
Inferential statistics
Null Hypothesis Significance Tests (NHST)
p-value
Pre-registration
Protocol paper
Reproducibility
Research fraud
Chapters #
Touched on or pertinent to Chapters 5 to 8.
Online resources #
None yet!
Dates #
First created 30.viii.23.
