These are very simple ideas coming out of the randomised controlled trial (RCT) world and are mainly used as a way of expressing the efficacy of one intervention relative to another based the comparative findings from those two arms of an RCT (or the aggregated findings across a number of such trials, i.e. from a meta-analysis of more than one trial). The restriction to the RCT world means you don’t see them a lot in the psychosocial intervention and psychological therapy worlds though they are sometimes computed for therapy versus medication trials or for trials of therapies versus treatment as usual. The usual argument, probably not entirely wrong, is that the NNTs for therapy interventions for common mental health problems are often as good as or better than those for pharmacological interventions. Let’s start with the NNT (it’s much more often reported than the NNH).
Details #
So what is an NNT? It’s exactly what it says if you complete the sentence it abbreviates. It’s the estimated number of clients who would have to receive the better intervention to result in one “recovery” (however that is defined). It comes out of the pharmacological therapy world and is easier to explain there. Suppose you have a well defined condition, let’s say something nasty, let’s say a bacterial meningitis and you want to know if a new antibiotic is better than the standard treatment. A good outcome is defined (and publicly documented before doing the study of course), here it might be complete recovery without neurological problems. Say the standard treatment results in 800 in 1,000 sufferers having that outcome and the new antibiotic shows 900 in 1,000 having that outcome. That difference is statistically significant (“X-squared = 38.435, df = 1, p-value = 5.66e-10”, i.e. a difference as big as that or bigger, given that total dataset size of 2,000, would happen less often than once in 1,766,784,452 times: way less likely than the usual “p < .05” criterion of statistical significance!) OK, but how much does it matter if perhaps the standard treatment is very cheap and the new antibiotic will be very expensive and healthcare resources are limited?
The NNT is 10: on the basis of the trial findings because nine of ten recover completely with the new antibiotic and only eight in ten with the standard treatment: difference 1 in 10, i.e. NNT to get one better outcome is 10.
What is an NNH: the same logic but applied to harmful outcomes, say the new treatment turns out to cause occasional but nasty effects, say it leaves people who took it deaf. That’s not a completely mad scenario: one of the problems with the antibiotic Gentamicin when it was introduced was that it took a while to recognise that it did cause hearng damage, sometimes severe, including in children born after their mother had been treated with it (if I remember correctly). Let’s say that none of the 1,000 who had the standard treatment were left deaf but 10 in the 1,000 who had the new antibiotic were left deaf. That’s a statistically significant difference again, less so but clearly: X-squared = 8.1407, df = 1, p-value = 0.004328, i.e. a one in 231 chance this or higher diference would have occurred by chance, now the NNH is 100 (1 in 100 left deaf in the new antibiotic group and none in the standard group: difference one in the 100 so NNH is 100.
All well and good for pharmacotherapy and other situations in which you may have very clear good or bad endpoints you can define and where you can allocate randomly and “blindly” participants in trials (with their informed consent of course) to interventions. All of that applies so rarely for psychosocial interventions (never?) that it’s probably not a very useful idea for us.
Try also #
Blinding
Double blind
Randomised controlled trials (RCTs)
Null hypothesis significance testing (NHST) paradigm
Inferential testing, “tests”
Chapters #
Not mentioned in the OMbook as computing them outside the RCT framework seems to me unlikely to have much robustness.
Online resources #
Not from me, not least because I don’t see the RCT framework as terrible useful in the psychosocial intervention and therapy worlds.
Dates #
First created 24.vii.24.
