This is part of the RCSC (Reliable and Clinically Significant Change) paradigm which itself is a way of categorising change in individuals (as opposed to change aggregated across groups of individuals in the traditional statistical approaches to measuring change. The RCI is a simple criterion: if the person’s score changes more than this value (the RCI value) in a health direction then their change is categorised as “reliable improvement”, if it is in the other direction it is categorised as “reliable deterioration” and if the change, whatever the direction, is less than the RCI value then the change is categorised as “no reliable change”.
Details #
The basic idea behind the RCI is simple: our measures are always imperfect, i.e. unreliable, which is the same as saying that they can be seen as the sum of a “true score” that we would see had we perfect measures and “error” which is random noise that contaminates our measurement. The more reliable the measure, the less the random noise. The RCI uses this very basic model of what is going on in measuring things (known as “Classical Test Theory”, CTT), with the assumption that in the wide population of people we might measure that true scores and noise are distributed according to the Gaussian (“Normal”) distribution. That gives a formula for the confidence interval for measurement of change between two occasions and the RCI is usually the the change such that if there were no true score change and only noise contributing to changes in scores, 95% of change would be classified as “no reliable change” and 2.5% incorrectly classified as “reliable improvement” and 2.5% again incorrectly classified as “reliable deterioration”.
The use of 95% for the criterion perhaps a bit seductively aligns the model with the typical “p < .05” criterion of “statistically significant change” in aggregate change evaluation, e.g. in the paired t-test or the non-parametric Wilcoxon test. However the models behind the p value in those “Null Hypothesis Significance Testing” method are actually rather different and the RCI, in CTT, is more a heuristic to tell you the criterion that would define that 95%, 2.5%, 2.5% expectation if no true change were happening in your dataset. It’s not really a sample/population model. That’s generally ignored!
The RCI is very easy to compute from the baseline standard deviation (SD) of the scores and the presumed reliability of the measure (details in https://www.psyctc.org/psyctc/root/stats/rcsc/ and the shiny app at https://shiny.psyctc.org/apps/RCI1/ makes it easy to get the RCI given those two numbers). Although most use of the RCI uses a 95% interval it is perfectly possible to compute the RCI for any other interval and for short measures, whose internal reliability is low, a 90% interval is often used.
How the RCI is used #
The RCI is a categorisation of individuals’ changes as the original publications intended. It is often used in that way but also, perhaps rather depressingly and reductionistically turned back into an aggregate summarising method as, typically in the UK IAPT project to compare services on their proportions of clients “achieving reliable change” and as “reliably recovered” (the latter involves the CSC, the other component of the RCSC paradigm).
History and citing the RCI #
The method and RCI idea was first published in Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336–352. In fact, as pointed out in Christensen, L., & Mendoza, J. L. (1986). A method of assessing change in a single subject: An alteration of the RC index. Behavior Therapy, 17, 305–308 the first paper got the formula for the RCI wrong, something the original authors immediately acknowledged in Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1986). Towards a standard definition of clinically significant change. Behavior Therapy, 17, 308–311. This all makes giving a canonical first citation for the RCI a bit messy as you can see.
A later paper, Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19, is often cited to avoid that historical complexity and, with my co-authors, I am told that our paper: Evans, C., Margison, F., & Barkham, M. (1998). The contribution of reliable and clinically significant change methods to evidence-based mental health. Evidence Based Mental Health, 1, 70–72. https://doi.org/10.1136/ebmh.1.3.70 is another readable introduction to the RCSC.
Try also #
Clinically Significant Change (CSC)
Confidence Intervals (CI)
Jacobson plot
Null Hypothesis Significance Testing (NHST)
Paradigm
Reliable and Clinically Significant Change (RCSC) (PSYCTC.org page)
Chapters #
Discussed in Chapter 5.
Online resources #
I have more detail about the RCI in https://www.psyctc.org/psyctc/root/stats/rcsc/.
Shiny app to compute the RCI for you given your baseline SD and reliability (and the interval you want).
Slightly more sophisticated shiny app to compute the RCI for you similarly but, given the dataset size, also gives you a range for the RCI taking into account the confidence interval of the SD used.
Dates #
First created 21.ix.23.
