On the Global Summit: do critiques of evidence hierarchies favour chiropractic?

Learning of a new citation to a previously published work is both gratifying and trepidatious. In their paper Extrapolating Beyond the Data in a Systematic Review of Spinal Manipulation for Nonmusculoskeletal Disorders: a fall from the summit, a team of four researchers quoted at length from my thesis, Hierarchies of Evidence in Evidence-Based Medicine. The quotation is:

The conclusion is that hierarchies in general embed untenable philosophical assumptions: principally that information about average treatment effects backed by high-quality evidence can justify strong recommendations, and that the impact of evidence from individual studies can and should be appraised in isolation. Hierarchies are a poor basis for the application of evidence in clinical practice.

Blunt, 2015, as cited in Goertz et al. 2021, p.3

The context for the piece is that the four co-authors were members of The Global Summit, an international collaboration involving 50 researchers, “to conduct a systematic review of the literature to determine the efficacy and effectiveness of SMT for the primary, secondary and tertiary prevention of non-musculoskeletal disorders.” (Côté et al. 2021) SMT is Spinal Manipulation Therapy, which is a technique (or rather a looser amalgam of techniques) employed by some chiropractors, osteopaths and other manual therapists. The Global Summit resulted in a systematic review which makes for very uncomfortable reading for SMT practitioners. They find that there is no evidence for the practice in “non-musculoskeletal disorders including infantile colic, childhood asthma, hypertension, primary dysmenorrhea, and migraine.” (ibid.) They go further, striking at the idea that SMT has physiological effects and calling for scrutiny and possibly defunding of the practice:

This finding challenges the validity of the theory that treating spinal dysfunctions with SMT has a physiological effect on organs and their function. Governments, payers, regulators, educators, and clinicians should consider this evidence when developing policies about the use and reimbursement of SMT for non-musculoskeletal disorders.

Côté et al. 2021, p.2

However, not all of the 50 researchers who attended the Global Summit were convinced. The breakaway foursome have methodological, philosophical and practical objections and differences from the majority group. They declined to be listed as co-authors of the Global Summit report. Extrapolating Beyond the Data is their response to the publication, listing their objections and laying out an alternative interpretation of the data and the wider literature on SMT. They invoke my argument against evidence hierarchies as part of a wider argument that it was misguided of the Global Summit group to focus only on RCTs in constructing their review, and that they have missed vital evidence in sidelining non-randomised research. They would have seen the group consider: “health-services research, practice-based studies, comparative-effectiveness studies, qualitative and observational studies, and expert panels.” (Goertz et al., 2021, p.3)

At conferences, I frequently encounter the objection that my pluralist stance and rejection of evidence hierarchies opens the door to charlatanism. Opponents of complementary and alternative medicine often feel that evidence hierarchies are a bulwark against attempts to use a melange of weak studies to substantiate effectiveness claims for therapies which have no scientific basis.

However, I have argued to the opposite claim, in both my thesis (2015) and in The Positivity Machine (2019). I claim that it is evidence hierarchies, and in particular the “Grades of Recommendation” taxonomies which extrapolate a recommendation for or against treatment from a level or ranking of evidence quality, which ease the way for alternative therapies to claim scientific backing, particularly where the existing literature of clinical trials is permeated by publication bias, methodological manipulation and selective reporting. By putting all scientific weight on studies of a particular sort, which are open to manipulation and can permit the careful cultivation of an evidence base skewed to support a claim of effectiveness, we centralise our risk of being misled. A philosophy of medical evidence which emphasises diversity, plurality and integration of evidence sources is more robust against manipulation and misleading claims.

However, it is clear that Goertz et al., at the least, feel that my argument supports a defence of SMT for non-musculoskeletal disorders. Although I am clearly on record as rejecting any claim for the efficacy or effectiveness of chiropractic (see Blunt, 2019), the arguments made by Goertz et al. require careful and fair consideration. Beginning, therefore, without prejudice of the expected outcome, I will set out here to review both The Global Summit’s systematic review, and the arguments brought against it by Goertz et al. with the intention of establishing whether they are correct that the Global Summit is flawed, whether my own arguments support that conclusion, and whether Goertz et al.’s suggested approach accords with my own, and what the outcome of following their approach and my own approach to the evidence base for SMT in non-musculoskeletal disorders would be.

This project will certainly take some time to develop. I write this only five days after the pre-publication of Extrapolating Beyond the Data online. When this review is completed, I will update this article to reflect my findings.

Bibliography: