The Authority of Evidence-Based Medicine

In the early part of the 20th century, the philosopher Ludwig Wittgenstein sought to demonstrate that metaphysical claims are meaningless. Statements which couldn’t be proven true in some way—through logic or evidence—weren’t even false, they had no meaning at all. But he ran up against a problematic irony. His book, Tractatus Logico-Philosophicus, which argued for this idea (amongst others) was constructed almost entirely of metaphysical claims—not least of which the iconic final line: “Whereof we cannot speak, thereof we must remain silent”.[1]

His solution, sometimes dubbed “Wittgenstein’s ladder”, was to accept this strange irony and agree that his book was a collection of nonsensical statements. He climbs a ladder to the roof, and when he reaches the roof, discovers that he can throw the ladder away. Yet, he is now on the roof. The book undermines the meaningfulness of its own statements. Nevertheless, the conclusions remain. He has shown his reader that metaphysics is meaningless through a string of meaningless metaphysical statements. Once the reader understands, they will be able to discard the very statements that produced that understanding.

Whatever you think of this quixotic line of reasoning, the proponents of Evidence-Based Medicine faced a similar dilemma. One of the fathers of EBM, David Sackett, responded with a similarly dramatic gambit. Evidence-Based Medicine denied authority as a basis for medicine. As the EBM movement developed, its pioneers rejected the notion that experienced clinicians and researchers held privileged positions—that they knew more or had a better kind of knowledge than less experienced colleagues and students, and that the best way for less knowledgeable juniors to learn was at their seniors’ intellectual fount.

According to EBM, the wisdom of the expert was an inadequate basis for medical practice. Experts’ “knowledge” may be unsubstantiated, based on unsystematic experiences and biased recollections. It may run wholly contrary to the newest and most rigorously tested scientific evidence. The experts may have prioritized their own expertise and experiences over hard data and results from clinical trials. They may have been ignorant of those trial results. No wonder, given that tens of thousands of new trial results are reported each year across hundreds of journals. Even the most attentive and dedicated individual could never keep abreast of the massive volume of information published annually. Experts’ ideas of what worked and what didn’t might be based not on epidemiological data and positive trial outcomes, but on theoretical mechanisms, which made them believe that a treatment should work. But, as proponents of EBM have taken pains to demonstrate, in healthcare should work very often fails to translate into does work.

EBM was initially an anti-authoritarian concern, rejecting traditional medical authorities in favour of data and the results of research. Any doctor (or even patient) no matter how junior could challenge any authority, no matter how venerable, to produce the research which underpinned any statement. ‘My many years of experience’ was no longer an adequate basis for a claim. Burgeoning young doctors should be taught to access and assess the medical literature, and made to interrogate the data to discover which treatments worked. They should not learn second-hand, rapidly obsolete information from authorities in lecture halls. They should learn to distrust anything that wasn’t explicitly grounded in research findings.

But here Evidence-Based Medicine has climbed a ladder, only to realize the ladder itself is infirm. EBM has made any claim of any authority subject to the demand for evidence—including their own. When the EBM proponents made a claim about the inadequacy of authority, about the primacy of the Randomised Controlled Trial (RCT), or about the way in which evidence should be assessed, their own philosophy demands the question: “Where is your evidence?” As we shall see, this question has been very difficult to answer, and has led to exceptionalism and contestable double-standards. Ultimately, Evidence-Based Medicine as an ideology fails its own test.

The philosophy of EBM itself is not something based on evidence, but on authority. We find ourselves in a Wittgensteinian rooftop stalemate, as the authorities of EBM declaim the invalidity of authority, only to realise that their own statements were similarly baseless. But now we have come to see that authority should not be accepted, we can reject it—in them as in others.

Who were these EBM authorities? David Sackett was perhaps the most prominent and influential EBM proponent. He founded the McMaster University department of clinical epidemiology and the Centre for Evidence-Based Medicine in Oxford. These two institutions, more than any others, carved out the ideology of EBM. Sackett’s student at McMaster, Gordon Guyatt, coined the term “Evidence-Based Medicine” in 1991, going on to become perhaps the most significant figure in the movement after Sackett.[2] Sackett understood his Wittgensteinian conundrum. Having established a formidable reputation and a position of great authority and influence, he could decry reliance on experts from the rooftops. As he put it in 2015, “I had long held the view that ‘experts’ inevitably became detrimental to the fields of their expertise”.[3]

Once he’d helped to establish Evidence-Based Medicine, though, he recognized that his authority was invalidated too. Sackett took this to heart, announcing in 1999 that he was officially retiring. His authority gave him undue influence, he believed, and he would step back from the frontiers of EBM. His noble seppuku was not an entire withdrawal from the medical literature, as he continued to publish papers of advice to clinicians and trialists, albeit mostly outside of the fundamental issues of EBM. In some ways, Sackett himself tried to embody the ladder for the EBM movement—setting himself up to convince the medical community of the EBM philosophy, before stepping back and acknowledging that EBM’s philosophy undermined his very authority. Sackett’s authority was the ladder to reach EBM’s rooftops, only to be kicked away.

Throughout his career, Sackett challenged conventional medical wisdom and demanded to see the evidence. If there was no evidence to his satisfaction, he went out and gathered it. This pursuit led him to great breakthroughs—demonstrating the benefits of aspirin for preventing stroke[4], and of surgical interventions to clear out plaque clogging the carotid arteries[5] (‘carotid endarterectomy’, of which much more in Chapter 3). It also led him to debunk the effectiveness of several commonly used treatments, which solidified his reputation for skepticism towards conventional wisdom.

He also became concerned that, as he and others accumulated more and more evidence, the doctors who practiced outside of academic communities would be unable or unwilling to access it. Before the internet made research somewhat more accessible, doctors would never be able to keep themselves up to date. Doctors who had trained under the traditional model of received wisdom would start from a basis riddled with claims which were unsupported by evidence, and would not benefit from the new evidence coming in. If Sackett was right, then, counterintuitively, doctors would get slowly worse (in terms of the outcomes for their patients) the longer they had been out of medical school, not better as they gained more experience.

In 1984, Sackett and his colleagues studied this radical hypothesis. They tested whether more experienced clinicians had a better knowledge than more junior doctors in a particular field – the treatment of hypertension (high blood pressure).[6] Their study supported his surprising conclusion: the longer since a doctor left medical school, the more outdated and inaccurate their understanding of the state of the art, and the worse their patients would fare. This even held in surgery, where practical skills and expertise, of the kind you’d expect to develop through long experience, were a central concern. In 2000, a study of outcomes in carotid endarterectomy, one of the surgeries Sackett helped to popularize through his research, showed that the longer it was since a surgeon had got their license, the higher the mortality rate in their patients.[7] In 2005, a meta-analysis reviewed 60 studies on this question, and found that half showed a negative correlation between how long a doctor had practiced, and their knowledge and the outcomes for their patients.[8] In many cases, it seemed, the more experienced the doctor, the worse their quality of care.

Sackett took these concerns personally, and again took his own medicine.[9] Some 20 years after first training as a practitioner, at the age of 49, Sackett decided to redo his residency despite being a well-respected professor. He believed he was not—or no longer—good enough to practice without retraining. He also retired from clinical medicine in 1999, stating: “I’d long been convinced that the judgement of acute-care, bedside internists began to deteriorate at age 65, and had long since resolved not to join them.”[10] When he passed away in May 2015, at the age of 80, he was still an important and influential contributor to the literature on randomized trials, but had kept himself aside from the debates surrounding the movement he helped to found. Sackett will rightly be remembered as one of the great figures of modern medicine.

By withdrawing from his position of influence because of the power of his authority, Sackett acted as a channel for EBM’s Wittgensteinian gambit. Yes, the claims of EBM were based on authority. But those authorities could be discarded once the key principles were understood. EBM’s process of gathering and appraising evidence could then be applied. Everything would be on the table for the challenge of evidence.

But this pretty portrait is not truly tenable in the case of Evidence-Based Medicine. If EBM’s ladder gambit left behind nothing but the notion that authority was an insufficient basis for medical practice, then perhaps the Wittgensteinian parallel could hold up. But the EBM movement pushed much more through on the basis of their own authority and status than just this rebellious anti-authoritarianism. The most important and influential component of the EBM movement’s final philosophy was the Hierarchy of Evidence. This hierarchy attempted to put all the various sources of evidence which the practicing clinician might draw upon into a ranking.

At the top were sources such as Randomised Controlled Trials (RCTs), and systematic reviews and meta-analyses of RCTs. RCTs as understood by the EBM movement are fundamentally experimental studies. They involve taking a group of patients and experimenting upon them by manipulating which treatments they receive. Systematic reviews bring together and summarise and analyse the results of a set of RCTs. Meta-analysis refers to a group of statistical techniques which can be applied to sets of RCT results within a systematic review. Both of these sources, many hierarchy authors assumed, could be regarded as inheriting the strength of the RCTs they were based upon.

Below the RCTs were observational studies. This broad class of research methods does not involve changing which treatments patients get. Rather, observers note which treatments patients got, and what happened to them, and investigate that data set. Some hierarchies go yet further, and place a range of other kinds of evidence below the observational studies, at an even lower rank. These sources of evidence include the range of expert opinion, mechanistic reasoning and biomedical rationale which the EBM movement had sought to reject. But the core assumption of hierarchies was the primacy of the RCT. Randomised trials provide the strongest, the best, the highest quality evidence—and everything else is second-rate information. The best situation, then, for an “evidence-based” practitioner, is to base all treatment decisions on data from RCTs.

This hierarchy of evidence – or more accurately, these hierarchies, as there have been a great many published under EBM’s auspices – does not follow on straightforwardly from the revolutionary rejection of authority that characterized early Evidence-Based Medicine. It was something new, something else. Spurning the pretentions to knowledge of the expert was one thing, but requiring that medical students learn (and existing practitioners re-learn) their craft by actively interrogating evidence themselves was another. If it was to gain traction, EBM would need to replace the experts with something more than just ‘the evidence’. Not all evidence is equal. But doctors and medical students don’t arrive at the clinic or the lecture hall already evidence aficionados. EBM’s first response was to try to train practitioners in evidence appraisal. To do that, and to do it at a massive scale, they needed core principles which they could disseminate easily and consistently, and which would be easy enough for their audience of busy doctors to understand and implement.

Many of the principles which EBMers espoused, and the backbone of the hierarchies which proliferated and permeated throughout healthcare, derived from the area of expertise of Sackett and his McMaster colleagues: clinical epidemiology.[11] It’s in clinical epidemiology that we find the first hierarchies of evidence [12], and find the germ of the evidence appraisal approaches which the EBM movement spread to a generation of practitioners [13]. Simple visual representations of core principles were always an efficient way to communicate complex ideas. The hierarchy was first used in recognizable form in 1979 by the Canadian Task Force on the Periodic Health Examination, of which Sackett was a prominent member. Most Task Force members were primarily epidemiologists, Sackett included. They drew on principles from epidemiology, which included emphasizing RCTs over observational studies as the pinnacle of study design.

But these weren’t experimentally validated, either as effective teaching techniques to train doctors in evidence appraisal, or as modes of operation to improve patient outcomes. They were methodological and practical doctrines which formed part of the clinical epidemiological furniture. They embedded assumptions about the kind of evidence which was important to clinical epidemiology, and the kinds of biases which were the greatest threat to causal inference in epidemiology. Much of this theoretical machinery was imported essentially unchanged into the clinical teaching philosophy, and then the clinical practice philosophy, of EBM. Much of it remains – tweaked, refined, extended and intensified – in modern EBM doctrine. But the authority upon which the EBM movement built its new approach to medicine is one of the authority, expertise and approach of its founding authorities and their discipline, not the authority of evidence. In time, ‘Whereof we cannot speak…’ becomes ‘Whereof we cannot speak on the basis of RCT evidence…’.

 

[1] Wittgenstein, Ludwig. Tractatus Logico-Philosophicus. Translated by Frank P Ramsey and Charles Kay Odgen. London: Routledge & Kegan Paul, 1922.

[2] Guyatt, G.H. ‘Evidence-Based Medicine’. ACP Journal Club: Supplement 2 to Annals of Internal Medicine 114 (1991): A-16.

[3] Sackett, David L. David L Sackett: Interview in 2014 and 2015. Edited by R Brian Haynes. Hamilton, Ontario: McMaster University, 2015.

[4] Barnett, H. J., J. W. McDonald, and D. L. Sackett. ‘Aspirin–Effective in Males Threatened with Stroke.’ Stroke 9, no. 4 (1 July 1978): 295–98. https://doi.org/10.1161/01.STR.9.4.295.

[5] Barnett, H. J., D. W. Taylor, M. Eliasziw, A. J. Fox, G. G. Ferguson, R. B. Haynes, R. N. Rankin, et al. ‘Benefit of Carotid Endarterectomy in Patients with Symptomatic Moderate or Severe Stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators’. N Engl J Med 339, no. 20 (12 November 1998): 1415–25.

[6] Evans, C. E., R. B. Haynes, J. R. Gilbert, D. W. Taylor, D. L. Sackett, and M. Johnston. ‘Educational Package on Hypertension for Primary Care Physicians.’ Canadian Medical Association Journal 130, no. 6 (15 March 1984): 719–22.

[7] O’Neill, Liam, Douglas J. Lanska, and Arthur Hartz. ‘Surgeon Characteristics Associated with Mortality and Morbidity Following Carotid Endarterectomy’. Neurology 55, no. 6 (26 September 2000): 773–81.

[8] Choudhry, Niteesh K., Robert H. Fletcher, and Stephen B. Soumerai. ‘Systematic Review: The Relationship between Clinical Experience and Quality of Health Care’. Annals of Internal Medicine 142, no. 4 (15 February 2005): 260.

[9] Sackett, ‘The sins of expertness and a proposal for redemption’.

[10] Sackett, David L. David L Sackett: Interview in 2014 and 2015. Edited by R Brian Haynes. Hamilton, Ontario: McMaster University, 2015, 57.

[11] See also Daly, J. (2005) Evidence-based medicine and the search for a science of clinical care. Berkeley, CA ; London: University of California Press.

[12] Canadian Task Force on the Periodic Health Examination. (1979) “The Periodic Health Examination”. CMAJ, 121, 1193-1252.

[13] e.g. Sackett, D.L., Haynes, R.B. & Tugwell, P. (1985) Clinical epidemiology : a basic science for clinical medicine. Boston: Little, Brown.; Sackett, D.L. (1981) “How to read clinical journals: IV. To determine etiology or causation”. CMAJ, 124, 985-990.; Sackett, D.L. (1986) “Rules of evidence and clinical recommendations on the use of antithrombotic agents”. Chest, 89(2 Suppl), 2S-3S.