Are hierarchies history?

Throughout my career as a philosopher of medicine, I have taken a special interest in hierarchies of evidence as used in Evidence-Based Medicine (EBM) and beyond. My critique of evidence hierarchies has adapted and developed alongside recent movements towards a less rigid hierarchalism in EBM, and with the development of approaches such as GRADE which rate and rank evidence quality in more sophisticated ways than the simple lists of old.

Nowadays, when I criticise hierarchies and their role in contemporary medicine and clinical research, I sometimes face the response that hierarchies are history. This argument usually goes that although hierarchies were used for a while in EBM, their heyday is behind us. We philosophers can rest assured that no simple structure such as a hierarchy will be unduly influencing the approach to evidence (and more importantly of course, the kinds of questions which even get asked) in medicine. Sometimes, the argument is that GRADE has superseded hierarchies. This argument is a little implausible, given the slew of widely-cited hierarchies which postdate GRADE. But undoubtedly GRADE’s influence curbs some of the more hard-line hierarchical claims and shows an implicit rejection of the too-strong claims for the universal superiority of randomised trials to observational evidence.

Nonetheless, when I discuss evidence hierarchies with practitioners – as opposed to philosophers or academic proponents of EBM – I find that “the” evidence hierarchy is perhaps better known today than the principles of Evidence-Based Medicine are. If the hierarchy is on its way out in EBM in theory, it could still be enjoying the spotlight in practice.

We could try to pursue a strategy similar to that used by Brian Wetherstone in his A history of philosophy journals to analyse patterns in the use of EBM and hierarchical keywords over time in medical journals. Or we could look at frequencies of citations of specific hierarchies to see how widely used they are in research publications. While such projects would no doubt be illuminating, I fear it would miss the crux of the issue here. While it might help us to assess whether claimants are right that hierarchies are on their way out in the EBM literature, this is hardly likely to reflect the level of interest of practitioners and practitioner-researchers in hierarchies. To see whether my experience of increasing interest in hierarchies is borne out in data, we need a data source which reflects broader interest than the developments of journal keywords.

As a first approach for this, I used Google Trends, Google’s open search traffic analytics platform, to analyse the popularity of search terms since the platform’s inception in 2004. Google Trends has the advantage that, unlike the academic literature, it should reflect interest in a topic beyond the academy – students, practitioners, evidence readers and analysts, policy-makers and academics are all likely to search for terms which interest them, particularly when they encounter them for the first time.

For a first pass, let’s compare the search traffic for evidence-based medicine as opposed to hierarchy of evidence. The blue line depicts search interest in evidence-based medicine, and the red hierarchies of evidence:

What we see is that EBM is unsurprisingly far out ahead of hierarchies of evidence in terms of search activity in the mid-2000s. But over time, searches for EBM decline, presumably as knowledge about it entrenches and there is less need to discover, or interest in, the topic. At the same time, interest in hierarchies is fairly consistent, and indeed begins to overtake searches for EBM in the last few years. This is true even though the terminology around hierarchies has fragmented (we could now be searching for levels of evidence, GRADE, an evidence pyramid, grades of recommendation, and so on) while the terminology around EBM has, I would submit, remained a little more consistent.

We can also explore the related search terms to see what else people who search for a hierarchy of evidence are looking for. In the last five years, “levels of evidence” is the most associated search term, and “NHMRC” quickly comes alongside that, too. This is suggestive that the NHMRC hierarchy has been about the most popular worldwide in recent years amongst this broad audience of Googlers. By comparison, the likes of GRADE is nowhere to be seen in related terms – this suggests either that GRADE has not overtaken traditional hierarchies as systematically as some of its proponents have claimed, or that GRADE is not being considered as a hierarchy of evidence or as part of that trajectory within EBM. There is most likely some combination of both going on. Certainly, there is no support here for the hope that those who search for hierarchies of evidence end up redirected onto GRADE. After NHMRC, the next most popular specific hierarchy searched for on Google is the Oxford Centre for Evidence-Based Medicine hierarchy, by Howick et al. Next comes a range of search terms for evidence pyramids.

Finally, we can use Google Trends’ “Rising” search terms feature to look at change over time and see if we can identify hierarchies which are gaining in popularity and influence recently. The biggest trend towards a specific hierarchy is the rising interest in the JBI Levels of Evidence from the Joanna Briggs Institute. This hierarchy is a little more detailed than many previous iterations. In the last 5 years, interest in the JBI levels has increased an estimated 110% amongst those searching for hierarchies of evidence. Another rising interest is in ASHA’s levels of evidence. This is largely just an adoption of the SIGN hierarchy, but indicates how the influence of hierarchies is spreading around different fields, with the American Speech-Language-Hearing Association now adopting a prominent positioning for a classical hierarchy. Evidence pyramids are also a rising interest, unsurprisingly.

Hierarchies of evidence have a central role in the history of Evidence-Based Medicine, that much is clear. But no matter how often we hear that simple list-style hierarchies are on the way out, their influence remains quite persistent. Even if one day we no longer need to distinguish between “Evidence-Based Medicine” and “Medicine”, it is likely that the most important artefact of the EBM movement will be its hierarchies. Even if EBM’s leading theoreticians find themselves post-hierarchy, the EBM literature, community, and those influenced by EBM beyond healthcare, are still under its influence. If proponents of EBM are truly trying to make hierarchies history, so far they are not succeeding.