The Pyramid Schema: The Origins and Impact of Evidence Pyramids

All things are defined by names. Change the name, and you change the thing.

Sir Terry Pratchett, Pyramids

There is a lesser-known riddle of the sphinx, lost to arcana, which goes: “When is a pyramid not a pyramid?” The answer: when it’s just a table in the guise of a pyramid.

In the late 1990s, evidence pyramids began to arise point-first from the literature of Evidence-Based Medicine (EBM), a movement within clinical medicine which has the goal of placing medical practice on sounder evidential footings. By the mid-2000s, evidence pyramids had bloomed across the EBM literature and were widespread across medical publications, textbooks and websites. Pyramids came to occupy as influential a position as the simple tables and lists which had preceded them, and to rival the prominence of the complex calculations of the more sophisticated evidence appraisal systems which arose alongside them. Evidence pyramids have come to define the visual imagination of EBM, to the point that a Google Images search in 2022 for “Evidence-Based Medicine” primarily returns evidence pyramids. A search for “Hierarchy of Evidence” is even more predominated.

• 

• 

Why are the pyramids here? Where did they come from? Who made them and what esoteric ancient wisdom do they represent? The origins of the evidence pyramids are masked by uncertainty. Few sources have attempted to identify the original evidence pyramid, and as a result, one of the most important artefacts of EBM comes from an unknown source. It is Why are the pyramids here? Where did they come from? Who made them and what esoteric ancient wisdom do they represent? The origins of the evidence pyramids have, until now, been masked by uncertainty. Few sources have attempted to identify the original evidence pyramid, and as a result, one of the most important artefacts of EBM has come from an apparently unknown source. It is this lacuna that this paper seeks to fill. First, the place of evidence pyramids within the broader trajectory of evidence appraisal schemas within EBM is mapped out. Second, I trace the origins of evidence pyramids to their creators, course writer Martin Mellish working with subject matter expert Betty Wagoner at the Medical Research Library of Brooklyn, and establish the intended interpretation of the design. Third, I consider what, if anything, a pyramidal presentation of an evidence hierarchy offers compared to tabular hierarchies, beyond an aesthetic choice, drawing on the accounts of Mellish and subsequent pyramid innovators Tomlin & Borgetto (2011). Finally, I present an argument that the pyramidal turn in evidence appraisal systems has, despite the more nuanced intentions of the original, been a retrograde step for the EBM movement.

---

1. Unearthing the Pyramids

Since the mid-1990s, the Evidence-Based Medicine movement has constructed and promulgated evidence hierarchies, which usually attempt to rank the quality or strength of the evidence a study provides, according to the methodology used in the study (see Blunt, 2015). Traditionally, Randomised Controlled Trials (RCTs) rank at or close to the top, sometimes alongside or beneath systematic reviews and meta-analyses of RCTs. Forms of observational study such as cohort and case-control studies occupy the rungs beneath. Languishing at the foot of the table are case reports, and sometimes other forms of evidence such as expert opinion, mechanistic reasoning and biological rationale.

These hierarchies have their own complex and contested history. They considerably pre-date the unveiling of the EBM movement, which came to prominence with the publication of Evidence-based medicine: a new approach to teaching the practice of medicine in the Journal of the American Medical Association in 1992 (EBM Working Group, 1992). Some sources (e.g. Rychetnik et al. 2002; Glasziou, Vandenbrouke & Chalmers, 2004) cite Campbell and Stanley’s highly influential Experimental and Quasi-Experimental Designs for Research (1963) as the original hierarchy. However, as I have discussed elsewhere (see Blunt 2015; Blunt 2019), Campbell & Stanley do not offer a methodology-driven ranking and are explicit about not interpreting their work in this way. While offering inspiration for the idea, it would not be accurate to term Campbell & Stanley’s work as offering an evidence hierarchy.

Elsewhere, Joseph Vere and Barry Gibson (Vere 2018; Vere & Gibson 2020) make a strong case for considering Archie Cochrane’s classic Effectiveness and efficiency (Cochrane 1972) as the progenitor of evidence hierarchies. While Cochrane again offers some semblance of a ranking, placing RCTs ahead of observational studies, which in turn outrank other forms of evidence, I contend that Cochrane’s piece primarily defends the claim that only RCTs offer high-quality evidence, and that other forms of evidence lack merit. In this form, it could be considered a proto-hierarchy with two levels (see Blunt 2020). But Cochrane is an inspirational figure for many EBM proponents (see Daly 2005), and his Effectiveness and efficiency is, with little doubt, the philosophical and ideological underpinning of the formal, structured hierarchies that followed. Whether we consider it the inspiration for the first hierarchies, or the first hierarchy in itself, is moot.

The first hierarchy recognisable as such to a modern EBM proponent was proposed by the Canadian Task Force on the Periodic Health Examination (CTF) in 1979. This hierarchy has most of the features replicated across many subsequent rankings. It proposes three tiers, with tier II itself subdivided into tier II-1 and II-2, perhaps drawing inspiration from the British university classification schema. Tier I is reserved for evidence from RCTs. Tiers II-1 and II-2 are various forms of observational study: cohort, case-control and “comparisons between times or places” (CTF 1979, p.1195). Finally, Tier III contains opinion, clinical experience, descriptive studies and expert committee verdicts. Most later hierarchies are riffs or expansions of this core structure.

Hierarchies preceded EBM, yet EBM did not immediately adopt hierarchies. Key early texts in the development of the distinctive EBM philosophical foundations did not include hierarchies or make any explicit reference back to the pre-EBM hierarchies (see e.g. Sackett & Rosenberg 1995; Davidoff et al. 1995; Sackett et al. 1997). This is despite the most prominent foundational texts sharing a co-author (David Sackett) with almost all of the pre-90s early hierarchies (Canadian Task Force 1979; Anonymous 1981; Sackett 1986; Sackett 1989). Hierarchies began to cross over into the EBM mainstream half a decade later, in the mid-to-late 1990s, primarily through the Users’ Guides to the Medical Literature series (e.g. Guyatt et al. 1995; McAlister et al. 1999; Guyatt et al. 2000). This was codified into a book in 2001 by Gordon Guyatt and colleagues, in which the opening chapter, entitled “The Philosophy of Evidence-Based Medicine”, delineated an evidence hierarchy as one of the two fundamental philosophical pillars of EBM (Guyatt et al. 2001). Other prominent early EBM or EBM-adjacent sources which had considerable influence at this time included Trisha Greenhalgh’s critical (but nonetheless widely adopted and copied) presentation of an evidence hierarchy in How to read a paper (1997), the Australian NHMRC’s adoption of a hierarchy in their clinical practice guideline development process (ANHMRC 1999), and the 1998 Levels of Evidence published by Oxford’s Centre for Evidence-Based Medicine (CEBM) (Phillips et al. 1998). The heyday of the signature evidence hierarchies was the late 1990s. Its codification as a foundational element of EBM thought came swiftly after, in the early 2000s.

While the Evidence Based Medicine movement’s hierarchalists primarily spent the late 1990s tabulating evidence according to methodology, the 2000s represented a more experimental period, in which proponents of evidence rating, ranking and grading systems explored some creative methods for portraying their proposals. At this time, early evidence pyramids began to rise to prominence. In contrast to the stark tables and lists which preceded them, early evidence pyramids were often colourful, vivid illustrations. Take, for example, the multicoloured pyramid published online on SUNY Downstate Medical Center’s EBM Tutorial page (see Fig. 2, below).

The rainbow colour-scheme and chunky MS Paint arrows make an immediate impression, and evoke an antithetical imagery to, for instance, the staid and academic style of previous evidence hierarchies, albeit conveying much the same information. For instance, consider another presentation of an evidence hierarchy from the same period, enshrined as one of the two philosophical precepts of EBM in the 2001 edition of the Users’ Guides to the Medical Literature, authored by EBM originator Gordon Guyatt and colleagues (see Fig. 3, below).

There are some notable differences here. The inclusion of N-of-1 randomized trials in Guyatt et al. (2001) is markedly distinct from the vast majority of hierarchies and never caught on within the broader literature. Meanwhile, the inclusion of “Animal research” and “In vitro (‘test tube’) research”, and the phrasing of “Ideas, Editorials, Opinions” in the SUNY pyramid similarly is atypical amongst EBM hierarchies. But the underlying idea is the same: a ranking of the evidence provided by studies, according to the underlying methodology of the study, which places RCTs and systematic reviews thereof at the top, and other forms of evidence below, going from observational studies down to expert opinion and mechanistic evidence.

Other evidence pyramids were more measured in appearance, but continued to offer visual flavour. Consider the widely-reprinted efforts by Glover et al. (2006/2011) and Melnyk & Fineout-Overholt (2005) (see Fig. 4 and Fig. 5, below).

These pyramids were not simple re-representations of tabular hierarchies. They often introduced novel components into the evidence appraisal processes. In each case, these authors have introduced some quirks into their pyramidal hierarchy designs. The use of “Single descriptive or qualitative study” is unique to Melynk & Fineout-Overholt (2005), although several other sources had included qualitative evidence in some form within their rankings prior to this point (e.g. Gray 1997, Weightman, Barker & Lancaster 2000, Craven 2001, Weaver et al. 2002), and “Expert committee opinions” is a rare inclusion which harks back to the 1979 CTF hierarchy. Glover et al.’s (2006/2011) pyramid integrates Critically Appraised Topics (CATs) and Critically Appraised Individual Articles. These forms of synopsised evidence summaries are essentially included in Brian Haynes’ influential ‘4S’ and ‘5S’ hierarchies for pre-appraised evidence (Haynes 2001, 2005), albeit under the broader heading of ‘Synopses’, but not widely acknowledged elsewhere. Pyramids, like evidence hierarchies more broadly, tend to incorporate a core structure (RCT > Observational studies > Anything else) but equally embed the more idiosyncratic evidence preferences of their authors.

Since the mid-2000s, when evidence pyramids first emerged in earnest, there has been a proliferation of the mode of portrayal. However, frequently the sources for the specific rankings of evidence in the pyramids are either uncited, or cite earlier evidence hierarchies which deployed a similar ranking but which do not use a pyramidal construction. Few of the mid-2000s pyramids offer either a source for a pyramidal representation of their hierarchy, or their own justification for or explanation of the significance of a pyramid as a structure. An exception is Akobeng’s early (2005) evidence pyramid. Akobeng writes: “The pyramid shape is used to illustrate the increasing risk of bias inherent in study designs as one goes down the pyramid.” (2005, p.840)

---

2. The Origins of Evidence Pyramids

To date, no study has conclusively determined which evidence pyramid was the first to depict an EBM-style hierarchy in this novel format. I contend that the first evidence pyramid recognisable as such by EBM proponents is the SUNY Downstate pyramid (see Fig. 2, above). Following an earlier preprint version of this paper (Blunt, 2022), Martin Mellish, one of the original authors of the SUNY Downstate EBM Tutorial, made contact to confirm his role as the creator of the evidence pyramid graphic. In this section, I draw on that correspondence (Mellish, personal communication, 2023), corroborated by Dr Andrea Markinson and Christopher Stewart of SUNY Downstate (Markinson, personal communication, 2022; Stewart, personal communication, 2022), to establish the origins of the image.

Based on a systematic review completed in 2020, I identified 195 distinct evidence hierarchies, including evidence pyramids, ranging from 1979 to the present. Amongst these, the earliest evidence pyramid was found in complementary and alternative medicine researcher Wayne Jonas’s (2001) The evidence house. However, Jonas is not putting forward the evidence pyramid as a novel contribution, instead presenting it as a foil against which to develop his alternative model (the titular ‘Evidence House’), which rejects hierarchical evidence appraisal. The source which Jonas cites for his pyramid-shaped hierarchy is Sackett et al.’s Evidence Based Medicine: what it is and what it isn’t (1996). This source does not contain an evidence hierarchy in any form. No antecedent versions which match Jonas’s hierarchy have been identified, and none could be found through an exhaustive search based on the distinctive terminology employed in his presentation (e.g., “more causal research methods”, “less causal research methods” as descriptions of the high and low ranked evidence, respectively).

A rival early pyramid identified in that search is the SUNY Downstate pyramid, typically cited to 2004. I can now confirm that this evidence pyramid is the first known pyramidal representation of an evidence hierarchy, and that it was created by Martin Mellish in the winter of 1996/7, as part of a compulsory online course in Evidence-Based Medicine for SUNY medical students. The course was initiated by Jack Lebowski. Mellish served as course writer and lead HTML programmer. Betty Wagoner, a reference librarian at the Medical Research Library of Brooklyn, served as lead content specialist and subject matter expert. As Mellish describes the collaboration, the conceptual content, including the general idea of incorporating a hierarchy of evidence, was primarily Wagoner’s, while the words and graphic design of the course, including the pyramid image itself, were Mellish’s (Mellish, personal communication, 2023). While the precise moment of the pyramid’s conception is not fully recalled, Mellish recounts that the idea likely emerged from the collaborative process: “Perhaps she simply said ‘you could visualize the different types of evidence like a pyramid’, or perhaps she said something like ‘each layer of evidence provides a foundation for the layers above’ and I replied ‘kind of like a pyramid?’” (Mellish, personal communication, 2023).

The graphic itself was produced quickly and informally. As Mellish recalls: “This graphic was knocked off in 20 minutes in Paint […] I can’t say I took it seriously at all” (Mellish, personal communication, 2023). The course as a whole was never intended to be “a universally valid, or even a theoretically defensible, framework for evaluating medical studies” (Mellish, personal communication, 2023). Rather, it was designed for medical students with extremely demanding coursework and hospital internships, who had to complete the course in their limited spare time. It was delivered online because no face-to-face teaching slot could be found. As Mellish puts it, the course “needed to be simple and to provide easy-to-remember rules of thumb” (Mellish, personal communication, 2023).

This context is illuminating. The evidence pyramid, which has become one of the most recognisable and widely reproduced artefacts of the EBM movement, was not the product of extensive deliberation about the most appropriate way to visualise the relationship between forms of medical evidence. It was a pragmatic design choice, produced in twenty minutes in Microsoft Paint, for a course aimed at time-poor students. That it subsequently achieved such influence is, as Mellish reflects, characteristic of a broader pattern: “It’s been a theme of my life that what I consider my original contributions to knowledge have ‘fallen deadborn from the press’, whereas things I casually throw off without thinking too much about them frequently become quite popular.” (Mellish, personal communication, 2023).

2.1 Corroborating the Timeline

The account given by Mellish is consistent with the documentary evidence available through web archives and the published literature. In 2011, when I first accessed and locally archived the SUNY EBM Tutorial page which contained the image of the pyramid, the page included a footnote reading “Last updated January 6, 2004”. In the absence of further information regarding the nature of the updates made in 2004, or any records of previous iterations, 2004 seemed a reasonable best estimate for the date of publication, which was the standard citation for this source in the EBM literature at that timeAmong the SUNY Downstate team attributed on the archived versions of the course page, the only other individual who could be identified and responded to queries prior to Mellish’s contact was Dr Andrea Markinson, Director of the Evidence in Practice Information Center at SUNY Downstate. Dr Markinson joined SUNY Downstate in 1997. In personal communication, Dr Markinson confirmed that she did not create the evidence pyramid, that the pyramid was not added during any of the later revisions of the tutorial in which she was involved, and indeed that the pyramid was, “created as one piece of the EBM tutorial that SUNY Downstate created in the 90’s but before I came to Downstate in 1997” (Markinson, personal communication, 2022). Christopher Stewart, who also joined SUNY Downstate in 1997 and maintains the course content, accepted this recollection (Stewart, personal communication, 2022). This is consistent with Mellish’s account of creating the graphic in the winter of 1996/7.

The SUNY pyramid is generally cited as “SUNY 2004” in the EBM literature, a date derived from a “Last updated January 6, 2004” footnote on a later version of the tutorial page. However, this specific pyramid was cited by multiple papers published before 2004. Forrest & Miller (2001a) reproduced a black-and-white version of the pyramid in a paper on evidence-based dentistry, and in a separate paper (2001b) provided a URL for the pyramid listed as accessed on 4th April 2001. Webb (2001) independently described the pyramid in the British Journal of Social Work.

Web archive analysis further corroborates the timeline: the Internet Archive’s earliest imprint of the Medical Research Library of Brooklyn’s version of the page dates to 11th May 2000 and contains the full-colour pyramid image. The tutorial’s archived contents page, with a copyright date footnote of 1997, lists the evidence pyramid as a linked component. All of this is consistent with Mellish’s account of creating the pyramid in the winter of 1996/7, and establishes that it predates the next earliest evidence pyramid in the published literature, that of Jonas (2001).

Based on these convergent sources, I confidently attribute this evidence pyramid to Martin Mellish (course writer and graphic designer) and Betty Wagoner (lead content specialist), created in the winter of 1996/7 and first published online as part of the SUNY Downstate EBM Tutorial no later than 1997. I endorse a citation to Mellish & Wagoner (1997). I further assert that this is the first known evidence pyramid.

---

3. The Meaning of Evidence Pyramids

Given that the precise authorship of SUNY’s original evidence pyramid cannot be established, it is unlikely to be possible to determine what its authors intended by presenting the Previous accounts of the evidence pyramid’s meaning have been hampered by the inability to identify its creator. In the earlier version of this paper (Blunt, 2022), I wrote that “given that the precise authorship of SUNY’s original evidence pyramid cannot be established, it is unlikely to be possible to determine what its authors intended by presenting the information in this form.” With the identification of Mellish as the pyramid’s designer, this gap can now be addressed. In this section, I first present Mellish’s intended interpretation, before considering alternative readings that have developed in the subsequent literature and examining the inventive reinterpretation offered by Tomlin & Borgetto (2011).

3.1 The Intended Interpretation: Foundation and Reality

Mellish’s account of the pyramid’s intended meaning combines two related ideas. The first is a relationship of dependency, or foundation. As Mellish writes:

“Why a ‘pyramid’? Because in vitro and animal studies, case files, expert opinions etc. may not be of much direct value in themselves, but they form an indispensable basis for higher-quality research by pointing out questions that might be fruitful to investigate. Without the base, the higher levels of the ‘pyramid’ would not exist.”

(Mellish, personal communication, 2023)

On this reading, the pyramid’s structure is not merely aesthetic. The lower tiers are genuinely foundational: they generate the questions, hypotheses and preliminary observations without which higher-tier research could not be designed or justified. An RCT does not materialise from nothing. It is typically preceded by mechanistic reasoning which establishes biological plausibility, by expert opinion which identifies the clinical question as worth pursuing, and perhaps by observational studies which provide preliminary evidence of an effect. The pyramid, so interpreted, represents this generative relationship.

The second element of Mellish’s intended interpretation is the narrowing prevalence of higher-tier study types. As Mellish explains, “the ‘pyramid’ narrowed because RCT and RCT + DB studies were rare at the time” (Mellish, personal communication, 2023). This is consistent with the late 1990s research landscape, in which randomised controlled trials were considerably less prevalent than they have since become.

This combined interpretation, foundation plus rarity, is considerably more philosophically substantive than most readings of evidence pyramids in the subsequent literature. It offers a picture in which the lower tiers of the evidence base are not merely inferior forms of evidence to be tolerated in the absence of better, but are rather necessary precursors without which the higher tiers could not exist or could not be suitably interpreted or applied. This is a notably different claim from the standard hierarchical reading, in which lower-tier evidence is simply of lower quality and should be superseded wherever possible.

3.2 An Interpretation Lost

This foundational reading aligns most closely, among the standard interpretive frameworks for pyramidal structures, with the dependency interpretation I discuss below. However, it is striking that this is not the interpretation which has predominated in the subsequent EBM literature. Few, if any, of the many authors who have reproduced or adapted evidence pyramids in the years since have articulated this foundational reading. Instead, the dominant interpretation has been purely hierarchical: higher is simply better, and the pyramid is merely a table reimagined.

This is perhaps unsurprising given the circumstances of the pyramid’s creation. The course for which it was designed was not intended as a contribution to the philosophy of evidence appraisal, but as a practical heuristic for busy medical students. No published paper accompanied the pyramid to explain its intended reading. As the image was reproduced, adapted and redrawn across the EBM literature, often without citation, and certainly without consultation of the original authors, any intended meaning beyond the simple ranking was progressively excised. The pyramid became, in effect, a table dressed in a pyramid’s clothing: a simple ranking of evidence types, with RCTs at the top and everything else below, conveyed through a shape whose structural implications were irrelevant.

3.3 The Narrowing Prevalence Interpretation

Setting aside the creator’s intentions, it is worth examining the interpretations that have been attributed to evidence pyramids in the broader literature. The most common is that the breadth of each tier represents the prevalence of that evidence type. The narrower the tier of the pyramid, the rarer evidence of that type. This will often be the case where ‘superior’ instances are more difficult to produce than lower-tier ones, and thus can be expected to be rarer. This explanation for the pyramidal shape has been promulgated in the secondary EBM literature. For instance, Yetley et al. (2016) reproduce an evidence pyramid and write:

 “The pyramidal shape qualitatively integrates the amount of evidence generally available from each type of study design and the strength of evidence expected from indicated designs. In each ascending level, the amount of available evidence generally declines.”

Yetley et al., 2016, p.11S.

Such an approach may reflect the design assumptions behind, for instance, Haynes’ ‘4S’, ‘5S’ and ‘6S’ hierarchies of pre-appraised evidence sources (see Fig. 6, below), which begin with individual studies, of which there is a vast proliferation, and move up towards synopses and systematic reviews. This interpretation sometimes seems viable. There are certainly more clinicians expressing opinions than there are studies addressing any given question. However, particularly since the rise of EBM and the promulgation of RCTs as the highest quality evidence through precisely such hierarchies, RCTs have proliferated while observational studies have declined in popularity. Particularly for evidence pyramids which draw fine-grained distinctions amongst observational studies (separating cohort studies from case-control studies, for instance, into separate tiers), this interpretation seems unlikely to track with real-world prevalence.

Certainly, this interpretation would seem inconsistent with the original SUNY pyramid, in which ‘Animal research’ and ‘In vitro (“test tube”) research’ are included at a tier below ‘Ideas, opinions, editorials’, despite surely being less frequent. While Mellish confirms that prevalence was part of the intended reading, this dimension applies unevenly across the tiers of the original pyramid.

3.4 The Dependency Interpretation

In other pyramidal representations, the relationship between tiers is one of dependency. Each level of the pyramid rests upon the one below. While reaching a higher point in the pyramid is preferable, one must rise sequentially through the tiers, or must satisfy the lower levels in order to satisfy the higher. In Maslow’s (1943) famous hierarchy of needs, one of the few other explicitly hierarchical structures commonly presented as a pyramid, each set of needs can only be satisfied once the tier of needs below have been met. Base tier needs are dependencies for second tier needs, and so on. Mellish’s intended interpretation is closest to this reading, though with an important distinction. In the Maslow model, dependency is strict: self-actualisation needs cannot be satisfied unless safety needs are first met. Mellish’s foundational reading is softer: lower-tier evidence typically informs and enables higher-tier research, but this is not a strict logical dependency. One can conduct an RCT without prior cohort studies, even if one seldom would. Evidence hierarchies are not generally intended as a research trajectory to be followed, starting with low-level evidence and working one’s way up as a research programme. It is unlikely that a sequential or dependency relation is intended for the majority of evidence pyramids, with the possible exception of Haynes’ (2001; 2005).

Nonetheless, the foundational interpretation offers a more philosophically interesting account of the relationship between forms of medical evidence than the purely hierarchical reading. It recognises that different forms of evidence serve different functions within the broader research enterprise, and that the forms of evidence typically ranked lowest (case reports, mechanistic reasoning, expert opinion) play an essential role in identifying questions, generating hypotheses, and establishing the preconditions for higher-tier research. This is broadly consistent with more pluralistic accounts of medical evidence (see e.g. Clarke et al. 2014; Parkkinen et al. 2018), and sits in marked contrast to the tendency within EBM to treat lower-ranked evidence types as simply inferior.

3.5 The Risk of Bias Interpretation

A final interpretation found in the literature is that the breadth of each tier represents the extent of bias, imprecision or risk involved in using evidence at that tier, pseudo-analogous to the width of a confidence interval. This matches Akobeng’s (2005) aforementioned explanation of their own pyramid: to “illustrate the increasing risk of bias inherent in study designs as one goes down the pyramid” (2005, p.840). While consistent with the intended interpretation of most pyramids by design, this offers very little meaning beyond the simple rank order. As such, Akobeng’s minimal interpretation offers no practical merit of presenting a hierarchy as a pyramid beyond the aesthetics.

3.6 Beyond the Original: Three-Dimensional Pyramids

Pyramids are three-dimensional structures. The majority of evidence pyramids in the EBM literature, however, are two-dimensional, and tend to make use of only one dimension. While most subsequent evidence pyramids stripped away the more nuanced reading intended by the original (the second dimension), one notable exception deserves extended discussion. Tomlin & Borgetto’s (2011) research pyramid for occupational therapy is the only evidence pyramid which makes genuinely meaningful use of the three-dimensional structure that the pyramid form affords.

In personal communication following the earlier version of this paper, Tomlin & Borgetto explained that their three-dimensional pyramid structure is derived from two meaningful axial considerations: “separating qualitative from quantitative research methods, and also distinguishing research designs which are inherently higher in internal validity […] versus those that are higher in external validity” (Tomlin & Borgetto, personal communication, 2023). These distinctions produce different “faces” of quantitative and qualitative research. The vertical juxtaposition of these three faces is intended to convey parity between them: “One should not be subordinated to the other, as typically happens in evidence ‘pyramids’” (Tomlin & Borgetto, personal communication, 2023).

Tomlin & Borgetto have subsequently developed a four-sided version of their pyramid, with descriptive research forming the base. As they describe it, this base represents “where systematic knowledge collection methodologies began” and is broadest “because each practitioner is capable of composing and publishing such studies.” Each subsequent tier represents “methodological innovations which lift the knowledge level/value to a greater height” (Tomlin & Borgetto, personal communication, 2023). This developmental reading resonates with Mellish’s foundational interpretation: the base is not the lowest-quality evidence, but the starting point from which more sophisticated methodologies progressively build.

Tomlin & Borgetto’s pyramid is distinctive in several respects. It is, to my knowledge, the only evidence pyramid where the three-dimensional structure is “meant to have a crucial meaningfulness” (Tomlin & Borgetto, personal communication, 2023). Most other evidence pyramids are, in effect, evidence triangles. Their model also explicitly addresses the tension between demonstrating effectiveness to external bodies via controlled research and informing practice decisions through evidence drawn from real-world clinical conditions, a distinction which the EBM literature tends to collapse. They have additionally been developing criteria for what they term “promotions” and “demotions”, conditions which adjust a study’s rating based on how it was implemented, analogous to the conditions discussed in the next section.

3.7 The Rhetorical Power of Pyramids

The intended interpretation notwithstanding, a pyramidal presentation may help to convey the sense of authority upon which EBM has often depended in cementing such evidence appraisal systems with medical practice. The uncertainty of the provenance has historically exempted the underlying assumptions of the pyramid from critique, as the hierarchy could be ascribed to historical precedent (see Blunt 2015, ch.2). The identification of the pyramid’s actual origins may itself prove a useful corrective to the reverence with which the form is sometimes treated.

The resemblance to other fixtures within epidemiological theory may also suggest inspiration for the design, and allow proponents to draw on the heuristic of acceptance of familiar forms. John Last’s ‘iceberg’ model of disease (Last, 1963) is a prominent way to represent hidden and subclinical suffering in a visual manner which has received thousands of citations in the epidemiological literature (Last, 2013). Last describes the popular appeal of such a structure as “durable and useful”, and a metaphor which is “a valuable communications aid, immediately grasped by everyone” (2013, p.1613). Last’s iceberg is frequently depicted as a pyramid (cf. Houben et al. 2022; Pfeiffer, 2002 – see Fig. 7, below), and has been reformulated as the “disease prevalence pyramid” (e.g. Zuccon et al., 2015). Elsewhere, the US Department of Health and Human Services’ 1994 For a Healthy Nation report promulgated the four-tier “Health Care Pyramid” (Gold et al. 1994). Confusingly, that model used the breadth of the pyramid’s base to convey the underfunding of population-based health services, an inversion of prestige when compared to EBM’s usage, in which the foundations are fundamental but underappreciated and the upper tiers are overvalued.

In summary, the original evidence pyramid carried a more substantive intended meaning than has generally been recognised, one in which the lower tiers are valued as indispensable foundations rather than dismissed as inferior evidence. This interpretation was largely lost as the format proliferated. Subsequent evidence pyramids have overwhelmingly adopted a purely hierarchical reading in which the pyramid adds nothing beyond the ranking already conveyed by a simple table. With the notable exception of Tomlin & Borgetto’s genuinely three-dimensional model, there is little practical or philosophical difference between most evidence pyramids and a table as approaches to express a ranking of evidence according to underlying methodology that would justify the choice of a pyramid structure rather than a table or list.

---

4. The Curse of the Pyramids

However, even if most evidence pyramids offer little in the way of additional information conveyed beyond that of tables, there remain several drawbacks of the form. I close by offering two deficiencies of pyramidal hierarchies which diminish the capacity and usefulness of a hierarchy so presented, to the conclusion that evidence pyramids are, on balance, a regressive step in the evolution of evidence hierarchies.

4.1 Conditionality

First, evidence pyramids are less apt to presenting conditional rankings. As such, evidence pyramids are far more likely to adopt a primarily or entirely unconditional ranking than would be available in a non-pyramidal presentation. I previously (Blunt 2015) distinguished between unconditional and conditional rankings in evidence hierarchies. Hierarchies primarily rate or rank evidence according to the method used to produce it. But a great many also make reference to other factors, by way of which they can offer a more fine-grained set of distinctions and acknowledge that not all studies of a given methodology produce evidence that is equally strong or of equal quality. Conditions are requirements other than the type of underlying methodology which produced a study, which are factored into the rating or ranking of the evidence within a hierarchical structure.

A wide range of conditions have been appended to qualify for a higher ranking in evidence hierarchies. Some are themselves evaluative. For instance, the original Canadian Task Force (1979) hierarchy requires that cohort or case-control studies be “well designed” to qualify for level II-1. Others require optional methodological features such as blinding (e.g. Davies & Nutley 1999). The size of studies or the precision of the results are frequently included as conditions, following Sackett’s influential Chest hierarchies (1986; 1989) which differentiate “large” and “small” studies, and evidence “with clear-cut results” and “with uncertain results”. Provenance is another common condition; for instance, the much-replicated LaForce (1987) hierarchy refers to studies “preferably from more than one center or research group”. Conditions can often be complex and lengthy, as exemplified in Cook et al.’s (1992) hierarchy which includes conditions such as “the lower limit of the confidence interval for the effect of treatment exceeds the clinically significant benefit”, “individual study results are homogeneous”, and “with low false-positive (alpha) and low false negative (beta) errors”.

While some simple conditions can be applied within a pyramid structure, the aesthetics of the design tends to limit the prevalence and level of detail of conditions applied. Particularly at the higher levels, as the pyramid narrows, there is seldom physical space to offer a nuanced set of conditions. Thus, the design choice may constrain the sophistication of a hierarchy. From the start, hierarchies have included conditions. But the pyramidal turn coincides with, and may have contributed to, the paring back of these conditions within EBM hierarchies. Reduced conditionality is a significant contributor to the oversimplification of evidence appraisal processes using evidence hierarchies.

It is particularly problematic that the pyramidal structure discourages conditionality at the highest echelons of the ranking. Most conditions are positively framed: evidence must meet the condition to attain a higher rank, rather than face downgrading for meeting an undesirable condition of a lower rank. This is sensible from the perspective of ensuring clarity in the ranking. For instance, if a ranking of tier 2 is reserved for “well-designed cohort studies”, while tier 3 is “cohort studies”, then it is clear where any cohort study ranks: it satisfies the conditions for 2 and 3, so is ranked at the highest level which it satisfies, tier 2. But if a ranking of tier 2 lists “cohort studies” while tier 3 lists “poorly designed cohort studies”, then ambiguity is introduced. Moreover, a higher ranking corresponds to a higher rating of quality or strength, and we would expect more criteria to be applicable to attain such a ranking. As such, higher echelons should tend to attract more conditions, for both practical and theoretic reasons. Pyramidal schemas push against this natural property, and thus discourage conditionality as a consequence of design.

This trend is borne out across the history of evidence hierarchies. Prior to 1997, when the first evidence pyramid was created, only one major evidence hierarchy was unconditional (Anonymous, 1981), amongst a preponderance of conditions. Around the turn of the millennium, unconditional hierarchies proliferated (e.g. McAlister et al. 1999; Briss et al. 2000; Craven 2001; McLeod 2001; McAlister & Sackett 2001). It would be premature to infer a causal relationship here. The aforementioned unconditional hierarchies are primarily in list or tabular form, and the turn towards pyramids could equally be informed by the reduction in complexity of hierarchies as opposed to a driver of that trend towards simplification.

But a dearth of conditions is a theme amongst evidence pyramids. The only hint of conditionality in the original SUNY pyramid is the inclusion of “double blind” in the description of RCT evidence, an optional methodological feature. However, there is no tier for unblinded or single-blind RCT evidence, suggesting that the author of the SUNY Downstate pyramid might simply equate RCT evidence with double-blind RCT evidence. Similarly, the early evidence pyramids are all entirely unconditional: Jonas (2001), Haynes (2001; 2005; DiCenso, Bayley & Haynes 2009), Akobeng (2005), Dagenais et al. (2006), Daly et al. (2007), Sprague, McKay & Thoma (2008), Bigby (2009), Crosswell & Kramer (2009) and Glover et al. (2006/2011) all provide unconditional evidence pyramids.

The first evidence pyramids to include minor conditions did not emerge until 2008, with Mantzoukis’s second tier reading “At least one well-conducted RCT” (2008). It is not until 2011 that an evidence pyramid included conditions at multiple levels; Tomlin & Borgetto’s inventive 2011 ranking includes a top-down view of a three-sided pyramid, with conditions including “related”, “blinded” and “prolonged engagement with patients” included at different tiers. But to achieve this, and despite their aesthetic innovation with the form, Tomlin & Borgetto feel compelled to include a separate table which replicates the pyramid in a traditional list-style hierarchy to give the full detail of each ranking. Notably, Tomlin & Borgetto have continued to develop what they term “promotions” and “demotions”, criteria for adjusting a study’s position based on implementation quality (Tomlin & Borgetto, personal communication, 2023). Almost every other evidence pyramid since has remained unconditional, or includes at most one or two brief conditions, primarily distinguishing among categories at the lower reaches of the pyramid.

4.2 Non-Categorical Rankings

A second drawback of the pyramid format is the inability to convey a non-categorical ranking in this design. In Blunt (2015), I distinguish categorical from non-categorical rankings, insofar as a categorical ranking implies that all evidence which meets the criteria for a particular rank or rating in a hierarchy definitively receives that rank or rating. The defining feature of a non-categorical hierarchy is that it is possible for some lower-ranked evidence to nonetheless be higher quality (or stronger, etc.) than some higher ranked evidence. In non-categorical hierarchies, either evidence might move up or down the rating scale according to further criteria (as in the popular GRADE systems, see e.g. GRADE Working Group 2004; Balshem et al. 2011), or the hierarchy is explicitly intended to be interpreted with some modifiers attached, for instance that tier 1 ranked evidence ‘tends to be high quality’ or ‘has a high likelihood to produce strong evidence’, etc. Non-categorical hierarchies avoid many of the philosophical criticisms directed towards hierarchical approaches to evidence appraisal, and are generally more flexible and defensible in the context of the highly variable character of the medical research literature.

While one could find inventive graphical ways to depict a non-categorical evidence pyramid, the format is not suited to this more nuanced understanding of medical evidence. Of the vast range of distinct evidence pyramids found across the medical literature, only one has ever offered a suggestion of a non-categorical interpretation: Murad et al.’s New Evidence Pyramid (2016) offers a wavy-tiered pyramid to convey that lower-ranked evidence sources are not necessarily always inferior to higher-ranked ones (see Fig.8, below).

Nonetheless, Murad et al.’s pyramid is unusual, subverting the expected aesthetic of a pyramid, and may be more open to interpretation than other non-categorical approaches. The wavy-tiered approach also can only accommodate a level of flexibility between two neighbouring tiers. In comparison, the GRADE framework allows for evidence which starts at a particular tier based on the underlying methodology to ultimately reach any level within the ranking based on a compounding of multiple independent conditions (see Balshem et al. 2011). This flexibility is difficult to envision within a pyramid framework, and has never been achieved.

---

5. Conclusion

In conclusion, the origins of evidence pyramids can now be traced with confidence to 1997, when course writer Martin Mellish created the graphic for a SUNY Downstate EBM tutorial, working from content provided by subject matter expert Betty Wagoner. The pyramid was produced in twenty minutes in Microsoft Paint, as a practical teaching aid for time-poor medical students, and was never intended as a theoretically definitive framework for evaluating medical research. Yet it has proliferated considerably across the Evidence-Based Medicine literature in the years since, becoming one of the most recognised artefacts of the movement.

The identification of the pyramid’s creator reveals that the original design carried a more philosophically substantive interpretation than has generally been attributed to it. In Mellish’s intended reading, the lower tiers of the pyramid are not merely inferior forms of evidence, but an indispensable basis for higher-tier research, forming the foundation of questions, hypotheses and preliminary observations without which RCTs and systematic reviews could not be conducted. This foundational reading offers a more coherent account of the relationship between different forms of medical evidence than the purely hierarchical interpretation which has predominated.

However, this intended meaning was almost entirely lost as evidence pyramids proliferated. The informal, undocumented circumstances of the pyramid’s creation meant that subsequent adopters of the format had no access to its intended interpretation. The resulting profusion of evidence pyramids has overwhelmingly treated the form as a simple ranking in which higher is better, adding nothing of substance beyond what would be conveyed by a table or list, while simultaneously constraining more nuanced design options.

The pyramid structure carries practical drawbacks which make it a retrograde step in the development of evidence appraisal systems. By cramping the space available towards the zenith of the structure, authors who adopt a pyramid format are discouraged by design from introducing conditions to achieve the highest levels. Pyramid structures also make it difficult to visually depict any flexibility between the levels to convey a more nuanced non-categorical interpretation of the ranking. With very few exceptions, most notably Tomlin & Borgetto’s (2011) genuinely three-dimensional model, which makes meaningful use of the pyramid’s spatial properties, evidence pyramids have typified a simplification of the appraisal system, both in comparison to the hierarchies which preceded them (e.g. Canadian Task Force 1979; Sackett 1986; LaForce 1987; Cook et al. 1992; Guyatt et al. 1995; Greenhalgh 1997) and in comparison to other contemporaneous trends within hierarchy development, such as the move towards more flexible designs such as the GRADE framework (GRADE Working Group 2004; Balshem et al. 2011).

Overall, the pyramidal turn in evidence hierarchy presentation is a negative force which detracts from the level of sophistication of evidence appraisal techniques promulgated in EBM, and counteracts the movement towards less rigid appraisal systems which move beyond the hierarchical model. It is a particular irony that the original evidence pyramid was intended to convey a richer, more foundational understanding of the relationship between evidence types, one more sympathetic to the value of evidence forms typically dismissed as inferior, but that this meaning was systematically stripped away as the format achieved the very prominence that its creator never anticipated.

---

References:

• Akobeng, A. K. (2005). “Understanding randomised controlled trials.” Archives of disease in childhood, 90(8), 840-844.
• Anonymous (acknowledged to be authored by David Sackett) (1981) “How to read clinical journals: IV. To determine etiology or causation”, CMAJ, 124, 985-990.
• Australian National Health and Medical Research Council (ANHMRC). (1999) A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines. Commonwealth of Australia.
• Balshem, H., et al. (2011) “GRADE guidelines: 3. Rating the quality of evidence”, J Clin Epidemiol, 64(4), 401-406.
• Bigby, M. (2009) “The Hierarchy of Evidence”, in Williams, H., Bigby, M., Diepgen, T., Herxheimer, A., Naldi, L., & Rzany, B. (Eds.). (2009). Evidence-based dermatology. (John Wiley & Sons.)
• Blunt, C.J. (2015) Hierarchies of Evidence in Evidence-Based Medicine (PhD Thesis) London School of Economics and Political Science, available at: http://etheses.lse.ac.uk/3284/
• Blunt, C.J. (2019) A Ghost of Progress: how hierarchies become fixtures, available at: https://cjblunt.com/a-ghost-of-progress-how-hierarchies-become-fixtures/
• Blunt, C.J. (2020) Random Reflections: Cochrane and the Origins of Hierarchies, available at: https://cjblunt.com/random-reflections/
• Blunt, C.J. (2022) The Pyramid Schema: The Origins and Impact of Evidence Pyramids [preprint], available at: https://cjblunt.com/the-pyramid-schema/
• Briss, P. A., et al. (2000) “Developing an evidence-based Guide to Community Preventive Services–methods. The Task Force on Community Preventive Services”, Am J Prev Med, 18(1 Suppl), 35-43.
• Campbell, D.T. & Stanley, J.C. (1963) Experimental and quasi-experimental designs for research (Chicago, IL: Rand McNally & Co.)
• Canadian Task Force on the Periodic Health Examination (1979) “The Periodic Health Examination”, Canadian Medical Association Journal, 121(9): 1193-1254.
• Cartwright, N. (2007) “Are RCTs the gold standard?” BioSocieties, 2(1), 11-20.
• Clarke, B., Gillies, D., Illari, P., Russo, F. & Williamson, J. (2014) “Mechanisms and the evidence hierarchy”, Topoi, 33: 339-360.
• Cochrane, A. (1972) Effectiveness and efficiency: Random reflections on health services (Nuffield Trust).
• Cook, D. J., et al. (1992) “Rules of evidence and clinical recommendations on the use of antithrombotic agents”, Chest, 102(4 Suppl), 305S-311S.
• Craven, O. (2001) “Screening for colorectal cancer using the faecal occult blood test: a critical literature review.” European Journal of Oncology Nursing, 5(4), 234-243.
• Croswell, J.M., & Kramer, B.S. (2009) “Clinical trial design and evidence-based outcomes in the study of liver diseases.” Journal of hepatology, 50(4), 817-826.
• Dagenais, S., Tricco, A. C., Bian, Z. X., Huang, W. H., & Moher, D. (2006) “Critical appraisal of clinical studies in Chinese herbal medicine.” Zhong xi yi jie he xue bao, Journal of Chinese integrative medicine, 4(5), 455-466.
• Daly, J. (2005) Evidence-based medicine and the search for a science of clinical care. (Berkeley, CA: University of California Press)
• Daly, J., et al. (2007) “A hierarchy of evidence for assessing qualitative health research”, J Clin Epidemiol, 60(1), 43-49.
• Davidoff, F., et al. (1995) “Evidence based medicine”, BMJ, 310(6987), 1085-1086.
• Davies, H.T.O. & Nutley, S.M. (1999) “The Rise and Rise of Evidence in Health Care”, Public Money & Management, 19(1): 9-16.
• DiCenso, A., Bayley, L. & Haynes, R.B. (2009) “ACP Journal Club. Editorial: Accessing preappraised evidence: fine-tuning the 5S model into a 6S model”, Ann Intern Med, 151(6), JC3-2, JC3-3.
• Evidence-Based Medicine Working Group (1992) “Evidence-based medicine. A new approach to teaching the practice of medicine.” JAMA, 268(17), 2420–2425.
• Forrest, J.L. & Miller, S.A. (2001a) “Enhancing your practice through evidence-based decision making: Finding the best clinical evidence”, Journal of Evidence-Based Dental Practice, 1(3): 227-36.
• Forrest, J.L. & Miller, S.A. (2001b) “Enhancing your practice through evidence-based decision making”, Journal of Evidence-Based Dental Practice, 1(1): 51-7.
• Glasziou, P., Vandenbroucke, J. & Chalmers, I. (2004) “Assessing the quality of research”, BMJ, 328, 39-41.
• Glover, J. et al. (2011) “EBM Page Generator”, from www.ebmpyramid.org
• Gold, M. et al. (1994) For a Healthy Nation: Return on Investments in Public Health (US Public Health Service & US Department of Health and Human Services: Hyattsville, MD.)
• GRADE Working Group. (2004) “Grading quality of evidence and strength of recommendations”, British Medical Journal, 328, 1490.
• Gray, J.A.M. (1997) Evidence-Based Healthcare. (NYC: Churchill-Livingstone)
• Greenhalgh, T. (1997) How to read a paper: the basics of evidence based medicine. (London: BMJ Pub. Group)
• Guyatt, G.H., et al. (1995) “Users’ guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group”, JAMA, 274(22), 1800-1804.
• Guyatt, G.H., et al. (2000) “Users’ Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users’ Guides to patient care. Evidence-Based Medicine Working Group”, JAMA, 284(10), 1290-1296.
• Guyatt, G.H., et al. (2001) “The Philosophy of Evidence-Based Medicine”, in Guyatt, G.H. et al. (eds.) Users’ Guides to the Medical Literature (Chicago, IL: American Medical Association).
• Haynes, R.B. (2001) “Of studies, syntheses, synopses, and systems: the 4S evolution of services for finding current best evidence”, ACP J Club, 134(2), A11-13.
• Haynes, R.B. (2005) “Of studies, syntheses, synopses, summaries, and systems: the 5S evolution of information services for evidence-based health care decisions”, ACP J Club, 145(3), A8-A8.
• Houben, R.M.G.J. et al. (2022) “Tuberculosis prevalence: beyond the tip of the iceberg”, Lancet: Respiratory Medicine, 10(6): P-537-9.
• Jonas, W.B. (2001) ‘The evidence house: How to build an inclusive base for complementary medicine’, in Western Journal of Medicine, 175: (2) 79-80.
• LaForce, F. M. (1987) “Immunizations, immunoprophylaxis, and chemoprophylaxis to prevent selected infections. US Preventive Services Task Force”, JAMA, 257(18), 2464-2470.
• Last, J.M. (1963) “The Iceberg: ‘Completing the clinical picture’ in general practice”, Lancet, 2: 28-31.
• Last, J.M. (2013) “Commentary: The iceberg revisited”, International Journal of Epidemiology, 42(6): 1613-5.
• Mantzoukas, S. (2008). “A review of evidence-based practice, nursing research and reflection: levelling the hierarchy.” Journal of clinical nursing, 17(2), 214-223.
• Maslow, A.H. (1943) “A Theory of Human Motivation”, Psychological Review, 50: 370-396.
• McAlister, F. A., et al. (1999) “Users’ Guides to the Medical Literature: XIX. Applying clinical trial results B. Guidelines for determining whether a drug is exerting (more than) a class effect”, JAMA, 282(14), 1371-1377.
• McAlister, F. A., & Sackett, D. L. (2001) “Active-control equivalence trials and antihypertensive agents.” American journal of medicine, 111(7), 553-558.
• McLeod R.S. (2001) “Evidence-Based Surgery.” In: Norton J.A. et al. (eds) Surgery. (Springer, Berlin, Heidelberg)
• Mellish, M. & Wagoner, B. (1997) “Guide to Research Methods: The Evidence Pyramid”, SUNY Downstate Medical Center: Medical Research Library of Brooklyn, EBM Tutorial, version 1.3, first known publication at latest 11 May 2000, archived at https://web.archive.org/web/20000511202739/http://www.servers.medlib.hscbklyn.edu/ebm/2100.htm
• Melnyk, B.M. & Fineout-Overholt, E. (2005) Evidence-Based Practice in Nursing and Healthcare: A Guide to Best Practice (Philadelphia: Lippincoot, Williams & Wilkins)
• Murad MH, Asi N, Alsawas M, & Alahdab F. (2016) “New evidence pyramid.” Evid Based Med. 21(4): pp.125-7.
• Parkkinen, V.-P., Wallmann, C., Wilde, M., Clarke, B., Illari, P., Kelly, M.P., Norell, C., Russo, F., Shaw, B., & Williamson, J. (2018) Evaluating Evidence of Mechanisms in Medicine: Principles and Procedures. SpringerBriefs in Philosophy: SpringerOpen.
• Pfeiffer, D.U. (2002) Veterinary Epidemiology: An Introduction (Royal Veterinary College: Hertfordshire, UK)
• Phillips, R. et al. (1998) “Levels of Evidence”. Centre for Evidence-Based Medicine, www.cebm.net.
• Rothwell, P.M. (2005) “External validity of randomised controlled trials: ‘To whom do the results of this trial apply?’” The Lancet, 365(9453), 82-93.
• Rychetnik, L., et al. (2002) “Criteria for evaluating evidence on public health interventions”, J Epidemiol Community Health, 56(2), 119-127.
• Sackett, D.L. (1986) “Rules of evidence and clinical recommendations on the use of antithrombotic agents”, Chest, 89(2 Suppl), 2S-3S.
• Sackett, D.L. (1989) “Rules of evidence and clinical recommendations on the use of antithrombotic agents”, Chest, 95(2 Suppl), 2S-4S.
• Sackett, D.L. & Rosenberg, W.M. (1995) “On the need for evidence-based medicine”, J Public Health Med, 17(3), 330-334.
• Sackett, D.L., et al. (1996) “Evidence based medicine: what it is and what it isn’t”, British Medical Journal, 312: 71-72.
• Sackett, D.L., et al. (1997) Evidence-based medicine: how to practice and teach EBM. (New York; Edinburgh: Churchill Livingstone)
• Sprague, S., McKay, P., & Thoma, A. (2008) “Study design and hierarchy of evidence for surgical decision making.” Clinics in plastic surgery, 35(2), 195-205.
• Tomlin, G. & Borgetto, B. (2011) “Research Pyramid: A New Evidence-Based Practice Model for Occupational Therapy.” American Journal of Occupational Therapy, 65, pp.189-196.
• Vere, J. (2018) Evidence based medicine: a critical analysis [PhD Thesis] University of Sheffield.
• Vere, J. & Gibson, B. (2020) “Variation amongst hierarchies of evidence”, Journal of Evaluation in Clinical Practice, 1-7.
• Wagoner, B. et al. (2004) “Guide to Research Methods: The Evidence Pyramid”, in SUNY Downstate Medical Center: EBM Tutorial, formerly available at: http://library.downstate.edu/EBM2/2100.htm
• Weaver, N., Williams, J. L., Weightman, A. L., Kitcher, H. N., Temple, J. M. F., Jones, P., & Palmer, S. (2002) “Taking STOX: developing a cross disciplinary methodology for systematic reviews of research on the built environment and the health of the public.” Journal of Epidemiology & Community Health, 56(1), 48-55.
• Webb, S.A. (2001) “Some considerations on the validity of evidence-based practice in social work”, British Journal of Social Work, 31: 57-79.
• Weightman A.L., Barker J., & Lancaster J. (2000) “A systematic approach to identifying the evidence. Project Methodology 3.” Health Evidence Bulletins Wales. Cardiff: Department of Information Services, University of Wales College of Medicine.
• Yetley, E.A. et al. (2016) “Options for basing Dietary Reference Intakes (DRIs) on chronic disease endpoints: report from a joint US-/Canadian-sponsored working group”, American Journal of Clinical Nutrition, 105(1): 249S-285S.
• Zuccon, G. et al. (2015) “Automatic detection of tweets reporting cases of influenza like illnesses in Australia”, Health Information Science and Systems, 3: S4-13.

---

Acknowledgements:

My thanks go to Prof. Burt Gerstman who first raised the question with me of the origins of evidence pyramids specifically.

Most sincere thanks are due to Pekka Louhiala who also posted about the origins of evidence pyramids on the Philosmed mailing list, and for their correspondence and effort in researching this topic and other hierarchies of interest.

I am particularly grateful to Martin Mellish for making contact following the publication of the preprint version of this paper, and for his generous and detailed correspondence regarding the creation of the evidence pyramid. His willingness to share both the factual details and the broader context of the pyramid’s origins has transformed this paper’s account of the evidence pyramid’s provenance.

Thanks also to Dr Andrea Markinson and Christopher Stewart of SUNY Downstate for their recollections of the SUNY pyramid and for corroborating the timeline of its creation.

My thanks to George S. Tomlin and Bernhard M. Borgetto for their detailed correspondence regarding the intended interpretation of their three-dimensional research pyramid, and for sharing their ongoing work in developing the model.

Most recent update: 11/02/26 (original publication)

A previous version of this paper appeared in 2022, and is available here: https://cjblunt.com/the-pyramid-schema/