Hierarchies of Evidence

Hierarchies of Evidence are a tool employed by many advocates of Evidence-Based Medicine. They are used to appraise evidence from a range of sources, as well as to teach medical students about evidence and evidence appraisal. My PhD thesis concerns the variation in hierarchies defended, and the range of philosophical interpretations of those hierarchies. In order to complete this project, it is necessary to create and maintain a database of hierarchies of evidence. This database is available here for the benefit of other researchers interested in Evidence-Based Medicine and hierarchies of evidence. Where possible, links are provided to a pdf of the relevant source. If not, links are provided to the journal listing or to google books. Please contact me if you notice any broken links. This database is up-to-date as of 29/06/16. If you know of any hierarchies not included in this list, please contact me. I am particularly interested in non-English hierarchies which are not available in English translation. The database currently only includes hierarchies available in English. For the purposes of this database, quite a broad selection criterion has been employed to maximise scope. The definition used here is:

“A procedure for making an inference about some epistemic property of the evidence provided by studies, or some epistemic relation which holds between the evidence provided by two or more studies, primarily based on a ranking of the design or methodology used in the studies.”

I also include for reference purposes a number of sources which are commonly called or cited as hierarchies or precursors to hierarchies, but which do not meet any reasonable definition of a hierarchy of evidence, in a separate section.

1979Canadian Task ForceCMAJCanadian Task Force on the Periodic Health Examination. (1979) "The Periodic Health Examination". CMAJ 121 1193-1252"
1981AnonymousCMAJ(1981) “How to read clinical journals: IV. To determine etiology or causation”. CMAJ, 124, 985-990.
1986SackettChestSackett, D. L. (1986) “Rules of evidence and clinical recommendations on the use of antithrombotic agents”. Chest, 89(2 Suppl), 2S-3S.
1987LaForceJAMALaForce, F. M. (1987) “Immunizations, immunoprophylaxis, and chemoprophylaxis to prevent selected infections. US Preventive Services Task Force”. JAMA, 257(18), 2464-2470
1989SackettChestSackett, D. L. (1989) “Rules of evidence and clinical recommendations on the use of antithrombotic agents”. Chest, 95(2 Suppl), 2S-4S.
1992Cook et al.ChestCook, D. J., et al. (1992) “Rules of evidence and clinical recommendations on the use of antithrombotic agents”. Chest, 102(4 Suppl), 305S-311S
1992McGowan et al.Infectious DiseaseMcGowan, J.E. et al. (1992) “Guidelines for the use of systemic glucocorticosteroids in the management of selected infections”. Journal of Infectious Disease, 165, 1-13
1993Ogilvie et al.CMAJOgilvie, R. I., et al. (1993) “Report of the Canadian Hypertension Society Consensus Conference: 3. Pharmacologic treatment of essential hypertension”. CMAJ, 149(5), 575-584
1995AnonymousBandolierAnonymous. (1995) “Evidence-Based Everything”. Bandolier, 12, 91-98
1995Guyatt et al.JAMAGuyatt, G. H., et al. (1995) “Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group”. JAMA, 274(22), 1800-1804
1995Byar/PiantadosiControlled Clinical TrialsPiantadosi, S. (1995). "David Byar as a teacher." Controlled Clinical Trials, 16 (4), p. 202 ~~ Byar's hierarchy is referenced by Olgin and Clarke but never published by Byar. Piantadosi published a version.
1995OlkinJCEOlkin, I. (1995) "Statistical and theoretical considerations in meta-analysis". Journal of clinical epidemiology (0895-4356), 48 (1), p. 133. ~~ An expansion of Byar's hierarchy as published by Piantadosi.
1997Evans et al.J clin oncolEvans, W. K., et al. (1997) “Lung cancer practice guidelines: lessons learned and issues addressed by the Ontario Lung Cancer Disease Site Group”. J Clin Oncol, 15(9), 3049-3059
1997GrayGray, J.A. Muir. (1997) Evidence-Based Healthcare. NYC: Churchill-Livingstone
1997GreenhalghBMJGreenhalgh, T. (1997) “How to read a paper. Getting your bearings (deciding what the paper is about)”. BMJ, 315(7102), 243-246
1997/2001GreenhalghBMJGreenhalgh, Trisha. (1997) How to read a paper : the basics of evidence based medicine. London: BMJ Pub. Group ~~ Greenhalgh uses nearly-identical hierarchies in her book as in her paper, but to quite different effect.
1997/2000Van Tulder et al.Spinevan Tulder, M., et al. (2000) “Exercise therapy for low back pain: a systematic review within the framework of the cochrane collaboration back review group”. Spine (Phila Pa 1976), 25(21), 2784-2796
1998ClarkeBMJClarke, M.J. (1998) ‘Ovarian ablation in breast cancer, 1896 to 1998: milestones along hierarchy of evidence from case report to Cochrane Review’ BMJ, 317 (7167): 1246-8
1998Edwards Russell & StottFam PractEdwards, A. G., Russell, I. T. & Stott, N. C. (1998) “Signal versus noise in the evidence base for medicine: an alternative to hierarchies of evidence?”. Fam Pract, 15(4), 319-322
1998Phillips et al.CEBMPhillips, R. et al. (1998) “Levels of Evidence”. Centre for Evidence-Based Medicine, www.cebm.net. ~~ no longer easily available online - pdf stored"
1999Chesson et al.SleepChesson, A. L., Jr., et al. (1999) “Practice parameters for the treatment of restless legs syndrome and periodic limb movement disorder. An American Academy of Sleep Medicine Report. Standards of Practice Committee of the American Academy of Sleep Medicine”. Sleep, 22(7), 961-968
1999Hoogendoorn et al.Scand J Work Environ HealthHoogendoorn, W. E., et al. (1999) “Physical load during work and leisure time as risk factors for back pain”. Scand J Work Environ Health, 25(5), 387-403
1999McAlister et al.JAMAMcAlister, F. A., et al. (1999) “Users' Guides to the Medical Literature: XIX. Applying clinical trial results B. Guidelines for determining whether a drug is exerting (more than) a class effect”. JAMA, 282(14), 1371-1377
1999Australian NHMRCANHMRCAustralian National Health and Medical Research Council (ANHMRC). (1999) A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines. Commonwealth of Australia: available at: http://www.health.gov.au/nhmrc/publicat/synopses/cp30syn.html"
1999Shekelle et al.BMJShekelle, P. G., et al. (1999) “Clinical guidelines: developing guidelines”. BMJ, 318(7183), 593-596
2000Briss et al.AM J Prev MedBriss, P. A., et al. (2000) “Developing an evidence-based Guide to Community Preventive Services--methods. The Task Force on Community Preventive Services”. Am J Prev Med, 18(1 Suppl), 35-43
2000Guyatt et al.JAMAGuyatt, G. H., et al. (2000) “Users' Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users' Guides to patient care. Evidence-Based Medicine Working Group”. JAMA, 284(10), 1290-1296
2000Roman Silberzweig & SiuEff Clin PractRoman, S. H., Silberzweig, S. B. & Siu, A. L. (2000) “Grading the evidence for diabetes performance measures”. Eff Clin Pract, 3(2), 85-91
2000Sackett et al.EBMSackett, David L. (2000) Evidence-based medicine: how to practice and teach EBM (2nd ed.). Edinburgh: Churchill Livingstone
2001Harbour & MillerBMJHarbour, R. & Miller, J. (2001) “A new system for grading recommendations in evidence based guidelines”. BMJ, 323(7308), 334-336
2001Harris et al.Preventive Services Task ForceHarris, R. P., et al. (2001) “Current methods of the US Preventive Services Task Force: a review of the process”. Am J Prev Med, 20(3 Suppl), 21-35
2001HaynesACP J ClubHaynes, R. B. (2001) “Of studies, syntheses, synopses, and systems: the 4S evolution of services for finding current best evidence”. ACP J Club, 134(2), A11-13
2001JonasWJMJonas, W.B. (2001) ‘The evidence house: How to build an inclusive base for complementary medicine’, in Western Journal of Medicine, 175: (2) 79-80
2001Khan Dinnes & KleijnenEJ Obst Gyn Rep BioKhan, Khalid S, Dinnes, Jacqueline & Kleijnen, Jos. (2001) “Systematic reviews to evaluate diagnostic tests”. European Journal of Obstetrics & Gynecology and Reproductive Biology, 95(1), 6-11
2001/2008Guyatt et al.ChestGuyatt, G., et al. (2001) “Grades of recommendation for antithrombotic agents”. Chest, 119(1 Suppl), 3S-7S
2002Ball & PhillipsEBonCallBall, C.M. & Phillips, R. (2002). “Levels of Evidence - March 2002”, Evidence Based On Call. Retrieved from http://www.eboncall.org/content/levels.html; No longer available online. Superseded by Howick et al. 2009.
2002Guyatt et al.UGMLGuyatt, G.H. et al. (2002) Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice: American Medical Association Press ~~ Uses the Guyatt et al. 2000 hierarchy but with different interpretation and discussion
2003ICSIICSIInstitute for Clinical Systems Improvement. (2003) Evidence Grading System. available at: http://www.icsi.org ~~ No longer available online. ICSI adopted GRADE in 2011."
2003EvansJ Clin NursEvans, D. (2003) “Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions”. J Clin Nurs, 12(1), 77-84
2003Mason & EcclesGREGMason, J. & Eccles, M.,. (2003) Guideline Recommendation and Evidence Grading (GREG). Newcastle, UK: University of Newcastle Centre for Health Services Research
2003Devereaux & YusufJ Int MedDevereaux, PJ & Yusuf, S. (2003) “The evolution of the randomized controlled trial and its role in evidence‐based decision making”. Journal of internal medicine, 254(2), 105-113.
2004Ebell et al.SORTEbell, M. H., et al. (2004) “Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature”. Am Fam Physician, 69(3), 548-556
2004Edlund et al.AANEdlund, W. et al. (2004) Clinical Practice Guideline Process Manual, Appendix 9. American Academy of Neurology: available at: www.aan.com"
2004GRADE Working GroupGRADEGRADE Working Group. (2004) “Grading quality of evidence and strength of recommendations”. British Medical Journal, 328, 1490
2004LangMcGill J MedLang, E. (2004) “The Why and How of Evidence-Based Medicine”. McGill Journal of Medicine, 8, 90-94
2004AnonymousThoraxAnonymous (2004) ‘Hierarchy of Evidence and Grading of Recommendations’, in Thorax, 59: 13-4
2004Wagoner et al.SUNYWagoner, B. et al. (2004) “Guide to Research Methods: The Evidence Pyramid”. SUNY Downstate Medical Center EBM Tutorial.
2004NICENICENational Institute for Health and Clinical Excellence (NICE). (2004) NICE: Guideline Development Methods (Vol. Vol.7: Reviewing and Grading the Evidence) ~~ No longer easily accessible online.
2005Melnyk & Fineout-OverholtMelnyk, B.M. & Fineout-Overholt, E.,. (2005) Evidence-Based Practice in Nursing and Healthcare: A Guide to Best Practice. Philadelphia: Lippincoot, Williams & Wilkins
2005HaynesACP J ClubHaynes, R Brian. (2005) “Of studies, syntheses, synopses, summaries, and systems: the 5S evolution of information services for evidence-based health care decisions”. ACP J Club, 145(3), A8-A8
2005Weightman et al.NICEWeightman, Alison, et al. (2005) Grading evidence and recommendations for public health interventions: developing and piloting a framework: Health Development Agency London, UK
2006Bhandari & GiannoudisInjuryBhandari, M. & Giannoudis, P. V. (2006) “Evidence-based medicine: what it is and what it is not”. Injury, 37(4), 302-306
2006NICENICENational Institute for Health and Clinical Excellence (NICE). (2006) Hypertension: Management of Hypertension in Adults in Primary Care, NICE Clinical Guideline 34
2006Fuster et al.EuropaceFuster, Valentin, et al. (2006) “ACC/AHA/ESC 2006 guidelines for the management of patients with atrial fibrillation: full text A report of the American College of Cardiology/American Heart Association Task Force on practice guidelines and the European Society of Cardiology Committee for Practice Guidelines (Writing Committee to Revise the 2001 Guidelines for the Management of Patients With Atrial Fibrillation) Developed in collaboration with the European Heart Rhythm Association and the Heart Rhythm Society”. Europace, 8(9), 651-745
2006Guyatt et al.ChestGuyatt, G.H. et al. (2006) “Grading Strength of Recommendations and Quality of Evidence in Clinical Guidelines”. Chest, 129, 174-178
2006Petrisor Keating & SchmetishInjuryPetrisor, B. A., Keating, J. & Schemitsch, E. (2006) “Grading the evidence: levels of evidence and grades of recommendation”. Injury, 37(4), 321-327
2006Schunemann Fretheim & OxmanWHOSchunemann, H. J., Fretheim, A. & Oxman, A. D. (2006) “Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations”. Health Res Policy Syst, 4, 21
2006Schunemann et al.ATSSchunemann, H. J., et al. (2006) “An official ATS statement: grading the quality of evidence and strength of recommendations in ATS guidelines and recommendations”. Am J Respir Crit Care Med, 174(5), 605-614
2007Daly et al.JCEDaly, J., et al. (2007) “A hierarchy of evidence for assessing qualitative health research”. J Clin Epidemiol, 60(1), 43-49
2007Petrisor & BhandariIndian J OrthopPetrisor, B. & Bhandari, M. (2007) “The hierarchy of evidence: Levels and grades of recommendation”. Indian J Orthop, 41(1), 11-15
2007Virgilio Chiapi & PalmarozziJ Am Ost AssVirgilio, R.F., Chiapa, A.L. & Palmarozzi, E.A. (2007) ‘Evidence-Based Medicine, Part 1. An introduction to Creating an Answerable Question and Searching the Evidence’, in Journal of the American Osteopathy Association, 107:295-7
2008Guyatt et al.UGMLGuyatt, G.H. et al. (2008) ‘How to use a patient management recommendation’ in UGML, pp.595-618
2008Guyatt et al.ChestGuyatt, G. H., et al. (2008) “Grades of recommendation for antithrombotic agents: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition)”. Chest, 133(6 Suppl), 123S-131S
2008Ho Peterson & MasoudiCirculationHo, P. M., Peterson, P. N. & Masoudi, F. A. (2008) “Evaluating the evidence: is there a rigid hierarchy?”. Circulation, 118(16), 1675-1684
2008SIGN 50SIGNScottish Intercollegiate Guidelines Network (SIGN). (2008) SIGN 50: A Guideline Developer's Handbook. available at: www.sign.ac.uk"
2008Sullivan & CambronNUHSSullivan, B.M. & Cambron, J.A. (2008) ‘Overview of Study Designs in Clinical Research’ [Presentation] ([email protected] CH5 Study Design)
2008Tang & BeagleholeJ Epid Comm HealthTang, KC, Choi, BCK & Beaglehole, R. (2008) “Grading of evidence of the effectiveness of health promotion interventions”. J Epidemiol Community Health, 62(9), 832-834
2008US Preventive Services Task ForceUSPSTFU.S. Preventive Services Task Force. (2008) U.S. Preventive Services Procedure Manual. AHRQ Publication No. 08-05118-EF
2009DiCenso Bayley & HaynesAnn Int MedDiCenso, A., Bayley, L. & Haynes, R. B. (2009) “ACP Journal Club. Editorial: Accessing preappraised evidence: fine-tuning the 5S model into a 6S model”. Ann Intern Med, 151(6), JC3-2, JC3-3
2009Howick et al.CEBMHowick, J. et al. (2009) “CEBM Levels of Evidence”. Centre for Evidence-Based Medicine, www.cebm.org"
2009Merlin Weston & TooherBMCMerlin, T., Weston, A. & Tooher, R. (2009) “Extending an evidence hierarchy to include topics other than treatment: revising the Australian 'levels of evidence'”. BMC Med Res Methodol, 9, 34
2010Marchevsky & WickAm J Clin PatholMarchevsky, A. M. & Wick, M. R. (2010) “Evidence levels for publications in pathology and laboratory medicine”. Am J Clin Pathol, 133(3), 366-367
2010Niederman & RichardsJ Am Coll DentNiederman, R. & Richards, D. (2010) “The challenge, access, risks, and deficits of evidence-based dentistry”. J Am Coll Dent, 77(4), 20-24
2010Swanson Schmicz & ChungPlast Reconstr SurgSwanson, J.A., Schmicz, D. & Chung, K.C.,. (2010) “How to practice evidence based medicine”. Plast Reconstr Surg, 126, 286
2010TidyTidy, C. (2010) “Different Levels of Evidence” Retrieved 06, 2012, from htttp://www.patient.co.uk/doctor/Different-Levels-of-Evidence-%28Critical-Reading%29.htm ~~ Tidy offers a number of different hierarchies here, some old, some novel.
2011Balshem et al.GRADEBalshem, H., et al. (2011) “GRADE guidelines: 3. Rating the quality of evidence”. J Clin Epidemiol, 64(4), 401-406
2011NICENICENational Institute for Health and Clinical Excellence (NICE). (2011) CG117: Tuberculosis: clinical diagnosis and management of tuberculosis, and measures for its prevention and control. London: RCP
2011EavesAesthet Surg JEaves, F. F., 3rd. (2011) “Got evidence? Stem cells, bias, and the level of evidence ladder: commentary on: ASAPS/ASPS position statement on stem cells and fat grafting”. Aesthet Surg J, 31(6), 718-722
2011Glover et al.EBM Page GeneratorGlover, J. et al. (2011) “EBM Page Generator”. www.ebmpyramid.org ~~ One of the most replicated images of evidence hierarchies - this is the earliest source I have been able to locate."
2011Gronsett Woodroffe & GetchiusAANGronseth, G.S., Woodroffe, L.M & Getchius, T.S. (2011) Clinical Practice Guideline Process Manual. AAN: available at: http://tools.aan.com/globals/axon/assets/9023.pdf.
2011TurnerEBVMATurner, S. (2011) ‘Hierarchy of Evidence’ in EBVMA, available at: ebvma.org/home/content/part-1-hierarchy-evidence
2011Guyatt et al.GRADEGuyatt, G.H. et al. (2011) “GRADE Guidelines: 1. Introduction--GRADE evidence profiles and summary of findings tables”. Journal of Clinical Epidemiology, 64, 383-394
2011Howick et al.CEBMHowick, J., et al. (2011) “The Oxford 2011 Levels of Evidence”. Oxford Centre for Evidence-Based Medicine, available at www.cebm.net"
2011Khan et al.CRCKhan, Khalid. (2011) Systematic reviews to support evidence based medicine: How to review and apply findings of healthcare research: CRC Press, p.19
2012Baral et al.HASTEBaral, Stefan D, et al. (2012) “The highest attainable standard of evidence (HASTE) for HIV/AIDS interventions: toward a public health approach to defining evidence”. Public Health Reports, 127(6), 572
2012HaighGroup AnalysisHaigh, R. (2012) ‘Being Economical with the Evidence’, in Group Analysis, 45: 70-83
2013Goldet & HowickJEBMGoldet, Gabrielle & Howick, Jeremy. (2013) “Understanding GRADE: an introduction”. Journal of Evidence‐Based Medicine, 6(1), 50-54
2014Canadian Task ForceCTFPHECanadian Task Force on the Periodic Health Examination. (2014) Procedure Manual. http://canadiantaskforce.ca/methods/ ~~ endorses GRADE but also offers an additional novel hierarchy."