I spend a lot of my time talking to healthcare practitioners of all kinds about evidence, formally and informally. I’m mostly focused on getting the word out that RCT results are not the only thing that matters in treatment recommendations, that hierarchies of evidence are pretty weak ways to evaluate evidence quality, and that a scientific approach to medicine doesn’t stop with study results but rather consists of systematically attempting to explain all of your data, then make and test predictions from those explanations.
In a lot of rooms, I get two diametrically opposed reactions to these arguments, which fall into two camps:
1: Obviously correct but not significant
- Philosophical assumptions and frameworks in medicine are tools, heuristics. They don’t actually matter.
- Tools like hierarchies are not meant to be taken seriously, and no one actually takes them all that seriously.
- Practitioners actually have a much more nuanced engagement with evidence than surface-level talk would suggest.
2: Obviously wrong and morally suspect
- These assumptions and frameworks are fundamental to the successful operation of medicine.
- The criticisms I level at these fundamental pillars of medicine pave the way for alternative therapies and authority-based medicine.
- Criticisms are anti-scientific and show a disrespect for the power of evidence, and open us up to methodological anarchism.
I often get both of these reactions from several people in a room, and while these are the most extreme versions of the reaction, they’re all things that’ve actually been said. My Q&A can get a bit fiery.
The advantage of getting these two same reactions in the same room is that they’re essentially mutually refuting. The existence of the person telling me that the hierarchy is just a fundamental fact about the nature of evidence refutes the person laughing because I’ve taken hierarchies so seriously when everyone knows they’re just a simplified tool. The person mocking anyone who’d take a hierarchy seriously should rebuff the person who thinks any doubt cast on hierarchies will undermine the scientific status of medicine.
I’ve developed two strategies to try to limit this polarised response, as it doesn’t really get me anywhere beyond a good laugh in the Q&A when audiences realise what’s happened. My responses are:
- To always include recent tweets and journal article titles specifically from the field of the audience members which show how seriously at least some people are taking hierarchies and show these assumptions being made in practice.
- To always develop my argument by walking through a significant case and draw out the methodological lessons on the way, rather than going straight in with the philosophy. That’s why this site is primarily peopled with case studies not with abstract philosophical arguments.
- To avoid philosophical language that might obscure the central point wherever possible.