A King’s College London research spotlight yesterday (Wanis & Davies, 2021) highlighted Hiba Wanis and colleagues’ work on the relationship between one-year survival rates and ethnicity in patients with brain tumours in England. Wanis, a PhD candidate at King’s, analysed data from 24,319 English adults who were diagnosed with a malignant brain tumour in the years 2012-2017, pulling data from the National Disease Registration Service. The work is yet to be published, but was presented at the National Cancer Research Institute festival, according to the Guardian’s coverage (Gregory, 2021). The findings were widely reported across the British press.
The headline-grabbing figure is that the white British subgroup of these patients had the highest rate of deaths within one year of diagnosis, across the ethnicity categories that Wanis et al. considered. The difference was startling enough to garner significant coverage. The starkest comparison is between white Britons and the “other ethnic” category, with King’s College reporting that “other ethnic” patients were 30% less likely to die within a year than white Brits. They also note differences between white British patients and Indian patients (16%), “other white” – i.e. non-British – ethnicity (17%) and unknown ethnicity (19%) (Wanis & Davies, 2021).
Outcomes research like this is, as I have regularly argued, highly valuable, and should receive more funding, attention and priority in medical reasoning and theorising (see e.g. Blunt 2018a, 2018b). It is systematically ignored and downgraded in the approach to clinical care developed by the Evidence-Based Medicine movement, despite often representing some of the most informative evidence we have in both making therapeutic decisions and monitoring clinical care impacts.
Yet, we must be attentive to design choices in outcomes research, as in the appraisal of any medical evidence. While we cannot yet read the full details of the research, the teaser of the analysis provided so far offers an intriguing example of the phenomenon of inconsistent lumping and splitting within medical research. Analysing data according to subgroup outcomes has received much negative attention, with concerns raised about data mining, small sample sizes, and over-eager attribution of causation. There are many ways to ward against these challenges. But a first principle must surely be to construct meaningful subgroups which are informed by clinical reasoning, existing evidence and biological rationale. Unfortunately, here, the ethnicity categories deployed are informed more by the vagaries of the NHS Digital system of ethnicity classification than any meaningful attempt to track culturally or biologically relevant aspects of ethnicity.
We see this foremost in three of the categories which showed reported differences in one-year survival rates compared to white British patients: “other ethnic”, “other white” and “unknown”. In philosophy, we evaluate a distinction – a system to categorise some domain into two or more groups – according to whether the categories are exhaustive and mutually exclusive. Exhaustiveness requires that everything in the domain is assigned to at least one category – nothing is left out entirely. This is particularly important in philosophical argumentation, because arguments based on inexhaustive distinctions are prone to presenting false dichotomies, and significantly limit the prospects for generalisation. In this case, an exhaustive categorisation of patients by ethnicity must assure us that every patient was ascribed to at least one ethnicity category. Mutually exclusive distinctions ensure that everything in the domain is ascribed to at most one category. Nothing can fall into multiple categories. Again, this is important in partitioning our domain and allowing for meaningful discussion according to category membership, as well as preventing an element of the domain being counted multiple times in the final analysis. A proper distinction is both exhaustive and mutually exclusive, i.e. everything in the domain is categorised into one and only one category. In this case, the researchers’ ethnicity distinction is proper insofar as it ascribes one and only one ethnicity to every adult brain cancer patient in England.
There is a quick and dirty way to ensure that a distinction is exhaustive: the use of one or more “catch-all” categories, which deliberately include just those elements of the domain which do not fit into our intensionally defined groups. An “other” category is a classic catch-all technique. However, catch-alls are very limiting. We can rarely say anything meaningful about the consequences of membership in a catch-all category, or construct coherent arguments drawing on catch-all membership as a starting point, because the catch-all category is – almost by definition – vulnerable to extreme heterogeneity. Nothing binds together the items in this category beyond their non-adherence to the other categories.
This problem is compounded, though, when the needed for mutual exclusivity prompts us to push any element in the domain which might fit into multiple categories out into a catch-all, too. That has clearly happened here: many patients might well fit into multiple ethnicity categories. The “other ethnic” catch-all has been constructed to include “all mixed ethnic groups and any other ethnic groups” (Wanis & Davies, 2021). “Other ethnic”, then, may well include patients who identify as white British, as Bangladeshi, as Indian – as members of several of the ethnicity categories. But the categorisation cannot admit of multiple group membership, leading to “mixed ethnic groups” being coalesced together and combined with any ethnic groups insufficiently large or prominent to get their own label. Similarly, the “unknown” category, which also showed a higher one-year survival rate than the white British groups, will include patients whose ethnicity is actually represented by one or more of the other categories. A person can be white British and that ethnicity can be unknown to a researcher, so this distinction lacks mutual exclusivity in theory – but is mutually exclusive in practice.
What can we say of the “other ethnic” and “unknown” groups? Little of any substance. Wanis is admirably cautious in giving any interpretation to their findings, telling King’s College’s research spotlight team that: “It is probably too early to speculate on what may lie behind these differences, but a number of factors may be involved. These include how early people ask their doctors about symptoms, how early in the disease a diagnosis is made, better reporting, lifestyle and cultural factors, deprivation, tumour characteristics and behaviour, and treatment options.” However, the stage at which people consult doctors about symptoms, stage of diagnosis, reporting behaviour, lifestyle and cultural factors, deprivation, tumour characteristics and treatment options in the catch-all categories will likely exhibit a full range of diversity. Any analysis of these factors within these subgroups can at best produce a misleading and reductive average, because of the lack of a meaningful property which binds the group together and could account for similarities in any of these dimensions. We should, then, likely disregard the findings with respect to “other ethnic”, “unknown” and potentially “other white” patients. The catch-all category does its job in ensuring that each patient counts once and only once for subgroup analysis by ethnicity, but cannot be a basis for meaningful attribution of difference.
The only relevant comparison that remains, then, is the 16% disparity between white British and Indian patients’ one-year survival – incidentally, the smallest of the four reported gaps. Here, we do need to attend carefully to the problems in subgroup analyses. When making multiple comparisons, we expect that some may appear significant by chance, especially where the numbers involved in a particular subgroup are small. Indeed, of the six subgroups for which Wanis & Davies (2021) reveals the numbers involved, Indian origin is the third smallest group, with 166 Indian patients dying of a brain tumour in the period studied. This compares to 13,339 white British patients who died in the same period. 52% of Indian brain tumour patients died, putting the number of Indian patients in the sample at an estimated 252, compared to 64% of white British patients dying, putting the number of white British patients in the sample at roughly 21,876. That’s 90% of the recorded cases, being compared to a little over 1% of the cases.
Generally, the larger the sample, the more accurate the one-year survival rate to the “true” propensity for survival in that group. We might expect, then, that the one-year survival rates in the white British patients would represent a much more accurate picture of the survival rate for such patients than does the survival rate found in Indian patients. The play of chance has a stronger pull on the Indian figures than the white British. Of course, there are well-researched statistical techniques to account for the imbalance in the sample size, and for multiple comparisons being made. Nonetheless, one does suspect that when none of the other meaningful ethnic groups met showed statistically significant disparity (the researchers also investigated Bangladeshi, Pakistani, Chinese, black African and black Caribbean ethnicity groups), the result is more likely a noisy outlier than an important signal. We will have to await the full publication to explore this further and unpack the methodology used.
But there is one more respect in which the decisions about lumping and splitting, how to divide up the domain, conflict rather than conform with the underlying medical realities. “Brain tumour” is, as Wanis is well aware (see Wanis et al. 2021) not a particularly meaningful category in itself. There are many different brain cancers, which exhibit a vast diversity of behaviour and characteristics. Indeed, given the gulf in severity and survival between the two most prevalent brain tumours alone – the utterly devastating and dismal glioblastoma (31.78% of brain tumour diagnoses in England), and the primarily benign meningioma (27.3% of brain tumour diagnoses in England, more than two thirds of which were benign) (Wanis et al. 2021), it is likely that the biggest factor predicting one-year survival in the rather artificial “brain tumour” patient is the form of tumour. Glioblastoma remains a dismal disease: fast-acting, highly recurrent, and with very limited treatment options. Even within the category of glioblastoma, there are at least two different subtypes of interest, which determine the likelihood of responsiveness to chemotherapy (Hegi et al., 2005), as I have previously analysed in detail (Blunt, 2019).
But the headline figures reported here relate only to one-year survival given brain tumour diagnosis. One-year survival is a commonly used metric for glioblastoma (though we might consider median survival a more valuable one) because of the aggression of the disease. But one-year survival rates are hardly likely to capture any effect on other forms of malignant brain cancer which operate more slowly or are more treatable. Really, we should expect the data presented by Wanis and colleagues to represent only the variation in glioblastoma (and other highly aggressive but far less prevalent brain tumours) survival rates by ethnicity. But in that case, the presence of the other tumour types seriously risks muddying the analysis, introducing far more noise than is necessary into the picture. Indeed, a valuable line of inquiry would be to ask whether there is ethnicity-based variability in the rates of different forms of brain tumour. If we were to accept that white Britons have a higher one-year death rate than other ethnicities, we might well suspect them to be more afflicted by glioblastoma than other patients.
Equally, though, such a result would be explained if white Britons were less afflicted by the less aggressive tumours than other patients. If white Britons experienced fewer than average meningiomas while experiencing an average rate of glioblastomas, we would expect to see their one-year death rates, when expressed as a percentage as they are here, be higher than those in other ethnicities, entirely as an artefact of the lower rate of meningiomas. While the media has treated this story as bad news for white Britons, there is an equally plausible explanation which is good news for the white Britons – a lower risk of meningioma with no higher risk of glioblastoma.
Of course, Wanis and colleagues have both the data and the clear intent to perform much of this analysis from the registry data they are scrutinising here, and we would indeed expect that the full published version of this work will disaggregate according to tumour type and reveal much more relevant information by doing so. But we can visualise an important lesson from this work: when splitting patients into subgroups, the biomedical realities must determine the order in which the splits are made. If we split by ethnicity, but lump by tumour type, the results will not be meaningful. If we first split by tumour type, and then look for ethnicity effects, the sample sizes will be vastly reduced and so will the likelihood of detecting a signal if one is there to be found. But at least we will stand a chance of identifying important patterns, and not setting off a comparatively misdirected media scrum.
Blunt, C.J. (2018a) The Avoidable Scandal: Benoxaprofen and Theories of Medical Evidence, available at: http://cjblunt.com/the-avoidable-scandal-benoxaprofen-and-theories-of-medical-evidence/
Blunt, C.J. (2018b) The Paracute Problem: Extracorporeal Life Support and the Demand for Trials, available at: http://cjblunt.com/the-parachute-problem-extracorporeal-life-support-and-the-demand-for-trials/
Blunt, C.J. (2019) The Dismal Disease: Temozolomide and the Interaction of Evidence, available at: http://cjblunt.com/the-dismal-disease/
Gregory, A. (2021) ‘White British brain tumour patients ‘more likely to die in a year”, The Guardian, 9 Nov 2021, available at: https://www.theguardian.com/science/2021/nov/09/white-british-brain-tumour-patients-more-likely-to-die-in-a-year
Hegi, M.E. et al. (2005) ‘MGMT Gene Silencing and Benefit from Temozolomide in Glioblastoma’, New England Journal of Medicine 352(10): 997–1003.
Wanis, H.A. & Davies, E.A. (2021) ‘Ethnicity could play a role in surviving brain tumours, study finds’, King’s College London: News Center – Spotlight on Research, available at: https://www.kcl.ac.uk/news/does-ethnicity-play-a-role-in-survival-from-brain-tumours
Wanis, H.A., Moller, H., Ashkan, K., & Davies, E.A. (2021) ‘The incidence of major subtypes of primary brain tumors in adults in England 1995-2017’, Neuro-Oncology, 23(8): 1371-82.
Last revision: 11/02/2022