In a study of 2 163 456 respondents from 8267 English primary care practices we show that case-mix adjustment of practice-level scores results in relatively few large adjustments (which were mainly positive), and many small adjustments (which were more often negative). However, the largest effects were on a distinct subset of practices whose patients were more likely than average to be from South Asian or other ethnic minorities, young, in poorer health, and living in deprived areas.
Although only a small number of practices would benefit significantly from case-mix adjustment, we propose that such adjustment should be applied because it meaningfully improves performance measurement for practices with less typical and often under-privileged patient populations, This would discourage practices from ‘cream-skimming’ by avoiding enrolling patients who could be seen as ‘hard to treat’, and increase perceptions of fairness and engagement in quality improvement activities.
In a study of 27 practices in the UK (patient race/ethnicity 97% white), Salisbury and colleagues conclude that ‘adjusting for patients' characteristics makes very little difference to practices' scores or to the performance of individual practices relative to other practices’.
6 Our study, with a larger sample of practices and including a broader mix of patient characteristics (practice N=8267; patient race/ethnicity 14% non-white), suggests that case-mix adjustment has a non-trivial impact on the assessment of performance for practices serving less typical, and often disadvantaged, patient populations. Unfairly disadvantaging such practices in performance measurement could have negative implications for retention and recruitment of healthcare staff working in them, and for attracting and retaining patients.
Our study builds on what is known from previous research by addressing some of the limitations of previous studies, which include smaller sample sizes, non-random selection of practices, and a low proportion of non-white respondents.
6 A particular strength of our study is the large sample size, enabling us to investigate the impact of case-mix adjustment on practices serving less typical patient populations. Because of the population basis of the sampling, our study is also not subject to problems such as regional variation in coding which may cause problems when using diagnostic coding for case-mix adjustment.
27Although parsimonious models are often desirable, one limitation of this study is the relatively small number (five) of case-mix adjustors included. While age, gender, ethnicity, self-reported health, and socio-economic deprivation are considered pertinent for case-mix adjustment in both the UK and the USA there are additional patient characteristics, such as language spoken at home, that have been used as case-mix adjustors in prior research
5 but were not measured in this study. Another limitation is the modest response rate to the survey (38%). However in our previous analysis of two questions associated with payment to practices we found minimal evidence of non-response bias,
23 and this is consistent with a meta-analysis of survey methodology literature showing that response rates are only weakly associated with non-response bias among studies employing methodology similar to ours.
28 There are limits to the generalisability of our findings. The impact of individual case-mix adjustors may vary between countries due to differences in, for example, the constituent racial/ethnic groups in each nation, and we were not able in our study to explore variation across nations or healthcare settings.
Implications for health policy and practice
Because case-mix adjustment reduces bias
4
5 and improves the validity of performance measurement (especially for some practices) it is integral to supporting patient choice and facilitating quality improvement in hospitals and in primary care.
5 In addition, by improving
perceived fairness, or face validity,
29 case-mix adjustment helps to maintain the credibility of pay-for-performance schemes, and focuses conversations on how to improve patient experiences
4 by avoiding arguments from providers that ‘my population is special’.
29 However, care must still be taken to communicate that negative adjustments are not ‘penalties’, and we recognise, like others,
2
30 that there is a risk that case-mix adjustment could remove incentives for providers to address disparities in care and institutionalise substandard care by ‘masking’ poor care provided to some patient subgroups. In order to make visible any disparities in the provision of care by socio-economic status or race/ethnicity and to minimise the risk of institutionalising substandard care, it is important that case-mix adjustment is used in conjunction with strategies that collect data to monitor healthcare disparities and report quality measures stratified by, for example, socio-economic position and race/ethnicity.
30Deciding when and how to adjust performance scores for case mix
Case-mix adjustment is most useful when patient characteristics vary substantially between providers, and where these patient characteristics are strongly related to performance measures.
5 Combined with previous US findings,
5
7 our data suggest that age and health status are broadly important adjustors across nations and healthcare settings. Our research also suggests that adjustment for race/ethnicity may be important to ensure equitable comparison, and, when results are linked to financial incentives, equitable distribution of resources. Socio-economic status/deprivation was less important in these data than is often the case in the USA; these differences may reflect differences in healthcare systems or in the measurement of socio-economic status.