In an era of continued health service modernisation, measuring the quality of care has become a major concern for funders and providers.8
Performance indicators are commonly used as measures of quality and efficiency within and between healthcare systems.
A number of high volume individuals and institutions have been shown to have better outcomes across a wide range of procedures and conditions.2
Halm et al
systematically reviewed volume versus outcome studies in the medical literature and found that 71% of all studies of hospital volume and 69% of studies of physician volume reported a statistically significant association between higher volume and better health outcome. This association was found to be in several disparate conditions, including pancreatic and oesophageal cancer, abdominal aortic aneurysms, paediatric cardiac problems, and in the treatment of AIDS.
The majority of the studies that compare health care performances are retrospective. The use of this type of data has the advantage of being readily available and provides information about large numbers of patients with different conditions. There are, however, potential pitfalls that can hamper the analysis and conclusions of these studies.5
One of the most important factors that needs to be taken into consideration to ensure reliable interpretation of the results is the issue of case mix. It is essential to try to ensure that data are derived from similar patient groups with similar clinical characteristics—that is, comparing like with like. Case mix is generally established by estimating the relative frequency of various types of patients seen by the provider in question during a given time.
Our previous study suggested that high volume surgeons tended to have lower complication rates when compared to low volume surgeons in the same unit.1
There are many reasons why this may occur but one of the most important to try to exclude is that some or all of these differences are explained by variations in difficulty between cases. If high volume surgeons were picking potentially easier cases to operate on (and are therefore more likely to complete their large lists), they would appear to have lower complication rates. Similarly, if low volume surgeons were doing potentially more difficult cases, they would require more time to do them and therefore have smaller lists. There is also a converse situation to consider. It has been suggested that higher volume surgeons do more potentially complicated cases (presumably because of a perception that they have more experience of them) despite the large numbers on their lists.
Our results do suggest that there is a difference in case mix between the high and low volume surgeons. Although these differences are not great in that there was a similar median PDS in both groups and the spread of individual risk factors was also even, they do suggest that in the sample chosen, the low volume surgeons were operating on potentially more difficult cases. This result may therefore explain, at least in part, the trend we previously demonstrated of lower complication rates for higher volume surgeons.
These results are consistent with a number of previous studies that have shown that subtle differences in disease severity or co-morbidity may partly explain the associations between volume and outcome. Halm et al
noted, in their review, that studies performing sophisticated risk adjustment using clinical data were less likely to report a positive effect of hospital volume on outcome than were studies that did not adjust for risk factors.2
Similarly, a review of seven published studies demonstrating reduced mortality rates with increased volume of coronary artery bypass graft surgery found a decrease in the relation as the degree of risk adjustment increased.9
In a comparative study evaluating cataract surgery outcomes, Willerscheidt and coworkers noted a variation in visual acuity results by surgeon and surgical volume, yet the relation was not significant when adjusted for the patient mix factor.3
Once again, care must be taken when interpreting volume outcome studies—especially at the level of individual surgeon. As a patient, choosing a high volume hospital or surgeon does not necessarily mean complications are less likely.
In an attempt to evaluate the predictive power of our newly designed scoring system, retrospective analysis of a sample of 100 cases revealed a difference in the average scoring of complicated cases verses non-complicated cases with higher PDS score associated with higher complication rates. This suggests that the PDS can appropriately predict the extent of possible surgical difficulty in that these cases are more likely to have complications. The results of this validation help to justify its comparative use in evaluating case mix.
We have not assessed other variables that could contribute to the final outcome such as surgeon’s experience or theatre environment. A prospective validation study is thus currently under way to assess the impact of other variables on the final surgical outcome.
There are certain potential biases in our study. The data extracted from the preoperative notes were variable, as the pre-assessment notes were recorded by ophthalmologists of different grades and experience. There were also differences in documented clinical details within the notes, although this bias was reduced by the fact that this would be equally true for both high and low volume surgeons patients. The operating surgeons themselves only saw a proportion of the patients preoperatively so their potential individual bias in clinical assessment and notes writing in the study was reduced.
Because of our retrospective validation and the fact that the PDS scores were comparative, we suggest our scoring system itself did not introduce significant bias into the results.
In conclusion, our study re-emphasises the importance of case mix adjustment in comparative assessment of healthcare quality. It is important not to overestimate the magnitude of the relation between higher volume and better outcome unless case mix adjustments have been made. For these results to be truly valid, the study needs to be repeated in different units with different patient populations.