|Home | About | Journals | Submit | Contact Us | Français|
Is improving but their contribution to improving quality of care remains unclear
Modern health care involves the routine collection of administrative data primarily for management and accounting purposes. Such databases include some clinical data (such as type of surgery, diagnosis, length of stay) that might be useful in monitoring quality of care.1 2 3 4 In this week's BMJ, Aylin and colleagues5 have used hospital episode statistics (HES) data, which are routinely collected by the UK National Health Service, to develop statistical models for predicting hospital mortality adjusted for case mix in three well defined clinical areas—cardiac surgery, aortic aneurysm repair, and colorectal cancer.
Previous comparisons of administrative databases and clinical databases or medical notes (chart review) have found administrative databases to be lacking in three important ways—scope (the relevant data not available), data quality, and ability to adjust for factors relating to patient case mix.1 2 3 4 5 6 This has led to the credibility of administrative databases being questioned, but as a result of several high profile events, this view of HES data may be changing.
In 2001, Dr Foster7 used HES data to produce standardised mortality ratios adjusted for case mix using methodology proposed by Jarman and colleagues.8 The methodology was later adopted by the Institute for Healthcare Improvement in the United States in its drive to reduce hospital mortality.9 In 2002, the inquiry into the high death rates after paediatric cardiac surgery in Bristol used HES (as well as a clinical database) to show that Bristol was a statistical outlier.10 The inquiry report stated that HES “was [sic] not recognised as a valuable tool for analysing the performance of hospitals. It is now, belatedly.” Furthermore, the inquiry also remarked that the “dual” system (HES and the clinical database) of collecting data in the health service was “wasteful and anachronistic.”11 In 2004, Harley and colleagues12 also used HES data retrospectively to show that Rodney Ledward, the discredited gynaecologist who was the subject of the Ritchie Inquiry, was also a statistical outlier. Also in 2004, the BMJ started publishing Dr Foster case notes,13 which draw on analyses of HES data undertaken by the Dr Foster Research Unit.7
The present study by Aylin and colleagues5 shows that HES based models to predict hospital mortality, in three well defined conditions, compare favourably with dedicated clinical databases. Although the choice and interpretation of some of the variables (such as year and deprivation) may be questionable, in statistical terms HES based models predict hospital mortality as well as their clinical counterparts. Clinical databases are often more costly; so are they still necessary?
In our view, it would be premature to discard clinical databases, because their purpose is not limited to predicting mortality. They may also measure longer term outcomes, incorporate rapid changes in treatments (administrative databases are constrained by inertia), and include other outcomes (such as quality of life) that are not often found on administrative databases. Furthermore, administrative databases seldom (without linkage) cover mortality adequately. In-hospital mortality is a key outcome only in a few important diseases. Other potentially useful process outcomes such as length of stay are also limited. Crucially, clinical databases are clinically owned vehicles driven to improve quality of care through a peer led educational process, as exemplified by the Department of Veterans Affairs.14 Currently, HES data are not.
We advocate that where HES based analyses are accurate they should be incorporated into the existing quality improvement framework alongside clinical databases. This would help clinicians to test their usefulness in delivering quality improvement and so develop trust in the quality of HES data. Only when HES based analyses are considered fit for purpose, after extensive comparison with clinical databases, would the criticism about “dual” databases be valid.11 Dual databases can be useful, however. For example, hospital death rates adjusted for case mix after surgery for congenital heart disease in the UK identified Oxford Radcliffe Hospital as a high outlier using HES data, but this was explained by incomplete case ascertainment in HES, which recorded 20% fewer cases than the central cardiac audit database.15
Another routinely collected data set—which unlike administrative databases has not been closely scrutinised—that is fit for purpose, clinically meaningful, and has no apparent credibility problems is laboratory data. Prytherch and colleagues16 17 showed that models for predicting hospital mortality produced from laboratory data were as good as the best models reported by Aylin and colleagues.5 This is even more remarkable as Prytherch and colleagues predicted deaths in general surgery and general medicine and not the specific areas selected by Aylin and colleagues.5 Most modern hospitals now have computerised laboratory databases so further research into the use of these databases is needed.
Ultimately a key purpose of data (and analyses) is to support continual quality improvement. While clinical databases have a track record in delivering improvement, the extent to which administrative databases can be incorporated into clinical quality improvement processes remains, by and large, to be seen.
Competing interests: None declared.
Provenance and peer review: Commissioned; not externally peer reviewed.