Explaining differences in English hospital death rates using routinely collected data

BMJ. 1999 June 5; 318(7197): 1515–1520.

PMCID: PMC27892

Explaining differences in English hospital death rates using routinely collected data

Brian Jarman, emeritus professor of primary health care,^a Simon Gault, research analyst,^a Bernadette Alves, research analyst,^a Amy Hider, statistician,^b Susan Dolan, research analyst,^a Adrian Cook, research analyst,^a Brian Hurwitz, senior lecturer,^a and Lisa I Iezzoni, professor of medicine^c

^aDepartment of Primary Health Care and General Practice, Imperial College School of Medicine, London W2 1PG, ^bDepartment of Medical Statistics and Evaluation, Imperial College School of Medicine, Hammersmith Hospital, London W12 0NN, ^cHarvard Medical School, Division of General Medicine and Primary Care, Department of Medicine, Beth Israel Deaconess Medical Center, 330 Brookline Avenue LY-326, Boston, MA 02215, USA

Contributed by

Contributors: BJ conceived, initiated, and coordinated the study and is the study guarantor. All the authors were involved in collecting, analysing, and interpreting the data; in particular, AH provided statistical support, and SG, BA, BJ, SD, and AC analysed the hospital episode statistics and the other routine data. The paper was written jointly by BJ, BA, BH, and LI, with all other authors commenting and editing the drafts and approving the final version.

Correspondence to: B Jarman

Accepted May 19, 1999.

This article has been cited by other articles in PMC.

Abstract

Objectives

To ascertain hospital inpatient mortality in England and to determine which factors best explain variation in standardised hospital death ratios.

Design

Weighted linear regression analysis of routinely collected data over four years, with hospital standardised mortality ratios as the dependent variable.

Setting

England.

Subjects

Eight million discharges from NHS hospitals when the primary diagnosis was one of the diagnoses accounting for 80% of inpatient deaths.

Main outcome measures

Hospital standardised mortality ratios and predictors of variations in these ratios.

Results

The four year crude death rates varied across hospitals from 3.4% to 13.6% (average for England 8.5%), and standardised hospital mortality ratios ranged from 53 to 137 (average for England 100). The percentage of cases that were emergency admissions (60% of total hospital admissions) was the best predictor of this variation in mortality, with the ratio of hospital doctors to beds and general practitioners to head of population the next best predictors. When analyses were restricted to emergency admissions (which covered 93% of all patient deaths analysed) number of doctors per bed was the best predictor.

Conclusion

Analysis of hospital episode statistics reveals wide variation in standardised hospital mortality ratios in England. The percentage of total admissions classified as emergencies is the most powerful predictor of variation in mortality. The ratios of doctors to head of population served, both in hospital and in general practice, seem to be critical determinants of standardised hospital death rates; the higher these ratios, the lower the death rates in both cases.

Key messages

Between 1991-2 and 1994-5 average standardised hospital mortality ratios in English hospitals reduced by 2.6% annually, but the ratios varied more than twofold among the hospitals
After adjustment for the percentage of emergency cases and for age, sex, and primary diagnosis, the best predictors of standardised hospital death rates were the numbers of hospital doctors per bed and of general practitioners per head of population in the localities from which hospital admissions were drawn
England has one of the lowest number of physicians per head of population of the OECD countries, being only 59% of the OECD average
It is now possible to control for factors outside the direct influence of hospital policy and thereby produce a more valid measure of hospital quality of care

Introduction

Wide variations in English hospital inpatient death rates have been observed over many years,¹^–⁴ and concerns have been expressed that such variations could reflect important differences in the quality of medical care available in different hospitals.⁵^,6 Hitherto, research has provided contradictory evidence about the relation of hospital mortality to quality of care.⁶^–⁹ While differences in patients’ age and severity of illness may explain some of the variation in hospital death rates, adjustment for age, sex, and severity leaves a large amount of unexplained variation.¹⁰^–¹⁵

Comparisons of hospital inpatient death rates, published annually in the United States as league tables, have resulted in lively discussion and debate about their compilation and usefulness.¹³^,16^–¹⁸ Meaningful comparison of hospital death rates requires adjustments for severity of illness, length of hospital stay, age, diagnosis, and type of admission. Suitably standardised hospital death rates are used both as indicators of quality of care and in the setting of standards in the United States.¹⁹^–²²

The NHS offers unique opportunities for examining the reasons for differences in hospital death rates because it provides a virtually closed system of care available to almost everyone in the country. Since 1987, data have been routinely collected nationally on every admission to hospital, providing a comprehensive database on all inpatient admissions. By linking other sources of routinely collected data to analyse inpatient hospital death rates, we attempted to ascertain differences in hospital mortality in England and to determine the main factors explaining the variation between hospitals over a four year period.

Methods

Data sources

We obtained data from three main sources: the NHS hospital episode statistics data system,²³ the national decennial census,²⁴^,25 and other routine NHS data such as hospital characteristics,²⁶^,27 hospital staffing levels, and general practitioner distribution over England.²⁸ For 51 hospitals, the results of a patient centred survey were available.²⁹

The hospital episode statistics database from 1991-2 to 1994-5 includes information on every inpatient spell in NHS hospitals in England. Each spell includes the following information: patient’sage, sex, postcode of residence, primary diagnosis and up to six additional sub-diagnoses coded with the International Classification of Diseases, Ninth Revision (ICD-9), type of admission (emergency or elective), and length of stay.

We obtained census data from 1991 at the level of the 8595 English electoral wards (average population 5500 residents), which provided a range of socioeconomic indicators³⁰^–³³ and the percentage of people with self reported, limiting, longstanding illness. The census data also contained information about the NHS facilities, hospices, and local authority or nursing home places available within each area.

We used other data sources, many routinely published by the NHS, such as numbers of hospital beds and indicators of staffing levels of hospital doctors and nurses and general practitioners per head of population.

Data extraction

NHS hospitals vary greatly in their size and purpose. Our goal was to compare roughly similar facilities, and we therefore selected the data using criteria based on type and size as well as on the quality of the data recorded in the hospital episode statistics database.

We looked at four years of data, from1991-2 to 1994-5. We excluded community and specialty institutions, small hospitals (under 9000 admissions during the four years), and hospitals without accident and emergency units. We also excluded any hospital that had poor quality data (more than 30% of inpatient episodes without a valid discharge or more than 30% of primary diagnoses recorded as “unknown”) for at least one of the four years. We included discharge records only—that is, episodes which ended in discharge (alive or dead) from the hospital rather than transfer to the care of another consultant within the hospital. In this paper, we use the terms admission and discharge to refer to the same outcome measure, namely the number of alive or dead hospital discharges; the term hospital refers to hospital trusts, which may occupy more than one site.

Discharges were included in the analysis if the primary diagnosis was one of 85 primary diagnoses which accounted for 80% of deaths. Weeliminated from the analyses all transfers between hospitals (2% of admissions and 3% of discharges). Data on deaths outside hospital were unavailable; it was therefore difficult to take account of differences in discharge practices that could affect comparisons of inpatient mortality. To address this situation, we recorded the availability of other NHS resources within each hospital health authority area, selected patients by lengths of stay of less than 14, 21, or 28 days, and used length of stay as a possible explanatory variable.

Measures of coexisting illness

Several studies stress the importance of adjusting for severity of illness in hospital admissions when comparing quality of health care.³⁴^–⁴¹ Since hospital statistics of inpatient episodes do not include detailed data on clinical severity, in addition to standardising for primary diagnosis, we calculated several measures of comorbidity based on discharge diagnoses for each hospital: the number of bodily systems affected by disease, the percentage of patient admissions with one of the 15 most serious primary diagnoses (responsible for 50% of all deaths), and the percentage both of cases and of deaths with comorbidities (that is, subdiagnoses) in each of the 85 diagnoses that led to 80% of all deaths. We ranked subdiagnoses by their univariable correlation with hospital standardised mortality ratios and created a measure of comorbidity by combining the top two or three comorbidity diagnoses. We used each of these measures in our model as independent estimates of the severity of illness treated.

Analysis

Because initial findings suggested that the percentage of emergency admissions was the strongest predictor of hospital standardised mortality ratios, we built up two models, the first (model A) included all admissions (both emergency and elective), and the second (model B) looked at mortality for emergency admissions only.

We conducted weighted multiple linear regressions that took account of the varying hospital volumes of cases (weights were defined as the reciprocal of the standard error squared where standard errors were derived using the normal approximation to a Poisson distribution of observed deaths). Each potential explanatory variable was used separately in a univariable regression model, and then multivariable analyses were performed. Backwards and forwards stepwise selection techniques were used, with a significance level of P=0.01. The adjusted R² was derived—this is the percentage of variation explained by the model after adjustment for the number of variables in the model.

The residuals were checked with standard diagnostic methods and were found to be satisfactory.⁴² The stability of the final model was checked by repeating the fitting procedure after removing observations with high influence. Fractional polynomials were also used to check for curvature in the explanatory variables, and no curvature was found.⁴³

Dependent variable—Our dependent variable was the hospital indirectly standardised mortality ratio, which is defined as the ratio of actual number of deaths to expected deaths multiplied by 100. We calculated death rates for the four years studied stratified by age (using 10 year age groups), sex, and the 85 primary diagnoses. These were used to calculate the expected deaths for each hospital by multiplying the number of hospital inpatient admissions in each stratum of age, sex, and primary diagnosis by the stratum specific rates. We also calculated hospital standardised mortality ratios using direct standardisation, which produced similar results to those from indirect methods.

Independent variables—The Appendix lists each of the independent variables considered in a univariable analysis. Three types of variables were used: aggregated discharge data such as the percentage of emergency cases, individual hospital data such as total number of beds, and community attributed data such as the percentage of patients with limiting longstanding illness. Aggregate discharge data was taken from the individual discharge records and aggregated across each hospital. Community data was taken from geographical areas (1991 electoral wards and 1995 health authorities), attributed from area of residence to each discharge (via postcode), and then averaged across discharges for each hospital.

Results

Descriptive statistics

We retained 183 acute general hospital trusts for analysis, roughly two hospitals per health authority in England. Over the four year study period, 7.7 million admissions were considered, of which 60% were classified as emergencies, accounting for 93% of all deaths considered (table (table1).1). These 183 hospitals covered 85% of all admissions (88% of emergency admissions) in the England hospital episode statistics data for the 85 diagnoses.

Table 1

Descriptive statistics for admissions to the 183 study hospitals during 1991-2 to 1994-5

Crude death rates varied between hospitals from 3.4% to 13.6%, with a mean mortality of 8.5%. The mean annual mortality fell from 9.2% to 7.6% over the four years. When annual death rates were standardised by age, sex, and primary diagnoses the mean hospital standardised mortality ratios fell from 104.9 to 97.0 (average annual fall of 2.6%).

Regression analyses

Length of stay proved not to be significant, and table table22 shows results only for all lengths of stay. It shows the predictors associated with hospital standardised mortality ratios at the 1% significance level and their regression coefficients. Table Table33 shows the univariable associations for these predictors.

Table 2

Results of stepwise regression analyses: variables associated with hospital standardised mortality ratios at 1% significance level and tabulated in order of selection.

Table 3

Univariable associations between hospital standardised mortality ratios and variables used in models A and B of regression analyses

For model A, based on emergency and elective admissions, the adjusted R² was 0.65. For model B, based on emergency admissions only, the adjusted R² was 0.50. The results show that in model A, after adjustment for the percentage of emergency admissions, the best predictors of hospital mortality were numbers of hospital doctors per 100 hospital beds and general practitioners per 100 000 population. The figure displays hospital standardised mortality ratios for three groups of hospitals and areas: low doctored (numbers of doctors per beds and general practitioners per head of population below mean values by at least ½ SD), with a mean hospital standardised mortality ratio of 112; high doctored (doctors per beds and general practitioners per head of population more than ½ SD above mean), with a mean hospital standardised mortality ratio of 88; and intermediate hospitals, with a mean hospital standardised mortality ratio of 99.

In model A higher hospital standardised mortality ratios were associated with higher percentages of emergency admissions, lower numbers of hospital doctors per hospital bed, and lower numbers of general practitioners per head of population. The numbers of hospital doctors of different grades were also considered as explanatory variables, but total doctors per bed was found to be the best predictor. Higher hospital standardised mortality ratios were also associated with four other factors: low standardised admissions ratios for the health authority where the hospital was located, higher percentages of live discharged patients who went home (that is, non-death discharges to normal residence), higher percentages of cases of comorbidities of bronchopneumonia or heart failure or fracture of neck of femur, and lower availability of NHS facilities per 100 000 population for the health authority where the hospital was located. At the 5% level of significance, only one other predictor entered the model—possession of a specialist renal unit, which was associated with lower hospital standardised mortality ratios. At the 1% level, only the proportion of emergency admissions, numbers of hospital doctors per bed, and numbers of general practitioners per head of population were significant.

For model B, the percentage of cases with comorbidities of bronchopneumonia or malignant neoplasm was a significant predictor: number of general practitioners per 100 000 population was no longer significant. At the 5% level of significance, two variables entered the model, the proportion of grade A nurses (auxiliary nurses in training) as a percentage of all hospital nurses and bed occupancy. High percentages of grade A nurse and high bed occupancy were associated with higher hospital standardised mortality ratios.

By removing the effect of factors directly beyond hospital control (that is, all except doctors per bed), it is possible to calculate a hospital standardised mortality ratio that is likely to be a more valid measure of hospital quality of care. When we did this the range of resulting hospital standardised mortality ratios narrowed to 79-125.

Discussion

We have calculated hospital death ratios adjusted for age, sex, and diagnosis and looked at their association with factors likely, on clinical grounds, to be associated with quality of care. We focused on factors in the hospital and in the community surrounding the hospital that took account of financial and human resources, such as the number of doctors and nurses per hospital bed and the number of general practitioners per head of population from which hospital admissions were drawn.

Implications of results

The overall standardised death ratio in the 183 hospitals studied decreased on average by 2.6% a year between 1991-2 and 1994-5, but the variation between hospitals remained large. The associations we found between lower numbers of general practitioners per head of population and higher death rates raise several possible explanations. When general practitioners are relatively overworked the patients whom they send to hospital may be relatively sicker; and in these areas patients are more likely to be admitted as emergencies: high percentage of emergency admissions was significantly correlated with low numbers of general practitioners per 100 000 population, that is, with high average list size (r=−0.35, P<0.001). In model A of our regression analysis a reduction of 5000 hospital deaths per year was associated with a 27% increase in hospital doctors (9000 more doctors) or an 8.7% increase in general practitioners (2300 more doctors). In other words, our results suggest that a 1% increase in the number of hospital doctors per bed (333 more hospital doctors if the number of beds remains unchanged) is associated with a 0.119% decrease in hospital standardised mortality ratios (186 fewer deaths), and a 1% increase in general practitioners per head of population (267 more general practitioners if the population is unchanged) is associated with a 0.368% decrease in hospital standardised mortality ratios (575 fewer deaths).

In discussing risk and safety in hospital practice, Vincent puts heavy clinical workloads at the top of a list of conditions in which unsafe acts may occur.⁴⁴ Compared with other countries in the Organisation for Economic Cooperation and Development (OECD), the United Kingdom has a low number of physicians per head of population,⁴⁵ although it is planned to change this (Department of Health press release 98/337, 14 August 1998). In 1994 the United Kingdom had 1.6 physicians per 1000 population, which is more than one standard deviation below the mean of 2.7 for the 28 countries recorded by the OECD, the UK average amounting to only 59% of the OECD average for that year.

A higher percentage of live discharges to patients’ homes was also associated with higher hospital standardised mortality ratios. This probably reflects the fact that, where there are more NHS facilities, hospices, and local authority or nursing home places available, patients are more likely to be discharged to one of these for recovery and any deaths that follow would not be in hospital. The number of NHS facilities per head of population in the district surrounding the hospital is also a good predictor—the more facilities, the lower the hospital standardised mortality ratio. This effect may be similar to that of non-home discharges—that is, where these facilities do not exist patients are more likely to remain in hospital to die.

The age standardised admission ratio was also an important predictor, with higher admission rates being associated with lower mortality ratios—possibly indicating that some hospitals may admit relatively higher percentages of less sick patients because they have lower thresholds for admission.

At the 5% level of significance, hospitals with a specialist renal unit had lower hospital standardised mortality ratios—possession of a renal unit possibly being a marker of the quality of hospital care generally. Measures of social deprivation of the area of residence were not significantly related to mortality ratios. However, the percentage of hospital nurses graded A (the lowest grade, which indicates auxiliary nurses in training) was associated with higher hospital standardised mortality ratios: this result further reinforces the relation between staffing factors and outcomes.

Contrary to recent US data,⁴⁶ teaching hospital status was significant at the univariable level, but, once adjusted for doctor:bed ratio in the multivariable regression, proved not to be significant. University teaching hospitals had 56% higher doctor:bed ratios than non-teaching hospitals (mean values 0.378 v 0.243 respectively).

Considerable care should be exercised in interpreting hospital mortality data. In view of the literature on case mix,⁹^,13^,16^,34^,36^,47 it is surprising that only one of our measures of comorbidity was significant in the model (table (table3),3), and this might be related to the lack of data on severity of illness. Data for individual hospitals could prove useful, especially if broken down by individual diagnoses or specialties, provided that the number of cases is sufficient to give narrow confidence intervals and the data adjustments described can be made.⁴⁸^–⁵¹ Results could prompt hospitals with high standardised mortality ratios to examine their care processes and staff ratios.

Future studies

We have found an association between mortality rates and doctor number (in hospital and in general practice). We know of no studies that have looked at this association before, and our findings need to be validated by further investigations. A matched pair study of patients admitted to hospitals with high and low standardised mortality ratios could help to elucidate these findings. In such an investigation detailed data would have to be collected to allow for accurate adjustment of case mix.

Most of the significant predictors in our two models are outside the direct influence of hospital policy (except doctor numbers per bed), and adjustment for these external factors narrows the range of mortality ratios. This finding indicates that variation in quality of hospital care is smaller than incompletely adjusted statistics suggest, and that our model may be used to produce more valid indicators of quality of care.

Figure

Hospital standardised mortality ratios (with 99% confidence intervals) for hospitals and areas with low, medium, and high staffing levels of doctors

Acknowledgments

We thank Professor John Henry and Dr Paul Aylin for reading the paper, Debbie Hart for data preparation, and the BMJ’s referees for their comments.

Appendix: Independent variables included in univariable analysis for each hospital showing those used in the regression analysis

Aggregate discharge data

Percentage of emergency cases^*
Percentage of live discharges who went home
Percentage of cases and deaths with comorbidity (subdiagnoses) of the 85 diagnoses leading to 80% of all deaths and combinations of those with the highest correlations with hospital standardised mortality ratios*
Percentage of cases and deaths with each of the top 15 diagnoses which account for 50% of all deaths*
Percentage of cases with comorbidity (subdiagnoses) of the two or three conditions most highly correlated with hospital mortality*
Average number of diseased bodily systems
Standardised admissions ratio for health authority where hospital located*
Average length of stay
Number of cases

Hospital data

Hospital doctors per bed* and per case
Percentage of nurses at grades A* to I, nurses per doctor and per bed
Number of hospital beds
Percentage of geriatric beds
Bed occupancy*
Location—inner London,* outer London,* or outside London
University teaching,* non-university teaching, other general hospital
Provision of a range of specialist units.
Hospital income per bed and per case
Total and first accident and emergency attendances
Hospital charter standards (including waiting times)
Results of survey of patient centred care (51 hospitals only)

Community attributed data^†

General practitioners per 100 000 population according to Office of National Statistics (based on health authority of patient residence,* individual data averaged at health authority of residence level)
General practice nurses per 1000 population according to Office of National Statistics in hospital local health authority
NHS facilities per 100000 population in hospital local health authority*
Underprivileged area score* (individual data averaged at electoral ward of residence level)
Percentage of patients with various social factors: elderly living alone, children aged under 5, one parent families,* social class V, unemployed, overcrowded accommodation, mobility,* ethnic minority(individual data averaged at electoral ward of residence level)
Percentage of patients with limiting longstanding illness(individual data averaged at electoral ward of residence level)
Provision of nursing homes, residential care homes in hospital local health authority area

Footnotes

Funding: None

Competing interest: Professor Jarman is the medical member of the Bristol Royal Infirmary inquiry, but this research was completed before he was appointed on 26 January 1999.

^*Variables with high adjusted R² from univariable regression entered into multivariable regression models

^†Based on electoral ward of patient residence and averaged for all admissions (aggregate health authority of hospital data except where stated)

References

1. Nightingale F. Notes on hospitals. 3rd ed. London: Longman Green; 1863.

2. Buckle F. Vital and economical statistics of the hospitals, infirmaries, etc of England and Wales for the year 1863. London: Churchill; 1865.

3. National Confidential Enquiry into Perioperative Deaths (NCEPOD) Report 1992. London: NCEPOD; 1992.

4. Jarman B, Lang H, Ruggles R, Wallace M, Gault S, Astin P. The contribution of London’s academic medicine to healthcare and the economy – report commissioned by the deans of the medical schools of the University of London. London: University of London; 1997.

5. Commission on Professional Hospital Activities. Risk adjusted hospital mortality norms 1986 workbook. Ann Arbor, MI: CPHA; 1987.

6. DuBois RW, Rogers WH, Moxley JH, Draper D, Brook RH. Hospital inpatient mortality: is it a predictor of quality? N Engl J Med. 1987;317:1674–1680. [PubMed]

7. Fottler DM, Slovensky DJ, Rogers SJ. Public release of hospital specific death rates, guidelines for health care executives. Hosp Health Serv Administration. 1987;32(3):343–356. [PubMed]

8. Blumberg MS. Comments on HCFA hospital death rate statistical outliers. Health Serv Res. 1987;21:715–739. [PMC free article] [PubMed]

9. Thomas JW, Hofer TP. Research evidence on the validity of risk-adjusted mortality rate as a measure of hospital quality of care. Med Care Res Rev. 1998;55:371–404. [PubMed]

10. Chassin MR, Park RE, Lohr KN, Keesey J, Brook RH. Differences among hospitals in Medicare patient mortality. Health Serv Res. 1989;24:1–31. [PMC free article] [PubMed]

11. Bradbury RC, Stearns FE, Steen PM. Interhospital variations in admission severity—adjusted hospital mortality and morbidity. Health Serv Res. 1991;26:407–424. [PMC free article] [PubMed]

12. Al-Haider AS, Wan TTH. Modelling organizational determinants of hospital mortality. Health Serv Res. 1991;26:303–323. [PMC free article] [PubMed]

13. Park RE, Brook RH, Kosecoff J, Keesey J, Rubenstein L, Keeler E. Explaining variations in hospital death rates, randomness, severity of illness, quality of care. JAMA. 1990;264:484–490. [PubMed]

14. Iezzoni LI, Ash AS, Shwartz M, Daley J, Hughes JS, Mackiernan YD. Judging hospitals by severity-adjusted mortality rates: the influence of the severity-adjustment method. Am J Public Health. 1996;86:1379–1387. [PubMed]

15. Iezzoni LI. The risks of risk adjustment. JAMA. 1997;278:1600–1607. [PubMed]

16. Goss MEW, Read JI. Evaluating the quality of hospital care through severity adjusted death rates: some pitfalls. Med Care. 1974;12:202. [PubMed]

17. Thomas JW, Holloway JJ, Guire KE. Validating risk-adjusted mortality as an indicator for quality of care. Inquiry. 1993;30:6–22. [PubMed]

18. Hartz AJ, Gottlieb MS, Kuhn EM, Rimm AA. The relationship between adjusted hospital mortality and the results of peer review. Health Serv Res. 1993;27:765–777. [PMC free article] [PubMed]

19. Lipworth L, Lee JAH, Morris JN. Case fatality in teaching and nonteaching hospitals, 1956-1959. Lancet. 1963;i:71.

20. Palmer HR, Reilly MC. Individual and institutional variables which may serve as indicators of quality of medical care. Med Care. 1979;18:693–717. [PubMed]

21. Luft HS, Hunt SS. Evaluation of individual hospital quality through outcome statistics. JAMA. 1986;255:2780. [PubMed]

22. Hartz AJ, Krakauer H, Kuhn EM, Young M, Jacobsen SJ, Greer G. Hospital characteristics and mortality rates. N Engl J Med. 1989;321:1720–1725. [PubMed]

23. Department of Health. Hospital episode statistics. England: financial years 1991-92, 1992-93, 1993-94 and 1994-95. London: DoH; 1993-1996.

24. Office of Population Censuses and Surveys. 1991 census definitions. Great Britain. London: HMSO; 1992.

25. Dale A, Marsh C, editors. The 1991 census users guide. London: HMSO; 1993.

26. NHS Executive. Hospital and ambulance services: the patient’s charter comparative performance guide 1993-1994. London: Department of Health; 1995.

27. Department of Health. Waiting times for first outpatient appointments in England: quarter ended 30 September 1994. Statistical bulletin 1995/3. London: DoH; 1995.

28. Department of Health. GMS (general medical services) GMP (general medical practitioner) census. London: DoH; 1997.

29. Bruster S, Jarman B, Bosanquet N, Weston D, Erens R, Delbanco TL. The patient’s view—a national survey of hospital patients. BMJ. 1994;309:1542–1546. [PMC free article] [PubMed]

30. Jarman B. Underprivileged areas: validation and distribution of scores. BMJ. 1984;289:1587–1592. [PMC free article] [PubMed]

31. Townsend P, Phillimore P, Beattie A. Bristol: Northern RHA, University of Bristol; 1986. inequalities in health in the Northern region.

32. Carstairs V, Morris R. Deprivation and health in Scotland. Aberdeen: Aberdeen University Press; 1991.

33. Office of Population Censuses and Surveys. Classification of occupations 1980. London: HMSO; 1980.

34. Greenfield S, Aronow HU, Elashoff RM, Watanabe D. Flaws in mortality data: the hazards of ignoring comorbid disease. JAMA. 1988;260:2253–2255. [PubMed]

35. Green J, Wintfeld N, Sharkey P, Passman LJ. The importance of severity of illness in assessing hospital mortality. JAMA. 1990;263:241–246. [PubMed]

36. Dicing with death rates (editorial) Lancet. 1993;341:1183–1184. [PubMed]

37. Orchard C. Comparing health care outcomes. BMJ. 1994;308:1493–1496. [PMC free article] [PubMed]

38. McKee M, Hunter D. Mortality league tables: do they inform or mislead? Qual Health Care. 1995;4:5–12. [PMC free article] [PubMed]

39. Goldfarb MG, Coffey RM. Case mix differences between teaching and non-teaching hospitals. Inquiry. 1987;24:68–84. [PubMed]

40. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chron Dis. 1987;40:373–383. [PubMed]

41. Orchard C. Measuring the effects of casemix on outcomes. J Eval Clin Pract. 1996;2:111–121. [PubMed]

42. Royston JP. A simple method for evaluating the Shapiro Francia W test for non-normality. Statistician. 1983;32:297–230.

43. Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling (with discussion) Appl Stat. 1994;43:429–467.

44. Vincent C. Framework for analysing risk and safety in clinical medicine. BMJ. 1998;316:1154–1157. [PMC free article] [PubMed]

45. OECD. Health data—a comparative analysis of 29 countries. Paris: OECD; 1998.

46. Kassirer JP. Hospitals, heal yourselves. N Engl J Med. 1999;340:309–310. [PubMed]

47. Goldstein H, Spiegelhalter DJ. League tables and their limitations: statistical issues in comparisons of institutional performance (with discussion) J R Stat Soc A. 1996;159:385–443.

48. Roemer MR, Friedman JW. Doctors in hospitals: medical staff organization and hospital performance. Baltimore: Johns Hopkins University; 1971.

49. Selker HP. Systems for comparing actual and predicted mortality rates: characteristics to promote co-operation in improving hospital care. Ann Intern Med. 1993;118:820–822. [PubMed]

50. NHS Executive. ‘Faster access to modern treatment’: how NICE appraisal will work. London: Department of Health; 1999.

51. McKee M. Indicators of clinical performance. BMJ. 1997;315:142. [PMC free article] [PubMed]

Articles from BMJ : British Medical Journal are provided here courtesy of
BMJ Publishing Group