Outcomes of variation in hospital nurse staffing in English hospitals: Cross-sectional analysis of survey data and discharge records

doi:10.1016/j.ijnurstu.2006.08.003

Int J Nurs Stud. Author manuscript; available in PMC 2010 June 30.

Published in final edited form as:

Int J Nurs Stud. 2007 February; 44(2): 175–182.

Published online 2006 October 24. doi: 10.1016/j.ijnurstu.2006.08.003

PMCID: PMC2894580

NIHMSID: NIHMS214821

Outcomes of variation in hospital nurse staffing in English hospitals: Cross-sectional analysis of survey data and discharge records

Anne Marie Rafferty,^a^f^* Sean P. Clarke,^b James Coles,^c^f Jane Ball,^d Philip James,^e Martin McKee,^f and Linda H. Aiken^b

^aFlorence Nightingale School of Nursing and Midwifery, King's College London, James Clerk Maxwell Building, 57 Waterloo Road, London SE1 8WA, UK

^bCenter for Health Outcomes and Policy Research, University of Pennsylvania, Philadelphia, PA, USA

^cCASPE Research, King's Fund, London, UK

^dEmployment Research, Hove, Sussex, UK

^eCHKS Ltd., Alcester, UK

^fLondon School of Hygiene and Tropical Medicine, London, UK

^*Corresponding author. Florence Nightingale School of Nursing and Midwifery, King's College London, James Clerk Maxwell Building, 57 Waterloo Road, London SE1 8WA, UK. Tel.: +44 20 7848 3561/3563; fax: +44 20 7848 3506. Email: anne_marie.rafferty/at/kcl.ac.uk (A.M. Rafferty).

The publisher's final edited version of this article is available at Int J Nurs Stud

See other articles in PMC that cite the published article.

Abstract

Context

Despite growing evidence in the US, little evidence has been available to evaluate whether internationally, hospitals in which nurses care for fewer patients have better outcomes in terms of patient survival and nurse retention.

Objectives

To examine the effects of hospital-wide nurse staffing levels (patient-to-nurse ratios) on patient mortality, failure to rescue (mortality risk for patients with complicated stays) and nurse job dissatisfaction, burnout and nurse-rated quality of care.

Design and setting

Cross-sectional analysis combining nurse survey data with discharge abstracts.

Participants

Nurses (N = 3984) and general, orthopaedic, and vascular surgery patients (N = 118 752) in 30 English acute trusts.

Results

Patients and nurses in the quartile of hospitals with the most favourable staffing levels (the lowest patient-to-nurse ratios) had consistently better outcomes than those in hospitals with less favourable staffing. Patients in the hospitals with the highest patient to nurse ratios had 26% higher mortality (95% CI: 12–49%); the nurses in those hospitals were approximately twice as likely to be dissatisfied with their jobs, to show high burnout levels, and to report low or deteriorating quality of care on their wards and hospitals.

Conclusions

Nurse staffing levels in NHS hospitals appear to have the same impact on patient outcomes and factors influencing nurse retention as have been found in the USA.

Keywords: Nurse staffing, Workforce planning, Patient outcomes, Nursing outcomes, Mortality, Failure to rescue, Staff outcomes, Job satisfaction, Burnout

1. Outcomes of variation in hospital nurse staffing in English hospitals

The impact of nurse staffing on patient outcomes has been controversial in the US and as contentious in the UK. A 2001 Audit Commission report on ward staffing in National Health Service (NHS) hospitals noted considerable variation across trusts in expenditures on nurse staffing but was not able to determine whether those differences were associated with variation in patient outcomes (Audit Commission, 2001). The Commission concluded, “Unless and until trusts that spend more [on staffing] can demonstrate a clear link with the quality of care that is delivered, movement towards a more even allocation of resources seems reasonable both for patients and staff.” (1, p. 15). The Healthcare Commission released a report in June 2005 suggesting that patients were more satisfied in hospitals with more qualified nurses but emphasized again the lack of evidence linking staffing to patient outcomes and the need for research to guide decision-making in this area (Healthcare Commission, 2005).

The Audit Commission's report coincided with the publication of the first results from the five-country International Hospital Outcomes Study. The International Hospital Outcomes Study, involving seven interdisciplinary research teams in five countries (US, Canada, England, Scotland and Germany), examined the extent to which the relationships between nurse staffing, the quality of the nurse work environment, and patient and nurse outcomes are similar across countries with well-resourced health care systems (Aiken et al., 2002a, b). It was seen that over 70% of nurses providing direct patient care in participating UK NHS hospitals in England reported that there were not enough nurses on their wards to provide care of high quality (Aiken et al., 2001). More than a third of nurses in these trusts scored in the high range on a standardized measure of job burnout, and almost 40% reported that they intended to leave their jobs within a year.

Recently published results from the US component of the International Hospital Outcomes Study documented a strong association between nurse staffing and mortality following common surgical procedures, and also showed that job dissatisfaction and burnout were associated with low staffing levels. In the US study, every one patient added to the average hospital-wide nurse workload increased the risk of death following common surgical procedures by 7% (Aiken et al., 2002a, b). There was a 31% difference in mortality between hospitals in which registered nurses cared for 8 patients each and those in which nurses cared for 4 patients each after taking into account patients' severity of illness and co-morbid conditions, and the size, technology level, and teaching status of the treating hospitals. Findings from the Canadian arm of the international study mirror those in the US, most notably, a staffing skill mix with a higher proportion of registered nurses was associated with significantly lower mortality (Estabrooks et al., 2005).

A robust evidence base of studies demonstrating that better hospital nurse staffing is associated with more favourable patient outcomes has stimulated both legislative and voluntary actions by hospitals in the US to improve staffing levels (Aiken et al., 2002a, b; Estabrooks et al., 2005; Moses and Mosteller, 1968; Hartz et al., 1989; Needleman et al., 2002; Page, 2004; Lang et al., 2004; Kazanjian et al., 2005; Spetz, 2004). While many concerns about the quality of hospital care are shared internationally (McKee et al., 1997; McKee et al., 1998), decision-makers in other countries have not always considered these US findings to be applicable to their particular national contexts. This paper reports on analyses of data on NHS hospitals in England from the International Hospital Outcomes Study and provides evidence of the kind sought by the Audit Commission on the relationship between nurse staffing levels and patient outcomes in England.

2. Methods

Nurse and patient data from 30 English hospital trusts analysed in this paper were collected in connection with the International Hospital Outcomes Study begun in 1999. The theoretical background and methods for the study are discussed elsewhere (Aiken et al., 2001; Aiken et al., 2002a, b; Estabrooks et al., 2005). Data were gathered from three sources. Information about hospital structure (such as size and teaching status) came from administrative databases. Nurses practicing in participating hospitals were surveyed to obtain data on patient-to-nurse ratios, staffing adequacy, working conditions, and quality of care indicators. Patient outcomes were derived from both nurse surveys and discharge abstracts. These data were merged to examine the influence of staffing and other hospital conditions on a variety of patient and nurse outcomes. The study protocol was approved by research ethics review boards at both the London School of Hygiene and Tropical Medicine and the University of Pennsylvania.

2.1. Samples and measures

2.1.1. Hospitals

The sampling frame for hospitals was a list of NHS trusts in 4 of the then 14 NHS regions (regional health authorities at the time of the study) that participated in the CHKS benchmarking program. CHKS provided commercial data benchmarking services to approximately two-thirds of NHS trusts in 1999 and thus provided an efficient infrastructure for recruiting hospitals to the study and a current database of standard patient discharge information that had been cleaned for analysis and adjusted for severity of illness. The four NHS regions were selected to ensure representation of trusts in urban and non-urban communities in different parts of England. All 32 of the trusts participating in the CHKS programme in each of these regions were approached, and all agreed to participate in both nurse surveys and sharing of their patient data. Due to one merger and problems in data collection at another trust, our final sample was 30 trusts.

A comparison of the trusts studied with acute trusts nationally is provided in Table 1. Of the 30 trusts, 10 had high-technology facilities (the capability to undertake cardiac, neurosurgical and renal procedures), 4 were provincial teaching hospitals, and 4 were London teaching hospitals (teaching status and technology status overlapped considerably). The trusts had between 368 and 2709 beds, with 11 having fewer than 600 beds, 12 having between 600 and 1000 beds and the remainder with over 1000 beds. Of the 30 hospitals in our sample and the 145 across England assigned to a specific geographical service area, 11 (45.8%) and 25 (19.2%), respectively, were designated as London hospitals. The high representation of London hospitals in the sample resulted from one of the four former Thames regions being in the sampling frame.

Table 1

Characteristics of the study hospitals compared with all acute care trusts in England, 1999

2.1.2. Nurses

We conducted surveys of nurses using lists provided by payroll offices at each trust. Nurses working full-time on medical and surgical wards, as well as nurses in other selected inpatient specialties involved in direct care were asked to complete a self-administered survey. Nurse managers, paediatric nurses, psychiatric nurses, midwives, as well as operating room nurses, were specifically excluded from the surveys. The survey questions were designed to collect data on basic demographic factors (education, age, duration in job, qualifications), nurses' workloads, nurses' evaluations of the quality of their work environment, the quality of care, their job satisfaction and their occupational health. Nurses received questionnaires on their units, in personally addressed envelopes, returning questionnaires by post to the research offices. They were assured of the confidentiality of their responses. Non-respondents received reminder cards at 4 and 9 weeks. Surveys were carried out from April to July 1999 yielding responses from 3984 nurses, a response rate of 49.4%.

The nurse staffing measure was derived from a series of questions inquiring about the number of patients on the respondent's ward during the last shift worked and the total number of nurses covering these patients (as well as the number of patients specifically assigned to the nurse herself/himself). The mean of all patient loads of all registered nurses and enrolled nurses (the latter corresponding roughly to practical nurses in the US) carrying at least one patient on the most recent shift worked in each hospital was used to derive a hospital-specific aggregate staffing measure. This measure of nurse staffing has been found to be superior to those derived from administrative databases because it includes only those nurses who have clinical caseloads (Aiken et al., 2002a, b).

Four nurse-reported outcomes were analysed. Job-related burnout was measured using the 9-item Emotional Exhaustion subscale of the Maslach Burnout Inventory, a standardized tool (Maslach and Jackson, 1986). Nurses who scored in the “high” burnout range with respect to published norms for the subscale were identified. Job satisfaction was rated by nurses on a 4-point scale from very satisfied to very dissatisfied. Nurses rated the quality of care on their units as excellent, good, fair or poor, and assessed whether the quality of care in their hospitals had improved, deteriorated or remained unchanged over the last year.

2.1.3. Patients

The patients studied were 20–85 years old and had been discharged in the 1998 calendar year from one of the 30 study hospitals. Hospitalizations studied were designated as having healthcare resource groups (HRG) codes (NHS Information Authority, 2005) representing specific types of general, orthopaedic, and vascular surgical cases that had been decided upon by an international consortium of researchers to represent commonly occurring surgical procedures undertaken by most hospitals (HRGs are analogous to Diagnosis Related Groups in the US). These case types closely parallel those in earlier US publications (Aiken et al., 2002a, b). The specific HRGs examined were: F01–F06, F11–F16, F21–F23, F31–F35, F41–F45, F51–F54, F61–F63, F71–F75, F81–F82, F91–F95, G02–G05, G11–G17, G21–G22, H01–H22, J01–J07, J29–J37, K01–K03, Q01–Q07, Q11, Q15–Q16.

Discharge abstracts analysed here were based on data collected for the nationally mandated Hospital Episode Statistics, a minimum data set of variables relating to each episode of care. Some 300 variables, ranging from demographic characteristics of patients and postcode, to administrative statistics (date of admission/discharge), identities of consultants, specialized diagnoses (up to 16) and procedures, source of admission, discharge destination and time on waiting list were included. The close working relationship developed between CHKS and client trusts facilitates a detailed and interactive process of data validation and correction leading to quality levels above those achievable through automated data checks. Consequently, the data analysed here can generally be presumed to be of higher quality than the raw data from the NHS system.

In addition to inpatient mortality, failure to rescue (FTR) was examined (deaths among patients who experienced complications). FTR is based on the premise that complication rates in hospitals may vary as a function of patient illness severity, whereas the survival rate among patients who experience complications may be closely related to quality of care (Silber et al., 1997). In the US, where this measure was developed, FTR is operationalized by scanning secondary diagnosis codes for evidence of life-threatening complications. Because of more limited secondary diagnosis coding in English hospital data (McKee et al., 1999), a FTR measure based on extended length of stay (LOS) (an indicator of deviations from a normal clinical course) (Silber et al., 2003) was developed in the course of this study. Patients whose stays exceeded 1.25 times the median for their HRG were identified as likely to have developed a complication.

The risk adjustment model used to account for differences in mortality risk among patients was developed using a logistic regression approach that incorporated data on demographic factors, procedures and diagnoses, interactions between procedures and diagnoses, and a number of other interaction terms. (Charlson et al., 1987; DesHarnais et al., 1988). The model was based on data for all patients in CHKS-member trusts for the study year. The model yielded scores representing each patient's odds of dying. The C-statistic (area under the receiver operating characteristic curve) (Hanley and McNeil, 1982) for the mortality model was 0.94.

2.2. Data analysis

Descriptive statistics were used to illustrate the characteristics of the patients and the nurses in the sample. Logistic regression models were used to estimate the effects of nurse staffing on the patient outcomes (mortality and FTR), two nurse-reported job outcomes, and two nurse ratings of quality of care. To account for the clustering of nurses and patients within hospitals, robust (Huber–White) procedures were used to derive the logistic regression parameters (Huber, 1967). We computed the likelihood of patients dying and dying following complications, and of nurses reporting high burnout, job dissatisfaction, fair or poor quality of care on their units, and deterioration in the quality of care in their hospitals in relation to staffing levels before and after controlling for a series of patient or nurse characteristics (depending on the type of outcome), and then again after control for hospital characteristics (size, bed size, technology). Nurse staffing was recoded as a categorical variable grouping institutions into quartiles. All analyses were performed using STATA version 8.0 (STATA Corp, College Station, Tex.) using p<0.05 as the statistical significance level.

3. Results

Across the 30 hospitals, 3984 nurses responded to the questionnaire, most of whom were registered nurses. As noted in Table 2, two-thirds were employed on medical-surgical units, 1 in 10 worked in an intensive care unit, and nearly 1 in 4 worked in another clinical area (accident and emergency, elderly, and an “other” category were the three most common of these). Slightly more than one-third of the nurses experienced high burnout and were dissatisfied with their jobs (36% each). Some 16% reported that quality of care on their units was only fair or poor and 27% believed that quality of care in their institutions had deteriorated in the previous year. About 8% of the respondent nurses held bachelor or higher degrees.

Table 2

Characteristics of nurses in the study hospitals (N = 3984)

A slight majority of the 118 752 patients were women, as noted in Table 3. The mean age of patients was 54.5 years. Of these patients, 2.3% died and 35.9% experienced extended lengths of stay and were considered to have had complicated hospital stays. A substantial minority (31.6%) of all patients, and 42.6% of patients with complicated hospital stays were admitted on an emergency basis. The two most common categories of surgeries were digestive and musculoskeletal, with those two groups accounting for about 60% of the cases. Among patients who experienced complications, the characteristics were quite similar except that considerably more (6.4%) died during their hospital stays. The body systems related to the admission diagnoses of the complicated cases were very similar to those of the larger group of patients.

Table 3

Characteristics of surgical patients in the study hospitals

Average hospital-level workloads ranged from 6.9 to 14.3 patients per nurse across the 30 hospitals, reflecting the substantial variation in nurse staffing across trusts noted by the Audit Commission. Dividing the hospitals into four even groups on the basis of their mean staffing levels led to the distribution of hospitals indicated at the top of Table 4. Logistic regression modelling with robust standard errors was carried out to identify any relationship between the average number of patients cared for by nurses in each hospital and negative outcomes for patients or nurses. Results of the fully adjusted models, taking into account all major characteristics of patients and nurses within the hospitals, are shown in Table 4.

Table 4

Patient and nurse outcomes in trusts with different staffing levels

Patients in hospitals in the upper quartile (where nurses had the heaviest patient loads) were 26% more likely to die overall and 29% more likely to die following complicated hospital stays than those in the lowest quartile. The nurses in the hospitals with the heaviest workloads were between 71% and 92% more likely to show negative job outcomes (burnout and job dissatisfaction), and to rate the quality of care on their wards as low and the quality of care in their hospitals as deteriorating. While increases in the risks of negative outcomes for patients and nurses in hospitals in the middle two quartiles were found in relation to those in hospitals in the bottom quartile on patient-to-nurse ratios, a number of these contrasts were not statistically significant. All, however, were in the expected direction. The clearest effects of staffing were seen when comparing the “best” staffed hospitals (lowest average workloads) versus the “worst” staffed hospitals.

4. Discussion

In this study of 30 hospital trusts in England, we found a large and consistent effect of nurse staffing on mortality outcomes in surgical patients as well as on nurse job outcomes and nurse ratings of quality of care. Hospitals in which nurses cared for the fewest patients each had significantly lower surgical mortality and FTR rates compared to those in which nurses cared for the greatest number of patients each. These findings are remarkably similar to those observed in 168 Pennsylvania hospitals studied at approximately the same time as part of the same international study (Aiken et al., 2002a, b). In addition to an overall mortality rate of 2.3% in the patient population in the current study (the figure in Pennsylvania for a group of comparable patients was 2.0%), the contrasts between highest and lowest staffing levels revealed a decrease in mortality risk of 31% in Pennsylvania versus 26% in England. Thus, this study's results fit squarely into a rapidly expanding body of literature documenting a link between better nurse staffing and better patient outcomes. It is, to our knowledge, one of the first hospital outcomes studies based on UK patient and nurse data (Jarman et al., 1999; UK Neonatal Staffing Study Group, 2002), and the only one clearly linking better nurse staffing with lower mortality for common surgical procedures.

Using the final fully adjusted model and keeping all other characteristics of patients and hospitals constant, we used direct standardization procedures (Bishop et al., 1975) to calculate that some 246 fewer deaths would have been seen in this subset of general surgery patients in 30 trusts had all been treated in hospitals with the most favourable staffing levels. Since our study involved only a sample of trusts and a subset of patients within those trusts, the number of lives that could potentially be saved through investments in nursing throughout NHS hospitals could be in the thousands every year. While this calculation incorporates a number of assumptions and must be interpreted with caution, it suggests the possible magnitude of the consequences of low staffing for the types of outcomes of greatest concern to policymakers and the public alike.

In addition to better outcomes for patients, hospitals with higher nurse staffing levels had significantly lower rates of nurse burnout and dissatisfaction. The nurses in the hospitals with the heaviest patient loads were 71% more likely to experience high burnout and job dissatisfaction than hospitals with the most favourable nurse staffing. Nurse burnout and dissatisfaction are precursors of nurse resignations (Sheward et al., 2005; Lake, 1998) and patient dissatisfaction (Vahey et al., 2004). Our findings, like those of the US study (Aiken et al., 2002a, b) suggest that better-staffed hospitals may be more successful in retaining their nurses. Hospitals and health systems internationally, as well as in the UK, are looking hard at maintaining and increasing the number of employed nurses to meet service and quality goals (Chancellor of the Exchequer, 2003)—retention of currently employed nurses is key to meeting these goals.

Our findings that hospitals with more favourable nurse staffing show the best outcomes for patients and nurses provide the kind of research evidence called for by the UK Healthcare Commission in its report on ward staffing (Healthcare Commission, 2005). The findings suggest that quality of care and nurse retention would improve if staffing levels across the NHS were brought more into line with those in the best-staffed hospitals in this study.

We have undertaken a number of tasks to verify the accuracy of our findings. The analyses were repeated in several different ways to ensure that the results reported here were robust to different cut-points for categorizing hospital workload levels to include and exclude slightly different groups of hospitals. Rates of poor outcomes for patients and nurses in hospitals in the lower (“better”) tertile on staffing were compared against those in hospitals in the upper two tertiles (a three-group categorization, introducing three more hospitals into the bottom and top groups) and in the lowest versus the other five quintiles (a five-group categorization taking one hospital out of those groups) were examined. The results were comparable to those in Table 4. Even more pronounced contrasts in outcomes were seen in trusts in the top and bottom fifths of hospitals on staffing. In addition to these sensitivity analyses, we confirmed that the results were robust to recalculating staffing statistics, restricting consideration to nurses from medical and surgical wards and excluding responses from a small number of nurses who reported very high patient loads (above 25 patients) on the last shift. Vascular patients, one of the diagnostic categories included among the general surgery patients studied, have higher risk for mortality than other general surgery patients. Thus, we tested the effects of staffing with and without including vascular patients and there were no differences.

Data were linked at the level of the hospital. While we were able to accurately classify nurses and patients into hospitals, we do not know to what extent the specific patients whose outcomes were studied here were cared for by the nurse respondents. The staffing statistics analysed were averages across shifts and specialties and while they are sound indicators of the availability of nursing time to patients across entire hospitals, they should not be interpreted as patient to nurse staffing ratios for implementation at the ward level.

The study has a number of strengths including carefully cleaned and validated hospital outcomes data, primary nurse survey data from a large sample of nurses in each study hospital, sound risk adjustment methods for the patient outcomes and appropriate controls for nurse and hospital characteristics. While measurement error and unmeasured differences in patient populations and hospital operations across acute hospitals may explain a portion of the effects seen here (as well as the lack of a consistent effect of staffing across hospitals in the middle range on staffing), we are confident that we have incorporated all of the relevant data available to us in this analysis.

In conclusion, this study is important in documenting that low levels of nurse staffing in UK hospitals have the same detrimental effects on patient outcomes and nurse retention that have been found in a large number of studies conducted primarily in North America. When considered alongside disturbing trends in the global nurse workforce (Buchan, 2002) it suggests that problems with the supply of nurses and the possible impacts of variability in nurse staffing levels on patients cannot be considered solely N. American problems. To our knowledge this is the first UK study to document lower mortality and improved nurse retention in hospitals with more favourable patient nurse ratios. There is an urgent need for action by health system and hospital leaders internationally to implement strategies, which promote the retention and sustainability of the registered nurses in the workforce. Shortage is not just about numbers but also about how the health system functions to enable nurses to use their skills effectively.

What is already known about the topic?

There is growing evidence from studies in the US that hospitals in which nurses care for fewer patients have better patient outcomes, but there is little evidence available internationally.

What this paper adds

This large-scale national study of nurse staffing in the UK supports US findings that patients and nurses in hospitals with the most favourable staffing levels have better outcomes than those in less favourably staffed hospitals.
Provides evidence that the positive relationship between low nurse: patient staffing ratios and favourable patient and nurse outcomes is an international phenomenon.

Acknowledgements

This research was supported by grants from the Nuffield Trust, the Commonwealth Fund of New York, and the National Institute of Nursing Research, National Institutes of Health (R01NR04513). The authors thank Douglas Sloane, Christine Hancock, James Buchan, and members of the International Hospital Outcomes Consortium for their contributions to this research.

We are grateful to the National Institute for Health and the Commonwealth Fund of New York for supporting the study.

References

Aiken LH, Clarke SP, Sloane DM, Sochalski JA, Busse R, Clarke H, et al. Nurses' reports on hospital care in five countries. Health Affairs (Millwood) 2001;20:43–53. [PubMed]
Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. Journal of the American Medical Association. 2002a;288:1987–1993. [PubMed]
Aiken LH, Clarke SP, Sloane DM. Hospital staffing, organization, and quality of care: cross-national findings. International Journal for Quality in Health Care. 2002b;14(1):5–13. [PubMed]
Audit Commission Acute Hospital Portfolio, Review of National Finding: Ward Staffing (No. 3) The Audit Commission; London: 2001.
Bishop YM, Fienberg SE, Holland PW. Discrete Multivariate Analysis: Theory and Practice. MIT Press; Cambridge, MA: 1975.
Buchan J. Global nursing shortages. British Medical Journal. 2002;324:751–752. [PMC free article] [PubMed]
Chancellor of the Exchequer Budget Statement, No. 3. 2003. 9 April.
Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. Journal of Chronic Diseases. 1987;40(5):373–383. [PubMed]
DesHarnais SI, Chesney JD, Wroblewski RT, Fleming ST, McMahon LF., Jr The risk adjusted mortality index: a new measure of hospital performance. Medical Care. 1988;26(12):1129–1148. [PubMed]
Estabrooks CA, Midodzi WK, Cummings GG, Ricker KL, Giovannetti P. The impact of hospital nursing characteristics on 30-day mortality. Nursing Research. 2005;54(2):74–84. [PubMed]
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36. [PubMed]
Hartz AJ, Krakauer H, Kuhn EM, Young M, Jacobsen SJ, Gay G, et al. Hospital characteristics and mortality rates. New England Journal of Medicine. 1989;321(25):1720–1725. [PubMed]
Healthcare Commission Ward Staffing. Commission for Healthcare Audit and Inspection; London: 2005.
Huber PJ. The behaviour of maximum likelihood estimates under non-standard conditions. Fifth Berkeley Symposium on Mathematical Statistics and Probability; Berkeley: University of California Press; 1967. pp. 221–233.
Jarman B, Gault S, Alves B, Hider A, Dolan S, Cook A, et al. Explaining differences in English hospital death rates using routinely collected data. British Medical Journal. 1999;318(7197):1515–1520. [PMC free article] [PubMed]
Kazanjian A, Green C, Wong J, Reid R. Effect of the hospital nursing environment on patient mortality: a systematic review. Journal of Health Services and Research Policy. 2005;10(2):111–117. [PubMed]
Lake ET. Advances in understanding and predicting nurse turnover. Research on Social Health Care. 1998;15:147–171.
Lang TA, Hodge M, Olson V, Romano PS, Kravitz RL. Nurse-patient ratios: a systematic review on the effects of nurse staffing on patient, nurse employee, and hospital outcomes. Journal of Nursing Administration. 2004;34(7/8):326–337. [PubMed]
Maslach C, Jackson SE. Maslach Burnout Inventory Manual. second ed. Consulting Psychologists Press; Palo Alto, CA: 1986.
McKee M, Rafferty AM, Aiken L. Measuring hospital performance: are we asking the right questions? Journal of Royal Society of Medicine. 1997;90:187–191. [PMC free article] [PubMed]
McKee M, Aiken L, Rafferty AM, Sochalski J. Organisational change and quality of health care: an evolving international agenda. Quality in Health Care. 1998;7:37–41. [PMC free article] [PubMed]
McKee M, Coles J, James P. `Failure to rescue' as a measure of the quality of hospital care: the limitations of secondary diagnosis coding in English hospital data. Journal of Public Health Medicine. 1999;21:453–458. [PubMed]
Moses LE, Mosteller F. Institutional differences in postoperative death rates: commentary on some of the findings of the National Halothane Study. Journal of the American Medical Association. 1968;203(7):492–494. [PubMed]
Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. New England Journal of Medicine. 2002;346(22):1715–1722. [PubMed]
NHS Information Authority Healthcare Resource Groups v4. 2005. [Last accessed 15 January 2005]. Available at http://www.nhsia.nhs.uk/casemix/pages/hrg.asp.
Page A, editor. Keeping Patients Safe: Transforming the Work Environments of Nurses. National Academies Press; Washington, DC: 2004.
Sheward L, Hunt J, Hagen S, Macleod M, Ball J. The relationship between UK hospital nurse staffing and emotional exhaustion and job dissatisfaction. Journal of Nursing Management. 2005;13:51–60. [PubMed]
Silber JH, Rosenbaum PR, Williams SV, Ross RN, Schwartz JS. The relationship between choice of outcome measure and hospital rank in general surgical procedures: implications for quality assessment. International Journal for Quality in Health Care. 1997;9(3):193–200. [PubMed]
Silber JH, Rosenbaum PR, Even-Shoshan O, Shabbout M, Zhang X, Bradlow ET, et al. Length of stay, conditional length of stay, and prolonged stay in pediatric asthma. Health Services Research. 2003;38(3):867–886. [PMC free article] [PubMed]
Spetz J. California's minimum nurse-to-patient ratios: the first few months. Journal of Nursing Administration. 2004;34(12):571–578. [PubMed]
UK Neonatal Staffing Study Group Patient volume, staffing, and workload in relation to risk-adjusted outcomes in a random stratified sample of UK neonatal intensive care units: a prospective evaluation. Lancet. 2002;359(9301):99–107. [PubMed]
Vahey DC, Aiken LH, Sloane DM, Clarke SP, Vargas D. Nurse burnout and patient satisfaction. Medical Care. 2004;42(Suppl. 2):II57–II66. [PMC free article] [PubMed]