|Home | About | Journals | Submit | Contact Us | Français|
Emphasis has been placed on quality and patient safety in medicine; however, little is known about whether quality over time has actually improved in areas such as patient safety indicators (PSIs).
To determine whether national trends for hospital PSIs have improved from 1998 to 2007.
Using PSI criteria from the Agency for Healthcare Research and Quality, PSIs were identified in the Nationwide Inpatient Sample (NIS) for all eligible inpatient admissions between 1998 and 2007. Joinpoint regression was used to estimate annual percentage changes (APCs) for PSIs.
Annual percent change for PSIs.
From 1998 to 2007, 7.6 million PSI events occurred for over 69 million hospitalizations. A total of 14 PSIs showed statistically significant trends. Seven PSIs had increasing APC: postoperative pulmonary embolism or deep vein thrombosis (8.94), postoperative physiological or metabolic derangement (7.67), postoperative sepsis (7.17), selected infections due to medical care (4.05), decubitus ulcer (3.05), accidental puncture or laceration (2.64), and postoperative respiratory failure (1.46). Seven PSIs showed decreasing APCs: birth trauma injury to neonate (−17.79), failure to rescue (−6.05), postoperative hip fracture (−5.86), obstetric trauma–vaginal without instrument (−5.69), obstetric trauma–vaginal with instrument (−4.11), iatrogenic pneumothorax (−2.5), and postoperative wound dehiscence (−1.8).
This is the first study to establish national trends of PSIs during the past decade indicating areas for potential quality improvement prioritization. While many factors influence these trends, the results indicate opportunities for either emulation or elimination of current patient safety trends.
In the past decade, much emphasis has been placed on the quality of medical care as famously noted in the Institute of Medicine's landmark report, To Err Is Human: Building a Safer Health System (Kohn, Corrigan, and Donaldson 1999). To address the need for quality monitoring, the Agency for Healthcare Research and Quality (AHRQ) established a set of patient safety indicators (PSIs) to assist in monitoring potentially preventable events for patients treated in hospitals (Patient Safety Indicators Overview 2006). AHRQ's Evidence-Based Practice Center at the University of California, San Francisco and Stanford University defined and expanded PSIs (Iezzoni, Foley, and Heeren 1992; McDonald 2002). The PSIs were developed through a comprehensive literature review, analysis of the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes, expert panel review, risk adjustment measures, and empiric analyses. The PSIs are measures that screen for potential adverse events that patients experience during hospitalization that are likely amenable to prevention by changes at the system or provider level and are frequently part of a hospital quality dashboard (Agency for Healthcare Research and Quality 2003a, b).
Patient safety indicators can provide a report card to determine whether quality on a national level has changed as a result of quality initiatives. Many quality initiatives have taken place through different national organizations that share the goal of improving patient safety over the past decade. For example, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), through its accreditation process, has had numerous initiatives. JCAHO has continued to mandate system-wide changes to the provision of care in health organizations over time through the following policies: Sentinel Event (1996), Quality Indicators (2000), Patient Safety Goals (2002), Universal Protocol (2004), and unannounced site accreditation visits (2006) (Joint Commission on Accreditation of Healthcare Organizations 2007, 2008). In 1998, the Leapfrog Group began as a consortium of health care purchasers and providers responding to both rising health care costs and concerns regarding quality of care and provided several quality initiatives for its over 37 million patients, including computerized physician order entry and evidence-based hospital referral to high-volume hospitals (Bahl et al. 2008). In 2006, the Accreditation Council for Graduate Medical Education, motivated largely by concerns for patient safety, enacted the 80-hour work week for residents (Accreditation Council for Graduate Medical Education 2003), the effects of which are still being debated. Also, the Institute for Healthcare Improvement has played a prominent role in providing forums and practices for quality improvement (Encinosa and Hellinger 2008).
Finally, a major change in quality improvement took place in October 2008, when the Centers for Medicare and Medicaid Services (CMS) announced that eight hospital-acquired conditions, which correspond to some PSIs, would have incremental payments disallowed (Rosenthal 2007; Centers for Medicare and Medicaid Services 2007). Given all of these initiatives, it is possible to anticipate changes in PSIs that correspond to specific quality interventions. For example, with the advent of the Leapfrog recommendations for computerized physician order entry, ICU physician staffing and Evidence Based Hospital Referral, a decline in the specific PSIs of transfusion reactions, selected infections due to medical care, and postoperative respiratory failure/deep venous thrombosis/sepsis should follow. With all of this focus on quality, the question remains: Has quality improved? Our objective is to determine whether quality is improving based on national trends for AHRQ PSIs from 1998 to 2007.
The database used in this study was the Nationwide Inpatient Sample (NIS) from the Healthcare Cost and Utilization Project (HCUP) for the years 1998–2007. The NIS contains de-identified discharge data, including up to 15 diagnoses and procedures, from 1,045 hospitals located in 38 states, approximating a 20 percent stratified sample of U.S. nonfederal hospitals (Healthcare Cost and Utilization Project (HCUP) 2007). The NIS is the largest all-payer inpatient care database in the United States. NIS contains information from hospitals located in the United States that are open during any part of the calendar year, excluding short-term rehabilitation hospitals, that supply data to HCUP. As the NIS sampling frame changes over time, NIS Trends Supplement data were used to address the sampling changes.
The PSIs are a set of indicators providing information on inpatient preventable adverse events (PAEs) following surgeries, medical procedures, and childbirth. The PSIs were initially released in March 2003 and most recently revised in March of 2008 and now include 20 hospital-level and seven area-level indicators. The PSI software (version 3.2) was used to analyze the PSIs for the years in the study using Statistical Analysis Systems (SAS) Software (2004). While most of the PSI's definitions may be gleaned from their title, Failure to Rescue is a unique title that refers to the inability to prevent a clinically important deterioration, such as death or permanent disability from a complication of an underlying illness (e.g., cardiac arrest in a patient with acute myocardial infarction) or a complication of medical care.
All 20 hospital-level PSIs were applied to the NIS database to identify potential PAEs for each year from 1998 to 2007. The AHRQ PSI module program flags patient discharges with ICD-9-CM codes corresponding to each PSI, applies external cause of injury codes (e-codes), and calculates crude, estimated, and risk-adjusted incidence rates. The rates were adjusted for age, sex, age–sex interactions, diagnostic related group (DRG) (Centers for Medicare and Medicaid Services 2005), certain hospital characteristics, and comorbidities (Elixhauser et al. 1998) for each PSI per year. Only discharge data from patient transfers and those lacking entries for age or sex were excluded. We applied the sampling weights designated by the NIS sampling design, clustered by hospital, and stratified the data according to the NIS stratum, to make national representative estimates of the number of procedures performed each year between 1998 and 2007. The PSI software automatically includes or excludes discharges based on the criteria for each PSI and constructs the numerator and denominator for each rate. For example, men are excluded from both the numerator and denominator for calculating the rates of all obstetric PSIs. It is possible a single hospitalization may have more than one PSI. Complete definitions of the PSIs, with specific inclusion and exclusion criteria, are described in AHRQ documentation (Agency for Healthcare Research and Quality 2003a, b).
To adjust for severity of illness and the confounding effect of comorbidities, we used the Elixhauser index to quantify the effect of 30 different comorbid conditions (Elixhauser et al. 1998). It distinguishes comorbidities from complications by considering only secondary diagnoses that are unrelated to the principal diagnosis. This is achieved through the use of DRGs included in the NIS and is applied to each rate by the PSI Software. Total Elixhauser scores per patient were averaged for each PSI per year using the above-mentioned PSI software. Levene's test for homogeneity of variance was used to assess changes in variance over time.
From 1998 to 2007, longitudinal change for each PSI was calculated using a full forward Joinpoint regression algorithm based on the likelihood ratio test statistic using the R Foundation for Statistical Computing (2008) (Kim et al. 2000; Czajkowski, Gill, and Rempala 2007). Statistical significance and p-values were determined by the Monte Carlo method (Kim et al. 2000). Trends are reported as annual percent change (APC) calculated by fitting a least squares regression line to the natural logarithm of the rates, using the calendar year as the regressor variable. The terms increasing or decreasing were used when the slope of the trend segment was statistically significant from zero (two-sided p < .05). The break points for the trends over time are established by the Joinpoint regression algorithm performed by R software. All models accounted for the clustered nature, admission within year-specific hospital cluster, of the study sample. Previous study demonstrates that the NIS sampling algorithm is an accurate national estimate over time when compared to a large cohort of population states who routinely and consistently reported over the same time frame (Wiener and Welch 2007).
Five of the PSIs were not included in the results. Complications of Anesthesia have small sample size and lack of validity. Four of the PSIs were unable to be risk adjusted due to the PSI software and small sample size: death in low-mortality DRGs; foreign body left during procedure; transfusion reaction; and obstetric trauma–cesarean delivery.
From 1998 to 2007, there were approximately 69 million discharges in the United States and 7.6 million PSI events. During this time frame, the aggregate numbers of PSI rates have fallen from 454.01 per 1,000 patient admissions to 358.15 in 2007. From the 15 appropriate PSIs, 14 showed statistically significant trends (APCs) between 1998 and 2007 (Table 1). Seven PSIs showed significant increasing APCs in the time frame of 1998–2007: postoperative pulmonary embolism or deep vein thrombosis (8.94), postoperative physiological or metabolic derangement (7.67), postoperative sepsis (7.17), selected infections due to medical care (4.05), decubitus ulcer (3.05), accidental puncture or laceration (2.64), and postoperative respiratory failure (1.46).
Seven PSIs showed significant decreasing APCs during the time period: birth trauma injury to neonate (−17.79), failure to rescue (−6.05), postoperative hip fracture (−5.86), obstetric trauma–vaginal without instrument (−5.69), obstetric trauma–vaginal with instrument (−4.11), iatrogenic pneumothorax (−2.5), and postoperative wound dehiscence (−1.8). The remaining PSI showed no statistically significant trend: postoperative hemorrhage or hematoma.
The PSIs that had the highest incidence rates were obstetric trauma–vaginal with instrument with a risk-adjusted rate of 141–204 per 1,000 discharges; failure to rescue with a risk-adjusted rate of 103–155 per 1,000 (failure to rescue is death after major complication such as cardiac arrest or pneumonia); and obstetric trauma–vaginal without instrument with a risk-adjusted rate of 32–52 per 1,000. The least frequent PSIs were postoperative hip fracture with a risk-adjusted rate of 0.23–0.44 per 1,000; postoperative physiologic and metabolic derangements with a risk-adjusted rate of 0.33–0.48 per 1,000; and iatrogenic pneumothorax with a risk-adjusted rate of 0.6–0.95 per 1,000.
The year-by-year change in smoothed risk-adjusted rates is displayed in Table 2. The smoothed risk-adjusted rates differ from the rates calculated by the joinpoint analysis, which fits lines to the incidence rates per year. The smoothed rates are those calculated directly from the adjusted numerator and adjusted denominator determined by the PSI software for each PSI. The coefficient used for direct adjustment of the risk-adjusted rate is calculated by the PSI program module and is not reported here.
This study is the first to establish national trends for the incidence of PSIs over the last decade. The NIS is the largest public database of hospitalized patient discharges. PSIs have become a standard method for screening administrative data for adverse events in an inexpensive and universally replicable manner which may be employed as diagnostic exam to prioritize areas for quality improvement. Our study sought to determine whether there has been an improvement for PSI rates from 1998 to 2007. The major findings are the statistically significant increasing and decreasing trends in PSIs.
Joinpoint analysis was chosen as a method of analysis due to its ability to identify changing trends over time. Originally applied to detect changes in cancer incidence (Kim et al. 2000), the joinpoint method has more recently been used to detect changes in cohort mortality (Jemal et al. 2005). Our analysis is the first to apply this method to detect changes in trends for PSIs.
Seven PSIs showed statistically significant increasing trends (postoperative pulmonary embolism or deep vein thrombosis, postoperative physiological or metabolic derangement, postoperative sepsis, selected infections due to medical care, decubitus ulcer, accidental puncture or laceration, and postoperative respiratory failure). While these seven PSIs differ in etiology and incidence, they are all bound by three general influences: severity of illness, surveillance, and surgery. The acuity of the patient's illness at presentation, including the number of comorbid conditions present, likely contributes to the increased rates of many of these PSIs. Despite adjusting for patient age and comorbid conditions, comorbid status may influence these rates by indirect means. Given the increase in life expectancy and subsequent rise in chronic medical conditions, there has been a general increase in the severity of illness in patients (Fry et al. 2005). For example, sicker patients with extensive past surgical history who may have been excluded for surgery in the past could now be receiving surgery, which may account for increasing PSI rates. This study underscores the possibility of increasing surgical acuity with the rising PSI of accidental puncture or laceration acting as a surrogate marker for re-operative surgery. In addition, patients with more comorbidities are frequently subjected to the effects of polypharmacy (Hajjar, Cafiero, and Hanlon 2007), which may contribute to increased rates of PSI postoperative respiratory failure and PSI postoperative metabolic and physiologic derangements. In addition to polypharmacy, increased drug-resistance (Diekema, BootsMiller, and Vaughn 2004) has become more prevalent in acute care hospitals and may contribute to the increasing rates of infections and thus of PSI selected infections due to medical care and PSI postoperative sepsis. The number of admissions in U.S. hospitals increased each year (National Center for Health Statistics 2007) and, despite the adjustment for admission rates, the increased rate of admission could contribute to these increased PSI rates by other means such as increased patient/provider ratios. Although length of stay per patient has decreased (National Center for Health Statistics 2007), there has been an increase in patient turnover and a decrease in hospital staffing (American Hospital Association 2007). High patient turnover, increasing hospital patient volume, and decreased staffing may contribute to all of the increasing PSI rates due to less medical attention per patient by health care staff. The effect of surveillance must also be taken into account. Unless a condition is documented, it does not exist in this surveillance system. While accuracy is paramount in quality reporting, the assiduity of surveillance may have the perverse effect of increasing PSI rates. For example, one study found a 10-fold increase in DVT rates based on a differing surveillance technique (Haut, Noll, and Efron 2007).
Finally, it should be mentioned that of the seven significantly increasing PSIs, five are directly associated with surgery: accidental puncture/laceration, post-op physiologic and metabolic derangement, post-op sepsis, post-op pulmonary embolus or deep vein thrombosis, and post-op respiratory failure. While post-op physiologic and metabolic derangement and accidental puncture/laceration are relatively rare, the remaining three significantly increasing PSIs are unfortunately high frequency. These high-frequency, significantly increasing surgically related PSIs (post-op sepsis, post-op pulmonary embolus or deep vein thrombosis, and post-op respiratory failure) are also three areas with readily identifiable means of improvement (Encinosa and Hellinger 2008). The remaining two significantly increasing PSIs not directly related to surgery are selected infections due to medical care, which is relatively rare, and decubitus ulcer, which may be influenced by the present on admission status.
Seven of the PSIs showed statistically significant decreasing trends: birth trauma injury to neonate (−17.79), failure to rescue (−6.05), postoperative hip fracture (−5.86), obstetric trauma–vaginal without instrument (−5.69), obstetric trauma–vaginal with instrument (−4.11), iatrogenic pneumothorax (−2.5), and postoperative wound dehiscence (−1.8). All of the obstetric complications have shown decreases in rates with the greatest drop in incidence for PSI birth trauma–injury to neonate, with an annual decrease of nearly 18 percent. These rates may be influenced by the increased use of both elective cesarean section and c-section for difficult labor as well as decline in the use of VBAC (Ford, Bateman, and Simpson 2006). Regarding PSI postoperative hip fracture, increased fall precautions during hospitalization as well as increases in availability of in-house physical therapy may have helped lower the rate (von Renteln-Kruse and Krause 2007). Use of ultrasound for placement of central lines and thoracentesis may have reduced rates of PSI iatrogenic pneumothorax (Karakitsos, Labropoulos, and De Groot 2006). The impact of resident work rules upon the decline of iatrogenic pneumothorax and other PSIs is not conclusive (Poulose, Ray, and Arbogast 2005). The decrease in failure to rescue may be due to increased solicitation of DNR/DNI status from patients upon admission, thus leading to better selection of candidates for resuscitation. Another potential contributor is better training of code teams, more rested resident physicians, and perhaps the implementation of rapid response teams (Dacey, Mirza, and Wilcox 2007).
Despite the widespread use of administrative databases, there are concerns regarding the use of claims data for quality assessment. These concerns include the accuracy of reporting of primary and secondary diagnoses (Bates et al. 1995), application of external cause of injury codes (Coben et al. 2006), and the sensitivity/specificity of PSI software (Agency for Healthcare Research and Quality 2003b). There is a need to continue to research the validity of the PSIs to improve the value of screening administrative data to target areas of further patient safety investigation. However, these concerns are partly mitigated by the consistent application and definitions of the PSI software over time. Another concern regarding the use of AHRQ PSIs is over-reporting of PAEs due to the possibility of including present on admission conditions to varying degrees as a PSI (Bahl et al. 2008). However, a recent study found that inpatient PSIs alone may actually underestimate the true impact of patient safety events by 20–30 percent (Encinosa and Hellinger 2008). There may also be concern for using the NIS over time; however, since 1998 the sampling methodology has remained constant (Jemal et al. 2005).
While these PSIs can be used as a screening tool, they may contain a significant number of false-positives due to coding methods. Only preventable events with corresponding ICD-9 codes can be flagged by the PSI software. Events like iatrogenic damage to nerve or urinary tract structures, which lack ICD-9-CM codes, cannot be included in the PSI software. Thus, there is bias inherent in the PSI software toward events that have established codes. In addition, there is a general bias toward surgical events rather than medical events. Also, although principal diagnosis is frequently accurately coded in administrative data, secondary or comorbid diagnoses are often underreported (Glance et al. 2008). Thus, events due to error are difficult to separate from those due to the acuity of the patients’ conditions.
Our analysis attempts to adjust for the majority of known confounders included in the PSI software as well as additional confounders. The calculation of rates was adjusted for age, sex, age–sex interactions, comorbidities, DRG cluster, and hospital characteristics like private versus public ownership, number of beds, and teaching status. PSI software adjustment for acuity of the patient's condition is by the Elixhauser method and has been shown to be superior to other reported methods (Southern, Quan, and Ghali 2004; Farley, Harley, and Devine 2006). However, we were unable to adjust for race due to the lack of consistent reporting with a substantial amount of missing data (approximately 30 percent).
There are differences between our results and previous findings, although our study is not directly comparable to those conducted by Romano et al. (2003) or Rosen, Zhao, and Rivard (2006) because of inconsistencies in years (1995–2000 and 2001–2004) and databases used (HCUP-NIS and VA-PFT, respectively). Most notably our decreasing trend in PSI failure to rescue is in agreement with the trend reported by Rosen but not Romano. This may be due to an actual change in the rate of this PSI that occurs during the year 1998–2000 and hence results in differing trends between analyses. However, our decreasing trends in the rates of neonatal and obstetric complications are in agreement. The trend for iatrogenic pneumothorax differs between our decreasing trend and a reported increasing trend by Rosen. This may be due to differences between the VA database and the NIS as a whole, or due to differences in practice and procedure between the VA system and U.S. civilian hospitals.
Patient safety is the topic of much research and debate in the medical community, particularly due to the apparent increasing rate of deaths due to medical error and rising cost of health care overall. In October 2008, the CMS announced plans to no longer reimburse hospitals for costs associated with eight complications it deemed preventable. Three of these hospital acquired complications directly correspond to three PSIs: decubitus ulcer, foreign body left during procedure, and transfusion reaction. Four of the other CMS no-pay complications correspond to three PSIs: selected infections due to medical care, postoperative sepsis, and postoperative hip fracture. The extra cost burden of these PSIs without reimbursement will be a large motivator for increased monitoring and prevention of these complications.
One area in which improvement has already begun has been investment in technology via the electronic medical record (EMR). A variety of governmental agencies have sought to increase research in patient safety through information technology, most notably Challenge Grants as part of American Recovery and Reinvestment Act of 2009 (National Institutes of Health and Health NIo 2009). Nationwide EMR will assist in greater consistency of reporting and best practices ordering. Despite numerous efforts toward education, it appears that “hard-wiring” best practices in the EMR for the prevention of adverse events is the best method of assuring compliance (Whitman, Cowell, and Parris 2008). In addition, specification of conditions present upon admission as comorbid conditions will assist in the accuracy of discerning PAEs from natural course of illness. New York, California, and Wisconsin already require such reporting with more states to follow suit (Glance et al. 2008).
Identifying areas for improvement is at the heart of patient safety initiatives. This study helps indicate areas for either improvement or imitation for either increasing or decreasing PSIs. As pointed out by others, more resources need to be devoted to patient safety research to best determine accurate measures of harms and means of prevention or mitigation (Pronovost, Needham, and Berenholtz 2006). Not all adverse events may be preventable entirely; some PAEs may be potentially eliminated if there is process compliance. Of note, a majority of the increasing PSIs were surgically related with the most frequent of these surgically related PSIs (post-op sepsis, respiratory failure, and pulmonary embolism or deep vein thrombosis) being amenable to improvement via established processes (Institute for Healthcare Improvement 2009). A greater emphasis on making surgery safer would certainly result in a decrease in PSIs. As Romano indicated, PSIs may require ongoing validation particularly when definitions change (Romano, Mull, and Rivard 2009). It should be noted that the definition changes have generally been more restrictive, indicating that rates should decline with more stringent definitions (Bahl et al. 2008). Whatever the role surveillance may play in PSI rates, these PSI events do exist and should be addressed. Emerging technology may force the need to assess other areas of patient safety such as increasing laparoscopy diminishing the need for post-op dehiscence as a PSI but perhaps leading to another measure such as post-op wound infection after laparoscopic Roux-en-Y gastric bypass, another proposed CMS no-pay condition. The impact of public reporting and analysis is also noteworthy. While the majority of joinpoint time periods were from 1998 to 2007, some of the PSIs noted change from 1998 to 2000–2003, which coincided with the Medicare Quality Monitoring System (MQMS) Report: Patient Safety, 2000 and 2001 (http://www.mathematicampr.com/publications/PDFs/patientsafetymqms.pdf; accessed May 2011). While our study demonstrated increasing severity of illness, and risk adjustment is known to provide comfort to physician consumers of quality data, over-reliance on risk adjustment may have the un-intended consequence of simply improving documentation and not stress the importance of continuing innovative reduction in patient harm. In this large, longitudinal, and national study, AHRQ PSIs provide a roadmap for patient safety improvement.
Joint Acknowledgment/Disclosure Statement:
Additional supporting information may be found in the online version of this article:
Appendix SA1: Author Matrix.
Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.