|Home | About | Journals | Submit | Contact Us | Français|
Medications represent a major cause of harm and are costly for hospitalized patients, but more is known about these issues in large academic hospitals than in smaller hospitals.
To assess the incidence of adverse drug events (ADEs) in six community hospitals.
Multicenter, retrospective cohort study.
Six Massachusetts community hospitals with 100 to 300 beds.
From 109,641 adult patients hospitalized from January 2005 through August 2006, a random sample of 1,200 patients was drawn, 200 per site.
ADEs and preventable ADEs.
Presence of an ADE was evaluated using an adaptation of a trigger instrument developed by the Institute for Health Care Improvement. Independent reviewers classified events by preventability, severity, and potential for preventability by computerized physician order entry (CPOE).
A total of 180 ADEs occurred in 141 patients (rate, 15.0/100 admissions). Overall, 75% were preventable. ADEs were rated as serious in 49.4% and life threatening in 11.7%. Patients with ADEs were older (mean age, 74.6 years, p<0.001), more often female (60.3%, p=0.61), and more often Caucasian (96.5%, p<0.001) than patients without ADEs. Of the preventable ADEs, 81.5% were judged potentially preventable by CPOE.
The incidence of ADEs in these community hospital admissions was high, and most ADEs were preventable, mostly through CPOE. These data suggest that CPOE may be beneficial in this setting.
Medications represent a major cause of harm in hospitalized patients and were the single most frequent cause in the Harvard Medical Practice Study, accounting for 19.4% of injuries1. In another study carried out in two large academic hospitals, there were 6.5 adverse drug events (ADEs) per 100 admissions2. Of these ADEs, 28% were preventable, and 56% of preventable ADEs occurred during prescribing3.
Computerized physician order entry (CPOE) systems have been shown to be effective in reducing medication error rates2,4. A randomized study at Brigham and Women’s and Massachusetts General Hospitals in Boston demonstrated that the introduction of CPOE reduced the serious medication error rate by 55% and the preventable ADE rate by 17%2. In another study at Brigham and Women’s Hospital, CPOE reduced the non-missed dose medication error rate by 81%5. A recent review found that CPOE systems can markedly decrease the rate of serious medication errors, and improve corollary order use and prescribing behavior, especially for nephrotoxic and renally excreted drugs4.
The evidence about the benefits of CPOE spurred several groups to initiate the Massachusetts Hospital CPOE Initiative in 2005, and legislation supporting it was passed in spring 20066. The initiative is coordinated by the Massachusetts Technology Collaborative (MTC) and the New England Healthcare Institute (NEHI) in collaboration with the Massachusetts Hospital Association (MHA) and the Massachusetts Council of Community Hospitals (MCCH) and has as its goal to implement CPOE in all community hospitals across the state within 4 years7. However, nearly all of the available data about the epidemiology of ADEs and CPOE systems come only from academic hospitals, many with internally developed CPOE systems. Both the epidemiology of medication safety and the relative benefits of applications may be different in community hospitals.
We therefore conducted a baseline study of the ADE rates in six community hospitals before introduction of CPOE to assess the potential for benefits and savings. The objectives of this study were to determine the baseline rate of ADEs in six community hospitals1, to compare the ADE rates with other hospitals2, to assess the rate of preventable ADEs3, and to estimate the potential benefit of CPOE-associated ADE rate reduction4.
We conducted a retrospective cohort study reviewing charts from patients hospitalized during the study period from 1 January 2005 to 31 August 2006. The Institutional Review Boards (IRB) at Brigham and Women’s Hospital as well as the IRB at each of the study sites approved the study.
Incidents were defined as any irregularity in the process of medication use and might represent an adverse drug event (ADE), a potential ADE, a medication error (ME), or none of these8; a trigger tool with clear incident definitions was used (see "Classification of Incidents" below). A medication error was considered to be an error anywhere in the process of drug ordering, delivering, or application regardless of whether or not it harmed the patient9. An ADE was defined as an injury resulting from medical intervention related to a drug3. ADEs may or may not be preventable. Consistent with earlier studies, ADEs were considered preventable if they were due to an error or were preventable by any means available3. A potential ADE was considered to be a medication error with the potential for harm. Harm might not occur because the error was caught before it reached the patient (intercepted potential ADE) or because the error reached the patient without actually harming them (non-intercepted potential ADE).
The primary outcome measures were ADEs and preventable ADEs. As secondary outcome measurements, we report the types and severity of the events, as well as the specific approach within CPOE, which might have prevented the ADE using categories that we had used in several previous studies25–27. These categories were not mutually exclusive; we asked reviewers to select what they believed was the most relevant category. Interrater reliability for presence of an ADE, preventability, and severity were assessed using the kappa statistic.
We studied care received by persons 18 years and older who had been hospitalized at six hospitals with 100 to 300 beds in Massachusetts. The hospitals volunteered to participate in the study and were selected to be reasonably representative of small to medium-sized hospitals in the state; all were considering implementing CPOE.
Three of the hospitals had house staff and three did not. We randomly selected 200 inpatient charts for review per study site, after calculating that that would give us a robust point estimate of incidence at each site. All admitting services were included with the exception of the psychiatric and neonatal services because different identification approaches would have been more effective for those populations10,11. A random sample was generated using a random number generator in the programming language Cache. The numbers of all of the records within the observation period were entered, and the random number generator was used to select the total sample of 1,200 medical charts needed.
Incidents were identified by trained study nurses using a modified version of the Institute of Healthcare Improvement (IHI) trigger instrument as described by Rozich et al. and ADE methods developed by investigators from the Center of Excellence for Patient Safety Research and Practice12,13. As an incident trigger example, the use of the opiate antagonist naloxone would initiate the search for an opiate overdosage. We made the following modifications to the IHI tool: in addition to the C. difficile-positive stool definition, we used the term “or yeast infection related to antibiotics,” and we added as a trigger a platelet count <50,000×106/μl. We excluded the following triggers for a variety of reasons: drug level triggers for lidocaine, gentamicin, tobramycin, amikacin, vancomycin, theophylline, and “customized to individual institution.”
Trained research nurses abstracted data from the random sample described above and completed electronic data forms, which included detailed descriptions of case findings. Data from these electronic forms were then downloaded into a physician reviewer database. Each incident was independently reviewed by two physicians who were blinded to site and prescribing physician, and the incidents were classified according to type, severity, and preventability using definitions published elsewhere3,8. First, it was classified as to whether or not it was an ADE, a potential ADE, or a medication error. Incidents were only counted as medication errors in this study if an error was present, but it was not a preventable ADE or did not carry sufficient potential for harm to be considered a potential ADE. Second, incidents classified as ADEs or potential ADEs were reviewed for severity as significant (e.g., rash), severe (e.g., two-unit gastrointestinal bleed), life threatening (e.g., transfer to ICU), or fatal. Third, preventability was classified using clinical judgment as probably preventable, definitely preventable, probably not preventable, or definitely not preventable. When summarizing the results, preventability was collapsed into two categories: preventable or not preventable. Finally, for each event that was judged preventable, reviewers were asked whether or not they felt it might have been preventable using CPOE, and what specific strategy might have been likely to have prevented the ADE (e.g., drug-laboratory check). In case of disagreement regarding type, severity, preventability of the incident, or prevention strategy for preventable ADEs, the physician reviewers met for reconciliation. If consensus could not be reached, third party reviewers evaluated the incident.
The percent agreement for ADE vs. not ADE was 90.7% (kappa 0.68, 95% CI 0.62-0.74). Regarding preventability, percent agreement was 80.5% (kappa 0.44, 95% CI 0.25-0.62). Agreement and kappa statistics were lowest for severity: 75.6% discerning life-threatening vs. serious or significant (kappa 0.29, 95% CI 0.09-0.49) and an agreement of 73.2% (kappa 0.41, 95% CI 0.24-0.58) between significant vs. serious or life threatening.
All categorical variables are reported as percentages as summary statistics. For comparison of categorical variables we used the chi-square and Fisher’s exact tests; for comparison of means we used the t-test and the ANOVA procedure. To account for hospital effects, we developed a fixed effects model using Poisson regression. We conducted statistical analyses using SAS 9.1 package (SAS Institute Inc., Cary, NC).
The demographic characteristics of the general patient populations at the six study sites during the observation period were generally fairly similar among the sites (Table 1). The mean age of all patients hospitalized during the study period was 64.2 years; patients at site 1 were much older (mean age, 72.7 years) compared to sites 3 and 6 (mean age of 59.6 years each) with a highly significant difference among the sites (p<0.001). Of all patients, 58.1% were women (range 47.2% to 67.6%). Patients were predominantly Caucasian (89.9%), with Hispanic patients accounting for 2.6% and African American patients for 2.2% overall. Age, gender, and race did not differ among the sites (p=0.99). The DRG weighted length of stay (LOS) of all admissions at the study centers of 4.80 days (SD 5.35, range 0-276) was significantly higher (p<0.001) than the DRG weighted national average LOS of 4.60 (SD 3.09, range 0-42.9; DRGs adjusted to admissions of study centers).
In comparison to patients without ADEs, those who had ADEs (Table 2) were significantly older with a mean age overall of 74.6 years (p<0.001), but did not differ among the sites (means 71.4 to 84.3 years, p=0.09). Patient characteristics among sites did not differ significantly in patients with ADEs, except for LOS (DRG weighted or not, p<0.001). Almost all patients (mean 96.5%) with ADEs were Caucasian (p<0.001). Typically, patients with ADEs were admitted by either the medical or the surgical service. The DRG weighted LOS of patients with ADE was longer by 0.77 days on average (p=0.03), and most of them had Medicare (63.5%); just less than a third had private insurance (31.9%).
During the study period, there were a total of 840 incidents. These included 180 ADEs (21.4%, rate 15.0/100 admissions) and 552 potential ADEs (65.7%, rate 46.0/100 admissions, Table 3). There was no difference across the sites in the incidence rate of ADEs (p=0.28). The incidents included 108 (12.9%) medication errors without potential for harm (although we did not make an effort to identify all medication errors in this study). A large proportion of ADEs were judged preventable (75.0%; range 68.0%-86.3%). Overall, preventable ADEs occurred with a frequency of 11.2/100 admissions; there was no significant difference comparing the preventable ADEs among the sites (p=0.36). Of the 552 potential ADEs, 492 (89.1%, 41.0/100 admissions) were not intercepted. This finding showed a range of 85.3% to 97.5% across the different sites (p<0.001).
Overall, there were 180 ADEs in 141 patients at the six study sites. A total of 108 (76.6%) had one ADE, 28 patients (19.9%) had two ADEs, 4 patients (2.8%) had three ADEs, and one patient (0.7%) suffered four ADEs.
Incidents by severity are shown in Table Table44 with weighted average rates over all sites. The distribution of severity of ADEs was similar to those of the potential ADEs: many (49.4%) were judged as serious (7.0/100 admissions), though life-threatening ADEs were much less common (11.7% of ADEs, 0.02/100 admissions). One fatal ADE occurred. Of the potential ADEs, 62.7% were judged serious (26.7 /100 admissions) and 4.7% were life threatening (0.03/100 admissions). Preventable ADEs had a higher level of severity that non-preventable ADEs (p<0.001, Table 5). Almost all life-threatening ADEs (95.5%) were considered preventable, as was the only fatal ADE. Non-intercepted potential ADEs also had a higher level of severity compared to intercepted potential ADEs (p<0.001).
The leading categories of drugs causing preventable ADEs (Table 6) were cardiovascular drugs (30.4%), followed by analgesics (17.0%) and antibiotics (12.6%); anticoagulants (10.4%) and neurological drugs (8.9%) were also common. In the 45 non-preventable ADEs the distribution was different: 35.6% of the cases had antibiotics as primarily responsible, 22.2% analgesics, and 11.1% cardiovascular drugs. Anticoagulants and neurological drugs were involved in 8.8% of non-preventable ADEs each.
Evaluation of prevention strategies (Table 7) suggested that 81.5% of all preventable ADEs and 82.8% of potential ADEs would be potentially preventable using CPOE linked with decision support. The most important strategy for preventable ADEs (27.4%) was to set up rules engines that check on laboratory values during the drug prescribing procedure, followed by renal dose checking (19.3%), drug dose suggestions and drug-age checking (each 8.9%), drug allergy checking (3.7%), and drug-drug interaction checking (2.2%). For potential ADEs, the most important strategies were the cumulative drug dose checking (19.2%) and drug dose suggestions (17.2%). Renal function checking had the potential to prevent 13.4% of potential ADEs.
In this study, we evaluated the rate and type of ADEs in community hospitals, and found that the rates were fairly similar among these hospitals, but higher than those reported for large academic hospitals by about a factor of two, with about one patient in seven suffering an ADE. In addition, a much higher proportion of the ADEs were preventable in this study compared to prior studies from academic sites, with over two thirds preventable in this study, compared to less than a third in earlier studies. In addition to the ADEs, there were nearly three times as many potential ADEs. Many of the preventable ADEs appeared to be potentially preventable using CPOE, with drug-laboratory and renal dose checking being the two most important strategies.
In contrasting these results to other prior work, one helpful comparator is the ADE Prevention Study, done at two tertiary referral university hospitals in the same geographical area, which found a rate of 6.5 ADEs and 5.5 potential ADEs/100 patient admissions3. Comparisons between the results must be made with circumspection for several reasons; the studies were conducted a number of years apart, somewhat different detection approaches were used in the two studies, and the patients in the earlier study were much younger—nearly 10 years younger on average, with a mean age of 52.5 years. In particular, the triggers sought in this study were more specific, although many of the same triggers were used in the earlier work. The earlier approach also included stimulated reporting (nurse investigators solicited information from nurses, pharmacists, and clerical staff at least twice daily to report incidents) as well as spontaneous reporting, although these two categories contributed very small numbers of events relative to chart review. In the present study, there was no interaction with the clinical staff prescribing the drugs. This should have biased the results toward finding more ADEs in the ADE Prevention Study, though we found the opposite. The differences in rates could also relate to differences in presence of house staff, work flow, and education of staff, or case mix. Furthermore, the ADE rates among the six community hospitals themselves also varied to some degree, although all had higher ADE rates than in the earlier study. In a particularly relevant study, Kilbridge et al. compared the automatically detected ADE rates between a university hospital and a community hospital using a detection rules engine14. They found a 1.4 times higher ADE rate of 6.2/100 admissions at the community hospital (4.4/100 admissions at the university hospital). They also found that the type of ADE encountered differed in the two hospitals: the rates of antibiotic-associated diarrhea, drug-induced hypoglycemia, and anticoagulation-related ADEs were significantly higher at the community hospital.
In this study, about seven in ten ADEs were judged preventable and up to 97.5% of potential ADEs were not intercepted, leaving room for improvement. CPOE systems have been found to be efficacious in reducing the serious medication error rate by half, and early versions reduced the preventable ADE rate by almost one fifth2,4. Better performance might be expected with additional decision support15,16. In a systematic review that evaluated the effect of CPOE with CDS on reducing ADEs, five out of ten studies showed a significant reduction of ADEs17.
The drug classes we found to be most often involved in preventable ADEs were cardiovascular drugs, analgesics, and antibiotics. Bates et al. found in the 1995 study that the drug classes responsible for preventable ADEs were 29% analgesics, 10% sedatives, and 9% antibiotics3. Regarding non-preventable ADEs, analgesics and antibiotics led the list with 30% each in that study. Thus, in this study, cardiovascular drugs appeared more prominent as a cause of preventable ADEs than in the prior work; this may well reflect to some extent the common use of these drugs in the elderly.
In the ADE Prevention Study, wrong dose errors were most frequent, followed by wrong choice, known allergy, wrong frequency, and drug-drug interaction18. In contrast, in this study, drug-laboratory issues were most frequent regarding ADEs, and dosing issues, allergy issues, and drug-drug interactions were much less important. It is unclear why these profiles are so different, though it is possible that more drug-allergy and drug-drug interaction issues are being detected in the pharmacy than at that time. However, for strategies for potential ADEs, drug dose interventions were the most important, followed by renal function checking. The potential of strategies using CPOE and the laboratory has been discussed previously, especially with respect to the importance of linking laboratory to drug data19. Our study shows that checks involving laboratory data (laboratory-drug and renal checks taken together) could reduce preventable ADEs by 46.7%. Hulse et al. found a very similar rate of 44.9% drug-laboratory issues in patients with potential drug problems20. Schiff et al. have described ten ways how laboratory parameters can help prevent medication errors such as suggesting contraindication of specific drugs, alteration of dosage and titration, as well as signaling signs of toxicity19.
Kappa statistics for inter-rater reliability using this methodology in prior studies have been shown to range from 0.81 to 0.98 for the presence of an ADE and 0.92 for preventability; kappa was lower for decisions regarding severity (κ=0.32 to 0.37)3. Inter-rater reliability in this study was similar to those earlier studies. In general, the range of kappa may vary substantially in rating incidents from 0.32-0.988.
The incidence of serious medication errors can be significantly reduced using CPOE systems, but implementing these systems is very costly. Because CPOE is so expensive, the return on investment for implementing it is of great public and policy interest. A recent study done by Kaushal et al. showed that in the case of a 720-bed tertiary referral teaching hospital over 10 years cumulative net savings of $28.5 million are met by a $16.7 million net operating budget resulting in savings of $9.5 million, although it is unclear how these results would translate to the community setting21. Another point is the reduction of LOS by preventing ADEs. In our study, patients with ADEs stayed 0.77 days longer in the hospital compared to patients without ADEs. This is only about half of what others have observed: Classen et al. found an excess length of hospital stay attributable to ADEs of 1.74 days, and Bates et al. found an increase of 2.2 days22,23. The difference may be partly explained by the study design (both of these studies used case controls); hence, the estimate of increase of LOS in this study may be conservative.
Our study has several limitations. This study was carried out in just six hospitals in one region, and they may not be representative of other community hospitals in other regions. Although random sampling with trigger tools is a widely acknowledged methodology for measuring ADE incidence in hospitals24, it almost certainly does not identify all ADEs occurring at a test site, and the ADEs found are probably not a random sample of all ADEs. However, we felt it was the most efficient way of detecting ADEs in the short term in institutions without CPOE or an EMR in place. We did not assess the reliability of nursing detection of signals in this evaluation, and some ADEs were undoubtedly missed. In some of the study sites CPOE systems were about to be implemented, which might have caused a higher awareness of the medical staff regarding drug safety issues as compared with to other sites. The study was intended to assess the incidence of ADEs in a group of community hospitals, and we did not intend to compare the hospitals with each other and had limited power to detect differences in rates between the hospitals.
In summary, we found that the incidence of ADEs in community hospitals was high, higher than rates in large academic hospitals measured previously, and a larger fraction of ADEs were preventable, although these comparisons must be made with circumspection given the differences between the cohorts. These data suggest that implementation of CPOE systems in community hospitals is likely to be beneficial if the benefits achieved in this setting are similar to those found in academic settings, which remains to be determined.
We thank Jason Lee, research assistant, for building the Access Database and the study nurses, Kris Martel-Waldrop, Cathy Foskett, Mary-Clare Hickey, Theresa McNeil, and Martha Vander Vliet, all RNs, for collecting data at the multiple study sites.
Conflict of Interest Disclosure Statements The Massachusetts Technology Collaborative supported the study. They commented on its design, but were not involved in collection, management, analysis, or interpretation of the data. They did approve the manuscript. Dr. Bates had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr. Bates is a co-inventor on Patent No. 6029138 held by Brigham and Women’s Hospital on the use of decision support software for medical management, licensed to the Medicalis Corporation. He holds a minority equity position in the privately held company Medicalis, which develops web-based decision support for radiology test ordering, and serves as a consultant to Medicalis. He is a consultant for Cardinal Health, which makes intravenous drug delivery systems. Dr. Hug has received financial funding from the Freie Akademische Gesellschaft, the Walter and Margarethe Lichtenstein Fund, and the University Hospital in Basel, Switzerland.
This study was funded by the Massachusetts Technology Collaborative, which is not responsible for the contents of the manuscript.