PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Am Geriatr Soc. Author manuscript; available in PMC 2009 February 1.
Published in final edited form as:
PMCID: PMC2633588
NIHMSID: NIHMS65928

Types, Prevalence, and Potential Clinical Significance of Medication Administration Errors in Assisted Living

Heather M. Young, PhD, GNP,* Shelly L. Gray, PharmD, MS, Wayne C. McCormick, MD, MPH, Suzanne K. Sikma, PhD, RN,§ Susan Reinhard, PhD, RN,|| Linda Johnson Trippett, RNC, MSN,§ Carol Christlieb, RN, MSN,* and Tiffany Allen, BS*

Abstract

OBJECTIVES

To describe the types and potential clinical significance of medication administration errors in assisted living (AL).

DESIGN

Cross-sectional observational study.

SETTING

This study was conducted in 12 AL settings in three states (Oregon, Washington, and New Jersey).

PARTICIPANTS

Participants included 29 unlicensed assistive personnel and 510 AL residents.

MEASUREMENTS

Medication administration observations, chart review, and determination of rates, types, and potential clinical significance of errors using standardized methodology.

RESULTS

Of 4,866 observations, 1,373 errors were observed (28.2% error rate). Of these, 70.8% were wrong time, 12.9% wrong dose, 11.1% omitted dose, 3.5% extra dose, 1.5% unauthorized drug, and 0.2% wrong drug. Excluding wrong time, the overall error rate dropped to 8.2%. Of the 1,373 errors, three were rated as having potential clinical significance.

CONCLUSION

A high number of daily medications are given in AL. Wrong time accounted for the majority of the errors. The bulk of the medications are low risk and routine; to promote optimal care delivery, clinicians need to focus on high-risk medications and residents with complex health problems.

Keywords: assisted living, medication administration, unlicensed assistive personnel, quality, medication errors

Assisted living (AL) settings increasingly serve persons who previously would have lived in a skilled nursing facility (SNF), offering a more-homelike and less-restrictive residential option at lower cost.14 With higher acuity, many AL residents have multiple chronic health conditions, increasing the need for ongoing monitoring and management. Core values in AL include choice, privacy, independence, and the potential to age in place.3,5,6 Although ALs cost less and provide less-institutional environments, these elements are offset by less professional oversight and the potential risk for negative health outcomes because of the need for more intensive health monitoring. A critical issue in AL is striking the appropriate balance between freedoms and risks for residents, staff, and the settings themselves.7

Nationwide, AL residents are a frail, older population with multiple chronic health problems. More than half are aged 85 and older, approximately 25% have significant cognitive impairment, and 77% receive help with medications. 2 Estimated medication use ranges from 3.8 to 6.2 regularly scheduled daily medications,811 higher than average medication use by older adults in the community (average 2.7–3.9) and lower than use in SNFs (average 8.9).12 Furthermore, there is a high prevalence of the use of several groups of high-risk medications in AL (e.g., psychotropics, cardiovascular agents, anticoagulants).811,13 In addition to multiple comorbidities and complicated medication regimens, this population experiences age-related changes in pharmocokinetics and pharmacodynamics that increase the potential for adverse drug events (ADEs).12,14

Unlike SNFs, where licensed nurses are engaged in medication administration, medications in AL are managed primarily by unlicensed assistive personnel (UAP), with limited professional involvement.15,16 Compounding the issue are high UAP turnover in AL settings17 and few structures and processes to monitor quality in medication management. 1820 The role of the registered nurse (RN) in AL is evolving in relation to AL regulations and nurse practice acts.7,15,16,21 Medication management is one of the three top quality-of-care concerns in AL.18 Medication safety research in AL is in an early stage, with limited information about medication profiles10,22 and error rates.23

Medication administration errors are defined as “any difference between what the patient received or was supposed to receive and what the prescriber intended in the original order.”24 The most widely accepted method of assessing medication errors is a standardized approach, including direct observation of medication administration, chart review of medication orders, and identification of discrepancies between observed and expected administration. 25,26 Using this method, observers are able to identify errors that escape the awareness of the person administering the medication, and the effect of the observer on the observed has been determined to be insignificant.27 One study25 compared methods and concluded that observation with chart review is the most sensitive approach; of 2,557 doses observed in hospitals over 7 months, investigators detected 300 errors using the observation method, 34 using chart review, and one according to incident report, yielding error rates of 11.7%, 0.7%, and 0.04%, respectively.24 Using the observational and chart review method in a study of 36 hospitals, the overall error rate of licensed staff was reported as 19%.26

Observational research of medication administration errors in long-term care is in its earliest stages. Using the method described above, one study28 compared error rates of nursing home personnel of different levels and reported the following error rates: RN, 34.6%; licensed practical nurse (LPN), 40.1%, and certified medication aides, 34.2%. In the only study of medication errors in AL, the overall rate was 3.6%,23 although this study used a limited methodological approach that consisted only of observations and did not include chart review to identify intended prescription, thus likely underestimating the error rate. The purpose of this article is to describe the types, prevalence, and potential clinical significance of medication administration errors made by unlicensed personnel in AL.

METHODS

This study was conducted in 12 AL settings in three states (Oregon, Washington, and New Jersey). Because AL is regulated at the state level, with significant variation in policies and resident mix, these states were selected to include differing models of delivery and levels of licensed staff involvement. 16 Inclusion criteria for staff were working on day or evening shift (when most medications are given) at least 20 hours/week and able to speak English.

Data sources for this study were observation of medication administration by staff and chart review of residents observed. The only AL resident identifiers collected were those necessary to connect the observations with the chart data. These identifiers were eliminated as soon as the two sources of data were matched. Informed consent was obtained from the medication aides (observed giving medications). As part of institutional review board approval process, a Waiver of Consent and Authorization (Health Insurance Portability and Accountability Act) for resident data was procured.

Data Collection and Analysis Procedures

The unit of analysis for this study was each medication administered. The sample goal was 400 observations per facility, for a total of 4,800 observations. Medication administration observations and record reviews were conducted using well-established standardized methods.24,25 The observers were clinical nurse specialists trained in the method and experienced with student supervision of medication administration. UAPs were informed that at any time they could ask the observer to step back should the observation interfere with resident care or comfort in any way. The observer accompanied the UAPs during medication administration on day and evening shifts, observing the preparation and administration of each dose, recording all UAP actions, as well as each medication and dose delivered, along with the time of delivery on a standardized data collection form. Related procedures (e.g., hand washing, checking pulse) were also recorded. After the observation, independent observers reviewed resident records and recorded prescriber’s orders using a standard data collection form. Interrater reliability was established according to two shared observations of an entire medication pass.

Calculation of Error Types and Rates

Observations of medication administration were compared with prescriber’s orders. Scoring was conducted according to established guidelines, with any deviation between prescriber’s orders and medication administered (including omitted doses) recorded as an error and categorized according to definitions of wrong time, wrong dose, unauthorized drug, extra dose, omitted dose, wrong form, wrong route, and wrong technique.24 Resident refusals were not counted as administration errors. Medication error rates were calculated by dividing the number of errors by the sum of the number of doses given plus the number of omissions and then multiplying by 100.

Determination of Potential Clinical Significance of the Errors

A geriatrician, a nurse practitioner in long-term care practice, and a pharmacist with specialized training in geriatrics, who were given written instructions for evaluation of medication errors, evaluated all errors for potential clinical significance. Two methods were used to determine the potential clinical significance of the medication errors. First, the methodology developed previously29 for classification of medication errors and ADEs was used, because many consider it to be the criterion standard.30 The panel of reviewers independently determined whether each medication error identified could theoretically result in a potential ADE, and if yes, they determined the theoretical severity of the potential ADE (significant or not), yielding a score of 0 or 1 (0=no potential ADE or potential but not significant ADE; 1= potential significant ADE). Examples of significant potential ADEs were provided in the training instructions (e.g., rash, diarrhea due to antibiotics, nausea resulting from oral potassium). For any discrepant ratings, the panel discussed and reached consensus. A conservative definition was used, assuming that the error was not repeated—ratings were made based on the assumption that it was a one-time error.

The methodology described above typically relies on access to clinical information for making decisions about whether an error was a potential or actual ADE. Given this, it is possible that the clinical significance would be under-estimated in this setting. Thus, the clinical significance of errors was also rated using methods that were used to examine the clinical significance of observations of dispensing errors by pharmacists.31 As a group, the reviewers determined for each medication error the potential to cause harm by assigning a score of 1 (important) to 5 (not important) and the likelihood of causing harm by assigning a score of 1 (very likely) to 5 (very unlikely). These two scores are summed, with a total score of 6 or less considered to be a clinically important error.

Statistical Analysis

Data were analyzed using SPSS version 15.0 (SPSS Inc., Chicago, IL). Facility characteristics were summarized using descriptive statistics. Frequencies of error types were calculated according to facility and state. Interrater reliability of the independent ratings of clinical significance was assessed using the intraclass correlation coefficient for each error type.32

RESULTS

Sample

Study sites included 12 AL settings, with geographic and organizational diversity and capacity ranging from 42 to 124 apartments, with an average size of 93 apartments (Table 1). Unlicensed staff were responsible for giving the medications to the residents in all settings. There were several differences across states, with the settings in New Jersey having a significantly lower percentage of Medicaid clients, Oregon having significantly lower annual resident turnover, and Washington having significantly higher staff turnover.

Table 1
Facility and Staffing Characteristics

Policies regarding medication management and nurse delegation varied across states (Table 2). Although Oregon is among the states with the frailest residents in AL, the regulations are the most permissive of the three states, with no requirement for certification or registration for AL medication aides and no state-required training. In Oregon, the facility and the RN are responsible for assuring on-the-job training for oral and topical medication administration. The RN must delegate insulin administration and blood glucose testing. In Washington, aides who receive delegation must be registered and complete state-required training. RNs delegate oral and topical medication administration, and UAPs are not allowed administer insulin. Finally, in New Jersey, medication aides are certified and complete state-required training, and RNs delegate administration of all medications.

Table 2
Medication Management Policies in Assisted Living Across States

Seventy-five percent of the settings used a corporate wholesale pharmacy for medication dispensing; the rest had contracted relationships with small local pharmacies. In Oregon and Washington, medications were most commonly dispensed in bubble-packed single-dose packages, and in New Jersey, multidose packaging including all medications given at a specific time was in higher use. All settings had consulting pharmacists, but regulations did not require intensive pharmacy review, nor was reimbursement for this service in place in any of the states. All states required that medications be locked and that residents be assessed for capacity to self-administer. Settings enacted the assessment in various ways, from standard tools to observations and interviews with residents. In Oregon, the medication aides used multidose trays to deliver the medications, and in Washington and New Jersey, medication carts were taken to the dining room or used in the rooms where medications were stored for delivery. One facility in Washington locked individual resident medications in each resident apartment, and aides delivered medications in these locations.

The medication aides (n=29) ranged in age from 18 to 57, with an average age ± standard deviation of 37.1 ± 12.6, and 96.6% were female. The sample included one African-American, one Native-American, one Asian-American, three Hispanic, and 15 Caucasian aides (eight did not disclose their ethnic background). Mean education level was 12.1 ± 1.4 years, with a range of 8 to 16. Forty-four percent received all their training on the job, 13.8% had attended a medication administration course, and 37.9% were certified nursing assistants.

For the 510 residents observed, the mean number of medications ordered was 13.1 ± 6.0, with 9.6 routine and 3.5 as-needed medications and a range of 2 to 34 medications per resident. This count included all prescribed medications given by staff to the residents, including vitamins and eye drops. This count did not include over-the-counter medications that residents had in their own rooms. Across settings, an average of 83.5% of residents were receiving assistance from staff with administration of medications.

Medication Observations

Across the 12 settings, 29 aides were observed over 56 medication passes on day and evening shifts, giving medications to 510 residents. In total, 4,866 medications were given, and all 510 resident records were reviewed for written orders. The number of medications given during a medication pass varied according to number of medication aides on duty, numbers of residents assigned, and time of day, from a low of 10 medications given to three residents during an evening medication pass, to a high of 375 medications given by one aide to 49 residents during a morning medication pass. All staff remained in the study during the observation period, and none requested that the observer withdraw from a resident care situation.

Medication Administration Errors

Figure 1 summarizes the overall error rates according to facility and in total. Facility policies indicated a 2-hour window for administering medications (1 hour before and after the scheduled time). With this strict definition, the overall number of errors observed was 1,373 of 4,866 total observations, for an average error rate of 28.2% (range 13.3–39.9%). Without including wrong time, the average error rate dropped to 8.2% (range 2.6–15%).

Figure 1
Medication error rates across facilities. NJ = New Jersey; WA = Washington; OR = Oregon.

Of the five types of errors, time was the most common error (medications given more than 1 hour before or after scheduled time), at 70.8%. The remaining error percentages in descending order were wrong dose (12.9%), omitted dose (11.1%), extra dose (3.5%), unauthorized drug (1.5%), and wrong drug (0.2%).

Potential Clinical Significance of Errors

After quantification and classification of the errors, each error was examined for its potential to cause harm to residents using a method previously identified.29 Errors were rated for potential for causing an ADE and potential for clinical harm. The clinicians completed an initial independent review, yielding high interrater reliability according to error type. Intraclass correlation coefficients were 0.98 for unauthorized drug, 0.88 for extra dose, 0.91 for omitted drug, 0.95 for wrong dose, and 0.99 for wrong time, all significant at the P<.001 level. After independent review, the three clinicians discussed differences and achieved 100% consensus on the significance of the errors. Using this approach, four errors (2 wrong dose and 2 unauthorized drugs) were rated as having potential for causing an ADE; four of these were rated as having potential for clinical harm.

Next, using the other method, errors were analyzed for potential clinical significance. Of the 1,373 errors, three were rated as having potential clinical significance (scored <6). The vast majority (1,361) were scored higher than 8. Table 3 displays all errors rated less than 8; as can be seen from this table, the most common medications rated as problematic are those that commonly undergo changes in dose according to patient parameters (e.g., laboratory values, symptoms). None of these errors were related to time.

Table 3
Comparison of Clinical Significance of Errors Scored Less Than 8 Using Two Methods

DISCUSSION

Medication management in AL settings is a prevalent and time-consuming process, with a high number of daily medications being given to residents with significant frailty. Because the goal of the study was to examine medications administered, all ordered medications that were dispensed by a pharmacy for UAPs to give to residents were included in the count. This could explain why the total number of medications was higher than previous reports in the literature, because it included such preparations as vitamins and eye drops for comfort. The total did not include additional medications or supplements taken independently by residents. This study represents the first reported use in AL of the standardized observation method developed previously. 25 Using the strict definitions outlined in the protocol, the overall error rate was 28.2%, compared with 19% in hospitals and 34.2% to 40.1% in SNFs.28 When time errors were removed from analyses in all settings, the error rate was 8.2% in AL, 10% in hospitals, and 7.4% in SNFs. Wrong time accounted for the bulk of the errors in AL (70.8%), compared with a 43% wrong-time rate observed in hospitals. Despite the high rate of wrong-time errors, none of these was deemed clinically significant. In other words, staff ensured that time-sensitive medications, such as insulin, were given according to the schedule, but were less rigid about delivering other medications at the appointed time.

The relevance of time is important when comparing error rates with those of hospitals, because AL settings are less regimented and more likely to incorporate resident preferences into care delivery. As other authors have asserted, 28 wrong-time errors can relate to systems issues such as staffing levels and facility policies, factors beyond the control of the person administering the medication. The bulk of the medications given are not time sensitive, and aides face the daunting challenge of administering a large quantity of medications in a short time frame, yet all the settings had policies that medications must be given within 1 hour of the scheduled time. Furthermore, most had standardized morning and evening medication pass times and did not customize the schedule according to resident preference or staff load. Under this scenario, a high number of wrong-time errors is not surprising, and given the lack of clinical significance of the errors observed, is probably not a meaningful indicator of quality. The practice of scheduling so many medications at once creates an artificial compression of time that could contribute to errors. Facility staff and regulatory agencies may consider liberalizing the scheduling of medications in this setting, focusing on clinical relevance, consistency, and resident preferences rather than on an arbitrary time goal. RNs in AL are well positioned to address medication scheduling, taking into account medication parameters (such as with meals), resident preferences, and staff delivery logistics.

None of the observed errors were rated highly likely to cause harm. Two methods were used to evaluate the clinical significance of errors. Using the first approach, four errors were rated as having potential for harm. Using the second approach, three errors were identified as having potential clinical significance, suggesting that the first method is more sensitive to errors, whereas the second provides a more-detailed rating. Two of the three errors rated as having potential clinical significance were unauthorized medications (i.e., not having an order for diazepam and insulin). The most interesting finding regarding clinical relevance was that errors occurred more commonly with higher-risk medications (e.g., insulin, warfarin, furosemide) associated with residents in less stable and predictable conditions. This is consistent with a recent audit of medications most commonly involved in errors in SNFs, implicating the drugs lorazepam, warfarin, insulin, hydrocodone, and furosemide. 33 A recent study of the risk of emergency department isits for ADEs that identified warfarin, insulin, and digoxin as responsible for 33.3% of these visits also supported the findings of the current study. The risk of ADEs leading to an emergency department visit was 35 times as high for these three medications as for medications always considered to be inappropriate according to the Beers criteria.34

Dosage changes, in response to symptoms or laboratory results, are more common with several high-risk drugs involved in errors, such as warfarin, furosemide, and insulin. It is likely that the furosemide and warfarin sodium errors were related to communication problems of getting the changes in dose transcribed or communicated to the person giving the medication. Systems could be improved to expedite communication about changes in regimen between primary care providers, pharmacies, and AL staff to promote timely transcription, ordering, dispensing, and administration. For the insulin errors, the errors were related to sliding-scale doses, which involved adding a standing dose to a sliding-scale dose. Given the low educational preparation of the medication aides, strategies to address mathematical errors could be employed.

There were several limitations to this study. First, the sample was a convenience sample of settings in three states. A larger randomly selected sample would strengthen the study. Because this was a cross-sectional observational study, it was assumed that the errors were one-time errors only, yielding a conservative interpretation of the clinical significance. An important next step would be to determine the rate at which errors are repeated, or not corrected, and to analyze the clinical significance of such oversights. This study did not examine self-administration of medications in AL, a practice of fewer than 20% of residents in the study settings. Future work could examine relative error rates of unlicensed staff versus resident self-administration. Finally, the effect of the observer on error rates is a common concern, although observers who were trained to minimize their effect observed participants in as unobtrusive way as possible during highly familiar routines. The methodologists who developed this approach have examined the effect of the observer and have concluded that there is minimal influence.27 In addition, direct observation as a common approach to measuring quality in long-term care was evaluated previously and the conclusion reached that observations were stable and did not seem to change behavior appreciably.35

In conclusion, this study supports the assertion that medication management is a salient clinical concern in AL, with a large quantity of medications given daily and a high level of resident dependency on staff for this service. UAPs generally do remarkably well with this complex task, given their level of training and preparation. The bulk of the medications are low risk and routine, and the risks appear to be minimal. Several higher-risk medications were identified, with the implication that attention to delivery of these medications should be prioritized to promote targeted prevention. Professional clinical attention, which is in limited supply in AL, could be targeted more strategically to promote optimal care delivery. For example, with the high number of medications ordered, clinicians could focus on high-risk residents and high-risk medications, the appropriateness of the medication regimen, and the efficacy of the pharmacological approaches, particularly for residents with multiple comorbidities.36 Health information technology is limited in AL, particularly linking settings to pharmacies and primary care providers, yet technology could enhance accuracy and timeliness of communication about medication management throughout the process, from prescribing through administering. Nurses in AL play multiple roles and have the potential to enhance the health of the community of residents by identifying residents at higher risk and prioritizing assessment and resident and staff education to promote optimal outcomes from the medication program.

ACKNOWLEDGMENTS

The authors wish to express deep appreciation to Ginette Pepper, RN, PhD, GNP, for methodological consultation; to the facility staff and residents in Oregon, Washington, and New Jersey who generously participated in this study; to Gail Maurer, PhD, and Sandra Howell-White, PhD, for project direction; to Sue MacDonald, ARNP, for analysis of clinical significance; to Juliana Cartwright, PhD, RN, and Glenise McKenzie, PhD, RN, for their thoughtful critique; and to George Knafl, PhD, for statistical consultation.

Conflict of Interest: The editor in chief has reviewed the conflict of interest checklist provided by the authors and has determined that the authors have no financial or any other kind of personal conflicts with this manuscript. This study was funded by National Institute of Nursing Research Grant R21 NR009102-01, Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, and the Robert Wood Johnson Foundation.

Sponsor’s Role: The funding agencies had no input into the design, methods, subject recruitment, data collections, analysis, or preparation of the manuscript.

Footnotes

This article was presented at the Annual Meeting of the Gerontological Society of America, November 2006, Dallas, Texas.

REFERENCES

1. Gelhaus L. More reasons to stay. Increasing services such as medication management and extra assistance with ADLs leads assisted living providers into uncharted territory. Provider. 2001;27:18–28. [PubMed]
2. DHHS. High service or high privacy assisted living facilities, their residents and staff: results from a national survey [on-line] [Accessed August 25, 2003]. Available at http://aspe.hhs.gov/daltcp/reports/hshpes.htm.
3. DHHS. Residents leaving assisted living: descriptive and analytic results from a national survey [on-line] [Accessed August 26, 2003]. Available at http://aspe.hhs.gov/daltcp/reports/alresid.htm.
4. AARP. Assisted Living in the United States [on-line] [Accessed August 25, 2003]. Available at http://research.aarp.org/il/fs62r_assisted.html.
5. Mollica R. State Assisted Living Policy: 2000. Portland: National Academy for State Health Policy; 2000.
6. Chapin R, Dobbs-Kepper D. Aging in place in assisted living: Philosophy versus policy. Gerontologist. 2001;41:43–50. [PubMed]
7. Sikma S, Young HM. Balancing freedom with risks: The experience of nursing task delegation in community-based residential care settings. Nurs Outlook. 2001;49:193–201. [PubMed]
8. Armstrong E, Rhoads M, Meiling F. Medication usage patterns in assisted living facilities. Consult Pharmac. 2001;16:65–69.
9. Garrard J, Cooper S, Goertz C. Drug use management in board and care facilities. Gerontologist. 1997;37:748–756. [PubMed]
10. Spore DL, Mor V, Larrat P, et al. Inappropriate drug prescriptions for elderly residents of board and care facilities. Am J Public Health. 1997;87:404–409. [PubMed]
11. Sloane PD, Zimmerman S, Brown LC, et al. Inappropriate medication prescribing in residential care/assisted living facilities. J Am Geriatr Soc. 2002;50:1001–1011. [PubMed]
12. Hanlon J, Ruby C, Shelton P, et al. In: Pharmacotherapy: A Pathophysiological Approach. DiPiro J, Talbert R, Yee G, Matzke G, Wells B, Posey L, editors. Stamford: Appleton & Lange; 1999.
13. Gruber-Baldini A, Sloane P, Zimmerman S, et al. Medication undertreatment in assisted living settings. Arch Intern Med. 2004;164:2031–2037. [PubMed]
14. Kane R, Ouslander J, Abrass I. Essentials of Clinical Geriatrics. New York: McGraw-Hill; 2003.
15. Mitty EL. Assisted living and the role of nursing. Am J Nurs. 2003;103:32–43. [PubMed]
16. Reinhard S, Young HM, Kane RA, et al. Nurse delegation of medication administration for older adults in assisted living. Nurs Outlook. 2006;54:74–80. [PubMed]
17. Mezey M. Nurse staffing in assisted living facilities. Am J Nurs. 2003;103:98. [PubMed]
18. US General Accounting Office. Assisted Living: Quality-of-Care and Consumer Protection Issues in Four States. (GAO Publication No. HEHS-97–93) Washington DC: US Government Printing Office; 1999.
19. Committee on Improving Quality in Long-Term Care. Improving the Quality of Long-Term Care. Washington, DC: National Academy Press; 2001.
20. Aud M, Rantz M. Quality concerns in assisted living facilities. J Nurs Care Qual. 2004;19:8–9. [PubMed]
21. Reinhard S, Young H, Kane R, et al. Final Report: Nurse Delegation of Medication Administration for Elders in Assisted Living. Baltimore: American Nurses Foundation; 2003.
22. Sloane P, Zimmerman S, Brown L, et al. Inappropriate medication prescribing in residential care/assisted living facilities. J Am Geriatr Soc. 2002;50:1001–1011. [PubMed]
23. Hyde J, Segelman M, Feldman S, et al. Medication management in Massachusetts assisted living settings. Consult Pharm. 1998;9:1001–1014.
24. Flynn E, Barker K, Pepper G, et al. Comparison of methods for detecting medication errors in 36 hospitals and skilled-nursing facilities. Am J Health Syst Pharm. 2002;59:436–446. [PubMed]
25. Barker K, Flynn E, Pepper G. Observation method of detecting medication errors. Am J Health Syst Pharm. 2002;59:2314–2316. [PubMed]
26. Barker KN, Flynn E, Pepper G, et al. Medication errors observed in 36 health care facilities. Arch Intern Med. 2002;162:1897–1903. [PubMed]
27. Dean B, Barber N. Validity and reliability of observational methods for studying medication administration errors. Am J Health Syst Pharmacy. 2001;58:54–59. [PubMed]
28. Scott-Cawiezell J, Pepper GA, Madsen RW, et al. Nursing home error and level of staff credentials. Clin Nurs Res. 2007;16:72–78. [PubMed]
29. Bates D, Boyle D, Vander Vliet MM, et al. Relationship between medication errors and adverse drug events. J Gen Intern Med. 1995;10:199–205. [PubMed]
30. Morimoto T, Gandhi T, Seger A, et al. Adverse drug events and medication errors: Detection and classification methods. Qual Saf Health Care. 2004;13:306–314. [PMC free article] [PubMed]
31. Flynn E, Barker K, Carnahan BJ. National observational study of prescription dispensing accuracy and safety in 50 pharmacies. J Am Pharm Assoc. 2003;43:191–200. [PubMed]
32. Maclure M, Willett WC. Misinterpretation and misuse of the kappa statistic. J Epidemiol. 1987;126:161–169. [PubMed]
33. Hansen RA, Greene SB, Williams CE, et al. Types of medication errors in North Carolina nursing homes: A target for quality improvement. Am J Geriatr Pharmacother. 2006;4:52–61. [PubMed]
34. Budnitz D, Shehab N, Kegler S, et al. Medication use leading to emergency department visits for adverse drug events in older adults. Ann Intern Med. 2007;147:755–765. [PubMed]
35. Shnelle J, Ouslander J, Simmons S. Direct observations of nursing home care quality: Does care change when observed? J Am Med Dir Assoc. 2006;7:541–544. [PubMed]
36. McCormick W, Boling P. Multi-morbidity and a comprehensive Medicare care-coordination benefit. J Am Geriatr Soc. 2005;53:2227–2228. [PubMed]