|Home | About | Journals | Submit | Contact Us | Français|
Recent studies have reported that epilepsy and seizures are common in nursing homes. Prevalence has been reported to range from 5-9% and antiepileptic drug (AED) use is even more common. Most of these studies have relied on various forms of nursing home records, but the validity of this source data, while assumed, has not been verified. This study evaluated the degree of agreement between the Minimum Data Set (MDS), both paper and electronic versions, and actual medical records available at the nursing home. Records of 144 residents were evaluated; agreement between paper and electronic versions of the MDS was 97.8%. Agreement between the paper version of the MDS and neurologists review of the nursing home record was 92.3%. However, the criteria for diagnosing epilepsy or seizure were not well documented. Nevertheless, the agreement among nursing home records, paper MDS and electronic MDS is great enough to allow the electronic MDS to be used as a research tool, but more investigation of the actual criteria used by nursing home physicians in diagnosing epilepsy and seizures is necessary.
1.1.With the leading age of the “baby boom” generation nearing 65 years of age, the number of elderly is poised to soar as this cohort ages. For persons aged 65 or older, the lifetime risk of becoming a nursing home resident is high, with some reports giving a figure as high as 43 to 46% (Kemper and Murtaugh1991; Liu et al 1994; Spillman and Lubitz, 2002). At any point in time, 4.5% of the elderly population in the U.S. is residing in NHs (Hetzel and Smith, 2001). Although there are relatively few studies of epilepsy and antiepileptic drug (AED) use in NHs, AED use is approximately 10% of all residents (Lackner et al. 1998; Schachter et al. 1998; Garrard et al. 2000). PHT is the most widely used AED in the USA with approximately 6% of all nursing home residents receiving it (Garrard, Cloyd et al 2000; Lackner et al. 1998; Schachter et al. 1998). Validity studies are needed to assure reliability of observations and reporting (Landis and Koch 1977). While these studies have relied on various forms of documentation, the validity of the source documents have never been carefully validated for epilepsy. As part of a National Institutes of Health sponsored grant on epilepsy in the elderly, we were able to obtain a computerized form of the Minimum Data Set (MDS) from Beverly Enterprises. The MDS is a standardized, clinical resident assessment instrument which is federally mandated for all residents of Medicare and Medicaid certified nursing facilities. A full MDS assessment is completed for each resident regardless of source of payment upon admission, and at least annually thereafter. Since 1998, the Health Care Financing Administration (HCFA) has required that MDS assessments be collected and stored as computerized records. This electronic data set, which includes information on epilepsy and seizure disorders, could provide a valuable source of clinical information on the incidence and prevalence of epilepsy and seizure disorders among nursing home elderly. Although it has been used and validated for other conditions (Abicht-Swensen, L. M. and Debner, 1999; Hartmaier, et al 1995; Hawes et al. 1995), the validity of information specific to epilepsy or seizure disorders documented in the MDS has not been published.
Our study had the following objectives: 1. to determine the validity of the paper MDS by calculating the extent to which documentation of epilepsy or seizure disorder on the paper MDS agreed with a neurologist's review of the nursing home chart (external criterion or gold standard); and 2. to determine agreement between the paper MDS assessment forms and their computerized counterparts.
2.1 Eleven nursing homes managed by Beverly Enterprises Inc. were selected; one as a pilot site in which to test protocols for data collection and study procedures and ten for the study. A complete list of elderly residents (65 years of age or older) compiled by nursing home staff, contained the nursing home record number, date of birth, date of admission, nursing home identified epilepsy or seizure disorder (yes or no) and current AED use (yes or no) of each elderly resident. Records for review were selected from each nursing home's list in three steps:
2.2 At each nursing home, abstractors searched selected residents' records for the most recent full MDS, either the last Annual MDS assessment or the MDS completed at admission. Epilepsy or seizure information was collected from section I, Items 1 and 3 on the paper MDS contained in the most recent MDS assessment (Table 1).
The study neurologist (IEL) conducted chart review sessions at each nursing home. The study neurologist was masked from knowledge of the resident's epilepsy or seizure disorder classification on the MDS. He then searched the remainder of each nursing home record for any mention of an epilepsy or seizure diagnosis and any supporting information. This included any consultations by a neurologist and results of MRIs and EEGs if available in the nursing home record. In many cases, a discharge summary from a hospital was also available for review.
2.3 Approval to collect these data was obtained from the University of Minnesota's International Review Board and Beverly Enterprises, Inc.
2.4 From the paper MDS, a resident was considered to have a seizure disorder or epilepsy if:
These same criteria were used for the computerized MDS. However, text documenting epilepsy or seizure disorder was not available in the computerized MDS file. From the neurologist's data collection session, the resident was considered to have epilepsy or a seizure if the neurologist saw a physician's note using these terms, a nurses note describing a seizure, a hospital discharge summary listing these terms, and any results of diagnostic tests or neurologist's consultations supporting use of these terms. The neurologist did not attempt make a diagnosis independently, rather limited the review to use of these terms by the physicians providing care for these patients.
2.5 Percent agreement between the study neurologist's determination of epilepsy or seizures from the nursing home record and documentation of epilepsy or seizure disorder from the “paper MDS” in the nursing home record was calculated using the study neurologist's determination as the “gold standard”. Percent agreement was also calculated between epilepsy or seizure disorder on the computerized MDS and epilepsy or seizure disorder on the “paper MDS” abstracted in nursing homes.
Statistical analysis was completed using SPSS® for Windows® software. Percent agreement, positive and negative predictive values, and the kappa statistic were computed for both validity comparisons (Streiner DL & Norman GR 2003). Agreement (yes/no) was compared across age groups (65-74,75-84,85+) and between genders using Chi Square tests. Length-of-stay (time since first admission) and time from MDS to data abstraction were compared between agreement groups (yes/no) using independent t-tests.
3.1 The eleven NHs in this study were located within one metropolitan area (Minneapolis-St. Paul). When all study procedures (protocols and logistics) were tested in the pilot nursing home, no changes in data collection or study procedures were needed. Therefore, data collected at this pilot nursing home were included with all other study results. Resident lists from the study's eleven nursing homes yielded 749 eligible residents. The cumulative list excluding sub-acute elderly nursing home residents was the sampling frame for resident selection into the study.
3.2 Sample selection resulted in 144 residents: 50 residents with nursing home identified seizures or epilepsy, 54 matched non-epilepsy residents, 36 residents on AEDs without seizures or epilepsy diagnosis listed, and 4 residents randomly chosen as epilepsy stand-ins The stand-ins were chosen from one nursing home which had no residents with epilepsy or seizures to provide the physician with additional test samples. Table 2 describes all nursing home residents selected for study. Agreement between the neurologist's review and paper MDS was not calculated for two nursing home records because the neurologist could not find sufficient information for comparison with paper MDS. Agreement between the paper and computerized MDS could not be calculated for four resident records as epilepsy or seizure disorder was indicated in text only, and our computerized data contained no text fields.
3.3 Overall, agreement between the documentation of epilepsy or seizure disorder on the paper version of the MDS and a neurologist's review of the nursing home record for documentation of epilepsy or seizure was 92.3% (131 agreements out of 142 paper MDS – neurologist pairs). The positive predictive value (likelihood the neurologist would find epilepsy or seizure, when it was documented in the paper MDS) was 87.8% (43 agreements in 49 instances of epilepsy or seizure disorder on the paper MDS). The negative predictive value (likelihood the neurologist would not find epilepsy or seizure when it was not documented on the paper MDS) was 94.6% (88 agreements in 93 instances of no epilepsy or seizure disorder on the paper MDS). The kappa (chance-corrected agreement statistic) was 0.83 considered as excellent agreement (Table 3).
3.4 Agreement between the documentation of epilepsy or seizure disorder on the paper version of the MDS and documentation of epilepsy or seizure disorder on the computerized MDS was 97.8% (137 agreements out of 140 MDS paper-computerized pairs) (Table 4). The positive predictive value (agreement with the paper MDS when epilepsy/seizure disorder was documented on the computerized MDS0 was 97.9% (47 agreements from 48 instances of epilepsy or seizure disorder on the computerized MDS), the negative predictive value (agreement with the paper MDS when epilepsy or seizure was not documented on the computerized MDS) was 97.8% (90 agreements from 92 instances of no documentation of epilepsy or seizure disorder on the computerized MDS). The kappa (chance-corrected agreement statistic) was 0.95 indicating excellent agreement.
3.4 Agreement was further analyzed by age group (65-74, 75-84 and 85+), gender, nursing home, length-of-stay and time from MDS. For agreement between the study neurologist and the paper MDS, the amount of documentation was also considered. Disagreements for both validation comparisons were too few to analyze (3 disagreements on paper MDS and 6 disagreements on computerized MDS). No significant relationships to agreement/disagreement were seen among any of these variables.
3.5 Supporting evidence for the diagnosis was not present on the majority of the nursing home records. A neurologists input was sometimes noted on a discharge summary from a hospital, and in less than 5% of cases was there a mention of an EEG prior to nursing home admission. No resident had a neurology consultation while in the nursing home.
4.1 Epilepsy and seizures are the most common serious neurological disorders affecting all ages. The incidence of epilepsy forms a U-shaped curve over a life-span, beginning with a high level in infancy, a lower level until age 55, and then an increasing level at each decade, with the incidence in elderly far exceeding that seen in children (Hauser et al 1993). With the aging of the population, epilepsy incidence and prevalence will increase greatly over the next decades, and the number of elderly with epilepsy will outnumber those in all other age groups (Leppik 2006). Use of AEDs in nursing homes is surprisingly high; surveys have shown that approximately 10% of all nursing home residents in the US are receiving an AED (Lackner et al 1998, Garrard et al 2000). Somewhat lower rates are reported from Europe, but the interesting finding that the young old (65-74 years of age) are much morel likely to use an AED then the oldest old is found on both continents (Galimberti et al 2006, Huying et al, 2006). But the choice of drugs varies greatly; phenytoin is most widely used in the US, carbamazepine in some European countries while phenobarbital is most widely used in others (Garrard et al 2000, Huying et al, 2006, Galimberti et al 2006).
4.2 The MDS is an assessment tool intended to address care issues in nursing homes (Frederiksen et al 1996; Gruber-Baldini et al, 2000). Only nursing home records were available from which to glean supporting diagnostic information. When available, reports from hospitals and clinics provided additional diagnostic data. Many physicians begin treatment after a single seizure in elderly patients because of the perceived risk of additional seizures. Therefore, documentation of a single seizure in nursing home records was considered clinically relevant to this study. Epilepsy/seizure disorders on the MDS were validated by comparing computerized MDS data to information on paper MDS and by comparing paper MDS data to a neurologist's review of nursing home records. A high degree of accuracy in the documentation of epilepsy/seizure disorders documented for both computerized and paper versions of the MDS is reported. Fortunately, our validation procedure was not influenced by some of the usual challenges that have faced other MDS validation studies. We used an external criterion and applied it to a small set of discrete items on the MDS (seizure disorder). The simple, objective focus or our study most likely contributed the precision of our validity measurement.
4.3 In the US, physicians are required to put an indication by ICD-9 codes for every drug prescription. Thus, when an AED is prescribed to prevent further seizures, a code for epilepsy or seizure is recorded. The criteria used by nursing home physicians to initiate an AED were not the focus of this study. Because of privacy rules restricting access to medical records, IEL was not able to obtain records other than which were available in the nursing home. While a patient may have had a neurological report prior to admission, these were not available in most cases, but at times a hospital discharge summary would mention a neurologist's involvement. It is possible that the epilepsy indications were given without the subjects meeting the epidemiological definition of two or more unprovoked seizures, and it is likely that many subjects had had only a single seizure prior to the initiation of an AED. The goal of this study was not to ascertain the accuracy of the diagnoses made by the nursing home physician, but rather validate the nursing home records, paper MDS, and electronic MDSs to be used in future research of the incidence and prevalence of AED use in the nursing home.
The MDS is a valid research tool for the study of an epilepsy or seizure indication in a nursing home, in both paper and computerized format. The results of this study have important implications for research on the use of AEDs among nursing home elderly. The age-specific incidence of epilepsy or seizure is highest in one of the most rapidly increasing segment of our population, those aged 65 and older, and most likely to be in nursing homes. Computerized MDS data provides a rich source of information on a relatively understudied disorder among elderly confined to nursing homes.
Supported in part by NIH Grant P-50-NS16308
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.