|Home | About | Journals | Submit | Contact Us | Français|
Convergent validation of the Kohlman Evaluation of Living Skills (KELS) to screen older adults’ capacity for safe and independent living.
Cross-sectional study correlating KELS with components of a Comprehensive Geriatric Assessment.
200 community-dwelling older adults aged ≥65 including 100 referred by adult protective services (APS) and 100 ambulatory patients matched on age, race, gender, and socio-economic status.
In-home comprehensive assessment
Kohlman Evaluation of Living Skills (KELS), Geriatric Depression Scale (GDS), Physical Performance Test (mPPT), Mini-mental state examination (MMSE), Knee Extensor Break Test, Executive test (EXIT25), CLOX 1 & 2, and an 8-foot walk test.
Older adults with abnormal KELS scores performed significantly worse on all tests except for the Knee Extensor Break Test. Accordingly, among the entire group, the KELS correlated with measures of executive function (EXIT25, r = .705, p <.001; CLOX 1, r = −.629 p<.001), cognitive function (MMSE, r=−.508, p<.001), affect (GDS, r= .318, p<.001) and physical function (mPPT, r= −.472, p<.001) but did not correlate with the Knee Extensor Break Test (r = −.068, p = .456). Among those referred by APS the KELS failed to correlate with only the 8-foot walk test (r = .175, p = .153) and GDS (r = .080, p = .450).
This study demonstrated the convergent validity of KELS with a battery of cognitive, affective, executive, and functional measures often used to determine older adults’ ability to live safely and independently in the community. KELS may be a valid and pragmatic alternative to screen for the capacity to live safely and independently among older adults.
The population of older adults is growing rapidly in the United States and by 2040 it is estimated that 89 million people will be at least 60 years of age.1 Advancing age has been associated with increases in disability and functional limitations.2,3 These limitations are often facilitated by age related changes in cognition, function, and affect, which may ultimately reduce ones ability to live safely and independently in the community.4–6 Depending on the extent and complexity of these impairments, rehabilitation assessments and restorative care may be necessary to re-establish an older adult’s ability to live safely and independently in the community. Multi-dimensional screening assessments that account for an older adult’s ability to function safely and independently in his or her home environment would facilitate healthcare providers in making critical decisions regarding longitudinal care and placement after hospital discharge.
Geriatric healthcare professionals often use Comprehensive Geriatric Assessment (CGA) in clinical and home-based settings to determine functional status in patients. The CGA is comprised of a battery of standardized cognitive, functional, and affective evaluations consisting of observational, self-report and objective measures.7 Studies have identified these domains as important components of functional status 7, 8, 9 and, using CGA evaluations, health care providers make recommendations regarding the ability of an older adult to live safely and independently in the community.7 CGA, however, is lengthy and typically involves a multidisciplinary team. In addition, CGA is methodologically heterogeneous, does not provide standardized scores, and is more difficult to conduct and interpret in the home setting.7 Recent evidence suggests only a weak association between commonly used CGA measures that rely on self-report of daily living activities and real-world functioning in older adults. It is suggested that objective assessments with direct association to real-world functioning be supplemented in place of these less reliable measures.10
Recent studies support utilizing the Kohlman Evaluation of Living Skills (KELS) to identify older adults who are in need of attendant care or supportive services to live safely in the community.11, 12 Originally developed to assess function in psychiatric populations and used predominantly by occupational therapists in inpatient and outpatient settings, KELS has been subsequently validated as a sensitive measure for differentiating among geriatric populations requiring different levels of personal assistance. In these validation studies, the KELS was reported to be an effective predictor of ability to live safely and independently in the community compared to established measures of basic and instrumental activities of daily living.11, 12 In addition, KELS has demonstrated good inter-rater reliability as well as construct and criterion validity for the KELS.12 The most recent study using the KELS among geriatric samples showed that the KELS significantly differentiated between elders living safely and independently in the community and elders in need of assistance to perform basic and or instrumental activities of daily living independent of cognitive, affective, medical, social and functional status.11
The purpose of this paper is to demonstrate that the KELS is a pragmatic, multi-dimensional screening tool for assessing an older adult’s capacity to live safely and independently in the community. Unlike many of the available tools for assessing this ability, KELS provides important performance-based indications of one’s ability to complete activities specific to safe and independent living. Therefore, this study will specifically evaluate the convergent validity of the KELS with a battery of tests that are typically used to assess cognitive, affective, and executive abilities, as well as functional status among a community-living sample of older adults mirroring a CGA. Implications of these findings for healthcare professionals are provided.
The data for this study were collected between April 1, 2005 and September 15, 2006 as part of a case-control study conducted by the Consortium for Research in Elder Self-Neglect of Texas (CREST). The CREST study was funded under the National Institutes of Health Roadmap Initiative and was designed to characterize urban community-living adults with geriatric self-neglect. Self-neglect is often accompanied by deterioration in ability to perform activities necessary for safe and independent living. One-hundred community-living adults reported to Adult Protective Services (APS) for geriatric self-neglect and 100 matched community living adults referred from a community geriatrics clinic were enrolled in this study. All self-neglect cases were recruited from APS and were identified as self-neglect by APS caseworkers using the standard definition(s) outlined by the Texas Human Resource Code 48.002(a)(4).13 Each APS referral had to meet the following inclusion criteria for the study: 1). APS substantiated self-neglect, 2). 65 years of age or older, 3). English Speaking and 4). Resident of Harris County. The community control group was matched with APS referrals using age±5 years, race, gender, and zip code (proxy for socio-economic status) and could have no prior evidence of self-neglect. English speaking was a necessary criteria based on the standardized measures used in this study. To ensure the safety and protection of the elderly participants, an ethicist was consulted when developing the consent process and prior to enrollment. The study was approved by an Institutional Review Board from the Baylor College of Medicine and the Harris County Hospital District in Houston, Texas.
All evaluations were one-time home-based assessments. After obtaining informed consent, a Comprehensive Geriatric Assessment (CGA) was conducted which included a full social and medical history, physical examination, pill count of all medications, and the following standardized measurements; Self-Rated Health and Mortality14, the Mini-Mental State Exam (MMSE)15 , the knee extensor break test16,17, Eight-foot walk test18, a modified version of the Physical Performance Test (mPPT)19, Kohlman Evaluation of Living Skills (KELS)20, Executive Cognitive Test (EXIT25)21, the executive clock drawing tests (CLOX I and CLOX 2)22 and the 15-item Geriatric Depression Scale (GDS)23. The EXIT25, CLOX 1 and CLOX 2 were not initially included in the battery, thus resulting in a smaller, but equally matched proportion of participants. These measures were added with the intent to provide validated screening tools that measured executive functions more specifically than broader cognitive assessments such as the MMSE.
The Mini Mental State Exam (MMSE) is a well-known, widely used cognitive screening test with established reliability and validity.15 The MMSE assesses the cognitive state in the areas of orientation, registration, memory, language, instructions and grapho-motor skills. The total score range is between 0–30 with scores less than 24 generally considered a positive screen for dementia.
The Knee Extensor Break Test assesses the muscle strength of the extensors of the knee using a hand held dynamometer.16 A Lafayette Manual Muscle Test Dynamometer was used to collect the data. The test administrator stabilized the lower extremity by placing her hand on the back of the leg to support the knee and then instructed the participant to fully extend and hold the leg while pressing down on the dynamometer. Three measurements were recorded for the knee extensors and the mean was recorded. Exclusion criteria for this test included if the participant was less than three months post hip or lower extremity surgery or if experiencing significant pain. The dynamometer is considered best practice when measuring muscle strength in therapy.
Timed walk tests are measures of lower extremity function.17,18 Participants walked at a pace they typically would walk in a grocery store. A measuring tape marked a clear eight-foot pathway on the floor, and the participant was instructed to start walking a couple of steps before the beginning of the tape measure. The faster of the two trials, in seconds, was recorded. The administrator also noted any assistive devices used such as a cane or walker.18
The modified Physical Performance Test (mPPT) objectively assesses functional capabilities in the following domains of physical function; write a sentence, simulate eating, lift a book onto a shelf, pick up a penny from the floor, turn 360-degreees, and a timed 50-foot walk.19 For the current study, the 50-foot walk item was eliminated (in lieu of the 8-foot walk test) and the scoring was modified with scores ranging from 0–24. The research team timed and scored each of the tasks using validated criteria with a score of <16 designated as a failing score for the current study.
The 15-item Geriatric Depression Scale (GDS) is used to screen for depressive symptomatology in older adults. It has been used extensively among geriatric practitioners and is efficient for clinical and home-based evaluations. A score of 5 or greater is often viewed as a positive indication of depressive symptomatology.23
The EXIT25 is a valid and reliable instrument to assess executive cognitive impairment.24 The EXIT25 is a 30-minute, 25-item interview scored from 0–50. The items assess the components related to frontal lobe dysfunction such as verbal fluency, design fluency, frontal release signs, motor/impulse control, imitation behavior, as well as others. The EXIT 25 has detected executive impairment in a variety of clinical conditions such as Alzheimer’s disease, Picks disease, sub-cortical vascular dementia, diabetes mellitus, major depression, HIV infection, schizophrenia and normal aging.25,21 The EXIT25 has also been used to predict functional status declines among aging adults.26 Scores of 15/50 or greater are indicative of clinically significant executive impairment.
The executive clock drawing test (CLOX) is a measure of executive impairment and is designed to differentiate between executive impairment and constructional apraxia.22 The CLOX is divided into an unprompted task that is sensitive to executive control (CLOX 1) and a copied version that is not sensitive to the same control (CLOX 2). A validation study of CLOX1 and CLOX 2 as a cognitive screen revealed the diagnostic accuracy rate of 84% for the CLOX 1 and 85% for the CLOX 2.27
The KELS was originally developed for psychiatric inpatients and has often been used to assess the elderly in acute care settings. The KELS contains 17 items and is used to assess ADLs and IADLs in five areas that include: self-care, safety and health, money management, transportation and telephone use, and work/leisure. The measure is a combination of self-report items (i.e. hygiene, cooking, social engagements, transportation, monthly income) and performance based items (i.e. writing a check and balancing a checkbook, purchasing items and receiving correct change, reading a phone bill, identifying hazards in pictures, dialing a telephone, balancing a budget, finding a number in a phonebook). Scoring ranges from 0 to 16 with a score of greater than or equal to 6 considered to be the best score for differentiating those who can live independently in the community versus those requiring assistance to live safe and independently in the community. Furthermore, the KELS has been shown to have strong construct and criterion validity as well as good inter-rater reliability (74% −94%).12,20
Independent samples t tests and chi-square distributions were used to compare means and proportions between the groups for continuous and dichotomous data. Pearson product moment correlations were computed to assess for linear associations between the KELS and the standardized measures mentioned above.
The baseline characteristics of the study population are provided in Table 1. A total of 11 characteristics are described and stratified by enrollment in the community-living comparison (CC) and Adult Protective Services referral (APS) groups. The two groups were significantly different on three variables: living alone, religious participation and number of medications. The CC group was less likely to live alone, more likely to have routine religious activities, and reported taking more medications. Although the groups were very similar in the types of medical conditions reported, the CC group was recruited from a Geriatric Outpatient Clinic thus likely explaining the differences in number of medications available for assessment in the CC group. It has also been postulated that older adults referred to APS may be less likely to seek routine medical care.28 Failure to reach statistical significance on the matching variables suggests that a successful matching of groups was achieved based on age, race, gender and socio-economic status.
Table 2 describes the mean value for the functional, cognitive, affective, and executive measures for the total study sample stratified by normal and abnormal scores on KELS. As shown in Table 2, statistically significant differences exist for each of the tests, with the exception of the Knee Extensor Break Test, between participants with normal versus abnormal KELS scores. For all comparisons, the significant associations were in the expected direction, with normal KELS scores being associated with better scores on cognitive, affective, executive and functional measures. Point biserial correlations between abnormal and normal KELS scores and continuous functional, cognitive, affective and executive function scores revealed the same associations. Furthermore, to enhance the clinical applicability of KELS in relation to the other commonly used screening measures, we elected to present only the data using previously established and validated cut-points for all measures.
Table 3 describes the correlation coefficients of the functional, cognitive, affective, and executive assessment measures with the KELS for all study participants and subgroups of those participants referred by APS and those receiving usual community based geriatrics care (CC group). In the complete sample, the KELS was significantly correlated with all the measures except for the Knee Extensor Break Test. The KELS scores for the CC group were significantly correlated with the 8-foot Walk, and the GDS. In the APS group, the KELS scores were significantly correlated with the Knee Extensor Break Test and the CLOX 2.
This study evaluated the convergent validity between KELS and standardized measures of cognition, affect, executive and functional status using data from known samples of community living older adults. Overall, older adults with abnormal KELS scored significantly worse on all standardized measures, with the exception of the Knee Extensor Break Test, compared to those with normal KELS scores. Furthermore, convergence between the KELS and the comparison measures was indicated by significant Pearson product moment correlations when assessed using the entire sample. Individual group correlations revealed a lack of significant convergence between the KELS and the Knee Extensor Break Test and CLOX 2 data in the community comparison group and with the 8-foot walk test and the GDS in the APS referral group. In all groups, the KELS converged most highly with executive cognitive function measures.
Accurate assessment of the older adult’s capacity to live safely and independently in the community poses many challenges for clinicians and other healthcare providers. Changes in cognition, affect, motivation, self-efficacy and functional status all may affect accuracy in one’s ability to self-report.10 Although self-report is the most common and time efficient clinical measure of functional status in older adults, recent literature reviews highlight a weak association between self-evaluation and real-world functioning.10
The KELS provides an objective assessment of the ability to carry out necessary real-world tasks. While some performance measures assess the ability to perform important functional activities through indirect measures, such as placing objects on shelves or writing out sentences, the KELS directly measures activities necessary for safe and independent living. A comprehensive assessment of multiple cognitive, functional, and affective domains is often required to make this determination and thus, isn’t always efficient or practical. Although the KELS is mainly administered by occupational therapists, it has been shown that with minimal training the KELS can be feasibly administered in the community-based setting by a variety of providers with minimal additional training.29 This study demonstrated convergent validation between a home-based KELS assessment and major components of a Comprehensive Geriatric Assessment of an older adult’s capacity for safe and independent living in the community.
Strong correlations between the KELS and executive function measures (i.e., EXIT25 and CLOX) matched or exceeded many of the other screening tools. These correlations suggest that KELS may provide an indication of potential limitations in the ability to plan, sequence, execute and/or monitor behaviors necessary for safe and independent living.30 The correlation of EXIT and CLOX to KELS is similar to the correlation of these two measures with IADLs.25 Correlations between the KELS and other assessments did vary in size, but most were within the moderate range suggesting important measurement characteristics of the KELS in relation to physical performance, depression and memory necessary for safe and independent living.
These findings should be viewed in light of certain limitations. This study was limited to English speakers based on the chosen standardized measures. Therefore, it is uncertain as to whether the findings would change with inclusion of Spanish only speakers. Difficulty in recruiting and consenting special populations of older adults such as geriatric self-neglecters may have reduced the true-score variability of the measures; thus, affecting the strength of the correlations. The executive function measures were implemented late in the study and the percentage of participants assessed with these measures is more limited. However, for each comparison there were sufficient data to protect against type II errors.
This study provides evidence for the convergent validity of the Kohlman Evaluation of Living Skills with standardized measures of cognition, affect, executive and functional status often used to screen an older adult’s ability to function safely and independently in the community. The KELS offers an appropriate and efficient alternative to timely and costly comprehensive geriatric assessments for determining an older adult’s ability to live safely and independently in their home. KELS provides a pragmatic clinical assessment and, unlike MMSE, EXIT, or CLOX, it also identifies the specific domains of safe and independent living that require intervention. A clinical advantage to using KELS, therefore, is that it provides the first outlines of a specific treatment plan. Furthermore, specialized clinical skills are not required to reliably administer and interpret the KELS, thus a wide variety of health professionals can be trained to screen older adults for impairments in ability to live safely and independently in their homes.
Although these findings warrant replication, they provide possible implications for the use of the KELS by clinicians and rehabilitative healthcare providers to screen for older adults functional ability. Further research is needed to determine the sensitivity of the KELS to measure change in function over time as a result of comprehensive care to restore older adults’ ability to live safely and independently at home.
Acknowledgment of financial support: This article is the result of work supported with resources and the use of facilities at the Houston Center for Quality of Care & Utilization Studies (HFP90-020), Michael E. DeBakey Veterans Affairs Medical Center, the Consortium for Research in Elder Self-Neglect of Texas (NIH P20-RR020626), and a bioethics grant from the Greenwall Foundation. Dr. Naik is also supported by the National Institute of Aging (K23AG027144) and a Doris Duke Charitable Foundation Clinical Scientist Development Award.
The authors would like to acknowledge Amy Wheeler, OT, for her review of an early draft of the manuscript and Kristin Cassidy for her editing of the final manuscript.
We certify that no party having a direct interest in the results of the research supporting this article has or will confer a benefit on us or on any organization with which we are associated AND, if applicable, we certify that all financial and material support for this research (eg, NIH or NHS grants) and work are clearly identified in the title page of the manuscript
Acknowledgement of presentations of material:
Preliminary results were previously presented during a Poster Presentation at the National Meeting of American Geriatrics Society in Seattle in May, 2007