|Home | About | Journals | Submit | Contact Us | Français|
Validation of process-of-care measures includes testing for a link with outcomes. We aimed to determine whether delivery of better quality of care for urinary incontinence (UI) and falls is associated with improved patient-reported outcomes.
Retrospective cohort study of older (age ≥ 75) ambulatory care participants in Assessing Care of Vulnerable Elders-2 (ACOVE-2) study, who screened positive for UI (n=133) and/or falls/fear of falling (n=328).
We measured composite quality scores (% quality indicators [QIs] passed per patient) and change in Incontinence Quality of Life (IQOL, range 0–100) scores or Falls Efficacy Scale (FES, range 10–40) before and after care was delivered (mean 10 months). Because eligibility for falls treatment QIs was dependent on the physician performing a physical exam, we calculated an alternative “Common Pathway” quality indicator (CPQI) score that assigned a failing score for falls treatment to unexamined patients.
Each 10% increment in receipt of recommended care for UI was associated with a 1.4 point improvement in IQOL score (p=.01). Falls quality was not related to FES score using the original composite score; however the CPQI score was related to FES (+.4 point FES per 10% increment in falls quality, p=.01).
Better quality of care for falls and UI was associated with measurable improvement in patient-reported outcomes in less than one year. The connection between process and outcome required consideration of the interdependence between diagnosis and treatment in the falls QIs. The process-outcome link demonstrated for UI and falls underscores the importance of improving care in these areas.
Urinary incontinence (UI) and falls are prevalent geriatric conditions1–4 associated with disability, morbidity, and poorer quality of life.2, 5–7 Despite the availability of effective treatments to improve UI and reduce the risk of falling,7–16 delivery of recommended care for these conditions in primary care settings is poor.17, 18
The Assessing the Care of Vulnerable Elders-2 (ACOVE-2) study was a controlled trial of an intervention to increase adherence to evidence- and consensus-based processes of care for three geriatric conditions (UI, falls, and dementia). The primary outcome of ACOVE-2 was the group-level score on 18 process-of-care indicators spanning these three conditions. The ACOVE-2 intervention was associated with moderate improvements in quality of care for falls (44% versus 23% of recommended care delivered for the intervention versus control site, or a 21 percentage-point improvement) and UI (37% versus 22%, or a 15 percentage point improvement) 19 Although ACOVE-2 was not designed to study clinical outcomes, patients were also given condition-specific symptom inventories for UI (the Incontinence Quality of Life [IQOL] survey20) and falls (fear of falling measured by the Falls Efficacy Scale [FES]21) prior to and after the quality improvement intervention was implemented. This paper presents a secondary analysis that examines the relationship between quality of care delivered during the ACOVE-2 study and these patient-reported outcomes. We hypothesized that delivering more recommended care processes to patients with UI and falls would result in decreased severity of symptoms as measured by the IQOL and FES, respectively.
ACOVE-2 enrolled 649 patients (age ≥75) to test a multi-component intervention to improve quality of care for UI, falls, and dementia.19 The intervention, which consisted of physician education, structured visit notes that guided physicians to provide recommended care, and community resource and education handouts for patients,22 was implemented at two large medical practices in southern California (each with intervention and control sites). Institutional review boards at RAND, the University of California, Los Angeles, and the Greater Los Angeles Veterans Affairs Healthcare System approved the research protocol.
Prior to appointments at both intervention and control sites, all patients aged ≥75 (n=2671, Fig 1) were screened for UI and falls during a 13-month observation window in 2002–2003. The screening questions, administered by office personnel, asked: Do you have a problem with UI (or your bladder) that is bothersome enough that you would like to know more about how it could be treated?” and “In the past 12 months, have you fallen 2 or more times? …Fallen and hurt yourself or needed to see a doctor because of a fall? …Been afraid that you would fall because of balance or walking problems?” Participants who screened positive for UI answered the IQOL survey, a measure of incontinence-related quality of life that is associated with clinical incontinence severity.20, 23 Those who screened positive for falls or fear of falling (i.e., answering yes to any of the three questions) were administered the FES,21 a measure of patient concern for falling during daily activities that is associated with future falls, gait and balance impairment, and disability.8, 21, 24–26
At study completion, patients were re-administered the IQOL and/or FES survey. Our analysis included patients who completed pre and post surveys. Patient characteristics are summarized in Table 1. Patients who participated in the study for both conditions were included in both analyses.
The outcome variable for the UI analysis was the change in severity of UI symptoms as measured by the IQOL survey.23, 27 The overall IQOL score ranges from 0–100 points, with higher scores representing better quality of life (milder symptoms). The overall IQOL is calculated as the sum of 22 questions scored on a 5-point scale, each item ranging from “extremely” symptomatic (1 point) to “not at all” (5 points), followed by re-scaling to the 100-point range. Sub-scores representing three sub-domains of incontinence-related quality of life are also calculated using subsets of the items (also followed by re-scaling to 0–100): avoidance and limiting behavior (e.g., “I worry where the toilets are in new places”), social embarrassment (e.g., “I worry about wetting myself”), and psychosocial consequences (e.g., “I feel depressed because of UI problems”).20 Our analytic sample included subjects with ≤3 missing items. We used multiple imputation (ordered logit models, 5 imputations with recombination of 5 datasets for all analyses) to obtain complete IQOL scores for 13 baseline interviews and 4 follow-up interviews with ≤3 missing items.28 We computed change scores by subtracting the baseline IQOL score from the follow-up IQOL score. Because higher IQOL scores represent less severe disease, a positive change in score represents improvement in the UI condition over time. In this study, we considered a 2-point improvement in IQOL as the minimally important effect size.
In a prior study of patients taking duloxetine to decrease UI, patients who perceived their overall change in symptoms as “a little better,” “much better,” and “very much better” in comparison to those with “no change” were associated with mean IQOL change scores of 2, 6, and 13 points, respectively. Those who reported a substantial improvement in incontinent episodes per day (at least 25% fewer) had mean IQOL change scores that were 5 points better than those who reported no improvement in incontinence episodes. 20, 27
The outcome variable for the falls analysis was change in FES, a measure of patient concern for falls during 10 daily activities (e.g., bathing, walking around the neighborhood).21 Responses ranged from least (1 point) to most concerned (4 points), yielding a final score with a range from 10 to 40.29 A higher FES score indicates more severe fear of falling, and the FES is associated with future falls and disability.8, 21, 24–26 To facilitate interpretation of results, the change scores for the falls analysis were calculated as baseline FES score minus follow-up FES score, so that a positive change score indicated improvement in the falls condition (i.e., less concern about falls over time). Our analytic sample for falls included patients with ≤2 missing FES items. We used multiple imputation to obtain complete FES scores for 16 baseline and 18 follow-up interviews with ≤2 missing items. Minimally important effect sizes for this version of the FES were not available, so to put the FES score into clinical perspective, we examined the findings from an intensive controlled multidisciplinary home visit intervention that reduced risk of falls by 23%. The pre-post difference in FES scores between the intervention and control groups was 1.4 FES points (+.2 points versus −1.2 points, p=.02).9
The ACOVE-2 process-of-care quality indicators [QIs] have been described previously in full,19 and we briefly outline the QIs (Tables 2 and and3)3) and scoring methods here. Patients with bothersome UI were eligible for three quality indicators for UI diagnosis (taking a UI-specific history, exam, and urinalysis, Table 2 QIs #1–3) and three QIs concerning treatment (checking post-void residual, discussing treatment options, and recommending behavior intervention prior to pharmacologic treatment, QIs #4–6). A patient who had fallen (twice in the past year or once with injury requiring medical attention) was considered to need a fall-specific history (Table 3, QI #1) and a gait and balance examination (QI #2). A patient who had not fallen but reported fear of falling was considered only to need a gait and balance examination (QI #3). If physical examination demonstrated abnormal balance then patients were eligible for treatment with physical therapy or assistive device (QI#4); if abnormal gait was found then patients were eligible for physical therapy (QI #5).
Evaluation of individual QIs was performed using all outpatient primary care and specialist medical records for a 13-month observation period for each patient. If the patient received the recommended care process, a score of 1 was awarded; if the process was not performed, a score of 0 was assigned. If the patient refused recommended care, full credit was awarded. Selected QIs (Falls QIs #3–5) were excluded from application to individuals with advanced dementia or life expectancy ≤ 6 months.30 In addition, if the patient was documented in the medical record as having received a workup and completed recommended therapies for UI or falls (referred to as “maximal treatment”), then QIs for that condition were excluded. Finally, patients who described fear of falling but did not fall before or during the observation period were excluded if they denied concern for falling on the FES.
For this patient-level analysis, we developed summary quality scores for each patient based on the individual QIs measured in ACOVE-2.. We first calculated a simple summary score for each patient for falls and/or UI care, calculated as the number of quality indicators passed divided by the number of QIs triggered, ranging from 0 to 100 percentage points.
Simple summary scores for UI patients were based on 4 to 6 UI QIs per patient. However, in contrast to the UI analysis, most of the falls sample (82%) triggered only 1 or 2 QIs. This comparatively smaller sampling of falls QIs per patient was a result of specific eligibility criteria for the treatment-QIs (#4 and #5). These QIs were triggered by an abnormal gait or balance exam, thus patients had to pass an exam-QI (#2 or #3) as a prerequisite. Since two-thirds of the falls sample (224 of the 328) were not examined, few patients overall were evaluated for care related to treatment. Considering the care of falls as a pathway from diagnosis to treatment, the simple summary scores for falls insufficiently captured the full pathway of care quality for this sample. Therefore we proposed an alternative scoring method that would restore evaluation of treatment for this patient-level analysis. Since we did not know the physical status of the unexamined patients, it was ambiguous for which treatment indicators patients would be eligible: QI #4 for balance problem, QI #5 for strength problem, both problems, or no problem. We combined QIs #4 and #5 into a single “Common Pathway” QI (CPQI) (3, rightmost columns).
Patients who did not receive an exam were presumed to have an undetected physical problem and scored accordingly for the CPQI. Those who were examined and found to have an abnormality (n=58) triggered one CPQI instead of QI 4 and/or QI 5. The overall effect of employing the CPQI was that all patients were evaluated for at least one treatment-QI, except those who had normal physical examinations (n=66 without gait, strength, or balance abnormality, consistent with the original ACOVE-2 scoring method).
For each of the individual falls and UI QIs, we compared the mean IQOL and FES change scores among those who passed versus failed the QI under consideration. For falls, QIs 4 and 5 were evaluated separately and then by the CPQI described above. We performed (two tailed) t-tests to compare mean IQOL and FES scores.
Next, we tested the relationship between patients’ summary UI and falls quality scores with IQOL or FES change scores, respectively, using unadjusted linear regression. We then performed multivariable regressions controlling for the following variables, chosen because of their potential effect on clinical outcomes: age, gender, number of ACOVE indicators triggered for all conditions (a proxy measure of co-morbidity), and time between interviews. To address confounding by disease severity, we controlled for number of UI or falls QIs triggered (for each analysis, respectively). We also controlled for whether the patient received care at an intervention versus control practice and tested an interaction term between quality of care and intervention assignment. Because patients’ care was performed by 36 primary care physicians, final multivariable models were modeled as hierarchical random effects models, with patients clustered within physicians.
For UI analyses, we considered change in overall IQOL scores as the main outcome as well as change in the three sub-domain scores. For the falls analyses, we performed all analyses using the simple summary score and the CPQI methods. We re-tested final results for sensitivity to imputed scores by limiting the sample to those with complete IQOL and FES scores. We used Intercooled STATA version 11.0 (College Station, Texas) for all statistical analyses.
Of 2671 patients age ≥ 75 who were screened for UI and falls/fear of falling, 235 subjects (9%) answered positively for new or bothersome UI symptoms and 500 (19%) answered positively for fear of falls or falling (Fig 1). There were 115 (4.3%) who screened positively for both UI and falls/fear of falling. For the UI sample, 32 (14%) were unable to complete the baseline IQOL surveys, and another 40 (20%) were lost to follow-up or had incomplete follow up IQOL surveys. During quality measurement for UI care, 30 patients (18%) were excluded based on ACOVE QI criteria. The analytic sample for the UI analysis was 133 patients who were evaluated for 650 UI QIs. For the falls sample, we excluded 89 (18%) participants due to baseline and 66 (16%) due to missing follow-up or incomplete FES surveys. Of the remaining 345 patients, 17 were not evaluated for falls quality based on ACOVE QI criteria. The analytic sample for the falls analysis was 328 patients who were evaluated for 579 QIs. The mean follow-up time was 9.8 months (range 4.2–14.6) for UI and 10.4 months (range 4.2–17.0) for falls. The final analyses contained 55 participants (41% of the UI sample and 17% of the falls sample) who were considered in both samples.
The mean age of the UI sample was 80. Four-fifths were female, and mean baseline IQOL score was 87. On average, participants worsened with respect to their UI symptoms over the 10-month observation window. This was consistent when measuring change in overall IQOL score (mean decline of 3.4 points [SD 16.3 points, p=.02] and two of the three subscales (mean decline of 3 points [SD 17.7 points, p=.045] for avoidance behavior, mean decline of 1.6 points [SD 18.6 points, p=.006] for psychological quality of life, and non-significant decline of 1.8 points [SD 18.8, p=.3] for the embarrassment subscale).
The unadjusted relationship between change in IQOL score and whether individual UI QIs were passed or failed is displayed in Table 2. For each of the six QIs, the direction of effect was positive (i.e., the mean IQOL change score was better for those who passed than those who failed), but only QI #1 was associated with statistically significant (t-test p<.05) improvement in overall IQOL and QI#2 with subscale change scores.
The mean summary UI quality score was 32.6% (SD 29.9%), with a mean score of 40.6% in the intervention group versus 23.0% in the control group. A 10% increment in unadjusted patient-level composite UI quality score was associated with an improvement in overall IQOL score of 1.23 points (p=.01) and improvement in 2 of the 3 subdomain scores (avoidance 1.06 [p=.07], psychological 1.30 [p=.03], and embarrassment 1.41 [p=.001]).
Multivariable random-effects models (controlling for age, gender, co-morbidity, intervention group, and time between interviews) showed that better UI quality is associated with improvement in UI-related quality of life. An improvement of 10 percentage points in UI quality was independently associated with 1.4 improvement (p=.01) in the overall IQOL score and two subscales (embarrassment [β=1.6, p=.008] and psychological [β=1.5, p=.02]). There was no significant interaction between quality of care and intervention versus control group status in predicting any of the UI outcomes (p=.6 to .8). Results were robust to limiting the sample to those with complete IQOL scores n=120).
We calculated the predicted outcomes for a hypothetical 80-year old woman over the full range of quality of care delivered in this study (Figure 2, upper graph) using the final multivariable fixed-effects model. Providing none, half or all recommended care for UI was predicted to result in a 7-point decrease, no change, and 7 point improvement in IQOL score over a mean 10-month follow-up, respectively. Applying these results to a group-level clinical practice improvement scenario, the expected improvement in mean IQOL for a group of older patients after improving UI care from 23% to 41% of recommended care (the mean summary scores for intervention versus control groups in the ACOVE-2 study) would be 2.5 IQOL points, a small but validated improvement in global UI symptoms.20, 27
The mean age of the falls sample was 81. Three-quarters were female, and mean baseline FES score was 19 (SD 7.5). Over the 10-month observation period, the overall change in concern for falling as measured by the FES score was essentially unchanged (mean improvement of .6 points, SD 6.7, p=.1).
We tested the unadjusted relationship between the change in FES scores and individual falls QIs (Table 3). For each QI, the mean improvement in FES change score was greater for subjects passing the QI compared to those failing the QI, but statistical significance (p<.05) was associated only with performing a falls exam (QI #3, 3.3 versus .6 point improvement [p=.03]) and the CPQI (3.9 versus .7 point improvement [p=.016]).
Most of the patients in our sample (n=292, 89.0%) had a single fall or fear of falling “event,” triggering 1 to 4 QIs (mean 1.5) by the simple scoring method and 1 to 3 QIs (mean 2.2) by the CPQI method instead of separate QIs #4 and #5. Thirty-six people (11.0%) had ≥ 2 events. The mean simple summary falls score was 31.3% (39.8% in the intervention group, 21% in the control group) and slightly lower via the CPQI method (30.3% overall mean; 39.2% intervention versus 19.6% control). Both the simple and CPQI scores were not related to unadjusted FES change scores (for both scores, β=.12 per absolute 10-percentage point improvement in quality, p=.2).
However, the multivariable random-effects analyses found that the CPQI score was related to improved FES change score (β=.41 FES points per 10-percentage point increment in quality, p=.01), while the simple summary score was not (β=.21, p=.2). Predicted outcomes for the CPQI model are displayed in Figure 2 (lower graph). For an 80 year-old woman, providing 0%, 50% or 100% of the recommended care would result in an improvement of .4, 2.4, and 4.5 FES points over the 10-month follow up period, respectively. Applying these results to a clinical practice quality improvement scenario, an improvement from 20% to 40% of recommended care would be expected to result in mean improvement of .8 FES points. This response was approximately two-thirds of the improvement in FES (1.4 points) associated with a multidisciplinary home-based controlled intervention to decrease falls.9 Results of the falls analysis were also robust to limiting the sample to those with complete FES scores (n=312).
In this secondary analysis of the ACOVE-2 practice-based quality improvement study, we hypothesized that better adherence to evidence-based quality indicators would improve patient-reported outcomes for falls and UI. We found a small but clinically meaningful improvement in incontinence quality-of-life (2.5 points) over ten months in response to a 15% percentage-point quality improvement for UI, a realistic level of improvement in quality that was achieved in the ACOVE-2 practice-based intervention.19 The IQOL response we observed in this analysis corresponds to prior studies of UI patients with a small improvement in global self-reported UI symptoms and half the improvement of those with a substantial decrease in incontinent episode frequency20, 27 We also found a small improvement in falls efficacy (.8-point improvement in the FES) associated with better quality of care for falls, i.e., the 20 percentage-point improvement achieved in the ACOVE-2 study. The response we observed was approximately two-thirds the effect on FES found in an intensive home-based falls-reduction intervention.9
Our results shed further light on our understanding of ambulatory care for geriatric conditions. For both falls and UI, our analysis extends prior interventional research by measuring the full spectrum of office-based care that includes diagnostic processes (history-taking and physical examination) as well as treatment. In this study, our broader practice-based approach was modestly linked with better outcomes, but not as tightly linked as in clinical intervention trials that improved falls efficacy 9, 13, 31, 32 and incontinence7–16 (e.g., pharmacologic or behavioral therapy).
Although the individual QI-level falls scores appeared to be positively related to better FES, the patient-level simple summary score was not. The simple summary score was an inadequate measure of comprehensive falls care because so many individuals in our sample did not receive physical exams. For these patients, gait and balance abnormalities could not be identified and therefore appropriate treatment could not be directed at improving their falls outcomes. Our results suggest that the association between process and patient-level outcomes for falls was restored by scoring with the CPQI, which assigned those without physical exam with an additional penalty for failure to treat. To our knowledge, this is the first report to test an alternative scoring method that addresses serial care measures in which failing to carry out an early care process results in exclusion from eligibility for downstream QIs. The CPQI scoring modification addresses this issue, emphasizing the importance of performing high-quality comprehensive care from screening to diagnosis to treatment to follow-up. These findings concerning falls care demonstrate that patient-level quality measures obfuscate detection of poor comprehensive care and should be avoided in future quality indicator design.
Although the ACOVE-2 intervention improved UI and falls quality of care at two intervention clinical practices,19 there remained substantial room for improvement. The quality of care delivered to individual patients was a better predictor of patient-reported UI- and falls-related outcomes than whether a patient was seen at an intervention versus control practice.
IQOL scores of the UI sample as a whole worsened over the relatively short follow-up interval (10 months), which was inconsistent with other studies that have found better IQOL scores as patients age and adapt to their UI symptoms.20 Rather than improving the quality of life for the sample, it appears that better quality of care attenuated a natural decline that occurred in our sample over 10 months, with only a small subset of patients (i.e., those with better than 75% of indicators passed) experiencing symptom improvement. The decline in IQOL may reflect the advanced age of our sample, which contrasts with prior IQOL studies that focused on younger populations. Some individuals in our older sample may also have thought their symptoms were a normal part of aging at baseline, but developed more concern about symptoms as a result of increased labeling and medical attention.33
A strength of our study design is that we used clinically detailed quality of care data on older outpatients that are not available in administrative datasets. We also administered measures of condition-specific symptom severity before and after the delivery of care. The data were collected prospectively in a community-based sample of older patients with falls and UI symptoms. However, the findings should be viewed in light of the limitations of the ACOVE-2 sample, which was not ethnically diverse and was assembled from only two medical groups. Our results also cannot be generalized to patients who cannot self-report their FES or IQOL. We also did not collect clinical outcomes in the ACOVE-2 study, for example frequency or severity of subsequent falls or number of incontinence episodes. Clinical severity measures would complement patient-reported outcomes in future quality improvement studies.
In conclusion, we found that the quality of care for falls and UI was associated with improvement in falls efficacy and UI-related quality of life. The link between better primary care for these conditions and improved outcomes should provide impetus for strengthening efforts to enhance care of these conditions within primary care practices.
Dr. Min is supported by the AHRQ (R21 HS017621), a Research Career Development Core award from the University of Michigan Claude Pepper Older Americans Independence Center, and the Geriatric Research, Education, and Clinical Care Center at the Veterans Administration Healthcare System in Ann Arbor. Dr. Min was on the UCLA faculty during the writing of this manuscript, supported by the UCLA Claude Pepper Older Americans Independence Center (NIA-UCLA K12 AG001004). Dr. Ganz is funded by the U.S. Department of Veterans Affairs, Veterans Health Administration, VA Health Services Research & Development (HSR&D) Service through the VA Greater Los Angeles HSR&D Center of Excellence (Project # VA CD2 08-012-1).
The original ACOVE-2 study was supported by a contract from Pfizer Inc to RAND. Pfizer did not serve a role in the design, analysis, or preparation of this retrospective analysis.
Author Contributions:Lillian C. Min: study concept and design, analysis and interpretation of data, and preparation of manuscript.
David B. Reuben: study concept and design, acquisition of subjects and/or data, analysis and interpretation of data, and preparation of manuscript.
John Adams: study concept and design, analysis and interpretation of data and preparation of manuscript.
Paul G. Shekelle: study concept and design, acquisition of subjects and/or data, analysis and interpretation of data, and preparation of manuscript.
David A. Ganz: study concept and design, analysis and interpretation of data, and preparation of manuscript.
Carol P. Roth: acquisition of subjects and/or data, analysis and interpretation of data, and preparation of manuscript.
Neil S. Wenger: study concept and design, acquisition of subjects and/or data, analysis and interpretation of data, and preparation of manuscript.