Search tips
Search criteria 


Logo of hsresearchLink to Publisher's site
Health Serv Res. 2005 June; 40(3): 865–886.
PMCID: PMC1361172

Just-in-Time Evidence-Based E-mail “Reminders” in Home Health Care: Impact on Patient Outcomes



To assess the impact and cost-effectiveness of two information-based provider reminder interventions designed to improve self-care management and outcomes of heart failure (HF) patients.

Data Sources/Study Setting

Interview and agency administrative data on 628 home care patients with a primary diagnosis of HF.

Study Design

Patients were treated by nurses randomly assigned to usual care or one of two intervention groups. The basic intervention was an e-mail to the patient's nurse highlighting six HF-specific clinical recommendations. The augmented intervention supplemented the initial nurse reminder with additional clinician and patient resources.

Data Collection

Patient interviews were conducted 45 days post admission to measure self-management behaviors, HF-specific outcomes (Kansas City Cardiomyopathy Questionnaire-KCCQ), health-related quality of life (EuroQoL), and service use.

Principal Findings

Both interventions improved the mean KCCQ summary score (15.3 and 12.9 percent, respectively) relative to usual care (p≤.05). The basic intervention also yielded a higher EuroQoL score relative to usual care (p≤.05). In addition, the interventions had a positive impact on medication knowledge, diet, and weight monitoring. The basic intervention was more cost-effective than the augmented intervention in improving clinical outcomes.


This study demonstrates the positive impact of targeting evidence-based computer reminders to home health nurses to improve patient self-care behaviors, knowledge, and clinical outcomes. It also advances the field's limited understanding of the cost-effectiveness of selected strategies for translating research into practice.

Keywords: Evidence-based medicine, reminders, home care, heart failure management, cost-effectiveness

Translating research into health care practice is a complex, challenging process aimed ultimately at yielding improvements in patient outcomes. Most of the literature in the field has focused on efforts to increase the use of clinical guidelines in hospitals and by physicians (Davis et al. 1995; Bero et al. 1998; Hunt et al. 1998; Grimshaw et al. 2001; 2004). Although such guidelines usually incorporate practices shown in clinical trials and/or judged by experts to produce clinically meaningful benefits for patients, relatively few “translation” studies have attempted to trace the impact of changes in practitioner behavior to actual improvements in patients' self-management of chronic conditions or to patient outcomes (Worrall, Chaulk, and Freake 1997).

This study moves out of the hospital setting and beyond physician practice changes to describe the impact and relative cost-effectiveness of a basic and an augmented e-mail reminder intervention designed to help home health nurses improve the self-management skills and outcomes of their heart failure (HF) patients. The interventions, described in a companion article by Murtaugh et al. (2005), delivered evidence-based, condition-specific information to nurses each time they began to care for a new HF patient—“just-in-time” to incorporate the recommended assessment and instruction practices into each patient's individualized plan of care. The interventions, selected for their reported effectiveness in other settings (Oxman et al. 1995; Bero et al. 1998; Grimshaw et al. 2001), were grounded in the “dual task theory of human performance.” This theory suggests that busy, information-overloaded clinicians will be more likely to perform “secondary” management tasks in conjunction with hands-on care when they receive well organized, cogent information for the right patient at the right time (Litzelman et al. 1993).

HF—a debilitating chronic condition characterized by symptoms such as severe shortness of breath, edema, and fatigue—is a high frequency, high cost disease that imposes a significant burden on older adults and their families. The leading cause of hospitalization among elders, HF accounted for nearly one million discharges in 1999 (Rich 2003). HF is also one of the major conditions for which older persons receive home health services. In 1999 it accounted for approximately 8.6 percent of Medicare home health discharges, 226,000 cases, and payments of $646 million (HCFA 2001). HF care has benefited over the past 20 years from the completion of several large-scale, randomized clinical trials and the development of authoritative evidence-based guidelines for clinical evaluation and treatment (Konstam, Dracup, and Baker 1994; Hunt et al. 2001; Rich 2003). Yet HF management among older adults has been suboptimal—marked by under use of effective therapies and lack of patient adherence to diet and medication regimens (Rich 2003).

To improve HF care and reduce hospital readmission, experts have recommended early discharge planning (Naylor et al. 1999; Phillips et al. 2004) and disease management approaches that entail post hospital follow-up, close monitoring of patient symptoms, and education regarding diet, medication, and exercise (Rich et al. 1995; Naylor et al. 1999; 2004; Stewart, Marley, and Horowitz 1999; Quaglietti et al. 2000; McAlister et al. 2001; Harrison et al. 2002; Weingarten et al. 2002; Rich 2003; Phillips et al. 2004). In theory, if not in practice, the role of a home health nurse encompasses the close patient follow-up that is a key element of disease management programs. Over the course of an HF home care episode—51 days on average (Centers for Disease Control and Prevention 2004)—home health nurses are routinely responsible for ongoing patient assessment, individualized care planning in consultation with the patient's physician, patient instruction in self-care management, monitoring of patient symptoms and support of patient adherence to medications and diet. Thus rather than mounting a new care management program, this study focused on the use of just-in-time, condition-specific information to improve the evidence-based knowledge and practices of home care staff already serving in the community. The interventions capitalized on existing intranet communication channels, so no significant organizational systems changes needed to be made. As an inexpensive and convenient mechanism, e-mail reminders were hypothesized to be an effective new medium for home care agencies and other decentralized health care practices to upgrade and standardize service provision and improve patient self-management and outcomes. The results reported here describe the comparative impact and cost-effectiveness of two variations of the information-based interventions.

Data and Methods

Study Design

The study employed a randomized design whereby a computerized algorithm assigned nurses to either a control group (usual care) or one of two intervention groups (basic or augmented) the first time they began caring for an eligible HF patient. A nurse's initial random assignment to a specific group determined the status for all new patients assigned to her care for the duration of the study. Both basic and augmented interventions provided the nurse with an e-mail reminder highlighting six HF-specific clinical guidelines. In addition, nurses in the augmented group received a laminated card focused on medication management, a prompter card to facilitate better physician–nurse communication, a self-care guide for patients (adapted from Konstam, Dracup, and Baker 1994), and follow-up outreach by a clinical nurse specialist (additional details about the content of the reminders can be found in Murtaugh et al. 2005). The e-mail reminder and, as applicable, the augmented materials were sent to the nurse each time a new eligible HF patient came onto her caseload (approximately 57 percent of nurses received reminders for one or two eligible patients, while 43 percent received reminders for three or more [Murtaugh et al. 2005]). While nurses were randomly assigned to treatment or control groups, random assignment of patients to nurses was not feasible. Patients referred to the study agency, however, were routinely assigned to a specific nurse based on where the patient lived and the nurse's overall caseload. Although not random, this assignment process was based on observable and exogenous factors that were controlled for in the analyses. Furthermore, agency staff responsible for assigning patients to nurses were blinded to the study.

Study Population

The study population consisted of patients aged 18 years or older who were admitted to a large, urban, nonprofit home care agency with a primary diagnosis of HF (ICD9-CM 428). Persons who were not cognitively able to give informed consent (as determined by the administration of the Short Portable Mental Status Questionnaire, Pfeiffer 1975), as well as non-English or non-Spanish speaking subjects were excluded from the study. The study was approved by the appropriate Institutional Review Boards.

Data Sources and Variable Definitions

Data were drawn from four main sources: (1) patient-level clinical and functional assessment data derived from the uniform home health assessment system (“OASIS”) mandated by the Centers for Medicare and Medicaid Services; (2) administrative data routinely collected by the agency's billing and human resources departments; (3) a patient survey designed to follow-up patients 45 days post home care admission; and (4) intervention cost data collected especially for the study.

Baseline measures of health and functional status, such as limitations in activities of daily living (ADLs) and instrumental activities of daily living (IADLs), mental status, cognitive functioning, and the presence and number of certain preexisting medical conditions were derived from the nurse's OASIS assessment conducted during the initial visit as part of routine nursing care. The OASIS data were also the source of information on patients' sociodemographic and social support characteristics. Administrative data were the source for additional control variables including provider nurses' baseline characteristics (e.g., gender, race/ethnicity, experience). The primary data source for outcome measures was a face-to-face patient interview conducted specifically for this study that employed a structured survey instrument to elicit detailed information on: (1) patient disease self-management, knowledge, and behavior; (2) the patient's clinical and functional status, including activities, limitations, and problems secondary to the cardiac condition, as well as general quality of life; and (3) health care utilization during the follow-up period. All interviews were conducted by trained interviewers blinded to the study groups and took place between August 2000 and November 2001. Whenever possible, standardized concepts and measures were used in the survey. Condition-specific clinical outcomes and functional status were derived from the Kansas City Cardiomyopathy Questionnaire (KCCQ) (Green et al. 2000).1 Depression was measured using the 15-item Geriatric Depression Scale (GDS) (Sheikh and Yesavage 1986). The EuroQoL EQ–5D was used to measure general quality of life (Brooks 1996). Additional items assessing self-management and knowledge of the disease were developed specifically for this study by the investigators in consultation with clinicians and providers. For example, to determine the extent to which patients recognized their HF medications, the interviewer collected the prescription bottles for all current medications, read the name and presented each medication to the patient, and asked the patient to indicate whether it was taken for his/her heart condition or related side effects.

Utilization data used for developing cost estimates were obtained through a combination of the agency's administrative records and self-reported data on medical care use collected as part of the patient interview. Two measures of cost—home-care-related and overall health care costs—were examined. Home care costs included administrative costs (i.e., the incremental cost of implementing the interventions, such as the cost of producing and distributing educational materials, and the cost associated with the consultant clinical nurse specialist), and costs associated with care provision (i.e., direct and indirect costs associated with the provision of home care visits by nurses, therapists, and home health aides). Overall costs included, in addition to the home care costs just described, resource costs associated with utilization of other health care services: specifically, the cost of hospital and emergency department (ED) services and physician visits during the study period. All medical services were valued using the average Medicare payment (or provider charges) for each type of service based on Centers for Medicare and Medicaid Services' data.

Estimation Procedure

Univariate descriptive statistics and multivariate regression models were utilized to analyze the data. The functional form of each regression varied according to the nature of the dependent variable. Ordinary least squares applied to a logarithmic transformation was used to model skewed continuous dependent variables such as the KCCQ summary, physical limitation, and total symptom scores. A probit specification was used to model treatment effects on binary dependent variables such as whether the patient reported skipping HF medications, was sure about when to take HF medications, or salted foods on a regular basis. Probit specifications also were used when modeling the likelihood of a high score (≥50) on clustered or bimodally distributed KCCQ scores (KCCQ quality of life and social limitation scores). Finally, an ordered probit specification was applied to hierarchical outcomes such as recognition of HF medications (does not recognize any, recognizes up to half, or recognizes more than half of own HF drugs). For all analyses, robust standard errors were computed to account for design clustering (i.e., multiple observations on patients for a given nurse) and heteroskedasticity effects.

In addition to the main variables of interest—membership in the basic or augmented treatment group—all multivariate analyses controlled for a wide array of patient, disease, nurse, and environmental characteristics that might confound the relationship between interventions and outcomes. These included baseline measures of patient health and functional status, such as the number of ADLs and IADLs, physical, social and cognitive functioning, and the presence and number of preexisting medical conditions. We also introduced controls for the patient's sociodemographic characteristics (age, gender, race/ethnicity, marital status, education, expected payment source, and baseline measures of social support), the provider nurse's baseline characteristics (gender, race/ethnicity, educational level, experience, employment status), and an indicator of whether the patient was served before or after the Medicare Prospective Payment System was implemented on October 1, 2000. Finally, measures of the nurse's caseload and borough of practice at the time of patient assignment were included to control for factors simultaneously affecting patient assignment to a specific nurse and patient outcomes/costs.

The magnitude of the intervention effects was estimated by comparing regression adjusted outcome scores (or probabilities) for the three intervention groups. Specifically, the regression equation for each outcome (e.g., KCCQ summary score) was used to calculate predicted (i.e., adjusted) outcomes for each individual assuming first treatment (e.g., augmented intervention) and then control status, holding the other variables constant. The average of the individual-level predicted values for each outcome represents the regression-adjusted score/probability in the presence and absence of each intervention.

Multivariate methods similar to those described above were used to obtain estimates of regression-adjusted treatment impacts on home-care-related and overall cost measures. All data analyses were conducted using SAS 8.2 and Stata 7.0 statistical software. Unless otherwise noted, positive differences indicate better outcomes (i.e., evidence that the intervention contributed to improved functioning or more effective self-management) holding other variables constant.

Empirical Results

Sample Characteristics

Of the 1,242 HF patients meeting study criteria, 158 (12.7 percent) were found to be ineligible for the survey during the telephone screener interview because of death or institutionalization. Of the 1,084 eligible respondents, 171 (15.8 percent) could not be located at the 45-day follow-up; 26 (2.4 percent) moved out of the area; and 259 (23.9 percent) refused to be interviewed (Figure 1). Complete interview data were, therefore, available for 628 patients (57.9 percent). Our analysis of the effectiveness and cost-effectiveness of the two information-based interventions focuses on these 628 subjects.2

Figure 1
Patient Flow Diagram

Table 1 presents selected sociodemographic and baseline health characteristics of the study participants. The mean age of the interviewed sample was approximately 72 years. About 40 percent of participants were black, non-Hispanic, and nearly one-third were of Hispanic descent. The majority (55 percent) had less than a high school education. Over 80 percent had been discharged from a hospital within 14 days prior to the home care admission. The severity of HF, as rated by the admitting home care nurse, was similar across groups. The patients averaged more than two chronic comorbid conditions in addition to their HF and more than five ADL and IADL dependencies. Although there were no marked differences in mean age, race, education, or baseline health characteristics among the three groups, there was a statistically significant difference between basic and control group members in the relative number of individuals in four broad age categories. Specifically, the proportion of basic intervention group members who were less than 65 and 75–84 years was lower than that for control group members, while the proportion 65–74 and greater than 85 years was higher. Both the basic and augmented intervention groups were 65 percent female while the control group was 77 percent female. Forty percent of the augmented intervention group reported an annual income of less than $10,000, compared with 52 percent of the control group.

Table 1
Key Sociodemographic and Baseline Health Characteristics of Study Patients (N=628)

Self-care Management

Adjusted probabilities for patient self-management indicators are reported in Table 2. Both interventions had a statistically significant effect (p=.002 and .023 for basic and augmented interventions, respectively) on patient recognition of HF medications as measured by a three-level indicator: (1) does not recognize any, (2) recognizes up to half, and (3) recognizes more than half. Patients in the basic and augmented intervention groups were significantly more likely to recognize more than half of their HF medications (38.4 and 35.0 percent, respectively) compared with the control group (26.3 percent). These improvements represented a gain of 46.0 percent for the basic group and 33.1 percent for the augmented group. Conversely, they were much less likely to recognize none of their HF medicines (31.1 and 34.3 percent versus 43.9 percent). There were marginally significant differences (p≤.10), in addition, between the augmented and control groups in the diet and weight management items. Patients in the augmented group were less likely to report that they salted their foods than those in the control group (23.3 versus 30.7 percent; p=.095) and were more likely to weigh themselves daily (27.4 versus 21.4 percent; p=.082). Although generally in the expected direction, the effects of the basic and augmented interventions on the remaining self-care management indicators were not statistically significant.

Table 2
Estimates of Treatment Effects on Self-Management Indicators

Clinical and Functional Outcomes

Table 3 reports the regression-adjusted effects of the interventions on patients' clinical and functional outcomes. There was a marked 6.2 point (15.3 percent) improvement in the mean KCCQ summary score of patients treated by nurses randomized to the basic intervention and a 5.2 point (12.9 percent) improvement for the augmented intervention, compared with patients receiving usual care. Patients in the basic and augmented intervention groups had mean KCCQ summary scores of 46.5 and 45.6, respectively, compared with a mean of 40.4 for patients in the control group (scores range from 0 to 100 with higher scores representing better outcomes). Both effects were statistically significant (p=.013 for the basic versus control group difference and p=.048 for the augmented versus control group difference).

Table 3
Estimates of Treatment Effects on Clinical and Functional Outcomes

The interventions also had a positive effect on three of the four dimensions of the KCCQ summary score. In particular, patients in the augmented group were more likely than their control group counterparts to score above the mid-point on the KCCQ quality of life scale (53.3 versus 44.6 percent, respectively—a difference of 8.7 percentage points or 19.5 percent; p=.042). Patients in both intervention groups also were more likely to have better (above mid-point) KCCQ social limitation scores; the adjusted probability was 7.4 percentage points (or 27.6 percent) higher for the augmented intervention group (p=.064) and 7.0 percentage points (or 25.2 percent) higher for the basic intervention group compared with the control group (p=.090). The basic intervention also had a positive impact on symptom domain scores with 55.6 percent scoring above the mid-point compared with 48.6 percent of control group subjects (p=.091).

Finally, patients in the basic intervention group scored significantly higher (p=.003) than those in the control group on the EuroQoL scale (48.9 compared with 39.3, respectively). There were no significant differences between control and intervention groups in self-efficacy, as measured by the KCCQ, nor in depression, as measured by the GDS.

Service Use, Cost, and Cost Effectiveness

Regression-adjusted estimates of patient service use are reported in the top of Table 4. The point estimates suggest a pattern of higher service use among patients treated by intervention nurses, particularly among those in the augmented group. However, service-specific differences were generally not statistically significant with one exception. Home-care-related visits—which include nursing, therapy, and home health aide visits—for both the basic and the augmented interventions were significantly higher than for usual care (43.6, 44.1, and 35.2 visits, respectively) (p=.048 and .053, respectively).

Table 4
Estimates of Service Use, Cost, and Cost Effectiveness for Selected Outcomes

Once service utilization measures were valued and aggregated to provide patient-level estimates of home-care-related and overall costs, differences across groups became more evident. As shown in the middle of Table 4, the average cost associated with administrative and direct home care provision was $2,814 for the usual care group, $3,371 for the basic intervention, and $3,425 for the augmented intervention group. Thus, the home-care-related cost of the basic intervention was about 19.8 percent higher (p=.062) and the augmented intervention about 21.7 percent higher than comparable cost associated with usual care (p=.058). The augmented intervention became relatively more costly when resources associated with patients' use of other medical care during the follow-up period were included in the calculation: overall costs for the control group were $4,996, compared with $5,869 for the basic intervention group (a cost increase of $873 or 17.5 percent; p=.084), and $6,330 for the augmented intervention (a cost increase of $1,334 or 26.7 percent; p=.020).

The interventions' cost-effectiveness with respect to two key outcomes—the KCCQ summary score and the EuroQoL quality of life scale—are shown in the bottom of Table 4.3 The cost of producing one unit of outcome was obtained by dividing the regression-adjusted difference in predicted cost between treatment and control groups (shown in the middle of Table 4) by the regression-adjusted difference in predicted outcomes (shown in Table 3) between treatment and control groups. Overall and home care-related unit costs were estimated separately for each of the two outcome measures (KCCQ and EuroQoL) and treatment groups (augmented and basic). For example, the overall cost of a unit improvement in KCCQ summary score for the basic intervention was $140.80 ($873/6.2). These unit costs were then multiplied by the number of outcome units necessary to produce a 5 percent improvement in the respective outcome (i.e., the number of units to produce a 5 percent improvement over regression-adjusted control group means for each outcome). For both outcomes, the basic intervention was a more cost-effective strategy than the augmented intervention. The home-care-related cost to produce a 5 percent improvement in the KCCQ summary score was $183, about three-quarters that of the $235 cost for the augmented intervention. The overall cost of a 5 percent improvement in the KCCQ summary score was $246 for the basic intervention, less than half of the $513 cost of the augmented intervention. A 5 percent improvement in the EuroQoL scale was achieved at an average of $116 for home-care-related costs and $181 for overall costs under the basic intervention; the augmented intervention was not effective in improving this outcome.


This study demonstrated that two relatively easy to administer, information-based interventions designed to strengthen the role of home health nurses in facilitating patients' adherence to complex HF treatment regimens yielded significant improvements in self-care management and clinical outcomes. Positive effects on critical dimensions of HF self-care management were found in the areas of medication knowledge for patients treated by nurses in the basic reminder group and for medication knowledge, diet, and weight monitoring for those treated by augmented group nurses. The effect on medication knowledge was particularly striking with basic and augmented interventions raising the probability that patients recognized over half of their HF medications by 46 and 33 percent, respectively. Both the basic and augmented interventions yielded positive effects on clinical outcomes measured by KCCQ HF-specific measures. The basic intervention also yielded improvement in general health related quality of life measured by the EuroQoL.

The self-care management and clinical outcomes findings are consistent with the content of both interventions (Murtaugh et al. 2005). Furthermore, that the augmented intervention improved more dimensions of patient self care than the basic is consistent with the fact that nurses in the augmented group received additional resources for each patient, including a comprehensive HF self-care guide to reinforce patient adherence, and that the augmented intervention generally yielded greater improvement in nurses' practices (Murtaugh et al. 2005). Nevertheless, even among patients served by nurses in the augmented group, levels of nonadherence were substantial, indicating room for further improvement.

Patients served by intervention nurses generally had better outcomes. They also experienced equivalent or higher service use compared with patients receiving usual care. This raises two questions. First, why did the two interventions increase rather than lower home care visits? Although the assessment and instruction practices emphasized by the interventions are in theory part of routine home care, many recommended practices were infrequently documented by usual care nurses (Murtaugh et al. 2005). Thus the finding of increased home care use suggests that effectively educating patients and developing their skills to self-manage their condition takes more time and effort than current routine care. Second, given improved disease management skills (in particular, among augmented intervention patients) and generally better outcomes, why did intervention group patients not experience lower use of medical care services than those served by control nurses?

Several considerations may be important in addressing the second question. First, the effects of improved self-care management might have resulted in lower downstream medical care use among the intervention groups relative to the controls had the follow-up period been longer. Alternatively, in the absence of a specific physician component, the interventions may have led to increased ED and hospital use as better-informed patients may have sought medical attention more immediately than those who were not as aware of their condition. This hypothesis is consistent with the patient self-care guide's prominent reminders to “call your doctor” in the presence of certain symptoms. Furthermore, seeking care from the ED is consistent with the characteristics of the study sample, which consisted primarily of low income, minority persons living in a city, like many other urban areas, characterized by a shortage of community-based physicians willing to serve Medicaid patients. Thus systemic barriers may have mitigated some of the effects of the interventions. In contrast, disease management programs shown to reduce hospitalization have generally provided systemic physician/cardiologist and nurse practitioner/cardiac nurse backup to avoid ED use and achieve optimal adjustment of pharmacotherapy over time (Stewart, Marley, and Horowitz 1999; Rich 2003). Still another possibility is that patients in the intervention groups experienced better outcomes as the result of their more effective use of all types of medical services: by educating patients about how to manage their condition, the intervention nurses may have educated patients on seeking the “optimal” combination of home health and other medical services.

The cost-effectiveness results indicate that, as implemented, the basic intervention produced roughly equivalent patient outcomes at lower cost than the augmented intervention. The near equivalence in patient outcomes across the two interventions was observed despite a generally better performance of the augmented intervention in improving nurses' practices (Murtaugh et al. 2005) and patient self-care behavior. Any incremental gains in patient-level HF-specific quality of life associated with the augmented intervention, however, were not sufficient to outweigh the incremental home care or overall medical care costs associated with its implementation. This finding highlights the importance of including cost calculations in considering whether and how to implement alternative quality improvement measures. It is also consistent with the most recent meta-analysis of research on strategies for guideline dissemination and implementation (Grimshaw et al. 2004), which, contrary to earlier reviews (Grimshaw et al. 2001) found that multifaceted interventions were not necessarily more effective than single interventions in promoting practitioner compliance.

The study contributes to the literature on translation research and disease management in two important ways. It goes beyond most other studies of the effectiveness of computer-based decision support systems by examining patient outcomes in addition to changes in care practices (Hunt et al. 1998). It also adds to our understanding of what strategies or combinations of strategies may be most cost-effective to improve patient outcomes in specific organizational settings for specific clinical conditions. This comparative, cost effectiveness approach has by and large been missing from translational research (Bero et al. 1998; Farquhar, Stryer, and Slutsky 2002; Grimshaw et al. 2004) and research on disease management (McAlister et al. 2001; Weingarten et al. 2002; Rich 2003).

The study, however, is not without limitations. It is based on patients served by a single certified home health agency providing care to an urban patient population that is more culturally and racially diverse, of lower socioeconomic status, and more likely to be dually eligible for Medicare and Medicaid than the population typically served by such agencies. In addition, specific baseline information about patient's disease knowledge and self-management behavior was not available. However, the inclusion of a wide range of preintervention characteristics—many of which are likely to be related to adherence and other behaviors at admission—mitigates the need to control for baseline measures of outcomes. Furthermore, because patient assignment to randomized nurses was carried out strictly by operations staff blinded to the study, there is no reason to believe that there would have been any systematic differences in patients' unobservable or unmeasured characteristics between the intervention and control groups.

In conclusion, this study demonstrated that providing “just-in-time” evidence-based practice information to dispersed home health nurses at the right time (i.e., “just in time” for them to incorporate it into care planning and patient instruction) significantly improved self-care management and outcomes in a predominantly poor minority HF patient population. In addition, the analysis yielded comparative information on the relative impact of both a simple e-mail reminder and a more resource-intensive multifaceted intervention, demonstrating the basic intervention to be the more cost effective in improving clinical outcomes. These findings are of significant import to the field of translational studies, where systematic reviews of the literature have shown that many studies used weak designs, embodied methodological flaws and did not include economic evaluations in their analyses (Grimshaw et al. 2001; 2004). They are also of great significance to the burgeoning field of home health care, where little work has been carried out to translate evidence-based guidelines into practice and where heightened regulatory scrutiny, compounded by increased economic pressure, is motivating providers to seek out and implement more efficient and effective care practices.


This project was supported by grant number 2 R01 HS010542 from the Agency for Healthcare Research and Quality (AHRQ). We would like to thank the many clinicians at the study agency who provided advice at key points during the project. We thank, in particular, Mary Ellen McCann, R.N., who developed the HF patient self-care guide prior to the initiation of the project and served as the “expert peer” to study nurses. We also would like to thank Doreen Wang, M.P.A., for her programming support as well as Amy Clark, B.A., and Lori King, B.A., for their assistance with the implementation of the intervention and subsequent data collection efforts.


1The KCCQ summary score consists of four domains—physical limitation, symptoms, quality of life, and social limitation. The physical limitation domain is six items assessing limitations in activities of daily living. The nine-item symptoms domain assesses the frequency and burden of HF-specific symptom (e.g., shortness of breath). Quality of life is assessed with three items asking about satisfaction and enjoyment of life. The social limitation scale of four items inquires if the patient's HF limits participation in social and recreational activities.

2A standard concern in experiments with longitudinal follow-up is the potentially biasing effects of sample attrition. Two conditions would be necessary for the impact estimates to be biased: (i) the pattern of attrition differed by intervention and control groups and (ii) attrition was correlated with the outcome measure being examined. Patients lost to follow-up were compared with those for whom complete data were obtained using baseline characteristics from the standard patient assessment on admission to home care (i.e., OASIS) and other administrative data available for all enrollees. These analyses were conducted separately by each of the main reasons for noninterview (death or institutionalization, unable to contact, and refusal) as well as by overall nonresponse. There was no evidence of attrition bias in this sample. For all attrition analyses, the intervention estimates were both small in magnitude and statistically insignificant.

3We opted to present cost effectiveness results based on the KCCQ summary score and the EuroQoL because these two outcomes summarize the patients' physical, social, and psychological functioning. With the exception of two outcomes related to diet and weight management, all cost effectiveness analyses revealed a pattern consistent with that reported above.


  • Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. “Closing the Gap between Research and Practice: An Overview of Systematic Reviews of Interventions to Promote the Implementation of Research Findings.” British Medical Journal. 1998;317(7156):465–8. [PMC free article] [PubMed]
  • Brooks R. “EuroQoL: The Current State of Play.” Health Policy. 1996;37(1):53–72. [PubMed]
  • Centers for Disease Control and Prevention Home Health Care Discharges. 2004. Available at
  • Davis DA, Thomson MA, Oxman AD, Haynes RB. “Changing Physician Performance. A Systematic Review of the Effect of Continuing Medical Education Strategies.” Journal of the American Medical Association. 1995;274(9):700–5. [PubMed]
  • Farquhar CM, Stryer D, Slutsky J. “Translating Research into Practice: The Future Ahead.” International Journal of Quality Health Care. 2002;14(3):233–49. [PubMed]
  • Green CP, Porter CB, Bresnahan DR, Spertus JA. “Development and Evaluation of the Kansas City Cardiomyopathy Questionnaire: A New Health Status Measure for Heart Failure.” Journal of the American College of Cardiology. 2000;35(5):1245–55. [PubMed]
  • Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O'Brien MA. “Changing Provider Behavior. An Overview of Systematic Reviews of Interventions.” Medical Care. 2001;39(8):112–45. [PubMed]
  • Grimshaw JM, Thomas R, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C. “Effectiveness and Efficiency of Guideline Dissemination and Implementation Strategies.” Health Technology Assessment. 2004;8(6):1–72. [PubMed]
  • Harrison MB, Browne GB, Roberts J, Tugwell P, Gafni A, Graham ID. “Quality of Life of Individuals with Heart Failure. A Randomized Trial of the Effectiveness of Two Models of Hospital-to-Home Transition.” Medical Care. 2002;40(4):271–82. [PubMed]
  • Health Care Financing Administration (HCFA) Health Care Financing Review. Baltimore, MD: U.S. Department of Health and Human Services; 2001. Medicare and Medicaid Statistical Supplement.
  • Hunt SA, Baker DW, Chin MH, Cinquegrani MP, Feldman AM, Francis GS, Ganiats TG, Goldstein S, Gregoratos G, Jessup ML, Noble RJ, Packer DP, Silver MA, Stevenson LW, Gibbons RJ, Antman EM, Alpert JS, Faxon DP, Fuster V, Jacobs AK, Hiratzka LF, Russell RO, Smith SC., Jr “ACC/AHA Guidelines for the Evaluation and Management of Chronic Heart in the Adult.” Journal of the American College of Cardiology. 2001;38(7):2101–13. [PubMed]
  • Hunt DL, Haynes RB, Hanna SE, Smith K. “Effects of Computer-based Clinical Decision Support Systems on Physician Performance and Patient Outcomes.” Journal of the American Medical Association. 1998;280(15):1339–46. [PubMed]
  • Konstam MA, Dracup K, Baker DW. Heart Failure: Evaluation and Care of Patients with Left-Ventricular Systolic Dysfunction. Clinical Practice Guideline No. 11. (AHCPR Publication No. 94-0612) Rockville, MD: Agency for Health Care Policy and Research, Public Health Services, U.S. Department of Health and Human Services; 1994.
  • Litzelman DK, Dittus RS, Miller ME, Tierney WM. “Requiring Physicians to Respond to Computerized Reminders Improves Their Compliance with Preventive Care Protocols.” Journal of General Internal Medicine. 1993;8(6):311–7. [PubMed]
  • McAlister FA, Lawson FME, Teo KK, Armstrong PW. “Randomised Trials of Secondary Prevention Programmes in Coronary Heart Disease: Systematic Review.” British Medical Journal. 2001;323(7319):957–62. [PMC free article] [PubMed]
  • Murtaugh CM, Pezzin LE, McDonald MV, Feldman PH, Peng TR. “Just-in-Time Evidence-Based E-mail ‘Reminders’ in Home Helath Care: Impact on Nurse Practices.” Health Services Research. 2005;40(3):849–64. [PMC free article] [PubMed]
  • Naylor MD, Brooten D, Campbell R, Jacobsen BS, Mezey MD, Pauly MV, Schwartz JS. “Comprehensive Discharge Planning and Home Follow-up of Hospitalized Elders: A Randomized Clinical Trial.” Journal of the American Medical Association. 1999;281(7):613–20. [PubMed]
  • Naylor MD, Brooton DA, Campbell RL, Maislin G, McCauley KM, Schwartz JS. “Transitional Care of Older Adults Hospitalized with Heart Failure: A Randomized, Controlled Trial.” Journal of the American Geriatric Society. 2004;52(5):675–84. [PubMed]
  • Oxman AD, Thomson MA, Davis DA, Haynes RB. “No Magic Bullets: A Systematic Review of 102 Trials of Interventions to Improve Professional Practice.” Canadian Medical Association Journal. 1995;153(10):1423–31. [PMC free article] [PubMed]
  • Pfeiffer E. “A Short Portable Mental Status Questionnaire for the Assessment of Organic Brain Deficit in Elderly Patients.” Journal of the American Geriatrics Society. 1975;23(10):433–41. [PubMed]
  • Phillips CO, Wright SM, Kern DE, Singa RM, Shepperd S, Rubin HR. “Comprehensive Discharge Planning with Postdischarge Support for Older Patients with Congestive Heart Failure: A Meta-Analysis.” Journal of the American Medical Association. 2004;291(11):1358–67. [PubMed]
  • Quaglietti SE, Atwood JE, Ackerman L, Froelicher V. “Management of the Patient with Congestive Heart Failure Using Outpatient, Home and Palliative Care.” Progress in Cardiovascular Diseases. 2000;43(3):259–74. [PubMed]
  • Rich MW. “Heart Failure in the Elderly: Strategies to Optimize Outpatient Control and Reduce Hospitalizations.” American Journal of Geriatric Cardiology. 2003;12(1):19–27. [PubMed]
  • Rich MW, Beckham V, Wittenberg C, Levan CL, Freedland KE, Carney RM. “A Multidisciplinary Intervention to Prevent the Readmission of Elderly Patients with Congestive Heart Failure.” New England Journal of Medicine. 1995;333(18):1190–5. [PubMed]
  • Sheikh JI, Yesavage JA. “Geriatric Depression Scale (GDS): Recent Evidence and Development of a Shorter Version.” In: Brink TL, editor. Clinical Gerontology: A guide to Assessment and Intervention. New York: The Haworth Press; 1986. pp. 165–73.
  • Stewart S, Marley JE, Horowitz JD. “Effects of a Multidisciplinary, Home-based Intervention on Unplanned Readmissions and Survival among Patients with Chronic Congestive Heart Failure: A Randomized Controlled Study.” Lancet. 1999;354(9184):1077–83. [PubMed]
  • Weingarten SR, Henning JM, Badamgarav E, Knight K, Hasselblad V, Gano A, Ofman JJ. “Interventions Used in Disease Management Programmes for Patients with Chronic Illness—Which Ones Work? Meta-analysis of Published Reports.” British Medical Journal. 2002;325(7370):925. [PMC free article] [PubMed]
  • Worrall G, Chauld P, Freake D. “The Effects of Clinical Practices Guidelines on Patient Outcomes in Primary Care: A Systematic Review.” Canadian Medical Association Journal. 1997;156(12):1705–12. [PMC free article] [PubMed]

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust