|Home | About | Journals | Submit | Contact Us | Français|
Optimal care for patients with diabetes is difficult to achieve in clinical practice.
To evaluate the impact of a registry and decision support system on processes of care, and physiologic control.
Randomized trial with clustering at the practice level, involving 7,412 adults with diabetes in 64 primary care practices in the Northeast.
Provider decision support (reminders for overdue diabetes tests, alerts regarding abnormal results, and quarterly population reports with peer comparisons) and patient decision support (reminders and alerts).
Process and physiologic outcomes were evaluated in all subjects. Functional status was evaluated in a random patient sample via questionnaire. We used multiple logistic regression to quantify the effect, adjusting for clustering and potential confounders. Intervention subjects were significantly more likely to receive guideline-appropriate testing for cholesterol (OR=1.39; [95%CI 1.07, 1.80] P=0.012), creatinine (OR=1.40; [95%CI 1.06, 1.84] P=0.018), and proteinuria (OR=1.74; [95%CI 1.13, 1.69] P=0.012), but not A1C (OR=1.17; [95% CI 0.80, 1.72] P=0.43). Rates of control of A1C and LDL cholesterol were similar in the two groups. There were no differences in blood pressure, body mass index, or functional status.
A chronic disease registry and decision support system based on easily obtainable laboratory data was feasible and acceptable to patients and providers. This system improved the process of laboratory monitoring in primary care, but not physiologic control.
Optimal diabetes care can improve outcomes and costs1. Although physiologic results have improved in the US, care remains suboptimal2,3. The Chronic Care Model highlights the importance of a population-based approach; involving community, patients and providers; and information technology4–6. Various combinations of population management, practice support, and patient outreach have been used for diabetes care in a variety of settings with mixed results7–16.
Many practice interventions for diabetes have been carried out in academic and urban centers. However, much primary care occurs in small private practices; half of visits in the US are to practices with two or fewer physicians17. These practices are less likely to have electronic systems with decision support for chronic illness care18.
The Vermont Diabetes Information System (VDIS) is a registry and decision support system based on the Chronic Care Model that targets both providers and patients, and was designed for easy integration into primary care offices with or without electronic medical records. We sought to determine its effect on processes of diabetes care (guideline-based laboratory testing), and physiologic outcomes (glycemic and lipid control) in a largely rural, community, primary care setting.
A full description of the decision support system, the study design, and the recruitment strategy has been reported19,20. In brief, VDIS is a laboratory-based registry defined by: use of the Chronic Care Model as an organizing framework, daily data feeds from otherwise independent laboratories, automatic test interpretation using guideline-based algorithms, use of fax and mail to report to providers and patients not easily reached by electronic networks, and report formats that are accessible and useful to patients and providers. The system collects pertinent laboratory results (hemoglobin A1C, cholesterol, creatinine, and urine protein) and provides five types of reports: accurate and timely faxed laboratory result flow sheets to providers, faxed reminders of overdue laboratory tests to providers, mailed reminders to patients overdue for testing, mailed alerts to patients with elevated test results, and mailed population reports to providers summarizing their entire diabetes roster19,20.
We identified 13 hospital-based clinical laboratories in Vermont and adjacent New York State that provided services to community practices. Two were excluded because they could not establish reliable data feeds. Eleven completed business associate agreements in accordance with the Health Insurance Portability and Accountability Act21.
We recruited 141 internal medicine or family medicine practices that used the participating laboratories (Fig. 1). Twenty-two were ineligible because they extensively used point-of-care testing (9); were participating in conflicting quality improvement activities (6); were undergoing a major transition such as a new practice or retiring provider (4); did not provide diabetes care (2); or used an electronic medical record that included reminders and decision support (1). We recruited from the 119 remaining practices until we met our target of 64.
The practice was chosen as the unit of randomization because of the sharing of patients and systems of care among primary care providers (PCPs) in the same office20. We randomized practices in blocks by hospital lab to balance the number of practices in each arm in each region. Participation required informed consent and a business associate agreement. Intervention practices received VDIS services while control practices received usual care. Orientation of the intervention practices to the system typically consisted of a one hour orientation session for the providers and staff and approximately 30 minutes of provider time to review patient rosters.
Clustering of patients within practices reduces statistical power in proportion to the degree that subjects within each cluster are similar. Therefore, we modeled sample size using methods which require an estimate of the intraclass correlation coefficient to use in a variance inflation factor22–24. Preliminary data indicated a standard deviation of A1C of 1.4%, an intra-class correlation of 0.02, and 125 subjects per practice20. Using alpha=0.05 and a power of 80%, we estimated that 22 practices per arm would allow the detection of a difference of 0.2 percentage points of A1C. To accommodate drop-out and unanticipated problems, we enrolled 64 practices. The observed intra-class correlation for A1C was higher than estimated (0.055)25 providing power to detect a change of 0.3 points of A1C. Practices were enrolled between June 2003 and January 2005 and followed until all had at least 24 months of observation.
Once a practice was enrolled, a list of patients with an A1C result in the previous two years was generated by the lab. PCPs confirmed those who had diabetes using chart review. Exclusion criteria were age under 18 years, patient receiving most diabetes care in another practice, or cognitive impairment that would limit the understanding of reminders, per the judgment of the PCP. We did not distinguish between Type 1 and Type 2 diabetes because national consensus guidelines do not differ substantially regarding testing frequency or therapeutic goals26, and because, in practice, it is often unclear which type is present. Patients were censored from the study when they died (526); were no longer with a participating practice because they moved or because of a practice change (1673); or at the end of the planned observation period. Three practices closed during the intervention period and two ended early to participate in other projects.
In order to more fully assess the study population, we conducted a home-based survey of subjects20. Randomly selected patients from each practice were invited by phone to participate until we had recruited approximately 12.5% of the study population (N=929). This target was chosen to allow for some attrition over the observation period while maintaining adequate sample size to include important covariates in modeling. Ninety percent of the baseline interviews were completed within 3 months of study start. The same patients were re-surveyed at the end of the observation period within 6 months of study completion. The surveyed population was slightly older (64.7 vs. 62.7 years, p=0.001), more likely to be female (54.3% vs. 50.5%, p=0.04), and more likely to be on time for A1C testing at study baseline (63.0% vs. 56.2%, p=0.001), though the mean A1C was not significantly different (7.12% vs. 7.07%, p=0.36).
The primary outcome was glycemic control measured by mean A1C and by the proportion of patients with A1C<7%. Secondary outcomes included mean LDL, proportion with LDL<100 mg/dl, and timeliness of testing. For A1C, on-time was defined as testing within 6 months if A1C<7% and 3 months otherwise; for lipid testing, annually if LDL<100 mg/dl, 6 months if LDL 100–129 mg/dl, and 3 months otherwise; for serum creatinine, annually; and for urine microalbumin testing, annually unless previous testing was abnormal, after which further testing was not required26.
Date of birth and sex were reported by the hospital laboratory. Other measures were obtained from the patient survey: self-reported race was categorized as “white” or other; education was classified as high school graduate or less; marital status was classified as “Married or living together as married” or other; household income was dichotomized as “<$30,000 per year” or “≥$30,000 per year.” Patients completed the SF-12 functional status measure27, the Diabetes Self Care Survey28, the Short Test of Functional Health Literacy29 (dichotomized at the median), the Self-administered Comorbidity Questionnaire30 and the Audit of Diabetes-Dependant Quality of Life (ADDQOL), a continuous measure of the impact of diabetes on quality of life (QOL) with lower scores representing lower QOL (possible range −9 to +9)31. We measured height, weight and determined the mean of three separate blood pressure readings using an automated sphygmomanometer.
At the follow-up survey, we asked patients to recall healthcare utilization in the past year32. We used visit costs from the Medical Expenditure Panel Survey (PCP $121; Specialist $235; ER $560;33,34; Hospital day $2,14735) multiplied by the number of uses recalled by the patient to estimate total cost per patient.
We used a general linear mixed model for outcomes with normally distributed residual errors, or a generalized linear mixed model for outcomes with a binomial distribution for residual errors36. Standard errors were adjusted for clustering within practice37. Intervention was a fixed effect in these models, while practice and patients nested within practice were random effects. To control for possible confounding, we adjusted for patient and practice characteristics that varied at baseline. For the analysis of on-time testing and physiologic control, we adjusted for the patient’s baseline value of the outcome, and the mean practice value at baseline. For utilization, we controlled for age, sex, marital status, education, health literacy, race, insulin use, comorbidity, and nominal variables representing the subject’s community and hospital. Baseline estimates of utilization were not available. Because the data were skewed, we also compared the unadjusted distributions in the two groups with the nonparametric Wilcoxon rank sum test. For functional status, quality of life, and self-care activities, we controlled for baseline patient values, age, sex, marital status, education, health literacy, race, insulin use and comorbidity.
The intervention induced testing in the active group compared to the control group, possibly generating bias from asymmetric observation38. To minimize this, after censoring we mailed letters to overdue control subjects to stimulate testing and collected data for another 90 days. Tests done after censoring were used to assess physiologic control, but not the proportion on-time. Because of the asymmetric ascertainment of the final lab values, we performed an additional analysis using multiple imputation by chained equations39,40. This method imputes missing data using known information about each case with multiple simulations, and then combines the results into a single imputed result. Variables used in the imputation were age, sex, baseline test results, on-time status of the four target tests at baseline and censoring, residence (NY, VT, or other), vital status at censoring, treatment group (active or control), duration of participation, date of enrollment, and indicator variables for each practice and hospital. Five imputations of 10 cycles each were combined for the final analysis.
Asymmetry also occurred in ascertainment of death, with 301 deaths recorded in the intervention arm (7.7%) and 222 in the control arm (6.4%) (P=0.27). We learned about intervention subject deaths through returned mail or interactions with PCP offices over the 32 months of observation. In the control arm, there was no communication with the subjects until study conclusion when we asked the PCPs to review their patient rosters to note any subjects known to have died. We also searched the Social Security Death Index for deceased subjects41,42. Deaths in the intervention group were reviewed to assure that no death was attributable to the intervention.
We analyzed all data on an intention-to-treat basis and had complete data for on-time testing at censoring for every subject. We used imputation to estimate the final A1C value for those patients who had no result recorded within 9 months before or 3 months after censoring, and the final LDL value if no result was recorded within 18 months before or 3 months after censoring according to our pre-specified analytic plan. Analyses were performed with Stata 10 (Stata Corporation, College Station, TX).
Patients were observed for a mean of 32 months (range, 1 day to 47 months; Table 1). Demographic characteristics were similar to the population of Vermont43,44. The results of the SF-12 Physical and Mental Component Summaries were similar to national samples of patients with diabetes27. Only 2% of the subjects were uninsured, reflecting regional characteristics and recruitment of subjects who were receiving care.
Subjects received care in small practices: 31 solo practices and 33 practices of two to six providers (65 Family Medicine physicians, 35 Internists, 18 nurse practitioners, and 14 physician assistants). The mean number of patients per PCP was 101 (median 82; range 1–395). No significant differences were observed between the two groups at baseline (Table 1).
At censoring, intervention subjects were more likely to receive guideline-appropriate testing than those randomized to control (Table 2). After adjusting for baseline differences and for clustering within practices, the differences were significant for lipids (OR=1.39), creatinine (OR=1.40), and urine protein (OR=1.74). The improvement in A1C testing (OR=1.17) was not statistically significant.
Cholesterol and A1C levels were similar in the two groups at censoring (Table 2). Adjustment for baseline test results, practice performance, and clustering revealed no significant differences. Missing laboratory results occurred more frequently in the control group (A1C: 34% vs. 32%, P=0.09; LDL: 23% vs. 20%, P<0.001). Analyses using imputed data were similar (Table 2).
At the follow-up survey no significant differences were seen in blood pressure, body mass, functional status, quality of life, or self-care, except for an improvement in exercise habits (Table 3). Recalled utilization was significantly lower in the VDIS group (Table 4). After adjustment, estimated total costs of care per person in the year prior to censoring were $2,426 lower in the intervention group than the control group (95% CI = [−4,647, −$205] P=0.03 by multivariate linear regression; P=0.03 by unadjusted Wilcoxon rank-sum test).
The system integrated into clinical workflow with minimal disruption. Fax volume averaged 4/practice/day. Ongoing support required less than 1 hour of information technology professional support per month.
In this large-scale, randomized controlled trial, the VDIS resulted in significant improvements in test ordering with no change in physiologic control or functional status. In order to be generalizable to typical primary care practices with minimal disruption to workflow, VDIS does not require added personnel, hardware, or software.
A similar diabetes decision support intervention targeted at both providers and patients in a large multispecialty group found no improvement in laboratory outcomes and a negative impact on timeliness of test ordering47. However, the decision support was not delivered at the point of care and may have been as much as 6 weeks out of date. These negative findings highlight the importance of delivering decision support information that is both timely and actionable. Another study in 24 practices in Minnesota, which included a case manager function, showed improvements in both process and outcome15. Other interventions have resulted in improvement in process but not laboratory outcomes8,10,16.
Systematic reviews suggest that quality improvement strategies in diabetes care produce modest results45,46. VDIS uses 5 of the 11 recommended strategies for improving care46: audit and feedback, electronic registry, clinician reminders, patient reminders, and abbreviated patient education (in the form of alert letters when results are significantly out of range). This approach led to improvements in patient testing without changing physiologic outcomes.
There are several possible explanations for the failure of improved testing to lead to improved control in our study. The level of control at baseline was very good—the mean A1C was 7.1% with 55% of subjects below 7%, compared to 7.7% and 42% for a national sample2, suggesting a possible “ceiling effect”48. This is supported by the observation that the strongest effect was on nephropathy testing, which had the poorest baseline performance. It is unknown if VDIS would improve outcomes in a population with a poorer baseline level of control.
Perhaps some patients without diabetes were included in the study population despite being confirmed by their PCP. However, 92% of all subjects had at least one A1C>6% at some time during the study, suggesting well-controlled diabetes at baseline rather than over-diagnosis.
It is possible that VDIS alone is not a sufficiently intensive approach49,50. Interventions that add personnel15,45,46 or more intensive behavioral interventions51 have been somewhat successful, though not uniformly52. The US health care payment structure is currently not supportive of a coordinated, team-based approach to chronic illness53,54, though initiatives such as the Patient Centered Medical Home are underway55. Our study findings support the idea that practices interested in population-based chronic disease management may consider a laboratory-based registry and decision support system as a transition to a more fully integrated future including a broader team and improved electronic systems.
It was the generally accepted basis of this experiment that improved monitoring would lead to improved physiologic control which would, in turn, lead to fewer complications, better quality of life, and lower costs. The failure of improved monitoring to influence physiologic control in this study does not suggest that monitoring is unimportant. Measurement is an essential step in any quality improvement process56. The challenge remains in designing interventions that appropriately balance clinical impact and ease of dissemination and use. Subsequent work has shown us that VDIS can serve as platform to support other systems improvements such as automatic pre-ordering of tests, planned diabetes visits, specialty referral, and added personnel for health behavior coaching or case management57.
The strengths of this study are the large sample, rigorous design, and long follow-up in small, independent, community practices (without sophisticated electronic systems) representative of how most primary care is delivered today. While the subjects were less racially diverse than the rest of the country, more rural, and less likely to have access to specialty care, they are similar to other Americans with diabetes in regard to age, income, sex, comorbid conditions, tobacco use, functional status, and other social and demographic characteristics. The cost data in this study are based on patient recall of clinical events, rather than review of clinical or administrative records. However, patient recall of hospitalizations and emergency room use is generally good32 and any error would apply to both control and intervention groups. We recently analyzed data from another population using actual insurance claims and found a similar magnitude of cost reduction58. Because VDIS includes several components directed at both providers and patients, it is difficult to tell which parts led to the improvements in testing.
A chronic disease registry and decision support system based on easily obtainable laboratory data was feasible and acceptable to patients and providers. It improved the process of laboratory monitoring in primary care, but not physiologic control. Utilization was lower among patients provided the service with no adverse impact on functional status.
The authors are grateful for the generous contributions made by the patients, providers, and staffs of the participating practices and hospitals.
Funded by the National Institute of Diabetes and Digestive and Kidney Diseases (R01 DK61167 and K24 DK068380).
Conflict of Interest Drs. MacLean and Littenberg, and Mr. Gagnon are principals of Vermedx, Inc., which distributes clinical decision support systems based on this work.
Clinical Trials Registration Number: ClinicalTrials.gov—NCT00109369