|Home | About | Journals | Submit | Contact Us | Français|
Although others have reported national disparities in the quality of diabetes care between the Veterans Affairs (VA) and private health care delivery systems, it is not known whether these differences persist among internal medicine resident providers in academic settings.
We compared the quality of diabetes primary care delivered by resident physicians in either a private academic health care system (AHS) or its affiliated VA health care system.
Cross-sectional observational study
We included patients who: had a diagnosis of diabetes, had >2 primary care visits with the same resident provider during 2005, and were not separately managed by an attending physician or endocrinologist. A total of 640 patients met our criteria and were included in the analysis.
Compared to the VA, patients in the AHS were more likely to be younger, female, have fewer medications, and be treated with insulin, but had less comorbidity. Patients in the VA were more likely to be referred for an annual eye exam (94% vs. 78%), receive lipid screening (88% vs. 74%), receive proteinuria screening (63% vs. 34%), and receive a complete foot exam (85% vs. 32%) in analyses adjusted for patient demographics and comorbidities (p-value <0.001 for all comparisons). In adjusted analyses, there were no significant differences in HbA1c, blood pressure, or LDL cholesterol control.
In spite of similar resident providers and practice models, there were substantial differences in the diabetes quality of care delivered in the VA and AHS. Understanding how these factors influence subsequent practice patterns is an important area for study.
Internal medicine residency training is a unique period of medical education when physicians develop long-standing practice patterns. Several recent position statements have called for reform of ambulatory care training to place residents in high functioning practice settings that provide the highest quality of care.1–3 The Accreditation Counsel for Graduate Medical Education recognizes practice-based learning and improvement as a core competency;4 however, formal measurement of the quality of care delivered by residents is not a part of traditional performance evaluation.
Others have reported shortcomings in management of diabetes and other chronic diseases by residents,5–8 which mirrors the widespread deficiencies reported in the United States.9,10 While internal medicine training programs routinely assess resident knowledge of accepted standards of care, knowledge does not always lead to appropriate practice.11–13 There are ongoing efforts to adapt the practice models in resident clinics to meet the challenges of chronic disease care;14 however, change is often slow and balanced among the competing financial and educational demands in graduate medical education.
One factor consistently associated with higher quality diabetes care is receiving services in the Veterans Affairs (VA) health system.15–18 The VA has implemented system-wide initiatives to better measure, report, and improve the quality of care delivered in their system. These efforts have resulted in substantial improvements in both process and outcome indicators of quality for diabetes. Although many residency training programs use both VA and private academic clinics for ambulatory care training, it is not known whether delivery of primary care in the VA health care system results in higher quality resident-delivered care.
We sought to compare the quality of diabetes primary care delivered by internal medicine resident physicians from the same internal medicine training program in either an academic health care system (AHS) or the Veterans Affairs health care system (VA).
The study was approved by the Institutional Review Board at both Duke University Medical Center and the Durham Veterans Affairs Medical Center. We conducted a cross-sectional medical records review of data from January 1, 2005 through December 31, 2005. Both the Duke University and Veterans Affairs health care systems employed electronic medical records for storage of clinic notes and laboratory results. All information was obtained by manual review of the electronic medical record notes by a research assistant, with the review limited to information during the 12-month study period in 2005.
Adult patients ≥18 years of age with a diagnosis of diabetes based on an International Classification of Diseases, Ninth Revision (ICD-9) code of 250.xx receiving care at the eligible clinics were identified for potential inclusion. To be included in the study, patients had to be assigned a resident primary care provider during 2005. Patients were excluded at the time of chart review if they were: co-managed by either an endocrinologist or a generalist attending physician separate from the resident’s care, seen by their resident provider less than twice during 2005, or had life-limiting co-morbidities that might significantly influence the diabetes goals of care. These included diagnoses of AIDS, metastatic cancer, use of home oxygen, or enrollment in hospice for any reason.
All primary care providers were residents in the Duke Internal Medicine Residency Program or the combined Medicine/Pediatrics or Medicine/Psychiatry residency programs. Residents were categorized according to their number of years in the residency program at the end of 2005, with those in their final year of training categorized as “senior residents” and all other categorized as “junior residents.” Patients cared for by interns at the end of 2005 were excluded due to the recent change in primary care provider. Residents were assigned to a clinic location for their primary care continuity clinic and remained there throughout their residency. Although there was no randomization procedure for clinic assignments, residents in the categorical program were usually assigned arbitrarily to ensure an adequate number of residents per site.
Patients in the AHS were selected from two different Duke University Health System resident clinics, while VA patients were from one VA resident clinic. The model for care in all three clinics was similar in that each resident had an assigned patient panel, for which they provided regular primary care. The residents would review all patients with faculty preceptors from the Duke Department of Medicine at the time of the patient visit, and the faculty would cosign the resident’s notes. Residents provided primary care at their clinic site an average of 1 half-day per week. The faculty physicians were general internists, with a small number of geriatricians and endocrinologists at the AHS who supervised the residents’ care. The ratio of residents to faculty physicians was similar in the two clinics. There was significant attending cross-over between the two health-care systems with several VA-based faculty physicians who provided attending supervision in the AHS clinics and vice versa.
Although the resident providers and supervising faculty were a part of the same Department of Medicine, there were important differences in the two health care systems, which are displayed in Table 1. These differences were largely in the electronic health record available at each site and are compared using criteria adapted from DesRoches and colleagues.19 In addition, there were three supplemental services for managing diabetes. First, the VA had an active diabetes quality monitoring and feedback for all primary care providers that occurred at regular intervals. Second, both clinics utilized clinical pharmacists to see patients and adjust medications as appropriate to achieve diabetes, blood pressure, and lipid targets of therapy. Finally, both clinics offered diabetes classes conducted in a group setting to offer self-management support.
Prior to manual chart review, we queried the electronic tracking and billing database for each clinic. This identified the list of potentially eligible patients seen at each clinic according to an ICD-9 code for a diabetes (250.x) related visit during the 12-month period. From this database, we were also able to collect the patient’s age, race, gender, and billing codes for important co-morbidities, including hypertension, chronic kidney disease, congestive heart failure, and coronary heart disease. Within the academic health care system we also collected information on health insurance.
Following electronic data pull, all patient charts underwent manual abstraction by trained research assistants. This abstraction verified patient demographic and comorbid conditions obtained from the electronic data pull. In addition, the abstractor collected patient information on the diabetes outcomes and additional covariates that may affect the outcomes. The latter included the total number of medications used by a patient (excluding topical creams and medical supplies), as medication count has been previously validated as a surrogate measure of comorbidity.20 We also recorded whether or not the patient was treated with insulin, as this has been associated with clinical outcomes21 and may identify patients in whom glycemic control is more difficult.
Two separate research assistants abstracted the data from patient charts. Each abstractor underwent training on chart abstraction and utilized a standardized electronic abstraction form to abstract an initial set of ten test charts. Their abstraction results were reviewed with the principal investigator in person, and feedback was given. This process was continued with an additional ten charts until there was agreement between the abstractor and study investigator on the data abstracted from the medical record.
The diabetes quality measures were taken from the Diabetes Quality Improvement Project (DQIP) accountability quality improvement set and are incorporated in both the VA External Peer Review Program and the American Diabetes Association Provider Recognition Program.22 For intermediate outcomes of disease control (i.e., HbA1c, LDL cholesterol, and blood pressure), we report the most recent ambulatory reading during the period under review. We did not have access to ophthalmology records outside of each health care system; therefore, we gave credit for the eye exam quality measure if there was any referral for retinopathy screening or reference to results obtained during the period under study in the progress notes. For urine microalbumin screening and monitoring, we gave credit for any urine microalbumin screen ordered during the period under study. For the comprehensive foot exam, we separately recorded visual inspection, assessment of pedal pulses, and monofilament testing. We defined a comprehensive foot exam as including all three elements. Because a comprehensive foot exam may be performed by non-physician providers (podiatrists or nurses) in the VA system as a part of the computer clinical reminder, we also recorded the provider performing the exam. A patient was considered to have aspirin use counseling when it was listed as an active medication or if there was a documented recommendation for patients to take aspirin in the progress note.
Continuous variables were summarized by the mean and standard deviation, while categorical variables were presented as frequency and proportions. Chi-squared tests were used to compare categorical variables, and t-tests were used for comparison between continuous variables. To evaluate the quality of care measures, we analyzed each outcome using logistic regression and examined both unadjusted and adjusted model results. In addition to our main effect of interest, health care system, the following patient-level covariates were included in adjusted models: age, race (white vs nonwhite), coronary artery disease, congestive heart failure, hypertension, chronic kidney disease, total number of medications, and whether the patient was treated with insulin. Modeling results are presented as expected outcome percentages and associated 95% confidence intervals for each health system. P-values for the difference between health systems are also provided. Expected percentages were obtained by setting the values of all covariates to their observed mean (or to the proportion of those experiencing the event for dichotomous variables) in the cohort. Because many of the provider clusters were very small, we did not adjust for provider clustering. All data were analyzed using SAS statistical software (SAS Institute, Cary, NC).
Based on the initial electronic screening of eligible patients with a diagnosis of diabetes and seen in a resident teaching clinic, we identified 602 potentially eligible patients in the academic health care system and 327 patients in the VA health care system. Each patient then underwent chart review by a research assistant to determine eligibility for inclusion. The reasons for exclusion were similar for patients in both the VA and AHS and are presented in the Figure Figure11.
The patient characteristics for those included in the two health care systems are shown in Table 2. Patients with diabetes in the VA health care system were older (mean age 64.2 years vs. 61.5 years; p=0.008), more likely to be male (93% vs. 34%; p<0.001), and more likely to have co-morbid coronary artery disease (49% vs. 25%; p<0.001), chronic kidney disease (16% vs. 9%; p=0.02) and hypertension (97% vs. 84%; p<0.001). There was a higher proportion of African-American patients in the academic health care system (72% vs. 49%; p<0.001). Only two patients in the AHS and no patient in the VA were identified as having Hispanic ethnicity; these patients were categorized in the racial category “Other” in Table 2. Patients in the academic health care system took fewer medications than patients in the VA (8.7 vs. 11.3; p<0.001); however, they were more likely to be treated with insulin (45% vs. 32%; p=0.001). The mix of junior and senior residents was similar at both clinic sites. The average number of included patients per resident was 10.5 (SD=4.0) in the VA compared to 7.2 (SD=4.0) in the AHS.
The unadjusted diabetes quality outcomes in each health care system are presented in Table 3. Patients in the VA health care system were significantly more likely than patients in the AHS to receive each of the recommended processes of care except an annual HbA1c. Prior to adjusting for patient differences, patients in the VA had a lower mean DBP (73 mmHg vs. 76 mmHg; p=0.001) and a lower HbA1c (7.2% vs. 7.9%; p<0.001) than patients in the AHS. There was no difference in the other intermediate outcomes of risk factor control.
The adjusted analyses comparing processes and outcomes between the two sites are presented in Table 4. After adjusting for patient characteristics, the differences in process of care measures were reduced; however, patients in the VA were still significantly more likely to be referred for an annual eye exam and to receive lipid screening, nephropathy screening, and a foot exam. After this adjustment, there were no longer any statistically significant differences in the intermediate outcomes of disease control.
In spite of the similarity in providers, patients receiving care from resident physicians in a VA system were significantly more likely to receive recommended diabetes processes of care than patients in a private academic health care system. After adjusting for differences in patient characteristics, there were no significant differences in glycemic, blood pressure, or cholesterol control between the two health care systems. These data suggest that the gap in diabetes quality of care between the VA health care system and private academic health care systems is present even in new resident providers who deliver care under the close supervision of faculty physicians.
Prior studies have demonstrated that patients receiving chronic disease care within the Veterans Affairs health system are more likely to receive recommended services and achieve better disease control.15,16,18 Four years after the publication of the Translating Research Into Action for Diabetes (TRIAD) study documenting differences in diabetes care between the VA and commercial managed care, our study provides evidence that this disparity persists. Our study is the first to compare diabetes quality of care between health care systems when care is provided by physicians at an early and uniform period in medical training and from a single training program. Comparing residents from a single training program reduces any bias from differences in provider knowledge, training, and experience that may influence prior comparisons between the VA and private health care delivery.
It is important to note that both health care systems in this study performed better on most of the process of care measures than national reports of diabetes quality of care.23 This may be due in part to secular trends toward improved diabetes care since the publication of data from TRIAD and the National Health and Nutrition Examination Survey. Alternatively, it may reflect higher quality of care in resident primary care clinics compared to usual community practice. Others have reported higher than average quality of care in resident training clinics than their related faculty practices.24,25 This may be due in part to residents having lighter patient loads than a busy faculty practice or the regular review and real-time feedback from the attending providers, which may contribute to higher delivery of recommended services.26
While both systems employed an electronic medical record for patient notes, laboratory tests, and procedures, the most notable difference between the two health care systems was the VA medical record systems’ clinical reminders and quality managers who provide ongoing provider feedback on performance. This system has been previously shown to significantly improve residents compliance with standards of ambulatory care.27 These clinical reminders in the VA can also prompt non-physician providers to perform the recommended processes of care. The impact of this is particularly seen in performance of the complete foot exam, more than half of which were performed by a provider other than the resident physician in the VA. Residents in the VA system also saw fewer patients than their colleagues in the AHS and may have provided higher quality care simply because of having more time with each patient.
While our findings suggest that the diabetes process of care outcomes significantly differed between the two health care systems, it is not clear that this translates to differences in patient-important clinical outcomes. Higashi and colleagues found that better performance on process quality measures was associated with improved survival among older adults.28 Similarly, better documentation of quality of care is associated with lower mortality in patients with acute myocardial infarction.29 However, this link between process of care and patient outcomes has not been well established for diabetes. The intermediate outcome measures of HbA1c, BP, and LDL-C are more closely linked to patient-important outcomes, and these measures were similar between the two health care systems in spite of substantial differences in the process of care. Further work is needed to better define the relationship between diabetes process of care and patient outcomes.
Some caution must be used in interpreting our results. While the quality of diabetes care clearly differed between these two health care systems, our results may or may not generalize to other sites. It is also possible that VA reminders facilitate better documentation of care rather than actual differences in care, something our study is unable to differentiate. In addition, there may be significant differences in the two patient populations that contribute to our findings and were not adequately addressed in the analysis. Consistent with the findings of others, the process of care measures were less sensitive to adjustment for patient covariates than the intermediate outcomes of disease control, suggesting our findings for process of care measures would likely be robust to adjustment for additional patient variables.
There are several unanswered questions about the relationship between residents’ educational experiences and their future practice patterns. While professional societies have recommended that residents practice in the highest quality outpatient clinics possible,1–3 it is not known whether physicians’ clinic experience during residency leads to durable changes in downstream practice patterns. Training physicians to provide high quality care includes several domains including knowledge, judgment, communication skills, work ethic, and professionalism. Many of these are difficult to measure and are not reflected in the disease focused quality measurement performed in this study. The quality measures we evaluated are appropriate for evaluating diabetes care because they are our best link to preventable patient-important outcomes; however, our findings suggest that these quality measurements better reflect the design and organization of the health care system rather than the knowledge or aptitude of an individual provider. This is particularly important as provider report cards are proposed as means to evaluate and allocate performance pay for primary care providers. While our results suggest continued room for improvement in chronic disease care, it is also encouraging that provider practice patterns for diabetes care are significantly shaped and potentially improved by high quality systems of care.
This research was supported by the The Joseph C. Greenfield, Jr. Scholars Endowment. The first author (BJP) was supported by a KL2 career development award through Duke University and the NIH (KL2 RR024127). This research was presented as a poster at the VA HSR&D 2009 National Meeting in Baltimore, MD, on February 11, 2009. The authors would also like to acknowledge Beverly McCraw, Jennette Wood Crowley and Mike Harrelson for their assistance with data collection.
Conflicts of Interest The authors have no conflicts of interest to report.
This research was supported by the The Joseph C. Greenfield, Jr. Scholars Endowment. The first author (BJP) was supported by a KL2 career development award through Duke University and the NIH (KL2 RR024127). This research has been presented as a poster at the VA HSR&D 2009 National Meeting in Baltimore, MD, on February 11, 2009.