Technology for Diabetes Care and Evaluation in the Veterans Health Administration: Longitudinal Approaches to Evaluate Health Care Quality and Outcomes: The Veterans Health Administration Diabetes Epidemiology Cohorts

J Diabetes Sci Technol. 2008 January; 2(1): 24–32.

Published online 2008 January.

PMCID: PMC2769712

Technology for Diabetes Care and Evaluation in the Veterans Health Administration

Longitudinal Approaches to Evaluate Health Care Quality and Outcomes: The Veterans Health Administration Diabetes Epidemiology Cohorts

Donald R. Miller, Sc.D.^1,² and Leonard Pogach, M.D., M.B.A.^3,⁴

¹Center for Health Quality, Outcomes and Economic Research, Bedford VA Medical Center, Bedford, Massachusetts

²Boston University School of Public Health, Boston, Massachusetts

³Center for Healthcare Knowledge Management, VA New Jersey Health Care System, East Orange, New Jersey

⁴University of Medicine and Dentistry, New Jersey Medical School, Newark, New Jersey

Correspondence to: Leonard Pogach, M.D., Center for Healthcare Knowledge Management, VA New Jersey Health Care System, 385 Tremont Avenue, East Orange, NJ 07018-1095; email address Leonard.Pogach/at/va.gov

Funding: This work was supported by grants from the Department of Veterans Affairs, including Health Services Research and Development (Drs. Miller and Pogach), the Quality Enhancement Research Initiative (Dr. Pogach), and Clinical Services Research and Development (Drs. Miller and Pogach).

Disclaimer: The opinions expressed are those of the authors and do not represent the opinion of the Department of Veterans Affairs.

This article has been cited by other articles in PMC.

Abstract

Objective

The Institute of Medicine proposed recently that, while current pay for performance measures should target multiple dimensions of care, including measures of technical quality, they should transition toward longitudinal and health-outcome measures across systems of care. This article describes the development of the Diabetes Epidemiology Cohorts (DEpiC), which facilitates evaluation of intermediate “quality of care” outcomes and surveillance of adverse outcomes for veterans with diabetes served by the Veterans Health Administration (VHA) over multiple years.

Methods

The Diabetes Epidemiology Cohorts is a longitudinal research database containing records for all diabetes patients in the VHA since 1998. It is constructed using data from a variety of national computerized data files in the VHA (including medical encounters, prescriptions, laboratory tests, and mortality files), Medicare claims data for VHA patients, and large patient surveys conducted by the VHA. Rigorous methodology is applied in linking and processing data into longitudinal patient records to assure data quality.

Results

Validity is demonstrated in the construction of the DEpiC. Adjusted comparisons of disease prevalence with general population estimates are made. Further analyses of intermediate outcomes of care demonstrate the utility of the database. In the first example, using growth curve models, we demonstrated that hemoglobin A1c trends exhibit marked seasonality and that serial cross-sectional outcomes overestimate the improvement in population A1c levels compared to longitudinal cohort evaluation. In the second example, the use of individual level data enabled the mapping of regional performance in amputation prevention into four quadrants using calculated observed to expected major and minor amputation rates. Simultaneous evaluation of outliers in all categories of amputation may improve the oversight of foot care surveillance programs.

Conclusions

The use of linked, patient level longitudinal data resolves confounding case mix issues inherent in the use of serial cross-sectional data. Policy makers should be aware of the limitations of cross-sectional data and should make use of longitudinal patient databases to evaluate clinical outcomes.

Keywords: A1c, amputations, databases, diabetes, registry

More than 20 million Americans have diabetes.¹ The direct (medical services) and indirect (disability and premature death) costs of treating diabetes exceed $130 billion per year.² Amputations, renal failure, visual loss, stroke, heart attack, and premature death reduce the length and quality of life of patients.³ However, a number of landmark studies have indicated that risk factor control can markedly decrease microvascular, and possibly macrovascular, complications of type 1 and type 2 diabetes.^4–7

Consequently, the federal government has set objectives for increased provision of care to persons with diabetes, as well as decreased rates of hospitalizations and complications. These rates are monitored on a population basis, with an emphasis on reducing disparities.^3,8 A number of private sector organizations and health care quality coalitions have developed, adopted, or endorsed process and intermediate outcomes for persons with diabetes for use by physicians and health care plans.^9–13

However, discerning actual progress in combating the diabetes epidemic has been difficult. National weighted population level reports from sources such as National Health and Nutrition Examination Surveys and Behavioral Risk Factor Surveillance System are generally based on comparisons of cross-sectional data, with risk adjustment for age and gender.¹⁴ Therefore, inferences made on changes in cross-sectional results in “quality” and “outcomes” can be biased by trends in “case mix.” For example, improvement in rates of diabetes-related adverse outcomes or intermediate outcomes of quality of care could result either from greater provision of appropriate care and/or an increase in the denominator of persons with incident onset diabetes (primarily type 2). Furthermore, at the regional level, such inferences are sensitive to assumptions about the estimated denominator of persons with diabetes.¹⁵

Evaluations of improvement within health care plans may also differ when results of randomized interventions with longitudinal follow-up¹⁶ are compared to operational reports using plan data.¹⁷ Some of these biases may result from the technical specifications used by the industry to define the denominator for performance measurement, i.e., a sampling frame for measurement that includes individuals who maintain enrollment with minimal disruption. Consequently, those who die within the year or leave the plan for any reason are not included.¹⁰ Thus, trends in outcomes based on serial cross-sectional data may be influenced by both in- and outmigrations of enrollees, in addition to the quality of care provided to enrollees, and inferences regarding performance confounded by selection biases. The Institute of Medicine emphasized that measures and rewards of performance should target multiple dimensions of care, initially including measures of technical quality, patient-centered care, and efficiency but transitioning toward longitudinal and health-outcome measures.¹⁸ Current national and industry snapshots are inadequate to meet this challenge; another approach is needed.

The Veterans Health Administration (VHA) provides a national laboratory in which to study trends in intermediate risk factor management and disease outcomes of diabetes on a national population level.¹⁹ The VHA provides comprehensive health care services to almost five million veterans annually and has the most advanced electronic health record in the United States.²⁰

Although most veterans who are users of the VHA system maintain some continuity of care within the VHA, more than half these patients are also enrolled in the federal Medicare program because of either age (65 years or older) or disability.²¹ For veterans with diabetes, this percentage rises to nearly three in four patients. Thus, in order for researchers to be able to obtain a more complete picture of these patients′ health care and outcomes, it is necessary to include Medicare claims with VHA patient data for care in the same period. While Medicare administrative data are not as rich as the clinical information in databases of the VHA, their utilization addresses continuity of follow-up and thus provides more accurate estimates of complication rates and resource utilization.

The Diabetes Epidemiologic Cohort (DEpiC) is a research database of all VHA patients with diabetes from 1997 to 2006.²² The objectives of the DEpiC are to determine diabetes prevalence and incidence, provide surveillance of rates of diabetes-related morbidity and mortality, and evaluate time-varying predictors of diabetes-related morbidity and mortality in the veteran population. The DEpiC has also been used to evaluate variation in the quality of diabetes care and outcomes with variation in race/ethnicity, mental health status, and other comorbidities and to advance methodological issues related to the use of computerized patient data for epidemiology and health services research.^23–42 In addition, the DEpiC is unique because of its size (over 1.5 million veterans with diabetes), national scope, and overrepresentation of individuals who are poor or have mental health conditions. This article reports on methodology used to develop the DEpiC and describes findings of methodological and policy importance in the evaluation of glycemic control and lower extremity amputations.

Methods

Data Acquisition

The health information infrastructure of the VHA makes the creation of a research registry possible, but it is necessary to obtain data from a number of sources and integrate it with appropriate quality checks (Figure 1). Available VHA data sources include inpatient and outpatient utilization data, prescription records, and laboratory test results, vital status, and patient surveys.⁴³ Most of these data are collected on an encounter/order/transactional basis at individual facilities with abstracts transferred to central repositories. In the past, Medicare data on veterans enrolled in the VHA were obtained directly from the Centers for Medicaid and Medicare Services [previously Health Care Finance Administration (HCFA)] with individual data use agreements and significant work required for accurate linkages and data processing. Currently, these data are obtained by the Veterans Affairs (VA) Information Resource Center and are made available to investigators under interagency data use agreements. Data elements include Beneficiary Files (Denominator File—one file for each year with demographics; Institutional Claims File (Part A); Non-Institutional Claims File-Part B (Physician/Supplier DME) Medicare Enrollment database (EDB); Group Health Master File—Medicare managed care enrollees; Provider Files; and typical claims data, collected on two forms UB-92 (also known as HCFA 1450)—used by institutional providers (hospitals in- and outpatients, home health agencies, hospices); and HCFA 1500—used by physicians and laboratories.

Figure 1.

Schematic of the Veterans Health Information Systems and Technology Architecture system supplemented with external Medicare claims data and mortality data to form the Diabetes Epidemiologic Cohort.

Reusing data from merges of all VHA patients with Medicare utilization has an additional advantage. The Medicare utilization files based on a national merge with VHA patient identifiers can be searched for patients who meet study criteria but who would not be identified in VHA files. For example, VHA patients with diabetes diagnoses in their Medicare claims do not all have such diagnoses in VHA data and may not get their prescriptions filled in the VHA. Searching the national files of Medicare utilization by VHA patients for the appropriate codes adds value to studies of specific conditions. Similar advantages may pertain to other study population definitions.

Data Quality

Merged Records

When merging VHA and Medicare data sets, it is advisable to check their face validity. The proportion of duplicate matches is important. For the VHA patient population, the distribution of the matches by age (e.g., the percent of those 65 and older who match and the number of matches for age less than 18) is also a potential indicator of problems.

Once the study sample has been confirmed, the next step is constructing data elements from VHA and Medicare files that are similar enough to allow them to be used in analyses. Although this will largely depend on the nature of the study, most studies of utilization will include hospitalization and outpatient utilization data. The first step any study must accomplish is reconciling fiscal years and calendar years. Because the VHA is a federal agency that utilizes fiscal years, it is necessary to convert Medicare calendar year files to fiscal years. In addition, because veterans enrolled in Medicare health maintenance organizations (HMO) do not have utilization data available, it is necessary to consider whether to include such individuals in individual analytic data sets.

Mortality

When the mortality status of all VHA patients is ascertained from four sources [the Beneficiary Identification Records Locator System, the Medical SAS Inpatient Datasets (for veterans who died during an inpatient stay), the Medicare Vital Status File, and the Social Security Administration's Death Master File], results are highly concordant with data from the National Death Index.⁴³

Laboratory and Pharmacy Data

VHA pharmacy and laboratory data are entered automatically and directly into the electronic health record (Veterans Health Information Systems and Technology Architecture) at the facility level, and there are monthly quality control evaluations. Nonetheless, it is possible that missing VHA and laboratory data can result from an incomplete transmission of facility VHA data collection or random nonentry of data, which can be addressed by the use of stochastic analyses.²⁹

There are also limitations in ascertaining non-VHA laboratory results and medication prescriptions that must be acknowledged. Although VHA data capture intermediate outcomes for most clinical users cross-sectionally, chart abstraction adds about 10% to cross-sectional adherence in the performance of low-density lipoprotein cholesterol (over 2 years) and hemoglobin A1c tests (over 1 year).⁴⁴ Consequently, inferences regarding processes of care are made most reliably using VHA fee-for-service Medicare data sets that capture whether tests are performed, although results are not available. Because the majority of veterans are of lower socioeconomic circumstances, we also assume that, prior to the implementation of the Medicare Part D benefit in 2006, we have captured medication usage on almost all veteran clinical users unless they are in a Medicare HMO for an extended period of time.

Demographic Data

Prior to 2004, patient race/ethnicity in the VHA information system was recorded based on an administrative or clinical employee's observation. Since 2004, the VHA has collected self-reported race in compliance with a new federal guideline from the Office of Management and Budget. Results from the VA Information Resource Center suggest that observer-recorded and self-reported data for whites and African-Americans can be used interchangeably across years without creating serious bias, but that observer-recorded race was less reliable for non-African-American minorities.⁴⁴

We have found it useful to improve the completeness and accuracy of race in three ways. Most patients have used VHA care for many years, and we have made use of the multiple entries for race over the years of data available. Furthermore, race/ethnicity in Medicare is self-reported and relatively complete. Combining Medicare data with enhanced VHA data on race/ethnicity decreased the proportions of missing data to 6.6% for race, <0.1% for gender (<0.1%), and 1.1% for marital status for patients prior to 2004. We have also demonstrated that evaluation of racial differences in the veteran population is sensitive to the inclusion of dual system use.³⁴

A third source for race/ethnicity data is the large national VHA surveys. For example, we have survey data from nearly 25% of the VHA patient population in 1999. Use of these data further enhances our race/ethnicity in significant subsets of the population. They also permit evaluation of the accuracy of known data and potential biases of unknown data. Consequently, we are able to compare VHA estimates of diabetes prevalence to the U.S. adult population by standardizing age, sex, and race (Figure 2).

Figure 2.

Comparison of standardized diabetes prevalence in the VHA and U.S. adult population (1999–2000). Adapted from Miller and colleagues²² and Centers for Disease Control and Prevention.⁵⁸

Results

Evaluation of Hemoglobin A1c Outcomes

Although the control of hemoglobin A1c levels is a major governmental priority in the health care of persons with diabetes, national trends in hemoglobin A1c values have been conducted using cross-sectional study designs in the absence of weighted, longitudinal population samples.¹⁴ Public reporting from accrediting organizations also utilizes cross-sectional data.¹⁰ However, hemoglobin A1c values are also related to patient level factors extraneous to health care systems, especially duration of diabetes and comorbid conditions. For example, even in the high-quality environment of landmark clinical trials,⁵ mean hemoglobin A1c levels rose over 10 years in study subjects with incident diabetes. Furthermore, these studies excluded most individuals with comorbid physical and mental health conditions that could minimize benefit and increase the risk of glycemic control. In addition, even in short-term (26–52 week) clinical studies with intention to treat to <7%, it is difficult to achieve this target in patients receiving insulin. In a meta-analysis of trials comparing glargine versus neutral protamine Hagedorn insulin, the mean achieved improvement in hemoglobin A1c was 7.58% in both groups, with about one-third in each reaching less than 7%.⁴⁶ Also, long-term follow-up is not available to evaluate whether the percent of “adherent” patients increased or decreased over time.

Consequently, multiple measures on individuals over time, termed “longitudinal data,” are crucial for assessing change over time within patients. Longitudinal data enable differentiation of cohort effects (patient population changes over time) from within-subject time effects. Longitudinal methods may give a better sense of how a patient's health care system is treating the disease over time than cross-sectional data, which overestimate improvement. This is illustrated in Figure 3.

Figure 3.

Comparison of trends of serial cross-sectional (with-in year) and longitudinal panel hemoglobin A1c results. A panel is composed of continued users with longer duration diabetes (in this panel the average duration was about 11 years), whereas cross-sectional (more ...)

In order to evaluate whether reductions seen in adherence to poor hemoglobin A1c following the introduction of performance measurement in the mid-late 1990s were possibly a consequence of the influx of new veterans,⁴⁷ we used a growth curve approach to fit longitudinal data of all veterans utilizing the VA from 1998 to 2000.^30,48 The deviation of patient level change in hemoglobin A1c from the average hemoglobin A1c growth curve for a given health care system is accounted for by random effects in the model. These random effects incorporate the correlation of multiple hemoglobin A1c measurements made on each patient. This use of growth curve methodology is attractive because it should result in meaningful assessments of quality of care as reflected in how health care systems influence patient hemoglobin A1c levels over time or, in other words, the patient response to the clinical setting.

We utilized this approach not only to evaluate overall population trends, but to evaluate facility level trends in the quality of care among diabetic veterans in 125 of the VHA facilities nationwide. The models ascribed to each facility an expected trajectory (“growth curve”) of hemoglobin A1c values for patients utilizing that facility as their home facility. A key methodological advance was the demonstration that hemoglobin A1c exhibits seasonality (Figure 4).²⁷ Our cohort was large enough to demonstrate that a winter–summer contrast was observed consistently in all age, sex, race, insulin use groups, and climate regions. We used the estimated facility level growth curve slopes to compare outcomes of diabetes care among the facilities, both with and without case mix adjustment.

Figure 4.

The use of longitudinal growth curves demonstrates hemoglobin A1c seasonality. The fluctuating hemoglobin A1c values are highest in late winter and lowest in late summer. Adapted from Tseng et al.²⁷

We reported that case mix-adjusted glycosylated hemoglobin values among veterans with diabetes decreased by -0.31% (range -1.90 to +1.03) over 2 years, indicating that improved glycemic control over time is not an artifact of changes in the population served by VA. These findings provide a convincing demonstration of the overall effectiveness of VA diabetic care management and suggest that with wider availability of the electronic health record, longitudinal data could eventually be used in larger health care plans.

Finally, our results on the evaluation of VHA facilities have demonstrated that risk adjustment of cross-sectional hemoglobin A1c using either administrative data³⁷ or self-reported health and socioeconomic status, as well as health behaviors,³⁹ has little predictive or explanatory value. Nonetheless, they significantly alter identifications of “best” and “worse” performing facilities.

Consequently, in the lack of a validated model to control for bias that could result from variation in case mix among facilities, the presence of seasonality that can lead to potential gaming, and inability to control for in- and outmigration, we have proposed that all or none cross-sectional threshold measures for optimal glycemic control are likely to be misleading for the purposes of public reporting and pay for performance.⁴⁹

Simultaneous Evaluation of Major, Minor, and Total Lower Extremity Amputations

Diabetes-related foot complications inflict an enormous individual and societal burden. Persons with diabetes have a 15% lifetime risk of a lower extremity complication, and such individuals are at increased risk of hospitalization, infection, amputation, and death.⁵⁰ Minorities and seniors incur a disproportionate number of amputations. Several U.S. governmental agencies utilize total amputation rates as a key indicator of foot health on a population basis.³ Healthy People 2010 have set a target of a 55% reduction in total amputation rates during the current decade, with a reduction in racial disparities,³ after amputation rates in 1990–2000 trended in the opposite direction from the target goals set for Healthy People 2000.

More recently, the Agency for Healthcare Quality and Research has utilized lower extremity amputations as prevention quality indicators for communities.⁵¹ However, the Organization for Economic Cooperation and Development⁵² has recommended major amputation rates for this purpose, in part because of the marked functional difference between minor and major amputations and in part because a minor amputation may precede a major one in the hopes of salvaging the limb.

However, any inferences regarding rates of amputations are less meaningful without individual level data. For example, the use of total amputation rates to assess population rates overestimates amputation rates because they are derived from the National Hospital Discharge Survey and the National Health Interview Survey. Because the National Hospital Discharge Survey samples hospital discharges, it does not necessarily reflect individual level results; persons hospitalized more than once a year will be overcounted.⁵³ Finally, because of recent increasing secular trends in incident diabetes, as well as increasing recognition of undiagnosed diabetes, the population risk for lower extremity complications may not be stable over multiple years.

However, in contrast to communities, health care systems are in a position to make improvements in foot health care delivery for their population of enrolled individuals, if provided with actionable information.⁵⁴ Individual level data permit definitions of persons with diabetes accurately and allow for determination of the maximal amputation level (major vs minor) per episode when multiple amputations were performed on a patient. Our methodology enabled us to evaluate present risk adjusted observed to expected amputation rates simultaneously for 22 regional units of the VHA using a four-quadrant map that could facilitate the management of foot care programs based on minor and major amputations (Figure 5).

Figure 5.

Quadrant rankings of regional outcomes based on the observed to expected (O:E) of major and minor amputations, with comparisons to total amputation rates. Adapted from Tseng et al.⁴²

Our proposed classification schema demonstrates that health care systems with similar total amputation rates can be differentiated further by observed to expected rates of major and minor amputations. We suggest that this detailed information can enable health care systems with access to individual level data to target the evaluation of foot health care delivery for persons with diabetes more accurately. For example, regions that are low outliers by major and minor amputation rates (first quadrant) could be benchmarked as best performers, thus warranting case study and dissemination of best practices. In contrast, those regions having higher rates of both major and minor amputation rates may need to be targeted immediately for a comprehensive program review and quality improvement interventions. Regions that are ranked as having high major but low minor rates (quadrant 4) could consider evaluating access and quality of their foot salvage care, as a low incidence of amputations could actually reflect an inappropriately conservative approach to nonhealing ulcers.⁵⁵ Finally, those with high minor rates and low major rates could assess their foot care systems to implement those that provide patient-centered care⁵⁶ and possibly improved outcomes.⁵⁷

Summary

The Diabetes Epidemiology Cohorts is a unique research resource used to study outcomes of veterans with diabetes, many of whom are older and Medicare enrolled. Through linked pharmacy, laboratory, and medical encounter data over multiple years, it is possible to evaluate determinates of intermediate outcomes and complications of diabetes, as well as to control for case mix issues inherent in the use of cross-sectional data.

This approach permits the use of special methods in epidemiologic and health services research to model dynamic cohorts with time-varying covariates, including physiological parameters (such as seasonality), risk adjustment (such as prevalent and incident diabetes), longitudinal analyses, and better definition of outcomes (such as amputation).

Consequently, the DEpiC has potential importance to federal policy makers in constructing and validating quality measures that are consistent with the challenges posed by the Institute of Medicine. It is a matter of concern that quality report cards structured around cross-sectional data may provide incentives for disenrolling the sickest patients and those with a longer duration of disease.¹⁸ The DEpiC provides a unique opportunity to construct less “gameable” quality measures that assess within-patient longitudinal care through multiple repeated measures on the same individual, enabling distinction of cohort effects (patient population changes over time) from within-subject time effects, which are often used in comparative quality assessments. It also may inform federal policy makers as to the impact of such factors as provision of preventive care and care coordination upon adverse diabetes-related outcomes of Medicare beneficiaries. Finally, although regional health maintenance organizations may have similar electronic health record capacity, the national scope of the VHA clinical population, and its potential for the evaluation of chronic complex illnesses, may be unique.

Abbreviations


DEpiC	Diabetes Epidemiologic Cohort

HCFA	Health Care Finance Administration

HMO	health maintenance organizations

VA	Veterans Affairs

VHA	Veterans Health Administration

References

1. National Center for Chronic Disease Prevention and Health Promotion. National Diabetes Surveillance System [cited 2007 Nov 4] Available from: http://www.cdc.gov/diabetes/statistics/index.htm.

2. Hogan P, Dall T, Nikolov P. American Diabetes Association. Economic costs of diabetes in the US in 2002. Diabetes Care. 2003;26:917–932. [PubMed]

3. U. S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion. Healthy People 2010. Chapter 5: Diabetes [cited 2007 Nov 4] Available from: http://www.healthypeople.gov/data/midcourse/comments/faobjective.asp?id=5&subid=10.

4. Diabetes Control Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993;329(14):977–986. [PubMed]

5. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33) Lancet. 1998;352:837–853. [PubMed]

6. Heart Outcomes Prevention Evaluation Study Investigators. Effects of ramipril on cardiovascular and microvascular outcomes in people with diabetes mellitus: results of the HOPE study and MICRO-HOPE substudy. Lancet. 2000;355(9200):253–259. [PubMed]

7. Collins R, Armitage J, Parish S, Sleigh P, Peto R. Heart Protection Study Collaborative Group. MRC/BHF Heart Protection Study of cholesterol-lowering with simvastatin in 5963 people with diabetes: a randomised placebo-controlled trial. Lancet. 2003;361(9374):2005–2016. [PubMed]

8. National Healthcare Quality Report. Agency for Healthcare Research and Quality [cited 2007 Nov 4] 2006. Available from: http://www.ahrq.gov/qual/nhqr06/nhqr06.htm.

9. National Voluntary Consensus Standards for Ambulatory Care July 2007. Standardizing Ambulatory Care Performance Measures. National Quality Forum [cited 2007 Nov 4] Available from: http://www.qualityforum.org/pdf/ambulatory/tbAMBALLMeasuresendorsed%2007-27-07.pdf.

10. National Committee for Quality Assurance. Technical Manual. Washington DC: 2006. HEDIS 2007 Volume 2.

11. National Diabetes Quality Improvement Alliance [cited 2007 Nov 4] Available from: http://www.nationaldiabetesalliance.org/organizations.html.

12. Recommended Starter Set : Clinical Performance Measures for Ambulatory Care. Agency for Healthcare Rearch and Quality [cited 2007 Nov 4] Available from: http://www.ahrq.gov/qual/aqastart.pdf.

13. 2007 Physician Quality Reporting Initiative. Specifications Document. Centers for Medicaid and Medicare Services [cited 2007 Nov 4] Available from: http://www.cms.hhs.gov/PQRI/downloads/Measure_Specifications_061807.pdf?agree=yes&next=Accept.

14. Saaddine JB, Cadwell B, Gregg EW, Engelgau MM, Vinicor F, Imperatore G, Narayan KM. Improvements in diabetes processes of care and intermediate outcomes: United States, 1988-2002. Ann Intern Med. 2006;144(7):465–474. [PubMed]

15. Shwartz M, Pekoz EA, Ash AS, Posner MA, Restuccia JD, Iezzoni LI. Do variations in disease prevalence limit the usefulness of population-based hospitalization rates for studying variations in hospital admissions? Med Care. 2005;43(1):4–11. [PubMed]

16. O'Connor PJ, Desai J, Solberg LI, Reger LA, Crain AL, Asche SE, Pearson TL, Clark CK, Rush WA, Cherney LM, Sperl-Hillen JM, Bishop DB. Randomized trial of quality improvement intervention to improve diabetes care in primary care settings. Diabetes Care. 2005;28(8):1890–1897. [PubMed]

17. HealthPartners Optimal Diabetes Care [cited 2007 Nov 4] Available from: http://www.qualitymeasures.ahrq.gov/summary/summary.aspx?ss=1&doc_id=9263.

18. Fisher ES. Paying for performance–risks and recommendations. N Engl J Med. 2006;355(18):1845–1847. [PubMed]

19. Krein SL, Hayward RA, Pogach L, BootsMiller BJ. Department of Veterans Affairs′ Quality Enhancement Research Initiative for Diabetes Mellitus. Med Care. 2000;38(6 Suppl 1):I38–48. [PubMed]

20. Kupersmith J, Francis J, Kerr E, Pogach L, Kolodner R, Perlin J. Advancing evidence-based care in diabetes through health information technology: lessons from the Veterans Health Administration. Health Aff (Millwood) 2007;26(2):w156–68. [PubMed]

21. Shen Y, Hendricks A, Zhang S, Kazis LE. VHA enrollees′ health care coverage and use of care. Med Care Res Rev. 2003;60(2):253–267. [PubMed]

22. Miller DR, Safford MM, Pogach LM. Who has diabetes? Best estimates of diabetes prevalence in the Veterans Health Administration based on computerized patient data. Diabetes Care. 2004;(27 Suppl 2):B10–21. [PubMed]

23. Helmer DA, Tseng CL, Brimacombe M, Rajan M, Stiptzarov N, Pogach L. Applying diabetes-related Prevention Quality Indicators to a national cohort of veterans with diabetes. Diabetes Care. 2003;26(11):3017–3023. [PubMed]

24. Safford M, Eaton L, Hawley G, Brimacombe M, Rajan M, Li H, Pogach L. Disparities in use of lipid lowering medications among people with diabetes. Arch Intern Med. 2003;163(8):922–928. 28. [PubMed]

25. Tseng CL, Rajan M, Miller DR, Hawley G, Crystal S, Xie M, Tiwari A, Safford M, Pogach L. Use of administrative data to risk adjust amputation rates in a national cohort of medicare-enrolled veterans with diabetes. Med Care. 2005;43(1):88–92. [PubMed]

26. Tseng CL, Greenberg JD, Helmer D, Rajan M, Tiwari A, Miller D, Crystal S, Hawley G, Pogach L. Dual-system utilization affects regional variation in prevention quality indicators: the case of amputations among veterans with diabetes. Am J Manag Care. 2004;10(11 Pt 2):886–892. [PubMed]

27. Tseng CL, Brimacombe M, Xie M, Rajan M, Wang H, Kolassa J, Crystal S, Chen TC, Pogach L, Safford M. Seasonal patterns in monthly A1c values. Am J Epidemiol. 2005;161(6):565–574. [PubMed]

28. Frayne SM, Halanych J, Miller DR, Lin H, Wang F, Pogach L, Sharkansky EJ, Keane T, Skinner KM, Rosen CS, Berlowitz DR. Disparities in diabetes care: impact of mental illness. Arch Intern Med. 2005;165(22):2631–2638. [PubMed]

29. Pogach L, Xie M, Shentue Y, Tseng CL, Maney M, Rajan M, Tiwari A, Kolassa J, Helmer D, Crystal S, Safford M. Diabetes healthcare quality report cards: how accurate are the grades? Am J Manag Care. 2005;11:797–804. [PubMed]

30. Thompson W, Wang H, Xie M, Kolassa J, Rajan M, Tseng CL, Zhang Q, Vardi V, Pogach L, Safford MM. Assessing quality of diabetes care by measuring longitudinal changes in hemoglobin A1c in the Veterans Health Administration. Health Serv Res. 2005;40:1818–1835. [PMC free article] [PubMed]

31. Greenberg JD, Tiwari A, Rajan M, Miller D, Natarajan S, Pogach L. Determinants of sustained uncontrolled blood pressure in a national cohort of persons with diabetes. Am J Hypertens. 2006;19(2):161–169. [PubMed]

32. Kern EF, Maney M, Miller DR, Tseng CL, Tiwari A, Rajan M, Aron D, Pogach L. Failure of ICD-9-CM codes to identify patients with comorbid chronic kidney disease in diabetes. Health Serv Res. 2006;41(2):564–580. [PMC free article] [PubMed]

33. Tseng C, Sambamoorthi U, Tiwari A, Rajan M, Frayne S, Findley P, Pogach L. Are there gender differences in diabetes care among veterans? J Gen Intern Med. 2006;(21 Suppl 3):S47–53. [PMC free article] [PubMed]

34. Halanych JH, Wang F, Miller DR, Pogach LM, Lin H, Berlowitz DR, Frayne SM. Racial/ethnic differences in diabetes care for older veterans: accounting for dual health system use changes conclusions. Med Care. 2006;44(5):439–445. [PubMed]

35. Sambamoorthi U, Tseng CL, Rajan M, Anjali T, Findley PA, Pogach L. Initial nontraumatic lower extremity amputations among veterans with diabetes. Med Care. 2006;44(8):779–787. [PMC free article] [PubMed]

36. Tseng CL, Sambamoorthi U, Tiwari A, Rajan M, Findley P, Pogach L. Diabetes care among veteran women with disability. Womens Health Issues. 2006;16(6):361–371. [PMC free article] [PubMed]

37. Maney M, Tseng CL, Safford M, Miller D, Pogach L. Impact of self-reported patient characteristics upon assessment of glycemic control in the Veterans Health Administration. Diabetes Care. 2007;30(2):245–251. [PubMed]

38. Tiwari A, Tseng CL, Kern EF, Maney M, Miller DR, Pogach L. Facility variation in utilization of angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers in patients with diabetes mellitus and chronic kidney disease. Am J Manag Care. 2007;13(2):73–79. [PubMed]

39. Pogach LM, Tiwari A, Maney M, Rajan M, Miller DR, Aron DC. Should mitigating comorbidities be considered in assessing healthcare plan performance in achieving optimal glycemic control. Am J Manag Care. 2007;13(3):133–140. [PubMed]

40. Zhang Q, Safford M, Miller D, Crystal S, Rajan M, Tseng CL, Pogach L. Short term statin exposure is associated with reduced all-cause mortality in persons with diabetes. Med Care. 2007;45(4):308–314. [PubMed]

41. Kahler KH, Rajan M, Rhoads GG, Safford MM, Demissie K, Lu SE, Pogach LM. Impact of oral antihyperglycemic therapy on all-cause mortality among patients with diabetes in the Veterans Health Administration. Diabetes Care. 2007;30(7):1689–1693. [PubMed]

42. Tseng CL, Helmer D, Rajan M, Tiwari A, Miller D, Crystal S, Safford M, Greenberg J, Pogach L. Evaluation of regional variation in total, major, and minor amputation rates in a national health-care system. Int J Qual Health Care. 2007 Dec;19(6):368–376. [PubMed]

43. Sohn MW, Arnold N, Maynard C, Hynes DM. Accuracy and completeness of mortality data in the Department of Veterans Affairs. Popul Health Metr. 2006 Apr 10;4:2. [PMC free article] [PubMed]

44. Sohn MW, Zhang H, Arnold N, Stroupe K, Taylor BC, Wilt TJ, Hynes DM. Transition to the new race/ethnicity data collection standards in the Department of Veterans Affairs. Popul Health Metr. 2006 Jul 6;4:7. [PMC free article] [PubMed]

45. Kerr EA, Smith DM, Hogan MH, Krein SL, Pogach L, Hofer TP, Hayward RA. Comparing clinical automated, medical record, and hybrid data sources for diabetes quality measures. Jt Comm J Qual Improv. 2002;28:555–565. [PubMed]

46. Rosenstock J, Dailey G, Massi-Benedetti M, Fritsche A, Lin Z, Salzman A. Reduced hypoglycemia risk with insulin glargine: a meta-analysis comparing insulin glargine with human NPH insulin in type 2 diabetes. Diabetes Care. 2005;28(4):950–955. [PubMed]

47. Sawin CT, Walder DJ, Bross DS, Pogach LM. Diabetes process and outcome measures in the Department of Veterans Affairs. Diabetes Care. 2004;(27 Suppl 2):B90–4. [PubMed]

48. Growth Curve Model. Appendix to reference 30 [cited 2007 Nov 4] Available from: http://www.stat.rutgers.edu/~slothrop/VA/Appendix.htm.

49. Pogach L, Engelgau M, Aron D. Measuring progress toward achieving hemoglobin A1c goals: pass/fail or partial credit. JAMA. 2007;297(5):520–523. [PubMed]

50. Reiber GE, Boyko EJ, Smith DG. Diabetes in America. Washington, DC: U.S. Govt. Printing Office; 1995. Lower extremity foot ulcers and amputations in diabetes; pp. 409–428. National Diabetes Data Group, editors.

51. Guide to Prevention Quality Indicators. AHRQ Quality Indicators [cited 2007 Nov 4] Available from: http://www.qualityindicators.ahrq.gov/downloads/pqi/pqi_guide_v31.pdf.

52. Nicolucci A, Greenfield S, Mattke S. Selecting indicators for the quality of diabetes care at the health systems level in OECD countries. Int J Qual Health Care. 2006;(18 Suppl 1):26–30. [PubMed]

53. Center for Organization and Delivery Studies, Agency for Healthcare Research and Quality, Rockville, Maryland. Epidemiology and Statistics Br, Div of Diabetes Translation, National Center for Chronic Disease Prevention and Health Promotion, CDC. Hospital discharge rates for nontraumatic lower extremity amputation by diabetes status–United States, 1997 [cited 2007 Nov 4] Available from: http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5043a3.htm.

54. Pogach L, Charns MP, Wrobel JS, Robbins JM, Bonacker KM, Haas L, Reiber GE. Impact of policies and performance measurement upon the development of organizational coordinating strategies for chronic care delivery. Am J Manag Care. 2004;10:171–180. [PubMed]

55. Jeffcoate WJ, van Houtum WH. Amputation as a marker of the quality of foot care in diabetes. Diabetologia. 2004;47(12):2051–2058. [PubMed]

56. Wrobel JS, Charns MP, Bonacker KM, Reiber GE, Pogach L. Diabetes-related foot care at 10 Veterans Affairs medical centers: must do's associated with successful microsystems. Jt Comm J Qual Patient Saf. 2006;32(4):206–213. [PubMed]

57. Wrobel JS, Charns MP, Diehr P, Robbins JM, Reiber GE, Bonacker KM, Haas LB, Pogach L. The relationship between provider coordination and diabetes-related foot outcomes. Diabetes Care. 2003;26(11):3042–3047. [PubMed]

58. Centers for Disease Control and Prevention (CDC) Prevalence of diabetes and impaired fasting glucose in adults–United States, 1999-2000. MMWR Morb Mortal Wkly Rep. 2003 Sep 5;52(35):833–837. [PubMed]

Articles from Journal of Diabetes Science and Technology are provided here courtesy of
Diabetes Technology Society