PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of qualsafetyQuality and Safety in Health CareCurrent TOCInstructions for authors
 
Qual Saf Health Care. 2007 August; 16(4): 248–251.
PMCID: PMC2464944
NIHMSID: NIHMS44290

Estimating costs of quality improvement for outpatient healthcare organisations: a practical methodology

Abstract

Background

Outpatient healthcare organisations worldwide participate in quality improvement (QI) programmes. Despite the importance of understanding the financial impact of such programmes, there are no established standard methods for empirically assessing QI programme costs and their consequences for small outpatient healthcare organisations.

Objective and methods

The costs and cost consequences were evaluated for a diabetes QI programme implemented throughout the USA in federally qualified community health centres. For five case study centres, survey instruments and methods for data analysis were developed.

Results

Two types of cost/revenue were evaluated. Direct costs/revenues, such as personnel time, items purchased and grants received, were evaluated using self‐administered surveys. Cost/revenue consequences, which were cost/revenue changes that may have occurred due to changes in patient utilisation or physician behaviour, were evaluated using electronic billing data. Other methods for evaluating cost/revenue consequences if electronic billing data are not available are also discussed.

Conclusion

This paper describes a practical taxonomy and method for assessing the costs and revenues of QI programmes for outpatient organisations. Results of such analyses will be useful for healthcare organisations implementing QI programmes and also for policy makers designing incentives for QI participation.

Deficiencies in the quality of care for chronic conditions are recognised as a major public health problem.1 To deal with these deficiencies, healthcare organisations throughout the world are being encouraged or required to take part in quality improvement (QI) programmes.2 Although QI programmes for chronic disease management have been found to be beneficial,3,4,5 their adoption has been much more limited than expected. One possible reason for this phenomenon is that outpatient healthcare organisations must expend considerable resources to implement QI programmes, but they may never financially benefit from them.6,7 To what extent these concerns are valid is uncertain. Given this uncertainty, outpatient organisations would benefit from indepth evaluations of how much these programmes cost to implement and run successfully.8 Such evaluations may also be valuable for policy makers designing and implementing financial incentives to encourage QI activities.2,9

Currently, little formal guidance exists for small organisations to empirically evaluate costs/revenues associated with QI programmes. There are several well‐established methods for evaluating the societal value of healthcare programmes (table 11).). Although methods such as cost‐effectiveness analysis represent the gold standard in the medical literature,10 these analyses do not provide the information small organisations need to make informed business decisions. Apart from these methods, the National Institute for Health and Clinical Excellence in the UK provides cost accounting tools for use by local health authorities to inform decisions regarding the adoption of new healthcare technologies.11 These tools are appropriate for regional policy decisions; however, they may still need further adaptation for empirical evaluation of programmes from the perspective of small healthcare organisations.12

Table thumbnail
Table 1 Application of standard health economic evaluation methods to evaluate quality improvement programmes

During a recent evaluation of a QI programme in US federally qualified community health centres, we developed an approach to empirically assessing costs that acknowledges formal health economic methods and accommodates common data limitations.

Methods

Between 2003 and 2005, we evaluated the financial impact of the Health Disparities Collaboratives using case studies13,14,15 of five health centres. The Health Disparities Collaboratives is an ongoing national programme to improve care for chronic conditions in federally qualified community health centres. Health centres are part of the US healthcare safety net and receive federal money to provide primary healthcare to the under/uninsured.16 The Collaboratives team members learn about chronic disease management and rapid QI,17,18,19,20 share best practices at learning sessions, and develop QI programmes specifically tailored to their organisations. We evaluated two types of costs/revenue:

  • direct costs/revenues, such as health professional time, items purchased, grants received;
  • cost/revenue consequences, which included financial effects due to changes in patient utilisation or physicians' behaviour ((figsfigs 1 and 22).
    figure qc19232.f1
    Figure 1 Direct costs/revenues.
    figure qc19232.f2
    Figure 2 Cost/revenue consequences.

Results

Direct costs/revenues

Key participants completed a retrospective survey on resources spent on the QI programme. Under prospective data collection, all individuals in the QI programme would track expenditures in real time. Costs included time spent on QI, and money spent on QI‐related purchases. Revenues included donations and grants received for the QI programme. We also collected data about expenditures on training sessions. For this QI initiative, participating health centres received a subsidy from the US government to cover travel expenses for these sessions.

Next we converted the time spent by personnel on the QI programme into dollars (table 22)) using the opportunity cost concept.21 We used the employee's hourly wage for employees who did not generate clinical revenues through encounters. For medical care providers, we used the revenue lost to the health centre that the provider could have generated by seeing patients. To find the revenue generated from an average medical encounter, we divided the total revenue medical care providers earned from medical visits in a fiscal year by the total number of medical encounters at the health centre. For example, from one of our centres, a licensed practical nurse earned $29 120/year including fringe in the year 2001. Her hourly wage ($14.00) was determined by dividing her salary by the total number of hours of work per year (52 weeks/year, 40 h/week). For a medical care provider, an hourly wage equalled the average clinical revenue generated per hour. For the physician in our example, an average encounter generated $53.92 in 2001. Assuming four visits an hour, this physician could potentially earn $215.68/h on average in 2001. In settings where clinics function in a capitated healthcare environment, one can use the salary and fringe method of assigning opportunity costs described earlier.

Table thumbnail
Table 2 Examples of calculating the costs of personnel time devoted to a quality improvement programme

We used surveys to collect information regarding QI revenues. During our study, healthcare organisations reported that their success in obtaining grants unrelated to the QI programme improved as a result of their participation in the QI programme; however, it was often unclear if the grants would have been received in the absence of the QI programme. For this analysis, we limited direct revenues to those grants/donations that were specifically obtained for the QI programme. When accounting for grants received for QI initiatives, it is critical that any time or usage restrictions on the grant money be taken into account. In the future, incentives such as pay‐for‐performance will also possibly be an important source of revenue generated from QI activities.2,22

Cost/revenue consequences

QI programmes may lead to changes in patient utilisation, test ordering, physician productivity, and insurance payor mix, which can affect a healthcare organisation's financial performance. Ideally, electronic billing data should be used to evaluate these changes. In the absence of electronic data, paper data or time and motion studies can be used as substitutes for some analyses. Regardless of the type of data available for analysis, an important first step in evaluating cost consequences is to determine the patients who most probably will be affected by a QI programme, which may involve multiple chronic conditions. We encourage organisations to begin by accounting for the full breadth of costs and benefits that they perceive are related to their overall QI programme.

Electronic billing data available

When working with electronic billing data, first identify the patients most likely to be affected by the QI programme. For this diabetes programme evaluation, we identified all patients who had a diagnostic code (International Classification of Diseases (ICD)‐9 code) for diabetes. We then extracted all billing information associated with these patients whether or not it was directly related to diabetes. This allowed us to look at all visits associated with these patients before and during the QI initiative.

In most billing systems, there will be a unique identifier for every charge, adjustment or receipt. Each identifier will be associated with a patient number and date, and transaction description. Other important fields include diagnostic and procedure codes, currency amount and who made the transaction. For our study, technical staff at the health centres extracted the data; if this is not possible, vendors of billing systems may be able to assist but at considerable cost. Multiple analyses of trends over time can be carried out with such data. We examined average number of encounters and revenue per patient, diagnostic test ordering and changes in number of patient encounters per provider. Patient–payor mix can also be evaluated, which may be especially relevant to clinics in the USA as QI programmes may attract sicker or uninsured patient populations to the clinic, which may be less profitable.

An important step in analysing costs is to assign the appropriate unit cost to each component of utilisation. Charges in the billing data are often unrelated to the actual cost of services provided. An organisation's cost to charge ratio can be used to convert charges into costs.23 If such a ratio does not exist, the costs can be determined using commonly accepted tariff/price lists (eg, National tariff 2006–7 for the UK) for the country of operation. When such unit costs are not available, we recommend the use of costs from similar facilities or countries.24,25 In our study, we used the Medicare Physician Fee Schedule for our US context.26,27 Once these assumptions are settled, costs can then be summed to evaluate whether total costs for the year per patient changed since the initiation of the QI programme. These figures should be adjusted for inflation23 so that they are comparable across years. We used the first year of the QI programme as baseline, and inflated figures in subsequent years by the healthcare consumer price index. This index may vary according to country.

Electronic billing data are not available

If a healthcare organisation lacks billing data or the ability to extract this information, it may be difficult to find a reliable source to identify trends in cost/revenue consequences. Annual accounting summary reports can provide a global picture of an organisation's finances; however, they cannot be used to determine whether observed trends are attributable to the QI programme. Furthermore, these reports may not be detailed enough to provide information on patient utilisation or physician productivity.

Without detailed electronic billing data, an organisation can still gain some insight into the clinical and cost consequences of a QI programme by analysing random samples of billing data and medical records of targeted QI patients. For instance, one can directly measure markers of clinical productivity such as number of daily patient visits on sampled days during the course of the QI intervention. To measure changes in service utilisation, insurance mix or test ordering, paper billing or medical records for a sample of patients can be analysed. In each case, define a pre‐QI baseline and measure changes relative to the baseline.

Discussion

Standard approaches to assessing the value and costs of healthcare programmes exist, but they are of limited value if healthcare organisations cannot use them in everyday practice to inform managerial decisions. To address this need, we have illustrated an approach to evaluating direct costs/revenues, and cost/revenue consequences that may help such organisations evaluate the financial effect of their QI programmes. These methods should be particularly relevant to small healthcare organisations working in resource‐constrained settings such as the developing world where it is extremely important to manage costs effectively.

For a given organisation, the nature of the cost evaluation will depend in large part on what data are available. Our approach is based on QI programmes implemented during the Health Disparities Collaboratives, which focused on improving management of chronic disease in community health centres across the USA.6 We customised our surveys and methods to the specific needs of the organisations we worked with. Tools and methods that retain the concepts used by our survey and database extraction methods can be developed to evaluate the costs/revenues of QI programmes in other healthcare settings.

There are several limitations to estimating costs using our methods. First, retrospective personnel recollection may be inaccurate. Ideally, data collection should occur in real time. Second, it is difficult to determine if secular trends are responsible for the changes observed in the cost/revenue consequences, or whether they result from the QI programme. This can be partially dealt with by comparing institutional trends with those observed regionally and by comparing costs/revenues associated with targeted patients to the overall patient population. Third, when organisations support multiple, ongoing QI initiatives, it may be difficult to account for resources specific for an individual component of the QI programme. Fourth, this analysis does not measure intangible costs/benefits associated with QI. Participants in this QI initiative anecdotally reported increased employee knowledge and initiative, and the spread of QI techniques to other organisational functions, as well as increases in burnout and the reallocation of resources away from other programmes. These may not be captured with financial data.

Our recommended approach to evaluating costs/revenues is primarily intended for leaders of outpatient healthcare organisations to determine the resources required/spent operating a QI programme. It may also provide policy makers important information for determining the amount of incentive needed to encourage QI participation among small organisations. In addition, these cost evaluations generate useful data inputs for traditional cost‐effectiveness analyses. The incorporation of empirically collected cost data in cost‐effectiveness analyses may help policy makers compare the relative cost effectiveness of different QI programmes and identify those that should be widely disseminated in the future.

Abbreviations

QI - quality improvement

Footnotes

This study was supported by an Agency for Healthcare Research and Quality (AHRQ) R01 (MC, R01‐HS010479), an AHRQ U01 (SB, MC and EH, U01‐HS013635), a National Institute on Aging (NIA) Career Development Award (EH, K23‐AG021963), a NIDDK Diabetes Research and Training Center (SB, MC and EH, P60 DK20595), the Chicago Center of Excellence in Health Promotion Economics (MC and EH, P30‐CD000147), a Robert Wood Johnson Generalist Physician Faculty Scholar Award (MC), and a Midcareer Investigator Award in Patient‐Oriented Research from the National Institute of Diabetes and Digestive and Kidney Diseases (MC, K24 DK071933‐01).

Competing interests: None declared.

References

1. McGlynn E A, Asch S M, Adams J. et al The quality of health care delivered to adults in the United States. N Engl J Med 2003. 3482635–2645.2645 [PubMed]
2. Roland M. Linking physicians' pay to the quality of care—a major experiment in the United Kingdom. N Engl J Med 2004. 3511448–1454.1454 [PubMed]
3. Chodosh J, Morton S C, Mojica W. et al Meta‐analysis: chronic disease self‐management programs for older adults. Ann Intern Med 2005. 143427–438.438 [PubMed]
4. Chin M H, Cook S, Drum M L. et al Improving diabetes care in midwest community health centers with the health disparities collaborative. Diabet Care 2004. 272–8.8 [PubMed]
5. Chin M H. Improving diabetes care with quality improvement collaboratives: sustaining gains over 4 years. Med Care (in press)
6. Huang E S, Brown S E S, Zhang J X. et al The cost consequences of improving diabetes care: the community health center experience. Jt Comm J Qual Patient Saf (in press) [PMC free article] [PubMed]
7. Huang E S, Zhang Q, Brown S E S. et al The cost‐effectiveness of improving diabetes care in US federally‐qualified community health centers. Health Serv Res. Published Online First: 16 May 2007. doi: 10.1111/j.1475–6773.2007.00734.x [PMC free article] [PubMed]
8. Leatherman S B D, Iles D, Lewin L S. et al The business case for quality: case studies and an analysis. Health Aff 2003. 2217–30.30 [PubMed]
9. Casalino L P. Markets and medicine: barriers to creating a “business case for quality”. Perspect Biol Med 2003. 4638–51 discussion 5234.51 discussion 5234 [PubMed]
10. Weinstein M C, Siegel J E, Gold M R, for the Panel on Cost‐Effectiveness in Health and Medicine et al Recommendations of the Panel on Cost‐Effectiveness in Health and Medicine. JAMA 1996. 2761253–1258.1258 [PubMed]
11. National Institute for Health and Clinical Excellence Costing tools. http://www.nice.org.uk/page.aspx?o = costingtools (accessed 9 March 2007)
12. Kilpatrick K E, Lohr K N, Leatherman S. et al The insufficiency of evidence to establish the business case for quality. Int J Qual Health Care 2005. 17347–355.355 [PubMed]
13. Yin R K. Enhancing the quality of case studies in health services research. Health Serv Res 1999. 34(5 Pt 2)1209–1224.1224 [PMC free article] [PubMed]
14. Ragin C C. Using qualitative comparative analysis to study causal complexity. Health Serv Res 1999. 34(5 Pt 2)1225–1239.1239 [PMC free article] [PubMed]
15. Ragin C C. The distinctiveness of case‐oriented research. Health Serv Res 1999. 34(5 Pt 2)1137–1151.1151 [PMC free article] [PubMed]
16. McAlearney J S. The financial performance of community health centers, 1996–1999. Health Aff (Millwood) 2002. 21219–225.225 [PubMed]
17. Bodenheimer T, Wagner E H, Grumbach K. Improving primary care for patients with chronic illness. JAMA 2002. 2881775–1779.1779 [PubMed]
18. Bodenheimer T, Wagner E H, Grumbach K. Improving primary care for patients with chronic illness: the chronic care model, part 2 [see comment]. JAMA 2002. 2881909–1914.1914 [PubMed]
19. Wagner E H, Glasgow R E, Davis C. et al Quality improvement in chronic illness care: a collaborative approach. J Qual Improv 2001. 2763–80.80 [PubMed]
20. Von Korff M, Gruman J, Schaefer J. et al Collaborative management of chronic illness. Ann Intern Med 1997. 1271097–1102.1102 [PubMed]
21. Perloff J M. Microeconomics, 3rd edn. Reading, MA: Pearson Addison Wesley 2004
22. Rosenthal M B, Frank R G, Li Z. et al Early experience with pay‐for‐performance: from concept to practice. JAMA 2005. 2941788–1793.1793 [PubMed]
23. Drummond M, O'Brien B, Stoddart G L. et alMethods for the economic evaluation of health care programmes. Oxford: Oxford University Press, 1987
24. Glick H A, Polsky D A, Schulman K A. Trial‐based economic evaluations: an overview of design and analysis. In: Drummond M, McGuire A, eds. Economic evaluation in health care. Oxford: Oxford University Press, 2001. 113–140.140
25. Schulman K, Burke J, Drummond M. et al Resource costing for multinational neurologic clinical trials: methods and results. Health Econ 1998. 7629–638.638 [PubMed]
26. Hsiao W C. The resource‐based relative value scale: an option for physician payment. Inquiry 1987. 24360–361.361 [PubMed]
27. Hsiao W C, Dunn D L, Verrilli D K. Assessing the implementation of physician‐payment reform. N Engl J Med 1993. 328928–933.933 [PubMed]

Articles from Quality & Safety in Health Care are provided here courtesy of BMJ Publishing Group