|Home | About | Journals | Submit | Contact Us | Français|
Quality reports or profiles of health care providers are inevitably based on only a measurable subset of the “outputs” of the organization. Hospitals, for example, are being profiled on their mortality in the cardiac area but not in some other areas where mortality does not seem to be the appropriate measure of quality. If inputs used for outputs included in the profile also affect outputs outside the scope of the profile, it can be taken into account in constructing a profile of the measured outputs. This paper presents a theory for how such a commonality in production should be taken into account in designing a profile for a hospital or other health care provider. We distinguish between “conventional” weights in a quality profile, and “optimal” weights that take into account a commonality in the production process. The basic idea is to increase the weights on discharges for which output is measured that use inputs that are important to other discharges whose outputs are not included in the profile.
Reports on the quality of health care providers and health plans are becoming increasingly common, supported by both public and private efforts to motivate providers to improve the quality of care.1 The efforts are diverse and reach no consensus about how much quality reports influence providers, or even about the mechanisms of any such influence. It seems inevitable, however, given both the high value and high cost of health care, that methods for the collection and dissemination of information will be subject to continued refinement, and the role of quality reporting will grow in importance in health care systems and policy.2
One well-recognized limitation of quality reports is that they are partial. For example, the National Commission on Quality Assurance (NCQA) collects data and disseminates ratings of health plans. It keeps track of 22 measures, including five for diabetes management, but care for cancer is tracked by only two screening rates (NCQA, 2004). Medicare’s recently created “Hospital Compare” program rates hospitals on eight measures for heart attacks, four for heart failure, five for pneumonia, two for surgical infection prevention, but none for other conditions.3 Several states, including Massachusetts, New York, New Jersey and Pennsylvania, focus on public reports on the quality of cardiovascular care (only) at hospitals using one measure (Shahian, Normand, Torchiana, et al., 2001).4
For several reasons, most quality reports are partial, and the most notable reason is cost. While the quality of some services is relatively easy to measure, the quality of others can be measured only at a great expense. For example, measuring survival rates in a hospital is very inexpensive. However, survival is a relevant measure of quality only for some of the hospital’s units (e.g., Cardiac) but not the others (e.g., Pediatric). Other reasons can prevent some outputs of a provider from being included in a quality report, such as nonverifiability of the quality of some services, small sample sizes, or difficulties in making risk-adjusted comparisons.
The partial nature of quality reports presents a problem from two perspectives. The most obvious is that because most activities of an organization are not captured in the profile, the omitted activities may be unaffected by the quality reports. But partial reports may be worse than neutral in their effect on unmeasured activities, as is recognized in a second, “multitasking,” perspective on partial reports: measuring and rewarding quality for some activities may divert resources away from unmeasured and unrewarded activities, in principal, therefore, rendering a partial report possibly worse than no report at all. Unintended consequences of partial quality monitoring have been observed in the health care literature.5
This paper puts forward another more hopeful perspective on the effects of a partial report. Measuring and rewarding one activity may spill over in positive ways to other activities at the same organization. Attention to quality in some dimensions may change the mindset of managers and workers, and lead to a generalized improvement in quality. At a more material level, improving the quality of care in one activity may improve quality elsewhere because of links through the production process. If quality in cardiac care is improved by a computerized drug monitoring system in a hospital, for example, quality for other areas of care may also benefit from fewer medication errors. We will refer to this as “commonality,” where inputs have effects across a range of activities.
A one-time study of commonality across services, however, would face fewer of the above-mentioned problems as reasons for why quality reports are often partial, such as cost of data collection, nonverifiabilty, small sample sizes, or difficulties in making risk-adjusted comparisons. Even if all outputs were not measured for all hospitals, for example, a large sample of hospitals can be used to measure the various elements of the quality of the different services that these hospitals provide, and to measure the degree of correlation among these various elements of quality. The main idea of this paper is to demonstrate how, once commonality across services has been estimated, a regulator can use even partial quality reports to induce a provider to provide the desired level of care not only of those services whose quality is measured, but also of those services whose quality is impossible or too costly to measure on a regular basis.
The central insight of our paper is conveyed by Table 1, meant to represent some aspects of the production process at a health care institution (hospital, health plan, nursing home). A profile is to be constructed, characterizing the quality of care at the institution. Suppose activities at a hospital consist of three types of discharges. Outputs of Discharges One and Two can be observed, measured, and used in the profile, but outputs of Discharge Three cannot. In the context of a hospital, for example, one could think of Discharges One and Two as discharge types related to the treatment of cardiac disease in adults such as acute myocardial infarction (AMI), heart failure, single and multi-vessel Coronary Artery Bypass Graft (CABG) surgeries, as well as Percutaneous Coronary Interventions (PCI) with and without the use of coronary stents. For all of these discharges, in-hospital all-cause mortality rates are considered to be a useful and valid outcome measure. Discharge Three can be thought of as representing medical discharges associated with Skin, Subcutaneous Tissue, and Breast Disorders, for which in-hospital mortality is very low, that mortality would not be a feasible (or even valid) measure of quality.
The rows in the table represent inputs used for each type of discharge. Input A could be, for example, use of the emergency room or laboratory services. An “X” in the columns for Input A indicates that patients in that discharge type make use of the emergency room or laboratory services. Input B is used by patients in Discharge Types One and Three, and Input C is only used by patients in Discharge Two.
Even though we can only use Discharges One and Two in the profile, outputs of all three types of discharges matter. The production process for Discharge One is more like that for Discharge Three than is the process for Discharge Two, in the sense that the same inputs are used for both types of discharge. By creating incentives for quality for Discharge One, we also therefore motivate quality for Discharge Three. When weighting the importance of the measurable Discharges One and Two in the profile, we can take this commonality into account and put more weight in the profile on Discharge One. If outputs for Discharges One and Two are risk-adjusted mortality rates, for example, this logic might imply that rather than simply averaging the scores according to the number of discharges of the two types, we should put more weight in the profile for discharges in Discharge One. How much more weight depends on the production relationship among the inputs and outputs, and on the valuation placed on the outputs of the different types of discharges.
The analysis in this paper is related to Glazer and McGuire (2005 and 2006) who draw the implications of demand and cost conditions for the form of quality reports in health care. Both papers demonstrate that when consumers have full information about product quality, and if the market equilibrium is inefficient, either because of adverse selection incentives or because firms enjoy some market power, then partial rather than full revelation of product quality can improve social and consumer welfare. Furthermore, if the product quality is multidimensional, then revealing only a weighted average of the different dimensions of quality may be the optimal reporting mechanism. In those papers, as in this one, optimal quality reports are derived as a solution to a “mechanism design” problem.
The analysis in our paper is related to the literature on “multitasking” (Holmstrom and Milgrom, 1991; Prendergast, 1999). In a multitasking analysis, a firm is rewarded for performance of one among many tasks. Efficiency objectives can be frustrated if rewarding one task causes a firm to cut back on some other unrewarded task. The multitasking perspective implies that doing nothing (or at least applying a low-powered incentive scheme) might be better than a partial system of rewards (even if it is a high-powered partial reward). The main idea of this paper is to show that commonality in production between the measured and the unmeasured tasks (e.g., they use the same inputs), can allow the multitasking problem to be at least partially overcome. Our problem also has a close analogy to the literature on optimal commodity taxation where the problem is to raise public funds by commodity taxes when at least one good (leisure) cannot be taxed. Higher taxes on commodities complementary to leisure in demand mitigates the problem of the untaxable good (Ramsey, 1927; Diamond and Mirrlees, 1971).
In Section 2 we set out a formal model of profiling that shows how information about the production process can, when some inputs are common across activities, be helpful in motivating quality of unmeasured outputs. We describe in that section the concept of an “optimal” profile and the set of weights on the measured outputs that maximizes the quality objective. We characterize the optimal weights when only a subset of the provider’s outputs are available to be used in the profile. Section 3 considers some next steps in the development of the ideas.
We begin by describing the efficient quality of health care at a hospital. The purpose of the profile is to encourage this pattern of efficient care. The hospital’s outputs are grouped into discharges, and the quality of each discharge type is a function of the quantity of inputs the hospital employs to produce the discharge. More inputs increase quality. By inputs we have in mind factors such as nursing time, laboratory systems, and emergency room staff. The hospital has a budget that limits total spending on all inputs.
Specifically, let i = 1, …, I be the types of discharges at a hospital. Index i could be a diagnosis, but other breakdowns of discharges could also be meaningful. For example, different patients (old, young) might be regarded as belonging to different types of discharges even if they had the same discharge diagnosis. Let ni be the (exogenously given) number of patients in discharge of type i. Let t = 1, …, T be the set of inputs the hospital uses. Let qt denote the quantity of input t and Qi(q1, …, qT) denote the quality of treatment provided to each patient in discharge type i (where qt > 0 for all t). We assume that for all i and t.
Efficiency is characterized in relation to a social objective of maximizing the sum of qualities of the discharges of all types at the hospital, . Other social objectives could also be specified, such as putting more value on treatment of some types of patients than others, but our analysis and results would be essentially unchanged by an alternative formulation of the social objective.
Total spending on inputs must be less than a budget, B. The socially efficient input vector, , is therefore the solution to the following problem:
The first-order conditions yield:
Equations (2) are a series of public-good like conditions that say that the marginal benefits, summing these across all discharge types, of a dollar spent on all inputs should be equalized.
We now describe how a hospital chooses inputs when motivated by a “profile.” To focus on commonality and the incentives created by the form of the profile, we assume in what follows that the hospital seeks to maximize a score on a profile subject to a budget. Financial incentives are in the background in our analysis. With the fixed number of discharges, and a fixed payment per discharge, revenue does not depend on a higher or lower score. Clearly, however, the point of attaining a higher score for a hospital is to attract more patients. The problem we pose and solve here remains relevant because whatever quality score is optimal for the hospital, it will want to attain that score efficiently. An equivalent approach to minimizing cost of attaining a certain score is to study the dual problem, maximizing the score subject to a budget. This is the approach taken here.
The hospital’s profile is given by , where αi, i = 1, …, I are some weights on discharge types. The focus of our analysis is how a regulator should choose these weights. The first step is to describe a hospital’s behavior when these weights are given.
Thus, given some weights α, we assume that the hospital chooses q = (q1, …, qT) so as to:
The first-order conditions of the hospital’s problem in (4) define the vector ( ):
We first consider what we refer to as a “conventional” profile, the typical way weights are chosen. We define a “conventional” profile as one that takes its weights from the social objective function. In our case, the weight given to discharge of type i is ni in a conventional profile. These weights are natural, weighting quality of each discharge equally. In this section, we analyze hospital behavior in the presence of such a conventional profile.
We say that a profile of weights αi, i = 1, …, I, implements the efficient quality if, given αi, i = 1, …, I, the first order conditions (5) and (6) yield for t = 1, …, T. It is easy to see that if αi = ni for i = 1, …, I, the efficient quality is implemented. In other words, if every discharge type can be used in a profile, and is weighted by the number of discharges of that type (as in the social welfare problem), the social objective is identical to the hospital objective and conventional profiling leads to efficient quality of all discharges.
In many cases, however, not all outputs of a hospital can be easily used in a profile. As discussed in the introduction, reasons that prevent all outputs of a hospital from being included in a profile may include: cost of data collection, nonverifiability of quality of some services, small sample sizes, or difficulties in making risk-adjusted comparisons. We will refer to a profile as “partial” if it cannot use all discharge categories in the profile.
To represent a partial profile, suppose quality cannot be observed and used in a profile for a subset of the discharge types. Specifically, suppose that quality information is only available for the first k types of discharges among the I types of discharges produced by the hospital. Consequently, for types i = k + 1, …, I, the αi must equal zero. In this context, a conventional partial profile is αi = ni for i = 1 … k, and αi = 0 for i = k + 1 … I.
We can next ask how a hospital chooses the quantity of inputs t, (for t = 1, …, T) in the presence of this conventional partial profile. Applying conditions from (5) and (6) above, the hospital’s choice, denoted ( ) can be described by the following equations:
The chosen ( ) will not in general equal the efficient vector ( ). The condition for efficiency (see (2) above) equalizes the full marginal benefits of spending on each input to each other input, using a summation across all discharge types. The difference between this and the conditions describing the hospital’s behavior with a partial profile is that the partial profile gives a hospital the incentive to care only about the impact of an input on discharge types 1 … k, and not on its impact on discharge types k + 1 … I.
The conventional partial profile fails to encourage the efficient selection of inputs. First, it creates no incentive for the hospital to devote resources to any input not used in production of discharge types 1 … k. Discharge types 1 … k might represent, for example, cardiac care at a hospital, with survival rates available for use in a profile. Other service areas of the hospital might not be profiled, such as obstetrics, psychiatry, and general medicine. In this case, the conventional partial profile would not reflect the contribution of an input specific to the obstetrical service. Other incentives or monitoring methods would be necessary to reward the hospital for spending on inputs that are not part of the production of cardiac care. No reweighting of discharge types 1 … k will address this problem.
Second, with a conventional partial profile, even inputs that are used in the production of discharge types 1 … k may not be chosen efficiently. The hospital will choose these inputs only so as to equalize the marginal benefits of spending on each input, but summing only over the k discharge types used in the profile. The condition for efficiency, by contrast, requires equalization of the marginal benefits summed over all discharge types. The hospital’s objective diverges from the social objective to the extent that an input involved in the production of discharge type 1 … k is also used in the production of types k + 1 … I. In essence, a conventional partial profile ignores any commonality in production outside of the profiled discharges. Suppose, for example, that the quality of the cardiac operating room does not affect the quality of discharges outside of the cardiac services. However, nurses staffing the general medical beds care for all medical patients including cardiac patients following operations. The quality of nursing staff therefore affects both cardiac and general medical discharges. Incentives in the conventional profile to allocate budget to the operating room in relation to general medical nursing staff will not reflect the full benefit of nursing staff.
This second problem can be addressed by a reweighting of the discharge types used in the profile. In what follows, we show how to design a partial profile that exploits the commonality of inputs.
We seek a profile that leads the hospital to choose inputs that maximize the social value of outputs. This type of problem is referred to in the economics literature as a mechanism design, or a principal-agent, problem and leads, in the case of a partial profile, to a solution that will differ, in general, from the conventional profile. We refer to the solution to the principal-agent problem as the “constrained optimal” profile.
Formally, the principal-agent problem when all αi’s are available for use in a profile can be described as follows:
The set of equations describing the hospital’s choice of inputs appear as a constraint on the principal-agent problem because the regulator cannot dictate choices to the hospital, but must take as given the hospital’s reaction to the incentives created by a profile. These are referred to as the incentive-compatible constraints.
An obvious solution to this principal-agent problem when all discharge types can be used in a profile is the conventional profile αi = ni, for i = 1 … I, leading the hospital to choose , t = 1 … T. Note that this solution for the αi may not be unique. A hospital’s choice of inputs is characterized by the T linear constraints above ((10) plus the budget constraint). I weights (the α’s) can be used in a profile. To satisfy these constraints at , if I > T, discharge types in a profile can be weighted in many ways to implement .
The principal-agent problem can be modified to represent the more relevant case where the regulator can only use a subset (i = 1, …, k) of the discharge types in a profile. This can be captured in the problem above by adding a set of constraints to the maximization, αi = 0, i = k + 1, …, I. Rather than simply rewriting the problem, we make some observations about the nature of the solution.
Introduction of the constraint that some αi must be zero does not necessarily imply that the regulator cannot construct a profile leading to efficient production. A solution with αi = 0 (i = k + 1, …, I), may be in the set of solutions to the earlier maximization without the new constraints. A necessary condition for a profile to achieve the efficient use of all inputs is that every input t that is used in the set of all I discharges must also be used in at least one of the 1 … k discharges available for profiling. Otherwise, is zero for that t, and a profile based on these k discharges can create no incentive for the hospital to devote resources to this input.
Consider the following system of (T − 1) equations:
This system of equation is the incentive compatible constraints where all the qt’s are fixed at the efficient level, while satisfying the constraints on the αi’s, for i = k + 1 … I, and the budget constraint (by definition of ). Hence, we have a system of T − 1 linear equations with k unknowns (where the unknowns are the α1, …, αk). If this system of equations has a solution, then it must be a profile that induces the first best choice of inputs. Whether or not a solution to this system exists is determined by a rank condition on the matrix of coefficients in the system, which comes down to conditions on the relationship between inputs and outputs at the hospital. In particular, if T ≤ k, and the equations are not linearly dependent, then at least one solution exists.6
An “optimal” partial profile, as a solution to the principal agent problem posed above, will generally exist, even if this constrained optimal profile is not able to achieve the first-best allocation of all inputs. In general, the partial profile solving the principal-agent problem will not be the same as the conventional profile. The optimal profile sets weights on the discharge types available for use in the profile to improve social welfare over the conventional profile.
Realistically, profiles of health care organizations must be partial, capturing only a subset of the outputs of the organization. When some of the inputs used to improve quality in the measured activities are also used in the unmeasured activities, weighting the measured activities to recognize commonality can motivate quality in the unmeasured activities as well.
The degree of commonality in hospitals is ultimately an empirical question, and as far as we know only a few papers have addressed this issue. Incentives from some payment systems in hospitals “spill over” onto patients from other payers. See Van Horn, Burns and Wholley (1997), Bernard (2000), and Wu (2005). One interpretation of these findings is that spillovers arise from common features of the production of care in hospitals. Asch et al. (2004) found better quality of medical care in the Department of Veterans Affairs compared to a comparable national sample of hospitals. The differences were largest for those measures actively profiled by the VA but also apparent in areas that were not explicitly profiled. Outside the hospital sector, Glied and Zivin (2002) find payer mix at the physician level affects “intensity” of visits over and above a payer-specific effect, again supporting the role of commonality. In the market for nursing home care, Grabowski, Gruber, and Angelelli (2005) support the existence of common elements of quality between Medicaid and private-pay residents, and Konetzka et al. (2006) find common elements of quality between long-stay and short-stay residents.
Moving the idea of a “constrained optimal” profile to practical application requires more research, primarily about the characteristics of the production process at hospitals or other institutions for which profiles are being constructed. The health services research literature contains considerable evidence of commonality in production at hospitals, nursing homes and health plans. The physical set up of a hospital or other institution is such that sharing of staff across activities defined by type of patient is natural. Furthermore, the underlying organizational rationale for the multi-output firm is economies of scope, a form of the kind of commonality we rely upon here. At the same time, we acknowledge that the existing evidence supporting commonality is too general to be usable yet as a basis for modifying profile construction. A reweighting of discharges at a cardiology unit to take account of quality at the rest of the hospital would need to be based on specific evidence about production commonalities.
Empirical estimation of the required parameters will be very challenging, essentially requiring estimation of the production function with many outputs and many inputs. To compound the challenge, some of the outputs may not be readily measurable - indeed, that is why these may not readily be included in the report. We should note, however, that special effort could be put into measurement in a one-time empirical study of inputs and outputs that might then inform the way a report is constructed. A “consumer price index” reports price levels, for example, based on spending patterns identified in surveys that are not done every year and for all sets of consumers. Analogously, a “constrained optimal report” for nursing homes could be based on a special study of productive relationships in a representative sample of homes.
We purposely titled our paper, “Mitigating the Problem..” rather than “Solving the Problem” of unmeasured outcomes. The analysis above indicates that, in theory, reweighting outputs can fully solve the problem. In practice, mitigating is the right goal. Empirical research that supports mitigating is easier to conceive. More resources for the ER might be found to affect time to cardiac catheterization for heart attack patients and positively affect the intake health status of patients across a range of conditions. This might tell us conclusively that discharges making heavy use of the ER should be overweighted without telling us by precisely how much overweighting there should be. Nonetheless, like many areas of public policy, we have enough to go on to mitigate the efficiency problem.
Care needs to be taken before using evidence on commonality across activities to modify profiles. The conventional profile creates incentives for the hospital to maximize the quality of the outputs within the cardiac unit, in our primary example, and this statement holds for any production process. While, as we have shown, other sets of weights may also satisfy this objective while improving incentives in other areas, this may not be so, and improving the incentives for quality outside the cardiac unit may involve a tradeoff in the motivation for quality within the unit. To put this bluntly, the profile of the cardiac unit set “optimally” so as to value patients equally in all parts of the hospital may call for a reduction in the incentives for quality for some inputs to cardiac care. Before taking such a step, the analyst needs to be on very strong ground. Any weighting deviating from the conventional profile requires reliable information about the technology of production.
Another concern that may arise has to do with providers’ heterogeneity. Not all hospitals use exactly the same production technology and, hence, the degree of commonality across services may be different for different hospitals. Payment systems such as the per discharge payment system used by Medicare and other payers assume a degree of homogeneity in hospital costs and production. A similar approach must underly any use of quality reports that compare outputs across facilities, whether or not they rely on commonality.
The leading assumption in our analysis in this paper is that for each hospital, the quality reported is a single aggregate measure. In principle, however, quality reports could be more detailed and could be defined over a subset of hospital services or over a specific domain (AMI mortality, for example). This obviously raises a whole set of new questions regarding the construction of optimal quality reports, such as: how detailed should quality reports be? Which services, if any, should be grouped together into one report and what weight should be assigned to the different services in each report? Answers to these questions are beyond the scope of this paper, but two important observations emerge: (i) if services are grouped together in a report, the weight assigned to each service in the aggregate report may depend on the degree of input commonality between these services and the services whose quality is not included in the report. (ii) Grouping services together into one report may provide the payer or regulator with an additional tool to induce high quality in those services for which quality is too costly to measure, if commonality across services exists.
Our analysis is also related to the growing phenomena of pay-for-performance mechanisms in health care. Under this type of mechanism, quality is not only measured and reported but also serves as one of the elements in the provider’s reimbursement scheme. Obviously, one of the most important questions that a payer must address when constructing such a mechanism is the “weight” to assign to each service that enters the reimbursement formula. One of the implications of our analysis is that if quality of some services cannot be easily measured and, hence, included in the hospital’s reimbursement formula, the weights assigned to the services that are included in the formula should depend, among other things, on the degree of commonality between the various services in exactly the same manner as discussed in our analysis above.
More broadly, we hope our paper stimulates others to think about policies in health care related to information reporting somewhat differently. Reporting the information is not the objective. The objective is to improve the quality of health care. New approaches for reporting are suggested by setting out that objective explicitly, and recognizing the behavior of actors in the health care system in response to the policies being considered.
*This research was supported by research grants from the National Institute of Mental Health (R34 MH071242), the Agency for Healthcare Research and Quality (P01 HS10803) and the Program in Health Systems Improvement at Harvard University. Paul Cleary, Richard Frank, Joseph New-house, Will White and Alan Zaslavsky provided helpful comments on an earlier draft. The authors thank Ting Liu, Treacy Silverstein, and Jaiyin Sun for able research assistance. Special thanks are due to Don Fullerton and two anonymous referees for excellent comments and suggestions on an earlier draft of this paper.
1Public payers, like Medicare (Clancy, 2006), and private coalitions, like the Leapfrog Group (Galvin and Milstein, 2002), are constructing quality reports intended to reward providers for improving quality. Medicare and the Federal Employees Health Benefit Plan (FEHP), have made quality reports about plans or providers available to beneficiaries for some time, either as experiments or on a regular basis (Wedig and Tai-Seale, 2002).
3For example, one of the measures for heart attacks is the percent of patients given beta blockers on arrival. See http://www.hospitalcompare.hhs.gov.
4A different approach is to ask consumers to give an overall rating of a plan or a provider as is done with the widely used Consumer Assessment of Health Plan (CAHPS) scores. See, for example, Schneider et al. (2001).
5Berlowitz, et al (2001) found a reorganization to deal with some aspects of quality lead to an increase in an unrewarded dimension, “pressure ulcers.” The most well-recognized unintended effect of quality regulation is discouraging health care providers from taking severely ill patients. See Dranove, Kessler, McClellan and Satterthwaite (2003) for discussion and an application.
6The resulting α’s may not all be positive.
Jacob Glazer, Thomas McGuire, and Sharon-Lise T. Normand (2008) “Mitigating the Problem of Unmeasured Outcomes in Quality Reports,” The B.E. Journal of Economic Analysis & Policy: Vol. 8: Iss. 2 (Contributions), Article 7.