The study by Azimi and Welch in this issue 1 raises several important questions about the interpretation of cost-effectiveness analyses. Perhaps the most important question is, “What constitutes a cost-effective medical therapy?” But related questions include how to detect systematic bias in the interpretation of cost-effectiveness analyses and how to interpret such analyses when there is no consensus about what to pay for medical therapy.
Cost-effectiveness analyses evaluate a health intervention by asking, “How much health benefit do we get for our money?” The purpose of such analyses is to help decision makers allocate health care resources efficiently.2 These analyses express costs in dollars and health benefits in units of health, such as a life saved, a case of cancer averted, or a quality-adjusted life-year (QALY) gained. They calculate a cost-effectiveness ratio: the ratio of dollars expended to health outcome obtained. The analyses compare one intervention with another, so they calculate cost-effectiveness as the difference in cost between two interventions, divided by the difference in health benefit obtained. For example, in a study that assessed the cost-effectiveness of testing cholesterol levels in children, compared with not testing, the cost-effectiveness ratio would be
In this example, if the cost-effectiveness ratio were $35,000 per life-year gained, one would expect to gain one more life-year for each additional $35,000 that was spent testing children for hypercholesterolemia. The cost-effectiveness ratio is most useful when expressed as dollars per life-year gained (with appropriate adjustments for quality of life, usually expressed as QALYs) because this metric enables one to compare the efficiency of health interventions for different conditions using the same units, much as one might compare different automobiles by the metric of miles per gallon.
But suppose we know that an intervention costs $35,000 per life-year gained; should we adopt it? To decide whether to perform an intervention, we must choose a cost-effectiveness threshold: the amount of money that we are willing to spend to gain one year of life. Is the appropriate threshold $25,000, $50,000, or $100,000? Azimi and Welch assessed how authors of cost-effectiveness analyses answered that question. They found that, for thresholds of less than $61,500 per life-year gained, the authors either favored implementing the intervention or drew no firm conclusion. For thresholds between $61,500 and $166,000, the authors disagreed about cost-effectiveness. For thresholds of more than $166,000, the authors concluded that the intervention should not be performed. What do these results mean? Why is there disagreement among authors about the appropriate cost-effectiveness threshold?
The choice of a cost-effectiveness threshold is a value judgment that depends on several factors.3 First, the choice depends on who the decision maker is, and what the purpose of the cost-effectiveness analysis is. Such analyses are performed from a particular perspective— that of patients, of the government, of society, or of payers. The perspective is important because it determines whose costs and whose benefits the authors include in the analysis. It is also important because these different entities may have different cost-effectiveness thresholds, and may use the analyses for different purposes. For example, if a consumer is trying to determine whether to buy a more expensive but more effective drug, the cost-effectiveness threshold will depend on that person's willingness to pay for improved quality or length of life. An insurer might choose a threshold based on market demand: a potential subscriber choosing among plans that have different thresholds (as implied by the services covered by the plan) would tend to purchase the one that matched his or her personal threshold. Thus, although a given decision maker should use the same cost-effectiveness threshold consistently, different decision makers may not choose the same threshold.
If the decision maker is the government, then the cost-effectiveness threshold will be set, in theory, by societal consensus. No such consensus exists currently,4––7 and the way to arrive at one is not clear. Furthermore, Garber and Phelps used economic principles to show that although use of a single societal cost-effectiveness threshold would enable efficient allocation of public health-care resources, in a heterogeneous population certain people will receive more health care than they would choose, and other people will receive less.3
The second factor in the choice of a cost-effectiveness threshold is how a decision maker values health outcomes and money, how she is willing to substitute one for the other, and what her attitude is about risk. People vary substantially in the amount of money they will spend to improve health, as evidenced in part by people choosing to buy different types of health insurance at different prices. Based on plausible assumptions about values and risk attitudes, Garber and Phelps showed that a reasonable cost-effectiveness threshold is about twice a person's annual income.3 In addition, a person who is more averse to the risk of serious illness and death will have a higher cost-effectiveness threshold (indicating a greater willingness to spend money to obtain health) than will a person who is less concerned about such risks.3
The third factor is the resources available. Just as a household budget will change as family income changes, a decision maker's cost-effectiveness threshold may change as resources change. On a societal basis, the use of antiretroviral therapy for HIV infection illustrates vividly the dependence of the cost-effectiveness threshold on resources: although antiretroviral therapy may be considered cost-effective in the United States, a cost-effectiveness threshold of $50,000 per life-year gained is completely implausible in the developing world, where per capita health spending may be less than $10 annually. Resources and the cost-effectiveness threshold tend to rise and fall together, all other factors being equal.
In summary, the choice of a cost-effectiveness threshold depends on who is making the decision; what the purpose of the analysis is; how the decision maker values health, money, and risk; and what the available resources are. Thus, the search for a single cost-effectiveness threshold is not likely to be fruitful. Given these considerations, it is not surprising that the authors studied by Azimi and Welch reached disparate conclusions.
As a practical matter, how can we interpret cost-effectiveness analyses? With recognition that different decision makers will not—and should not necessarily—agree with one another, I interpret the results of such analyses as providing general guidance on whether an intervention is reasonably efficient, of questionable efficiency, or inefficient. Most, but not all, decision makers in the United States will conclude that interventions that cost less than $50,000 to $60,000 per QALY gained are reasonably efficient. An example is screening for hypertension, which costs $27,519 per life-year gained in 40-year-old men.3, 8 For interventions that cost $60,000 to approximately $175,000 per QALY, certain decision makers may find the interventions sufficiently efficient; most others will not agree. For example, coronary artery bypass grafting for patients who have single-vessel disease and moderate angina costs $88,087 per life-year gained (in 1993 dollars).3, 9 Few decision makers will conclude that interventions that cost more than $175,000 per QALY are justifiable. Cost-effectiveness analysis alerts us to interventions for which lack of efficiency is an important consideration.
Azimi and Welch also raise the question of whether funding by industry influences authors to support high-cost alternatives or to use a higher cost-effectiveness threshold. In their study, the median threshold of the industry-funded studies was $32,678 per life-year gained (higher than the $9,500 threshold that the authors observed in other studies), a value that in itself would cause little controversy. We cannot determine from the study whether the recommendations of industry-funded analyses were inappropriate or were systematically biased. Nonetheless, as consumers of cost-effectiveness analyses, we should recognize that manufacturers face enormous economic incentives to demonstrate the cost-effectiveness of their products, and that the potential for bias exists regardless of funding source. We should be mindful of the source of funding for a study, and should look for statements that indicate that the authors had full scientific and editorial independence, and that disclose the nature of financial ties with the sponsor. Although scientific and editorial independence do not guarantee an unbiased study, their lack creates many opportunities, often inadvertent, for subtle bias.
The broader question that Azimi and Welch pose is whether cost-effectiveness analyses contain costs. As they acknowledge, however, cost containment is not the purpose of cost-effectiveness analyses. Rather, we should ask whether cost-effectiveness analyses prevent us from wasting money on interventions that provide minimal benefit relative to cost. That question warrants careful study; however, in evaluating the influence of cost-effectiveness analyses, we must realize that spending money wisely does not necessarily mean spending less money.
Cost-effectiveness analysis is a tool to help us understand what we get in return for the money we spend on health care. In a determination of whether to offer an intervention, economic efficiency is only one of many factors that deserve consideration. There may be good reasons to offer an inefficient intervention, and there may be good reasons not to offer an efficient intervention (such as concerns about equity or ethics). Used with an understanding of their limitations, cost-effectiveness analyses can inform decisions about the use of an intervention. We should not confuse the scalpel with the surgeon, however: cost-effectiveness analysis is a tool that cannot substitute for value judgments. We must still decide how much money we are willing to spend to improve our health.—Douglas K. Owens, MD, MSc, VA Palo Alto Health Care System and Departments of Medicine and Health Research and Policy, Stanford University, Palo Alto, Calif.