|Home | About | Journals | Submit | Contact Us | Français|
To determine the cost-effectiveness of three alternative high-quality treatments for attention deficit hyperactivity disorder (ADHD) relative to community care (CC) and to determine whether cost-effectiveness varies with the presence of comorbid disorders.
The study included 579 children ages 7–9.9 with diagnosed ADHD at six sites. Data for the study were distilled from administrative data and from interviews with parents, including estimates of the child's functional impairment. These analyses focus on changes in functional impairment over 14 months.
The study involved a large clinical trial that randomized participants to one of four arms: routine CC, intensive medication management (MedMgt), multicomponent behavioral treatment, and a combination of behavioral treatment and medication.
We assessed the cost-effectiveness of the alternatives using costs measured from a payer perspective. The preferred cost-effective treatment varies as a function of the child's comorbidity and of the policy maker's willingness to pay. For pure (no comorbidity) ADHD, high-quality MedMgt appears likely to be cost-effective at all levels of willingness to pay. In contrast, for some comorbid conditions, willingness to pay is critical: the policy maker with low willingness to pay likely will judge MedMgt most cost-effective. On the other hand, a policy maker willing to pay more now in expectation of future costs savings (involving, for example, juvenile justice), will recognize that the most cost-effective choice for comorbid conditions likely involves behavior therapy, with or without medication.
Analyses of costs and effectiveness of treatment for ADHD must consider the role of comorbidities.
Attention problems are among the most common of mental/emotional/behavioral disorders among children and youth (U.S. Department of Health and Human Services 1999). Youth with attention deficit hyperactivity disorder (ADHD) are more likely to be involved in a variety of costly behaviors including substance use and abuse (Daley 2004) and delinquency (Richardson 2002) as well as to receive costly services. These children also are likely to receive additional school services, including (but not limited to) special education (Jones, Foster, and Gottschall 2005). Jones et al. (2005) estimate that the costs of such services can exceed $10,000 per year. These costs represent only some of the costs that appear on public budgets. These behaviors generate even larger costs to society, such as the costs to victims of violence or the broader costs of school failure (Haveman and Wolfe 1984). These figures suggest that a policy maker's willingness to pay for effective treatments should be quite high. As discussed below, this willingness to pay is a key determinant of a program's cost-effectiveness.
Fortunately, effective treatments exist, notably behavioral treatment and psycho-stimulants. A key issue in the field involves the degree to which each of these treatments is more (or less) effective and cost-effective when combined with the other (Jensen 2000).
Assessment of treatment effectiveness is complicated by comorbidities, including conduct disorder and anxiety. These conditions amplify the link between ADHD and service use (Vostanis et al. 2003) and raise the risk for poor long-term outcomes, such as violence and substance abuse (Arseneault et al. 2000).
In contrast to efficacy and effectiveness, little is known about the cost-effectiveness of alternative treatments, either separately or in combination. To address this issue, we use data from one of the largest and most influential studies in the field, the Multimodal Treatment Study of Children with ADHD (MTA) (The MTA Cooperative Group 1999). The MTA was designed to compare the effectiveness of psychosocial (behavioral) treatment versus medication, and determine whether the two modalities are more effective when delivered together.
An earlier cost-effectiveness analysis of MTA data focused on core attention and related symptoms in the whole sample (Jensen et al. 2005). The analyses presented here extend that earlier work in three ways. First, we examine a broader measure of functional impairment, the Columbia Impairment Scale (CIS). Second, we examine how cost-effectiveness varies with comorbidities. Third, we use new methodology for examining cost-effectiveness of treatments, the cost-effectiveness acceptability curve (CEAC) (Hoch, Briggs, and Willan 2002).
Two areas of prior work inform this research: that on the cost-effectiveness of treatment for ADHD and earlier research on MTA.
In addition to the Jensen et al. analyses (discussed below), economic evaluations of ADHD treatment have been published for Canada and the United Kingdom (UK). Miller et al. (1998) compared six treatments: methylphenidate, dextroamphetamine, pemoline (high-dose and low-dose), nondrug therapy, combined therapy, and no treatment. The study considers costs and outcomes over a 1-year period; the study's perspective is that of third-party payers. On the basis of prior research, treatment effects were determined using a measure of symptoms, the Conners Teacher Rating Scale (CTRS). The methylphenidate strategy was more effective and cheaper than other medication therapies and behavioral therapy as well as combination therapy. Relative to no treatment, methylphenidate was associated with incremental costs of 498 CAN-$ per 6-point (or 1 SD) improvement of the CTRS score.
In the United Kingdom, Gilmore and Milne (2001) examined the cost-effectiveness of different medications from the perspective of the UK National Health Service (NHS). Methylphenidate was cost-effective in children with hyperkinetic disorder according to ICD-10 criteria. This study considered neither behavioral nor combined treatment strategies, however (Gilmore and Milne 2001). The UK National Institute of Clinical Excellence (NICE) estimated the cost per quality-adjusted life year (QALY) gained by methylphenidate at £9,200 to £14,600 (Lord and Paisley 2000). In a further study from the perspective of the UK NHS, a once-daily modified-release preparation of methylphenidate showed extended dominance over immediate-release methylphenidate (both combined with behavioral treatment) over a wide range of assumptions (Schlander 2004). To date, all UK analyses have used a 1-year time horizon.
The MTA study has been the subject of substantial research and some debate. Five hundred and seventy-nine children ages 7–9.9 were randomized to one of four treatment arms: assessment and referral to community care (CC); intensive, systematic medication management (MedMgt); multicomponent behavioral treatment (Beh); and a combination of behavioral treatment and medications (Comb). The MTA-administered interventions were high quality. In the case of MedMgt, dosage was carefully titrated, and participants met with their physician monthly, who consulted the child's teacher regarding classroom performance. Beh was multicomponent and targeted multiple aspects of the child's life. (For details, see Wells et al. 2000.) These components included parent training, a two-part school intervention component, and a child treatment component anchored in an intensive summer treatment program. Each treatment continued for 14 months, although Beh was faded to monthly parent group sessions over the last 3–5 months.
Results indicated that for ADHD symptoms, children receiving Comb or MedMgt showed significantly greater improvement than those given CC or Beh alone. Much debate focused on the question of whether the study really implied that medication (only) was the ideal treatment. Although preliminary reports in the media seemed to support this claim, the full portfolio of findings was more nuanced. The most dramatic findings focused on symptoms, but analyses of broader outcomes (such as social skills, parent–child relationship, and academic performance) revealed that the combination therapy was modestly superior to medication alone (Swanson et al. 2001).
The MTA study collected a comprehensive battery of assessments, and our analyses focused on three key measures: the CIS (a comprehensive measure of child functioning) (Bird, Shaffer, Fisher, Gould, Stagnezza et al. 1993); the Services for Children and Adolescents, Parent Interview (SCAPI, a measure of service use Hoagwood et al. 2004; P. Jensen, Hoagwood et al. 2004) and the Diagnostic Interview Schedule for Children (Shaffer et al. 1993), used to assess the presence of DSM-IV diagnoses, including ADHD, conduct disorder, depression, and anxiety disorders.
The CIS is a 13-item questionnaire completed by parents and measures impairments in the preceding 6 months across four areas: interpersonal relations, broad psychopathology domains (for example, depression, anxiety, or behavior problems), schoolwork, and use of leisure time (Bird et al. 1993). The scale has good internal consistency and construct validity and has been used in multiethnic samples. The analyses below focus on the change in impairment between baseline and the 14-month treatment endpoint.
Use of mental health services was assessed using the revised SCAPI. This measure was administered at 3-month intervals during treatment and asked the families to report on the use of any medical and school services, community mental health services, or juvenile justice services. This measure provided information on the specific types and amounts of services used, medication costs, and costs associated with primary care versus specialty mental health services. Reliability and validity of this measure are excellent (Hoagwood et al. 2004; Jensen et al. 2004).
Following procedures outlined in Jensen, Hinshaw et al. (2001), we defined four comorbidity subgroups: ADHD-only (32 percent), ADHD+internalizing comorbidities only (anxiety or depression) (14 percent), ADHD+externalizing comorbidities only (conduct or oppositional defiant disorder) (30 percent), and ADHD+both comorbidities (25 percent).
As in the Miller et al. (1998) study, all costs were included regardless of whether they were paid for by a patient, an insurer, or any other third party. Although not accounting for the discounts offered and negotiated fees obtained by various parties, this approach approximates a payer perspective (Jensen et al. 2005). The direct costs associated with providing each of the treatment arms were calculated by using the real treatment costs of the MTA Study, while excluding costs associated with the research component of the study, such as research staff time and costs associated with administering the study instruments. All the costs were assumed equal across sites and were adjusted for inflation to year 2000 dollars by using the consumer price index.
Costs of the medications were calculated by using the National Data Drug File Plus (http://www.firstdatabank.com/knowledge_bases/nddf_plus/). It provides average wholesale prices of all drugs from various pharmaceutical companies and in varying pill sizes. The following prices were used for each medication: Ritalin, Novartis, 10-mg pill, 0.059 ¢/mg; Dexedrine, Smith-Kline Beecham, 10-mg pill, 0.092 ¢/mg; imipramine, Allscripts, 50-mg pill, 0.013 ¢/mg; Cylert, Abbott Pharmaceutical, 37.5-mg pill, 0.046 ¢/mg; and bupropion, Geneva, 100-mg pill, 0.009 ¢/mg.
The costs of the psychiatrist, psychologist, pediatrician, teacher, and aide times were calculated using the hourly wage for these professionals based on their respective annual salaries. The adjusted yearly salary of the psychiatrists with the consumer price index was determined to be $142,919.80, based on data from the American Medical Association's Socioeconomic Monitoring System survey (Center for Health Policy Research 1999). Psychologist salaries were calculated as $80,523.24 (Scheffler, Ivey, and Garrett 1998). Teacher salaries (for those who worked in the summer treatment program) and teachers' aides' salaries were determined using data from the Bureau of Labor Statistics. Appropriate levels of fringe benefits were added to the salaries to capture total compensation. (For teacher's aides, a rate of 23.5 percent was used; for psychologists and psychiatrists, we used a rate of 27.4 percent.) Finally, the cost of the summer treatment program was calculated using hourly wages of program staff.
Traditional methods of cost-effectiveness analysis revolve around incremental cost-effectiveness ratios (ICERs) (Drummond et al. 1997). Until recently, ICERs were often provided with no sense of the precision of the estimates. Addressing this problem, however, turned out to be easier said than done, largely because of the irregular statistical properties of ratios. A variety of solutions have been proposed involving the delta method, Feiler's method, and bootstrapping (Briggs, Wonderling, and Mooney 1997).
Even if a satisfactory confidence interval of sorts could be developed, conceptual problems remain. In particular, negative regions of the confidence interval are difficult to interpret. A negative ICER may mean that a new treatment is more costly and less effective than the old or that it is less costly and more effective. Clearly, the implications for the decision maker are different (indeed, completely opposite).
For that reason, two alternatives have been proposed: net benefits (NB) and the closely related net health benefits. Although both involve manipulating the cost-effectiveness ratio, we use the former here (Lothgren and Zethraues 2000; Sendi and Briggs 2001; Hoch et al. 2002).
The key question in cost-effectiveness is whether the ICER of a new treatment relative to an existing one exceeds a policy maker's or society's willingness to pay (λc) for the outcome of interest. This outcome is measured in nonmonetary terms. If the ICER is less than λc, then the new technology or treatment is desirable—i.e.,
Because of sampling error, this statement is probabilistic, and thus confidence intervals for the ICER are needed. As mentioned above, the bounds of this interval can be difficult to calculate and/or interpret. For that reason, one can manipulate (1) into the following:
(2) represents “NB,” and a positive value of NB clearly indicates that the new treatment is preferred. A key feature of equation (2) is that it no longer involves a ratio and that one can calculate the P(NB>0) at alternative values of λ, and these values can be plotted, generating the so-called CEAC. The CEAC provides the policy maker with the information he or she needs—what is the probability the new treatment is cost-effective at alternative values of the outcome of interest (or levels of willingness to pay) (Claxton 1999). All analyses were conducted with STATA version 8 (StataCorp. 2004).
Tables 1 and and22 describe the sample's demographics as well as key variables across the four treatment arms. The latter provides the mean for costs and functional impairment as well as the 95 percent confidence intervals. Three features of Table 2 stand out. First, one can see that the average individual improved, regardless of comorbidity conditions or treatment arm. (Improvement corresponds to negative values of the CIS variable.) The CIS scores have been standardized so comparisons across groups or treatments correspond to effect sizes. Second, for a given comorbidity condition, costs vary across the four treatment arms in every instance and outcomes vary across arms for two of the four comorbidity conditions. The two exceptions involve functioning for the ADHD (only) subgroup and for the conduct-disordered subgroup (p =.40 and .16, respectively). Third, within a treatment arm, outcomes and costs varied a good deal less across comorbidity conditions. Supplemental analyses reveal that the variation in costs was not significant for any of the four arms. To some extent, functional impairment did vary. For three arms, variation across subgroups is significant at the .05 level.
How large are these effects in terms of changes in behaviors that parents, providers, and policy makers value? To assess their practical magnitude, one can look at specific items in the scale. For example, for the comorbid conduct disordered, moving from CC to combination therapy reduces the likelihood that “getting into trouble” is a “bad problem” from 19 to 7 percent. For those with comorbid anxiety, moving from CC to the other arms reduces the likelihood that “behavior at school” is a “bad problem” from 50 to 10 percent.
Figure 1 provides the CEAC for the entire sample. The outcome in this case is a continuous variable—the WTP figures (λ) correspond to a 1 SD improvement in functioning. For each value of λ (on the horizontal axis), the net health benefits were calculated for each individual. This tabulation involved a simple function of the improvement they experienced multiplied by λ, and from this figure, costs were subtracted. Then using bootstrapping (with 500 replications) the probability that a given treatment had the highest NB was calculated for each arm. This probability is plotted on the vertical axis. At modest levels of willingness to pay, MedMgt is nearly certain to be cost-effective. At somewhat higher levels of willingness to pay, Comb therapy becomes cost-effective. Beh therapy is dominated—other treatments are more effective and less costly (Drummond and Mcguire 2001).
Figure 2, however, provides CEAC for the four subgroups. Although MedMgt appears cost-effective at all levels of willingness to pay for ADHD-only children, the picture changes for ADHD-Anx children: Beh may be cost-effective choice at higher levels of willingness to pay. Likewise, for youth with ADHD and comorbid CD/ODD-only, MedMgt may be cost-effective choice at low levels of willingness to pay, but Comb may be the more likely cost-effective choice at higher levels of willingness to pay. And for ADHD+CD/Anx youth, MedMgt is the only treatment with a reasonable probability of being cost-effective but that occurs only at very low levels of willingness to pay.
Figure 3 shows the precision of the statistical uncertainty surrounding the choice of treatment for children and youth in the different subgroups. In some instances, the resulting estimate is rather imprecise, reflecting substantial uncertainty. However, note that the estimates of net health benefits are still large in practical terms. To highlight this fact, we present box and whisker plots for net health benefits for each treatment and combordity subgroups at one level of willingness to pay, $50,000. Nearly all of the distributions are positive (i.e., to the right of the broken line at 0). For those with both comorbidities, for example, one can see that the three high-quality treatments all average roughly $75,000 in net health benefits. Those estimates are very imprecise, however, generating the statistical uncertainty depicted in Figure 2. Particularly revealing is the fact that the potential net health benefits are greatest for the group for which the statistical uncertainty is also greatest.
These analyses generate several important insights into the choices that a policy maker may face when deciding how to use scarce health care dollars to improve health outcomes for children with ADHD. First, our analyses reveal that the cost-effective treatment varies as a function of the child's comorbidity and of the policy makers' willingness to pay. For “pure” (no comorbidity) ADHD high-quality medication care (MedMgt) appears certain to be cost-effective at all levels of willingness to pay. In contrast, for comorbid conditions, willingness to pay is critical: the policy maker who is willing to pay only a little likely will judge MedMgt cost-effective. On the other hand, a policy maker willing to pay more now to avert future costs (involving, for example, juvenile justice) may recognize that the most cost-effective choice likely involves behavior therapy, either with or without medications (depending on the comorbidity—cf. Figure 2).
These analyses also indicate that in instances of comorbid ADHD (with both anxiety-depression and conduct/oppositional disorders), considerable uncertainty faces the policy maker seeking to make a cost-effective choice. For these complex cases, the likelihood of making a cost-effective choice falls dramatically: a policy maker hoping to pay (or avert) $30,000 or more has at best a 50–50 chance of making a cost-effective choice when opting for Comb interventions. Alternatively, for the policy maker willing to pay (hoping to eventually avert) $10,000 for high-quality MedMgt, the likelihood of making a cost-effective choice is high (above 80 percent), but this falls dramatically if the policy maker is hoping for cost-savings greater than that.
Similar to the findings of the original report by Jensen et al., these findings suggest that under many (but not all) circumstances, high quality MedMgt offers the policy maker reasonable certainty of making a cost-effective choice, but in instances of specific comorbidities and at higher levels of willingness to pay, intensive behavior therapy (for ADHD/Anx children) and combined treatments (for ADHD with CD/ODD) may be cost-effective choices. Our findings demonstrate that high costs alone do not rule out cost-effectiveness; however, such treatments may be more likely to be cost-effective when carefully targeted. Future studies must address whether these findings apply to longer-term follow-up periods, other types of outcomes, or other patient subgroups or risk profiles.
These results suggest that the “either medication or therapy” choice that often roils mental health policy is a false one. Optimal treatment may involve both types of treatment, and this conclusion is emerging for several disorders (e.g., the treatment of depression in adolescents [March et al. 2004]). The MTA Study grew a great deal of attention from researchers and policy makers regarding the superiority of MedMgt to behavior therapy alone (Pelham 1999). As earlier research showed, however, the apparent advantage of MedMgt was greatest for symptoms. Regarding functioning, this research shows that behavior therapy plays a key role for two of the three groups for which definitive recommendations can be made.
Obtaining high-quality combination treatment in the real world, however, may be easier said than done. Medicaid and SCHIP are major sources of funding for children's mental health services, and the coverage they provide could in theory finance combination therapy like that offered under MTA (Howell 2004). As evinced by the experiences of those in the CC arm, however, the care actually received may fall far short of those standards (Semansky and Koyanagi 2003). Other studies of larger community samples confirm that only a minority of children receive multimodal therapy (dosReis et al. 2004).
A variety of barriers hinder receipt of high-quality services, including stigma and low reimbursement rates, as they do the receipt of any mental health services (U.S. Surgeon General 1999). Additional barriers, however, are specific to children's mental health, including limitations in the number of relevant professionals. (For example, there are only 5,000 child psychiatrists practicing in the entire United States [Manderscheid and Henderson 2001].)
These problems may worsen as states struggle with the high costs of Medicaid coverage, especially prescription drugs. (One common limitation, limiting coverage to generics, is likely to affect care less for attention deficit disorder than for other disorders for which front-line medications are not available in generic form [Koyanagi, Forquer, and Alfano 2005].)
Another key policy implication involves the redistribution of resources among children and youth with attention problems. One implication of these analyses is that MedMgt alone may be cost-effective for cases uncomplicated by other comorbidities. Providing these cases with high-quality MedMgt may free resources needed to provide more costly treatment to more complex cases. It is striking that for no subgroup was behavioral treatment alone cost-effective. If upheld by longer-term results, these findings suggest that individuals should be provided with either more or less intensive (and costly) treatment, depending on the nature of their comorbid conditions but that such treatments should as a rule include high-quality MedMgt.
Finally, a key to interpreting these policy implications involves the practical sense in which these treatments are cost-effective. Although we have measured the costs of the treatments from a payer perspective, we have considered the benefits of the treatment from a broader perspective, that of a policy maker. Such a policy maker, however, would have to realize that many of the benefits of these model treatments are experienced outside of the mental health sector. Many of the benefits, for example, would accrue to the criminal or juvenile justice systems or even outside of government altogether (e.g., the costs to victims). Such savings are indeed the justification for many public programs. However, it often takes a far-sighted policy maker to recognize that expenditures on better mental health services now may be realized outside of that agency and only over time. As policy makers consider implementing evidence-based treatments in real-world settings, this issue will recur—better treatments may not be less costly for the mental health agency but may produce substantial benefits for society over time.
The study has several limitations. First, the CC arm may not generalize beyond the communities included as study sites, which typically were in and around academic medical centers. Given that context, it is striking that “treatment as usual” fared so poorly given the resources likely available in those communities. Second, the measure of treatment costs is limited to the use of mental health services. However, services use in other sectors (such as child welfare) might potentially influence a policy maker's willingness to pay and could be included in a benefit–cost analysis. This area is an important one for future research.
The MTA is a cooperative treatment study performed by six independent research teams in collaboration with the staff of the Division of Clinical and Treatment Research of the National Institute of Mental Health (NIMH), Rockville, MD, and the Office of Special Education Programs (OSEP) of the U.S. Department of Education (DOE). The NIMH Principal Collaborators were Peter S. Jensen, M.D., L. Eugene Arnold, M.Ed., M.D., John E. Richters, Ph.D., Joanne B. Severe, M.S., Donald Vereen, M.D., and Benedetto Vitiello, M.D. Principal Investigators and Co-investigators from the six sites are as follows: University of California at Berkeley/San Francisco (UO1 MH50461): Stephen P. Hinshaw, Ph.D., Glen R. Elliott, M.D., Ph.D.; Duke University (UO1 MH50447): C. Keith Conners, Ph.D., Karen C. Wells, Ph.D., John S. March, M.D., M.P.H.; University of California at Irvine/Los Angeles (UO1 MH50440): James M. Swanson, Ph.D., Dennis P. Cantwell, M.D., Timothy Wigal, Ph.D.; Long Island Jewish Medical Center/Montreal Children's Hospital (UO1 MH50453): Howard B. Abikoff, Ph.D., Lily Hechtman, M.D.; New York State Psychiatric Institute/Columbia University/Mount Sinai Medical Center (UO1 MH50454): Laurence L. Greenhill, M.D., Jeffrey H. Newcorn, M.D.; University of Pittsburgh (UO1 MH50467): William E. Pelham, Ph.D., Betsy Hoza, Ph.D., Helena C. Kraemer, Ph.D. (Stanford University) is statistical and design consultant. The OSEP/DOE Principal Collaborator is Ellen Schiller, Ph.D.
All data were collected under the approval of the Institutional Review Boards for the Institutions involved.