PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Pharmacoeconomics. Author manuscript; available in PMC 2013 August 1.
Published in final edited form as:
PMCID: PMC3423960
NIHMSID: NIHMS399399

PREFERENCE-BASED HEALTH-RELATED QUALITY OF LIFE OUTCOMES IN CHILDREN WITH AUTISM SPECTRUM DISORDERS: A COMPARISON OF GENERIC INSTRUMENTS

J. Mick Tilford, Ph.D.,1,2,6 Nalin Payakachat, Ph.D.,2 Erica Kovacs, Ph.D.,3 Jeffrey M. Pyne, MD,4 Werner Brouwer, Ph.D.,5 Todd Nick, Ph.D.,6 Jayne Bellando, Ph.D.,6 and Karen Kuhlthau, Ph.D.7

Abstract

Background

Cost-effectiveness analysis of pharmaceutical and other treatments for children with autism spectrum disorders (ASDs) has the potential to improve access to services by demonstrating the value of treatment to public and private payers, but methods for measuring quality adjusted life years (QALYs) in children are understudied. No cost-effectiveness analyses have been undertaken in this population using the cost per QALY metric.

Objective

This study describes health-related quality of life (HRQoL) outcomes in children with ASDs and compares the sensitivity of two generic preference-based instruments relative to ASD-related conditions and symptoms.

Methods

The study design was cross-sectional with prospectively collected outcome data that was correlated with retrospectively assessed clinical information. Subjects were recruited from two sites of the Autism Treatment Network (ATN): a developmental center in Little Rock, Arkansas, and an outpatient psychiatric clinic at Columbia University Medical Center in New York, NY. Children that met DSM-IV criteria for an ASD by a multi-disciplinary team evaluation were asked to participate in a clinical registry. Families of children with an ASD that agreed to be contacted about participation in future research studies as part of the ATN formed the sampling frame for the study. Families were included if the child with the ASD was between 4 and 17 years of age and the family caregiver spoke English. Eligible families were contacted by mail to see if they would be interested in participating in the study with N=150 completing surveys. HRQoL outcomes were described using the Health Utilities Index Mark III (HUI-3) and the self-administered Quality of Well-Being scale (QWB-SA) obtained by proxy via the family caregiver.

Results

Children were diagnosed as having autistic disorder (76%), pervasive developmental disorder (PDD-NOS) (15%), and Asperger's disorder (9%). Average HUI-3 and QWB-SA scores were 0.68 (SD=0.21, range of 0.07 to 1) and 0.59 (SD=0.16, range of 0.18 to 1) respectively. The HUI-3 score was significantly correlated with clinical variables including adaptive behavior (ρ=0.52; p<0.001) and cognitive functioning (ρ=0.36; p<0.001). The QWB-SA score had weak correlation with adaptive behavior (ρ=0.25; p<0.001) and cognitive functioning (ρ=0.17; p<0.005). Change scores for the HUI3 were larger than the QWB-SA for all clinical measures. Scores for the HUI3 increased 0.21 (95% CI: 0.14–0.29) points across the first to third quartile of the cognitive functioning measure compared to 0.05 (95% CI: −0.01–0.11) for the QWB-SA. Adjusted R2's also were higher for the HUI3 compared to the QWB-SA across all clinical measures.

Conclusions

The HUI-3 was more sensitive to clinical measures used to characterize children with autism compared to the QWB-SA score. The findings provide a benchmark to compare scores obtained by alternative methods and instruments. Researchers should consider incorporating the HUI-3 in clinical trials and other longitudinal research studies to build the evidence base for describing the cost-effectiveness of services provided to this important population.

INTRODUCTION

Autism spectrum disorders (ASDs) are characterized by impairments in social skills, communication, and cognitive and behavioral functioning.[14] Children with ASDs can exhibit severe tantrums, noncompliance, destructiveness, and self-injury.[57] They may require less sleep and have frequent awakenings during the night [811]. Successful pharmaceutical and other interventions for children with ASDs thus have the potential to improve their quality of life outcomes, yet we know little about the relative impact of different ASD-related impairments on general health-related quality of life (HRQoL). In addition, given the varied symptoms that are common among children with ASDs, there is need for evidence on the cost-effectiveness of ASD interventions to assist with prioritizing services. Only one study has examined HRQoL outcomes for children with ASDs across the complete spectrum of disorders and in relation to ASD severity and common behavioral characteristics. Kuhlthau et al. measured HRQoL outcomes for children enrolled in the Autism Treatment Network using the PedsQL questionnaire.[12] Survey responses to the PedsQL were linked with clinical data describing the child's cognitive ability, adaptive functioning, ASD-related symptoms, and behavior problems. Findings from the study showed HRQoL deficits in all domains of health including physical, psychosocial, emotional, social, and school-functioning relative to healthy children. In addition, ASD-related symptoms were associated with decrements in HRQoL.

Evidence of associations between ASD-related symptoms and HRQoL suggests that effective treatments for children with ASDs have the potential to reduce associated symptoms and improve HRQoL for the child. Such associations also permit targeting or development of interventions for specific behavioral characteristics that might produce the greatest gains in HRQoL, although such evidence would require appropriate confirmation. Interventions could be prioritized using cost-effectiveness analysis and other relevant criteria, which would permit a more rational allocation of resources by informing public and private payers of the value of services.[13] Despite the importance of developing treatment protocols for children with ASDs that can optimally reduce ASD-related symptoms and improve HRQoL, no studies have reported on the cost-effectiveness of ASD services using a cost-utility or cost per quality-adjusted life year (QALY) framework.

The lack of information on cost-effectiveness may be related to the limited information available that identifies preference-based HRQoL outcomes in children with ASDs. Preference-based HRQoL outcomes are valued on a 0 to 1 scale where “0' represents death and “1” represents perfect health. Preference-based HRQoL outcomes need to be combined with life years in order to calculate QALYs, which are commonly viewed as the preferred metric for cost-effectiveness analysis.[14] When cost-effectiveness analyses are conducted with QALYs, they, in principle, permit standardized comparisons with other mental and physical conditions as well as conditions affecting different age groups.[15] Different interventions targeted at different impairments associated with a condition (e.g. behavioral problems, sleep issues, and communication issues – all of which are common for children with ASDs) can thus be compared in terms of their efficiency expressed as costs per QALY gained. Measurement of preference-based HRQoL outcomes in the context of child health conditions, however, has typically lagged behind adult conditions.[1618] Measuring preference-based HRQoL in children raises a number of methodological issues, including the need to use proxy respondents such as parents.[19;20] We are aware of only two small case series that report preference-based HRQoL outcomes in children with ASDs.[21;22] Both studies of children with ASDs used the Health Utilities Index Mark 3 (HUI-3) instrument to describe HRQoL outcomes with the child's caregiver as a proxy respondent. Information on the clinical characteristics of the child was not included in these descriptions so it is not possible to relate differences in ASD-specific outcomes to differences in QALY scores.

This paper seeks to explore further the validity of generic preference-based instruments to describe HRQoL in relation to disease-specific health outcomes for children with ASDs. A number of studies have examined the sensitivity of different generic instruments to describe preference-based HRQoL in relation to disease-specific health outcomes in adult conditions including schizophrenia,[23;24] substance use disorders,[25] and other physical [26;27] and mental health conditions.[28] Studies test the sensitivity of different instruments because the choice of instrument has the potential to influence estimated cost-effectiveness ratios.[29] As different instruments have different domain structures, some instruments may be better suited for economic evaluations of a given condition relative to other instruments.[30] While the literature on the “comparative effectiveness” of different instruments to measure QALYs in different adult conditions is large and growing,[30] few studies have compared instruments for conditions affecting children.[31;32]

Thus, the current study has two main objectives: 1) we seek to evaluate the construct validity of two instruments for describing preference-based HRQoL in children with ASDs and 2) identify the magnitude of potential QALY gains from treatment based on relationships between clinical variables and the HRQoL instruments. Information on construct validity is necessary to determine whether some generic instruments may be considered better suited for measuring HRQoL scores in children with ASDs relative to others. For this study, we compare the HUI-3 with the Quality of Well-being Self-Administered (QWB-SA) scale. The HUI-3 has been used in a number of studies involving children [33;34] as well as autism.[21;22] The QWB-SA was used for comparison in this study because it has been shown to work well in mental health conditions in adult populations.[23;35] We provide an indication of the magnitude of changes in relation to clinical parameters using both instruments to inform future cost-effectiveness analyses of ASD interventions.

METHODS

Participants and Study Design

The study used a cross-sectional and prospective design to obtain outcome measures that were correlated with retrospectively captured clinical data. Participants for the study were recruited through two sites of the Autism Treatment Network (ATN) funded by Autism Speaks: a developmental center in Little Rock, Arkansas, and an outpatient psychiatric clinic at Columbia University Medical Center in New York, NY. At these two clinical sites, children suspected of having an ASD completed a multidisciplinary evaluation that included diagnostic, cognitive, behavioral, and physical assessments. Children that meet DSM-IV criteria for an ASD were asked to participate in a clinical registry. The clinical registry contains information on ASD-related symptoms and severity, cognitive functioning, and other clinical information described below. Families of children with an ASD that agreed to be contacted about participation in future research studies as part of the ATN formed the sampling frame for the study. Families were included if the child with the ASD was between 4 and 17 years of age and the family caregiver spoke English. Eligible families were contacted by mail to see if they would be interested in participating in the study.

Eligible families were sent a packet that contained a recruitment letter, consent forms, and instruments to measure preference-based HRQoL outcomes for the child and the caregiver. The information sheet requested that the primary caregiver of the child complete the survey. The survey contained two separate packets that were clearly labeled “caregiver items about caregiver” and “caregiver items about child.” In this study, we consider only the caregiver responses about the child's health. Families were contacted up to three times by mail or phone (follow-up calls) to get the surveys returned. Families that signed HIPPA forms, consent/assent forms, and returned the surveys were provided a $25 gift certificate. Survey instruments were formatted such that data could be scanned into SPSS using Remark Classic OMR® software. REMARK automatically flags fields that include multiple responses or are left blank. Data from the returned surveys was then merged with clinical information from the ATN. The study was approved by the institutional review boards at Columbia University and the University of Arkansas for Medical Sciences.

Instruments

Two generic preference-based HRQoL instruments, the HUI-3,[36] and the QWB-SA,[37;38] were selected for this study. These instruments are widely used in economic evaluations for patients with different conditions although the QWB-SA has been used less frequently in children despite being recommended as a generic instrument for use in cost-effectiveness analysis by the US panel on cost-effectiveness in health and medicine.[39] Studies in children that used the QWB-SA to measure preference weights reached mixed conclusions concerning its sensitivity.[40;41] Tilford et al generated preference scores for children with traumatic brain injuries using the QWB-SA [42] and Smith-Olinde et al compared scores for the HUI-3 and QWB-SA in children with hearing loss.[32] While there was evidence of sensitivity to clinical outcomes in both of these studies, the HUI-3 appeared to be the better choice for studying outcomes in children with hearing loss.

The HUI-3 describes an individual's health in terms of eight attributes: vision, hearing, speech, mobility, dexterity, cognition, emotion, and pain/discomfort with five or six levels per attribute. Caregivers were asked to report on the health states of the child over a 3-day period to be consistent with the QWB-SA described below. A multiplicative scoring function is used to calculate the HUI-3 index; the values range from −0.36 (some health states are considered worse than dead) to 1 (perfect health state). The HUI has strong theoretical and empirical foundations and employs a multi-attribute utility function based on standard gamble weights obtained from a community sample. [36]

The QWB-SA is a self-administered preference-weighted measure combining three scales of functioning (mobility, physical activity, and social activity including completion of role expectation) with a measure of symptoms and problems (58 symptom complexes: CPX) to produce a point-in-time expression of well-being that ranges from 0 (for death) to 1.0 (for asymptomatic full function). The CPX symptom complex includes two conditions (sexuality and hangovers) that are not applicable to children of all ages and thus, are not included in the survey. Caregivers were asked to report their child's health on the four subscales over the 3-day recall period. Preference weights for the QWB-SA health states were derived from a representative sample of the community using categorical rating scales. Because the QWB-SA uses visual analog scales in determining weights, many investigators do not consider it to represent a utility value.[43] We used the QWB-SA in this study because of prior concerns that instruments other than the QWB-SA were insensitive to mental health outcomes in adult populations [44] and deemed it necessary to test it in a mental health condition affecting children.

Clinical Measures

To test the construct validity of the preference-based HRQoL instruments, we assessed their correlations with ASD-specific diagnostic instruments, behavioral measures, symptoms, and measures of cognitive functioning. All of the clinical measures were obtained at the time of the child's first visit to the ATN site. For most of the children, clinical data was obtained within 1 year of the survey data. Approximately 90% of the clinical data was obtained within 2 years of obtaining the survey data.

All children had a clinical diagnosis of autism spectrum disorder meeting DSM-IV-TR criteria (e.g., Autistic Disorder, PDD-NOS, or Asperger's Disorder) and confirmed by scores meeting or exceeding cutoffs for classification with ASD on the Autism Diagnostic Observation Schedule (ADOS). The ADOS is a semi-structured autism observation measure that has become the gold standard for assessing autistic behavior and is administered as part of the ATN initial comprehensive evaluation. An overall measure of autism severity was constructed from scores on the Autism Diagnostic Observation Schedule (ADOS) following recent work.[45] The ADOS-calibrated severity score provides a metric to quantify ASD severity with relative independence from the child's age and IQ. The score ranges from 1 to 10 with scores of 1–3 indicating a non-spectrum classification on the ADOS and scores of 4 and above indicating greater severity of autism on the ADOS.

Adaptive skills are an aspect of a child's development that is a major factor in future prognosis concerning the ability to function successfully and independently.[46] We measured adaptive skills using the Vineland-II Adaptive Behavior Scales.[47] The Vineland-II consists of four major adaptive domains: communication, socialization, and daily living skills, and motor skills (age < 6 years), which contribute to a single adaptive behavior composite score. The Vineland-II is a valid and reliable individually administered semi-structured caregiver interview designed to measure adaptive behavior in individuals from birth through age 90. The Vineland-II interview form is scored by the clinician assigning 0 to behaviors that are never performed by the individual, 1 to sometimes or partially, and 2 to usually performed by the individual. The Vineland-II has proven to be sensitive to changes in development over time. The composite and domain scores are expressed as standard scores with a mean of 100 and standard deviation of 15. The Vineland-II adaptive behavior composite score was used in this study, with higher scores indicative of better adaptive functioning.

Cognitive functioning for children with ASDs can range from low to high across any level of ASD symptom severity [48;49] and may produce an independent effect on HRQoL after controlling for ASD symptom severity.[45] Cognitive functioning was determined based on results of an individually administered, formal test of general cognitive abilities. We used one of three cognitive tests chosen on the age of the child and clinical preferences of the ATN clinician involved in the initial assessments. All three cognitive measures yield an overall composite score that is expressed as a standard score with a mean of 100 and standard deviation of 15 to describe an individual's cognitive ability and are comparable measures of general intelligence. The tools are the Stanford-Binet Intelligence Scales (5th Edition), the Mullen Scales, or the Bayley Scales. The Stanford-Binet Intelligence Scale is an individually administered formal test of intelligence used with individuals as young as 2 years and yields an IQ value. The Mullen is an individually administered comprehensive measure of cognitive functioning for children from birth through 68 months of age and yields a cognitive composite score, the Early Learning Composite. The Bayley Scale is an individually administered comprehensive measure of cognitive functioning for children from birth through 42 months of age and produces a Cognitive score. The Stanford Binet was used the most often, followed by the Mullen. To differentiate children with intellectual disability or significant delay, we use a cut-off of 69 on each of the cognitive instruments, as this cutoff corresponds to scores at or below the 2nd percentile rank and two or more standard deviations below the mean compared to same-age peers.

The ATN assessment battery includes information on a number of ASD-specific symptoms (e.g., social interactions, sensory issues, self-stimulatory and repetitive behaviors) as well as number of associated behavior symptoms (e.g., aggression, hyperactivity, and sleep disturbances) to characterize the child's behavioral adjustment. Some of the symptoms are parent-reported on an ATN custom parent-report form designed to capture parents' concerns about the child's behavior and the extent to which the behavior has been experienced as a problem from the parents' perspective. Other symptoms are clinician-reported using a diagnostic checklist and aimed at assessing the presence or absence of the core symptoms of ASD. We provide data on both sets of symptoms as reported by parents and clinicians to assist with the identification of conditions or behavioral adjustment patterns that might have large impacts on preference-based HRQoL.

We hypothesize that increasing impairment associated with an ASD would result in lower HRQoL scores from the two instruments. Thus, to test whether an instrument is sensitive to clinical outcomes, we would expect a statistically significant negative relationship between the HRQoL scores and the ADOS severity scale and significant positive relationships between the HRQoL scores and the cognitive functioning scales and the Vineland scales. We are unable to state a priori what symptoms are more likely to be associated with HRQoL scores, but we would expect overall that the presence of ASD-related symptoms would decrease HRQoL scores.

Statistical Analysis

Preference-based HRQoL scores were calculated from the HUI-3 and the QWB-SA instruments according to their scoring manuals. Associations among the two preference-based HRQoL instruments and clinical measures were tested using ordinary least squares regression analysis and Spearman correlation coefficients. We explored if there was any difference among the HRQoL scores by severity, adaptive behavior, cognitive functioning, and other symptoms. Ordinary least square regression was used because there was little evidence of ceiling or floor effects in the HRQoL scores. Restricted cubic splines with 3 knots were used in the analysis in order to relax assumptions of linearity. Restricted cubic splines allow continuous data to fit the ordinary least squares model without assuming a linear relation.[50] All regression analyses controlled for age and gender. Other demographic variables had no measurable impact on the estimated coefficients and thus, were not included. To test the predictive accuracy of the models, an adjusted R2 was calculated and validated using 150 bootstrap samples to address the potential for model over-fitting. All statistical analyses were performed using SAS 9.2 (Cary, North Carolina: SAS Institute) and the open source R (version 2.11.1) statistical computing language (R Development Core Team, 2005). The RMS library in R was used to construct regression models.

RESULTS

Data collection for the study began March, 2010 at the Little Rock ATN site and August 2010 at the Columbia New York ATN site. Data collection for the study remains ongoing with 150 surveys returned (88 returned from Little Rock and 62 from Columbia). The response rate for Little Rock was 59% out of 149 eligible participants and 46% out of 135 eligible participants at Columbia. 10% of families diagnosed with an ASD through the ATN in Little Rock and 5% of families in New York elected not to participate in the registry and could not be contacted for this study.

Table 1 provides demographic characteristics of enrolled children and their families. Children ranged in age from 4 to 17 with a mean of 8.6 (SD = 3.3). Consistent with the condition, the vast majority of children in the sample were male (85.3%). The sample contained a higher proportion of Caucasian children (78.7%) relative to African-American (8.7%) and Hispanic children (6.7%) than would be expected on the basis of the US population and the populations of New York and Arkansas respectively. Population-based surveys typically find similar racial distributions for children with autism and the study area.[51] In most cases, the survey respondent was the mother of the child with the ASD. At the time of the survey, 73.3% of caregivers reported being married with 25.4% reporting being divorced, separated, or never married.

Table 1
Demographics/characteristics of children with ASDs and their family (n=150).

Percentage distribution of responses on HUI-3 and QWB-SA are presented in Table 2. The speech and cognition domains appear to contribute the most to the HUI-3 scores and some contributions were made by the emotion and pain domains (Table 2a). Examination of the distribution of health states in the speech domain indicates that 20.7% of the children had level 1 speech – able to be understood completely when speaking with strangers or friends – whereas, 10.0% of caregivers reported that their child was unable to be understood when speaking to other people (or unable to speak at all). Similarly, 22.0% of caregivers reported level 1 cognition (able to remember most things, think clearly and solve day to day problems) and 45.3% reported having a little difficulty with these tasks. The percentage of missing responses increased with the emotion, cognition, and pain domains, which are more subjective measures of HRQoL. This pattern of response is consistent with associated difficulties in using caregivers as proxy respondents.

Table 2
Percentage distribution of responses for children with ASDs on two HRQoL instruments

Table 2b provides the percentage of responses for specific items of the QWB-SA CPX scale based on whether the child experienced the problem on any day over the three day recall period and the associated disutility weight for that problem. Examination of Table 2b indicates similarities and differences in responses to the QWB-SA items relative to the HUI-3. In particular, for both instruments, speech problems represent the condition with the highest percentage of problem responses with 54.1 percent of caregivers reporting stuttering/unable to speak clearly over the 3-day period. A significant percentage of children exhibited confusion/memory loss (26.0%) as well as other mental health conditions such as trouble falling asleep/staying asleep (34.7%), frustration/irritation/losing temper (52.0%) and excessive worry/anxiety (28.7%). Because only the symptoms experienced by the child with the highest disutility weight over the 3-day recall period is included in the calculation of the QWB-SA score, confusion and memory loss (0.559) represents an important contributor to the overall QWB-SA score for a significant number of children.

Table 3 provides mean HRQoL scores by diagnosis. The mean score for the HUI-3 averaged 0.66 (SD = 0.23) with a range of −0.03 to 1.0. The QWB-SA score averaged 0.59 (SD = 0.16) with a range from 0.18 to 1.0. Mean scores for the HUI3 increased for children with PDD-NOS relative to Autistic Disorder (0.70 vs. 0.64; p=0.283) and Asperger's Disorder (0.79 vs. 0.64; p=0.026). In contrast, mean scores for the QWB-SA were similar for both children with PDD-NOS and children with Asperger's Disorder (0.62) and were not significantly different from QWB-SA scores for children with Autistic Disorder.

Table 3
Mean preference-weighted scores by ASD diagnosis

Table 4 describes the clinical measures in relation to the HRQoL summary scores for the child using Spearman correlation. The ADOS severity score ranged from 2 to 10 with a mean of 7.2 (SD = 1.8). It had insignificant correlations with both the HUI-3 and the QWB-SA scores. The Vineland II Adaptive Behavior Composite Score averaged 67.4 (SD = 11.2) and had significant moderate correlation with the HUI-3 (ρ =0.521, p = < 0.001). Other Vineland II domain scores were also significantly correlated with the HUI-3 scores especially the communication, the daily living skills, and the motor skills domains. The QWB-SA scores were weakly correlated with the Vineland II composite and the domain scores. Their correlations, however, were statistically significant except for the Vineland II motor skill domain. The cognitive ability scores (based on the Stanford-Binet, the Mullen, or the Bayley) had statistically significant correlations with the HUI-3 score (ρ = 0.359, p < 0.001)) and weak correlation with the QWB-SA score (ρ = 0.166, p < 0.05).

Table 4
Clinical characteristics and correlations with HRQoL summary scores

Table 5 and Figure 1 provide parent-rated and clinician-rated symptoms associated with ASDs in relation to the HUI-3 and QWB-SA scores. We report HRQoL scores for the child unadjusted for age and gender following a recent suggestion.[52] To avoid problems with multiplicity, the significance level alpha of 0.05 was corrected using the Tukey, Cimnera, and Heyse adjustment.[53] For Table 5, 14 outcomes were analyzed using an adjusted significance level of 0.014. The p-values correspond to the spearman correlation coefficients between the ordinal variable symptoms and the two HRQoL scores. In particular, we investigate whether there was a trend between symptom severity and the HRQoL scores.

Figure 1
Clinician-rated symptoms associated with autism disorders in relation to the preference-weighted scores.
Table 5
Parent-reported ASD-related symptoms in relation to HRQoL scores.

Table 5 provides an indication of the extent of the different symptoms and exploratory findings on the magnitude of the HRQoL scores in relation to the extent of the problem. For example, in Table 5 only 15% of children did not have problems with language use and understanding and for these children, HRQoL scores for both the HUI-3 and QWB-SA were elevated relative to children with mild problems (18%), moderate problems (35%), or severe problems (33%). The HUI-3 score changed from 0.84 (SD = 0.09) for children that did not have language use and understanding problems to 0.51 (SD = 0.25) for children with severe language use and understanding problems (p < 0.01). For the QWB-SA, the mean score for the children without the symptom was 0.69 (SD = 0.16) and 0.51 (SD = 0.13) for the children with severe problems (p < 0.01). Other symptoms with large changes in HRQoL scores included attention span, hyperactivity, self-stimulatory and repetitive behaviors, and loss of or losing skills they previously had. The last symptom (loss of skills) had a low prevalence with only 4% of children having severe problems and another 4% having moderate problems. However, in both problem states and for both the HUI-3 and the QWB-SA, HRQoL scores ranged towards the lower end of the distribution of overall scores (0.43 – 0.49).

Figure 1 reports clinician-rated symptoms as to whether the problems are present or absent in relation to the HUI-3 and QWB-SA scores. For Figure 1, because 12 outcomes were analyzed, confidence intervals plots for the HUI3 and QWB-SA are presented using the adjusted significance level of 0.015. In general, the pattern of change scores for the HUI-3 and the QWB-SA associated with symptoms that were present in the child relative to children without symptoms was similar. However, for the QWB-SA, no significant differences in HRQoL scores were found among the clinician-reported ASD symptoms. In contrast, there were five clinician-reported symptoms where the HUI-3 score differed significantly including lacking spontaneous seeking to share enjoyment etc., delay or total lack of spoken language, lack of play for developmental level, stereotyped and repetitive motor mannerisms, and persistent preoccupation with parts of objects. Similar to the parent-rated symptoms, the changes in scores for the HUI-3 tended to be larger relative to the QWB-SA for a number of symptoms. For example, for the 38% of children who were identified as having a persistent pre-occupation with parts of objects, mean values for the HUI-3 score changed by 0.15 points compared to 0.06 with the QWB-SA.

Table 6 reports OLS regression coefficients and adjusted R2's from an analysis of the two instruments in relation to the clinical measures as the clinical measure change from the first to third quartile of their distribution. Effects for all predictors are presented as regression coefficients and indicate the change in HRQoL scores (HUI-3 or QWB-SA) as the value of the clinical variable changes from the 25th percentile (q1) to the 75th percentile (q3).[50] The findings in Table 5 indicate that the HUI-3 has better explanatory power in all of the estimated models relative to the QWB-SA based on adjusted R2 values. The R2 for the regression analysis using the Vineland composite had the highest adjusted R2 for both the HUI-3 (0.32) and the QWB-SA (0.03). Other components of the Vineland also had adjusted R2 values that were higher than the ADOS severity score. The ADOS severity score was negatively associated with the HUI-3 as expected although not significant. The QWB-SA did not have the expected sign or significance with the ADOS severity score. The ADOS severity score model performed poorly in terms of adjusted R2 when either the QWB-SA or the HUI-3 was used as the dependent variable. In general, the coefficients from the HUI-3 are larger than the QWB-SA.

Table 6
OLS regression of clinical measures and preference-weighted scores.

DISCUSSION

Information on the cost-effectiveness of pharmaceutical and other treatment services for children with autism is lacking. One reason for the lack of information may be the limited data on preference-based HRQoL outcomes associated with ASD-related conditions and their validity. Data on preference-based HRQoL outcomes is necessary for calculating QALYs in a cost-effectiveness analysis permitting comparisons across different conditions. Such information has the potential to identify the comparative value of services for children with ASDs to other conditions covered by private and public health insurance. Cost-effectiveness information also can help identify optimal treatment strategies and reduce unnecessary treatment variations.

In the US, treatment services for children with autism vary substantially according to the child's state of residence. Much of the variation in services is due to differences in funding at the state level for special education and Medicaid services. States also have been active in pursuing policies to increase access to services for children with autism through mandates on private insurers. Both Pennsylvania and Arkansas have passed comprehensive bills to support autism services requiring health insurers to cover yearly behavioral and clinical treatments at a cost of up to $36,000 and $50,000 respectively. Despite considerable variation in spending on services by states, there is scant evidence on the value of increased spending as information necessary for economic evaluations is lacking. Indeed, reports from the insurance industry criticize treatment emphasis on applied behavioral analysis – an expensive therapy – because they argue the value of alternative treatment services has not been demonstrated.[54]

This study had two objectives. We sought to test the sensitivity of two generic instruments for calculating preference-based HRQoL outcomes for children with ASDS and to identify the magnitude of potential QALY gains from treatment. This initial investigation finds evidence of construct validity for both the HUI-3 and QWB-SA, but the HUI-3 score appears to be more sensitive to ASD symptoms and severity for children with ASDs. Correlations between the clinical variables, especially the Vineland-II and cognitive functioning, were generally higher for the HUI-3 compared to the QWB-SA (Tables 3, ,4,4, and and6).6). Changes in the scores from the two instruments were generally in the correct direction when compared across varying levels of ASD-related symptoms. The magnitude of score changes was higher at the mean for the HUI-3 consistent with the larger range in scores for the HUI-3 relative to the QWB-SA and is likely a better choice for describing QALY gains from treatment or prevention (Tables 3 and and5).5). The scales differ across a number of dimensions beyond score range including time of administration, domains of health, and valuation strategies for assigning weights. The HUI-3 is easier to administer and appears to capture health states associated with language better than the QWB-SA. Still, it is clear that more research is necessary to be able to determine which generic HRQoL instrument would be most suitable in this context as other instruments suitable for children were not considered and evidence from other study designs, especially randomized trials, is needed.

Only two prior studies have measured preference-based HRQoL for children with ASDs. One study [21] reported on a sample of children (N=105) eligible for support programs using the HUI-3 with family caregivers as proxy respondents. Mean HUI-3 scores from that study (0.433) were generally inconsistent with mean scores from this study (0.66) likely due to selection bias associated with support eligibility. The second study [22] reported mean HUI-3 scores (0.61) for autistic disorder (N=11) that were more consistent with mean scores (0.64) for autistic disorder from the current study. While the prior studies provide HRQoL scores that can be used to measure the disutility for the health state “autism,” information regarding HRQoL scores associated with different behavioral states and symptoms associated with autism were not available. The current study provides initial evidence on how variation in behavioral states and symptoms for children with autism may be related to preference-based HRQoL. A full mapping of ASD conditions into preference-based HRQoL using larger samples and additional HRQoL instruments remains an area for future research. Findings from this study suggest that the HUI-3 is a useful instrument to measure the effectiveness of services for children with ASDs and should be included in clinical trials.

The large change in HRQoL across the range of ASD-related conditions and symptoms may also signal that preventing such conditions could have significant spillover effects for family members of children with ASDs. Theoretical and empirical work on developing estimates of family and caregiver effects, in terms of health and well-being, for use in cost-effectiveness evaluations are in the early stages of development.[5558] It will be interesting to assess whether the general associations between ASD-related conditions and symptoms in children translate into similar associations between the health and well-being of family members and caregivers. Incorporating family effects in economic evaluations of effective interventions for children with ASDs would provide a fuller account of (health) gains in economic evaluations. If treatment of children result in an increase in health in family members and caregivers as well, the total number of QALYs gained due to the intervention may increase, resulting in a more favorable cost-effectiveness ratio.[55]

In this study, we found the largest correlations between the HRQoL scores and the clinical measures describing behavioral conditions and symptoms. None of the correlations between the clinical measures and the HRQoL scores was strong as the two types of measures provide information on different constructs.[14] It needs noting that correlations between the HRQoL scores and the ADOS severity score were weak at best. Correlations between the HRQoL scores and cognitive functioning were better, but still not as strong as the behavioral measures. Future research will need to better elucidate the interactions between cognitive functioning and behavioral conditions associated with ASDs.

The study includes a number of limitations. First, we used caregivers as proxy respondents to obtain information on HRQoL for the child. Use of other methods to obtain HRQoL scores in this population such as direct elicitation techniques or discrete choice experiments, may generate findings that differ from the present study. In addition, this study uses two generic instruments with weights based on adult respondents. Instruments designed for use in children with weights developed from a child perspective [19] may also alter study findings. For example, this study did not test the sensitivity of the HUI-2 instrument that was designed to better reflect preferences of children [59] because the domains of the HUI-3 appear most appropriate for neurodevelopmental conditions. Responses to the HUI-3 indicate that most caregivers could respond to the questions about their child, although a higher percentage of caregivers did not complete answers in the most subjective domains – emotion, pain, and cognition. Whether other methods or instruments for obtaining preference-based HRQoL scores, such as the AQoL-8D [60] could significantly influence QALY estimates and associated cost-effectiveness ratios remains to be answered. At a minimum, the estimates in this study provide an important benchmark for comparing alternative methods and future studies.

The findings also could be influenced by the timing of the clinical and survey assessments. Children who enrolled in the ATN at the beginning of the registry will have had a longer time period between clinical assessment and responses to the HRQoL instruments. If the child's ASD-related conditions and symptoms changed over time, this could reduce the correlation between the clinical measures and the HRQoL scores. While many children with ASDs do improve over time, improvement does not occur rapidly if at all. We have information on dates for the clinical assessment and survey response and tested whether time significantly influenced reported estimates. We found a positive, but insignificant relationship between time differences in clinical measures and preference scores. Controlling for the timing of instrument administration did not significantly change the findings because of correlation with the child's age.

Finally, the study is limited by the relatively small sample size and the population studied. Our sample is not representative of population-based surveys as we enrolled from treatment clinics with presumably higher severity subjects. Also, we did not include non-English speaking respondents in the sample. Because of the small sample size, it is not possible to account for other factors that may be associated with HRQoL score changes across different levels or severity of clinical conditions and symptoms. For example, our descriptive analysis indicates a large change in scores associated with language development, but we are unable to assess the extent to which other clinical conditions are correlated with language development. If other unmeasured factors are correlated, it can reduce the impact of language development on HRQoL outcomes. Because of this limitation, it is not currently possible to use the reported scores as “off the shelf” weights for cost-effectiveness evaluations. Future research with larger sample sizes will be needed to map or cross-walk clinical conditions and symptoms into HRQoL scores. Ideally, data on HRQoL scores should be obtained over time from randomized controlled trials.[43] The primary contribution of this study is to suggest that such data collection schemes are feasible using the HUI-3.

A larger sample size also will be necessary to assess whether the “correct” health domains are included in the instruments used in this study.[30] Future research may consider psychometric approaches based on item response theory to assess this issue. The framework used in this study, where clinical information is combined with responses from preference-based HRQoL instruments, is a promising approach for psychometric testing of the instruments. Indeed, a major strength of the study is the use of a clinically-identified sample of children diagnosed with an ASD following standardized protocols.

CONCLUSIONS

Despite the need to identify optimal treatment strategies for children with ASDs and describe the value of services to public and private payers, research describing health outcomes in relation to clinical conditions is lacking. This study provides associations between ASD-related conditions and symptoms and preference-based HRQoL outcomes. We find support for the use of generic preference-based instruments to describe HRQoL in children with ASDs, especially for the HUI-3 instrument. Researchers should consider incorporating generic instruments to describe preference-based HRQoL in clinical trials and other longitudinal research studies involving children with ASDs to build the evidence base describing the cost-effectiveness of services provided in the care of this important population.

ACKNOWLEDGEMENT

The project was supported by a grant (R01MH089466) from the National Institute of Mental Health with JM Tilford and KA Kuhlthau serving as Principal Investigators. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health. The authors acknowledge the members of the Autism Treatment Network (ATN) for use of the data. The data for the study was collected as part of the ATN, a program of Autism Speaks. Further support came from a cooperative agreement (UA3 MC 11054) from the U.S. Department of Health and Human Services, Health Resources and Services Administration, Maternal and Child Health Research Program, to the Massachusetts General Hospital. The work described in the manuscript represents the independent efforts of the authors with no restrictions coming from the funding source or the ATN. None of the authors of this study reported a conflict of interest associated with the preparation of the manuscript. Maria Melguizo, Nupur Chowdhury, Rebecca Rieger, and Latunja Sockwell provided excellent research assistance.

Reference List

1. Strock M. Autism Spectrum Disorders (Pervasive Developmental Disorders) National Institutes of Mental Health, National Institutes of Health, U.S. Department of Health and Human Services; Bethesda, MD: 2004. NIH Publication No. NIH-04-5511.
2. Nordin V, Gillberg C. Autism spectrum disorders in children with physical or mental disability or both. I: Clinical and epidemiological aspects. Dev Med Child Neurol. 1996;38(4):297–313. [PubMed]
3. Rapin I. Autistic children: diagnosis and clinical features. Pediatrics. 1991;87(5 Pt 2):751–760. [PubMed]
4. Rapin I. The autistic-spectrum disorders. N Engl J Med. 2002;347(5):302–303. [PubMed]
5. Aman MG. Management of hyperactivity and other acting-out problems in patients with autism spectrum disorder. Semin Pediatr Neurol. 2004;11(3):225–228. [PubMed]
6. Arnold LE, Vitiello B, McDougle C, Scahill L, Shah B, Gonzalez NM, et al. Parent-defined target symptoms respond to risperidone in RUPP autism study: customer approach to clinical trials. J Am Acad Child Adolesc Psychiatry. 2003;42(12):1443–1450. [PubMed]
7. Militerni R, Bravaccio C, Falco C, Fico C, Palermo MT. Repetitive behaviors in autistic disorder. Eur Child Adolesc Psychiatry. 2002;11(5):210–218. [PubMed]
8. Couturier JL, Speechley KN, Steele M, Norman R, Stringer B, Nicolson R. Parental perception of sleep problems in children of normal intelligence with pervasive developmental disorders: prevalence, severity, and pattern. J Am Acad Child Adolesc Psychiatry. 2005;44(8):815–822. [PubMed]
9. Polimeni MA, Richdale AL, Francis AJ. A survey of sleep problems in autism, Asperger's disorder and typically developing children. J Intellect Disabil Res. 2005;49(Pt 4):260–268. [PubMed]
10. Limoges E, Mottron L, Bolduc C, Berthiaume C, Godbout R. Atypical sleep architecture and the autism phenotype. Brain. 2005;128(Pt 5):1049–1061. [PubMed]
11. Oyane NM, Bjorvatn B. Sleep disturbances in adolescents and young adults with autism and Asperger syndrome. Autism. 2005;9(1):83–94. [PubMed]
12. Kuhlthau K, Orlich F, Hall TA, Sikora D, Kovacs EA, Delahaye J, et al. Health-Related Quality of Life in children with autism spectrum disorders: results from the autism treatment network. J Autism Dev Disord. 2010;40(6):721–729. [PubMed]
13. Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-Effectiveness in Health and Medicine. Oxford University Press; New York, Oxford: 1996.
14. Neumann PJ, Goldie SJ, Weinstein MC. Preference-based measures in economic evaluation in health care. Annu Rev Public Health. 2000;21:587–611. [PubMed]
15. Neumann PJ, Greenberg D. Is the United States ready for QALYs? Health Aff (Millwood) 2009;28(5):1366–1371. [PubMed]
16. Tilford JM. Cost-effectiveness analysis and emergency medical services for children: issues and applications. Ambul Pediatr. 2002;2(4 Suppl):330–336. [PubMed]
17. Griebsch I, Coast J, Brown J. Quality-adjusted life-years lack quality in pediatric care: a critical review of published cost-utility studies in child health. Pediatrics. 2005;115(5):e600–e614. [PubMed]
18. Grosse SD, Prosser LA, Asakawa K, Feeny D. QALY weights for neurosensory impairments in pediatric economic evaluations: case studies and a critique. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):293–308. [PubMed]
19. Prosser LA, Hammitt JK, Keren R. Measuring health preferences for use in cost-utility and cost-benefit analyses of interventions in children: theoretical and methodological considerations. Pharmacoeconomics. 2007;25(9):713–726. [PubMed]
20. Prosser LA. Current challenges and future research in measuring preferences for pediatric health outcomes. J Pediatr. 2009;155(1):7–9. [PubMed]
21. Petrou S, Kupek E. Estimating preference-based health utilities index mark 3 utility scores for childhood conditions in England and Scotland. Med Decis Making. 2009;29(3):291–303. [PubMed]
22. Petrou S, Johnson S, Wolke D, Hollis C, Kochhar P, Marlow N. Economic costs and preference-based health-related quality of life outcomes associated with childhood psychiatric disorders. Br J Psychiatry. 2010;197:395–404. [PubMed]
23. Pyne JM, Sullivan G, Kaplan R, Williams DK. Comparing the sensitivity of generic effectiveness measures with symptom improvement in persons with schizophrenia. Med Care. 2003;41(2):208–217. [PubMed]
24. McCrone P, Patel A, Knapp M, Schene A, Koeter M, Amaddeo F, et al. A comparison of SF-6D and EQ-5D utility scores in a study of patients with schizophrenia. J Ment Health Policy Econ. 2009;12(1):27–31. [PubMed]
25. Pyne JM, French M, McCollister K, Tripathi S, Rapp R, Booth B. Preference-weighted health-related quality of life measures and substance use disorder severity. Addiction. 2008;103(8):1320–1329. [PMC free article] [PubMed]
26. Brazier J, Roberts J, Tsuchiya A, Busschbach J. A comparison of the EQ-5D and SF-6D across seven patient groups. Health Econ. 2004;13(9):873–884. [PubMed]
27. Marra CA, Woolcott JC, Kopec JA, Shojania K, Offer R, Brazier JE, et al. A comparison of generic, indirect utility measures (the HUI2, HUI3, SF-6D, and the EQ-5D) and disease-specific instruments (the RAQoL and the HAQ) in rheumatoid arthritis. Soc Sci Med. 2005;60(7):1571–1582. [PubMed]
28. Lamers LM, Bouwmans CA, van SA, Donker MC, Hakkaart L. Comparison of EQ-5D and SF-6D utilities in mental health patients. Health Econ. 2006;15(11):1229–1236. [PubMed]
29. McDonough CM, Tosteson AN. Measuring preferences for cost-utility analysis: how choice of method may influence decision-making. Pharmacoeconomics. 2007;25(2):93–106. [PMC free article] [PubMed]
30. Lipscomb J, Drummond M, Fryback D, Gold M, Revicki D. Retaining, and enhancing, the QALY. Value Health. 2009;12(Suppl 1):S18–S26. [PubMed]
31. Feeny D, Furlong W, Saigal S, Sun J. Comparing directly measured standard gamble scores to HUI2 and HUI3 utility scores: group- and individual-level comparisons. Soc Sci Med. 2004;58(4):799–809. [PubMed]
32. Smith-Olinde L, Grosse SD, Olinde F, Martin PF, Tilford JM. Health state preference scores for children with permanent childhood hearing loss: a comparative analysis of the QWB and HUI3. Qual Life Res. 2008;17(6):943–953. [PubMed]
33. Buysse CM, Raat H, Hazelzet JA, Hulst JM, Cransberg K, Hop WC, et al. Long-term health status in childhood survivors of meningococcal septic shock. Arch Pediatr Adolesc Med. 2008;162(11):1036–1041. [PubMed]
34. Prince FH, Geerdink LM, Borsboom GJ, Twilt M, van Rossum MA, Hoppenreijs EP, et al. Major improvements in health-related quality of life during the use of etanercept in patients with previously refractory juvenile idiopathic arthritis. Ann Rheum Dis. 2010;69(1):138–142. [PubMed]
35. Pyne JM, Sieber WJ, David K, Kaplan RM, Hyman RM, Keith WD. Use of the quality of well-being self-administered version (QWB-SA) in assessing health-related quality of life in depressed patients. J Affect Disord. 2003;76(1–3):237–247. [PubMed]
36. Feeny D, Furlong W, Torrance GW, Goldsmith CH, Zhu Z, DePauw S, et al. Multiattribute and single-attribute utility functions for the health utilities index mark 3 system. Med Care. 2002;40(2):113–128. [PubMed]
37. Kaplan RM, Anderson JP, Wu AW, Mathews WC, Kozin F, Orenstein D. The Quality of Well-being Scale. Applications in AIDS, cystic fibrosis, and arthritis. Med Care. 1989;27(3 Suppl):S27–S43. [PubMed]
38. Kaplan RM, Anderson JP. A general health policy model: Update and Applications. Health Services Research. 1988;23(2):203–235. [PMC free article] [PubMed]
39. Gold MR, Patrick DL, Hadorn DC, Kamlet MS, Torrance GW, Fryback DG, et al. Identifying and Valuing Outcomes. In: Gold MR, Siegel JE, Russell LB, Weinstein MC, editors. Cost-Effectiveness in Health and Medicine. Oxford University Press; New York, Oxford: 1996. pp. 82–134.
40. Czyzewski DI, Mariotto MJ, Bartholomew KL, LeCompte SH, Sockrider MM. Measurement of Quality of Well Being in a Child and Adolescent Cystic Fibrosis Population. Medical Care. 1994;32(9):965–972. [PubMed]
41. Bradlyn AS, Harris CV, Warner JE, Ritchey AK, Zaboy K. An investigation of the validity of the quality of Well-Being Scale with pediatric oncology patients. Health Psychol. 1993;12(3):246–250. [PubMed]
42. Tilford JM, Aitken ME, Goodman AC, Fiser DH, Killingsworth JB, Green JW, et al. Child health-related quality of life following neurocritical care for traumatic brain injury: an analysis of preference-weighted outcomes. Neurocrit Care. 2007;7(1):64–75. [PubMed]
43. Ruchlin HS, Insinga RP. A review of health-utility data for osteoarthritis: implications for clinical trial-based evaluation. Pharmacoeconomics. 2008;26(11):925–935. [PubMed]
44. Donald SC, Unutzer J, Schoenbaum M, Duan N, Lenert LA, Sturm R, et al. Can utility-weighted health-related quality-of-life estimates capture health effects of quality improvement for depression? Med Care. 2001;39(11):1246–1259. [PubMed]
45. Gotham K, Pickles A, Lord C. Standardizing ADOS scores for a measure of severity in autism spectrum disorders. J Autism Dev Disord. 2009;39(5):693–705. [PMC free article] [PubMed]
46. Paul R, Miles S, Cicchetti D, Sparrow S, Klin A, Volkmar F, et al. Adaptive behavior in autism and Pervasive Developmental Disorder-Not Otherwise Specified: microanalysis of scores on the Vineland Adaptive Behavior Scales. J Autism Dev Disord. 2004;34(2):223–228. [PubMed]
47. Sparrow SS, Cicchetti DV, Balla DA. Vineland-II Adaptive Behavior Scales. AGS Publishing; 2005.
48. Coplan J. Atypicality, intelligence, and age: a conceptual model of autistic spectrum disorder. Dev Med Child Neurol. 2003;45(10):712–716. [PubMed]
49. Coplan J, Jawad AF. Modeling clinical outcome of children with autistic spectrum disorders. Pediatrics. 2005;116(1):117–122. [PubMed]
50. Harrell FE., Jr. Regression Modeling Strategies with Applications to Linear Models, Survival Analysis, and Logistic Regression. Springer; New York: 2001.
51. Nicholas JS, Charles JM, Carpenter LA, King LB, Jenner W, Spratt EG. Prevalence and characteristics of children with autism-spectrum disorders. Ann Epidemiol. 2008;18(2):130–136. [PubMed]
52. Russell LB. Comment on Petrou and Kupek: use unadjusted HUI3 scores, not adjusted disutilities. Med Decis Making. 2010;30(1):143. [PubMed]
53. Zhang J, Quan H, Ng J, Stepanavage ME. Some statistical methods for multiple endpoints in clinical trials. Control Clin Trials. 1997;18(3):204–221. [PubMed]
54. Rothenberg B, Samson D. Special Report: Early Intensive Behavioral Intervention based on Applied Behavior Analysis among Children with Autism Spectrum Disorders. Technology Evaluation Center. 2009;23(9):1–61. Available from: URL:http://www.bcbs.com/blueresources/tec/vols/23/23_09.pdf. [PubMed]
55. Basu A, Meltzer D. Implications of spillover effects within the family for medical cost-effectiveness analysis. J Health Econ. 2005;24(4):751–773. [PubMed]
56. Brouwer WBF, Van Exel NJA, Tilford JM. Incorporating caregiver and family effects in economic evaluations of child health. In: Ungar WJ, editor. Economic Evaluation in Child Health. Oxford University Press; New York: 2009. pp. 55–76.
57. Bobinac A, van Exel JA, Rutten FF, Brouwer WB. Health Effects in Significant Others: Separating Family and Care-Giving Effects. Med Decis Making. 2010 [PubMed]
58. Bobinac A, van Exel NJ, Rutten FF, Brouwer WB. Caring for and caring about: disentangling the caregiver effect and the family effect. J Health Econ. 2010;29(4):549–556. [PubMed]
59. Torrance GW, Feeny DH, Furlong WJ, Barr RD, Zhang Y, Wang Q. Multiattribute utility function for a comprehensive health status classification system. Health Utilities Index Mark 2. Med Care. 1996;34(7):702–722. [PubMed]
60. Richardson J, Sinha K, Iezzi A, Khan M. Modelling the Utility of Health States with the Assessment of Quality of Life (AQoL) 8D Instrument: Overview and Utility Scoring Algorithm. Monash University, Melbourne, Centre for Health Economics; 2011. Research Paper 63.