|Home | About | Journals | Submit | Contact Us | Français|
To describe the development and validation of an instrument to measure parents' attention deficit-hyperactivity disorder (ADHD) treatment preferences and goals.
Parents of children 6–12 years diagnosed with ADHD in the past 18 months were recruited from 8 primary care sites and an ADHD treatment center (autism excluded). A 16-item medication and 15-item behavior therapy preference scale and a 23-item goal scale, developed following literature review, 90 parent and clinician semi-structured interviews, and input from parent advocates and professional experts, were administered to parents. Parent cognitive interviews confirmed item readability, clarity, content, and response range. We conducted an exploratory factor analysis, assessed internal consistency and test-retest reliability, and construct and concurrent validity.
We recruited 237 parents (mean child age 8.1 years, 51% Black, 59% from primary care, 61% of children medication naive). Factor analyses identified 4 medication preference subscales (treatment acceptability, feasibility, stigma, and adverse effects, Cronbach's α 0.74 to 0.87); three behavior therapy subscales (treatment acceptability, feasibility, and adverse effects, α 0.76 to 0.83); and three goal subscales (academic achievement, behavioral compliance, and interpersonal relationships, α 0.83 to 0.86). The most strongly endorsed goal was academic achievement. The scales demonstrated construct validity, concurrent validity (r= 0.3–0.6) compared to the Treatment Acceptability Questionnaire and Impairment Rating Scale and moderate to excellent test-retest reliability (ICC= 0.7–0.9).
We developed a valid and reliable instrument for measuring preferences and goals for ADHD treatment, which may help clinicians more easily adhere to new national treatment guidelines for ADHD that emphasize shared decision making.
Shared decision making (SDM) involves the active participation of both clinicians and families in treatment decisions, the exchange of information, discussion of preferences, and a joint determination of the treatment plan.1 This process helps families and clinicians balance the risks and benefits of different options in the context of personal values. Clinical trials have shown that families participating in SDM using standardized decision aids had increased knowledge, less uncertainty, greater participation in decision making, and were less likely to remain undecided about treatment.2 Based on these findings, the Institute of Medicine (IOM) has prioritized research on SDM in the setting of chronic illness3 and the 2010 Patient Protection and Affordable Care Act supports the implementation of SDM in clinical settings.4 The assessment of treatment preferences5 and goals from families provides a foundation for SDM.
ADHD is an ideal prototype for SDM in pediatrics because it results in impaired academic achievement, self-esteem, and interpersonal relationships for over 4 million children,5, 6 there are multiple evidence-based treatment options,7, 8 and personal and cultural values influence their selection.9–12 Even though ADHD has many adverse consequences, families, especially those in minority groups, are often reluctant to accept effective treatments for ADHD which include both stimulant medication and behavior therapy, alone or in combination.13, 14 In response, new national guidelines and a revised toolkit for ADHD treatment from the American Academy of Pediatrics (AAP) recognize this problem, stress that “family preference is essential in determining the treatment plan,” and provide forms to list treatment goals.8, 15 Despite this prioritization, clinicians continue to lack validated tools to elicit from families their preferences and goals for ADHD treatment. In fact, no measures have been developed to formally determine the goals and multiple domains of preference of parents of children with ADHD. To address these knowledge gaps in a clinical context ideal for SDM, this study was designed to develop and validate an instrument to assess parents’ ADHD treatment preferences and goals.
Subjects were recruited from 4 urban and 4 suburban primary care practices within The Children’s Hospital of Philadelphia (CHOP) Pediatric Research Consortium (PeRC), a primary care practice-based research network, and the CHOP Center for Management of ADHD, a specialty clinic staffed by an interdisciplinary team. These settings were chosen in order to develop an instrument that was useful in distinct practices settings and with a diverse population of families. Eligible parents had children between 6 and 12 years who were enrolled in grades kindergarten or higher, had been diagnosed with ADHD within the last 18 months, and were English speaking. Children with autism spectrum disorders and intellectual disability were excluded. In the primary care practices, children were diagnosed by a pediatrician using the Vanderbilt Rating Scale.16, 17 For children diagnosed at the ADHD Center, a combination of the following were used: structured diagnostic interview (K-SADS-PL)18, the ADHD Rating Scale-IV,19 Behavior Assessment System for Children (BASC)20, and an interview with parents to evaluate impairments.
A member of the research team approached eligible families, identified by clinic staff or using rosters generated from the electronic medical record, introduced the study and obtained consent. The final survey was completed within 20 minutes. Participants received $10 as an incentive. Chart review was then used to confirm the date of ADHD diagnosis, ADHD subtype, and comorbidities. Subjects were excluded if chart review demonstrated that they were not eligible.
The preference and goal items were developed after the completion of 60 semi-structured interviews with parents and 30 with clinicians addressing shared decision making in ADHD.21 In addition, we consulted an advisory panel with expertise in ADHD comprised of 2 practicing general and 2 developmental pediatricians, a school psychologist, a medical anthropologist, and a leader of a national ADHD advocacy group. Based on a review of existing measures assessing treatment acceptability, we used the Treatment Evaluation Inventory22 and the ADHD Knowledge and Opinion Rating Scale (AKOS) to inform our preference measure.23 We included medication and behavior therapy as the 2 treatment options in the preference scales because they were endorsed by practice guidelines from the American Academy of Pediatrics (AAP) and by results of the Multimodal Treatment of ADHD (MTA) Study.7,15 Although we found no instrument that directly captured ADHD treatment goals, we considered items from the Life Participation Scale for ADHD, an instrument that assesses treatment-related improvements in adaptive functioning, including quality of life, social development, and emotional regulation.24–26 Ultimately, we developed a draft instrument with three separate sections: (1) medication treatment preferences, (2) behavior therapy treatment preferences, and (3) treatment goals. Iterative rounds of review lead to revisions in the scales’ content as well as technical elements including response categories and word choice. To incorporate the perspectives of children, we reviewed the instrument with 3 children (≥ 10 years of age) who had been diagnosed with ADHD. These children responded favorably to the instrument and did not suggest changes.
We then pilot-tested the instrument with 6 parents of children recently diagnosed with ADHD and conducted cognitive interviews. These 30–45 minute interviews assessed readability, clarity, response range, and content and lead to the revision of multiple items and the rewriting of the survey instructions. The finalized measure included 3 sections, 15 items addressing preferences for behavior therapy, 16 items for medication treatment, and 23 goal items with responses Likert-scaled from 0 to 4. An example of the introduction, format, and response options for these scales are shown in Figure 1. Each goal scale item included two parts, the first indicating if each item was a concern for the parent now and the second indicating if the parent wanted to work on this concern with the child’s doctor or psychologist. Because the responses for these items were highly correlated (r=0.83–0.94), all subsequent analyses are presented based exclusively on parent responses to the statement “This is a concern for me now.”
The percentage of missing data was determined for each item. Missing data was rare. There were no more than 5% of subjects missing data for any of the preference and goal items; also, only 3, 4 and 4 out of 237 parents had more than 2 missing values for the medication preference scale, behavior therapy preference scale, and goals scale, respectively. We used multiple imputation methods and imputed 10 data sets to complete missing data on the preference and goal scale items (≤5% missing for each item) and for baseline behavior therapy receipt (14% missing) in a manner designed to avoid bias in our results and produce correct confidence intervals.27 All statistical analyses included the imputed data.
To identify the latent structure of the preference and goal scales, we conducted exploratory common factor analysis.28–30 The number of factors retained for rotation was determined using minimum average partials,31 parallel analysis,32 and Bartlett’s Test33 supplemented by a visual scree test.34 Factor solutions were rotated orthogonally (Varimax) and obliquely (Promax) to determine the most parsimonious factor structure. Factor analysis yields patterns of coefficients, known as factor loadings, that indicate the strength of the association between individual items and proposed factors. Pattern coefficients ≥0.40 were predetermined to indicate a salient association between an item and factor. Non-salient items and those that loaded on multiple factors were dropped from the scale to obtain a simple structure. Factor structures were evaluated according to the following criteria: (1) three or more unique and salient loadings on each factor; (2) internal consistency reliability (Cronbach’s alpha ≥0.70 for the overall scale and for each factor35); and (3) theoretical and clinical meaningfulness of the proposed factors.
We described the study population using descriptive statistics. Content validity of the scales was established through the previously described process of instrument development. Once the factor structures of the preference and goal scales were identified, we computed the mean ratings of the items of each scale and of each factor (range: 0–4). Accounting for the 10 imputed datasets, subscale means were then compared. The mean item ratings were then used to assess construct validity, a measure of whether a scale performs as hypothesized.36
To establish construct validity, we used a series of linear regression models to test pre-specified hypotheses. We hypothesized that, in the subset of children who were diagnosed at least 3 months prior to enrollment, preference ratings would indicate a greater acceptance of medication among parents of children currently receiving medication for ADHD treatment, and a greater acceptance of behavior therapy among parents of children currently receiving that treatment. We also hypothesized that parents who were at an earlier stage of decision making according to an item from the Ottawa Health Decision Centre toolkit would have lower mean item ratings for both medication and behavior therapy preferences.37 In addition, we hypothesized that parents of children with combined-type versus inattentive ADHD would have higher goal item ratings for the domain of behavioral compliance, that parents of children with problematic performance in academic areas (assessed by the Vanderbilt Performance Scale16, 17) would have higher goal item ratings for the domain of academic achievement, and that parents of children with Oppositional Defiant Disorder (ODD) would have higher goal item ratings for the domain of interpersonal relationships. P values <0.05 were considered significant.
In order to test the concurrent validity of the preference scale, we simultaneously administered the Treatment Acceptability Questionnaire, Parent version (TAQ-P),38 a validated measure of the acceptability of medication and behavior therapy. To test the concurrent validity of the goal scale, we administered the Impairment Rating Scale, Parent version,39 a reliable and valid instrument that assesses the impact of ADHD in distinct contexts. Using Pearson correlation coefficients and adjusting for multiple comparisons using Sidak’s correction,40, 41 we correlated the mean overall and subscale scores for the preference scales with those of the TAQ-P and the mean overall and subscale goal scores with those of the Impairment Rating Scale. Correlations were designated as small (r=0.10–0.29), medium (r=0.30–0.49), and large (r≥0.50).42
Twenty-one families were re-administered the survey by phone 7–14 days following the completion of initial measures. Test-retest reliability for the overall scales and for individual factors was determined using an interclass correlation coefficient (ICC) with a value of ≥0.80 indicating excellent agreement, between 0.61 and 0.79 indicating moderate agreement, and between 0.41 and 0.60 indicating fair agreement.
Factor analysis was performed using SAS version 9.1 (SAS Institute, Cary, NC), and remaining statistical analyses were performed using Stata versions 10.0 and 11.0 (StataCorp, College Station, TX). The study was approved by the CHOP Institutional Review Board.
Characteristics of participants are described in Tables 1 and and2.2. We approached a total of 205 potential subjects at CHOP primary care centers and recruited 141 eligible subjects from this population, with 21 (10%) refusing. The remaining 43 were found to be ineligible. Those who refused were significantly more likely to identify their race as white or other (p=0.05), and have an older child (p=0.04). We recruited an additional 96 subjects from the CHOP ADHD Center, with only 2 (2%) refusing. These subjects did not differ significantly by race or gender from the overall population receiving care at the ADHD Center. Our total enrollment included 237 eligible participants.
A 4-factor oblique solution best met the criteria for factor retention for the medication preference scale. The 4 domains were treatment acceptability, feasibility, stigma, and adverse effects, each with Cronbach’s α coefficient ≥0.7 (Table 3A). All of the items loaded significantly on one factor, and were retained in the final model. This factor solution accounted for 59% of the total variance. The “acceptability” subscale addressed overall comfort with and belief in the effectiveness of medication, “feasibility” included items related to practical barriers to medication treatment, “stigma” items measured negative perceptions of the child or family by others due to medication treatment, and “adverse effects” addressed issues regarding side effects of medication.
A 3-factor oblique solution best met the criteria for factor retention for the behavior therapy scale. The 3 domains were treatment acceptability, feasibility, and adverse effects, each with Cronbach’s α coefficient ≥0.7 (Table 3B). Fourteen of fifteen items loaded on any one of the factors, and were retained. This factor solution accounted for 49% of the total variance. The “acceptability” subscale addressed overall comfort with and belief in the effectiveness of behavior therapy, “feasibility” included items related to practical barriers to behavior therapy, and “adverse effects” perceived negative consequences of receiving behavior therapy for the child.
A 3-factor oblique solution best met the criteria for factor retention for the goals scale. The three domains were academic achievement, behavioral compliance, and interpersonal relationships each with Cronbach’s α coefficient ≥0.8 (Table 3C). Sixteen items were retained in the final instrument. We dropped six items that did not load or loaded on multiple factors. This factor solution accounted for 50% of the total variance. The “academic achievement” subscale addressed learning, “behavioral compliance” included items related to following rules at school and home, and “interpersonal relationships” included items addressing getting along with peers and family members (see the online appendix for the final preference and goal scales).
Table 4 presents the mean rating for each factor and each item on the preference and goal scales. The mean item rating for the overall behavior therapy preference scale was significantly higher than the mean item rating for the medication preference scale (p<0.001), suggesting that subjects overall had a greater preference for behavior therapy than for medication. Of medication rating subscales, scores were the lowest for acceptability and adverse effects (p<0.0001), indicating that these were the most common barriers to medication treatment. For behavior therapy, feasibility scores were lowest (p<0.001) suggesting that practical concerns were the primary barrier to parents preferring behavior therapy. On the goals scale, improved academic achievement was the most strongly endorsed goal (52% of the study population) followed by behavioral compliance (46%) and interpersonal relations (7%) (Table 4). 8% of the sample endorsed more than one goal most strongly.
Twenty one of the first subjects recruited, who had similar characteristics to the overall sample (p>0.05 for all comparisons), completed the instrument again 2 weeks after their initial enrollment. The preference and goal scale demonstrated excellent test-retest reliability for the overall scales, with an ICC of 0.9 for each overall scale. ICCs for the overall scales and subscales are shown in Table 5.
The preference and goal scales performed as hypothesized (Tables 6 and and7).7). Subjects whose children were on ADHD medication at the study start had higher medication preference mean item ratings than those who were not (p<0.0001). Although not statistically significant, patterns were similar for behavior therapy. Additionally, subjects who reported being at a later stage of decision making had higher medication preference means (p<0.0001) and non-significantly higher behavior therapy scores (p=0.1). Subjects who rated their children’s school performance as somewhat problematic or problematic had higher mean item ratings for the academic achievement goal domain than others (p<0.001). Parents whose children were diagnosed with combined-type ADHD more strongly endorsed the goal of behavioral compliance than those whose children had inattentive type (p<0.001). Finally, subjects whose children had a comorbid diagnosis of ODD had a significantly higher mean item rating for the interpersonal relationships goal domain than subjects whose children did not have ODD (p<0.001).
The scales demonstrated concurrent validity. The medication preference scale was highly correlated with the Medication TAQ-P scores (r=0.60), and the behavior therapy preference scale was moderately correlated with the Behavior Therapy TAQ-P scores (r = 0.44). Additionally, higher scores on the interpersonal relationships treatment goal domain were highly correlated with Impairment Rating Scale items addressing relationships with playmates and siblings (r=0.62 and 0.61, respectively), and moderately correlated with the items addressing relationships with parents and the overall family (r=0.31 and 0.41, respectively). The academic achievement goal domain was highly correlated with the Impairment Rating Scale item relating to school performance (r=0.62).
Clinicians currently lack tools for measuring families’ preferences and goals for ADHD treatment, a foundation for SDM and a priority in new guidelines from the American Academy of Pediatrics.43 Through an iterative process that included literature review and engaged a diverse group of families, clinicians and content experts, we developed a reliable and valid instrument to assess parents’ preferences and goals for ADHD treatment. Each scale and subscale demonstrated internal consistency reliability with α≥0.74, above the standard threshold of 0.7. Scales performed as hypothesized in the analysis of construct validity, demonstrated moderate to high concurrent validity, and had excellent test-retest reliability.
Our results extend prior research findings by using exploratory factor analysis to identify domains that shape parents’ preferences for medication and behavior therapy. While previous scales including the TAQ-P measured only the acceptability of ADHD treatment,22, 23, 38 we included items addressing multiple dimensions of preference, including those that might be especially salient for minority groups.10, 13, 14, 43–45 Through this process, we identified four factors that influence medication preference: treatment acceptability, feasibility, stigma and adverse effects. Among these factors, scores for acceptability and adverse effects were lowest suggesting that these were the most important barriers for parents to medication treatment. In contrast to medication, we found three factors that shaped behavior therapy preference: acceptability, adverse effects and feasibility. Consistent with prior studies that have highlighted practical barriers to behavior therapy receipt,21, 46 we found that feasibility scored lowest among behavior therapy preference factors, which suggests that families identify practical barriers as most important in limiting participation in behavior therapy.
While prior instruments including the Impairment Rating Scale have assessed the impact of ADHD on different aspects of a child’s life,39 no prior instrument has been developed to assess families’ goals for ADHD treatment. Our results indicate that families consider goals in 3 broad domains: academic achievement, behavior compliance, and interpersonal relationships. Highlighting the need to coordinate ADHD care, each domain spans more than one setting; academic achievement depends upon success in the classroom and homework completion, behavior compliance necessitates children following both family and school rules, and improving interpersonal relationships requires addressing interactions with teachers, peers, and family. Among the goal domains and consistent with existing literature on ADHD,47 academic achievement was the most commonly endorsed goal, rated highest by 52% of the study population. However, 46% of parents most wanted to address behavior problems, and 7% interpersonal relationships. Reflecting that problems in ADHD span multiple domains for individual children, families sometimes endorsed multiple goals.7
The variation in goals between families underscores the importance of measuring goals in order to match families with evidence-based treatments most likely to achieve the outcomes they prioritize. For example, although medication for symptom control may be beneficial to help families reach any of these goals,7 children with academic problems may especially benefit from educational consultation with reading or math specialists, organizational skills training, or behavior therapy focused specifically on homework strategies.47 In contrast, those prioritizing improved behavior may benefit from behavior therapy focused on parent training and consultation with the teacher in order to foster family-school collaboration.47, 48 To improve interpersonal relationships, social skills training accompanied by behavior therapy focused on promoting effective peer relationships, such as through supervised play, may be most beneficial.47, 48 As these examples illustrate, by tailoring evidence-based treatment to match priorities for families, goal-directed care has the potential to improve outcomes. Prospective study is warranted.
Targeting treatment toward families’ goals with evidence-based treatments they prefer may also contribute to improved treatment adherence in ADHD. Although systematic reviews have found that adherence strongly predicts improvement for children on all ADHD treatments,49 multiple studies have documented limited adherence to treatment even in tightly controlled research settings.13, 14, 50–52 Reflecting challenges in promoting adherence, research has shown that treatment acceptability alone is not associated with improved adherence in ADHD.23 Despite these barriers, the approach of measuring and targeting treatment toward families’ goals may directly promote adherence since adherence depends upon families recognizing that their regimen is beneficial.53 In addition, by measuring multiple dimensions of preference as opposed to only acceptability, clinicians may be able to more effectively elicit and address obstacles to treatment that could compromise adherence. Additional research will be needed to assess the benefits of this approach in clinical practice.
This study had several limitations. While the study sample included an even balance of white and black parents from diverse practice settings, only 4% of the sample was Hispanic and all study practices were from one health system in a single region of the United States, potentially limiting generalizability. Additionally, all study subjects were receiving care for ADHD. Findings may not reflect the perspectives of families who have not pursued treatment. Although subjects were diagnosed using valid instruments, the study team did not independently perform a standardized diagnostic evaluation. Our sample was too small to complete a confirmatory factor analysis or to conduct exploratory factor analyses for subpopulations. Although prior work has demonstrated the unique perspectives of children on ADHD treatment,54, 55 the measurement of children’s preferences and goals was beyond the scope of this study.
We expect that this instrument will ultimately be helpful in both clinical practice and research. In order to facilitate clinical use, work will be needed to develop a manual and scoring system and pilot test optimal workflows for using the ADHD preference and goal instrument with families. A better understanding of the responsiveness of the instrument to changes in preferences and goals over time, the impact of prior treatment receipt on preferences and goals, as well as the association of measured preferences and goals with subsequent treatment choices and clinical outcomes will provide a foundation for use of the tool in research. With this tool, scholars should be able to better determine whether prescribed treatments are consistent with families’ preferences and whether outcomes achieved match the families’ priorities.
We developed a reliable and valid instrument to assess parents’ preferences and goals for ADHD treatment. Although additional research is needed to confirm the factor structure of these scales, they may help pediatric clinicians meet new treatment guidelines from the AAP that prioritize SDM. Our analyses demonstrate that preferences for both medication and behavior therapy are multidimensional constructs. Salient barriers to treatment for individual families’ may be better recognized using the preference measures. By measuring goals and tracking progress toward them, researchers and clinicians may also be better able to assess if current treatments are helping families achieve outcomes that they most value. Such an approach may help both families and clinicians promptly identify and troubleshoot barriers to treatment success and keep families engaged in the treatment process.
The Institute of Medicine prioritized research to foster shared decision making (SDM) in chronic illness and guidelines for ADHD prioritize this process. We developed a novel instrument to assess parents’ preferences and goals for ADHD treatment, a foundation for SDM.
We thank Fran Barg, James Massey, the clinicians at CHOP’s ADHD Center, Mark Ramos, and Russell Localio for their help with the conduct and analysis of this research. Marie Paxson of Children and Adults with Attention Deficit/Hyperactivity Disorder provided advice to the research team and reviewed the scales. We also thank the network of primary care physicians, the patients, and the families for their contributions to clinical research through the Pediatric Research Consortium at The Children’s Hospital of Philadelphia.
Research Support: This research was supported by Award Number K23HD059919 from the Eunice Kennedy Shriver National Institute of Child Health & Human Development. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Eunice Kennedy Shriver National Institute of Child Health & Human Development or the National Institutes of Health.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Financial Disclosure and Conflicts of Interest: None.