|Home | About | Journals | Submit | Contact Us | Français|
Attention-deficit hyperactivity disorder (ADHD) places a high burden on families and communities, including enormous medical and school-related costs (Harpin, 2005; Pelham, Foster, & Robb, 2007) that contribute to the public health concern regarding the increasing prevalence and impact of early onset mental health disorders (Perou et al., 2013). Access to empirically supported mental health care for children with ADHD varies widely across geographic areas. In non-metropolitan areas, access to psychiatry (Holzer, Goldsmith, & Ciarlo, 1998; Thomas & Holzer, 2006) and psychology (American Psychological Association, 2014; Michalski, Mulvey, & Kohout, 2010; Rural Health Information Hub, n.d.; Smalley et al., 2010) providers with specialty training in child mental health care is limited. This service gap is filled by primary care providers (PCPs) who diagnose and prescribe medications and community therapists who provide psychotherapy. However, most PCP’s do not provide pharmacological treatment that is consistent with best practice guidelines (Leslie & Wolraich, 2007; Wolraich et al., 2011), and most therapists do not provide the evidence-based behavioral interventions indicated for ADHD (Hoagwood, Kelleher, Feil, & Comer, 2000). New models of care are needed to bring child mental health expertise to underserved communities and to promote the delivery of evidence-based mental health treatments.
The Patient Protection and Affordable Care Act (2010) and the Institute of Medicine (Lustig, 2012) recommend telehealth technologies for reducing geographic disparities in health care availability and quality. The Health Resources and Services Administration (HRSA) defines telehealth as, “The use of electronic information and telecommunications technologies to support and promote long-distance clinical health care, patient and professional health-related education, public health, and health administration (Telehealth, n.d.). When telehealth relies on synchronous (interactive) technologies, such as videoconferencing, to deliver medical care to patients, the Centers for Medicare and Medicaid (CMS; Centers for Medicare & Medicaid Services, 2014; Medicaid.gov Telemedicine, n.d.) uses the term telemedicine; and when that care specifically involves mental health or psychiatric services, the terms telemental health (TMH) and telepsychiatry, respectively, are generally used (Yellowlees, Shore, & Roberts, 2010). Asynchronous, or delayed, telehealth technologies disseminate evidence-based interventions through the self-administration of intervention tools or by providing training materials through recordings of clinical protocols or documents through patient portals and websites (Epstein et al., 2008). Synchronous and asynchronous telehealth technologies show potential for distance training of community-based clinicians in evidence-based treatments of complex disorders (Archambault, Mansfield, Evans, & Keitner, 2014; Davies, Yeung, Mori, & Nixon, 2012; Siminerio, Ruppert, Huber, & Toledo, 2014; Socolovsky et al., 2013).
More attention has been given to the use of telehealth to support PCPs in delivering guideline-driven pharmacotherapy to children with mental health conditions (Greenberg, Boydell, & Volpe, 2006; Lau, Way, & Freemont, 2011; Myers, Valentine, & Melzer, 2007; Myers, Vander Stoep, Zhou, McCarty, & Katon, 2015; Yellowlees, Hilty, Marks, Neufeld, & Bourgeois, 2008) than to the use of these technologies to provide psychotherapy (Nelson, Barnard, & Cain, 2003; Reese et al., 2013; Reese, Slone, Soares, & Sprang, 2012; Xie et al., 2013) or to train therapists to deliver evidence-based interventions. No studies have addressed the effects of providing child telehealth services on caregivers who, in underserved communities, bear a disproportionate burden for coordinating services and managing child behavioral disorders.
Caregivers of children with ADHD in both community and clinical samples have described high levels of depression, anxiety, and substance use problems (Anastopoulos, Guevremont, Shelton, & DuPaul, 1992; Baldwin, Brown, & Milan, 1995; Bussing et al., 2003; Chronis et al., 2003; Chronis-Tuscano et al., 2013; Evans, Sibley, & Serpell, 2008; Hinshaw, 2007; Rockhill, Violette, Vander Stoep, Grover, & Myers, 2013) and decreased quality of life (Harpin, 2005; Heath, Curtis, Fan, McPherson, 2015; van den Hoofdakker et al., 2010). The stress and strain of raising children with ADHD have shown persistence for follow-up periods of up to eight years (Barkley, Fischer, Edelbrock, & Smallish, 1990; Barkley, Fischer, Edelbrock, & Smallish, 1991). Parenting stress adversely affects child-rearing practices and child disruptive behaviors (Barry, Dunlap, Cotten, Lochman, & Wells, 2005; Kazdin & Whitley, 2003; Podolski & Nigg, 2001; Qi & Kaiser, 2003) that may improve with parent behavior training (Anastopoulos, Shelton, DuPaul, & Guevremont, 1993; Pisterman et al., 1992; Wells et al., 2000). Conversely, child ADHD and disruptive behaviors may affect both parent functioning and their response to interventions (Heath at al., 2015; van den Hoofdakker et al., 2010). Thus, there is some indication of reciprocity in parent-child functioning and foci for intervention.
If telehealth is to be integrated into mainstream mental health practice for families of children with ADHD, interventions with demonstrated efficacy when conducted in person will need to be replicated using telehealth technologies (Baum, Epstein, & Kelleher, 2013).
The Children’s ADHD Telehealth Treatment Study (CATTS) was a community-based randomized controlled trial (RCT) that compared the effects of two service delivery models for improving the care and outcomes for children with ADHD and their caregivers. The CATTS trial tested the effectiveness of a hybrid telehealth service-delivery model with combined pharmacotherapy and caregiver behavior training for reducing children’s ADHD-related symptoms and caregivers’ distress (CATTS service delivery model) compared to treatment in primary care with a single teleconsultation session (augmented primary care: APC). Children in the CATTS service-delivery model achieved significantly greater improvements than children in the APC model for inattention, hyperactivity/impulsivity, oppositional/defiance, and adaptive functioning (Myers, et al., 2015). The current investigation compares caregiver responses to these two service delivery models. We hypothesized that (a) caregivers of children randomized to the CATTS service-delivery arm would experience greater reductions in distress compared to caregivers of children randomized to the APC arm, and that (b) a significant part of the effects on caregiver distress would be explained by improvement in child ADHD symptoms.
We propose mediation of caregiver outcomes by treatment-related improvements in child symptoms and behaviors – specifically, inattention, hyperactivity/impulsivity, and oppositional/defiant behaviors, as well as adaptive functioning – for several reasons. Oppositional/defiant behaviors in children with ADHD have been linked to parent distress separately from inattentive and hyperactive/impulsive symptoms (Podolski & Nigg, 2001) and typically constitute the reason for seeking treatment (Evans, Owens, Bunford, 2014). Behavioral interventions constitute the best evidence-base for the treatment of oppositional defiant behaviors (Fabiano et al., 2009; McMahon and Forehand, 2003) and transactional models support improvement in parent distress and functioning with improvement in disruptive child behaviors (Chronis, Chacko, Fabiano, Wymbs, & Pelham, 2004; Davies and Cummings, 1998). Therefore, we hypothesized that our caregiver behavior training intervention would be associated with reductions in children’s oppositional-defiant behaviors with a reciprocal reduction in caregiver distress and disaggregate these three types of ADHD-related symptoms in the current study.
The CATTS trial was approved by the Institutional Review Board of Seattle Children’s Research Institute and monitored by a Data Safety and Monitoring Board. Informed consent and assent were obtained from all caregivers and children who were included as participants in the study. The CATTS trial methods (Vander Stoep & Myers, 2013), interventions (McCarty, Vander Stoep, Violette, & Myers, 2015), and children’s outcomes (Myers, et al., 2015) have been described in detail elsewhere.
Between November 2009 and August 2012, we recruited children from the practices of 88 PCPs in seven communities in underserved areas of Washington and Oregon states (Myers et al., 2015; Vander Stoep & Myers, 2013). All study services were provided at community clinics that had high bandwidth, point-to-point videoconferencing connections (384 kbits/sec to 1.0 MB/sec).
Child participants included boys and girls between ages 5.5 and 12.9 years who met diagnostic criteria for ADHD (American Psychiatric Association, 1994), attended school, and resided with English-speaking caregivers who were their legal guardians. Exclusion criteria included being in state custody, unavailability of the legal guardian, or having medical, developmental, or psychiatric disorders that required interventions beyond the scope of the study (Vander Stoep & Myers, 2013).
Eligibility, recruitment and randomization procedures were determined in a three step process. First, we reviewed medical records to evaluate exclusionary criteria. Caregivers then completed the Child Behavior Checklist online (CBCL; Achenbach & Rescorla, 2001). If the CBCL diagnostic subscale for ADHD was above the cutoff (T > 65), caregivers met in-person at the community telehealth clinic with a CATTS therapist who administered informed consent and three modules of the Computerized Diagnostic Interview Schedule for Children-IV (CDISC-IV; Shaffer et al., 1996) to confirm a diagnosis of ADHD and to establish the presence of common comorbid conditions. Consistent with other approaches (Jensen et al., 2001; Lewczky, Garland, Hurlburt, Gearity, & Hough, 2003), we considered an anxiety disorder (AD) present if caregivers endorsed criteria on the CDISC module for generalized anxiety disorder or if the anxiety disorder diagnostic subscale on the CBCL showed a T-score > 70. We used the same approach to determine the presence of oppositional defiant disorder (ODD). Children were then administered assent. Caregivers completed a baseline assessment.
We carried out block randomization with stratification by age group (5.5–9.9 vs. 10.0–12.9 years) and site (N=7). A statistician generated a set of random numbers with an allocation ratio of 1:1 for assignment in consecutive order to one of two service models: (1) the CATTS delivery model and (2) the APC model (Vander Stoep & Myers, 2013).
Caregivers completed the assessments remotely from personal computers with data entered directly into the outcomes database (Geyer et al., 2011) at baseline, and 4, 10, 19, and 25 weeks post-randomization. The 25-week assessment was timed for completion halfway between the end of the CATTS service delivery (22 weeks) and return to their PCP’s care (27 weeks). Overall, caregivers completed a mean of 4.8 ± .7 assessments; 86.5% of caregivers in the CATTS service arm and 90.2% in the control arm completed all five of the assessments. The CONSORT diagram is available elsewhere (Myers et al., 2015).
The CATTS service delivery model used a hybrid approach with telehealth and in-person services (Hilty & Yellowlees, 2015) to deliver to families six sessions of combined treatment (MTA Cooperative Group, 1999). We used synchronous videoconferencing to provide access to child psychiatry services including pharmacotherapy. We used multiple asynchronous telehealth technologies to enhance the skills of master’s level community therapists to deliver in-person an evidence-based caregiver behavior training intervention for children with ADHD (McCarty, et al., 2015). These skilled therapists could serve as an enduring resource to their communities. All sessions were recorded, and sessions were randomly selected to rate clinician adherence to their intervention protocols using checklists that outlined the essential treatment components. Telepsychiatrists and therapists showed high fidelity to their protocols (Myers, Vander Stoep, & Lobdell, 2013; Vander Stoep & Myers, 2013).
Our decision to provide combined treatment was based on empirical evidence (Majewicz-Hefley & Carlson, 2007; MTA Cooperative Group, 1999; Strand et al., 2012) and the practice guidelines for the treatment of ADHD (Pliszka & AACAP Workgroup on Quality Issues, 2007; Wolraich et al., 2011) indicating that a combined treatment model yields optimal outcomes for children with ADHD and comorbid ODD and/or ADs. We anticipated that PCPs would predominantly refer children with ADHD and a comorbid disorder to the trial based on our expectation that PCPs would manage uncomplicated cases of ADHD (Epstein et al., 2008; Knapp et al., 2012; Leslie, Weckerly, Plemmons, Landsverk, & Eastman, 2004; Power, Mautone, Manz, Frye, & Blum, 2008; Wolraich et al. 2011) and our experience providing telepsychiatry consultation to PCPs (Myers et al., 2010). The six CATTS sessions were spaced three to four weeks apart over a 22-week period. The two service-delivery arms have been described elsewhere (McCarty, et al., 2015; Myers, et al., 2015; Vander Stoep & Myers, 2013) and are summarized below.
The principal investigator (KM) developed a manual and trained child psychiatrists to deliver evidence-based diagnostic and pharmacological care for ADHD through videoconferencing based on the five ADHD algorithms in the Texas Children’s Medication Algorithm Project (TCMAP; Pliszka, et al., 2006). Telepsychiatrists were instructed to adjust medication to reach a treat-to-target goal of 50% reduction in ADHD-related symptoms as determined by caregiver ratings of the child’s current symptom severity (Rockhill, Tse, Fesinmeyer, Garcia, & Myers, in press). At each session, telepsychiatrists also delivered psychoeducation about the neurobiology of ADHD (Stahl, 2008). The modules included: (1) Role of Medication in ADHD Treatment; (2) Central Nervous System (CNS) Involvement: ADHD Symptoms and Treatment Focus; (3) CNS Involvement: The Prefrontal Cortex and Executive Functioning; (4) Conditions Comorbid with ADHD; (5) Long Term Course and Potential Consequences of ADHD; (6) Review and Implications for the Individual Child.
A PhD-level psychologist co-Investigator (CM) developed a protocol for training community therapists to deliver caregiver behavior training. A training manual described the core elements of each session with sample scripts for teaching skills to caregivers and a series of recorded intervention sessions with two volunteer families (McCarty, et al., 2015). The six-session behavior training protocol incorporated key elements of evidence-based parent training programs that have demonstrated effectiveness in reducing disruptive behaviors within the broad developmental range from 5 to 12 years old (Fabiano et al., 2009; McMahon & Forehand, 2003). Session modules included: (1) Understanding ADHD and Your Child: Understanding Antecedents and Consequences of Behavior; (2) School Advocacy; (3) Praising and Ignoring Skills; (4) Giving Clear Instructions and Follow-through; (5a) Timeout and Other Consequences (5.5–9 years old) and (5b) Point System for Behavior (10–12 years old); and (6) Putting it All Together.
During training, the master’s level therapists in each study community viewed digitized recordings of sessions with the two volunteer families accessed through an asynchronous secure website, reviewed these training cases with the psychologist, and practiced the intervention with two additional volunteer families recruited from their communities. After receiving this training, they began to implement the caregiver behavior training in-person to study participants. During the training sessions, therapists gave caregivers handouts and assigned skills to practice between sessions. The PhD psychologist carried out small group supervision of therapists via telephone.
Prior to each telepsychiatry session, the therapist shared clinical data, such as vital signs, rating scales, homework, and quiz scores with the telepsychiatrist through a website, “WebCATTS;” (Vander Stoep & Myers, 2013). Near the end of each telepsychiatry session, the therapist joined the videoconference to transition the family to the caregiver behavior training component. Between intervention sessions, therapists exchanged documents with the CATTS research staff through an asynchronous web portal and participated in weekly research team meetings through videoconferencing.
At the close of the combined session, the telepsychiatrist and therapist documented their clinical decision-making in WebCATTS which then collated the information into a report that was sent to the family and the PCP. Children returned to the care of the referring PCP following study participation with a final report that contained a summary of the child’s progress and recommendations for follow-up monitoring and treatment.
We included an active teleconsultation control arm (APC model) for several reasons. Because the telepsychiatry consultation model was a well-established service in participating communities, it would not have been ethical to withhold this long-standing option from PCP’s. Further, the availability of two active arms was an attractive offering for PCP’s and families and aided recruitment in participating underserved communities (Baquet, Commiskey, Mullins, & Mishra, 2006). Finally, we anticipated that the comparison of two telehealth models would provide valuable information to stakeholders seeking to establish a telehealth service.
Children randomly assigned to the APC service delivery arm received a single consultation with a telepsychiatrist who then made treatment recommendations to the referring PCP. The PCPs implemented these recommendations at their discretion and were not restricted from implementing any interventions they deemed appropriate or referring children to other services.
We assessed four domains: parenting stress, depression, caregiver strain, and family empowerment that reflect the challenges that caregivers experience in raising children with ADHD. All domains were measured with validated instruments (Rockhill, et al., 2013). Parenting stress was measured at the baseline and 25 week assessments. Other domains were measured at all five assessment points (baseline, 4, 10, 19, 25 weeks).
The Parenting Stress Index (PSI) assesses the types of stressors that caregivers experience in caring for children with special needs (Abidin, 1995). Stressors fall into categories of objective life circumstances (e.g., my spouse and I don’t do as many things together), the caregiver’s judgment of the child’s activity level, and the caregiver’s subjective feeling of being trapped by parenting responsibilities (e.g., I feel that my child’s needs control my life). For the CATTS trial, we administered the 20 items that comprise three of the PSI subscales: role restriction (seven items assessing caregiver perception that the child’s demands are a source of frustration and control or restrict their personal freedoms), isolation (six items assessing whether caregivers feel socially isolated from their peers, relatives and other supports); spouse (seven items assessing whether caregivers perceive that they are lacking the emotional and physical support of their significant other to manage the child).
High test-retest reliability and construct and predictive validity have been demonstrated for the PSI in multiple studies (Abidin, 1995), including differentiating between parents of children with and without developmental delays (Lafiosca, 1981) and yielding elevated scores for mothers of children with ADHD (Barkley, 1988; Barkley and Cunningham, 1980).
PSI items were rated on a five point Likert-type scale from 1 (strongly disagree) to 5 (strongly agree) with higher scores indicating more stress. The total score across 20 items was used in analyses. For caregivers who did not have a spouse, we prorated scores for the 13 role restriction and isolation items to a 20-item equivalent. In the CATTS sample, the internal consistency coefficient for the baseline total PSI score was .89.
The Patient Health Questionnaire (PHQ-9) is a self-report scale that measures the severity of nine depressive symptoms experienced in the past two weeks (Kroenke & Spitzer 2002; Kroenke, Spitzer, & Williams, 2001). This scale is widely-used in clinical and research settings worldwide and has demonstrated high sensitivity and specificity for distinguishing between adults with and without depressive disorders (Kroenke and Spitzer, 2002). Sensitivity to change has been demonstrated in depression treatment studies (Löwe, Kroenke, Herzog, Grafe, 2004). Item response options range from 0 (not at all) to 3 (nearly every day). Scores ≤ 4 indicate no depression (no proposed treatment); scores 5 to 9 indicate mild depression (watchful waiting; repeat PHQ-9 at follow-up); scores 10 to 14 indicate moderate depression (consider counseling and/or pharmacotherapy), scores 15 to 19 indicate moderately severe depression (immediate initiation of psychotherapy and/or pharmacotherapy) and scores 20 to 27 indicate severe depression (immediate initiation of pharmacotherapy and, if severe impairment or poor response, expedited referral to a mental health specialist for psychotherapy and/or collaborative management). A cutoff score of 10 has been shown to have highest sensitivity and specificity for detecting major depressive disorder (Kroenke, Spitzer, Williams, 2001). In our study, baseline internal consistency (Cronbach alpha) for the PHQ-9 was .82.
The Caregiver Strain Questionnaire (CSQ) was developed within a system of care framework to assess the “demands, responsibilities, difficulties and negative psychic consequences of caring for a relative with special needs” (Brannan, Heflinger, & Bickman, 1997). Child behavior problems are proposed to be one of the strongest predictor of caregiver strain. The CSQ developers conceived of strain as resulting from the child’s emotional or behavioral disorders as filtered through the family’s perceptions and resources (Bickman et al. 1995; Brannan, et al., 1997). Thus, strain encompasses both the outwardly observable impact, as well as the less observable emotional impact of caring for a family member with special needs (Brannan & Heflinger, 2002; Sales, Greeno, Shear, & Anderson, 2004).
Caregiver strain is strongly associated with the severity of child symptoms (Brannan and Heflinger 2001; Bussing et al. 2003; Farmer, Burns, Angold, & Costello, 1996; Sales, Greeno, 2004), affects the family’s ability to mobilize supports, including mental health services (Angold et al., 1998; Brannan and Heflinger 2005; Brannan, Hefflinger, & Foster, 2003), and increases when families experience more barriers to care (Brannan and Helflinger, 2006).
The CSQ has 21-items and two subscales: objective strain (e.g., missing work due to your child’s problems) and subjective strain (e.g., feeling sad as a result of your child’s problems). Good convergent validity has been demonstrated with regard to other indicators of caregiver well-being (Khanna et al., 2012). CSQ items have five response options ranging from 1 not at all a problem to 5 very much a problem reflecting how much of a problem each potential source of objective or subjective strain the caregiver experienced during the past 6 months due to the child’s problems. Higher scores indicated greater strain. In the CATTS trial, the internal consistency coefficient for the baseline CGSQ was .92.
The Family Empowerment Scale (FES) has 34 items that indicate caregivers’ understanding of their child’s needs and their ability to advocate for their child with mental health problems (Koren, DeChillo, & Friesen, 1992). As with the CSQ, the FES was developed within the system of care framework as a tool to assess the effectiveness of interventions or programs designed to increase the empowerment of parent or other family caregivers. The FES has three subscales: the family subscale queries caregivers about their ability to handle the child within the family context; the services subscale queries caregivers about their sense of competence in working with professionals; the community subscale includes items about caregivers’ ability to advocate for their children. The five response options range from 1 (never) to 5 (very often), with a higher score reflecting greater empowerment. The subscales are highly correlated (Koren, et al., 1992), indicating a core component of the caregiver’s sense of empowerment (Florian and Elad, 1998).
Psychometric testing of the FES has demonstrated high validity and reliability in samples of parents of children with serious emotional disturbance (Koren, et al., 1992) and ADHD (Singh et al., 1995) and covariation with child functioning over the course of mental health treatment (Resendez, Quist, & Matshazi, 2000). In the CATTS trial, the internal consistency coefficient for the baseline FES was .93.
Child ADHD symptoms were assessed with caregiver ratings on the Vanderbilt ADHD Diagnostic Parent Rating Scale (VADRS-Parent; Jellinek, Patel, & Froehle, 2002). The scale is based upon DSM-IV criteria for ADHD (American Psychiatric Association, 1994). Subscales include inattention (9 items), hyperactivity/impulsivity (9 items), oppositional defiant behaviors (8 items), and role performance related to academic, classroom and interpersonal functioning (8 items). Items are rated on a 0 (“never”) to 3 (“very often”) Likert scale. The sum of the 18 inattention and hyperactivity/impulsivity items yields a total ADHD scale score. The VADRS-Parent scale has demonstrated solid psychometric properties (Wolraich et al., 2003).
In the CATTS trial, consistency coefficients for baseline ADHD and ODD symptom scores were high, ranging from .84 to .89, and for performance was moderate (.53). Concurrent validity of the item total of the VADRS-Parent and the ADHD symptom scores on the CDISC-IV was high (r = .79). Correlations between the total ADHD scores and the two component scores were very high (r > 0.90).
At baseline, participants in the two arms had similar proportions meeting VADRS-Parent diagnostic criteria for inattention (intervention versus control: 83% versus 82%), hyperactivity (67% versus 58%), combined ADHD (60% versus 52%), and ODD (61% versus 51%). Children in both study arms improved over time. However, compared with children in the augmented primary care arm, at 25 weeks lower proportions of children in the CATTS intervention arm continued to meet diagnostic criteria for inattention (23% versus 48%), hyperactivity (16% versus 31%), combined ADHD (12% versus 26%), and ODD (16% versus 26%) scales” (Myers, et al., 2015; page 269).
We conducted intention-to-treat analyses, including all participants, irrespective of the number of treatment sessions they completed. Besides being the analytic approach that we had proposed prior to conducting the study, of children assigned to the CATTS arm, 85% completed five or six sessions of the six session program. Furthermore, when we compared the 15% of children exposed to 2–4 sessions with the 85% children with 5–6 sessions, we found no statistically significant between-group differences in child symptom or functioning scores at baseline, 19-weeks or 25-weeks.
We compared baseline demographics, comorbid conditions, and caregiver distress scores between the two study arms, using two-sample t-tests for continuous variables and Chi-square tests for categorical variables. We adjusted regression analyses by baseline characteristics that were found to differ significantly between the two arms. For each of the four caregiver scales, we discarded a score from analyses if the caregiver answered fewer than 75% of the items contributing to the total scale score. For measures with 0–25% items missing, we assigned to the missing items the mean value of the items the caregiver completed and calculated the total score accordingly.
To test the first hypothesis that caregivers of children assigned to the CATTS service-delivery condition would experience greater reductions in distress, we compared the trajectories of caregiver outcomes in the two arms using longitudinal data analyses. Given that multiple assessments were made within individual participants, and participants were nested within seven study sites, we chose multilevel mixed effects regression models to evaluate primary outcomes. Specifically, subject and site-specific random effects were initially included in the regression models to account for the within-subject and within-site correlations. The site random effect was dropped from the final models because its variance component was nearly zero when the subject random effect was included. In addition, we used robust standard error estimates for the inference. The mixed effects model approach has the advantage of utilizing all available data even if a participant missed assessments. In each regression model we included intervention status (CATTS vs APC), time (week 0, 4, 10, 19, and 25), and an intervention-by-time interaction term. Time was modelled as a discrete predictor. Our evaluation of the effectiveness of the CATTS intervention was based on the coefficient of the interaction terms, which estimated how absolute differences in scores from baseline to each post-baseline follow-up assessment differed across the CATTS intervention and comparison arms. We then visually depicted the outcomes for caregivers of children in the intervention and comparison conditions by plotting trajectories of mean caregiver distress scale scores and point-wise 95% confidence intervals for baseline and 4, 10, 19, 25 weeks post-randomization assessments. We calculated a Cohen’s d (Cohen, 1988) corrected for uneven groups to estimate effect sizes for baseline to 25-week changes in caregiver outcome scores.
To test the second hypothesis that the effects of the CATTS intervention on caregiver distress may be partially mediated by improvements in child clinical outcomes, we operationalized potential mediators as improvements in children’s clinical outcomes as changes from baseline to 19-week follow-up in children’s inattention symptoms, hyperactivity/impulsivity symptoms, oppositional-defiant symptoms, and role performance scores. We modeled each of the 25-week caregiver outcomes separately. Mediation models included the baseline scores on the relevant caregiver scale as covariates. Following the MacKinnon (2008, 2011) framework, we tested for mediation using path analysis and a structural equation modeling (SEM) approach.
McKinnon requires that two conditions be met for establishing mediation (McKinnon & Fairchild, 2009): 1) the intervention is significantly associated with mediator, and 2) the mediator is significantly associated with the outcome while controlling for intervention status. In a previously published paper, we established that the CATTS telehealth service delivery model was associated with greater improvement in each of the child outcomes that we incorporated into the mediation model (Myers, et al., 2015). Results of the test of the first hypothesis (described above) established whether the CATTS service delivery model was associated with an improvement in caregiver outcomes. We then estimated and tested whether improvement in child symptoms, individually and combined, significantly mediated this intervention effect. We used bootstrap estimates and bootstrap confidence intervals on the key coefficients to test the mediated effect of improvements in child symptoms and performance. We considered an intervention effect to be significantly mediated by a child outcome if its associated bootstrap confidence limits did not overlap with zero. We also estimated the percent of the intervention effects mediated by improvements in all the child outcomes combined. Mediation analyses were conducted using the SEM module in Stata 12.1 software.
Baseline characteristics of the children and caregivers in the two study sample are summarized in Table 1. Eighty-eight (n=88) PCPs referred 223 children, predominantly boys (n=163; 73.0%), with mean age 9.25 (± 2.0) years, of European/White ancestry (n=204; 91.5%).
On the CDISC-IV, 25% (n=55) child participants met criteria for ADHD alone. Comorbidity was common: 75% (n=168) had at least one comorbid disorder; 28% (n=62) met criteria for both ODD and AD; 41% (n=93) met criteria for ODD, and 6% (n=13) met criteria for AD.
Based on the VADRS-Parent symptom ratings, 56.1% (n=125) of children met VADRS-Parent criteria for combined type ADHD; 82.5% (n= 184) met symptom criteria for inattention; 62.3% (n = 139) met symptom criteria for hyperactivity/impulsivity; and 56.1% (n =125) met criteria for ODD.
The demographic and clinical characteristics of child participants in the CATTS telehealth service delivery and APC arms were comparable at baseline with one exception. A higher proportion of children in the CATTS service delivery arm had ADHD with both comorbid disorders, and, therefore, we adjusted for comorbidity status in the regression analyses of child outcomes.
Caregivers were predominantly biological mothers (n=173; 77.6%), married or cohabitating (n=159; 71.6%), with some post high school education (n=145; 68.1%), and a median annual family income in the range of $35,000 to $75,000. Caregiver demographic characteristics at baseline were well balanced across the two service delivery arms; however, on clinical measures, caregivers in the CATTS service model showed significantly higher baseline PSI score. We adjusted regression analyses by baseline PSI scores. Baseline scores on the four caregiver scales were highly correlated. The bivariate correlation coefficient was particularly high between baseline PSI and CSQ scores (r=.67, p < .001).
Of a possible six combined intervention sessions, participants randomized to the CATTS service delivery model (n=111) attended an average of 5.2 (range 0 to 6) telepsychiatry sessions and an average of 5.1 (range 0 to 6 sessions) caregiver behavior training sessions.
Figure 1 graphically depicts mean scores at baseline, 4, 10, 19, and 25-weeks for the four scales measuring distress of caregivers in each of the two study arms. Summarized in Table 2 are differences between the CATTS service model and the APC model in the longitudinal course of caregiver outcomes from baseline to 25-week assessment, as determined by the multilevel mixed effects regression models. Caregivers in both service models demonstrated improvements over the course of the study. Caregivers who participated in CATTS service delivery showed significantly greater reductions from baseline to 25 weeks in PSI (f3 = −4.59, 95% CI = [−7.87, −1.31], p <.01), PHQ-9 (f3 = −1.41, 95% CI = − [2.74, −0.08], p <.05), and CSQ (f3 = − 5.41, 95% CI = [−8.58, −2.24], p <.001), and significantly greater increases in FES (f3 = 6.69, 95% CI = [2.32, 11.06], p <.01). Effect sizes for 0 to 25-week changes in caregiver distress measures based on Cohen’s d corrected for uneven groups were estimated at 0.59 (moderate) for PSI change, 0.45 (small to moderate) for CSQ change, and −0.44 (small to moderate) FES change and 0.27 (small) for PHQ-9 change.
Bivariate correlation coefficients between 0–19-week improvements for each pair of child symptom and performance outcomes were statistically significant, ranging from −0.38 to 0.66. As shown in Table 3, a substantial proportion of the CATTS service model effects on caregiver distress occurred through the mechanism of reducing children’s symptoms and improving their role performance. The proportions of intervention effects on 25-week caregiver outcomes that were mediated via baseline to 19-week improvements in child symptoms and performance were: parenting stress (PSI; 41%), caregiver depression (PHQ-9; 48%), caregiver strain (CGSQ; 43%), and family empowerment (FES; 26%). For parenting stress and caregiver strain outcomes, total indirect effects of improvements in all child scales combined were statistically significant. Taking improvements in the four child clinical outcomes independently, only reduction in ODD symptoms made a statistically significant contribution towards the intervention effect on caregiver strain (See Table 3).
This paper reports the results of a randomized controlled trial comparing the effects of two service delivery models on reducing distress of caregivers of children with ADHD living in geographically underserved communities. The children and caregivers assigned to the CATTS service delivery model received multiple sessions of multi-modal treatment delivered remotely via a hybrid service model using videoconferencing with a psychiatrist and in-person services from a community therapist who was trained and supervised remotely using telehealth technologies. Families assigned to the APC model received treatment in primary care augmented by a single teleconsultation session. Over time, caregivers in both service models reported significantly decreased levels of distress and depression symptoms, and increased levels of family empowerment. Effects were significantly greater for caregivers in the CATTS service delivery model. Further, combined child symptom improvements mediated reductions in parenting stress and caregiver strain, and improvements in caregiver strain were significantly mediated by treatment-induced decreases in child ODD symptoms.
Prior studies have shown treatment engagement, continuity, and adherence to be low for families of children with ADHD (Adler & Neirenberg, 2010; Bussing et al., 2003). In the CATTS trial, the majority of families who were assigned to the CATTS service delivery arm attended the full six sessions of combined treatment, reflecting high acceptability and engagement (Myers, et al., 2015; Vander Stoep & Myers, 2013). This finding portends well for the ability of telehealth technologies to extend the reach of empirically-supported psychotherapeutic treatments to children and caregivers in geographically underserved communities. It also suggests next steps to build on the few small studies supporting the delivery of caregiver behavior training through synchronous videoconferencing (Reese, et al., 2012; Tse, McCarty, Vander Stoep, & Myers, 2015; Xie, et al., 2013).
In each of the six CATTS sessions, we used synchronous and asynchronous telehealth technologies to deliver evidence-based treatments directly to children with ADHD and their caregivers. This hybrid model parsimoniously utilized asynchronous telehealth technologies to enhance the skills of local therapists in evidence-based behavioral interventions who could then remain an enduring resource for their communities. The model used synchronous technologies to deliver the scarce resource of expert psychopharmacology from child psychiatrists. The primary target of the CATTS service delivery model was reduction in child ADHD-related symptoms. While the CATTS model did not incorporate interventions that directly targeted caregiver depression, stress, strain or empowerment, it was effective in improving these outcomes. Our findings are consistent with results of studies conducted in-person showing beneficial effects of behavior management training for caregivers of children with ADHD (Anastopoulos et al., 1993; Chronis, Jones, & Raggi, 2006; Owens et al., 2003; Pelham, Wheeler, & Chronis, 1998; Pisterman et al., 1992; Wells et al., 2000) and the benefits for both caregivers and children of providing combined interventions, particularly for children with ADHD and a comorbid disorder (Jensen et al., 2001; MTA Cooperative Group, 1999). These results suggest directions for future research that tests the feasibility and effectiveness of delivering parent-child interventions through telehealth technologies and assesses caregiver implementation of behavior training skills (Chronis et al., 2006; Chronis-Tuscano et al., 2013; Comer at al., 2014).
To provide context for interpreting the meaning of the changes in the caregiver outcomes scales observed in the CATTS trial, we evaluated caregiver distress start and endpoints in light of findings from prior research reporting mean scale scores in normative samples or in outcome studies. The baseline to 25-week decrease in mean item scores from M= 2.55 to M= 2.30 for the 20-item PSI among caregivers participating in the CATTS service delivery model reflected a greater percent change (9.8%) than the 14-month decreases from M= 2.28 to M= 2.12 (7.0%) reported for participants in the combined treatment group in the MTA study (Wells et al., 2000) on the 12-item (Short Form) Parenting Distress (PD) subscale of the PSI (PSI-PD Short Form; Abidin, 1990). More recently, Kolko and colleagues (2014) reported 6-month mean item decreases in the PSI-PD from M= 2.05 to M= 1.92 (6.3%) for participants in a pediatric collaborative care intervention for children with behavioral disorders.
Baseline PHQ-9 mean total scores for caregivers receiving the CATTS service model of M= 7.1 indicated, on average, mild depression. At 25-weeks the mean PHQ-9 score for these caregivers had fallen to M= 4.0, reflecting no depression (Kroenke and Spitzer, 2002).
The CSQ was developed for use in system of care research to characterize caregivers of children with serious emotional disturbance (SED). At baseline, mean item CSQ scores for caregivers in the CATTS service model, M= 2.4, were comparable to those endorsed by families of children with SED in the Fort Bragg study, M= 2.5, reflecting “a little to some” strain” (Brannan, et al., 1997). At 25-week assessment in the CATTS group, the mean item score had dropped to M= 1.8, a significant change, unlike families in the large Fort Bragg study who did not show a significant drop in caregiver strain after participating in a continuum of care service delivery model (Bickman, Lambert, Andrade, Penaloza, 2000). The overall 25.0% decrease from M= 2.4 to M= 1.8 in mean item CSQ scores over 25 weeks for caregivers in the CATTS service model compares favorably to the 20.8% decrease from M= 2.4 to M= 1.9 over 32 weeks for parents enrolled in a publicly funded mental health program (Accurso, Garland, Haines-Schlagel, Brookman-Frazee, & Baker-Ericzen, 2015) and to a small randomized trial that found no benefit for parents enrolled in a parent-to-parent support program for school children designated as having serious emotional disturbance (Kutash, Duchnowski, Green, & Ferron, 2011).
FES mean total baseline scores for caregivers in the CATTS group (M= 115.3) were more similar to the scores of caregivers in the original scale development study (Koren, et al., 1992) who were not involved in any advocacy activities (M= 125.9) than those of caregivers who were engaged in advocacy activities (M= 144.8). The increment of improvement in CATTS participants by 25-weeks to M= 127.9, while significant, did not bring the total mean score to the level observed among caregivers of children with SED who were recruited from parent advocacy organizations. Members of advocacy organizations would be expected to report higher empowerment scores than parents living in underserved communities raising similarly affected children. The final mean total FES score for the CATTS caregivers was comparable to the mean score for parents who received Wraparound services in a public mental health system for six months (M= 132.8) (Bruns, Pullmann, Sather, Brinson, & Ramey, 2015.
Taken together, this evaluation of caregiver outcomes in the CATTS group vis a vis prior studies that used the four scales revealed that caregivers who received services via the CATTS service delivery model experienced statistically and clinically significant improvements in their distress and empowerment.
The CATTS trial has implications beyond the effectiveness of telehealth service delivery. Our results contribute to the small body of research showing the beneficial effects for caregivers of improvement in children’s ADHD symptoms (Accurso, Garland, Haine-Schlagel, Brookman-Frazee, & Baker-Ericzen, 2015; Heath et al., 2015), and the study is the first to formally test these indirect effects. The findings suggest that effective treatment of child ADHD may counter the well-documented detrimental effects of child ADHD on caregiver mental health (Accurso, et al., 2015).
Future research should parse the differential benefits for caregivers of improvements in child ADHD symptoms versus ODD behaviors as our findings indicate that decreased ODD behaviors have particular salience for ameliorating the consequences of caring for a child with ADHD and ODD. As evidence-based interventions for ODD focus on behavior training, rather than pharmacotherapy, our results support the use of telehealth technologies in providing caregiver behavior training.
While not statistically significant, the mediational effect of improvement in child inattentive symptoms on CATTS-induced reduction in parenting stress was quite strong, and appears clinically relevant. When a child has difficulty with focus, forgetfulness, and disorganization during daily tasks, his/her caregiver is required to devote considerable time and resources to instructing and monitoring. Inattentive symptoms often attract less attention or concern than hyperactive or impulsive symptoms or ODD behaviors, but our mediational findings suggest the importance of adequately managing inattentive symptoms.
Over 40% of intervention effects on parenting stress, caregiver strain, and family empowerment were mediated by improvements in child symptoms and functioning, but a large proportion of intervention effects on caregiver distress were not. The caregiver training component of the CATTS intervention focused primarily on helping caregivers to manage child behaviors and to advocate for their children in school settings. Thus, it is plausible that the behavior training itself had a direct effect on caregiver distress through the acquisition of knowledge about ADHD, improved parent-child relationships, and/or perceived social support of the caregiver from the therapist. Such an interpretation is consistent with a recent meta-analysis finding that behavioral interventions have beneficial effects on parenting children with ADHD in studies that did not target parent functioning as a primary intervention goal (Daley et al, 2014). Future research is recommended that is designed to further clarify the effect of behavior training on caregiver distress and to elucidate other child, caregiver, and family-level factors that are potential mediators of caregiver outcomes.
Our study findings must be evaluated in light of several limitations. First, is the choice of comparison condition. Our APC comparison group set a high bar compared to treatment as usual in the community that is commonly used as the comparator in health services and intervention studies. Likely, the increment of improvement in outcomes rendered through the CATTS service delivery model would have been wider had treatment as usual served as the comparison. The benefit of the current approach is the demonstration of the added value for caregivers of providing a short term expert intervention over a single consultation to PCPs. Second, the broad referral base of 88 PCPs across seven sites with varied documentation and tracking methods precluded an accurate quantification of the additional services that families in either group may have received from mental health providers, schools, PCP’s or other community agencies over the course of the study. We did not document other service use on the part of participants. Had this information been available and incorporated into the analyses, it might have affected the interpretation of the contribution to parent well-being of services delivered via the CATTS model. The likelihood is quite low that families in these underserved communities had access to alternative behavioral health services where empirically-supported parent training was delivered with fidelity.
Third, the trial was designed to test the effectiveness of a telehealth service delivery model with a two-component, guideline-based intervention and, therefore, we cannot evaluate the separate contributions of the telepsychiatry and the caregiver behavior training components. The high rate of treatment adherence and completion of intervention sessions yielded insufficient variability in intervention content or dosage to perform post-hoc analyses to parse the relative benefits of the two treatment components. The upside of the current approach is the successful demonstration of use of a hybrid model that delivered both components of guideline-based care (medication and behavior training) in underserved locales.
Fourth, our 25 week assessment provided a short term follow-up half way between the last CATTS session (22 weeks) and the first scheduled PCP appointment (27-weeks) for children in the CATTS condition. The Multi-model Treatment of ADHD (MTA) study demonstrated that intervention groups started to regress to the mean within 24 months of treatment completion (The MTA Cooperative Group, 2004). We have insufficient evidence to suggest a different course for treatment offered through telehealth. Fifth, results may not be generalizable across all populations, as our participants represented a relatively homogeneous sample recruited from underserved communities in the Pacific Northwest (Table 1). Finally, the constructs used in the outcome and mediation analyses were all measured with caregiver-report instruments. Caregiver ratings of their children’s status may not be independent of the current emotional status of the caregiver and do not fully capture functioning of children in settings such as school or extracurricular activities. All parent distress measures were highly correlated. Our study findings did not point towards any differentiation among these measures.
Overall, the CATTS trial contributes new evidence to the emerging body of knowledge that supports the use of telehealth technologies to deliver evidence-based services to children with mental and behavioral disorders. This research highlights the process of how service delivery innovations work and is the first study to show that ADHD intervention leads to changes in caregiver outcomes that are mediated through changes in child symptoms and performance. This new evidence will spark conversations among stakeholders from advocacy organizations and primary care, mental health, and health policy sectors who are working to rectify disparities in access to evidence-based mental health care.
The CATTS study was supported by funding from the U.S. National Institute of Mental Health [1R01MH081997 and R01MH081997-04S1]. The authors declare that they have no conflict of interest.
Funding The design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation of the manuscript was supported by funding from the U.S. National Institute of Mental Health [1R01MH081997 and R01MH081997-04S1]. Funding for pilot data collection was provided by the University of Washington Institute of Translational Health Sciences, Small Pilot Project Grant program ; the University of Washington Royalty Research Fund program [65-4020]; and the American Academy of Child and Adolescent Psychiatry Abramson Fund . The CATTS trial is registered with Clinical Trials: http://clinicaltrials.gov/show/NCT00830700.
All procedures performed involving human participants were in accordance with the ethical standards of the Institutional Review Board of Seattle Children’s Research Institute and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.
Contributors. We would like to thank: Research Coordinators – Heather Violette, PhD and Kelly Thompson, MSW; CATTS Therapists – Joan Arrasmith, MEd, Laura Aviles, MSW, Aloma Burrows, MEd, Corrie Piper MSFT, Sarah Rundell, MA, Heather Violette, PhD, Lila Waldron, MS; Telepsychiatrists –Lynda Lee Carlisle, MD, Kelley Fellman, MD, Daniel Ferber, MD, William French, MD, Michael Kisicki, MD, Ian Kodish, MD, PhD, Carol Rockhill, MD, PhD, MPH; Research Staff – Sarah Grover, Gina Kim, John Geyer, Caitlin Lobdell, Jane Koltracht, Yuet Juhn Tse; Community Champions- Lynn O’Leary MD & Blaine Tolby MD, PhD; and the Families and Primary Care Providers of Washington and Oregon who participated in the CATTS Trial.
Presentations Earlier versions of this paper were presented at the 16th Biennial Scientific Meeting of the International Society for Research in Child and Adolescent Psychopathology in Leuven, Belgium, June 13–16, 2013 and at the 60th Annual Meeting of the American Academy of Child and Adolescent Psychiatry in Orlando, FL, October 22–27, 2013. This research was supported by the National Institute of Mental Health [1R01MH081997 and R01MH081997-04S1]. The CATTS trial is registered with Clinical Trials: http://clinicaltrials.gov/show/NCT00830700.
Dr. Ann Vander Stoep, Associate Professor, Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Department of Epidemiology, University of Washington School of Public Health, Seattle Children’s Research Institute, (206) 543-1538.
Dr. Carolyn A. McCarty, Research Associate Professor, Department of Pediatrics, University of Washington School of Medicine, Seattle Children’s Research Institute, Seattle, WA 98145-5005.
Dr. Chuan Zhou, Research Associate Professor, Department of Pediatrics, University of Washington School of Medicine, Adjunct Research Associate Professor, Department of Health Services, School of Pubic Health, Seattle Children’s Research Institute, Seattle, WA 98145.
Dr. Carol M. Rockhill, Associate Professor, Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Medical Director, Outpatient Psychiatry and Behavioral Medicine, Seattle Children’s Hospital, MS OA.5.154, Seattle Children’s Research Institute, PO Box 5371, 4800 Sand Point Way, NE, Seattle WA 98105.
Dr. Erin N. Schoenfelder, Assistant Professor, Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle Children’s Hospital, MS OA.5.154, Seattle Children’s Research Institute, PO Box 5371, 4800 Sand Point Way, NE, Seattle WA 98105.
Dr. Kathleen Myers, Professor, Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Director, Telemental Health Service, Seattle Children’s Hospital, MS OA.5.154, Seattle Children’s Research Institute, PO Box 5371, 4800 Sand Point Way, NE, Seattle WA 98105, (206) 987-1663.