Analyses were conducted to compare symptom and functioning outcomes between the treatment and control conditions using parent and teacher ratings on the DBD and IRS, as well as teacher ratings on the CPS. For all participants, there were six measurement occasions, beginning with the intake assessment and then once each month from January through May. Descriptive statistics for each outcome measure over each measurement occasion are provided in . All outcome data were analyzed using hierarchical linear modeling (HLM) in SAS 9.1 (proc mixed procedure). Time points were nested within students, and the student factor was modeled as random. Time was modeled as a linear trend; the quadratic trend of time was not statistically significant at the .05 level for any outcome. In each case, we investigated Condition × Time interactions, because a significant interaction would indicate the treatment and control groups changed differently over time. Additionally, a potential Condition × Time × Site interaction was investigated in all analyses to assess the possibility of differential impact of the CHP at the two participating schools, but only significant findings are reported. Although site would conceptually be a random factor, it was modeled as a fixed factor because there were only two sites. Similarly, cohort effects were also evaluated and revealed no significant differences; accordingly, cohort was not included in the outcome analyses.
Descriptive Statistics for Outcome Variables Over Six Measurement Occasions
We examined the potential for a Condition × Time × Medication Status interaction; however, in no cases did medication status significantly moderate outcomes. There was a significant main effect of medication use on several of the variables including parent ratings of inattention on the DBD and parent ratings of peer relationships, parent–child relationships, family impairment, and overall severity on the IRS. In each case, the severity of the impairment and symptoms was greater for participants taking medication. If medication status was not balanced across conditions, it could potentially bias the estimation of the treatment effects; however, rates of medication use were equivalent at baseline (50% control and 48% treatment) and similar throughout the study (67% control and 52% treatment).
We analyzed teacher satisfaction data, which were collected at the end of both school years, using analysis of variance (ANOVA) techniques in SPSS 15.0. We also analyzed end-of-year student report card data using a life table analysis because previous research suggests that psychosocial interventions such as those used in the CHP may be most helpful in preventing serious academic decline, rather than improving student performance relative to baseline performance (Schultz et al., 2009
In order to evaluate the effect of treatment on ADHD-related impairment, similar analyses were conducted using parent and teacher ratings on the IRS. There were no significant Condition × Time interactions for the parent ratings of social and academic impairment on the IRS, and no significant changes were noted in the teacher ratings of participants’ social impairment. However, analyses of teacher ratings of participants’ academic progress revealed a significant Condition × Time interaction in the predicted direction, t(213) = −2.04, p = .04. Specifically, participants in the treatment condition were rated as appreciably more impaired at intake, but these ratings declined (d = −0.45) to a level below that of participants in the control condition (d = 0.10) by the end of the school year, suggesting that teachers observed less academic impairment in the treatment group as compared to the control group.
Teacher ratings on the CPS were also analyzed using HLM (proc mixed
procedure) in SAS 9.1. When all core course teacher ratings were averaged together, there were no significant Condition × Time interactions. Since previous studies have documented high rates of inconsistency between middle school teacher ratings of student symptoms and impairments (e.g., Evans, Allen, Moore, & Strauss, 2005
), the teacher ratings were also analyzed separately by class. The Condition × Time interaction was statistically significant for language arts, t
(188) = −3.09, p
< 0.01, and social studies, t
(179) = −2.26, p
= 0.02. In both cases, impairment in classroom performance decreased over time for the treatment group (d
= −0.37 and −0.23, respectively), but increased for the control group (d
= 0.40 and 0.29, respectively). A similar pattern emerged for math teachers, although the effect was not statistically significant. Although there was no statistically significant Condition × Time interaction in science teacher ratings, there was a three-way Condition × Time × Site interaction, t
(166) = 2.02, p
= 0.04. Specifically, impairment increased for the control group and decreased for the treatment group at one site, while the opposite was true at the other site, washing out the overall Condition x Time interaction. Additionally, in social studies there was a three-way Condition x Time x Site interaction, t
(179) = 2.40, p
= 0.02, suggesting that the significant Condition × Time interaction was mostly due to an interaction at one site (treatment d
= −0.35, control d
= .76) and not the other (treatment d
= −0.12, control d
Average core course GPAs were used as an additional outcome measure of academic functioning. In previous research, we have noted a curvilinear trend in GPA, characterized by declining grades in late fall and spring followed by partial recovery late in the year (Evans, Langberg, et al., 2005
). Additionally, the second semester grading periods corresponded to the after-school intervention program, so the trend was predicted to change for the treatment group during this time. Thus, we analyzed a two-part trend over time by modeling grade as a function of an intercept (predicted grade halfway through the year), experimental condition, linear trend of time in the first semester, linear trend of time in the second semester, condition by first semester, and condition by second semester. The results of this analysis for core GPA suggested that there were no significant differences between the trends of the treatment and control groups during the first semester, t
(220) = 1.04, p
= .30, or the second semester, t
(220) = 0.93, p
= .35. Follow-up analysis of the constituent classes revealed that differences between treatment and control over time only reached significance in one instance; specifically, treatment participants appeared to outperform control participants in the second semester in their math classes, t
(225) = 2.51, p
= 0.01. The magnitude of improvement over first semester performance for the treatment group was small (d
= .14), as most of the apparent benefit of treatment was relative to a performance decline in the control group (d
Similarly, previous research has indicated that even in cases when statistically significant benefits are not found using continuous measures of GPA, differences may exist in the proportion of participants who experience failure events over time (Schultz et al., 2009
). In the present study, the proportion of students who experienced failing grading periods—defined as overall GPA dropping below 1.0—at some point in the school year was 32% in the treatment group and 39% in the control group. Although these differences are in the predicted direction, life table analysis revealed that the trend did not reach statistical significance (Z
= 0.18, p
Changes in ADHD symptoms were assessed over time using parent and teacher ratings on the DBD. The estimated intercepts and slopes of the treatment and control groups for these and all rating scale outcomes are reported in . The results suggest that there were no significant Condition × Time interactions for teacher ratings of either inattention or hyperactivity–impulsivity. On parent ratings, a significant Condition × Time interaction was found on ratings of hyperactivity–impulsivity, t(163)= −3.37, p < .01, but not for ratings of inattention. Specifically, parent ratings of hyperactivity–impulsivity remained stable for participants in the control condition (d = −0.16), while parent ratings of participants in the treatment condition indicated a decline in symptoms over time (d = −1.03). In follow-up analyses of the parent ratings, a significant three-way interaction was noted when school site was entered into the model (Condition × Time × Site),t (163) = 2.32, p = .02. Specifically, improvement in ratings for treatment participants at one of the schools (d = −1.76) far exceeded improvement for treatment participants at the other (d = −0.40), suggesting differential impact of the CHP as a function of unspecified contextual factors.
Estimated Intercepts and Slopes When Modeling Change in Parent and Teacher Ratings of Participant Performance Over Time
The third question being addressed in this study has to do with the degree of teacher satisfaction with the program. All core course teachers who taught the treatment participants (n = 89) received the Teacher Evaluation of Services at the end of school year. The return rate was 76% (n = 68). The mean response to all items was close to three (range = 2.8 to 3.1), which was typically anchored by mostly satisfied. Based on ANOVA analysis, there were no significant site differences on any of the items; however, there were significant differences between the 2 years of the study, with teachers’ satisfaction with the program significantly greater during the second year than the first on four of the seven items. Specifically, a main effect of time was noted for the items related to whether teachers would recommend the CHP to friends (Item 3; F = 8.098, p = .006, η2 = 0.11), satisfaction with the amount of help students received (Item 4; F = 10.528, p = .002, η2 = 0.14), satisfaction with interactions with the CHP staff (Item 6; F = 4.667, p = .004, η2 = 0.12), and on the item related to the responsiveness of the CHP staff (Item 7; F = 2.259, p = .011, η2 = 0.10). In all instances, the responses suggested that there were significant improvements in teacher perception of the program from year 1 to year 2. The mean scores for the other three items improved from the first to the second year, but not to a statistically significant degree.
Finally, we examined the number of parents who chose to participate in family treatment sessions provided by our research clinician or obtain assistance entering other services from outside agencies. Out of the 31 participants in the treatment condition who completed the FCU, only two began individual sessions with the clinician to target specific behavioral concerns, and only one family requested assistance obtaining services in the community based on the results of their FCU. The first two families requested extra assistance addressing their child’s behaviors at home, and the latter family requested more intensive family therapy to address concerns in the marital relationship.