PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Consult Clin Psychol. Author manuscript; available in PMC 2010 February 23.
Published in final edited form as:
PMCID: PMC2827251
NIHMSID: NIHMS170321

A Multicenter, Randomized Clinical Trial of a Cognitive Remediation Program for Childhood Survivors of a Pediatric Malignancy

Abstract

Survivors of childhood cancer whose malignancy and/or treatment involved the central nervous system may demonstrate a consistent pattern of neurocognitive deficits. The present study evaluated a randomized clinical trial of the Cognitive Remediation Program (CRP). Participants were 6- to 17-year-old survivors of childhood cancer (N = 161; 35% female, 18% Hispanic, 10% African American, 64% Caucasian, 8% other) who were at least 1 year off treatment and who manifested an attentional deficit. They were enrolled at 7 sites nationwide. Two thirds of the participants were randomly assigned to cognitive remediation. All participants were assessed using a battery of academic achievement/neurocognitive tests and parent/teacher measures of attention. The CRP resulted in parent report of improved attention and statistically significant increases in academic achievement. Effect sizes were modest but were comparable with those for other clinical trials of brain injury rehabilitation and for psychological interventions in general. The CRP is presented as a potentially beneficial treatment for many survivors of pediatric cancer. Long-term clinical significance remains unproven. Further work is needed to improve effect sizes and treatment compliance and to address the needs of other populations with pediatric brain injury.

Keywords: brain injury rehabilitation, childhood cancer/treatment, clinical interventions

Cure rates for many childhood cancers have been positively influenced by the introduction of central nervous system (CNS) prophylaxis (Margolin & Poplack, 1997) and by improvements in tumor resection techniques, reduced cranial radiation therapy, and refinements in chemotherapy (Heideman, Packer, Albright, Freeman, & Rorke, 1997). Children who survive the most common pediatric tumors, however, are at risk for declines in cognitive functioning and psychosocial deficits that continue into adulthood (Hoppe-Hirsch et al., 1995; Mulhern, Merchant, Gajjar, Reddick, & Kun, 2004; Ris, Packer, Goldwein, Jones-Wallace, & Boyett, 2001). In fact, there is general consensus that not only do CNS treatments for leukemias and brain tumors significantly affect neuropsychological development but that there is a consistent pattern of deficits involving vigilance attention, working memory, spatial awareness, processing speed, and self-monitoring (Butler, Hill, Steinherz, Meyers, & Finlay, 1994; Butler, Kerr, & Marchand, 1999; Mulhern & Butler, 2004, 2006; Spiegler, Bouffet, Greenberg, Rutka, & Mabbott, 2004). As a result, school performance is often adversely affected, especially for mathematics.

Most research on the effectiveness of cognitive rehabilitation, also termed cognitive remediation, has been directed toward improving neuropsychological and behavioral performance with adults. In two comprehensive reviews of evidence-based studies on brain injury rehabilitation, Cicerone and coauthors (Cicerone et al., 2000, 2005) determined that remediation, although typically characterized by small-to-moderate treatment improvements, is an effective therapeutic process. The National Institutes of Health published a consensus statement (National Institute of Neurological Disoders and Stroke, 2002) that contained the same conclusion.

Few randomized Phase 3 clinical trials have been conducted among young people with a brain injury, and most of these trials have been in the area of traumatic brain injury. One study compared remediation with a control condition, and the results indicated significant improvement on neuropsychological tests of attention and memory (Hooft et al., 2005). Measures of treatment generalization, such as academic achievement, were not administered. A clinical trial that compared two rehabilitation treatments for children and adolescents who had sustained a traumatic brain injury also provided encouraging results (Braga, Da Paz, & Ylvisaker, 2005).

Butler (1998) began directing traditional brain injury rehabilitation techniques toward children who had been treated for leukemia and brain tumors, because, in these populations, CNS insults result in neuropsychological impairment similar to that for other types of brain injuries. An initial case study documented improvement on a test of attention under conditions of vigilance following treatment of a child who had received cranial irradiation. As a result of this study, a combination of therapies was developed into a programmatic treatment approach entitled the Cognitive Remediation Program (CRP). Twenty-five 2-hr sessions were prescribed, and expected goals were determined. The CRP was pilot tested on 31 off-treatment survivors of pediatric cancer; 10 participants served as nonintervention comparison participants. The CRP resulted in significant improvement on a continuous performance test (Butler & Copeland, 2002). On the basis of additional supportive evidence (Butler & Mulhern, 2005), this study provided the foundation for the current Phase 3 clinical trial.

Our group tested the efficacy of this integrative CRP using a multicenter, nationwide, randomized clinical trial. To our knowledge, this is the most ambitious test of cognitive remediation for brain injury with either children or adults. It was specifically hypothesized that the CRP would result in improved academic achievement and cognitive functioning. The primary outcome functions were assessed within five rationally derived domains: academic achievement, brief focused attention, working memory, memory recall, and vigilance. In addition, secondary outcome measures of parent and teacher reports regarding children's attention abilities as well as participant self-reports on learning strategy acquisition and self-esteem were obtained. Significant improvements in all of these areas were predicted.

Method

Participants

Participants were 161 survivors of a childhood malignancy that involved CNS disease and/or treatment to the CNS. They were 6–17 years of age and were at least 1 year off treatment. Sample size was projected from power analyses. The diagnostic categories included brain tumors, leukemia, bone marrow transplant involving total body irradiation, and non-Hodgkins lymphoma. We initially used a pseudorandom approach for sampling from those populations but changed our approach to targeting high-risk participants, such as patients with a brain tumor, in order to ensure accrual goals. Participants were accrued at seven institutions in seven states in the following ratios of CRP/wait list control participants: Oregon (24/11), Pennsylvania (15/8), Tennessee (12/8), Texas (19/8), California (16/8), New York (9/5), and Ohio (13/5).

An attentional disturbance documented by scores on the Continuous Performance Test (CPT; Conners, 1992) and the Conners' Parent Rating Scale: Long Version—Revised (CPRS: LV–R; Conners, 1997) defined enrollment. To be eligible, a participant must have received a Clinical Competence Index (CCI) greater than 50% or have a level of errors of omission greater than T = 60 on the CPT. Test–retest correlation coefficients for these respective indices are 0.84 and 0.89. Analyses of covariance that compared nonclinical control participants and individuals with diagnosed attentional difficulties on the CPT have been reported as statistically significant at the p < .001 level for both errors of omission and the CCI. Additionally, the participant must have received a T score of greater than 60 on either the Cognitive Problems/Inattention scale or the Diagnostic and Statistical Manual Inattentive scale of the CPRS: LV–R.

Internal reliability coefficients for the Cognitive Problems/Inattention scale across age and gender groupings range from .81 to .90. Similarly, coefficients for the Diagnostic and Statistical Manual Inattentive scale range between .92 and .95. Confirmatory factor analysis of CPRS: LV–R items, as reported in the manual, indicates significant ratings for both scales. Values range between 0.6 and 0.7 and are all statistically significant at the p < .05 level. Finally, a full-scale IQ of 50 or more, as assessed with the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999), was required. As with all Wechsler scales, the WASI has excellent psychometric properties. The average stability coefficients for children range from .88 to .93 for the IQ scales. Correlation coefficients between the WASI and the Wechsler Intelligence Scale for Children—Third Edition (WISC–III; Wechsler, 1991) for IQ measures range between .76 and .87. All CRP participants were English speakers and were enrolled in school or receiving homebound instruction. Participants were excluded if there was a history of a documented attention-deficit disorder prior to the diagnosis of cancer or brain tumor. Institutional review board approval for this study was obtained at each participating institution, including the data management site. No adverse events were reported.

The study design was a Phase 3 randomized controlled clinical trial. Recruitment extended from January 2001 to November 2003. Of the 654 potential participants, 444 were screened, and 173 were eligible for the randomized trial. A total of 161 individuals were enrolled in the study (excluding 2 randomized participants who were later determined to be ineligible for participation). Figure 1 is the study flowchart. Approximately 80% of participants assigned to CRP completed at least 15 of the 20 sessions, and 83% completed posttreatment (T2) assessments. Of the wait list control participants, 98% completed T2. This difference was statistically significant (Wald statistic = 4.39, p = .04). For the CRP participants, 75% completed the 6-month follow-up evaluation (T3). Project-specific time constraints made it impossible for us to collect the T3 data on control participants and still offer them the CRP. T3 data are presented in Table 3 for visual inspection. Although these data reflect no obvious declines in scores, no statistical analyses were conducted.

Figure 1
Study flowchart. ADD = attention-deficit disorder; CRP = Cognitive Remediation Program.
Table 3
Individual Measure Means and Standard Deviations

Participants were randomly assigned by the central data manager to the treatment condition or to a wait list control group at a ratio of 2:1. The decision was made on the basis of (a) preliminary data suggesting the CRP intervention was likely to be beneficial and (b) our inability to deliver the CRP to all control participants within the period of the grant if a 1:1 approach was used. Concerns over meeting recruitment goals if all participants were not offered treatment also entered into this decision. Thus, a T3 follow-up evaluation on control participants was not conducted, because they were offered the CRP after the T2 assessment. Both groups were assumed to be receiving special education services if needed, as that is the current standard of care for outpatient pediatric patients with brain injuries.

Sample descriptive information (see Table 1) demonstrated that there were no statistically significant differences between the two groups on any of the demographic variables. The two groups did differ on performance IQ, with the CRP participants scoring at a lower level compared with the wait list control participants. The groups were not significantly different in errors of omission or commission on the CPT.

Table 1
Comparisons of Group Characteristics at Study Entrance

Intervention

The CRP is a tripartite model that combines interventions derived from three approaches: brain injury rehabilitation, educational psychology, and child clinical psychology (Butler & Copeland, 2002). Participants in the CRP were seen for a total of up to 20 two-hr weekly sessions over 4–5 months. Specific CRP interventions were programmatic but individualized. The CRP has three interdependent components: (a) hierarchically graded massed practice, (b) strategy acquisition, and (c) cognitive–behavioral interventions. These components are not orthogonal. Participants completed a modified version of the Attention Process Training cognitive rehabilitation program. Developed by Sohlberg and colleagues (Sohlberg, Johnson, Paule, Raskin, & Mateer, 1999; Sohlberg & Mateer, 1999), it is designed to strengthen multidimensional aspects of attentional processes. The intervention for the clinical trial described here was identical to our CRP approach, as described previously (Butler & Copeland).

All site principal investigators (PIs) and assessment/CRP research assistants (RAs) met for training prior to participant accrual. Subsequently and throughout the course of the study, RAs submitted training tapes demonstrating in vivo competence with a CRP participant to the two principal investigators (Robert W. Butler, Donna R. Copeland). These tapes were submitted at the beginning and middle of treatment, and each RA received verbal and written feedback. CRP therapists were graduate-level clinical psychology students, equivalent health care professionals, or postdoctoral fellows. To ensure continued treatment integrity, we conducted regular conference calls with site PIs and RAs over the period of the study. It was not possible for us to blind RAs to the treatment condition, but separate RA positions were maintained for assessment and treatment.

Measures

All participants completed a baseline battery of neuropsychological tests that we had selected to assess attentional functions, memory, new learning, and academic achievement. Parent and teacher reports of attentional abilities and participant self-reports of esteem and quality of life were obtained as well. We used rational-based data reduction to consolidate the individual measures. Participants in the CRP treatment group were reevaluated upon completion of the intervention and again 6 months postintervention. Participants in the wait list control group were retested 4–6 months after baseline. Parents and teachers were not blind to treatment status.

Primary Measures

Academic Achievement (Index 1)

Standardized academic achievement tests were used as indicators of the degree to which CRP treatment gains generalized to performance at school. The following measures were completed by all participants.

Wide Range Achievement Test—Third Edition (WRAT–3; Wilkinson, 1993)

This test provides standardized scores in reading decoding, spelling, and arithmetic computation. Coefficient alpha values for the three WRAT–3 scales range from .85 to .95. Scores on this measure of academic achievement correlate in the moderate but statistically significant range (.5–.6) with WISC–III full-scale IQ scores.

Calculation and Applied Problems (Woodcock–Johnson Tests of Achievement—Revised; Woodcock & Johnson, 1989)

The manual reports impressive split-half reliability coefficients that range between .80 and .90. Additionally, correlation coefficients between the Woodcock–Johnson Tests of Achievement—Revised, as reported by Spreen and Strauss (1998), indicate moderate-to-high consistency with other measures of academic achievement, as documented by correlations that place in the .50–.70 field. These two subtests have a strong attentional component. The Calculation subtest measures the ability of the child to perform mathematical operations using arithmetic skills and formulas. The Applied Problems subtest requires the child to use mathematical skills to solve practical problems. The effects of reading ability are minimized in these subtests.

Reading Comprehension (Peabody Individual Achievement Test—Revised; Dunn & Markwardt, 1970)

The Peabody Individual Achievement Test is a well-standardized and widely used instrument for individuals 5–18 years of age. All subtests, including the Reading Comprehension measure, demonstrate internal consistency and test–retest reliability coefficients above .90. The measure is significantly correlated with a test of receptive vocabulary, the Peabody Picture Vocabulary Test—Revised (Dunn & Dunn, 1981), at a statistically significant level, with validity coefficients reported in a range from .50 to .72.

Arithmetic (WISC–III; Wechsler, 1991)

This subtest involves orally administered word problems that require knowledge of mathematical concepts. As documented in the reference manual, reliability and validity are excellent.

Brief Focused Attention (Index 2)

Digit Span (WISC–III; Wechsler, 1991)

Digit Span contains two parts. In Digits Forward, the examiner reads a number series aloud at the rate of one digit per second, and the participant is asked to repeat each series in the exact order in which it was read. Digits Backward is a measure of working memory. The psychometric properties of this measure are excellent and well documented.

Sentence Memory (Wide Range Achievement Test of Memory and Learning [WRAML]; Sheslow & Adams, 1990)

The examiner reads sentences aloud, and the participant is instructed to repeat each sentence immediately. The manual reports alpha coefficients that range from .78 to .90; the median coefficient value for the various subtests from the WRAML, including Sentence Memory, varies from .90 to .96. Scores from the WRAML also correlated with other measures of attention and memory. Specifically, the WRAML General Memory Index is reported to have a correlation coefficient of .80 with the Stanford–Binet Short-Term Memory Index.

Stories (Children's Memory Scale; Cohen, 1997)

The Stories test is a measure of immediate and delayed recall for verbal material. Two age-appropriate, brief stories are read, and recall is recorded immediately and a half hour later. Administration was altered, in that cuing regarding a delayed recall was not provided. Across all age groups, immediate and delayed Stories subtest scores have reliability coefficients that range between .70 and .81. The manual reports an intercorrelation between immediate and delayed recall on the Stories subtest of .88.

Rey Auditory Verbal Learning Test (Trial 1 [RAVLT]; Rey, 1964)

A list of 15 words is read, and the participant names as many of them as he or she remembers. Initially developed in France, this test of attention and verbal learning is widely used in the United States and other countries. It has been reported that, over intervals up to 1 year, the measure has moderate but significant test–retest reliability (Uchiyama et al., 1995). Research on the factor structure of the RAVLT is supportive of its validity as a measure of verbal learning, and scores correlate significantly with those for the California Verbal Learning Test (Crossen & Wiens, 1994). The list is presented five more times. The Trial 1 score was used as a brief focused attention variable. There are excellent normative data available for developmental populations (Baron, 2004).

Working Memory (Index 3)

Digits Backward (WISC–III; Wechsler, 1991)

This test is described above. Psychometric properties have been addressed.

Stroop Color–Word Test (Trial 3; Golden, 1978)

Participants are required to look at color words, colors, and color names written in a noncorresponding color and to name them as quickly as possible. The interference condition is considered the primary index of working memory. Spreen and Strauss (1998) reported studies that reflect trial-to-trial reliabilities for this measure that are at or above .75. Test–test reliability coefficients are higher. Factor analytic studies suggest that the interference condition of the Stroop Color–Word Test appears to be primarily related to working memory abilities as opposed to general intelligence (Baron, 2004).

Trail Making Test B (Reitan, 1969)

This test assesses the ability to alternate one's attention between sequences of numbers and letters. The Trail Making Test B is a measure of attention and working memory. Interater reliability and internal consistency are acceptable, as documented by coefficients that range from .67 to .98 over several studies (Spreen & Strauss, 1998). In fact, Spreen and Strauss concluded, “In summary, the Trail Making Test is a well established, sensitive test of visual search and sequencing backed by a solid body of research and normative data” (1998, p. 539). It has been validated with children and adolescents (Baron, 2004).

Brief Test of Attention (Schretlen, 1997)

On this test, which is used for assessment of divided attention, a taped voice reads 10 lists of letters and numbers. There are normative data on children ages 6 years and above. The manual indicates that an internal reliability coefficient alpha of .80 was obtained on the normative sample. The Brief Test of Attention correlated highly with Trail Making Test B (r = −0.55, p < .001) and also with Digits Backward from the Wechsler scales (r = .53, p < .001).

Memory Recall (Index 4)

Stories (Delayed Recall)

The psychometric properties of this test have been described above.

Rey–Osterrieth Complex Figure Test (Delayed Recall; Lezak, Howieson, & Loring, 2004)

The participant is asked to re-create a drawing from memory a half hour after he or she has copied it. We used this measure as an index of nonverbal memory. It is reported that reliability coefficients comparing immediate to one half hour delay recall are in the moderate range (.47–.59), as documented by Spreen and Strauss (1998). The validity of the complex figure as a measure of memory has been documented by Waber and Holmes (1986). As with the RAVLT, appropriate normative data have been obtained on the complex figure measure for children and adolescents (Baron, 2004).

RAVLT (Delayed Recall of Trial 1; Rey, 1964)

We conducted this test as described in the Brief Focused Attention section but asked the participant to recall the original list of words after a half hour delay. Psychometric properties are reported above.

Vigilance (Index 5)

CPT–II (Conners, 1992)

The attention/concentration dependent variables were omissions, hit reaction time, variability, and the CCI. We used the scores obtained on the screening test for the initial assessment. Reliability and validity data for the CPT–II are reported above.

Secondary Measures

Learning/Learning Strategies

Strategies Assessment Measure

On this test, which assesses the number of metacognitive strategies that an individual has learned and retained, the participant is asked what behaviors he or she used to (a) prepare for the activity, (b) maintain on-task behavior, and(c) assess postactivity performance. The score is the total number of strategies verbalized. Raw scores were converted to z scores. This is a self-report measure that has no established psychometric qualities.

Parent/Teacher Ratings of Attention

CPRS: LV–R (Conners, 1997)

This test provides two indices of inattention and hyperactivity based on caregiver rating. Reliability and validity are described above.

Conners' Teacher Rating Scale: Long Version—Revised (CTRS: LV–R; Conners, 1997)

This measure is similar to the CPRS: LV–R but is based on teacher report. One index is used to assess inattention, and another is used to identify hyperactivity. The psychometric properties of the CTRS: LV–R are extremely similar to those of the CPRS: LV–R. Confirmatory factor analysis reports an overall loading of 0.49 on the Cognitive Problems/Inattention scale, and the intercorrelation between this loading and the Attention Deficit/Hyperactivity Disorder scale is significant (p < .05) but modest in terms of explained variance (r = .44). Internal consistency coefficients for the CTRS: LV–R are above .80 over all age and gender groups.

Self-Esteem

Culture-Free Self-Esteem Inventory, Second Edition (Battle, 1992)

This is a psychometrically sound child/adolescent self-report measure of self-image that assesses overall independent reliance and perception of academic adjustment. It addresses self-image over the course of the developmental span. Test–retest correlations for the normative sample of the Culture-Free Self-Esteem Inventory range from .81 to .89. Correlation coefficients with other indices of psychological adjustment range from .66 to .91 for both gender groupings, as documented by the technical manual.

Results

The first step in the statistical analyses was data reduction, which we accomplished by collapsing most individual measures into five indices prespecified in the protocol: (a) academic achievement, (b) brief focused attention, (c) working memory, (d) memory recall, and (e) vigilance. Additionally, parent/teacher and self-report measures of attention and self-esteem were analyzed. Tests were age standardized and, when necessary, converted to a common metric. For each component scale, we used a simple regression on age to generate standardized residuals. The slope of the curve was allowed to change at 8, 10, 12, 14, and 16 years. The residuals were converted to z scores and then averaged. Internal consistency of the summary measures was assessed with Cronbach's alpha.

We used a repeated-measures model for incomplete data in all analyses. The model included a common mean for the baseline assessment and group-specific means at follow-up. A linear and quadratic term for age was included, with no time interaction. The term was centered at 10 years, roughly the average age in this study. The covariance of the repeated measures was assumed to be homogeneous across the two groups but was allowed to vary over time. The within- and between-group changes were estimated using linear contrasts of the estimated parameters. The difference in the change from baseline (T1) to T2 between groups was the primary endpoint, and the test was based on the t statistic associated with the linear contrast. There was no difference in days between T1 and T2 assessments between the two groups, t(1, 140) = −0.46, p = .65. The mean interval for CRP participants was 227.5 days (SD = 71.3) and for control participants was 221.1 days (SD = 96.1).

Effect sizes were estimated with the standard deviation of the baseline assessment as the denominator (see Table 2) and the standard deviation of the T1 to T2 change (see DISCUSSION). We calculated power for predetermined differences using the first definitions for an alpha level of .05. Table 2 summarizes group comparisons for all dependent variables. Given that some measures were age corrected and others were not, the most important clinical interpretive anchor in this table is the effect size. Table 3 presents means and standard deviations for all individual dependant variables. Five index measures were assembled on the basis of rational development of related variables. Table 4 lists these indices and shows internal consistency with individual variable correlations for each overall summary measure.

Table 2
Comparison of Group Performance on Study Variables
Table 4
Construction of Indices/Summary Measures

Primary Measures

Academic Achievement (Index 1)

Seven scales measured academic achievement. Three focused on language, and four focused on mathematics. Exploratory factor analysis produced a single factor among the seven measures that explained over 94% of the common variance. Although the language and mathematics variables did partition into separate components, they were strongly correlated and virtually redundant. There was no change over time in the control group, but a statistically significant improvement within the CRP group was noted at T2. The differences between groups were statistically significant.

Brief Focused Attention (Index 2)

Both groups showed significant changes on measures of focused attention over the initial 6-month period. There was no statistically significant difference between the two groups.

Working Memory (Index 3)

For working memory, both groups demonstrated improved performance from T1 to T2. The difference between the groups was not statistically significant.

Memory Recall (Index 4)

For memory recall, both groups improved from T1 to T2. The difference of rehabilitative success was not significantly significant.

Vigilance (Index 5)

All participants demonstrated improved performance in vigilance over time. At T2, there was no statistically significant difference between the groups.

Secondary Measures

There were no significant differences between T1 and T2 in the control group with regard to learning strategies. However, the CRP group acquired significantly more learning and metacognitive strategies. It should be noted that these data do not necessarily reflect the degree to which the strategies were used on a regular basis.

Parent responses to the Conners' rating scales revealed fewer cognitive problems, improved attention, and a reduced tendency toward attention deficit/hyperactivity disorder symptoms in the participants who received the CRP as compared with the participants in the control condition. Differences between the two groups at T2 were statistically significant.

Teacher ratings indicated a perception of fewer symptoms of inattention or cognitive problems in the CRP group but not in the control participants. The difference at T2 between the two groups, however, was not statistically significant. For the teacher measure of attention deficit/hyperactivity disorder, no significant effects were obtained.

No statistically significant changes in self-esteem were apparent in either group from T1 to T2.

Treatment-Related Variables

The number of sessions completed and changes from T1 to T2 in the primary outcomes were compared across sites. We collected these data to assess the possible impact of site-based treatment fidelity challenges. With very few exceptions, consistency was maintained. Participants from the Oregon Health and Science University completed more sessions than did those participants at the Children's Hospital of Philadelphia site, χ2(6, N = 108) = 21.1, p < .01. The Strategies Assessment Measure was the only variable that differed, with participants from the Oregon Health and Science University acquiring more strategies than did participants at the other sites overall and participants at University of Texas/M. D. Anderson Cancer Center manifesting more strategies than did participants at St. Jude Children's Research Hospital, χ2(6, N = 87) = 15.5, p = .02. No other statistically significant differences emerged.

We analyzed compliance, defined by number of treatment and assessment sessions, and older age (Wald = 5.50, p = .02) and African American origin (Wald = 4.90, p = .03) were associated with completion of fewer than 18 treatment sessions. Gender, socioeconomic status, and time since diagnosis were not related to treatment compliance. Noncompletion of T2 assessment was related to enrollment in the CRP condition (Wald = 4.40, p = .04) and to older age of the participant (Wald = 3.66, p = .05).

Discussion

Our original hypotheses were that five primary indices regarding academic achievement and neurocognitive functioning would be significantly improved following treatment of participants with the CRP. The results from this Phase 3 clinical trial are equivocal. Participants experienced statistically significant improvement in academic achievement, incorporated more metacognitive strategies, and, on the basis of parental report, manifested improved attention. However, there were no statistically significant differences in neurocognitive functioning, even though trends were supportive of mild gains in neuropsychological development. It is our expectation that the clinical significance of the CRP will improve over time, particularly if caregivers continue to emphasize the skills that were taught. Nevertheless, this matter remains untested and is in need of further evaluation. We are now analyzing moderating and mediating factors regarding participants who benefited from the CRP versus those who did not. These are critical issues that will guide us in our efforts to provide more effective rehabilitative services.

Brain injury rehabilitation is a demanding task for both therapists and patients. The process is particularly complex in children/adolescents, because their brains are undergoing healing and development simultaneously, and the involvement of caregivers is critical. At this time, we report mixed results, with statistical significance in some areas and positive trends. In fact, as presented in Table 2, effect sizes (relative to the baseline standard deviation) tend to be within the .1–.5 range. As most of the measures were moderately correlated over time (ρ = approximately 0.5), estimation of the effect size relative to the standard deviation of the change scores resulted in similar estimates. The one exception was academic achievement, which was strongly correlated over time. The effect size estimates for this domain increased from .19 to .53.

An effect size in the range of .5 is within the medium range. Thus, participants who completed our CRP demonstrated improvement in their ability to successfully complete tests of arithmetic- and language-based functions, even though our treatment was not directed toward education. In sum, treatment produced the most prominent impact on measures of generalization. It remains to be seen whether or not these results will be stable and if they will have future ramifications in terms of postsecondary education.

It should be noted that brain injury rehabilitation is commonly characterized by small-to-medium effect sizes (Cicerone, 1999; Cicerone et al., 2000) and has typically been reported to result in very limited improvement in cognitive, educational, behavioral, and social domains (Anderson & Catroppa, 2006). Although we have evidence of generalization of functioning to the academic arena, our findings are consistent with this pattern of limited effect.

Psychological and medical interventions are associated with modest treatment effects (Meyer et al., 2001). Even with this evidence-based caveat, we are extremely interested in determining the degree to which our intervention will have a positive and lasting effect not only on future school performance but on development and quality of life. Ongoing research will be necessary to document these possibilities, and longitudinal designs will be necessary in this regard. There are clear questions regarding clinical significance. Our study did document improvement in academic functioning. However, these improvements are not clearly associated with neurocognitive functioning, which we considered to be a primary outcome variable of this clinical trial. This fact has caused our team to reevaluate the manner in which, we believe, that change is occurring in children/adolescents with brain injuries. The current study is very robust in terms of sample size and research methodology, but our results are less robust than anticipated. There are positive aspects to this clinical trial, in terms of significant improvement in academic performance and some aspects of attention in children who have suffered deficits following their CNS disease/treatment, but the findings also amplify the fact that much work must be directed toward pediatric brain injury rehabilitation.

Summary measures of neurocognitive functioning did not, for the most part, reveal a statistically significant level of improvement for the CRP participants. Composite measures all had acceptable internal consistency but were lowest on working memory. Working memory is emerging as an important mediator/moderator of intelligence and achievement declines in this population (Beebe et al., 2005; Palmer et al., 2003). Additional research is needed for development of an index that is sensitive enough to reliably measure this multidimensional cognitive function. Although CRP-treated participants demonstrated improvements on most neuropsychological measures, many control participants also demonstrated improvements. This result is likely due, in part, to practice effects. Our next generation of clinical trials must address this issue of concern, perhaps through the use of alternate form testing.

In addition, it is critical that researchers introduce strategies to increase compliance and intervention potency. Clearly, there is a need for greater developmental focus within this population, as advocated by Anderson and Catroppa (2006), who emphasized a multimodal approach that includes family- and school-based interventions. These have been the guiding theoretical and practical principles of the CRP approach, but greater emphasis is needed if we are to achieve more robust results.

We assessed the effect that treatment would have on self-esteem, as measured by the participant's report of mastery. Although there were no statistically significant differences between pre- and posttreatment scores on the culture-free self-esteem measure for either group, it may be naive for us to expect that these changes would occur so quickly.

Data were analyzed under an “intent-to-treat” model. This approach most accurately reflects real-world expectations, in that not all clinical patients complete a prescribed intervention trial, and it minimizes misinterpretation of data. Thus, the comparisons between study and control participants are not based on the effectiveness of the CRP as prescribed but are influenced by additional factors, such as compliance and dropout. In effect, the true potency of the CRP was not tested in the pure sense. As shown in Figure 1, only 60% of participants in the CRP arm completed the entire regimen. However, 80% completed at least three quarters of the therapeutic intervention. Nevertheless, compliance is a concern. The CRP is a demanding commitment for families, and we are currently developing treatment revisions to address these issues. These revisions will help caregivers navigate the educational system more effectively, so the CRP can be incorporated into classroom work.

The current findings are, in our opinion, encouraging but also sobering. It is likely unreasonable for us to expect to be able to rehabilitate children/adolescents with a brain injury to a pre-CNS insult level of functioning. We believe, however, that it is incumbent on us to devise strategies that will ensure rehabilitation, compliance, and increased program potency. In addition, more extensive follow-up treatment and the possible benefits of adding booster sessions over time should be investigated.

Within the area of psychotherapy research, a subtractive model is typically encouraged. Once an intervention is proved effective, researchers dismantle their treatment methods to determine the specific causative intervention. In our opinion, this may be an ineffective approach for brain injury rehabilitation. Instead, we propose an additive strategy, given the difficulty in teaching significant others and the individual with neuropsychological impairment how to manage his or her cognitive resources, particularly when the individual is a child or adolescent.

We are instituting treatments to increase the impact of the CRP that are based on proven caregiver interventions (Sahler et al., 2005). The use of these innovative methods represents a potential advance for pediatric cancer survivors and, we hope, all populations with pediatric brain injury.

Childhood brain injuries can have a devastating effect both on the ability of the individual to benefit from schooling and develop his or her foundation for a productive adult life and on family and social relationships over the course of the individual's life. The entire rehabilitation process suffers from poor funding and from treatments administered by individuals without formal training in brain injury rehabilitation. Continued research should be directed toward the administration of effective rehabilitation techniques. We believe that our data support this directive. The development of a comprehensive and collaborative team that includes the patient, therapist, caregivers, educational professionals, and other involved individuals who will marshal the necessary resources to promote commitment, involvement, and the lifetime use of skills taught during rehabilitation is essential. A new standard of care in pediatric outpatient brain injury rehabilitation must be advanced. The educational system is underfunded, and there are few adequately trained faculty and staff. Medical caregivers in the area of clinical neuropsychology, psychiatry, and pediatric neurology need to become more involved and should be appropriately funded.

Acknowledgments

This research was supported by National Cancer Institute Grant RO1 CA83936. The authors take this opportunity to remember Raymond K. Mulhern, who was an integral member of this project. Each author knew Ray personally and collaborated with him on this and other endeavors.

Contributor Information

Robert W. Butler, Department of Pediatric Hematology/Oncology, Oregon Health & Science University.

Donna R. Copeland, Department of Pediatrics, University of Texas/M. D. Anderson Cancer Center, Houston.

Diane L. Fairclough, Colorado Health Outcomes Program, Preventive Medicine and Biometry, Colorado Health Sciences Center, Denver.

Raymond K. Mulhern, Department of Behavioral Medicine, St. Jude Children's Research Hospital, Memphis, Tennessee.

Ernest R. Katz, Department of Pediatrics and Psychology, Children's Hospital Los Angeles.

Anne E. Kazak, Department of Oncology, The Children's Hospital of Philadelphia.

Robert B. Noll, Departments of Pediatrics, Psychiatry, and Psychology, Children's Hospital of Pittsburgh.

Sunita K. Patel, Divisions of Population Science and Pediatrics, City of Hope National Medical Center, Duarte, California.

Olle Jane Z. Sahler, Department of Pediatrics, University of Rochester Medical Center.

References

  • Anderson V, Catroppa C. Advances in postacute rehabilitation after childhood-acquired brain injury: A focus on cognitive, behavioral, and social domains. American Journal of Physical Medicine and Rehabilitation. 2006;85(9):767–778. [PubMed]
  • Baron IS. Neuropsychological evaluation of the child. New York: Oxford; 2004.
  • Battle J. Culture-free self-esteem inventories. 2nd. Austin TX: PRO-ED; 1992.
  • Beebe DW, Ris MD, Armstrong FD, Fontanesi J, Mulhern R, Holmes E, et al. Cognitive and adaptive outcome in low-grade pediatric cerebellar astrocytomas: Evidence of diminished cognitive and adaptive functioning in National Collaborative Research Studies (CCG 9891/POG 9130) Journal of Clinical Oncology. 2005;23(22):5198–5204. [PubMed]
  • Braga LW, Da Paz AC, Ylvisaker M. Direct clinician-delivered versus indirect family-supported rehabilitation of children with traumatic brain injury: A randomized controlled trial. Brain Injury. 2005;19(10):819–831. [PubMed]
  • Butler RW. Attentional processes and their remediation in childhood cancer. Medical and Pediatric Oncology. 1998;30(1):75–78. [PubMed]
  • Butler RW, Copeland DR. Attentional processes and their remediation in children treated for cancer: A literature review and the development of a therapeutic approach. Journal of the International Neuropsychology Society. 2002;8(1):115–124. [PubMed]
  • Butler RW, Hill JM, Steinherz PG, Meyers PA, Finlay JL. Neuropsychologic effects of cranial irradiation, intrathecal methotrexate, and systemic methotrexate in childhood cancer. Journal of Clinical Oncology. 1994;12(12):2621–2629. [PubMed]
  • Butler RW, Kerr M, Marchand A. Attention and executive functions following cranial irradiation in children. Journal of the International Neuropsychological Society. 1999;5:108.
  • Butler RW, Mulhern RK. Neurocognitive interventions for children and adolescents surviving cancer. Journal of Pediatric Psychology. 2005;30(1):65–78. [PubMed]
  • Cicerone KD. Commentary: The validity of cognitive rehabilitation. Journal of Head Trauma Rehabilitation. 1999;14(3):316–321. discussion 322–324. [PubMed]
  • Cicerone KD, Dahlberg C, Kalmar K, Langenbahn DM, Malec JF, Bergquist TF, et al. Evidence-based cognitive rehabilitation: Recommendations for clinical practice. Archives of Physical Medicine and Rehabilitation. 2000;81(12):1596–1615. [PubMed]
  • Cicerone KD, Dahlberg C, Malec JF, Langenbahn DM, Felicetti T, Kneipp S, et al. Evidence-based cognitive rehabilitation: Updated review of the literature from 1998 through 2002. Archives of Physical Medicine and Rehabilitation. 2005;86(8):1681–1692. [PubMed]
  • Cohen M. Children's Memory Scale. San Antonio, TX: Psychological Corporation; 1997.
  • Conners CK. Conners' Continuous Performance Test (Version 3.0) [Computer software] Toronto, Canada: Multi-Health Systems; 1992.
  • Conners CK. Conners' Rating Scales—Revised. North Tonawanda, NY: Multi-Health Systems; 1997.
  • Crossen JR, Wiens AN. Comparison of the Auditory–Verbal Learning Test (AVLT) and California Verbal Learning Test (CVLT) in a sample of normal subjects. Journal of Clinical and Experimental Neuropsychology. 1994;16(2):190–194. [PubMed]
  • Dunn LM, Dunn ES. Peabody Picture Vocabulary Test—Revised. Circle Pines, MN: American Guidance Services; 1981.
  • Dunn LM, Markwardt FC. Peabody Individual Achievement Test—Revised. Circle Pines, MN: American Guidance Services; 1970.
  • Golden CJ. Stroop Color and Word Test. Chicago: Stoelting; 1978.
  • Heideman R, Packer R, Albright L, Freeman C, Rorke LB. Tumors of the central nervous system. In: Pizzo P, Poplack D, editors. Principles and practice of pediatric oncology. 3rd. Philadelphia: Lippincott-Raven; 1997. pp. 633–681.
  • Hollingshead AB. Two factor index of social position. New Haven, CT: Yale University Press; 1965.
  • Hooft IV, Andersson K, Bergman B, Sejersen T, Von Wendt L, Bartfai A. Beneficial effect from a cognitive training programme on children with acquired brain injuries demonstrated in a controlled study. Brain Injury. 2005;19(7):511–518. [PubMed]
  • Hoppe-Hirsch E, Brunet L, Laroussinie F, Cinalli G, Pierre-Kahn A, Renier D, et al. Intellectual outcome in children with malignant tumors of the posterior fossa: Influence of the field of irradiation and quality of surgery. Child's Nervous System. 1995;11(6):340–345. discussion 345–346. [PubMed]
  • Lezak M, Howieson D, Loring D. Neuropsychological assessment. New York: Oxford; 2004.
  • Margolin J, Poplack D. Acute lymphoblastic leukemia. In: Pizzo P, Poplack D, editors. Principles and practice of pediatric oncology. 3rd. Philadelphia: Lippincott-Raven; 1997. pp. 409–462.
  • Meyer GJ, Finn SE, Eyde LD, Kay GG, Moreland KL, Dies RR, et al. Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist. 2001;56(2):128–165. [PubMed]
  • Mulhern RK, Butler RW. Neurocognitive sequelae of childhood cancers and their treatment. Pediatric Rehabilitation. 2004;7(1):1–14. discussion 15–16. [PubMed]
  • Mulhern R, Butler R. Neuropsychological late effects. In: Brown RT, editor. Comprehensive handbook of childhood cancer and sickle cell disease: A biopsychosocial approach. New York: Oxford University Press; 2006.
  • Mulhern RK, Merchant TE, Gajjar A, Reddick WE, Kun LE. Late neurocognitive sequelae in survivors of brain tumours in childhood. Lancet Oncology. 2004;5(7):399–408. [PubMed]
  • National Institute of Neurological Disorders and Stroke, National Institutes of Health. Traumatic brain injury: Hope through research. 2002. Retrieved from http://www.ninds.nih.gov/disorders/tbi/detail_tbi.htm.
  • Palmer SL, Gajjar A, Reddick WE, Glass JO, Kun LE, Wu S, et al. Predicting intellectual outcome among children treated with 35–40 Gy craniospinal irradiation for medulloblastoma. Neuropsychology. 2003;17(4):548–555. [PubMed]
  • Reitan R. Manual for administration of neuropsychological test batteries on adults and children. Bloomington: Indiana University Press; 1969.
  • Rey A. L'examen clinique en psychologie [Clinical tests in psychology] Paris: Presses universitaire de France; 1964.
  • Ris MD, Packer R, Goldwein J, Jones-Wallace D, Boyett JM. Intellectual outcome after reduced-dose radiation therapy plus adjuvant chemotherapy for medulloblastoma: A Children's Cancer Group study. Journal of Clinical Oncology. 2001;19(15):3470–3476. [PubMed]
  • Sahler OJ, Fairclough DL, Phipps S, Mulhern RK, Dolgin MJ, Noll RB, et al. Using problem-solving skills training to reduce negative affectivity in mothers of children with newly diagnosed cancer: Report of a multisite randomized trial. Journal of Consulting and Clinical Psychology. 2005;73:272–283. [PubMed]
  • Schretlen D. Brief Test of Attention. Odessa, FL: Psychological Assessment Resources; 1997.
  • Sheslow D, Adams W. Wide range assessment of memory and learning. Wilmington, DE: Jastak Associates; 1990.
  • Sohlberg MM, Johnson L, Paule L, Raskin SA, Mateer CA. Attention Process Training II (APT–II) Wake Forest, NC: Lash & Associates; 1999.
  • Sohlberg M, Mateer CA. Attention Process Training I (APT–I) Wake Forest, NC: Lash & Associates; 1999.
  • Spiegler BJ, Bouffet E, Greenberg ML, Rutka JT, Mabbott DJ. Change in neurocognitive functioning after treatment with cranial radiation in childhood. Journal of Clinical Oncology. 2004;22(4):706–713. [PubMed]
  • Spreen O, Strauss E. A compendium of neuropsychological tests. 2nd. New York: Oxford; 1998.
  • Uchiyama CL, D'Elia LF, Dellinger AM, Becker JT, Selmes OS, et al. Alternate forms of the Auditory–Verbal Learning Test: Issues of test comparability, longitudinal reliability, and moderating demographic variables. Archives of Clinical Neuropsychology. 1995;10(2):133–145. [PubMed]
  • Waber DP, Holmes JM. Assessing children's memory productions of the Rey–Osterrieth complex figure. Journal of Clinical and Experimental Neuropsychology. 1986;8(5):563–580. [PubMed]
  • Wechsler D. Wechsler Intelligence Scale for Children. Third. San Antonio, TX: Psychological Corporation; 1991.
  • Wechsler D. Wechsler Abbreviated Scale of Intelligence. San Antonio, TX: Psychological Corporation; 1999.
  • Wilkinson GS. The Wide Range Achievement Test. Third. Wilmington, DE: Wide Range; 1993.
  • Woodcock RW, Johnson MB. Woodcock–Johnson Tests of Achievement. Itasca, IL: Riverside; 1989.