|Home | About | Journals | Submit | Contact Us | Français|
The true benefit of pharmacological intervention to improve cognition in schizophrenia may not be evident without regular cognitive enrichment. Clinical trials assessing the neurocognitive effects of new medications may require engagement in cognitive remediation exercises to stimulate the benefit potential. However, the feasibility of large-scale multi-site studies using cognitive remediation at clinical trials sites has not been established.
Patients with DSM-IV schizophrenia from nine sites were randomized to a cognitive remediation condition that included the Posit Science Brain Fitness auditory training program with weekly NEAR ‘bridging groups,’ or a control condition of computer games and weekly healthy lifestyles groups. Patients were expected to complete 3–5 one-hour sessions weekly for 40 sessions or 12 weeks, whichever came first.
The primary outcomes were feasibility results as measured by rate of enrollment, retention, and completion rate of primary outcome measures. Within the 3-month enrollment period, 53 of a projected 54 patients were enrolled and 47 completed the study. Thirty-one patients completed all 40 sessions and all patients completed all primary outcome measures. Preliminary efficacy results indicated that after 20 sessions, patients in the cognitive remediation condition demonstrated mean MCCB composite score improvements that were 3.7 (95% CI: 7.34, 0.05) T-score points greater than in patients in the computer games control group (F=4.16, df=1,46, p=0.047). At the end of treatment, a trend favoring cognitive remediation was not statistically significant (F=2.26, df=1,47, p=0.14).
Multi-site clinical trials of cognitive remediation using the Posit Science auditory training program with the NEAR method of weekly bridging groups in traditional clinical sites appear feasible.
Neurocognitive impairment affects almost all patients with schizophrenia1, ranges from moderate to severe,2 and is strongly correlated with functional outcomes3,4. Current antipsychotic treatment does little to improve cognitive impairment5,6. While several studies are ongoing to explore new treatments for cognitive impairment in schizophrenia, no pharmacologic approaches to improve cognition have yet been successful7–9.
Many patients who enroll in pharmacologic enhancement studies have little daily cognitive stimulation. It is possible that experimental pharmacologic interventions offer minimal or no benefit when patients are evaluated in this context. Analogous to the need for physical exercise in an individual who takes steroids to increase muscle mass, schizophrenia patients in pharmacological intervention trials may require systematic cognitive training to “exercise” any newfound cognitive potential that they may have acquired from drug treatment10.
Cognitive remediation has been defined as “a behavioral training based intervention that aims to improve cognitive processes (attention, memory, executive function, social cognition or metacognition) with the goal of durability and generalization” (Cognitive Remediation Experts Workshop, April, 2010). This area of work is rapidly evolving with many methodological barriers and potential advances. Cognitive remediation may provide an excellent platform for enriching the cognitive environment of patients engaged in pharmacologic trials to improve cognition11. Several studies and meta-analyses suggest that cognitive remediation produces medium effect size improvements in cognitive performance and, when combined with psychiatric rehabilitation, also improves functional outcomes12,13. Patients find these programs to be enjoyable and engaging, and they have been linked with increases in participant self-esteem14. Previous cognitive remediation trials have reported significant effects12–14, including those studies that have tested the Posit Science auditory module compared to a control condition of computer games15–17.
Ongoing treatment with cognitive remediation may thus provide schizophrenia patients with the necessary cognitive enrichment and motivation to demonstrate the true potential of effective cognitive enhancement with pharmacologic intervention. However, it is not clear whether a cognitive remediation intervention could be successfully employed in the large multi-site clinical trials that are designed to assess drug safety and efficacy. Most of the cognitive remediation studies performed to date in patients with schizophrenia have been implemented at single sites with highly trained research personnel and methods developed at those sites; thus the generalizability of these methods to research sites with limited cognitive remediation experience is not well established.
The goal of this study was to assess the feasibility of implementing a cognitive remediation program in a network of sites that do not specialize in this area of research with the following aims: 1. Determine the feasibility of training staff to provide evidence-based cognitive remediation to patients with schizophrenia in a set of clinical sites without substantial cognitive remediation experience but with clinical trials experience. 2. Determine the feasibility of conducting a controlled, randomized clinical trial to test the effectiveness of cognitive remediation for individuals with schizophrenia or schizoaffective disorder. 3. Determine if a cognitive remediation protocol can be implemented with a high level of protocol adherence by sites and low attrition and good treatment adherence by patients. We also provide preliminary multi-site efficacy data on the effect of a cognitive remediation program on neurocognition and related outcomes.
Nine sites from the Schizophrenia Trials Network, which was originally created to conduct the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) schizophrenia trial, were involved in the study. Seven of the sites had no previous experience with cognitive remediation trials. Each site was asked to enroll six patients with chronic DSM-IV schizophrenia over the course of 3 months, estimating a reasonable rate of recruitment for a large-scale efficacy trial. A total of 53 patients were entered into the study. Patients were required to: be 18–55 years of age; have Positive and Negative Syndrome Scale (PANSS)18 Hallucinatory Behavior, Unusual Thought Content, and Conceptual Disorganization ratings of no greater than moderately severe (≤ 5), have learned English before age 12; able to complete the MATRICS Consensus Cognitive Battery (MCCB)19 at the baseline assessment; obtain a raw score of 37 or greater on the Wide Range Achievement Test, Reading subtest, 3rd edition (WRAT-3)20 to establish minimum reading level (sixth grade) and estimated premorbid IQ; and be able to state specific goals relevant to the intervention that they would like to achieve. Patients were excluded from participating in the study if they met any of the following criteria: psychiatric hospitalization in the 8 weeks prior to randomization; adjustments in antipsychotic treatment in the 4 weeks prior to randomization; current regular use of an anticholinergic medication; DSM-IV diagnosis of alcohol or substance (other than nicotine) abuse within the last month or dependence within the last 6 months; history of mental retardation or pervasive developmental disorder; or other neurological disorder. IRB approval for the protocol was obtained at all sites. Informed consent was obtained for all patients.
The program was based upon suggestions from an NIMH Working Group on multi-site cognitive remediation methods10. It was designed to be completed over eight weeks, and consisted of 40 hours of computerized cognitive remediation training or control computer games, and 5 hours of group sessions. Patients were permitted up to 12 weeks to complete the 40 hours of training. For each of the three assessment visits, patients received $20 to help compensate for effort and for transportation costs. To help pay for travel to receive cognitive remediation or the control intervention for this research study, patients received limited reimbursement ($5) for transportation costs.
Patients randomized to the cognitive remediation group received the computerized Posit Science Brain Fitness auditory training program, which has demonstrated successful results in schizophrenia15,16. This program targets cognitive ability hierarchically in an adaptive design, with repeated exercises aimed at lower cognitive levels (e.g. auditory perception) before advancing to more complex cognitive constructs such as verbal memory21. Performance is rewarded through the use of novel and amusing visual and auditory embellishments, animated graphics, and an accumulation of points for each successful trial.
To help patients apply their cognitive skills to their everyday functioning, study patients receiving the auditory training cognitive remediation program also attended ‘bridging groups’ that met every week. The goals of these groups were to teach patients how to apply newly acquired cognitive skill to everyday tasks, promote group identity, and promote socialization, as conducted in the Neuropsychological Educational Approach to Remediation, or NEAR, program22. These bridging groups were very structured with detailed manuals. Group leaders were trained and certified by the developer of NEAR (AM), who also supervised the leaders to assure fidelity with the manual.
A control condition was included to determine the effects of the randomization process and randomized treatment on recruitment, informed consent, retention, patient burden, maintenance of blinded conditions for assessing outcomes, and to control for nonspecific treatment effects, such as supportive interactions with research personnel, reimbursements, experience with computers and computer activities, and general engagement of attentional systems.
Patients in this condition played enjoyable computer games for the same number of hours as active training patients and received the same amount of contact with personnel as the experimental group. Patients used 10 computerized games selected and studied by Fisher et al15.
To control for possible general benefits of group participation, study patients receiving the control condition received a healthy lifestyles group developed by Ganguli and colleagues23.
Each site randomly assigned three patients to cognitive remediation and three patients to the computer games control group. Treatments were open-label, but cognitive testers were blinded to treatment assignments. The success of the blinding procedures was evaluated as an outcome measure in this feasibility study.
The key indicators to evaluate the feasibility of this study were: rate of enrollment per site and overall, retention of patients in the trial, number of sessions completed during the course of the trial, blinding effectiveness, and completion rate of primary outcome measures.
All efficacy outcomes were evaluated at midpoint (after 20 hours of participation in the assigned intervention, no more than 6 weeks post randomization) and end of study (after 40 hours or 12 weeks of participation in the assigned intervention, whichever came first).
The most basic of the auditory training exercises requires users to make gradually more difficult distinctions between frequency modulation (FM) “sweeps” of auditory stimuli increasing or decreasing in frequency as the sweeps become progressively faster and are separated by shorter interstimulus intervals. As a positive control for task learning, patients’ performance on this task was determined in both treatment conditions. The outcome measure for this task was the shortest stimulus duration and inter-stimulus interval (which were equal to each other) for trials where patients were able to perform the task at 67% accuracy15.
The MATRICS Consensus Cognitive Battery (MCCB) was used to assess cognitive functioning in each of seven domains. T-scores are created for each cognitive domain and an overall composite score, which was the primary efficacy outcome.
The University of California, San Diego Performance Based Skills Assessment, 2nd Edition (UPSA-2)24 is a performance-based measure of the extent to which patients are capable of performing specific functional living skills such as household chores, communication, finance, transportation, and planning recreational activities25.
The Cognitive Assessment Interview (CAI)26 is a 10-item interview-based measure of cognitive function that incorporates ratings from 3 primary sources- subject, informant, interviewer. The presence of an informant interview was requested but not required.
Additional measures included: the Positive and Negative Symptom Scale (PANSS)27 was the primary assessment instrument for psychopathology 27 the Clinical Global Impression Scale (CGI-S)28 rating severity of psychopathology; the Specific Levels of Functioning Scale (SLOF)29 measuring function in the areas of self-maintenance, social functioning, and community living skills; the Rosenberg Self-esteem scale30; the Intrinsic Motivation Inventory-Schizophrenia Research (IMI-SR) scale31; and the Perceived Competency Scale (PCS)32. The overall study methodology quality was measured with the Clinical Trial Assessment Measure, a 15-item reliable and valid scale (range 0–100) of trial methodology for psychological treatment studies33.
The primary outcome for this trial was determination of the feasibility of conducting a multisite trial using cognitive remediation. Additional key outcomes were the completion rate of the primary and secondary outcome measures.
Exploratory efficacy analyses assessed whether there were significant changes after 20 sessions, which was the midpoint of the treatment schedule, and at endpoint, beyond expected practice effects (0.2 SDs) in the composite scores from the MCCB34. The data were analyzed using a mixed model approach to repeated measures using change from baseline to each post-baseline time point as the response, baseline as a covariate, and treatment, visit, and treatment-by-visit interactions as the predictors. Exploratory analyses were performed using the same methodology for all MCCB subscales and the other outcomes.
Demographic and clinical characteristics are described in Table 1. The two treatment groups did not differ at baseline on any of the 17 clinical or cognitive outcome measures except the Perceived Competency Scale (P<.02), which would not have been statistically significant if corrected for multiple comparisons. The baseline, session 20 and end of treatment values for all outcomes are presented in Table 2.
The Consort Figure (figure 1) describes the sample sizes for each treatment arm. Our stated goal was for 8 of 9 chosen sites to participate in the trial and for each site to enroll 6 patients. All nine sites participated and 8 of them enrolled 6 patients with one site enrolling 5. Of the 53 patients in the study, 47 met criteria for study completion, 25 in the cognitive remediation group and 22 in the computer games control group. Thirty-one patients (16 in the cognitive remediation condition and 15 in the control condition) completed all 40 sessions, and 16 patients completed fewer than 40 sessions in 12 weeks of treatment. Of these 16 patients, 9 patients in the cognitive remediation group completed a mean of 24.1(SD=7.5) sessions, and 7 patients in the control group completed a mean of 30.0 (SD=5.42) sessions in 12 weeks. Of the 6 patients who did not complete the study, 2 were not willing to maintain the time commitment and 4 withdrew for unspecified reasons
The overall study quality as rated by the Clinical Trials Assessment Measure was 87, which is equal to the top score given by Wykes et al17 in their review of 40 cognitive remediation trials where the mean Clinical Trials Assessment Measure rating was 57.4(SD 12.3).
All patients completed all MCCB tests at all time points. No more than three patients were missing data on any of the secondary outcome measures, including all three visits. All patients had interviewer ratings for the CAI; informant interview data were missing for 8, 13, 14 patients at baseline, 20 sessions, and end of study respectively for the cognitive remediation group and for 10, 11, 12 patients at those time points for the control group.
The blinding procedures were very effective. All cognitive testers completed a form asking them to estimate whether they were unblinded about patients’ treatment condition. In two cases, one in each treatment condition, testers estimated “definite” unblinding, and both reports were correct. In 15 additional cases, testers estimated “possible” unblinding. However, less than 50% of those estimates were correct, which was less than chance. In nine cases, testers were reasonably certain that a patient was receiving the cognitive remediation, but in only 4 of those cases was the tester was correct. In 6 cases the tester was reasonably certain that a patient was receiving the control intervention, but in only 3 of those cases was the tester correct.
Patients receiving the cognitive remediation intervention had substantial mean improvements on the auditory discrimination task (see Table 2), which were significantly greater than in patients in the control condition, who did not improve during the course of the study. This indicates that patients in the cognitive remediation group significantly improved their psychophysical efficiency in discriminating rapidly presented auditory FM sweeps, while control condition patients did not.
After 20 weeks, patients receiving the cognitive remediation treatment condition had MCCB composite scores adjusted for baseline of 32.7 compared to 29.0 for patients receiving the control condition (F=4.16, df=1,46 p=0.047; see Table 2). This mean change of 3.7 T-score points had a 95% confidence interval of 7.34, 0.05. At the end of treatment, patients receiving cognitive remediation had MCCB composite scores adjusted for baseline of 32.4 compared to 29.7 for those in the control condition (F=2.26, df=1,47, p=0.14; mean change = 2.73; 95% CI: 6.39, 0.93). Post hoc analyses suggested that the MCCB domains contributing most strongly to improvement in the cognitive remediation group at the 20 session point were verbal learning (p<.05), social cognition, and reasoning and problem-solving (p’s<.10). In patients receiving cognitive remediation, improvement in the auditory frequency discrimination task was not correlated with changes in the MCCB composite score or its component domains (all r’s <0.31; all P’s >.14).
There were no significant treatment-related differences on any of the secondary outcome measures (Table 2).
This study assessed the feasibility of conducting a randomized and blinded multi-site trial using a computerized auditory cognitive remediation intervention with weekly bridging groups compared to a control condition of computer games paired with healthy lifestyles weekly group meetings. The burden on patient treatment and assessment was considerable, with a 40-session target over the course of 8–12 weeks, and a large battery of clinical and cognitive assessments at baseline, midpoint, and endpoint. In terms of training, enrollment, patient engagement, and study completion, multi-site trials of cognitive remediation using these active and control interventions appear to be feasible, supporting large-scale efficacy trials of cognitive remediation and the use of cognitive remediation as a potential platform for large-scale drug trials. All sites were able to train testers and cognitive remediation specialists to the satisfaction of the intervention developers. Further, there were no missing cognitive measures on the MCCB, and almost no missing data on the clinical outcomes. The retention rate was very high, with 25 of 27 patients (93%) completing the cognitive remediation arm of the study. This rate is comparable to previously published single-site studies using similar methodologies13,15, and the average retention rate of 89% reported in Wykes’ meta-analysis of 40 studies17. This retention rate is also consistent with similarly sized government-funded trials assessing the effects of pharmacologic interventions to improve cognition in schizophrenia7. The sites in this study were from academic medical centers, and it is not clear whether such encouraging results would be replicated in non-academic industry trials. It is also not clear how these results would generalize to other cognitive remediation interventions. However, it appears that the requirement of this cognitive remediation intervention or a similar one in a drug trial for cognition in schizophrenia is unlikely to significantly affect recruitment, retention, or the completion of the study data set.
Although the intention of this study was not to test the efficacy of the cognitive remediation intervention, it is noteworthy that patients randomized to this condition demonstrated significant improvement in the FDA-recommended cognitive outcome, the MCCB composite score, after 20 sessions of treatment. This amount of benefit is consistent with previous single-site studies15, and similar to that obtained by McGurk et al12 with a different cognitive remediation package and within 0.1 effect size units of the mean improvement reported in the recent meta-analysis of Wykes et al17. However, in the current study there was not additional benefit to the MCCB at the endpoint of this study. Due to an enrollment and study completion deadline for this project, patients were given only 12 weeks to complete the 40 sessions, which may not have been sufficient time to gain the maximal benefit of treatment. It is possible that since 9 of the 25 patients did not receive the entire 40-session intervention, the efficacy of the cognitive remediation was reduced. Patients in the auditory training cognitive remediation group demonstrated robust improvement in the auditory frequency discrimination task at 20 sessions, but no additional improvement by the end of the study. Little is known about the ideal or minimal dose, frequency, and time course of cognitive remediation treatment response, which may differ across individuals10,35–37, and these issued need to be examine in future studies.
Because this was a feasibility and pilot study, it used a small sample size. However, the effect size for cognitive improvement with the auditory training intervention coupled with NEAR bridging groups is consistent with potential significant benefit for patients given sufficient statistical power to test this hypothesis. These results and the clear feasibility of the study design in a multi-site trial provide positive proof-of-concept empirical evidence that two new directions in clinical trial design are possible and worthy of investigation. First, they suggest that a large well-powered multi-site trial for assessing the true efficacy of cognitive remediation interventions is feasible. Further, they suggest that drug trials that aim to provide an enriched cognitive environment for patients can successfully implement cognitive remediation across multiple sites that have no prior experience with this intervention.
This work was supported by the National Institute of Mental Health (grant number N01 MH900001).
We acknowledge the great contributions of the patients and their families who devoted their time and energy to this research. Posit Science provided the Brain Fitness software and facilitated training for its use during the trial without cost.
We also acknowledge the contributions of the staff members at each of the 9 sites: Medical College of Georgia, University of Iowa, University of Massachusetts Medical School, University of Miami Miller School of Medicine, University of Minnesota, University of North Carolina at Chapel Hill, University of Pennsylvania School of Medicine, University of Texas Health Science Center at San Antonio, and Yale University School of Medicine