The strong association between cognitive impairment and functional disability, together with the treatment refractoriness of cognitive deficits, make the development of new therapies to enhance cognition in schizophrenia one of the most pressing challenges in psychopharmacology. An important step towards developing treatments targeting impaired cognition in schizophrenia came with the successful completion of the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative (1
). With renewed interest from the Pharmaceutical Industry and the support of the FDA, this remarkable effort has already contributed to a broader re-conceptualization of the treatment of schizophrenia that goes beyond relapse prevention and symptom control to focus on reducing disability and improving functional outcome. Through this MATRICS process a battery of tasks reflecting seven domains of cognitive functioning was developed (3
). The tasks selected were mostly pen and paper measures widely used in clinical neuropsychology and were selected based on criteria that experts indicated were essential for clinical trials. Thus, the measures needed to be brief (10-15 minutes) and have known strong psychometric properties (e.g., high test-retest reliability, limited practice effects, etc.). The existing literature on these characteristics and the performance of the measures compared to other similar tasks in a large scale prospective study played key roles in selection (4
). These measures have become widely used in clinical trials focusing on cognitive functioning in schizophrenia and will have helped to provide an energizing force for research designed to identify novel treatment approaches for cognition in schizophrenia.
During the MATRICS process, considerable discussion occurred regarding whether a different approach, using measures from cognitive neuroscience, should be used in developing the suggested battery. The cognitive neuroscience approach uses highly refined tasks that have been developed to examine the function of specific cognitive systems. These theoretical model systems have been conceptualized, specified in mechanistic terms and validated using exhaustive experimentation in which task parameters are varied and performance measured to test predictions arising from the models. Importantly, these constructs have been increasingly defined in terms of brain function, using non-invasive imaging techniques such as ERP and fMRI to characterize the functioning of well specified cognitive systems in the human brain. Further, many of these constructs have also been validated in animal models that have allowed very precise specification of the pattern of neural activity in supporting distinct cognitive mechanisms in specific brain regions and the neuropharmacological substrates of these mechanisms.
Given the experience that the field has with clinical neuropsychological tasks and their desirable psychometric properties, why should we be considering a shift to a more cognitively mechanistic approach? There are two critical reasons. The first is that that this cognitive approach allows us to examine the function of specific cognitive component processes. By this we mean that this approach allows us to distinguish the function of one specific cognitive system (e.g. specific aspects of working memory, long term relational memory, or different kinds of attention such as vigilance, focused attention and selective attention) from another, and to distinguish between these specific cognitive impairments and the generalized deficits that schizophrenia patients manifest as a result of reduced motivation, sedation from medications, general inattentiveness and poor test taking skills. Typically, clinical neuropsychological measures involve tasks that engage a combination of cognitive processes. The complexity of standardized clinical neuropsychological tests has direct implications for their ability to provide sensitive measurement of drug effects on cognition in schizophrenia. For example, if a drug has been developed in an animal model of working memory, it might actually improve this system but not show an effect on a complex task such as the Spatial Span (a working memory measure in the MATRICS battery) because the other cognitive systems that also support performance on this task are unaffected by the drug. Variance from cognitive mechanisms unaffected by the drug, but engaged by a task, may limit its sensitivity to drug effects on a specific cognitive system. An argument against this approach might be that if performance on the task predicts functional outcome, then it may be that improvement of all components of even complex tasks would be necessary in order to translate into improved functional outcome. However, it is possible that the relationship between performance on a multicomponential task and functional outcome is actually being driven by impairments in just one of the specific cognitive processes tapped by the more complex measure. Improvement in this specific process might then be linked to improved functional outcome, even if the other non-specific factors that could lower task performance were not improved. If so, then the use of complex cognitive tasks may mask improvement in the critical cognitive processes that would actually drive improved life function. The corollary to this is that, if a task can provide a relatively pure measure of an specific cognitive process, it is possible that it may increase the fidelity of the signal originating from the effects of a drug on its biological substrate (5
). This latter benefit is of course dependent on our ability to link cognitive systems with their biological substrates (including neuropharmacological targets), a knowledge base which while still a work in progress is developing rapidly at this time.
The second critical reason to identify and measure specific cognitive processes is that this is the approach that both animal modelers and human cognitive /affective neuroscientists use. Importantly, most early drug development and much of the basic science work that has led to the development of potentially procognitive drug therapies in schizophrenia and other disorders has used the technology of cognitive neuroscience. As such, using a technology in clinical trials that matches that used in the development and initial testing of these drugs may facilitate the transition to human testing and better assure positive results in early drug development. Further, non-invasive imaging studies using cognitive neuroscience methods can allow us to measure drug effects on cognitive and emotional processes and on its neural substrates (6
). fMRI may be used to measure brain activity in circuits supporting specific cognitive functions with a relatively high degree of anatomical specificity while other methods such as ERP and MEG can index the time course and magnitude of cognitive processing in the brain.
Despite the potential advantages of using an approach derived from the basic science literature, several important concerns arose during the MATRICS process regarding cognitive neuroscience based tools. The first was that there was no general consensus regarding which specific cognitive mechanisms were impaired in schizophrenia. A second concern was that there were often no standardized versions of the tasks that cognitive neuroscientists use to measure specific cognitive processes or the function of the brain circuitry supporting these functions. Third, it was noted that the psychometric properties of experimental cognitive tasks were largely unstudied and therefore unknown. Finally, in some cases, measures from cognitive neuroscience were considered, but their development had not emphasized psychometric properties optimal for detecting individual differences and cognitive change, so they were not selected for the final MATRICS battery. There is no question that poor measurement properties such as floor and ceiling effects and poor test re-test reliability would severely limit the usefulness of these tasks in the drug development process. At the last of the MATRICS meetings, the “New Approaches” meeting, these issues were revisited in depth and it was concluded that an important future research agenda would be to address these limitations so that a new brain based approach to measuring cognition in schizophrenia could be developed.
The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) conference series, funded by an R13 Award from the NIMH, is seeking to take a significant step forward on this path. One of the goals of the MATRICS process was to develop a an agreed upon cognitive battery that measured each of seven empirically derived cognitive domains in schizophrenia, based largely on factor analytic studies of neuropsychological task performance in this disorder (3
). The MATRICS selection process was tailored to select the best brief measures currently available for these domains, based on task properties such as known reliability, low practice effects, relationship to functional outcome, practicality, and tolerability (1
). In contrast, the purpose of CNTRICS was to select constructs that have prominence in current cognitive neuroscience and substantial promise for delineating elementary cognitive processes that may be more closely connected to neural systems. CNTRICS did not require that measures currently available have the psychometric properties required for selection for the MATRICS Consensus Cognitive Battery. Rather, CNTRICS emphasized section of cognitive constructs and associated tasks with the goal of promoting future research that would examine and improve (if need be) the psychometric characteristics of such tasks to facilitate their use in translational research and in clinical trials.
Guided by a Steering Committee of experts in basic cognitive neuroscience, clinical studies of cognition in schizophrenia, and animal pharmacology, the CNTRICS projects has as its goal the development of a set of experimental cognitive tasks that can be used behaviorally as well as in noninvasive functional imaging studies to enhance treatment development for impaired cognition in schizophrenia. This is being accomplished through a set of three consensus-building meetings, preceded by a set of interactive web based surveys to include a broader range of participants than can be accommodated at the meetings. The first of these meetings addressed the question of which component processes of cognition should be targeted for treatment development in schizophrenia. This meeting was held in February 2007 in Bethesda, Maryland. The second meeting, held in St. Louis in September 2007 focused on psychometric and practical issues of using experimental cognitive tasks to measure treatment effects on cognition in schizophrenia. The final meeting, which will be held in Sacramento, California in March 2008, will focus on identifying specific tasks that could profitably be used to measure treatment effects on cognition in schizophrenia and enhance translational research. These tasks will then need to undergo psychometric evaluation and are likely to be refined and optimized before they will meet psychometric criteria established during meeting 2. The status of the CNTRICS project may be found at http://cntrics.ucdavis.edu
. This manuscript and the accompanying 6 papers report the results of the first CNTRICS conference, in which we focused on developing an agreed upon set of cognitive systems and specific component processes of cognition to be targeted for measurement development.
The Development of Criteria Used to Select Cognitive Constructs and Mechanisms
One of the challenges for CNTRICS was to identify and select those constructs most relevant for understanding impaired function in schizophrenia. To aid in this effort, the CNTRICS executive committee developed an initial list of potential selection criteria (see ). Like the MATRICS Neurocognition Committee, we felt it important to involve as many individuals as possible in the process, as the FDA and the NIMH are more likely to benefit from the consensus views of a large group than the opinions of only a small subset of the field. We then use a web-based survey to ask individuals from a wide range of expertise domains to rank the criteria in terms of their relevance for the CNTRICS process. Experts included individuals from academia and industry, the basic and clinical sciences, as well as individuals with experience in clinical trials and cognitive rehabilitation in schizophrenia. We used several methods to generate the list of individuals asked to participate in the survey, including: 1) the names of those individuals who were involved in the MATRICS project; 2) individuals serving on the editorial boards of basic and clinical cognitive science, cognitive neuroscience, and schizophrenia related journals; 3) individuals from as many small and large pharmaceutical, biotechnology and other relevant industry organizations as could be identified by the CNTRICS steering committee.
Importance Ratings of Initial Criteria
We asked these individuals to rank each potential criterion on a 5-point scale, ranging from 0 (Not Necessary) to 4 (Very Essential). A total of 141 individuals completed this survey, and shows the results for the total sample as well as separately for those individuals from academia and industry. The intraclass correlation coefficient (ICC) using the mean of all raters (since that is the data upon which decisions were based), was .97 using the absolute value method. The ICC for just the academic raters was .97 as well. The ICC for just the industry raters was .86. As can be seen in , academic and industry participants rated many of the same criteria highly, including ease of measurement in humans, evidence for impairment in schizophrenia, links to functional outcome in schizophrenia, the clarity of the cognitive and neural mechanisms, and the existing links to neuropsychopharmacology. The six most highly rated criteria were used in a subsequent survey to assess the relevance of cognitive constructs to the CNTRICS process. In addition, the academic responders rated the amenability for use in human imaging studies as very helpful, and the industry respondents rated the availability of an explicit animal model as very helpful. Given that the CNTRICS process is focused on helping to translate paradigms from human cognitive neuroscience (which often uses fMRI techniques) and animal cognitive neuroscience, we felt it important to include these criteria as well. Even though there was not necessarily a clear discontinuity in the ratings of the first 6 and the remainder of the criteria, we felt that this was the maximum number of criteria we could reasonable ask individuals to rate for a range of cognitive constructs.
As the next step, the CNTRICS executive committee developed an initial list of cognitive constructs and mechanisms that have been studied in the basic human and animal cognitive neuroscience literature, and which we thought likely to be relevant to understanding functional outcome in schizophrenia (see ). Many of these constructs were phrased in terms of traditional parsing schemes for understanding subcomponents of cognitive domains. We recognized that the expert basic science presentations and discussions at the first CNTRICS meeting might suggest a different parsing of constructs and mechanisms. In fact, this is exactly what happened, and the cognitive constructs that were selected as most ready for translation at the first CNTRICS meeting were in many cases substantially different than those proposed during our initial survey. Further, we fully recognize that even the parsing and definitions developed at the first meeting are only one way of “carving cognition at its joints” and we do not intend our definitions of constructs and cognitive domains to be reified in any way. Rather they represent a contemporary cognitive theoretical starting point for the translational cognitive neuroscience process which is the core of the CNTRICS process and not an end unto themselves.
Pre-meeting Web Survey Based Ratings of Cognitive Constructs
Prior to the meeting we once again used a web-based survey to ask individuals from a wide range of expertise domains (selected in the same way as described for the first survey) to rank the pre-meeting constructs in terms of the degree to which they met each of the 8 criteria selected in the first web-based survey. shows the results of these ratings for the entire sample of 110 respondents. For each construct, we show the mean, median and modal response for each of the 8 criteria, as well as the number of individuals who felt that there was insufficient information to rate this construct (shown in parentheses). In addition, we show the Intra class correlation coefficient for each criterion, using the constructs within domain as the replication factor. As can be seen in , there was a good deal of variability in these initial ratings, with some constructs receiving relatively high ratings across most criteria and others receiving very low ratings across most criteria. Of note, relatively few of these constructs received high ratings on the criterion of “Linked to Functional Outcome in Schizophrenia.” However, many respondents noted that this was because of an absence of research on this question, rather than because of negative published findings. These ratings were provided to all individuals attending the first CNTRICS meeting as additional information to spur discussion and consensus building.
The meeting process itself consisted of a series of formal presentations by prominent cognitive neuroscientists (see ) to provide a state of the art perspective of the cognitive and neural architecture of six different cognitive systems, identified during the survey phase as showing impairments in schizophrenia. In preparing these presentations, the basic scientists were asked to focus on the state of the science in their field, but with an emphasis on constructs most likely to be relevant to understanding function in schizophrenia. In order to help these basic scientists to understand the literature on schizophrenia in their area of cognition, we asked a clinical researcher with schizophrenia expertise to serve as an advisor to the basic scientist. The basic scientists and clinical scientists involved in preparing these presentations are listed in . The talks were followed by breakout groups during which component processes to be targeted for measurement development (and eventually treatment development) were identified. These targeted mechanisms were further discussed, refined and prioritized at a wrap up session involving the entire group on the second day of the meeting. As can be seen in and noted above, the discussion among the experts present in the breakout groups suggested a different parsing of cognition that the CNTRICS steering committee has initially proposed. The cognitive constructs and definitions outlined in represent the agreement and opinions of the basic cognitive scientists present at this meeting and are an excellent launching point for the remaining CNTRICS meetings. Further, no assumptions were made of “orthogonality.” It was assumed that there would be overlapping mechanisms across one or more of the cognitive systems under consideration.
Basic Scientists and Clinical Advisors at Meeting 1
List of Constructs Selected as Most Ready for Translation
In the following six papers, the background talks and consensus-building results are summarized across the six cognitive systems that were considered at the meeting. Constructs were categorized as 1) Ready for measurement development (those constructs that the participants felt met many of the criteria well); 2) Promising, but in need of more basic cognitive neuroscience research to specify and validate the construct; 3) Promising, but role in schizophrenia needs more investigation; and 4) Not recommended for further measurement development. In total, eleven aspects of cognition across six different cognitive systems were endorsed by the participants as targets for measurement development (see ). Constructs that were felt to belong to the other three categories are also described in the remaining six manuscripts. Not surprisingly, across the different cognitive systems there was also a degree of overlap in component processes identified as high priority targets, suggesting that certain common underling cognitive deficits may be accounting for abnormal task performance across a range of cognitive systems. It is therefore likely that the true set of targeted mechanisms may be somewhat smaller than eleven, and that some of the various tasks to be developed may in fact engage a set of shared general-purpose mechanisms that are impaired on schizophrenia.
CNTRICS is a work in progress and at its completion a major challenge -- psychometric benchmarking and task optimization -- will remain to be completed. Thus, post-CNTRICS research and development activities will involve studies that will examine and enhance (where needed) the psychometric properties of tasks developed based on the modern tools of cognitive neuroscience, so that these tools can be used in future pro-cognitive treatment development in schizophrenia. However, the successful development of a set of cognitive neuroscience based tools should prove to be a valuable asset to those seeking to develop effective therapies for the cognitive deficits that disable many individuals with schizophrenia. As the CNTRICS process unfolds we hope to bring the full force of the new knowledge and technology that is cognitive neuroscience to bear on the effort to develop effective therapies for impaired cognition in schizophrenia.