PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Schizophr Res. Author manuscript; available in PMC 2012 March 1.
Published in final edited form as:
PMCID: PMC3050090
NIHMSID: NIHMS254579

The MCCB Impairment Profile for Schizophrenia Outpatients:Results from the MATRICS Psychometric and Standardization Study

Abstract

The MATRICS Psychometric and Standardization Study was conducted as a final stage in the development of the MATRICS Consensus Cognitive Battery (MCCB). The study included 176 persons with schizophrenia or schizoaffective disorder and 300 community residents. Data were analyzed to examine the cognitive profile of clinically stable schizophrenia patients on the MCCB. Secondarily, the data were analyzed to identify which combination of cognitive domains and corresponding cut-off scores best discriminated patients from community residents, and patients competitively employed vs. those not. Raw scores on the ten MCCB tests were entered into the MCCB scoring program which provided age-and gender-corrected T-scores on seven cognitive domains. To test for between-group differences, we conducted a 2 (group) × 7 (cognitive domain) MANOVA with follow-up independent t – tests on the individual domains. Classification and regression trees (CART) were used for the discrimination analyses. Examination of patient T-scores across the seven cognitive domains revealed a relatively compact profile with T-scores ranging from 33.4 for speed of processing to 39.3 for reasoning and problem-solving. Speed of processing and social cognition best distinguished individuals with schizophrenia from community residents; speed of processing along with visual learning and attention/vigilance optimally distinguished patients competitively employed from those who were not. The cognitive profile findings provide a standard to which future studies can compare results from other schizophrenia samples and related disorders; the classification results point to specific areas and levels of cognitive impairment that may advance work rehabilitation efforts.

Keywords: MCCB, cognition, schizophrenia, profile

1. Introduction

The MATRICS Consensus Cognitive Battery (MCCB) was developed to address the absence of a uniform, standardized method by which to measure cognition in clinical trials of cognition-enhancing drugs (Marder & Fenton, 2004). The MCCB, now accepted as a standard by the U.S. Food and Drug Administration, is comprised of ten tests that assess seven cognitive domains (speed of processing, attention/vigilance, working memory, verbal learning, visual learning, reasoning and problem solving, and social cognition) (Nuechterlein et al., 2004). Previous articles have detailed the process of MCCB development, co-norming, and evaluation of co-primary measures for clinical trials (Green et al., 2008; Kern et al., 2008; Nuechterlein et al., 2008). In this paper, we extend previous findings by examining the cognitive profile of chronic, clinically stable schizophrenia outpatients on the MCCB using data from the two studies involved in its development (Psychometric and Standardization Study, PASS Phase I and II). Secondarily, we address two discrimination questions relevant to cognitive dysfunction in schizophrenia. Specifically, which combination of MCCB cognitive domains and corresponding cut-off scores best discriminate: a) schizophrenia individuals from community residents, and b) vocational status within the schizophrenia group.

2. Methods

2.1 Participants

PASS Phase I was conducted to evaluate the psychometric properties of tests in the beta version of the battery (20 tests) and included 176 schizophrenia and schizoaffective disorder outpatients from five academic sites (Duke University, Harvard University, University of Kansas, Maryland Psychiatric Research Center, and UCLA) (Nuechterlein et al., 2008). PASS Phase II gathered normative data on the MCCB and included 300 community residents aged 20-59 from the same sites (Kern et al., 2008). Inclusion criteria for schizophrenia participants and community residents are described in earlier publications (Kern et al., 2008; Nuechterlein et al., 2008) and summarized below:

For schizophrenia participants, inclusion criteria included: (a) a DSM-IV diagnosis of schizophrenia or schizoaffective disorder, depressed subtype, based on SCID interview, (b) no medication changes in the previous month, (c) clinical stability, (d) age 18–65 years, (e) no substance dependence in the past 6 months, (f) no substance abuse in the past month, (g) no neurological disease or head injury, and (h) no substance use or excessive alcohol consumption in the days prior to testing or no excessive lifetime alcohol or substance use.

The community sample included representative numbers of persons according to gender, race/ethnicity, and educational attainment based on the 2000 U.S. Census. Inclusion criteria were: (a) no history of diagnosis of schizophrenia or other psychotic disorder, (b) no neurological disease or head injury, (c) no mental retardation or pervasive developmental disorder, (d) not currently taking any medications that may interfere with test performance (e.g., narcotics for pain), and (e) no recent alcohol or substance use or excessive lifetime alcohol consumption or substance use. Table 1 presents the demographic characteristics of the schizophrenia and community resident samples.

Table 1
Demographic and Clinical Characteristics.

2.2 Study procedures

All study participants were administered the ten tests that comprise the MCCB. Persons in the schizophrenia sample were administered a beta version that included the ten MCCB tests that were interspersed with ten other cognitive tests that were candidates for inclusion in the final battery. Community residents received only the ten tests that made up the final MCCB. Hence, the groups differed by the number of tests they were administered and test order. Table 2 includes the ten MCCB tests, according to domain, and their corresponding dependent measures. Schizophrenia participants were also administered the Brief Psychiatric Rating Scale (BPRS) (Lukoff et al., 1986) for assessment of psychiatric symptoms and the Birchwood Social Functioning Scale (SFS) (Birchwood et al., 1990) supplemented by sections from the Social Adjustment Scale (SAS) for assessment of community and work functioning (Weissman and Paykel, 1974). Employment status was determined from the SAS which distinguishes regular paid work from assisted work (e.g., job coach), supported work (e.g., sheltered workshop), volunteer work, and no work related activities based on activity over the past three months. Our analyses focused on distinguishing regular paid work from the other categories.

Table 2
MCCB Cognitive Domains, Tests, and Dependent Variables.

2.3 Statistical analyses

Statistical analyses were performed using SPSS 17.0. For the MCCB impairment profile, patients' raw scores from each of the ten MCCB tests were entered into the MCCB scoring program to produce age- and gender-corrected T-scores for the seven cognitive domains (normative mean = 50; standard deviation = 10) 1. These descriptive data allow for a straightforward interpretation of severity of cognitive impairment against norms representative of the demographic make-up of persons in the U.S. as described in the 2000 Census Report. To test whether the patient profile was significantly different from that of community residents, the data were analyzed using a one-way MANOVA with group (schizophrenia participants vs. community residents) as the between-subjects variable and age- and gender-corrected T-scores from the seven cognitive domains as the dependent variables. Follow-up contrasts (parallel to two sample t - tests) were conducted on each of the respective cognitive domains. Within the schizophrenia group, additional follow-up contrasts were conducted to determine whether there were any domains of relative strength or weakness by comparing the average performance level on each cognitive factor with the mean of the remaining six.

To examine which combination of cognitive domains and corresponding cut-off scores best distinguished persons with schizophrenia from community residents, data were analyzed using classification and regression trees (CART) (Breiman et al., 1998). CART is a data mining technique that derives decision trees to predict values of continuous (regression) or categorical (classification) outcomes from a set of predictor variables. In contrast to traditional regression or discriminant function analyses, CART is non-parametric and does not assume a monotonic relationship between predictors and outcomes. Trees branch and grow iteratively by identifying optimal cut-points for key discriminating variables in the predictor set. For classification problems, the tree grows until a stopping criterion is met or no further improvement in correct classification of study participants is possible. With this distribution-free and flexible approach, CART can often represent a complex set of overlapping predictor variables and achieve good classification with a few simple “if-then” rules. CART was also applied to address classification of persons with schizophrenia on vocational status. For both analyses, we used uncorrected T-scores to allow examination of age and gender as separate predictors. The predictor variables for these analyses included T-scores for the seven MCCB domains plus the demographic variables age and gender. We also examined each MCCB domain for their overall importance/centrality to the prediction model. This measure provides information on the robustness of selected variables to serve as a proxy for other measures within the tree.

3. Results

3.1 MCCB impairment profile

The age- and gender-corrected MCCB cognitive profile of the schizophrenia group is illustrated in Figure 1. T-score means ranged from 33.4 for processing speed (greatest impairment) to 39.3 for reasoning and problem-solving (least impairment). The MANOVA results revealed an overall effect of diagnostic status with the schizophrenia group significantly impaired relative to the community resident group (F(7,448) = 56.04; p < .001). Schizophrenia participants showed significant impairment relative to community residents on each of the seven MCCB cognitive domains (all ps < .001). Within the schizophrenia group, the outlier analyses that compared each cognitive domain with the mean of the remaining six revealed speed of processing and working memory to be most impaired (t(167) = 6.241; p < .001; t(167) = 2.302; p = .023, respectively) and reasoning and problem-solving to be least impaired (t(167) = 4.384; p < .001).

Figure 1
Cognitive Impairment Profile of Schizophrenia Individuals on the MCCB (age- and gender-corrected T-scores).

3.2 CART analyses for schizophrenia vs. community resident discrimination

The CART results for discrimination of persons with schizophrenia from community residents are presented in Figure 2 along with the independent variable importance for each of seven MCCB domains. A cut-off T-score of 43.9 (27th percentile) on speed of processing splits the sample into two groups, one of which is 87.6% community residents and the other of which is 66.4% persons with schizophrenia. Both these groups are better discriminated than the original sample which was 63% (n=300) controls and 37% (n=176) patients. Viewed as a classification rule, the new model correctly identifies 81.8% (144/176) patients and 75.7% (227/300) controls or 77.9% of participants overall. Discrimination was further improved with the additional consideration of social cognition performance which resulted in one node that was 92.4% community residents; another node that was 79.4% schizophrenia patients and two nodes that were relatively evenly split. Group discrimination was not further improved by other MATRICS scores or the demographic variables of age and gender. In sum, persons with schizophrenia were best distinguished from community residents based on their processing speed which was further improved with consideration of level of social cognition performance.

Figure 2
Results from CART Analyses for Discrimination of Schizophrenia Individuals vs. Community Residents.

3.3 CART analyses for employment status within the schizophrenia sample

Figure 3 presents the CART results for discriminating patients who were in competitive employment from those who were not as well as the independent variable importance for each of the seven MCCB domain predictor variables included in the model. A cut-off T-score of 30.5 (2nd percentile) on speed of processing splits the sample into two groups, one of which is 89.4% non-workers and 34.5% workers. Both these groups are better discriminated than the original sample of patients which included 74.4% (n=131) non-workers and 25.6% (n=45) workers. Viewed as a classification rule, the new model correctly identifies 45.0% (59/131) of non-workers and 84.4% (38/45) of workers or 55.1% of patients overall. Discrimination was further improved with the additional consideration of visual learning and attention/vigilance. On the left hand side of the tree, a cut-off T-score of 43.0 (24th percentile) on the BVMT-R (visual learning) resulted in one node that was 94.6% non-workers. On the right hand side, a cut-off T-score of 47.9 (42nd percentile) on the CPT-IP (attention/vigilance) resulted in one node that was 50.0% workers. Discrimination was not further improved by other MATRICS scores or the demographic variables of age and gender. In sum, these findings indicate that marked impairments in processing speed best distinguished non-workers from workers, and the additional consideration of visual learning ability further discriminated the two. In contrast, workers were less clearly distinguished from non-workers by level of cognition.

Figure 3
Results from CART Analyses for Discrimination of Schizophrenia Individuals according to Employment Status.

4. Discussion

These findings are the first reported on the MCCB impairment profile of schizophrenia outpatients from the sample used in developing this test battery. As such, they provide a standard of comparison for future studies examining the profile of cognitive impairment in other schizophrenia samples and comparison groups as well as clinical trials testing the efficacy of cognition-enhancing drugs. In interpreting these data, it is important to note that the patient and community resident samples included in MCCB development were designed to be broadly representative of adults in the U.S. and included men and women from a wide range of age and education, differing racial and ethnic backgrounds, and urban and rural settings. Also, a key feature in developing the MCCB was the co-norming of tests in the final battery. Although a number of the individual tests that comprise the MCCB are widely used neuropsychological tests with normative data, this combination of tests had never been administered together as a single unit and co-normed in a representative community sample. It should also be noted that the component tests of the MCCB were selected with a focus on their use within the context of clinical trials and were evaluated based on test-retest reliability, utility as a repeated measure, relationship to functional outcome, practicality and tolerability, and more broadly sensitivity to change. Hence, other measures might be more ideal to characterize the cognitive impairments of schizophrenia outside of this context.

Results from the cognitive profile analyses revealed schizophrenia patients to be impaired relative to community residents on each of the seven MCCB cognitive domains. The range of impairment across domains was relatively compact with the breadth of impairment severity covered by six-tenths of a standard deviation. Of the seven MCCB cognitive domains, speed of processing and working memory were most impaired (4th and 7th percentiles, respectively). Greater impairment observed in these domains may be associated with the fact that they were measured using multiple tests and may have greater reliability relative to other MCCB domains assessed by a single test. In terms of generalizability of these findings to other schizophrenia samples, a Norwegian study (Holmen et al., 2010) of early onset schizophrenia spectrum disorder patients found a z-score impairment range of .8 to 1.8 on the MCCB with the curious exception of social cognition that was found to be equivalent with the study's normal comparison group. It should be noted that z-scores were derived relative to the study's normal control group not the MCCB normative sample. At the other end of the age spectrum, it is not known if these profile findings apply to older patient samples.

The findings for speed of processing are in keeping with independent investigations and meta-analyses examining a commonly used measure of processing speed, digit symbol type tests. In a study of 127 schizophrenia patients and 127 demographically matched controls (Palmer et al., 2010), processing speed measured by digit symbol and symbol search tests was found to show greater levels of impairment than cognitive measures of verbal comprehension, perceptual organization, working memory, and auditory memory. Similarly, in a meta-analysis of 37 studies, the digit symbol test was found to yield a large mean effect size (g = 1.57) and the magnitude was greater than that found for measures of episodic memory, working memory, and executive functioning (Dickinson et al., 2007). In a separate meta-analysis of 43 first episode schizophrenia samples, Cohen's d was 1.59 for the digit symbol test, remarkably consistent with that found in more chronic samples (Mesholam-Gately et al., 2009). In the present analyses, Cohen's d for the BACS symbol coding test was 1.35. For working memory, meta-analyses and reviews (Aleman et al., 1999; Heinrichs and Zakzanis, 1998; Lee and Park, 2005) indicate a mean effect size ranging between .45 to .82 which are considerably lower than we found. However, a number of studies in the meta-analyses examined experimental tasks of working memory that may have had lower task difficulty levels than the ones included in the MCCB. A recent memory study of chronic schizophrenia and schizoaffective disorder outpatients that included the MCCB measure of working memory, Letter-Number Span, reported more comparable levels of impairment (Cohen's d = 1.17) (Kern et al., 2010).

Although processing speed and working memory were found to be the most impaired areas of cognition relative to other domains, these findings should be interpreted in the context of the method of measurement used to derive T-scores by the MCCB scoring program. Processing speed and working memory are the only MCCB domains assessed by more than one test. Including multiple tests to assess a particular cognitive domain improves reliability and ability to capture the breadth of a construct. However, a form of measurement bias can occur when comparisons are made with domains assessed by a single test. This is because T-scores for domains which include multiple tests are renormalized versions of the sums of T-scores from the component tests. This ensures that the domain scores have a common metric in the sense that their means and standard deviations in the normative sample are the same regardless of the number of component tests. However, the resulting patients' T-scores for domains assessed by multiple tests tend to be lower than those assessed by a single test. This measurement artifact occurs for the following reason. If the component tests are perfectly correlated, the renormalization results in the mean difference between the patient and control domain scores being the average of the group differences on the individual tests. Otherwise, the group difference in the domain score is adjusted by a factor that increases as a) the correlations between the component tests decrease and b) the number of tests increases. Intuitively, this is because when the correlations are imperfect, each test contributes additional unique variance in defining between group differences. As a group, schizophrenia patients generally perform worse than the normative sample on the individual MCCB tests, and the correlations between tests within a given domain (e.g., speed of processing) are less than 1.0 since the tests were designed to capture different facets of functioning. As a result, the combined patient and control group score distributions shift further apart with the inclusion of each additional test, and interpretation of findings between MCCB domains must include consideration of this measurement artifact. However, pertinent to interpretation of the present findings, it is noteworthy that the mean T-scores for the individual tests most representative of speed of processing and working memory were only slightly higher (showing less impairment) than the T-score composites for the corresponding domain (BACS symbol coding = 35.2 (12.0) vs. speed of processing = 33.4 (11.9); Letter-Number Span = 36.7 (12.0) vs. working memory = 35.4 (12.1)).

Speed of processing and social cognition best distinguished persons with schizophrenia from community residents. The finding for speed of processing is not surprising. Symbol coding measures are among the most sensitive to detection of cognitive dysfunction (Lezak, 1995; Wechsler, 2008). These measures have a longstanding history in detecting early signs of cognitive impairment across a wide range of neurological disorders (Storandt and Hill, 1989; Strauss and Brandt, 1986). The test places demands on cognitive processes involved in sustained attention, working memory, graphomotor speed, as well as strategy formation (Glosser et al., 1977). One hypothesis about its sensitivity as a measure of cognitive impairment is based upon the number of cognitive processes involved in task performance. That is, because the test is polyfactorial, its sensitivity may be increased relative to measures that involve fewer performance limiting processes. Patient vs. community resident discrimination was further improved by the inclusion of a measure of social cognition, the MSCEIT Managing Emotions branch. Though this test assesses a narrow element of social cognition, perhaps impairment in this area is particularly central to schizophrenia. Alternatively, the MSCEIT's unique algorithm-based scoring method, which differs from other MCCB tests, may have contributed to its importance as a discriminator.

Speed of processing, visual learning, and attention/vigilance contributed to distinguishing persons with schizophrenia who were competitively employed vs. those who were not. Interestingly, the combination of cognitive domains and cut-off scores that identified workers differed from those that identified non-workers. Marked impairments in processing speed (below the 2nd percentile) along with impairments in visual learning ability best distinguished non-workers from workers. In contrast, level of cognitive functioning appeared less critical to distinguishing workers from non-workers. Other studies have also found processing speed, learning and memory, and attention to be related to employment status with a possible role for executive functioning as well (Bellack et al., 1999; Bryson and Bell, 2003; Evans et al., 2004; Gold et al., 2002; Lysaker et al., 2005; Milev et al., 2005), and the findings for processing speed may extend more broadly to prediction of a number of areas of functional outcome (Harvey et al., 2009). Bellack et al. (1999) found persons with schizophrenia with good vocational histories (GVH) to perform better from those with poor vocational histories (PVH) on a broad array of cognitive tests assessing processing speed, learning and memory, attention, and executive functioning, as well as general intelligence. Classification of GVH vs. PVH using discriminant analyses revealed that cognitive measures that best identified GVH differed from those for PVH, a finding similar to ours. Based on these findings, it appears easier to identify cognitive determinants of unemployment than it is to identify them for employment.

Acknowledgements

The authors are grateful to the study participants for their time and effort devoted to participation in this study; and we wish to thank the research staff at the five performance sites for their work in recruitment, testing, scoring, and data management. We thank Kellie M. Smith, M.A. for her assistance in the preparation of the manuscript.

Role of funding source

Funding for the MATRICS Initiative was provided through Contract N01MH22006 from the National Institute for Mental Health to the University of California, Los Angeles (Dr. Marder, PI; Dr. Green, Co-PI; Dr. Fenton, Project Officer). Funding for this study came from an Option (Dr. Green, PI; Dr. Nuechterlein, Co-PI) to the NIMH MATRICS Initiative. The NIMH had no further role in study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of interest

Author Kern is an officer for MATRICS Assessment, Inc. and receives financial compensation for his role within the non-profit organization; Authors Green and Nuechterlein are officers within MATRICS Assessment, Inc. but do not receive any financial remuneration for their respective roles. No other authors have any conflicts of interest with respect to this manuscript.

Contributors

Authors Green, Nuechterlein, and Marder designed the study and wrote the protocol. Authors Sugar, Lee, and Kern performed the data analyses. Author Kern wrote the first draft of the manuscript with contributions from authors Gold and Dickinson. All authors contributed to and have approved the final manuscript.

1Note: The MCCB scoring program uses a regression-based approach for determining ageand gender-corrected T-scores based on a linear effects model. The MCCB scoring program will produce T-scores for individuals outside the age range of the normative sample by extending these linear age effects. For small extension beyond the normative age range, this modeling is likely still appropriate. However, we caution assumption of this linear correction for ages far outside the normative range.

References

  • Aleman A, Hijman R, de Haan EHF, Kahn RS. Memory impairment in schizophrenia: A meta-analysis. Am. J. Psychiatry. 1999;156:1358–1366. [PubMed]
  • Bellack AS, Gold JM, Buchanan RW. Cognitive rehabilitation for schizophrenia: Problems, prospects, and strategies. Schizophr. Bull. 1999;25:257–274. [PubMed]
  • Birchwood M, Smith J, Cochran R, Wetton S, Copestake S. The social functioning scale: the development and validation of a new scale of social adjustment for use in family intervention programs with schizophrenic patients. Br. J. Psychiatry. 1990;157:853–859. [PubMed]
  • Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and Regression Trees. Chapman & Hall/CRC; Boca Raton, FL: 1998.
  • Bryson G, Bell MD. Initial and final work performance in schizophrenia: cognitive and symptom predictors. J. Nerv. Ment. Dis. 2003;191:87–92. [PubMed]
  • Dickinson D, Ramsey ME, Gold JM. Overlooking the obvious: a meta-analytic comparison of digit symbol coding tasks and other cognitive measures in schizophrenia. Arch. Gen. Psychiatry. 2007;64:532–542. [PubMed]
  • Evans JD, Bond GR, Meyer PS, Kim HW, Lysaker PH, Gibson PJ, et al. Cognitive and clinical predictors of success in vocational rehabilitation in schizophrenia. Schizophr. Res. 2004;70:331–342. [PubMed]
  • Glosser G, Butters N, Kaplan E. Visuoperceptual processes in brain damaged patients on the Digit Symbol Substitution Test. Int. J. Neurosci. 1977;7:59–66. [PubMed]
  • Gold JM, Goldberg RW, McNary SW, Dixon L, Lehman AF. Cognitive correlates of job tenure among patients with severe mental illness. Am. J. Psychiatry. 2002;159:1395–1401. [PubMed]
  • Green MF, Nuechterlein KH, Kern RS, Baade LE, Fenton WS, Gold JM, et al. Functional co-primary measures for clinical trials in schizophrenia: Results from the MATRICS Psychometric and Standardization Study. Am. J. Psychiatry. 2008;165:221–228. [PubMed]
  • Harvey PD, Keefe RS, Patterson TL, Heaton RK, Bowie CR. Abbreviated neuropsychological assessment in schizophrenia: prediction of different aspects of outcome. J. Clin. Exp. Neuropsychol. 2009;31:462–471. [PMC free article] [PubMed]
  • Heinrichs RW, Zakzanis KK. Neurocognitive deficit in schizophrenia: A quantitative review of the evidence. Neuropsychology. 1998;12:426–445. [PubMed]
  • Holmen A, Juuhl-Langseth M, Thormodsen R, Melle I, Rund BR. Neuropsychological profile in early-onset schizophrenia-spectrum disorders: Measured with the MATRICS battery. Schizophr. Bull. 2010;36:852–859. [PMC free article] [PubMed]
  • Kern RS, Hartzell AM, Izaguirre B, Hamilton AH. Declarative and nondeclarative memory in schizophrenia: What is impaired? What is spared? J. Clin. Exp. Neuropsychol. in press. [PubMed]
  • Kern RS, Nuechterlein KH, Green MF, Baade LE, Fenton WS, Gold JM, et al. The MATRICS Consensus Cognitive Battery, part 2: co-norming and standardization. Am. J. Psychiatry. 2008;165:214–220. [PubMed]
  • Lee J, Park S. Working memory impairments in schizophrenia: A meta-analysis. J. Abnorm. Psychol. 2005;114:599–611. [PubMed]
  • Lezak MD. Neuropsychological Assessment. 3rd ed. Oxford University Press; New York: 1995.
  • Lukoff D, Nuechterlein KH, Ventura J. Appendix A: Manual for the Expanded Brief Psychiatric Rating Scale (BPRS) Schizophr. Bull. 1986;12:594–602.
  • Lysaker PH, Bryson GJ, Davis LW, Bell MD. Relationship of impaired processing speed and flexibility of abstract thought to improvements in work performance over time in schizophrenia. Schizophr. Res. 2005;75:211–218. [PubMed]
  • Marder SR, Fenton WS. Measurement and treatment research to improve cognition in schizophrenia: NIMH MATRICS Initiative to support the development of agents for improving cognition in schizophrenia. Schizophr. Res. 2004;72:5–10. [PubMed]
  • Mesholam-Gately RI, Giuliano AJ, Goff KP, Faraone SV, Seidman LJ. Neurocognition in first-episode schizophrenia: A meta-analytic review. Neuropsychology. 2009;23:315–336. [PubMed]
  • Milev P, Ho BC, Arndt S, Andreasen NC. Predictive values of neurocognition and negative symptoms on functional outcome in schizophrenia: A longitudinal first-episode study with 7-year follow-up. Am. J. Psychiatry. 2005;162:495–506. [PubMed]
  • Nuechterlein KH, Barch DM, Gold JM, Goldberg TE, Green MF, Heaton RK. Identification of separable cognitive factors in schizophrenia. Schizophr. Res. 2004;72:29–39. [PubMed]
  • Nuechterlein KH, Green MF, Kern RS, Baade LE, Barch DM, Cohen JD, et al. The MATRICS Consensus Cognitive Battery, part 1: Test selection, reliability, and validity. Am. J. Psychiatry. 2008;165:203–213. [PubMed]
  • Palmer BW, Savla GN, Fellows IE, Twamley EW, Jeste DV, Lacro JP. Do people with schizophrenia have differential impairment in episodic memory and/or working memory relative to other cognitive abilities? Schizophr. Res. 2010;116:259–265. [PMC free article] [PubMed]
  • Storandt M, Hill RD. Very mild senile dementia of the Alzheimer type: II. Psychometric test performance. Arch. Neurology. 1989;46:383–386. [PubMed]
  • Strauss ME, Brandt J. Attempt at preclinical identification Huntington's disease using the WAIS. J. Clin. Exp. Neuropsychol. 1986;8:210–218. [PubMed]
  • Wechsler D. Wechsler Adult Intelligence Scale - Fourth Edition: Technical and interpretive manual. Pearson Assessment; San Antonio, TX: 2008.
  • Weissman M, Paykel E. The Depressed Woman: A Study of Social Relationships. University of Chicago Press; Chicago, IL: 1974.