PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Psychiatr Res. Author manuscript; available in PMC 2009 June 15.
Published in final edited form as:
PMCID: PMC2696336
NIHMSID: NIHMS42158

Usefulness of the UCSD Performance-based Skills Assessment (UPSA) for Predicting Residential Independence in Patients with Chronic Schizophrenia

Abstract

The objective of this study was to examine the sensitivity and specificity of a performance-based measure of functional capacity, the UCSD Performance-Based Skills Assessment (UPSA) for the prediction of independent living status in patients with chronic schizophrenia-related conditions. A sample of 434 adults with schizophrenia or schizoaffective disorder was administered the UPSA and assessed for independent living status. Participants were classified as “independent” if they were living alone in an apartment, house, or single-resident occupancy (e.g., hotel room) and non-independent if they resided in a care facility (e.g., Board-and-Care home, Skilled Nursing Facility). Receiver Operator Characteristic (ROC) curves were calculated with the UPSA and Mattis’ Dementia Rating Scale (DRS) scores as predictor variables and residential independence as the state variable. Of the 434 participants, 99 (23%) were living independently at the time of assessment. The discriminant validity of the UPSA was adequate (ROC area under the curve = 0.74; 95% CI: 0.68–0.79), with greatest dichotomization for the UPSA at a cutoff score of 75 (68% accuracy, 69% sensitivity, 66% specificity), or 80 (68% accuracy, 59% sensitivity, 76% specificity). The UPSA was also a significantly better predictor of living status than was the DRS, based on ROC (z = 2.43, p = .015). The UPSA is a brief measure of functional capacity that predicts the ability of patients with schizophrenia to reside independently in the community.

Keywords: Schizophrenia, Independence, Functional Outcomes, Cognitive Function

1. Introduction

Many individuals suffering from schizophrenia are not capable of residing independently in the community, instead residing under assisted-living arrangements such as in a Board and Care (B&C) homes or skilled nursing facilities. Although it is likely that multiple factors play a role in independent living skills, research suggests that the cognitive impairments associated with schizophrenia limit the patients’ abilities to live and function in the community. For example, recent research has examined various determinants of real-world functional performance in patients with schizophrenia, including the severity of neurocognitive deficits. These studies indicate that neurocognitive functioning is related to interpersonal skills, engagement in community activities, and work skills (Bell & Bryson, 2001; Bowie et al., 2006; Bryson & Bell, 2003; Bryson et al., 1998). For example, Green et al. showed that neurocognitive deficits significantly impaired patients’ daily functioning abilities (Green, 1996; Green et al., 2000), including success in psychosocial skill acquisition (e.g., learning basic conversation skills and medication management), acquisition of instrumental skills and social problem-solving ability (e.g., role-playing performance of skills), and community outcomes/daily activities (e.g., occupational functioning, social attainment, degree of independent living). Although their review and meta-analysis suggested that neurocognitive functioning was the single best predictor of functional outcome, Green and colleagues (2000) called for more research identifying how neurocognition is related to functional outcome. Recent research has sought to answer this question through examining functional capacity. These studies have examined the ability to perform specific, functionally relevant, everyday living skills in controlled assessment settings. In a study of older patients with schizophrenia, Bowie et al. (2006) examined the role of neurocognitive functioning and functional skills capacity in predicting real-world functional performance (e.g., personal care skills, interpersonal skills, community activities) as rated by case managers with extensive contact with the patients. Functional capacity was measured with the UCSD Performance-based Skills Assessment (UPSA; Patterson et al., 2001), a performance-based measure assessing the ability to adequately perform a number of everyday functional tasks such as comprehending written material, planning activities, and managing money. The results indicated that while neurocognitive functioning significantly predicted everyday functional outcomes as rated by the case managers, this relationship was mediated, in large part, by performance-based measures of functional capacity.

Although these studies indicate that neurocognitive functioning, functional capacity, and “real-world” functional performance are interrelated, few studies have examined the types of concrete outcomes that are relevant to healthcare providers and medical decision-makers, such as the ability to live independently. Along these lines, Revheim and Medalia (2004b) examined the relationship between cognitive functioning, problem-solving skills, and community status (inpatient vs. outpatient) in a sample of 162 individuals diagnosed with schizophrenia or schizoaffective disorder. They found that cognitive functioning was correlated with problem-solving ability, and that problem-solving skills best predicted residential status. Similarly, Twamley et al. (2002), in a study of outpatients with schizophrenia, found that the UPSA performance was correlated with neuropsychological performance, and that UPSA scores significantly predicted level of independence in living.

The ultimate functional expression of problem-solving and cognitive abilities, that is, residing independently in the community, may rest heavily on the ability to perform basic functional tasks such as paying bills, scheduling medical appointments, complying with a work schedule, and shopping for food and household supplies. The development of a “gold standard” measure that captures these domains and is capable of accurately predicting the ability to reside independently in the community is needed for healthcare providers and family members who must make decisions regarding safe living situations for patients (e.g., whether to discharge an inpatient with schizophrenia to a B&C facility or independent living environment). This becomes particularly important when considering the “deinstitutionalization” of people with mental illness over the past 50 years, including the release of persons residing in psychiatric hospitals to alternative facilities in the community (Lamb & Bachrach, 2001). Indeed, Lamb and Bachrach (2001) report that between 1955 and 1998, the number of occupied state hospital beds in the United States was reduced from 339 per 100,000 population to 21 per 100,000 population. In California, the pioneer in deinstitutionalization, this number was 3 per 100,000 (Lamb et al., 2001).

As discussed by Obuchowski (2005), the preferred method for providing clinically useful information is the use of ROC analysis, which has several advantages over other methods. First, the ROC curve provides much more information about how a test performs than just a single estimate of the test’s sensitivity and specificity. Second, ROC curves allow clinicians to examine the costs and benefits of a test’s sensitivity and specificity for various cutoff scores on the test. Third, upon weighing these costs and benefits, clinicians can choose the optimal cutoff for each patient. In addition to providing clinically meaningful information, ROC curves are often the only valid method of comparing two or more diagnostic tests.

The purpose of this study was to assess the ability of the UPSA, a brief measure of functional capacity, to independently predict the level of residential independence in 434 patients diagnosed with primary psychotic disorders (i.e., schizophrenia or schizoaffective disorder). We further compared the UPSA with other measures possibly related to residential independence, such as measures of positive and negative symptoms, depressive symptoms, and neurocognitive performance. As discussed above, comparison of these measures for predicting residential independence was assessed by means of ROC curves. We hypothesized that the UPSA would be significantly better than these measures for predicting residential independence.

2. Methods

2.1. Participants

Four hundred thirty four individuals diagnosed with either schizophrenia or schizoaffective disorder were the participants for this study. All subjects were ongoing research participants at the University of California, San Diego (UCSD) Advanced Center for Interventions and Services Research (ACISR), which examines, among other aspects, functional abilities of these patients. Diagnosis of schizophrenia-related disorders was established by DSM-IV-based chart diagnosis using the patients’ research charts. All participants volunteered and provided written, informed consent to participate in research studies.

2.2. Measures

2.2.1. Functional Capacity

All the participants were administered the UPSA (Patterson et al., 2001), which assesses a person’s ability to perform a variety of everyday living tasks. The UPSA involves role-play tasks similar in complexity to situations that a community-dwelling person is likely to encounter. Patients are first tested in the area of Comprehension/Planning and are asked to read two stories. The first describes an outing to the beach on a hot sunny day and the second depicts a trip to the zoo to see the giant pandas on a cold rainy day. After reading the stories, participants are asked a few questions to evaluate their comprehension of the material and then requested to list five items necessary to bring or wear in order to spend the whole day at the beach/zoo. Points are given for answers deemed appropriate (e.g., swimsuit, towel, picnic lunch, sunscreen, etc. for the beach and umbrella, raincoat, camera, money, walking shoes, etc. for the zoo). This part of the assessment takes about eight minutes to complete.

The area of Finance tests one’s ability to count change and write checks. In the first part of this task patients are provided with real currency (coins and bills) and asked to count out given amounts (e.g., $12.17, $6.73, $1.02) and make change from ten dollars. The second part of the task involves filling out a check to pay a utility bill. Patients are shown a real bill from a utility company (e.g., San Diego Gas and Electric) and are required to make out a check. Points are given for each correct element of this process (e.g., the check is made out to SDG&E, the written amount corresponds to the bill, the check is signed, etc.). These tasks take about eight minutes to complete.

The next area tested is Communication. Patients are provided with an unplugged telephone and asked to role-play a number of scenarios. First they are asked to show the tester what number they would dial if they had an emergency. The appropriate response is to dial 9-1-1. A second task involves calling information to get a specific telephone number and then dialing that number from memory. Next, patients are requested to read a medical appointment confirmation letter and then role-play calling the hospital to reschedule the doctor’s appointment. In addition, subjects are asked to describe how the letter requested them to prepare for the medical appointment (e.g., fast for a blood draw) and what two items they need to bring with them to the doctor (e.g., insurance card and list of medications). There are a total of nine communication subtasks that require about five minutes to complete.

The area of Transportation includes the use of public transportation. Subjects are provided with bus schedule information from the San Diego Transit District and asked questions about the cost of their fare, the telephone number for schedule information, which bus to ride to a specific destination, where they would get off the bus to transfer to another bus, and the location of the trolley stations on a map. This task requires about five minutes to complete.

The final task measures Household Management skills. Patients are provided with a recipe for rice pudding and asked to prepare a written shopping list. They are then presented with an array of 29 items that one possibly would have on hand in their pantry (e.g., pasta, jelly, cereal, soup, rice, canned tuna, toothpaste, canned vegetables, crackers, etc.). Patients are requested to read the recipe, check the pantry, and then prepare a list of the missing items they need to buy in order to make the rice pudding. Points are given for each correct item on the shopping list. This task is timed for five minutes. Overall, the UPSA requires approximately 30 minutes to administer and to score in total. Inter-rater reliability of ratings is excellent (Patterson et al., 2001). Participants receive scores for each of the 5 subscales (range = 0–20), which are summed to create a summary score ranging from 0 to 100. Previous research has established that the UPSA is highly related to neurocognitive functioning (Bowie et al., 2006; Twamley et al., 2002) and performance on a number of functional abilities (Bowie et al., 2006).

2.2.2. Cognitive Performance

Mattis’ Dementia Rating Scale (DRS; Mattis, 1973) was administered as an assessment of overall cognitive functioning. This scale consists of a number of subtests assessing various cognitive abilities including attention, initiation/perseveration, constructional abilities, conceptualization, and memory. Scores on these scales are summed to create an overall score (maximum score = 144), with higher scores indicating better functioning.

2.2.3. Clinical Symptoms

The Positive And Negative Syndrome Scale (PANSS; Kay et al., 1987) was used to assess positive and negative symptoms of psychosis. The PANSS is a 30-item instrument with each item rated on a seven-point severity scale. Seven items are summed to reflect overall positive symptoms, another seven contribute to the negative syndrome score.

2.2.4. Depressive Symptoms

The Hamilton Rating Scale for Depression (HAM-D; Hamilton, 1969) was administered to evaluate patients’ severity of depressive symptoms. A trained observer provided ratings and on a 0-to-4 or 0-to-2-point scale on each of the 17 items, with higher scores indicating greater depressive symptoms. The HAM-D is a brief yet comprehensive measure of depressive symptoms, with well documented reliability and validity (Hamilton, 1969; 1974).

2.2.5. Classification of Living Status

Participants were classified as “independent” if they were living alone in an apartment, house, or single-resident occupancy (e.g., hotel room) and non-independent if they resided in a care facility (e.g., Board and Care, Skilled Nursing Facility). Our dataset also included 134 individuals who were residing in an apartment or house with another individual (e.g., spouse, family member, etc). However, they were not included in the present analysis because we were unable to determine how much assistance these patients were provided by their “roommates”.

2.2.6. Antipsychotic Medication Doseage

We conducted chart reviews of medication use (i.e., amounts, types, and frequencies of antipsychotic medication use). Typical and atypical medications were converted to mg chlorpromazine equivalents using published formulae (Jeste & Wyatt, 1982; Woods, 2003).

2.3. Statistical Analysis

A Receiver Operating Characteristic (ROC) curve was plotted for the UPSA. The ROC curve shows the sensitivity versus one minus the specificity for every possible cutoff point; optimal cutoff points are determined by visually assessing which score combines maximum sensitivity and specificity. The area under curve (AUC) with 95% confidence intervals was used as an indicator of the ability of the UPSA to differentiate patients who were living independently in the community (e.g., living alone in an apartment) or not living independently (e.g., living in a B&C facility). In addition, we used the methods described by Hanley and McNeil (1982; 1983) to compare the AUCs for other measures potentially predictive of residential independence (e.g., cognitive performance; positive, negative, and depressive symptoms). Finally, using the cutoff established by the AUC analysis, we conducted additional analyses to describe characteristics (e.g., age, cognitive functioning) also prevalent in those scoring above and below this cutoff.

3. Results

Characteristics of those living independently and non-independently are presented in Table 1.

Table 1
Comparison of Participants in Independent vs. Non-Independent Living Situations

Of the 434 participants, 99 (23%) were residentially independent. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and percent correctly identified as residentially independent for different cutoff scores on the UPSA are shown in Table 2. Briefly, sensitivity refers to how good the test is at correctly identifying people who reside independently, whereas specificity is the test’s ability to correctly identify people who are not independent (Loong, 2003). Similarly, positive predictive value is the proportion of patients with positive test results who are correctly classified as independent, whereas negative predictive value is the proportion of patients with negative test results who are correctly classified as dependent (Altman & Bland, 1994). The ROC curve is shown in Figure 1. The estimated AUC for the UPSA was 0.74 (95% CI: 0.68–0.79), which was significantly greater (p < .001) than the area of no information (an area of 0.50). As seen in Table 2, the best UPSA cutoff scores were 75 and 80. At a cutoff of 75, the UPSA correctly identified 68% of the sample, with a sensitivity of 0.69 and specificity of 0.66. At a cutoff of 80, the UPSA correctly identified 68% of the sample, with sensitivity and specificity of 0.59 and 0.76, respectively.

Figure 1
ROC: Receiver Operator Characteristic; UPSA Total score. UPSA = UCSD Performance-based Skills Assessment.
Table 2
Sensitivity, Specificity, Positive and Negative Predictive Values and Percent Subjects Correctly Identified as Residentially Independent with UPSA cutoffs

3.1. Comparison of UPSA with Other Predictors of Residential Independence

To establish whether the UPSA was better than other predictors of residential independence, we produced ROC curves (with 95% confidence intervals) for positive and negative symptoms, depressive symptoms, and DRS total scores. For these analyses, PANSS positive scores were available for 427 of the original 434 participants. PANSS negative data were available for 425 participants, DRS scores were available for 355 participants, and HAM-D scores for 421 participants. The estimated AUC for PANSS positive scores was 0.55 (95% CI: 0.49–0.62). The PANSS negative AUC estimate was 0.46 (95% CI: 0.40–0.52). The DRS AUC estimate was 0.65 (95% CI: 0.58–0.73). Using the estimate of average correlation provided by Hanley and McNeil (1983), we compared the AUCs for the UPSA and DRS for the 355 participants who had data available on both tests; the UPSA AUC estimate of 0.74 (95% CI: 0.68–0.81) was significantly greater than that of the DRS detailed above (z = 2.43, p = .015). ROC curves for the UPSA and DRS are presented in Figure 2.

Figure 2
Receiver Operator Characteristic analysis of comparison: Prediction of residential independence. The UPSA and DRS AUC estimates were 0.74 (95% CI: 0.68–0.81) and 0.65 (95% CI: 0.58–0.73), respectively, with the UPSA AUC significantly greater ...

Finally, we compared demographic and functional differences between the patients scoring above versus below an UPSA cutoff of 75. Results of these analyses, presented in Table 3, suggest that, compared to those scoring <75, individuals scoring 75 or above displayed fewer positive and negative symptoms and performed significantly better on the DRS. These findings suggest that in addition to residential independence, scores on the UPSA are sensitive to other clinically meaningful outcomes. However, the findings presented above suggest that UPSA scores are the best single predictor of residential status.

Table 3
Characteristics of Individuals Scoring Below versus Above UPSA Cutoff of 75

4. Discussion

Healthcare providers and other medical decision-makers for patients with chronic schizophrenia and other serious mental illnesses often must make decisions on whether their patients are capable of residing independently or would be better served by assisted care settings such as B&C facilities. Although many factors play a role in this decision, including neurocognitive functioning, level of positive, negative, and depressive symptoms, our study sought to establish the usefulness of the UPSA, a brief assessment of functional capacity, for prediction of residential independence. We found that the UPSA was significantly better than chance and better than classical clinical features of schizophrenia such as positive and negative symptoms and global cognitive functioning at predicting residential independence.

It has been suggested that assessments focusing on functional skills (e.g., the UPSA) do not adequately take other cognitively relevant features of everyday functioning into account. Specifically, Revheim and Medalia (2004a) argued that tests of functional capacity (e.g., observing a patient’s ability to pay bills) do not take into account important functional features such as remembering to pay bills or otherwise initiate functionally relevant tasks. Although not discounting this observation, our data demonstrates that performance on the UPSA is highly related to global cognitive performance (see Table 3) and serves as a significantly better predictor of residential independence than does global cognitive functioning. Thus, if this assessment could incorporate assessments of tendencies toward initiation, it might be even more sensitive.

The UPSA offers several advantages relative to other clinical measures (e.g., DRS, PANSS, HAM-D). For example, the brevity of the UPSA (i.e., 30 minutes) makes it preferable to lengthy batteries assessing multiple cognitive domains. Indeed recent research (Bowie et al., 2006; Keefe et al., 2006) has indicated that the prediction of real-world outcomes is not enhanced by cognitive performance when scores on the UPSA are considered. In addition, unlike assessments of global cognitive functioning (e.g., DRS) healthcare professionals do not need specialized training in order to administer the UPSA. Further, our data suggest that clinical symptoms, particularly negative symptoms, failed to provide any meaningful advantage to predicting residential independence. However, participants scoring 75 or above on the UPSA had significantly fewer positive and negative symptoms, indicating these symptoms are captured by performance on the UPSA. Therefore, we believe the UPSA serves as a brief measure of both functional capacity and as a proxy measure of global cognitive functioning and clinical symptoms that can adequately predict residential independence.

In addition to the brevity and ease with which the UPSA may be administered, there are advantages to establishing an UPSA cutoff for predicting residential independence. For example, researchers conducting clinical trials on the benefits of medications (e.g., cognitive-enhancing drugs, antipsychotic medications) or psychosocial interventions may seek to move participants above the UPSA threshold of 75 as an indication that these treatments improve the likelihood that patients may live independently. Indeed, Patterson et al (2006) found that a behavioral intervention, known as Functional Adaptations and Skills Training (FAST), significantly improved performance on the UPSA, with average scores for patients in the FAST intervention improving from 60 to 70. These results were similar to those of a small pilot study of Hispanic patients with schizophrenia examining the effects of a behavioral intervention for improving functional capacity (Patterson et al., 2005). Although these studies did not examine the percentage of participants moving above scores of 75, they provide evidence that the UPSA is sensitive to treatment and that patients can make substantial gains on the UPSA. We encourage future investigations to examine not only overall change on the UPSA, but the success of treatments for moving patients above a cutoff of 75.

Our study provides preliminary evidence that the UPSA is a valid instrument for predicting independent living status. However, data from this study were cross-sectional rather than prospective in nature. Therefore, one interpretation of our findings is that patients living in independent living situations have greater opportunity to practice the skills assessed on the UPSA (e.g., counting change, scheduling appointments) or that patients living in B&C settings are limited in their ability to practice these skills, thus accounting for differences in UPSA scores. If true, we believe this adds strength to our previous assertion that psychosocial skills-training interventions (rather than psychotropic medications) may be useful for improving patients’ functional abilities, given their emphasis on repeated practice of skills. While the cross-sectional nature of this study is a limitation, we believe assessment of patient ability to perform these tasks may still be useful in terms of discharge planning. Yet, future research is needed to determine if the UPSA serves as a prospective predictor of residential independence in patients with schizophrenia.

Our study did not include a clinical assessment of functioning such as the Global Assessment of Functioning (GAF) scale of the DSM-IV (American Psychiatric Association, 2000). We believe the UPSA would perform particularly well compared to clinical assessments such as GAF scores. Previous studies have found GAF scores to be poor predictors of living status and functioning (Revheim et al., 2004a; Roy-Byrne et al., 1996) and suggest that relying on the GAF to assess patients’ functioning may be problematic. Nonetheless, we recommend that future studies should compare the AUC for clinical ratings of functioning such as GAF scores to that of the UPSA. Future research should also examine whether using the UPSA in conjunction with clinical assessments provides a better assessment than either of these assessments alone. Finally, future work should examine the predictive power of the UPSA prospectively across multiple placement settings to determine success in those settings and generalization to other functional outcomes.

We did not assess other indicators of functional outcome such as employment. We believe future studies should examine whether the UPSA adequately predicts employment status as a means of further validating the UPSA’s usefulness as a clinical tool. We suggest two methods of establishing the UPSA’s validity for predicting work-related outcomes. First, similar to our current study, ROC curves could be used to establish cutoffs for predicting employment status (e.g., employed vs non-employed; full-time vs part-time). Second, survival curves for maintaining employment over time could be calculated. Relevant to this point, it was recently reported (Rosenheck et al., 2006) that work outcomes are tightly linked to disability status and less strongly related to cognitive performance. It would be of interest to determine if UPSA scores were more relevant than cognitive test scores at predicting employment when disability status is also considered.

Cutoff scores between 75 and 80 most accurately predicted residential independence, although this accuracy was 68%. It should be noted, however, that our sample did not include participants who resided in care settings where maximum supervision is necessary (e.g., inpatient settings). Instead, our sample consisted entirely of community-dwelling patients. We believe that inclusion of an inpatient sample would further increase the UPSA’s specificity and sensitivity and might further enable healthcare providers to differentiate cutoffs at which individuals may be safely moved into the community (i.e., higher levels of independence). As such, we strongly encourage this line of research.

In sum, we find that the UPSA serves as an adequate predictor of residential independence for individuals suffering from chronic schizophrenia. Furthermore, the UPSA was superior to measures of psychiatric symptom severity and global cognitive performance in predicting residential independence. Additionally, the UPSA is a brief assessment that can be easily administered to patients.

Acknowledgments

This research was supported, in part, by MH 62554 by the National Institute of Mental Health to Dr Patterson, MH 66248 by the National Institute of Mental Health to Dr Jeste, and by award MH 63116 by the National Institute of Mental Health to Dr. Harvey, as well as the VA VISN-22 and VA VISN-3 MIRECC.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • Altman DG, Bland JM. Diagnostic tests 2: Predictive values. BMJ. 1994;309:102. [PMC free article] [PubMed]
  • American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4 Text Revision. Washington, DC: American Psychiatric Association; 2000.
  • Bell MD, Bryson G. Work rehabilitation in schizophrenia: does cognitive impairment limit improvement? Schizophrenia Bulletin. 2001;27:269–79. [PubMed]
  • Bowie CR, Reichenberg A, Patterson TL, Heaton RK, Harvey PD. Determinents of real-world functioning performance in Schizophrenia: Correlations with cognition, functional capacity, and symptoms. American Journal of Psychiatry. 2006;163:418–425. [PubMed]
  • Bryson G, Bell MD. Initial and final work performance in schizophrenia: cognitive and symptom predictors. The Journal of Nervous and Mental Disease. 2003;191:87–92. [PubMed]
  • Bryson G, Bell MD, Kaplan E, Greig T. The functional consequences of memory impairments on initial work performance in people with schizophrenia. The Journal of Nervous and Mental Disease. 1998;186:610–5. [PubMed]
  • Green MF. What are the functional consequences of neurocognitive deficits in schizophrenia? The American Journal of Psychiatry. 1996;153:321–30. [PubMed]
  • Green MF, Kern RS, Braff DL, Mintz J. Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the “right stuff”? Schizophrenia Bulletin. 2000;26:119–36. [PubMed]
  • Hamilton M. Standardised assessment and recording of depressive symptoms. Psychiatria, Neurologia, Neurochirurgia. 1969;72:201–5. [PubMed]
  • Hamilton M. General problems of psychiatric rating scales (especially depression) In: Pichot P, Olivier-Martin R, editors. Psychological Measurements in Psychopharmacology. Basel, Switzerland: S. Karker; 1974.
  • Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36. [PubMed]
  • Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148:839–43. [PubMed]
  • Jeste DV, Wyatt RJ. Understanding and Treating Tardive Dyskinesia. New York: Guilford Press; 1982.
  • Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin. 1987;13:261–76. [PubMed]
  • Keefe RS, Poe M, Walker TM, Kang JW, Harvey PD. The Schizophrenia Cognition Rating Scale: an interview-based assessment and its relationship to cognition, real-world functioning, and functional capacity. The American Journal of Psychiatry. 2006;163:426–32. [PubMed]
  • Lamb HR, Bachrach LL. Some perspectives on deinstitutionalization. Psychiatric Services. 2001;52:1039–45. [PubMed]
  • Loong TW. Understanding sensitivity and specificity with the right side of the brain. BMJ. 2003;327:716–719. [PMC free article] [PubMed]
  • Mattis S. Dementia Rating Scale. Odessa, FL: Psychological Assessment Resources; 1973.
  • Obuchowski NA. ROC analysis. American Journal of Roentgenology. 2005;184:364–72. [PubMed]
  • Patterson TL, Bucardo J, McKibbin CL, Mausbach BT, Moore D, Barrio C, Goldman SR, Jeste DV. Development and Pilot Testing of a New Psychosocial Intervention for Older Latinos With Chronic Psychosis. Schizophrenia Bulletin. 2005;31:922–930. [PubMed]
  • Patterson TL, Goldman S, McKibbin CL, Hughs T, Jeste DV. UCSD Performance-Based Skills Assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophrenia Bulletin. 2001;27:235–45. [PubMed]
  • Patterson TL, Mausbach BT, McKibbin C, Goldman S, Bucardo J, Jeste DV. Functional Adaptation Skills Training (FAST): A Randomized Trial of a Psychosocial Intervention for Middle-Aged and Older Patients with Chronic Psychotic Disorders. Schizophrenia Research. 2006;86:291–99. [PubMed]
  • Revheim N, Medalia A. The independent living scales as a measure of functional outcome for schizophrenia. Psychiatric Services. 2004a;55:1052–4. [PubMed]
  • Revheim N, Medalia A. Verbal memory, problem-solving skills and community status in schizophrenia. Schizophrenia Research. 2004b;68:149–58. [PubMed]
  • Rosenheck R, Leslie D, Keefe R, McEvoy J, Swartz M, Perkins D, Stroup S, Hsiao JK, Lieberman J. Barriers to employment for people with schizophrenia. The American Journal of Psychiatry. 2006;163:411–7. [PubMed]
  • Roy-Byrne P, Dagadakis C, Unutzer J, Ries R. Evidence for limited validity of the revised global assessment of functioning scale. Psychiatric Services. 1996;47:864–6. [PubMed]
  • Twamley EW, Doshi RR, Nayak GV, Palmer BW, Golshan S, Heaton RK, Patterson TL, Jeste DV. Generalized cognitive impairments, ability to perform everyday tasks, and level of independence in community living situations of older patients with psychosis. The American Journal of Psychiatry. 2002;159:2013–20. [PubMed]
  • Woods SW. Chlorpromazine equivalent doses for the newer atypical antipsychotics. The Journal of Clinical Psychiatry. 2003;64:663–7. [PubMed]