An important performance-based functional capacity measure was developed by Patterson et al.37
This scale, referred to as the University of California San Diego (UCSD) Performance-Based Skills Assessment (UPSA) was designed to address limitations in other performance-based assessments that had been validated with dementia patients. The scale takes about 30 minutes to complete and assess performance in 5 domains—household chores, communication, finance, transportation, and planning recreational activities.
Items in this scale involve performing a variety of skilled tasks, including manipulating money, making routine and emergency calls, reading maps and schedules, and performing shopping tasks. The scale is scored on a 0- to 100-point basis, with scores in the 5 domains set to 20 points each. Interrater reliabilities of ratings were excellent (intraclass correlation coefficient
.91) and test-retest reliabilities over a 1-week period of independent ratings were good (intraclass correlation coefficients ranged from .74 to .94), with the total score having a test-retest reliability of .93 at a 2-week retest.
A recent construct validity study suggested that scores on the UPSA (expressed as a total score) completely mediated the effects of NP impairment on a measure of everyday outcome rated by case managers.20
Interestingly, the zero-order correlations between NP performance and both UPSA scores and ratings of everyday outcome were substantial, while the correlations between negative symptoms, positive symptoms, and depression and UPSA scores were quite small. Thus, in that study, it appeared as if the UPSA had substantial construct validity as a measure of the functional outcome construct. Further evidence for the validity of the UPSA was provided in a recent study examining the sensitivity of UPSA scores to independence in residential situation.38
In that study, 434 participants were evaluated with the UPSA and 99 (23%) were living independently at the time of assessment. The discriminant validity of the UPSA was adequate (receiver operating characteristic [ROC] area under the curve
0.74; 95% confidence interval: 0.68–0.79), with greatest dichotomization for the UPSA at a cutoff score of 75 (68% accuracy, 69% sensitivity, 66% specificity) or 80 (68% accuracy, 59% sensitivity, 76% specificity). The UPSA was also a significantly better predictor of living status than cognitive performance assessed by the Dementia Rating Scale (DRS; z
In addition, a brief version of the UPSA, referred to as the UPSA-Brief has been developed.39
Using the same sample of 434 patients, a factor analysis of all the items was performed, finding a single-factor solution. The UPSA-Brief was created from the 2 subscales that loaded most heavily on this factor, namely the finance (factor loading
.85) and communication (factor loading
.80) subscales. Interestingly, these are the 2 shortest subscales on the instrument and require the fewest special props. The brief version of the UPSA was correlated quite strongly with total scores on the instrument, Pearson r
.91, as well as with concurrent validators such as the DRS (Pearson r
< .001). Finally, a cutoff score of 60 on the UPSA-Brief yielded accuracy of 70%, sensitivity of .82, and specificity of .58. An ROC analysis suggested that there was essentially no difference in the long and brief versions for their relationship with residential independence.
To date, the UPSA has not been used as a treatment outcome measure in a completed study. It is currently being utilized in treatment studies associated with the National Institute of Mental Health's MATRICS-TURNS initiative. Information will be available soon as to the relative sensitivity of the UPSA (as well as the UPSA-Brief), compared with the MATRICS NP battery, to treatment-related changes induced by a pharmacological intervention.
An additional potential limitation of the current versions of the UPSA is a vulnerability to ceiling effects. Because the rest was developed for use in older patients, there is some potential for ceiling effects in younger patients. A potential solution to this problem is to include some supplemental, more challenging, items from other functional outcome rating scales.
The Test of Adaptive Behavior in Schizophrenia (TABS40
) is a performance-based measure of adaptive functioning designed to address limitations of other available measures including inadequate assessment of the ability to initiate and of the ability to identify problems that occur in the course of performing functional activities. For example, performance-based tests tend to ask a participant how they would solve a particular problem. However, it is not clear that the individual would have been able to identify the fact that a problem existed. In addition, because performance-based tests tend to ask the individual to respond to a contrived situation, there is little room to assess the individual's ability to initiate or to adopt a flexible problem-solving style if necessary.
The TABS is designed to assess the abilities necessary to perform goal-directed activity including initiation, planning, problem identification, problem solving, sequencing, appropriate inhibition, and persistence in the context of 6 functional areas (work and productivity, medication management, independent living, shopping, basic hygiene, and social skills).41
TABS items demand considerable initiation (eg, spontaneously generating items that would be necessary to stock an empty bathroom), allow the subject the chance to identify specific problems on their own, prior to they being pointed out by the examiner (eg, identify that he or she was short changed, identifying that he or she will run out of medication), and provide additional points for spontaneously offering solutions (eg, spontaneously announcing a plan to remedy a problem with running out of medication). TABS scores are calculated as percentages with higher scores indicating better adaptive behavior.
In a study of 264 individuals with schizophrenia or schizoaffective disorders, tested on 2 occasions 3 months apart, the TABS demonstrated good test-retest reliability (.80) and interitem consistency (.84). Moreover, TABS scores were found to show good evidence of convergent validity, by being moderately to strongly correlated with other measures of functional outcome, negative symptoms, and neuropsychological test scores. However, measures of positive symptoms were not found to be related to TABS performance (discriminant validity). Thus, these preliminary data suggest that the TABS has reasonable evidence of construct validity. The sensitivity of the TABS to the effects of psychosocial or pharmacological treatments needs to be examined.
As reported in Velligan et al,40
one criticism that could be leveled at the TABS is that once a problem has been identified during the course of the first administration of the test (eg, not enough medication to fill the containers), the subject would be aware of the problem on repeated administrations. Repeated measurements of the TABS were obtained in the validation study at 3-month intervals. During that interval, there was little clinical evidence that subjects “remembered” problems that they had identified or that had been pointed out to them at previous testing sessions. Practice effects may be problematic in clinical trials with shorter intervals.