|Home | About | Journals | Submit | Contact Us | Français|
Approximately 45% of alcohol abusers suffer from mild to moderate cognitive impairments including difficulties with verbal memory, executive functioning, and visuospatial ability (Grant, 1987 Grant, 1994). Between 33% to 50% of polysubstance abusers have a wide variety of cognitive deficits including problems in mental flexibility, abstract reasoning, and visuospatial scanning (Fals-Stewart and Bates, 2003; Grant et al., 1976). Studies have also documented lasting cognitive decrements among individuals who chronically abuse cocaine, solvents, or sedative-hypnotics (Fals-Stewart and Bates, 2003; Rosseli and Ardila, 1996). Cognitive problems have been shown to impact on retention in substance abuse treatment in multiple ways. SA patients low in cognitive ability are less likely to engage in treatment, and are less motivated for treatment than those higher in cognitive ability (Aharonovich et al., 2006; Katz et al., 2005). Cognitive problems also restrict the patient’s ability to learn skills taught in treatment sessions (Fals-Stewart and Lucente, 1994). Finally, patients with cognitive deficits are also more likely to be removed from treatment due to failure to follow rules (Fals-Stewart, 1993).
Given the problems associated with cognitive impairments among patients in substance abuse treatment, it is useful for clinicians to know whether patients have cognitive difficulties in order to develop an effective treatment plan. Neuropsychological assessment is known to be a valid and reliable method of evaluating cognitive difficulties. This method is highly resource-intensive, however, requiring several hours of time by skilled personnel to administer and interpret a complete battery of tests.
A self-report measure of cognitive complaint may have substantial value in SA treatment settings, as it permits a quick and easily administered snapshot of cognitive concerns that offers a window on how patients perceive their own mental functioning in daily life. One such instrument is the Patient’s Assessment of Own Functioning Inventory (PAOFI). The PAOFI was designed to evaluate a patient’s sense of his or her functional capacity in everyday activities. The items were derived from the actual complaints of patients referred for psychiatric and neurological assessment and grouped by rational analysis into subscales concerning memory, language and communication, use of hands, sensory-perception, higher level cognitive and intellectual functions, and work/recreation. The rational approach was substantiated by a factor analytic clustering of items into the final subscales (Chelune et al., 1986).
The potential usefulness of the PAOFI has been questioned however, due to concerns about the ability of patients to accurately assess and report their cognitive status. Few studies have found strong relationships between patients’ self assessment of cognitive difficulties, and actual cognitive performance (Rourke et al., 1999a, b). Moreover, several studies have found strong relationships between cognitive self-assessment and other non-cognitive clinical variables, such as character pathology (Chelune et al., 1986), depression (Rourke et al., 1999a, b), or extent of substance use (Shelton and Parsons, 1987). It has therefore been difficult to distinguish whether the scale measures awareness of actual cognitive impairment, perceived decline in cognition, or other factors associated with psychic distress.
In order to evaluate the value of the PAOFI for clinical use it is necessary to establish norms for clinical as well as non-clinical populations. Also, given the known effects of depression on self-report of cognition, it is important to determine whether this relationship is the same for various categories of patients as well as non-clinical populations.
In the study that follows, we attempt to quantify the independent contributions of depression, actual level of neurocognitive performance, and decline in cognitive performance from premorbid function to cognitive complaints in two samples, one in substance abuse treatment and the other comprised of non-clinical controls. Our predictions were as follows:
The SA sample consisted of 73 veterans enrolled in a substance abuse day treatment program within the VA Healthcare System in a northeastern state. Diagnosis of substance abuse or dependence was a precondition for entrance into the treatment program. Most patients came directly from detoxification and had observable symptoms of substance abuse recorded in their medical record. Participants were recruited by clinician referral, self-referral, or peer referral to participate in a randomized controlled trial of cognitive remediation treatment. All measures were completed as part of the intake battery for this study. All participants gave written informed consent following procedures approved by the VA Human Studies Subcommittee.
Non-clinical participants were 150 English-speaking adults recruited between September 1999 and June 2005 in a clinical research protocol at the University of California at San Diego HIV Neurobehavioral Research Center (UCSD-HNRC). These participants were screened by PhD level clinicians using the SCID (DSM-IV) for histories of current substance-related disorders (e.g., alcohol and or drug dependence), psychosis, mental retardation, or any neurological and medical conditions that may have adversely impacted cognitive functioning (e.g., HIV infection, seizure disorders, traumatic brain injury). All participants provided informed, written consent following procedures approved by the UCSD IRB and were administered the PAOFI, Beck Depression Inventory (BDI) and neurocognitive measures as part of a broader neuropsychological, neurological, medical, or psychiatric evaluation.
A modified version of the PAOFI was used, consisting of three subscales that directly address cognitive functions. The Memory subscale (10 items) was composed of questions regarding the patients’ ability to recall information relevant to everyday life (e.g., How often do you forget events which have occurred in the last day or two?). The Language and Communication subscale (10 items) pertained to verbal communication skills (e.g., How often do you have difficulty thinking the names of things?). The Higher Cognitive Functions subscale related to executive functions such as planning and organization (e.g., Do you have more difficulty now than you used to in planning or organizing activities; i.e., deciding what to do and how it should be done?). A higher rating on any item indicated a lesser degree of impairment. Each item was rated on a Likert scale from 1 (Almost Always) to 6 (Almost Never). Likert ratings of 1–3 were given a score of 1 indicating impairment, and ratings from 4–6 were given a score of 0 indicating no impairment. This method of scoring the PAOFI is consistent with the method employed by the scale’s authors (Chelune et al., 1986).
Depression was assessed via the Beck Depression Inventory (BDI). The non-clinical sample completed the BDI, and the SA sample was given the BDI-II. The BDI is a 21 item self-report scale that measures attitudes and symptoms of depression (Beck et al., 1961). Items on the scale ask about how the subject has been feeling in the last week, with a set of at least four possible answer choices of varying intensity from 0: I do not feel sad, to 3: I am so sad or unhappy that I can’t stand it. Internal consistency for the BDI ranges from 0.73 to 0.92 with a mean of 0.86 (Beck et al., 1988). The BDI-II (Beck and Steer, 1993) was revised to conform more closely to the symptom descriptions listed in the DSM-IV (American Psychiatric Association, Association, 1994). Each of the 21 items corresponds to a symptom of depression, and is summed to give a single total score. A total score of 0–13 represents minimal symptoms; 14–19 indicates mild depression; 20–28 is moderate; and 29–63 represents severe depressive (Beck and Steer, 1993). Although the original BDI and the revised version (BDI-II) have been found to be highly correlated (r= 0.94), average scores on the BDI-II are 3 points higher than the BDI (Lightfoot and Oliver, 1985). To compensate for the difference in scoring, we added 3 points to the BDI scores for the non-clinical sample.
Each sample completed an extensive battery of neuropsychological tests that were administered by trained, professional staff under standard conditions. In the interest of clarity, only those measures that were the same for both samples are discussed in detail. These included (1) the Wisconsin Card Sorting Test-64 (Kongs et al., 2000) which assessed Executive cognitive functions; (2) The Benton Controlled Oral Word Association Test, a measure of verbal fluency requiring participants to generate words beginning with the letters F,A and S, and the category Animals under time constraints (Benton, 1994); (3) WAIS-III Digit Symbol Coding and (4) WAIS-III Symbol Search, which measure processing speed; (5) Trailmaking Test A, a paper and pencil measure of speeded processing and (6) Trailmaking test B, which measures the executive function of mental flexibility (Lezak, 1995), and (7) the Wide Range Achievement Test-Third Edition Reading subtest (WRAT-III; Snelbaker et al., 2001), a reading screen used to estimate premorbid intellectual functioning. In addition to these measures the SA sample was also administered the following tests of memory: WMS-III Logical Memory I and II, Visual Reproduction I & II. With the exception of Trails A & B, for which we used raw scores (time to complete task), all neuropsychological test scores were converted to standard scores according to published norms.
Demographic differences between the two groups were assessed using t-tests for continuous variables such as age and years of education, and Mann Whitney U tests for categorical variables, such as race/ethnicity. White substance abusers outperformed non-Whites on reading level and speeded executive processing. Race was therefore included as a covariate in relevant analyses.
The number of cases with missing data varied widely, from a low of three cases for the PAOFI subscales, to a high of 88 for the WRAT-III. Our statistical analyses employed a pairwise deletion strategy in which cases missing data for any given variable were left out of analyses only for that variable, rather than being eliminated from all analyses. This strategy was used to minimize the effect of missing data on the power of our analyses.
Direct comparisons of raw scores between PAOFI subtests was not feasible because the Higher Cognitive Ability subscale contained 9 items whereas the other subscales each contained 10 items. All PAOFI raw scores were therefore converted to Z-scores based on the mean and standard deviation of the non-clinical sample. This conversion rendered each subject’s scores as standard deviation units from the non-clinical mean.
T-tests were used to compare SA patients to non-clinical controls on the various domains of cognitive processing, PAOFI scores, and BDI scores. Bonferroni corrections were calculated by dividing the p-value for significance (0.05) by number of variables in each set of comparisons, and appear at the foot of each table.
We considered PAOFI scores of eight or more to be clinically significant. This cutoff represented one standard deviation (SD=4.5) above the average number of endorsements (M=3.0) from the non-clinical sample.
To determine whether PAOFI complaints were due to awareness of objective decline in performance, we used WRAT-III scores as a measure of premorbid cognitive functioning to predict performance on the other neurocognitive measures. Regression coefficients were computed using data from the non-clinical participants. These coefficients were then used to compute predicted values for each neurocognitive measure. Race was included in the regression equation as a binary variable (1=white, 0=non-white) to control for differences in race on the WRAT-III. Predicted values were subtracted from the actual scores for each individual and converted to z-scores based on the standard deviation of the predicted scores from the non-clinical sample. These standardized regression residuals were then averaged across the neurocognitive measures to create a unitary index of “cognitive decline”. The resulting z-score reflected how well each individual did on the cognitive tests compared to the performance expected by our estimate of premorbid verbal intelligence, while controlling for race differences on the WRAT-III.
Patients who scored more than one standard deviation below expected performance (z-score of -1) were identified as “decliners”. We used t tests to compare differences between decliners and non-decliners on PAOFI total score. We used a Chi Square test to compare the proportion of decliners among patients with clinical PAOFI elevations to those with subclinical PAOFI scores.
Data reduction for neuropsychological test variables was performed by grouping data for all cases based upon factor analysis. We computed a principal components analysis of the standard scores for the various neurocognitive measures using a Varimax rotation with Kaiser normalization, retaining only those factors with an eigenvalue greater than 1. This yielded a three factor solution that accounted for 71% of the variance. The factor loadings were as follows: Factor 1 (WCST Factor) included WCST Total Errors, Perseverative Responses, and Total Categories Completed. Factor 2 (Processing Speed Factor or PSF) included WAIS-III Processing Speed Index, Trails A and Trails B. Factor 3 (Verbal Fluency Factor or VFF) included COWAT F-A-S and Animals scores. Pearson correlations were performed between factor scores and PAOFI scores. Logical Memory I and II and Visual Reproduction I and II were not included in the factor analysis because these data were only available for the substance abuse sample. Pearson correlations between these measures and PAOFI scores were performed separately for the substance abuse group only.
To assess the relative contributions of mood disturbance and cognitive performance to PAOFI complaints, we conducted Pearson correlations of BDI scores with PAOFI Total and the neurocognitive factors for each sample separately. We then computed separate stepwise regression analyses for each group, with PAOFI total score as the dependent variable and the three cognitive factors and adjusted BDI scores entered as independent variables. Finally, to determine the impact of BDI scores on the relationship between cognitive performance and self report of impairment, we computed separate partial correlations of the cognitive factors and PAOFI scores, while controlling for BDI scores.
The substance abuse sample was significantly older and more ethnically diverse than the non-clinical sample, however both groups had the same level of education (see Table 1). We found small but significant correlations (r=0.12 – 0.14) between age and PAOFI endorsement for the whole sample, however analyses within samples were not significant. Other research (Heaton et al., 2004) found no effects of age on PAOFI endorsement. Non-clinical Caucasian participants reported more memory complaints than non-clinical African Americans at a trend level (t=2.1, df=81, P=.03). However this difference did not emerge between SA Caucasians and African Americans.
Our first hypothesis was that substance abusers would endorse more cognitive complaints than controls. We found that SA patients reported twice as many PAOFI complaints overall than non-treatment seeking controls (6.5 vs. 3.0; P <.001). Thirty-two percent of the SA sample reported a clinically significant level of cognitive complaints (n=24) compared to 13% (n=20) of the non-clinical sample (χ2(1)=11.4, P<0.001). The most frequent complaint among both substance abusers and controls was in the area of memory (see Table 2).
Our second prediction was that SA patients would have poorer cognitive performance. We found that the cognitive performance of the SA participants was generally lower than non-clinical controls, falling in the low end of the average range for published norms. The substance abuse sample underperformed compared to non-clinical controls on measures of executive functioning represented by the Wisconsin Card Sorting Test and Trails B. There were also deficits in processing speed, as evidenced by the WAIS processing speed index, and Trails A. Verbal fluency (FAS and Animals), which measured executive functioning and processing speed, was lower in the SA group.
Our third hypothesis regarding the construct validity of the PAOFI was not supported. Pearson correlations between PAOFI scores and the three cognitive factors were not significant for either group. Although memory was a primary complaint category among substance abusers, there were no associations between any PAOFI complaint categories and measures of Logical or Visual memory. There was a marginal relationship between the Verbal Fluency factor and PAOFI total for the substance abuse group (r=0.27, p=.025).
We also predicted that SA patients would show greater cognitive decline and that the construct validity of the PAOFI would be supported by its association with cognitive decline. There were significantly more subjects with clinically meaningful cognitive decline in the substance abuse sample (n=27) than in the non-clinical sample (n=1; X2(1)=25.0, P<0.001). However, those demonstrating cognitive decline had no more PAOFI complaints than those whose cognitive abilities were as expected (t (131)=0.98, P>0.05). Moreover, the within-group association between subjects with clinically meaningful declines in neuropsychological test performance and clinically meaningful PAOFI scores was non-significant (Substance abuse X2(1) = 0.01, P> 0.5; Non-clinical X2 (1)=0.15, P>0.05). There were no differences between decliners and non-decliners in terms of age, race, or level of education.
Our fourth prediction was that substance abusers would report greater depression and that depression scores would make an independent contribution to explaining PAOFI variance. Substance abuse and non-clinical samples were significantly different in their endorsement of depressive symptoms on the BDI measures (P<.001). The median adjusted BDI score for the substance abuse sample was more than twice that of the non-clinical sample (See Table 2). We also found that PAOFI total correlated with Adjusted BDI scores for both groups. However the correlation in the non-clinical group (r=0.66, P<0.001) was almost twice that of the SA group (r=0.35, P<0.001). Multiple regression showed that for the participants in substance abuse treatment, BDI scores accounted for 12% of the variance in PAOFI total. For the non-clinical group BDI scores accounted for 44% of the variance in PAOFI total.
The substance abuse sample had PAOFI scores that were significantly higher than the non-clinical control group, endorsing on average twice as many complaints of cognitive dysfunction. Overall, memory was the most common category of endorsement for both samples, but all categories of complaints were endorsed more by SA’s. A third of the substance abuse sample had a clinically meaningful number of complaints on the PAOFI, which was a significantly higher frequency than that found in the non-clinical sample. These findings support the discriminant validity of the PAOFI in distinguishing between the complaints of substance abuse and non-clinical samples.
We also found that neuropsychological test performance of our SA sample was generally lower than standardized norms, and lower than our sample of non-clinical controls. When we categorized subjects by decline in neuropsychological performance from a premorbid estimate (based on the WRAT-III), we found that more than one third of the substance abuse sample showed evidence of decline, compared to the single individual in the non-clinical sample. These findings of cognitive compromise in substance abuse were predicted and are consistent with other research (Carlin and O’Malley, 1996; Fals-Stewart, 1993; Fals-Stewart and Bates, 2003; van Gorp et al., 1999) highlighting the need for continued assessment and appropriate treatment for cognitive difficulties in this population.
Our final group comparison was on the BDI and showed that the substance abuse sample exhibited more depressive symptomatology than the non-clinical sample. This was not surprising given the psychosocial difficulties, the depressive effects of chronic substance abuse, and high co-morbidity of depression with substance abuse.
The central question regarding the construct validity of the PAOFI was whether individual variation in scores was closely associated with cognitive impairment and/or decline. Correlational analyses failed to find significant associations between the number of PAOFI endorsements and neuropsychological test scores. Statistical significance was corrected for multiple comparisons, so that small and directionally expected relationship to verbal fluency did not reach threshold. PAOFI endorsements were also not associated with evidence of decline in neuropsychological performance. This was true even when subjects were categorized into those with clinically meaningful PAOFI scores.
The only statistically significant association found for PAOFI scores was with the BDI. While 44% of PAOFI score variance was explained by BDI scores for the non-clinical sample, a much smaller proportion (12%) was explained by BDI scores for the substance abusers. For the non-clinical sample greater depression may have been a primary source for greater self-experience of impairment, even though neuropsychological results do not confirm performance deficits or decline. Perhaps, for the substance abuse sample, factors other than depression, such as a more disorganized lifestyle or greater psychosocial stressors may be contributing to the perception of cognitive impairment. For example, Shelton and Parsons (1987) found that the amount and frequency of drinking was associated with PAOFI scores in alcoholics, but failed to find an association with objective neuropsychological test performance.
The PAOFI is not unique in failing to yield associations with actual cognitive performance. Two other measures of cognitive complaint have shown similar null results. Using the Neuropsychological Impairment Scale (NIS), Errico et al. (1990) found that the self report of alcoholics was unrelated to actual cognitive performance, and highly influenced by patient’s mood state and frequency of drinking. Similarly, Horner et al (1999) found no relationship between cognitive complaint and neuropsychological performance in using the Cognitive Failures Questionnaire (mCFQ) with a sample of mixed substance abuse patients entering treatment.
It is possible that the PAOFI may relate more closely to cognitive impairments in clinical populations other than substance abuse. For example, Poutiainen and Elovaara (1996) found that HIV patients who reported a high level of cognitive complaints demonstrated poor verbal memory. Those who complained of motor abnormalities also demonstrated lapses in processing speed and cognitive flexibility. They concluded that HIV patients with cognitive declines were aware of their problems, and that these problems are distinguishable from general complaint tendencies associated with depressive disorders. Similarly, Rourke and colleagues (1999a) found that HIV patients who complained of cognitive deficits had difficulties with attention and working memory. Even so, scores on the Beck Depression Inventory predicted 84% of the variance in PAOFI endorsement in their HIV+ sample, whereas actual neuropsychological performance predicted between 12 and 25%. In contrast, a regression analysis conducted by Bassel et al., (2002) revealed that working memory was the best neurocognitive predictor of PAOFI complaints in HIV patients, and explained a similar amount of variance as mood disturbance. It is possible that the nature of cognitive decline in HIV is distinct from substance abuse in its severity, onset and course, such that HIV patients may be better aware of their cognitive deficits than substance abusers. A direct comparison between our results and those of Rourke et al, (1999a) and Bassel et al (2002) cannot be made as our analyses did not include a measure of working memory.
This is the first study to compare the validity of self-reported cognitive problems between a mixed sample of substance abusers and a non-clinical sample. It is also the first study to quantitatively compare the impact of depression on PAOFI scores in non-clinical and SA groups. Depression was three times more strongly associated with PAOFI complaints in our non-clinical sample than for substance abusers. Our substance abuse sample was actively engaged in clinical care. As such they may have been in greater psychosocial distress and had greater reason to be concerned about cognitive problems whether or not they were depressed. Among non-treatment seeking individuals with otherwise normal cognition, concerns about cognitive problems may have intensified during episodes of depression.
Given the failure to associate PAOFI scores with cognitive deficit or decline in the current study, some concerns about the usefulness of the PAOFI warrant mention. The PAOFI has domain-specific subscales but no current research has shown them to predict impairment in domains of actual cognitive performance. Further research examining the relationships between individual subscales and other dependent variables such as psychosocial adjustment, improvement in treatment, or other measures of objective cognitive performance may be helpful in evaluating the utility of these subscales and the total score.
This study has several limitations. First, the sample was entirely male. It is therefore unknown whether these findings will generalize to females. Second, participants in the two groups were recruited on opposite sides of the continent, and were very different in demographic makeup. It is therefore uncertain whether geographic or demographic factors that were unaccounted for in the analysis may have contributed to our findings. The lack of similarity between the two groups does not, however, preclude our ability to answer the essential question of whether a diagnostically heterogeneous sample of substance abusers, taken from a real-world treatment setting are sufficiently different in terms of their cognitive makeup and tendency to complain of cognitive difficulty from a sample of non-treatment-seekers. Third, the neuropsychological assessment, although representative, was not comprehensive and our method of assessing decline was approximate. It is possible that relationships between self-reported cognitive difficulties on the PAOFI and actual impairment or decline exist but went undetected due to the number of cases with missing data on our measure of premorbid functioning.
Finally, most of the PAOFI variance in the substance abuse sample has gone unexplained. Had other measures related to SA populations been employed, such as severity of abuse or measures of psychosocial stress, perhaps relationships would have been revealed. Given the importance of self-experience in behavioral outcomes, it would be worthwhile to understand what SA patients are responding to when they answer PAOFI questions and how those answers relate to subsequent treatment outcomes.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.