|Home | About | Journals | Submit | Contact Us | Français|
Despite substantial research on overall decision-making capacity (DMC) levels in schizophrenia, factors causing individuals to make errors during decision-making regarding research participation or treatment are relatively unknown.
We examined the responses of 84 middle-aged and older patients with schizophrenia or schizoaffective disorder on a structured DMC measure—the MacArthur Competence Assessment Tool for Clinical Research—to determine the frequency and apparent cause of patients’ errors.
Most errors were due to difficulty recalling the disclosed information (seen in 65.5% of patients), particularly the study’s procedures, potential risks/discomforts, and purpose. Errors attributable to concrete thinking, thought disorder or other psychotic symptoms, or perceived coercion were rarer.
Informed consent procedures might be improved for this population by providing information in a way that facilitates learning and memory, such as iterative disclosure of the information, corrective feedback, and emphasis of key points (e.g., the study’s purpose, procedures, and potential risks).
Valid consent for research requires that participants have the capacity to use disclosed information to make a decision about participation1. Persons with schizophrenia are at increased risk of impaired decisional capacity (DMC), but there is substantial between-person heterogeneity in DMC even within this diagnostic group2. Thus, judgments about an individual’s DMC cannot and should not be based solely on his/her diagnosis3.
Concerns about DMC for research consent among people with schizophrenia have stemmed, in part, from worries that the associated cognitive deficits and/or psychotic symptoms—delusions, hallucinations, and thought disorder, for example—could impair the abilities needed to make informed decisions, as well as from the supposition that persons in this population may be susceptible to coercion4. There is a wealth of empirical data documenting that severity of cognitive deficits, and to a lesser degree, negative symptoms, are the strongest predictors of impaired DMC in patients with schizophrenia (reviewed in Palmer and Savla5). There has been less empirical focus on the specific nature of difficulties or the errors manifest by patients when DMC is being assessed. Dunn and Jeste6 reported findings on the content of such errors, finding that, relative to healthy comparison subjects, patients with psychotic disorders had greater difficulty answering questions regarding requirements of the study as well as the potential risks and benefits associated with participation.
A potentially useful complement to the data provided by Dunn and Jeste6 would be information on the form or process of errors (such as distinguishing those due to forgetfulness versus those due to psychotic thought processes). Such information could inform compensatory strategies to maximize the effectiveness of consent procedures. These strategies are needed because attempts to improve the consent process have produced mixed results thus far, depending on the approach used and the population studied7. Marson et al.8 have addressed this issue in mild and moderate probable Alzheimer’s disease by focusing on the nature of errors made by patients during a treatment decision. The authors found that the Alzheimer’s group were more likely than normal controls to make certain kinds of errors—primarily communication difficulties (e.g., incomprehensible responses, language production problems such as paraphasias), comprehension problems, factual errors, and losing set. To our knowledge, however, there have been no published studies of this type examining the consent-related errors made by patients with schizophrenia during research (or treatment) consent procedures.
As a preliminary effort to elucidate the nature of decision making errors patients make during the process of consent to research, we examined the responses of patients with schizophrenia or schizoaffective disorder during administration of a structured capacity interview, the MacArthur Competence Assessment Tool for Clinical Research (MacCAT-CR)9. We expected that patients would make a variety of errors, reflecting the diffuse and heterogeneous patterns of cognitive impairment that are found in these disorders (reviewed in Palmer et al.10). However, the specific nature of DMC-related errors has not been examined in this population; therefore, the current study was designed to be descriptive and exploratory in nature.
Participants were 84 patients with schizophrenia or schizoaffective disorder who participated in a larger longitudinal study of DMC among middle-aged and older persons with a history of psychoses11. DSM-IV12 diagnoses were made by patients’ clinical care providers. Data for these 84 participants was originally collected as part of a larger study of capacity to consent to research on the short- and long-term side-effects of second generation (“atypical”) antipsychotic medications, which also included a smaller number of persons with mood disorders with a history of psychotic features, in addition to the larger schizophrenia sample. All subjects in the large study completed the MacArthur Competence Assessment Tool for Clinical Research (MacCAT-CR, described below). However, as also described below, the current analyses were based on secondary coding of the individual responses on each MacCAT-CR item. Given the lack of prior research focused on the error processes in the MacCAT-CR, we had no basis for estimating effect sizes for power analyses/determination of sample size. However, since the secondary coding was a labor intensive process, we decided to do so with the MacCAT-CR from the first 100 consecutive participants. For the present report, we decided to exclude 16 individuals who had diagnosis other than schizophrenia or schizoaffective disorder because the primary question was whether psychotic symptoms affect DMC in patients with a primary psychotic disorder.
The 84 participants had a mean age of 54.1 years (SD = 8.4) and education of 12.2 years (SD = 2.8); 38.1% women and 64.3% Caucasian. Many (54.8%) were living in board and care facilities; 31% were in apartments/houses, 3.6% were in assisted living facilities, 2.4% were in single room occupancies, and 1.2% were homeless. Participants’ psychiatric characteristics were assessed with the Positive and Negative Syndrome Scale (PANSS)13, 17-item Hamilton Depression Rating Scale (HAM-D), and the Birchwood Insight Questionnaire14. Participants’ mean symptom levels were as follows: schizophrenia positive symptom severity (PANSS Positive Total) = 15.2 (SD = 5.9, Range = 7–30), schizophrenia negative symptom severity (PANSS Negative Total) = 13.1 (SD = 5.4, Range = 7–32), depression severity (HAM-D) = 9.4 (SD = 5.8, Range = 0–28), and insight (Birchwood Insight Total) = 8.3 (SD = 3.0, Range = 0–12). Participants were clinically stable outpatients, and these inventories indicate that most individuals’ psychiatric symptoms were mild in severity.
The consent process for this study of informed consent involved each potential participant meeting with a member of our research staff who reviewed the IRB approved consent form with the potential participant and answered any questions that he or she had. All participants provided written consent to participate by signing the IRB approved consent form. Per the guidelines of our IRB, formal assessment of capacity to consent was not required for enrollment in this study as it meets the federal definition of “minimal risk”15. (Please see Saks et al.16 for a detailed discussion of the issue of decisional capacity for research on informed consent). Further, because of the minimal risk nature of this study, a lower level of comprehension ability was needed than might be the case for the typically more complex and risky randomized clinical drug trial16. However, our staff is trained to be attuned to apparent confusion among participants, and two potential participants were excluded due to a clear lack of understanding regarding the nature of this project. Although two other participants had legal conservators who gave permission for patient participation (i.e., in addition to the consent provided directly by the patients themselves), no participants were enrolled via proxy consent.
Participants completed the MacCAT-CR9, a semi-structured interview that assesses: 1) Understanding disclosed information, 2) Appreciating the information in terms of one’s personal situation, 3) Reasoning with the information, and 4) Expressing a choice. The MacCAT-CR items were tailored in reference to a naturalistic study of the short- and long-term side effects of antipsychotic medications to which subjects were actually considering enrolment17. This naturalistic study would involve cognitive, motor, and symptom assessments, and participants’ medications would not be changed.
The MacCAT-CR is a 20–30 minute semi-structured interview. The MacCAT-CR interviews for this study were conducted by trained Bachelor’s-level research assistants under the supervision of two of the study authors (BWP and LBD). As per standard MacCAT-CR administration9, interviewers were trained to ask follow-up questions so that respondents could clarify any ambiguous responses. Also, information provided during the Understanding subscale was re-explained if a participant’s initial response indicated that he/she did not fully comprehend the information. [The standard MacCAT-CR administration includes one such re-explanation/re-query per item, but as one of the goals of the overall study was to evaluate the value of further explanation, we permitted up to two re-explanations (see Palmer & Jeste, 200618, Palmer et al, 200719)]. Interviewers were instructed to record subjects’ responses to each item verbatim, to the extent possible, without disrupting the flow of the interview. These recorded responses served as the key data for the present error analyses.
Participants were referred on to the study described by the MacCAT-CR if they expressed interest in enrolling in that study and they were considered to have adequate DMC to provide informed consent to that study. (Specifically, if individuals’ total Understanding score was less than or equal to 15 by trial 3, they were considered to have inadequate DMC for participation in that study. Furthermore, individuals had to respond correctly to three critical items on the MacCAT-CR regarding 1) the potential risks of the study, 2) the right to withdraw, and 3) the voluntary nature of the study in order to be referred on to that study. For individuals who exceeded these requirements, clinical judgment was also used to evaluate whether they had adequate DMC to consent to that study).
Individual patients’ responses on the MacCAT-CR were examined by three of the authors (ARK, BWP, and LBD) using a coding scheme developed for these purposes. Steps used in the coding scheme development process are outlined in Figure 1. (Additional details about this process and specific scoring criteria are available from the authors upon request). Raters individually examined and coded each MacCAT-CR protocol using the final version of the coding scheme. (Table 1 lists the interrater agreement rates for each code that resulted from this step). Then, raters met to discuss their ratings of each participant’s MacCAT-CR and to generate final codes as agreed upon via group consensus. Any coding discrepancies were discussed, and agreement on all of the codes was reached, such that no codes remained disputed. All remaining analyses reflect these consensus codes.
Error categories for the MacCAT-CR Understanding subscale included 1) difficulty recalling the disclosed study information, 2) perseveration, 3) illogical responses, 4) psychosis / delusions apparent in responses, and 5) overly concrete and/or personalized responses (i.e., overly emphasizing his/her potential for gain from study participation). For individuals who were coded as having difficulty recalling study information, it was noted whether their performance improved following repetition of the information. Error categories for the Appreciation subscale included 1) difficulty appreciating the goals of the study (i.e. distinguishing research goals from personal benefit) and 2) difficulty appreciating the study procedures (i.e. recognition that assessments were for study purposes, not clinical care). Error source codes for the Reasoning subscale included reasoning that was considered 1) vague, 2) illogical, 3) limited (e.g., only one reason for choice given when two are required per standard MacCAT-CR scoring criteria), or 4) inconsistent (i.e., reasoning did not correspond with final choice). On the Expression of a Choice subscale, it was noted whether or not patients clearly expressed a choice. (As this is what determines the quantitative score on this subscale, separate coding of this item was not required).
The frequencies of all codes described above were tallied. For patients coded as having difficulty recalling information on the Understanding subscale, specific items for which they struggled to recall information were recorded. Given concern regarding the potential for coercion in this population, we also recorded the number of patients who correctly recognized that study participation would be voluntary.
70 out of 84 (83.3%) participants were coded as demonstrating at least one error on the MacCAT-CR, and most occurred during the Understanding subscale. The frequency of each error is provided in Table 2. The most common source of errors, seen in 55 of the 84 participants, was difficulty recalling study information. Table 3 lists the specific information that these 55 patients had difficulty recalling. Of these patients, with repetition of relevant information, 36.4% showed perfect or near-perfect recall, 47.3% recalled more information but continued to miss important details, but 16.4% showed no recall improvement.
76 out of 84 (90.5%) patients indicated, after the first presentation of the study information, that they knew study participation would be voluntary (i.e. there would not be any negative consequences for them if they chose not to participate). Only one patient failed to demonstrate this knowledge following repetition of the disclosed study information.
The majority of patients (83.3%) demonstrated at least one error during the DMC interview (MacCAT-CR). The most common source of errors by far was difficulty recalling aspects of the study information disclosed during the Understanding subscale (65.5% of patients). Further evidence of the importance of memory processes and/or systems in DMC among schizophrenia patients is provided by Eyler et al.20. Using functional magnetic resonance imaging, these authors found that patients’ performance on the MacCAT-CR Understanding subscale was correlated with brain activation during a verbal learning task in regions known to support learning and memory including the right hippocampus and bilateral parahippocampus. Together, these findings suggest that efforts to enhance the research consent process for patients with schizophrenia and schizoaffective disorder should focus on factors that facilitate initial acquisition and processing of the disclosed information. Indeed, in the present study, patients struggled to recall even the most basic points of the study information (e.g., 42.9% had difficulty recalling the purpose of the study). It may be that the large amount of detailed information provided during consent procedures, in combination with patients’ impaired memory abilities, makes it difficult for them to learn and retain the key points. Our findings do suggest, however, that repetition of the study information ameliorates recall errors in most cases. Together, these findings highlight the importance of structuring the disclosed study information in such a way as to emphasize key points and repeating the information in order to facilitate recall.
Patients’ responses were also notable for the errors they did not make. Ethical concerns have been raised surrounding the notion that psychotic symptoms per se (e.g., delusional thinking) might impede DMC. However, in the present analyses, there was no evidence to support detrimental effects of psychosis on DMC for research participation among this outpatient sample. These findings complement quantitative studies that found no relationship between DMC for research participation and the positive symptoms of schizophrenia in samples of outpatients and/or inpatients21. However, some quantitative studies have found a negative relationship between positive symptoms and DMC, including a study of DMC for research22 and a study of DMC for treatment23, both of which were of inpatient samples. On average, individuals in our sample were rated as having mild psychiatric symptoms, although some patients did have moderate symptoms. On the PANSS, 28% of the sample was rated as having delusional symptoms of at least moderate severity and 35.4% of the sample was rated as having hallucinatory symptoms of at least moderate severity. Thus, despite psychotic symptoms of this severity being present among our sample, they were not evident during patients’ responses on the MacCAT-CR, suggesting that these symptoms did not affect DMC. Nevertheless, conclusions regarding the impact (or lack thereof) of psychotic symptoms on DMC must be tempered as our sample consisted primarily of clinically stable outpatients. A limitation of the present study is that results cannot be generalized to more acutely ill or inpatient populations. Future research should explore whether inpatients exhibit a different pattern of DMC-related errors.
Despite concerns regarding the potential for coercion in this population, the fact that the vast majority of patients in the present study recognized the voluntary nature of participation suggests that perceived coercion is uncommon. Minimal previous work has examined the possibility of coercion in research enrollment in mentally ill populations. Moser and colleagues assessed susceptibility to coercion in the research context among 30 incarcerated mentally ill people (of whom 5 had schizophrenia or schizophrenia-related disorders). Although patients with worse neuropsychological functioning showed higher susceptibility to coercion, no evidence of actual coercion was found24. Despite these null findings, there remains the possibility of undue influence and therapeutic misconception (of which one aspect may be conflating the role of the clinician and researcher—particularly if that happens to be the same individual)25.
In the present analyses, patients were observed as making more errors during the Understanding subscale than during the remaining MacCAT-CR subscales. This finding may be counterintuitive in that adequate appreciation and reasoning abilities would logically seem to be dependent upon good understanding of the study information. However, we suspect that this finding is an artifact driven by the nature of the MacCAT-CR in that the Appreciation and Reasoning subscales contain far fewer items than the Understanding subscale. Thus, those subscales are likely less sensitive in eliciting patient error.
In interpreting our results, it is important to remember that participants’ incorrect responses were classified according to the apparent source of the error and that we cannot be certain of the true source of the error. For example, errors classified as “difficulty recalling” could be due to difficulty encoding, storing, or retrieving the information. Alternatively, this type of error may have resulted from a number of other intrapersonal or contextual factors26. Personal limitations such as poor attention, low motivation, or fatigue may have affected an individual’s responses, as may have situational demands, such as being presented information that was too complex or being presented information in a manner unconducive to memory. Furthermore, deficits in pragmatic or social communication27 may have contributed to difficulties understanding the questioners’ intent or responding appropriately.
It is also important to recognize that, in the context of the present study, decisional capacity was assessed in reference to consent for an outpatient naturalistic study of the short- and long-term side effects of FDA-approved antipsychotic medications. Contrast this context with that of a placebo controlled clinical trial conducted in an acute inpatient psychiatry setting, and/or in the context of capacity to competently refuse treatment among acutely psychotic persons (e.g., refusal of hospitalization or medication). Clearly, the association between capacity and psychopathology may differ across these different contexts. Moreover, the considerations of respect for autonomy versus protection of those with diminished capacity for authentic autonomous decision making may be more complex, or at minimum, distinct. Thus, the types of errors, and the source of errors seen in the present sample may not readily generalize (either in terms of statistical association or ethical implications) to distinctly different contexts.
This study has additional limitations. As the protocol was not originally designed for a qualitative examination of the data, some errors may reflect interviewer (rather than patient) error—e.g., failing to record a patient’s response verbatim. The way we evaluated the MacCAT-CRs also introduced limitations. First, a degree of subjectivity is inherent in how raters classified patients’ responses, which likely impacted the initial interrater agreement rates (Table 1) prior to the final codes being determined by group consensus. Subjectivity may affect interrater reliability28, which itself puts an upper limit on validity29. However, common procedures to improve apparent objectivity (such as more structured response formats) do not necessarily improve reliability30, and even perfectly reliable measures may lack validity29. The bottom line is that the ideal (most reliable and valid) method of classifying respondent errors during capacity assessment is not presently known. The present study illustrates one potential approach, but additional research on error types would be useful in developing and evaluating other potential methods which could be more routinely applied in determining where subjects in a given study or population are most likely to have difficulty with routine consent procedures. A second limitation is that some MacCAT-CRs were evaluated more than once during the coding scheme development process, in order to refine our coding methods. Third, although we feel that our group decisions represent a more accurate depiction of patients’ responses, rating each MacCAT-CR via group consensus limited individual raters’ independence. Finally, there are several known limitations to the MacCAT-CR measure itself31, limiting, to some degree, the validity of our codes. As mentioned above, the Appreciation and Reasoning subscales contain few items, constraining the amount of information that was elicited from patients and making comprehensive evaluation of their appreciation and reasoning abilities difficult.
Nevertheless, these results inform ways to improve consent procedures to increase the likelihood that patients with schizophrenia are making informed decisions regarding research participation. Given patients’ recall difficulties, key study information should be highlighted and repeated during consent procedures, while some information may require less explanation. For example, in the present sample, the vast majority of patients recognized that participation would be voluntary, but a significant proportion had difficulty recalling the purpose of the study. This suggests that spending a great amount of time covering voluntariness is unnecessary, while additional explanation of other key points is warranted.
The finding that some patients provided rationales for participation that were considered overly vague or otherwise limited in content suggests that some patients have difficulty verbalizing their reasons. Alternatively, such lack of clear response may be due to limitations in the Reasoning subscale. As probing may be required to obtain adequate information from patients about their decision-making process and reasons, consent procedures should include an interactive discussion of patients’ reasons for and against participation. This would have the additional salutary effect of providing researchers the opportunity to clarify or re-explain aspects of the study, should patients’ reasoning seem faulty or vague.
Researchers should be aware that potential participants may confuse the goals and purposes of research and clinical care32. In the present study, some patients did have difficulty appreciating the differences between the two or overemphasized how being in the study might benefit them personally. However, they were being asked to consider consenting to a minimal-risk study that could actually have some benefit for them (e.g., they would receive comprehensive evaluations), so therapeutic misconception is not as concerning in this situation. Future studies should continue to evaluate the nature and prevalence of therapeutic misunderstanding in higher-risk studies, like randomized placebo-controlled clinical trials. Qualitative analysis of patients’ responses during DMC assessment within such contexts could elucidate points of confusion for patients and suggest ways to improve consent procedures to reduce therapeutic misconception. For all studies—but higher-risk protocols in particular—consent procedures should emphasize the differences between research and clinical care. Again, an interactive discussion would help the researcher understand patients’ beliefs and address any misconceptions.
Interventions designed to improve DMC have yielded promising, but mixed, results33. Consideration of the errors that patients make during DMC processes, as suggested by the present study, may inform the development of more targeted interventions. Modifications to consent procedures that should be evaluated in future studies include emphasis and repetition of key study information, clarification and emphasis of distinctions between research and clinical care, and expanded discussion and assessment of patients’ abilities to appreciate and reason about the presented information. Finally, consistent with ethical obligations of researchers to ensure that participants have understood the relevant information, the consent process should be viewed as a conversation and information sharing process, rather than merely the reading and signing of the consent form by potential participants34.
Knowledge of the types of errors that individuals make during decision-making for research participation may also help IRBs to provide further guidance to researchers regarding the informed consent process. IRBs aptly regulate what study information is disclosed to potential participants but typically give less attention to how that information is conveyed. As described above, our results point out some weaknesses in the informed consent process, at least for this particular population of clinically-stable outpatients with schizophrenia, and suggest possible strategies for improvement. IRBs could use this information to encourage researchers to employ more novel and interactive methods of information disclosure.
Allison R. Kaup, San Diego State University / University of California, San Diego Joint Doctoral Program in Clinical Psychology Department of Psychiatry, University of California, San Diego.
Laura B. Dunn, Department of Psychiatry, University of California, San Francisco.
Elyn R. Saks, Gould School of Law, University of Southern California, Department of Psychiatry, University of California, San Diego.
Dilip V. Jeste, Department of Psychiatry, University of California, San Diego, Sam and Rose Stein Institute for Research on Aging, University of California, San Diego.
Barton W. Palmer, Department of Psychiatry, University of California, San Diego, San Diego Veterans Medical Research Foundation.