|Home | About | Journals | Submit | Contact Us | Français|
Drs. Choi and Medalia designed the study, contributed to writing the grant and protocol, supervised research assistants in collecting the data, and made significant contributions to writing up the manuscript. Both authors approved the final manuscript.
Amotivation is a telling hallmark of negative symptomatology in schizophrenia, and it impacts nearly every facet of behavior, including inclination to attempt the difficult cognitive tasks involved in cognitive remediation therapy. Experiences of external reward, reinforcement, and hedonic anticipatory enjoyment are diminished in psychosis, so therapeutics which instead target intrinsic motivation for cognitive tasks may enhance task engagement, and subsequently, remediation outcome. We examined whether outpatients could attain benefits from an intrinsically motivating instructional approach which (a) presents learning materials in a meaningful game-like context, (b) personalizes elements of the learning materials into themes of high interest value, and (c) offers choices so patients can increase their control over the learning process. We directly compared one learning method that incorporated the motivational paradigm into an arithmetic learning program against another method that carefully manipulated out the motivational variables in the same learning program. Fifty-seven subjects with schizophrenia or schizoaffective disorder were randomly assigned to one of the two learning programs for 10 thirty-minute sessions while an intent-to-treat convenience subsample (n=15) was used to account for practice effect. Outcome measures were arithmetic learning, attention, motivation, self competency, and symptom severity. Results showed the motivational group (a) acquired more arithmetic skill, (b) possessed greater intrinsic motivation for the task, (c) reported greater feelings of self competency post-treatment, and (d) demonstrated better post-test attention. Interestingly, baseline perception of self competency was a significant predictor of post-test arithmetic scores. Results demonstrated that incorporating intrinsically motivating instructional techniques into a difficult cognitive task promoted greater learning of the material, higher levels of intrinsic motivation to attempt the demanding task, and greater feelings of self efficacy and achievement to learn.
As cognitive remediation interventions for schizophrenia emerge from laboratory efficacy trials to be disseminated into the community, it is important to investigate methods to maximize outcome in hospital clinics and community day programs (Bell et al., 2005, Heinssen et al., 2000, Hodge et al., 2008, Kern et al., 2008, McGurk and Wykes, 2008, Silverstein and Wilkniss, 2004, Twamley et al., 2008). Studies are now underway to investigate possible mediating factors, such as learning potential, treatment intensity, and group versus individual modalities (Kurtz et al., 2007). However, very little has been published about the interplay between neurocognitive recovery and the psychological states which influence the actual learning process. One such mechanism beginning to receive more empirical examination is the role of intrinsic motivation (IM) and its associated derivatives in neurocognitive recovery (Barch et al., 2008, Medalia and Richardson, 2005, Velligan et al., 2006). A recent paper by Nakagami et al. (2008) studied the meditational role of IM between neurocognition and psychosocial outcome in community-based psychiatric rehabilitation programs. The authors found that IM promoted neurocognitive improvement and that IM was a decisive mechanism for explaining the relationship between neurocognition and psychosocial functioning in schizophrenia.
The underpinnings of IM derive from the needs for competency, autonomy, and an interpersonally supportive climate (Ryan and Deci, 2000). In healthy control college students, IM is fundamental to persistence and adherence on tasks that involve intricate information processing (Deci and Ryan, 2008). In much the same way, when patients with psychosis are intrinsically motivated for a difficult treatment, they engage in targeted behaviors because of the interest, enjoyment, and satisfaction derived from their engagement in the activity, rather than exclusively due to external rewards (Medalia and Freilich, 2008). Consequently, intrinsically motivated behaviors are repeated without external reinforcement, making their maintenance within a treatment setting more likely. Recent studies have also found impairments in various aspects of the human motivation and reward system in schizophrenia that play a critical role in hedonic experiences of anticipatory pleasure (Gard et al., 2007) and the ability to understand the inherent value of training tasks and external rewards (Gold et al., 2008). Therefore, it is pragmatic and scientifically prudent to focus efforts on targeting and increasing the innate enjoyment and value patients place on the training itself rather than relying on platforms of external incentive.
Up until recently, a major focus of most program development in cognitive remediation has been the conception and design of cognitive training tasks. The field has relied on experimental neuroscience to supply the drill and practice training exercises and ensure activities are relevant to targeted neuroanatomical regions (Wexler et al., 2000, Wykes et al., 2002). The emphasis has been on the validity of exercises as they pertain to principles of neural plasticity. As a first step in the development of this field, this focus was understandable and empirically necessary. However, the time is now ripe to take advantage of the wealth of empirical literature from learning and motivational psychology, which offers multiple insights into how people with compromised brain and motivation systems acquire and maintain cognitive skills. Learning theories and motivational science provide the techniques and forum to understand the underlying learning processes involved in the acquisition of cognitive skills, and importantly, provide guidance in how to effectively teach cognitive skills to promote engagement and maximize learning. In fact, learning and motivation psychology are fields specific to teaching people how to learn new skills despite impairments, disabilities, motivational deficits, and task difficulty (Elliot and Dweck, 2005). It is therefore a reasonable progression for relevant learning and motivational science findings to be used to hone cognitive treatment for schizophrenia, especially in terms of improving engagement for demanding cognitive tasks and effectively disseminating the treatment into the community.
The literature on teaching techniques that enhance IM in normal students emphasizes the beneficial role of presenting learning exercises in meaningful contexts of inherent appeal to the student. This can be achieved even when the learning content involves abstract concepts, by embedding the material in “fantasy” contexts involving topics, themes or characters of interest to the students (Parker and Lepper, 1992). Other instructional techniques that are known to enhance IM to learn involve personalizing incidental features of the learning exercise, and giving students some control by providing choices over aspects of the activity with which they can engage (Cordova and Lepper, 1996). Providing people with a menu of choices increases the sense of control and autonomy, which according to motivation theory, in turn increases IM (Deci and Ryan, 2008). This three-pronged instructional approach was found to enhance learning in arithmetic (Cordova and Lepper, 1996), a demanding subject for many adolescent students and adults alike. So far, the effects of this motivational approach on enhancing learning in individuals with schizophrenia have not been examined.
This goal of this study was to investigate the utility of applying the three-pronged motivational approach studied by Cordova and Lepper (1996) to enhance the learning process in adults with schizophrenia/schizoaffective disorder. Specifically, we examined whether or not adults with schizophrenia/schizoaffective disorder could attain cognitive and psychological benefits from an approach which (a) presented learning materials in a meaningful context, (b) personalized material into themes of high interest value, and (c) offered choices so that patients could increase their control over the learning process. We directly compared two methods of computer-based learning in people with schizophrenia. One method incorporated the motivational approach of contextualization, personalization, and choice into an arithmetic learning program while another method carefully manipulated out the motivational techniques in the same learning program. It was hypothesized that relative to participants in a similar program without motivational elements, adults with schizophrenia engaged in a computerized arithmetic learning program which incorporates personalization, contextualization, and choice would (a) show greater acquisition of arithmetic skills, (b) report greater feelings of self-competency, and (c) report higher levels of intrinsic motivation for the demanding task.
Participants in the study were (a) 57 outpatients, ages 21 to 55, diagnosed with schizophrenia or schizoaffective disorder enrolled in a computer-based arithmetic learning program and a comparable convenience subsample of 15 outpatients with schizophrenia/schizoaffective disorder not involved in the training protocol. Participants were recruited at the VA Connecticut Healthcare System in West Haven, CT, and the New York State Psychiatric Institute in New York City. Prior to participation in the study, participants had to be psychiatrically stable for at least 30 days on any psychotropic regimen. Diagnosis was confirmed by the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-IV) (First, 1996). Any participants with significant auditory/visual impairment, lack of fluency in English or medical illnesses known to impair brain function, other than schizophrenia, were excluded. Participants who met criteria for current substance abuse/dependence were also excluded, including those with active substance abuse 30 days prior to intake.
Estimated premorbid intelligence for each participant was established with the vocabulary subtest scaled score from the Wechsler Adult Intelligence Scale-Revised (WAIS-R; Wechsler, 1981) and the reading subtest standard score from the Wide Range Achievement Test-Third Edition (WRAT3; Wilkinson, 1993).
The Continuous Performance Test-Identical Pairs Version (CPT-IP; Cornblatt et al., 1988) was used to measure the capacity to sustain focus on critical information in an attention-demanding visual environment. The CPT-IP serially presents two types of stimuli (numbers and nonsense shapes) which appear on the screen at a rate of 1 stimulus per second. The participant is instructed to respond as quickly as possible by pushing a control button whenever two identical stimuli are presented simultaneously. Major indices analyze performance by number of correct responses to target, false positives, random errors, and d-prime.
Self-reported IM was measured using an adapted version of the Intrinsic Motivation Inventory (IMI; Plant and Ryan, 1985), a 7-point Likert-type scale designed to assess a participant’s subjective experience of an activity specifically in an experimental setting. The instrument assesses the participant’s interest/enjoyment, effort, and value/usefulness while performing a given activity (“I enjoyed doing this activity very much”), with higher scores indicative of greater IM for the task. The scale is highly associated with germane constructs of motivation for health-related behaviors, including perceived competency for attempting challenging tasks and autonomous treatment engagement (Deci et al., 1994). The adapted version validated for schizophrenia has 21 items and possesses good internal consistency (alpha=.92) and test re-test reliability (ICC=.77) (Choi et al., 2009).
Perception of self-efficacy is a central constituent of IM and a strong predictor of high levels of motivation in educational and treatment settings (Williams and Deci, 1996). The Perceived Competency Scale (PCS; Williams and Deci, 1998) was used to measure the participant’s outlook on completing and mastering the learning exercises. The PCS consists of 4 items on a 7-point Likert-type scale ranging from “not at all true” to “very true” (i.e. I feel confident in my ability to learn the computer program; I am able to achieve my goals in this program), with higher scores indicative of greater feelings of self competency for the task. The questionnaire has shown good validity and consistency in repeated studies examining its factor loadings related to internalized motivation and interest, and possesses excellent internal consistency (Cronbach’s alpha=.80–.94) (Williams and Deci, 1996).
The Treatment Self-Regulation Questionnaire (TSRQ; Ryan et al., 1995) was used to ascertain why participants engaged in the learning program. The degree of autonomous behavior for a treatment regimen is highly correlated to IM for the treatment. If a person's motivation for health behaviors is relatively autonomous, then motivation for achievement is internally driven and the probability of successfully completing treatment is dramatically increased (Williams and Deci, 1998). The TSRQ consists of 18 items on a 7-point Likert-type scale ranging from “not at all true” to “very true”, with higher total scores on the Relative Autonomy Index indicative of greater feelings of autonomy for treatment participation. Items query as to why the participant entered and continued in the program (i.e. I decided to enter this learning program because people told me to). The TSRQ posses high reliability and internal consistency in studies examining motivation for psychotherapeutic treatments and continued program attendance (Cronbach’s alpha=.80–.87).
Baseline arithmetic skill and direct learning of arithmetic operations post-treatment was measured by a 48-item paper-and-pencil arithmetic test used by Columbia University Teacher’s College to assess general arithmetic ability in young adults. It comes in 4 alternative forms to address possible “training to test” effects, and evaluates the participant’s knowledge and calculation skills in addition, subtraction, division, multiplication, use of parentheses, and order of operations (e.g. “(4 + 5)×5 = __”). Overall pre-treatment skill level and direct learning in arithmetic was measured by the total number correct on tests, which ranged from 0 to 60.
Psychiatric symptomatology was measured by the expanded Brief Psychiatric Rating Scale (BPRS), an updated version of the original BPRS (Overall and Gorham, 1962). The expanded BPRS is a 24-item, self-report measure which quantifies the level and presence of psychopathology on a 7-point Likert type scale ranging from “not present” to “extremely severe”. We parsed the BPRS into the standard 4-factor solution of the 24-item BPRS (Kopelowicz et al., 2008) to examine any possible impact the interventions may have on symptom categories.
Treatment intensity is a valuable functional barometer which needs to be considered when examining the link between treatment engagement and behavior in schizophrenia treatment studies (Heinssen and Cuthbert, 2001). Treatment intensity was determined by the number of days it took participants to complete the assigned intervention. Since a research assistant conducted a set number of sessions a week where participants were allowed to come in anytime during those sessions to work on the lessons, treatment intensity was determined by the participants themselves.
A description of the study was given to all participants who provided written informed consent in accordance with each respective hospital Institutional Review Board. Following baseline assessment on all measures, 57 participants were randomly assigned to one of two conditions: (a) the motivation condition, which employed the motivational paradigm into a personalized and contextualized fantasy arithmetic program which allowed choice over game elements (b) the comparison condition, which consisted of a simple adaptation of the arithmetic program void of any fantasy context, personalization of instructional references, or choice of learning elements. The learning programs consisted of 10 thirty-minute sessions to be completed over a 4 week period. Importantly, participants could work on the lessons at their pace. A research assistant conducted 4 sessions a week where participants were allowed to come in anytime during those sessions to work on the lessons. Post-testing was completed within 2 days of the last lesson by a research assistant blind to the randomization. The convenience subsample (n=15) underwent the same battery of tests, four weeks apart, to directly control for practice effects associated with taking the same measure on multiple occasions within a relatively brief period of time (Goldberg, 2007). To mimic conditions of the actual treatment provided to the training groups, the subsample was briefly shown a computerized math learning exercise each time and instructed to complete the self-reports of motivation, perceived competency, and treatment motivation in the context of attempting the math lesson.
Arithmetic provides a gauge of direct domain-specific learning that allows the researcher to quantify the degree of material absorption from a specific lesson or intervention. Arithmetic measures have a distinct history of being used to assess dorsolateral frontal lobe and angular gyrus functioning in adults (i.e. MMSE), to the extent that such measures produce a quantifiable result that accurately reflects an individual’s attention and working memory on multi-step problem-solving tasks. Therefore, an arithmetic modality of treatment and assessment served as a valid instrument to measure direct learning and changes in executive ability from an intervention.
In this study, two equivalent versions of a commercially-available computer math game (“How the West was 1 + 3 × 4”) were created with motivational variables selectively manipulated to examine the impact of variables on attention and the learning process. Both parallel versions involved the same fundamental procedure. The game board consists of a numbered line from 1 to 60 with the first one to reach 60 declared the winner. In each trial, working memory and executive ability are needed to combine three numbers generated by the computer into a valid arithmetic expression. Participants employ arithmetic knowledge and mental flexibility by using any number of addition, subtraction, division, multiplication, and parenthetic operations to create the correct expression. The resulting value of the created expression is the number of spaces the participant can advance on the screen. If the participant cannot provide a valid arithmetic expression using the generated numbers, the computer provides instructional feedback in the form of a brief arithmetic lesson and offers the correct answer. However, the participant was not allowed to advance unless a correct answer was provided. Both versions had identical feedback and lesson content. This program was a single player program where the participant challenged the computer at varying levels of difficulty.
In the motivation version, participants played a version called the “Wild Wild West” (see Figure 1). In this version, relevant Western theme icons and story lines are incorporated into the game board and prologue. All participants assigned to the motivation condition completed a questionnaire prior to the first session inquiring about specific personal preferences such as the first names of close friends, favorite foods or hobbies, and nicknames. This information was inserted into the personalized fantasy context replacing generic references for characters and icons. Participants also had control over game elements by choosing from a menu of options during the lessons (e.g. difficulty level, type of player icon such as a stagecoach or train).
Participants in the control condition played a parallel but austere adaptation called the “Math Game”. In contrast to the motivation condition, the “Math Game” was void of any contextual environment, personalized instruction and preferences, or choice of game elements, as the participant had to merely move a triangle to a finish line by generating and solving the same arithmetic operations (see Figure 1).
First it was confirmed that the distribution of all dependent measures conformed to assumptions underlying the use of parametric statistical procedures. To ensure that the three groups were similar on baseline tests scores, we compared each of these scores across the three groups via one-way analyses of variance (ANOVA). To assess the direct impact of the arithmetic learning programs on arithmetic performance, attention, symptoms, and motivational constructs, we compared post-assessment scores on total number correct on the arithmetic test, major indices of the CPT-IP, BPRS total and factor scores, and total scores on the IMI, PCS, and TSRQ. We entered these scores into a series of one-way ANOVAs, with condition as a between-subjects variable. To strengthen this analysis, we also conducted a series of one-way analyses of covariance (ANCOVA). This allowed for greater control of subtle, but potentially influential baseline differences in performance that are not necessarily picked up when comparing similar baseline scores. We included Tukey HSD to correct for multiple comparisons, and also calculated effect sizes (ES) between the motivation math game and math game groups. Finally, to explore the possible role of baseline attention and psychological structures as independent variables for predicting change in arithmetic ability, we conducted a step-wise multiple regression analysis for subjects in the arithmetic training conditions, with baseline arithmetic scores entered first, treatment self regulation second, attention third, then intrinsic motivation, followed by perceived self competency. This order of entry was based on past associations found between each construct and learning in arithmetic (Choiet al., 2009).
No significant differences were evident between the two math game groups or convenience subsample on demographic or clinical variables (see Table 1). There were also no significant between-group differences on any of the baseline measures (Table 2) (all p’s>.10). As shown in Table 2, the one-way ANOVA for post arithmetic ability revealed a main group effect (F [2, 68] =7.13, p=.04) with both math game conditions producing improvement relative to the non-trained subsample participants. However, the post hoc showed that the motivational math game condition made greater gains in arithmetic relative to the math group (p=.03). In addition, there was a main group effect in intrinsic motivation for the arithmetic lessons (IMI; F [2, 68]=6.86, p=.03), perceived self competency (PCS; F [2, 69]=5.76, p=.05), and the number of false positives on the CPT-IP (F [2, 67]=4.98, p=.05), with the motivational math game condition producing improvements relative to the comparison math game and non-trained subsample participants at post-assessment. Effect size calculations further underscore the differences between groups as a result of the motivational intervention, with medium to large effect sizes noted in motivation, perceived competency, false positives on the CPT-IP, and direct learning (Table 2). Interestingly, other major indices of the CPT-IP were not significant between groups at post-testing (d-prime, p=.07; correct responses to target trials, p=.09; random errors, p=.12) nor were there any significant symptom changes among the groups on the BPRS or its factors (Positive Factor, p=.28; Negative Factor, p=.08; Agitation-Mania Factor, p=.27; Depression-Anxiety Factor, p=.32) or the TSRQ. (p=.21). These analyses were re-run with ANCOVA using baseline scores as covariates, but there were no changes in this pattern of findings, thus demonstrating further that the minute differences in baseline scores did not influence outcome.
Significantly, the motivational math game group completed the assigned 10 session in less time than the comparison math group (t=3.7, p=.04). One-way analysis of covariance (ANCOVA) was calculated for all outcome analyses using this barometer of treatment intensity as a covariate, but no effect was found for treatment intensity on improved scores in the motivational math group (range of ps>.12).
To further explain the relationship of baseline attention and psychological structures to change in arithmetic ability following treatment, we entered baseline performances on arithmetic, CPT-IP false positives (the only significant CPT index between groups), IMI, PCS, and TSRQ into a multiple linear regression equation. The model, which partitions equally exclusive components of the overall variance for each variable, explained a significant portion of the variance in the total number of correct on the arithmetic test at the end of the learning programs (R2=.59, F[1,56]=6.81; p=.001). Higher PCS scores (β =.40, t=2.68, p=.03) predicted greater change in arithmetic scores following the intervention, even after variance attributable to measures of attention, perceptions of treatment autonomy, and intrinsic motivation were accounted for. The results of the stepwise regression of baseline arithmetic ability (step 1), perceptions of treatment autonomy (step 2), attention (step 2), intrinsic motivation (step 3), and perceived self competency (step 5) are presented in Table 3. In the first 4 steps, 48% of the variance is explained, with the addition of intrinsic motivation (IMI) nearly making a significant contribution to post arithmetic ability similar to baseline arithmetic ability. However, at the final step, only perceptions of self competency (PCS) predicted post arithmetic scores, thereby, presenting clear evidence of the association between perceptions of self competency and improvements in learning, even when variance for baseline arithmetic ability, attention, motivation, and feelings of treatment autonomy were controlled.
This research was conducted with the broad intention of elucidating the role of IM in cognitive recovery and identifying treatment components which measurably contribute to improved cognitive performance in schizophrenia. The results found medium to large effect sizes when incorporating instructional techniques that enhance IM into a difficult learning task. The IM instructions promoted greater learning of the material, higher levels of IM to attempt the demanding exercise, and greater perceived self competency to learn. Furthermore, only for the group receiving motivation enhancing instructions did these benefits generalize to a non-trained test of attention, specifically an index measure on the CPT-IP that gauges attentional resource allocation. These benefits were evidenced after a brief stint of 10 sessions, with cognitive demands highly controlled for between the two groups learning arithmetic. Instruction and feedback were identical in content and carefully designed to curtail alternative cognitive interpretation as to why the motivational group benefited more from the arithmetic program. Overall, these results suggest that people with schizophrenia do respond to motivationally enhancing instructional techniques, and that incorporation of these techniques into training exercises significantly boosts the amount of learning that occurs. This is consistent with how IM affects learning behavior in healthy samples (Cordova and Lepper, 1996), and suggests some qualitative similarities in the determinants of IM in schizophrenia and healthy controls (Choi and Medalia, 2009).
Motivational theory posits that perceived competency and autonomy are key determinants of IM (Deci and Ryan, 2008). This is supported by research in healthy controls that found beliefs of self-and-content mastery can be so influential they can delineate the degree of improvement on challenging cognitive tasks even more so than general intelligence and cognitive ability (Molden and Dweck, 2000). The findings in this study support the notion that these constructs are operative in schizophrenia; instructional techniques that enhanced perceptions of self competency also enhanced learning. So, similar to the non-psychiatric population, people with schizophrenia must believe their actions can produce the outcomes they desire (self competency) or else they may have little incentive or motivation to take on challenging cognitive tasks in remediation programs (Schunk and Zimmerman, 2006, 2007).
Another product of enhancing motivation was that subjects who were exposed to IM instruction strategies chose to attend sessions at a higher rate than those in the comparison math game condition, thereby increasing treatment intensity. Although increasing frequency of attendance for remediation programs is generally considered a desirable result so that patients can attain an adequate dosage in the briefest span of time, consistent with other studies that looked at treatment intensity as a process variable (Kurtz et al., 2008), we did not find greater treatment intensity to be related to better outcome. In our case, greater treatment intensity was not a mediator of learning, but a product or rather a demonstration of how appealing motivational manipulations can be, even for adults with cognitive impairment.
Despite the strong effects obtained in this study, the results should not be taken as an endorsement of exclusive use of the particular motivationally enhancing instructional strategies used. Using fantasy to create opportunities for personalization, choice, and context is only one way to achieve motivational enhancement. The motivational math game was well received by our sample, with subjects noting how much they enjoyed the fantasy component (“I liked beating the stagecoach”, “It makes math almost fun”). However, these specific techniques may not be received so well by every type of learner, and for some, the fantasy gaming experience may be pitched too low to offer any benefits. For example, people who are highly goal directed and focused on the utility of an activity may find the fantasy elements to be a distraction. For these learners, competency and autonomy are also important for sustaining motivation to learn, but the instructional techniques that enhance these psychological states may be different (Eccles and Wigfield, 1995). The results of this study are not intended to focus attention on the specific instructional manipulations used, but rather the role of IM enhancing instructions on learning in schizophrenia. Nevertheless, this study contributes to the growing literature on cognition and IM in schizophrenia. Further investigations examining the potential relationship between our study of treatment-specific IM, trait IM (Barch et al., 2008), and cross-situational IM (Nakagami et al., 2008) may yield new insights into the relationship between motivation, cognition, and psychosocial rehabilitation.
A limitation to be addressed in future trials is that this study did not examine what specific effect individual motivational instructions had in improving outcome but rather gauged the synergic value of incorporating all three strategies. In addition, the study may have been insufficiently powered to detect changes on the primary index of the attention measure we used. For the CPT-IP, the index most frequently reported is d-prime, as it is a more inclusive index than false positives for signal detection. We are not certain why the motivation group made significant gains on the false positive index but not d-prime. It could be that the arithmetic learning program specifically isolates the training task of inhibiting incorrect responses as the participants must constantly attend and select the correct numbers/symbols to complete the equations while ignoring irrelevant numbers/symbols. As reported in the Results section, d-prime does appear to be approaching significance (p=.07) for the motivational group, so there does seem to be a trend emerging that points to greater motivation influencing overall attentional gains, and with a larger sample we may have found significant changes on d-prime, as well. Another potential limitation of the study is that patients with better computer skills prior to intervention may have had an advantage on the computer programs during treatment. Follow-up studies that extricate the active motivational ingredient(s) within the instructional paradigm and stratify participants according to computer familiarity and skill may provide additional insight into efficacious treatment and participant characteristics that influence response to computer-based rehabilitation treatment in patients with schizophrenia.
Drs. Choi and Medalia would like to thank Joanna Fiszdon, Ph.D., Kellie Smith, M.A., and Katie Tobin, Ph.D., for their assistance in data collection.
Role of Funding
This research was supported by a NIMH 1 R03 MH071733-01A2 to Drs. Choi and Medalia. NIMH had no further role in study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflict of Interest
Drs. Choi and Medalia have no conflict of interests to disclose.