|Home | About | Journals | Submit | Contact Us | Français|
Background: The “clinical high risk” (CHR) construct was developed to identify individuals at imminent risk of developing psychosis. However, most individuals identified as CHR do not convert to psychosis, and it is unknown whether these nonconverting individuals actually recover from an at-risk state.
Methods: Eighty-four prospectively identified patients meeting CHR criteria, and 58 healthy comparison subjects were followed in a 2-year longitudinal study. Analyses examined rates of conversion, clinical, and functional recovery. Proportional cause-specific hazard models were used to examine the effects of baseline and time-varying predictors on conversion and remission. Trajectories of symptoms and psychosocial functioning measures were compared across outcome groups.
Results: Competing risk survival analyses estimated that 30% of CHR subjects convert to psychosis by 2 years, while 36% symptomatically remit and 30% functionally recover by 2 years. Lower levels of negative and mood/anxiety symptoms were related to increased likelihood of both symptomatic and functional recovery. CHR subjects who remitted symptomatically were more similar to healthy controls in terms of both their baseline and longitudinal symptoms and functioning than the other outcome groups.
Conclusions: Nonconverting CHR cases represented a heterogeneous group. Given that nonconverted subjects who remitted symptomatically also presented initially with less severe prodromal symptomatology and showed a distinct normative trajectory of both symptoms and psychosocial functioning over time, it may be possible to refine the CHR criteria to reduce the number of “false positive” cases by eliminating those who present with less severe attenuated positive symptoms or show early improvements in terms of symptoms or functioning.
Schizophrenia and other psychotic disorders are seriously disabling, and available treatments are limited and mostly palliative in nature. In response to the devastating personal and societal costs of psychotic disorders, researchers have emphasized the importance of early identification in the hopes of preventing psychosis. Consequently, clinical risk criteria were developed to identify individuals at imminent risk for developing the illness, yet prior to full onset of psychotic symptoms.1,2 These clinical high risk (CHR) criteria are defined by the presence of attenuated psychotic symptoms and/or genetic risk and functional deterioration. Although a variety of psychosis risk criteria have been operationalized (eg, basic symptoms criteria3,4), those referring to imminent risk prediction (eg, ultrahigh risk1 and CHR2) have yielded conversion rates over 2 years of approximately 35%, with rates reportedly declining to 12% in some studies.5–9 While the CHR criteria represent a strong and reliable predictor of imminent psychosis outcome, it is notable that the majority of putatively prodromal individuals do not go on to develop a psychotic illness.
A question of major importance is whether the nonconverting CHR patients represent “false positives” from the perspective of risk ascertainment. That is, to the extent that the presenting symptoms and functional deficits that define the CHR syndrome actually remit within the 2 years of ascertainment, such individuals would no longer be considered to be at imminent risk for psychosis. Currently, very little is known about the predictors of symptomatic remission and functional recovery among individuals identified as meeting criteria for a putative prodromal syndrome. Such knowledge would aid in efforts to refine the risk syndrome criteria and limit exposure of false positive cases to interventions that carry some side effect burden.
Simon and Umbricht10 attempted to address the false positive issue by investigating 1-year diagnostic outcomes in individuals initially identified as fulfilling CHR criteria. Findings indicated that 59% no longer met criteria for a prodromal syndrome after 1 year, suggesting that current criteria for a psychosis risk state are likely to “mislabel” individuals with transient psychotic-like symptoms. While that study provided a critical first step in addressing the false positive problem, it was limited by an absence of information related to individuals’ psychosocial functioning, the use of a dichotomized outcome variable (“cases” vs “noncases”), and reliance on a single follow-up time point, thus precluding analysis of the trajectory of change in symptoms over time. More recently, Addington and colleagues11 investigated the clinical and functional status of those who did not convert to psychosis over a 1- to 2-year follow-up period, finding that the nonconverting group showed significant improvement in attenuated positive symptoms (APSs), negative symptoms, and social and role functioning over time, with over half showing a remission of prodromal symptoms. While this study characterized outcomes among nonconverters, several unanswered questions remain; in particular, what are the factors that predict symptomatic improvement and functional recovery? What is the time course of recovery? And lastly, is it possible to differentiate such clinical outcomes at the time of ascertainment or is continuous monitoring required? In order to address some of these questions and to better characterize outcomes in individuals who appear to remit from an at-risk state, it is important to investigate the course of both clinical symptoms and psychosocial functioning.12
The primary aims of the current study were to examine the trajectories of clinical symptoms and social and role functioning in a sample of CHR youth who were not observed to develop a psychotic disorder over the 2-year follow-up compared with CHR youth who converted to psychosis. In particular, we hoped to identify factors, such as negative symptoms, mood and anxiety symptoms, and psychosocial functioning, that were associated with higher rates of conversion and remission and to determine the rates at which CHR subjects experienced those events. In particular, we examined whether baseline symptom severity and psychosocial functioning were significantly associated with the likelihood of symptomatic remission and functional recovery, in order to determine whether these variables could aid in revising CHR criteria to reduce the number of cases falsely identified as high risk.
CHR participants (N = 125) were recruited from the Staglin Music Festival Center for the Assessment and Prevention of Prodromal States (CAPPS) at the University of California at Los Angeles (UCLA) from 2001 to 2009. The participants enrolled in the current study were help-seeking individuals who were referred from the Los Angeles Unified School District (20%), the UCLA Neuropsychiatric Institute (25%), and community mental health centers (30%). Another 25% of our subjects were respondents to our website and/or advertisements. These individuals or their families contacted our clinic for an initial screening interview with the Structured Interview for Prodromal Syndromes (SIPS).13,14 Approximately, 25% of participants who completed the initial screening met the inclusion criteria (as described below), of which 94% consented to participate in the CAPPS program. Healthy control subjects (N = 58) were also recruited through advertisements posted at local schools and career fairs.
All CHR participants met inclusion criteria based on receiving a diagnosis of a “prodromal syndrome,” as defined by the SIPS.13,14 According to the SIPS, a prodromal syndrome is characterized by the presence of (1) APSs, (2) brief intermittent psychotic symptoms (BIPS), or (3) a 30% decline in global functioning over the past year and either a diagnosis of schizotypal personality disorder (SPD) or a first-degree relative with psychosis (genetic risk and deterioration; GRD). Further detailed information pertaining to SIPS syndromes, inclusion criteria, and interrater reliability are described elsewhere.13,15 CHR participants were excluded if they met Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, (DSM-IV) criteria for an Axis I schizophrenia-spectrum diagnosis based upon a Structured Clinical Interview for the DSM-IV TR Axis I disorders (SCID-I/P)16 interview. Healthy controls did not meet DSM-IV criteria for a psychiatric disorder as determined by the SCID-I/P, have a first-degree family history of a psychotic disorder, or meet criteria for a prodromal syndrome. In addition, participants were excluded from the study if they had a neurological disorder, drug or alcohol dependence within the past 6 months, or a Full Scale IQ below 70. All interviews were conducted by MA or PhD level psychologists who had participated in an in-depth “gold-standard” training program regarding the administration and scoring of the SIPS and SCID-I/P.
All participants completed informed consent or assent for the study and were compensated for their participation in all assessments. Parental informed consent for minors was also obtained. The UCLA Institutional Review Board approved study protocol and informed consent procedures.
Course of clinical symptoms and conversion to psychosis were assessed using the Scale of Prodromal Symptoms (SOPS), which is the symptom severity rating scale derived from the SIPS interview.14 Four domains of symptoms were measured: positive, negative, disorganized, and general symptoms. The present study focused primarily on the positive and negative scales. The SOPS positive subscale measures symptoms related to unusual thought content, suspiciousness, perceptual disturbances, grandiosity, and disorganized communication. Negative symptoms include anhedonia, avolition, flat affect, academic/occupational impairment, and decreased ideational richness. Symptoms were rated based on a 7-point severity scale, with 0–2 representing absent, questionable, or mild; 3–5 indicating moderate, moderately severe, or severe attenuated psychotic symptoms; and 6 representing a fully psychotic level of intensity. SOPS ratings were based on symptom severity over the past month. Conversion to psychosis is defined as the experience of positive symptoms at a psychotic level of intensity that have occurred for more than 1 hour/day at an average frequency of 4 days/week over a 1-month period. Psychotic intensity is defined as having full conviction in the experience of a positive symptom and/or experiencing psychotic symptoms that are seriously disorganizing or dangerous.
Our measure of mood and anxiety symptoms was based on the SOPS “dysphoric mood” item, which assesses depressed mood and anxiety. Questions used to evaluate this item include: “Do you ever generally just feel unhappy for any length of time?” and “Have you felt more nervous or anxious lately?” We opted to use this index rather than a DSM-IV diagnosis of mood or anxiety disorder, given that SCID interviews were only conducted at baseline and 1-year follow-up timepoints (or at the time of conversion), and we wished to maximize the amount of available data on the trajectory of mood/anxiety symptoms. Within our sample, there was 87% convergence between this SIPS item (ratings of 3 or higher) and an SCID-based DSM-IV diagnosis of a mood and/or anxiety disorder.
Psychosocial functioning within the past month was assessed with the Global Functioning Scale (GFS): Social and the GFS: Role.17,18 Both scales are scored on a 10-point scale, with 10 indicating the highest level of functioning, with scores of 6 and below indicating gradually increasing impairment across domains of functioning. Both scales were administered as direct clinical interviews, with each scale having its own set of probes. These scales have high interrater reliability and construct validity.19
After an intake evaluation that determined study eligibility, participants were assessed at baseline and as close as possible to 3, 6, 12, and 24 months thereafter. Clinical and functioning measures were collected at each time point. At the time of data analysis, no control subjects had reached the 24-month time point.
Statistical analyses were performed in SAS Version 9.2 and the R programming language. Our primary interest was in comparing CHR subjects who ultimately convert to psychosis to those who eventually experience a clinical or functional recovery (ie, false positives). For this study, we considered clinical recovery to have taken place when a subject no longer exhibited prodromal symptoms at the subpsychotic level of intensity (ie, the subject had ratings of ≤2 for all the SOPS positive symptom items). Ratings of 7 and higher on the GFS: Social and Role scales were designed to reflect levels of functioning within the normal range.19 Therefore, functional recovery was defined as achieving a score of 7 or higher over a 1-month period on “both” of these scales. Our primary predictive measures included SOPS positive and negative symptoms, represented by the total summed score of SOPS items P1–P5 (positive) and N1–N6 (negative), mood and anxiety symptoms as measured by the SIPS dysphoric mood item (as rated on a continuous, 0–6 scale) and the 2 psychosocial functioning scales.
While subjects who converted or remitted during our study could be classified with certainty, those who dropped out or completed the study without experiencing one of the primary outcome events were “censored,” as they ultimately could potentially have been part of either group. We therefore used survival analysis methods to examine factors relating to conversion and remission. We took the number of days from the baseline measurement, rather than the nominal visit time, because our measure of time both because there was substantial variation in actual visit spacing and because the continuous scale yields better results for survival analyses. First, to assess rates of conversion and remission, we used a competing risks model.20,21 In this framework, subjects are considered censored for the remission outcome once they have converted and vice versa, thereby taking into account the fact that these events are not independent. We considered 2 pairs of competing outcomes: (1) conversion and symptomatic remission and (2) conversion and functional remission. Cumulative incidence plots, which are the competing risks analog of Kaplan-Meier survival curves, were created to show estimated rates of conversion and remission over time. Next, to examine symptomatic and functional predictors of outcome, we used proportional cause-specific hazards models. In the competing risks context, these models can be fit using a Cox regression for each individual outcome with the competing event treated as a censoring point. We considered 2 types of predictors in our cause-specific hazards models: (1) Time-varying covariates to examine the overall effects of symptoms and functioning on outcomes on conversion and recovery and (2) Baseline covariates to assess the ability to predict outcome at the time of ascertainment. The primary predictors used in these models were positive and negative symptom scores, mood/anxiety symptom scores, and social and role functioning scores. Follow-up models were fit adjusting for age, gender, and medication status to check whether results remained robust. Survival analyses included all CHR subjects who had at least one follow-up measurement. Finally, we plotted trajectories of each of the symptom and functional variables with subjects grouped as healthy controls, CHR-converted, CHR-remitted, and CHR-neither converted nor remitted (CHR-NRC). While the ultimate outcome for this final set of subjects is unknown and thus we cannot perform formal tests based on these class assignments, the trajectory plots provide important information about the comparative courses of illness for these groups.
Of the 125 CHR patients enrolled in the study, 84 subjects had at least one follow-up clinical evaluation, with the majority having at least 2 follow-up visits (59.5%; 34 with 2 visits, 14 each with 3 and 4 visits, 16 with 5 visits, and 6 with 6 visits). Forty-one individuals were lost to follow-up, reflecting a 33% attrition rate. Independent sample t tests and/or chi-square tests found no significant baseline differences between those with and without usable data for the survival analyses (gender, χ2 = 1.09, P = .30; age, t = 1.77, P = .08; mother’s education, t = 0.45, P = .65; father’s education, t = 0.40, P = .69; positive symptoms, t =0.37, P = .71; social functioning, t = 1.36, P = .18; and role functioning, t = 0.55, P = .58). Although age did not differ significantly between the 2 groups, there was a trend for the subjects lost to follow-up to be younger.
There were no significant differences between patients and healthy controls on demographic variables (age, gender, parental education, and ethnicity) although as expected patients had significantly higher levels of positive symptoms and poorer psychosocial functioning (all P values < .0001). The demographic characteristics of the study participants at baseline are reported in table 1. Of the 84 subjects with follow-up data available, 27 converted to a psychotic disorder during the study, representing a 32% simple conversion rate (ie, not accounting for censoring). Of these 27 converters, 10 were originally diagnosed with a BIPS syndrome (37%) and 17 were diagnosed with APS (63%). Interestingly, 5 of these subjects experienced a temporary remission of their positive symptoms before ultimately converting. Of the 57 CHR subjects who did not convert during the study, N = 48 (84%) entered with an APS Prodromal Syndrome, 7 (12%) had a BIPS Syndrome, and 2 (4%) had the GRD syndrome.
The most prevalent DSM-IV diagnoses among the 84 CHR subjects at baseline were anxiety (N = 13 among those who ultimately converted and 27 among those who did not) and mood disorders (N = 11 in the converter group and 26 in the nonconverted group), with the majority of subjects (N = 53) having at least one of these diagnoses. A total of 16 of the 84 CHR subjects had a family history of psychosis in a first-degree relative, 4 in the converted group, and 12 in the nonconverted group. Fifteen of the CHR subjects who did not convert during follow-up entered the study on an antipsychotic medication. Among those who did convert during the study, 13 were taking an antipsychotic medication at study entry. All subjects received case management services offered by the CAPPS program and the majority of CHR patients (approximately 70%) received outpatient treatment services.
Cumulative incidence functions were computed based on competing risks models for (a) conversion vs symptomatic remission and (b) conversion vs functional recovery. The corresponding curves are shown in figure 1. Both models yield an estimated conversion rate of 30% at 2 years (95% CI 18–41%), with corresponding rates of symptomatic remission of 36% (95% CI 24–48%) and functional recovery of 30% (95% CI 20–41%). Of the 57 subjects who did not convert during the follow-up period, 17 met criteria for both symptom remission and functional recovery, 15 experienced symptom remission without functional recovery, and 15 met criteria for functional recovery without having had their symptoms remit.
Next, we fit proportional cause-specific hazards models examining the impact of symptoms and functioning on conversion and symptomatic remission. All models were fit using Cox regression, treating the competing event as a censoring point. The first set of models used symptoms and functioning as time-varying covariates so that a subject’s observed values at each of their visits contributed to the overall estimate of the impact of the predictor on the outcome of interest. The results are summarized in table 2. In these models, SIPS total negative symptom score is an extremely strong predictor of conversion (P = .002) when either symptomatic or functional recovery is the competing event. In a Cox regression model, the hazard ratio (HR) plays the role of relative risk. Here, the negative symptom score had a HR of approximately HR = 1.10 (95% CI 1.04–1.17) in the 2 models, implying a 10% increase in the hazard of conversion for every additional 1 point increase in negative symptoms. Social functioning was individually significant (P = .04, HR = 0.71, CI 0.52–0.99), with each additional 1-point increase on the social functioning scale corresponding to a 29% reduction in risk of conversion. Anxiety/mood symptoms and role functioning were not individually significant in this model. In a combined model, using all the predictors and adjusting for age, gender, and baseline medication use, negative symptoms remained highly significant (P = .01) and antipsychotic use was significantly associated with a higher risk of conversion (P = .02).
When symptomatic remission was treated as the outcome with conversion as the competing event, total negative symptoms (P = .001, HR = 0.89, 95% CI 0.83–0.95), dysphoric mood/anxiety symptoms (P = .0004, HR = 0.65, CI 0.51–0.83), social functioning (P = .01, HR = 1.47, CI 1.11–1.95), and role functioning (P = .01, HR = 1.32, CI 1.06–1.63) were all significant individual predictors. These results are in the expected direction, with more severe symptoms being associated with lower rates of remission, and better functioning associated with higher rates of remission. These variables all remain individually significant after adjusting for age, gender, and baseline medication status. However, in the model combining all predictors, only anxiety/mood symptoms remains significant, suggesting substantial overlap in the explanatory power of these variables. When functional recovery was the outcome, social, and role functioning were not included in the model to avoid circularity, but total negative symptoms (P < .0001, HR = 0.83, CI 0.77–0.89) and dysphoric mood/anxiety (P = .004, HR = 0.73, CI 0.60–0.90) were both significant individual predictors. Negative symptoms remained highly significant (P = .0002) in the combined model, which also adjusted for age, gender, and medication use but dysphoric mood/anxiety symptoms did not.
Next, we considered baseline values of the above covariates as time-fixed covariates to see whether they provided significant predictive value for eventual outcome. Consistent with the time-varying results, total negative symptom score was a significant predictor of conversion when symptomatic remission was the competing event (P = .02), although it did not remain significant when functional recovery was the competing event. Baseline social functioning was also a significant individual predictor of conversion when symptom remission was the competing event. Baseline negative symptom score was a significant predictor of functional recovery (P = .0004) and dysphoric mood/anxiety symptoms trended toward significance (P = .07), but these variables were not significant predictors of symptomatic remission.
We also examined whether baseline positive symptoms and medication status were associated with rates of conversion and remission. Higher baseline total positive symptom score was associated with a higher rate of conversion when functional recovery was the competing outcome (P = .03, HR = 1.13, 95% CI 1.01–1.26) and was trending (P = .08, HR = 1.10, 95% CI 0.99–1.22) when symptomatic remission was the competing outcome. Lower initial positive symptoms also showed a trend-level association with increased likelihood of symptomatic remission (P = .07, HR = 0.92, 95% CI 0.84–1.01). Baseline medication antipsychotic and antidepressant use were not individually significant predictors of any of the outcomes, although antipsychotic use was trending (P = .08) for predicting conversion and, as noted above, did add significant explanatory power in the multivariate model for this outcome.
To better understand the how CHR subjects’ symptoms and functioning evolve over time in comparison with those of healthy subjects, we plotted positive symptoms, negative symptoms, dysphoric mood/anxiety symptoms, social functioning, and role functioning scores over time for the 84 individuals included in our survival analyses plus the 58 healthy controls (figure 2). For all the measures, the trajectories cluster strongly by group, with the highest functioning of the remitters approaching the levels of the healthy controls, and the trajectories of those who neither convert nor remit (CHR-NRC) intermediate between those who converted and those who remitted. A similar pattern is observed when functional recovery is used as the competing event (figure not shown.). The trajectory plots indicate that, for most CHR subjects, psychosocial functioning appears more temporally stable than symptoms.
While the CHR syndrome13,14 represents one of the most reliable predictors of psychosis yet identified, predictors of symptomatic and functional recovery among nonconverting putatively prodromal individuals—who represent the majority of the CHR population—have not been systematically investigated. Do nonconverting prodromal patients represent false positives from the perspective of risk ascertainment? Our findings suggest that, at minimum, about one-third of individuals identified as CHR for psychosis are likely to represent false positives, to the extent that within 2 years of ascertainment, the severity of their positive symptoms has declined significantly from baseline and they no longer manifest the clinical features that define the CHR criteria. Better concurrent social and role functioning also were associated with an increased likelihood of prodromal symptom remission, with a 1 point higher score being associated with a 47% and 32% increase, respectively, in the relative likelihood of symptomatic remission over 2 years. Less severe negative and mood/anxiety symptoms at baseline were associated with functional recovery. Thirty percent of the CHR subjects in our study who did not convert experienced a full clinical recovery, as indicated by the achievement of both normative functioning and positive symptom remission. However, it is possible that the presence of functional impairment—despite remission of positive symptoms—may be at least partially attributable to higher rates of comorbid mood and anxiety symptoms in more functionally impaired cases, although we do not have sufficient data to test this hypothesis. Given that functional deficits are associated with a number of comorbid disorders present in CHR populations, such as mood and anxiety disorders,15 the observed pattern of functional impairment should not necessarily be equated with a continuation of psychosis risk, at least not without the concomitant profile of a recent onset or worsening of APSs.
CHR subjects who did not convert or remit over the course of the study are more difficult to characterize from the perspective of risk classification. Although these patients experience positive symptoms at a less severe level over the course of the study than patients who convert to psychosis, their functioning is remarkably similar to that of the converted patients. The stable trajectories of subthreshold positive symptoms and functional deficits in these individuals are consistent with the conceptualization of persistent functional disability and low-grade subpsychotic symptoms in the context of SPD. As such, it is possible that up to 50% of nonconverting CHR cases “progress” to SPD or a subthreshold variant of SPD. Although such individuals clearly share many genetic and neural risk factors in common with schizophrenia22,23 and youth diagnosed with SPD appear to be at risk of developing schizophrenia,24 the rate of conversion from a diagnosis of SPD to that of schizophrenia in the adult population appears to be modest.25 Further work is needed to determine a profile of risk indicators that could be used at the time of ascertainment to separate this subgroup from the CHR cases who are most likely to convert to psychosis, which would further increase the predictive power for psychosis onset by CHR criteria.
The conclusiveness of the findings of this study is, of course, constrained by the length of the follow-up interval employed. The results of the present study warrant a prospective replication, preferably over a longer follow-up period. Whether the lifetime risk for psychosis in the CHR-remitted group is comparable to the population base rate and how frequently patients experience a “relapse” of symptoms in the prodromal range of severity or an ultimate conversion after showing symptomatic remission are important remaining questions that can only be answered in longer-term follow-up studies. In the present study, a single item on the SIPS was used to determine the presence and severity of mood and anxiety symptoms. Despite the very good convergence of this measure with DSM-IV diagnoses of mood and anxiety disorders, this should nevertheless be considered as a preliminary finding needing further validation. Another limitation of the present study was the relatively young age of the participants and the possibility that some of the subjects had not gone through the full period of the “risk window.” Although medication status was not significantly related to clinical remission or functional recovery in this naturalistic study, it is possible that some of the nonconverting patients who were on antipsychotics at study entry may have converted to psychosis had they not been on those medications.
Our study, as with several other recently published studies investigating CHR individuals, found that the majority of the sample did not develop psychosis over the study period,5–9 suggesting that some refinement of the current “CHR” criteria is warranted. Given that nonconverted subjects who remitted symptomatically also presented initially with less severe prodromal symptomatology and showed a distinct, more normative (ie, more similar to typically developing adolescents), trajectory of both symptoms and psychosocial functioning over time, it may be possible to refine the psychosis risk syndrome criteria to reduce the number of false positive cases by eliminating those who present with less severe APSs or show early improvements in terms of symptoms or functioning. Also consistent with previous studies,5,8,26,27 here we found that psychosocial functioning was a robust predictor of clinical outcome. Further, rapid initial improvement in positive, negative, and mood/anxiety symptoms are indicators for better outcomes such as remission. Thus, eliminating subjects with low symptom levels at the time of ascertainment or who show improvements over short-term follow-up could greatly reduce the number of subjects falsely identified as at high risk for psychosis. Whether the criteria could be modified without a loss of sensitivity remains an important question. Although we were not able to quantitatively evaluate sensitivity in the current study, inspection of the data ranges for converting and recovering subjects suggests that any loss in sensitivity posed by the above modifications may be relatively minor. Improved risk classification is critical for targeting interventions to those only at highest risk of developing psychosis and minimizing exposure of false positive cases to potential adverse events.
National Institute of Mental Health (MH65079 to T.D.C.; P50 MH066286 to T.D.C., C.E.B., C.S.; T32 MH082719 to D.S., S.J.). National Alliance for Research on Schizophrenia and Depression Young Investigator Award (Maxine and Jack Zarrow Investigator Award to C.E.B.); donations from the Rutherford Charitable Foundation and Staglin Music Festival for Mental Health to the University of California–Los Angeles Foundation.
The authors have declared that there are no conflicts of interest in relation to the subject of this study.