Search tips
Search criteria 


Logo of schbulschizophrenia bulletinsubscriptionscontact uscurrent issuemy basketarchivemy accountsearchcontact this journaloxford journalsabout this journal
Schizophr Bull. 2010 January; 36(1): 126–135.
Published online 2009 October 30. doi:  10.1093/schbul/sbp117
PMCID: PMC2800145

CHoice of Outcome In Cbt for psychosEs (CHOICE): The Development of a New Service User–Led Outcome Measure of CBT for Psychosis


Outcome measures for cognitive behavior therapy for psychosis (CBTp) have been derived from pharmacological studies, focusing on symptom change rather than outcomes such as distress or fulfilment. This study presents the development and psychometric properties of a new outcome measure (CHoice of Outcome In Cbt for psychosEs [CHOICE]), which reflects more strongly the aims of CBTp and the priorities of service users. Service users who had received CBTp participated in focus groups to discuss their outcome priorities, using a topic guide generated by a panel of experts in CBTp. A qualitative thematic analysis was undertaken to reach consensus on themes and generate items. Response scales were constructed for 3 dimensions: severity, satisfaction, and importance. The resulting questionnaire was piloted with service users who had not received CBTp, stratified by service type, ethnicity, and first language to ensure that it was user friendly and applicable prior to CBTp. The psychometric properties of the measure were then examined in a sample of 152 service users. Twenty-four items, and 2 of the dimensions (severity and satisfaction), were retained in the final measure. A factor analysis revealed a single psychological recovery factor interspersed throughout with both CBTp and recovery items. Test-retest reliability, construct validity, and sensitivity to change following CBTp were confirmed. The CHOICE measure is unique in being the first psychometrically adequate service user–led outcome measure of CBTp. It provides the opportunity to examine the evidence base for CBTp with an assessment approach that prioritizes service user definitions of recovery and CBT aims.

Keywords: subjective, measurement, cognitive behavior therapy, trials, schizophrenia


Measurement of outcome in cognitive behavior therapy for psychosis (CBTp) has so far focused on psychotic symptom reduction and improved function as primary outcomes. Indeed, all the randomized controlled trials (RCTs) of CBTp selected in recent meta-analyses reported symptom reduction, and most reported global function as primary outcomes.16 This strategy was partly chosen to be compatible with RCTs in medication outcome studies and has proved worthwhile in demonstrating the efficacy of CBTp in informing recent national and international guidelines for the treatment of psychosis.79

The influential cognitive models of psychosis of Garety and colleagues,1013 Chadwick and Birchwood,1417 and Morrison,18 however, identify a variety of cognitive and emotional processes associated with the development and maintenance of psychotic symptoms: distressing emotions and, specifically, anxiety11,19; beliefs about voices20; distress and emotional and behavioral dysfunction that arise directly from beliefs; and the content of distressing appraisals and beliefs themselves. Additionally, focused CBT approaches have addressed relapse prevention, negative beliefs about psychosis, self-esteem, and stigma.2124 The resulting manuals emphasize a broad range of CBTp targets, which in turn influence the selection of outcome measures.

The suggestion here is that CBTp focuses on something other than, or in addition to, symptom reduction, and that the psychological model used informs both CBT target and outcome measurement. RCTs have tended to accommodate the potential broad focus of CBTp by adding a number of secondary outcomes. Evaluation of secondary outcomes has thus involved dimensional measures of symptoms, such as the Psychotic Symptoms Ratings Scales (PSYRATS)25 to identify conviction, preoccupation, distress, and disability, in addition to an array of specific measures, such as the Beck Depression Inventory (BDI),26 Beck Anxiety Inventory (BAI),27 and Beliefs About Voices Questionnaire.28,29 Voices Compliance Scale,30 Voices Power Differential,20 Omniscience Scale,20 Personal Beliefs about Illness Questionnaire,31 and Rosenberg Self-esteem Scale.32 However, this use of multiple specific measures presents both statistical and methodological problems for RCTs. The advancement of the evidence base for CBTp requires outcome measures that are both specific to CBTp but sufficiently generic to be of relevance to therapy derived from all psychological models.

In addition, CBTp is a collaborative process, and the targets of CBTp on an individual basis are informed by the needs and wishes of the service user. Service user views and the concept of recovery are becoming increasingly central to the organization of mental health service provision.7,8,33 Yet, consideration of service users’ views suggests that both their targets for intervention and their concepts of recovery differ from those commonly held by clinicians. Service users report that symptoms can be less debilitating than, for example, social exclusion and stigma, emotional problems, and difficulties with relationships.34 Symptom reduction is not necessarily sufficient to improve quality of life, with issues such as empowerment, choice, control, and personal fulfilment being of central concern.35 Therefore, service users’ views of recovery can diverge from the clinician model of symptom reduction and return of function,36,37 in promoting a different course where the return to a previous functional level may be seen as a negative outcome, and the development of a new direction in life is paramount.33 There are few measures within psychosis that are designed to capture these needs, although some do address service user priorities such as quality of life and empowerment38,39 and define outcomes such as quality of life from a service user perspective.40

From the foregoing, we conclude that a measure of CBTp suitable for assessing outcome in RCTs should incorporate both CBTp priorities and those of service users. It should be reliable, valid, and suitable for administration before and after therapy; specific enough to capture change; but sufficiently generic to apply across different CBTp approaches and models. Brevity and ease of administration are also important.

The goal of this study was to develop a new outcome measure of CBTp suitable for use in RCTs as well as in individual practice, in collaboration with service users. A self-report format was chosen to assess the service user perspective. This format affords greater flexibility for self or researcher administration, without the need for lengthy training. In keeping with the notion that there is more to outcome than problem reduction, a dimensional structure was incorporated with separate dimensions for severity, satisfaction, and importance for each concern. While severity and satisfaction are commonly assessed, a dimension of importance to the service user was included after consideration of the literature on “utility values” or relative values to an individual of different health states.4143 This dimension aimed to capture the current relative health value (importance) of problem areas to the individual. See online supplementary material for a copy of the measure.



The study was designed to identify outcomes of CBT for psychosis (CBTp) considered important by both service users and cognitive behavioral therapists and to develop a reliable and valid questionnaire to assess these outcomes. Initially, a consultation among experts in CBTp generated broad probe areas for discussion of CBTp outcomes. The measure development was then conducted with service user researcher involvement (A.S.) through 7 stages of design, implementation, and analysis: (1) qualitative analysis of service user focus group discourses, (2) Delphi analysis and measure adaptation after 2 follow-up interviews with each focus group member, (3) initial piloting and adaptation of the complete measure, (4) factor analysis in a large sample, (5) test-retest reliability, (6) construct validity against traditional outcome measures, and (7) sensitivity to change.

Initial Consultation to Form Focus Group Topic Guide

A consultation among experts in the field of CBTp (n = 5: P.G., E.K., E.P., J.S., K.E.G.) yielded a list of broad probe areas for discussion in the focus groups, which were finalized following further written feedback from 3 of the original experts (P.G., E.K., J.S.) and 8 CBTp therapists working in a specialist CBTp service, the Psychological Interventions Clinic for oUtpatients with Psychosis (PICuP) Clinic (South London and Maudsley NHS Foundation Trust). These probes were developed to facilitate initial generation of a broad range of items for the questionnaire. The 6 probes were (1) mood/emotions, (2) empowerment/self-confidence/self-esteem, (3) coping with/understanding of/approach to problems, (4) day-to-day functioning/quality of life, (5) psychotic experiences/symptoms, and (6) relationships with people.

Development of the Measure

Participant characteristics for each stage are summarized in table 1.

Table 1.
Participant Characteristics for Stages 1–7

Stage 1—Focus Groups

Potential focus group participants were all service users who received therapy with the PICuP Clinic between January 2003 and December 2004 (n = 76). People were excluded if they were under 18 years, withdrew from therapy, received therapy from the group facilitator, or were too unwell to take part. Fifty-one participants were invited to take part, of whom 12 agreed. All interested participants (24% of all eligible users) took part in either the pilot (n = 3 participants) to inform the final format and topic guide or the focus group (n = 9 participants) to discuss possible and desired outcomes. They were audiotaped for transcribing and were facilitated by a service user researcher with first-hand CBT experience (A.S.) and a CBTp research clinician (K.G.). A research assistant (S.W.) made additional notes.

Focus groups followed a preprepared semistructured interview format, with the relative importance and nature of change being explored for all emerging outcomes. The 6 probe areas identified by CBTp experts were offered to facilitate item generation only when these were not covered spontaneously. Written feedback was obtained from each group member in the closing stages of the group on the outcomes considered most important.

A combined thematic and content analysis was applied to the transcribed discourses and written feedback using a multiple coding approach: Three researchers (research clinician, service user researcher, research assistant) all analyzed the full transcript of each group and produced independent theme structures. Potential items were either mapped on to the broad predetermined outcome areas (content analysis) identified by CBTp therapists or were new and emerging from the discourses (thematic analysis). The validity of themes was ascertained through a process of triangulation whereby independent analyses were subjected to a consensus between the 3 researchers to identify common themes and through respondent validation of themes during the follow-up interviews as described below.

Stage 2—Follow-up Interviews and Delphi Process

To capture outcomes that were held consistently over time and to reduce the impact of group pressure, the preliminary outcomes were fed back in 2 additional semistructured interviews with each focus group member. Focus groups and first and second interviews were held at approximately 1-month intervals to allow for interim analyses and preparation of the next interview document.

Participants were asked in follow-up interviews whether the derived outcomes made sense given the discussions in the focus groups and whether they were important and achievable through CBT. They were also asked to comment on the language and the format of the measure, preferences over the measurement scale, and inclusion of outcome descriptors. The pooling of all participant responses to all items revealed that 97% of outcomes made sense, 85% were deemed achievable through CBT, and the mean importance rating was 6.0 (range of 5.4–6.6) on a 7-point scale.

Following Delphi procedures, and in order to move toward a consensus, participants rated the importance of items on a 1- to 7- point scale for each outcome. At the second interview, participants were presented for each item with their own and the mean importance rating for the group from the first interview. They were then asked to rerate the importance of each item. The Delphi analysis compared the mean discrepancies between the individual and group means at time 1 compared with those at time 2. A paired samples t test demonstrated progression toward a consensus with significantly smaller discrepancies between individual and group ratings in the second, compared with the first, interviews (t = 3.34, df = 27, P = .002, 95% confidence interval [CI] = 0.06–0.23).

Stage 3—Pilot Interviews

The pilot measure, entitled CHOICE (CHoice of Outcome In Cbt for psychosEs) in recognition of the tailoring to service user priorities and CBT outcome principles, comprised 26 items, one of which, “coping,” was divided into 6 subsections to provide 31 items in total. As a result of service users’ emphasis on the individual nature of the psychosis experience and of goals, an additional 2 items were left blank for service users to complete.

A new group of people with psychosis (n = 15), who had not previously received CBT, completed the pilot measure. This group was selected to encompass service users in a range of psychiatric settings, ethnicity, and first language to ensure representativeness across the diverse population of people with psychosis. The psychiatric settings consisted of an inpatient ward, a local outpatient community team, and the PICuP clinic (n = 5 per setting). Within each setting, there were 2 individuals from black and minority ethnic populations and 1 person with English as their second language.

Participants also provided feedback on the language and clarity by underlining and commenting on problem areas and on the user friendliness by completing multiple choice questions relating to ease of completion, length, clarity, and emotions elicited. Sixty-seven percent of service users found the questionnaire intermediate or easy to complete, 80% found it intermediate or enjoyable, and 93% found it acceptable in length. The majority of service users (67%) thought that the outcomes were clearer without a description. Descriptors were therefore delegated to a supplementary booklet to be used by the administrator of the measure in the event that further explanation was required.

Factor Analysis

A factor analysis was undertaken to reduce item redundancy and to elucidate the factor structure. A 2-factor solution was predicted based on the dual goals of capturing both the aims of CBTp and the recovery priorities of service users. The analysis was based on data from 152 participants who completed the outcome measure as part of consecutive standard assessments for the PICuP clinic (baseline, pre-, mid-, or posttherapy assessments) and who gave informed consent for their data to be used for research. A principal axis factoring model was employed and compared using direct oblique and varimax rotations. Of the initial 31 outcomes, items entered into the final model were determined following an iterative process involving investigation of Pearson interitem correlations, corrected item to total correlations, and stepwise omission of highly correlated redundant items, items with low intercorrelations (<0.3), and those with low item-total correlations. Personal goal items were excluded from the factor analysis due to the smaller number of participants (n = 91) who identified individual goals. The factor solution was decided through consideration of the scree plot, eigenvalues, and predicted eigenvalues derived from a Monte Carlo parallel analysis of 100 randomly generated solutions with the same number of participants and items. Of the 3 dimensions (severity, satisfaction, and importance), those entered into the final model were decided after consideration of sensitivity to change and intercorrelations.

Test-Retest Reliability

Test-retest reliability was undertaken for each dimension, using the intraclass correlation (ICC) coefficient, with subject as a 1-way random effect.44 The analysis incorporated the first 30 participants who either completed the measure as part of a standard clinic assessment and subsequently returned a postal retest copy or who completed the questionnaire at baseline and again at the second pretherapy assessment. The postal return rate was 30% (n = 19), and 11 people completed the questionnaire at both their initial (baseline) and second (pretherapy) assessment.

Construct Validity

The cross-sectional validation study comprised the first 120 people, involved also in the factor analysis, who completed the measure alongside their standard baseline or posttherapy assessments for the PICuP clinic. Construct validity was investigated for each dimension against 5 standard self-report outcome measures of psychotic symptoms, emotional problems, quality of life, and “cognitive insight,” described below.

Psychosis Symptoms

The PSYRATS25—the PSYRATS comprises 2 multidimensional assessment scales: one each for delusions and hallucinations. The delusion scale comprises 6 items that form a 2-factor structure of distress/disruption and preoccupation/conviction.24,44 The Hallucinations Scale has 11 items but a less clear structure with both 3- and 4-factor solutions reported, both of which include a distress/disruption/negative content factor.25,45,46 For the purposes of the current study, validity was investigated against the factors reported by Steel et al.45

Emotional Problems

  1. BDI-II26—a 21-item measure of depressive symptoms and cognitions
  2. BAI27—a 21-item measure of physiological anxiety symptoms

Quality of Life

The Manchester Short Assessment of Quality of Life Scale (MANSA)47—a 16-item measure that assesses satisfaction with life across 12 subjective measures and 4 objective measures including areas such as employment, finances, leisure, friendships, relationships, personal safety, accommodation, and physical and mental health. Each item is rated on a 7-point scale with extremes labeled “couldn't be worse” and “couldn't be better.”

Cognitive Insight

The Beck Cognitive Insight Scale (BCIS)48 is a 15-item scale that evaluates patients’ self-reflection and overconfidence in their interpretations of their experiences. The scale contains 2 subscales of self-certainty and self-reflexivity.

Sensitivity to Change

Finally, the sensitivity of the measure to change over a course of therapy was investigated for each dimension for the first 33 participants who completed therapy with the PICuP clinic (average 19 sessions [range = 5–47]) and who completed the measure before and after CBTp with a psychology assistant, independent of therapy delivery.


Factor Analysis

Preliminary analysis revealed high intercorrelations between the dimensions of severity and satisfaction (r = 0.82 for the final scale) and a similar factor structure for each of these dimensions when conducted separately. Ratings of item importance were high (mean = 8.51 [range = 5.06–10] on a 10-point scale), but this dimension did not correlate significantly with either severity or satisfaction and did not change significantly with CBT, as discussed below. In the interests of brevity and simplicity, it was therefore decided to delete the importance dimension from the measure. The ratings of severity and satisfaction were then combined into mean scores across dimensions for the factor analysis. Following stepwise elimination of redundant, highly correlated items and those with low interitem and item-total correlations, a total of 24 items, 5 of which were subcategories of coping, were retained (see table 2). The majority of correlation coefficients between these items were 0.3 and above, the Kaiser-Meyer-Olkin value was 0.92, and the Bartlett's Test of Sphericity reached significance, thus supporting the suitability of the data for factor analysis.

Table 2.
Unrotated Factor Structure Item Loadings for 1-Factor Solution

A principal axis factoring analysis of the 24 items, using SPSS Version 12, revealed the presence of 4 factors with eigenvalues exceeding 1, which explained 47.0%, 5.4%, 5.1%, and 4.6% of the variance, respectively. However, the scree plot revealed a clear break after the first factor. A single-factor structure was supported by the Monte Carlo parallel analysis in which only one factor exceeded the predicted criterion eigenvalue (actual eigenvalue 11.28 compared with a criterion eigenvalue of 1.812), compared with 100 randomly generated datasets with the same sample size (24 variables × 152 participants). The single factor, several high loading items, and communalities in the 0.5 range indicated that the 152 participants and 24 items provide sufficient power and a robust factor structure.49,50 Hence, the final single-factor solution explained 47% of the variance. Inspection of the histogram of mean factor scores revealed that these were normally distributed across the full range of possible scores, with a mean of 4.52 and an SD of 1.93 (See figure 1). The unrotated factor loadings are presented in table 2.

Fig. 1.
Distribution of mean CHoice of Outcome In Cbt for psychosEs factor scores

Test-Retest Reliability

The dimensions had high internal consistency (Cronbach α = .83 for severity and .88 for satisfaction) and were reliable over time (test-retest ICC = 0.73, 95% CI = 0.51–0.86, P < .001 for severity, and ICC = 0.79, 95% CI = 0.61–0.90, P < .001 for satisfaction). Preliminary analysis also suggested good internal consistency and reliability for mean severity and satisfaction with individual goals (n = 14 participants).

Construct Validity

Construct validity was investigated using Pearson correlation analyses between the severity and satisfaction symptom dimensions and the appropriate total and factor scores derived from the measures of psychotic symptoms, emotion problems, quality of life, and cognitive insight. The PSYRATS delusions scale was completed by 83 of the 120 participants, and the PSYRATS voices scale by 69 participants. In all correlation analyses, α was set at the 1% level to control for multiple comparisons. The results are summarized in table 3. Of note, both the severity and satisfaction dimensions correlated positively with Quality of Life (MANSA) and negatively with emotional measures (BDI/BAI), with trends for negative correlations with PSYRATS delusions distress, voices distress (CHOICE severity), and disruption/control (CHOICE severity). Construct validity analysis for mean severity and mean satisfaction with personal goals revealed a similar pattern of correlations for the 91 participants who identified one or more individual goals.

Table 3.
Pearson Correlation Coefficients Between CHOICE Dimensions and Measures of Psychotic Symptoms, Emotional Problems, Quality of Life and Cognitive Insight

Sensitivity to Change Following CBT

Thirty-three people completed the CHOICE measure at baseline and immediately following CBT (see figure). Paired samples t tests revealed a significant improvement on the CHOICE severity and satisfaction dimension scores from before to after therapy (t = −4.1, df = 32, 95% CI = −2.57 to −0.87, P < .001, for severity and t = −4.3, df = 32, 95% CI = −2.97 to −1.06, P < .001, for satisfaction).

Fig. 2.
Improvement in Mean Severity, Satisfaction, and Personal Goal Score Following Cognitive Behavior Therapy for Psychosis.

Sixteen people additionally recorded either 1 or 2 personal goals. These included, eg, “learning to deal with panic attacks,” “commencing guitar playing, artwork and tai chi,” “understanding how my past relates to the present,” “thinking with optimism and hope,” and “finding peace and happiness within myself.” Where an individual recorded 2 personal goals, for the purposes of investigating change following CBT, the mean score was taken across the 2 goals at baseline and again at follow-up for that individual. The significant improvement following CBT was maintained for personal goals (t = −3.12, df = 15, 95% CI = −5.13 to −0.96, P = .007), where the magnitude of change was even greater (mean improvement = 3.05 points).


This study sets out to design, in collaboration with service users, a new measure of outcome in CBTp that reflects more accurately than existing assessments both the targets of CBTp and the priorities of service users.

The Psychometric Properties of the Measure

The final measure was a 2-dimensional 24-item self-report questionnaire, which provides mean scores for (1) severity and (2) satisfaction with a range of difficulties. An additional item, left blank, was included to record and evaluate individual intervention targets that were not included elsewhere.

The measure had good test-retest reliability, face and construct validity, and sensitivity to change. It was designed in full consultation with service users to capture the outcomes that they viewed as the most important and realistic to obtain through CBTp. The outcomes were presented and phrased in language deemed appropriate by service users and were acceptable and understandable prior to the start of therapy.

The Dimensions of the Measure

Severity and satisfaction were agreed, a priori, to be distinct dimensions as they have been demonstrated empirically in previous studies to relate to different factors. Subjective ratings of problem severity and problem change are influenced by the severity of delusions and hallucinations51 and the ability to subjectively appraise experiences.52 Subjective quality of life, which is comparable to satisfaction, is associated with psychosocial factors and depression,53,54 while satisfaction specifically is influenced by age, service user expectations, and gratitude.55 Although these 2 dimensions were closely related in the current study, they were maintained as separate dimensions for the reasons described. In light of the relationship between psychotic symptom severity and subjective rating of problems, it is of relevance that test-retest reliability was good for both dimensions, thus validating the reliability of service user responding.

Importance ratings in the current study were high but were not correlated with either severity or satisfaction and were not sensitive to change with CBT. Furthermore, the large number of items, each scored with separate scales and without explicit instructions to compare across items, suggests that this dimension captured absolute and not relative ratings of importance. The high and stable importance ratings added further evidence that the measure captured the important health goals of service users, but the insensitivity of this dimension to change precluded its inclusion within the outcome measure.

The Content of the Measure

The measure is interspersed throughout with both generic recovery–related items and CBT-specific items (see Supplementary Material). Recent service user–led research56 has identified a number of important factors in recovery from psychosis, which included “rebuilding the self,” “rebuilding life,” and “hope for a better future.” Subthemes of these factors overlap with CHOICE items such as “understanding myself and my past,” “self-esteem,” “self-confidence,” “positive ways of relating to people,” “a positive purpose and direction in life,” “a sense of being in control of life,” and “hope and optimism.” Similar themes have been echoed in other studies that emphasize internal aspects of recovery such as self-direction, hope, empowerment, coping, and establishing the self as separate from symptoms.57 Bellack33(p437) recently stated that recovery should not be “defined so broadly as to make its achievement unimportant.” At every stage in the development of the measure, we aimed to reconcile the capture of important broad recovery themes with the need for items that were specific and measurable.

In addition to capturing outcomes related to recovery from psychosis, the CHOICE measure includes outcomes that seem to be more specifically related to targets of CBTp. Relevant outcomes include “The ability to approach problems in a variety of ways,” “The ability to step back from overwhelming experiences (e.g. thoughts or voices),” “The ability to question the way I look at things,” and “Knowing I am not the only person who has unusual experiences,” the latter reflecting both the CBT focus on normalizing experiences and the recovery process of moving from social exclusion to inclusion.

The construct validity findings emphasize the close relationship between the CHOICE measure scores and both mood (depression and anxiety) and quality of life. Near significant trends were also found between both CHOICE dimensions and the delusions distress factor, as well as between the CHOICE severity dimension and the two voices factors of distress and disruption/volume/control. Hence, the CHOICE measure appears to capture general distress, psychosis-specific distress and quality of life outcomes. The closer relationship to general anxiety, distress and quality of life than to psychosis specific distress emphasises the breadth of treatment priorities for CBTp service users. This is consistent with the nature of CBTp as more than a quasi-neuroleptic14 aimed at symptom change. CHOICE therefore seems to capture psychological recovery in terms of therapy success or failure from the subjective perspective of the service user.

The Structure of the Measure

Despite the fact that the items were predicted to belong to 2 broad general themes, (ie, recovery and CBTp items), only one main factor was found. This finding may reflect the fact that CBT and recovery outcomes are closely enmeshed with each other. An exploration of the factor structure revealed that the items with the highest loadings could be conceptualized as those likely to yield more immediate changes and the goals of a more recovery and behavioral (coping) nature (eg, A sense of being in control of my life, ways of dealing with unpleasant feelings and emotions, a positive purpose and direction in life), whereas those with the lower loadings (with the exception of feeling safe and secure) were those more reminiscent of cognitive outcomes and processes of change in CBTp58 (eg, the ability to see things from another point of view, the ability to approach problems in a variety of ways, understanding my experiences, understanding myself and my past, and the ability to question the way I look at things.) This factor structure may have been affected by the timing of the sampling, which mainly included people prior to the start of therapy (74%) for whom recovery and coping may have been a more relevant and cohesive set of outcomes than cognitive and process goals.


The focus group members were self-selecting volunteer service users from a specialist CBTp clinic. In addition, the factor analysis was based on data provided by a single client group, each recruited from a specialist CBTp clinic. This sample may not be generalizable to other individuals with psychosis, and the factor analysis may require replication with a new sample. Furthermore, the service users involved in the measure development were all longer term service users, and the outcomes generated may not necessarily capture the phase-specific needs and priorities from people in their first episode. It is possible that the test-retest reliability may have been lower had the 70% who did not return their postal retest data been included, although test-retest data on approximately 20% of these responders were subsequently collected at their next follow-up interview. Reliability remained good despite the longer time to retest in this latter group.

The validity of the distinction between severity and satisfaction was not confirmed in the current study, despite a clear a priori distinction on theoretical grounds. This may have reflected some conceptual overlap between severity and satisfaction ratings for items tapping psychological recovery that are subjectively rated and that are compared with similarly subjective ratings of mood and quality of life. This distinction warrants further investigation.

The CHOICE questionnaire is clearly sensitive to change over time but requires further investigation using a RCT design to demonstrate sensitivity to change specifically with therapy.

On a related point regarding outcome measurement in RCTs, the limited correlation with psychosis-specific distress and lack of correlation with self-reflection and self-certainty on the BCIS demonstrate that the CHOICE measure complements, rather than replaces, the need for other psychosis symptom measures. In addition, validation against objective measures of function such as employment, education, and independent living should be investigated further in future research.

The Utility of CHOICE in Research and Therapy

The CHOICE measure is unique in being the first psychometrically adequate service user–led outcome measure of CBTp. The psychological recovery focused outcomes are broad enough in scope to capture the essence of CBTp across models but specific enough to be sensitive to change. The opportunity is thus provided to advance the evidence base for CBTp across models with an assessment approach that places service user definitions of recovery at the fore. In light of the structure of the measure, CHOICE may also provide useful information on the process of change in CBTp. Therapeutically, the CHOICE measure has value as a clinical tool for facilitating the development of shared goals and for evaluating progress and outcome. The recovery focus may place the measure in an ideal position to fill the need identified for additional instruments to facilitate and assess service users’ experiences of change and recovery during CBTp.51


South London and Maudsley NHS Foundation Trust (Research and Development grant).

Supplementary Material

[Supplementary Data]


We would like to thank all the service user focus group members, service users, and PICuP CBTp therapists who contributed to the development of this measure.


1. Wykes T, Steel C, Everitt B, Tarrier N. Cognitive behavior therapy for schizophrenia: effect sizes, clinical models, and methodological rigor. Schizophr Bull. 2008;34:523–537. [PMC free article] [PubMed]
2. Pfammatter M, Junghan UM, Brenner HD. Efficacy of psychological therapy in schizophrenia: conclusions from meta-analyses. Schizophr Bull. 2006;32(suppl 1):S64–S80. [PMC free article] [PubMed]
3. Zimmermann G, Favrod J, Trieu VH, Pomini V. The effect of cognitive behavioral treatment on the positive symptoms of schizophrenia spectrum disorders: a meta-analysis. Schizophr Res. 2005;77:1–9. [PubMed]
4. Jones C, Cormac I, Silveira da Mota Neto JI, Campbell C. Cognitive behaviour therapy for schizophrenia. Cochrane Database Syst Rev. 2004;(4):CD000524. [PubMed]
5. Tarrier N, Wykes T. Is there evidence that cognitive behaviour therapy is an effective treatment for schizophrenia? A cautious or cautionary tale? Behav Res Ther. 2004;42:1377–1401. [PubMed]
6. Pilling S, Bebbington P, Kuipers E, et al. Psychological treatments in schizophrenia: II. Meta-analyses of randomized controlled trials of social skills training and cognitive remediation. Psychol Med. 2002;32:783–791. [PubMed]
7. National Institute for Health and Clinical Excellence. Schizophrenia: Core interventions in the treatment and management of schizophrenia in primary and secondary care. NICE Clinical Guideline 82. London, UK: NICE; 2009.
8. National Institute for Health and Clincial Excellence. Core Interventions in the Management of Schizophrenia in Primary and Secondary Care. London, UK: NICE; 2002.
9. Lehman AF, Kreyenbuhl J, Buchanan RW, et al. The Schizophrenia Patient Outcomes Research Team (PORT): updated treatment recommendations. Schizophr Bull. 2004;30:193–217. [PubMed]
10. Garety PA, Bebbington P, Fowler D, Freeman D, Kuipers E. Implications for neurobiological research of cognitive models of psychosis: a theoretical paper. Psychol Med. 2007;37:1377–1391. [PubMed]
11. Garety PA, Kuipers E, Fowler D, Freeman D, Bebbington PE. A cognitive model of the positive symptoms of psychosis. Psychol Med. 2001;31:189–195. [PubMed]
12. Freeman D, Garety PA. Connecting neurosis and psychosis: the direct influence of emotion on delusions and hallucinations. Behav Res Ther. 2003;41:923–947. [PubMed]
13. Freeman D, Garety PA, Kuipers E, Fowler D, Bebbington PE. A cognitive model of persecutory delusions. Br J Clin Psychol. 2002;41:331–349. [PubMed]
14. Birchwood M, Trower P. The future of cognitive-behavioural therapy for psychosis: not a quasi-neuroleptic. Br J Psychiatry. 2006;188:107–108. [PubMed]
15. Birchwood M. Pathways to emotional dysfunction in first-episode psychosis. Br J Psychiatry. 2003;182:373–375. [PubMed]
16. Birchwood M, Chadwick P. The omnipotence of voices: testing the validity of a cognitive model. Psychol Med. 1997;27:1345–1353. [PubMed]
17. Chadwick P, Birchwood M. The omnipotence of voices. A cognitive approach to auditory hallucinations. Br J Psychiatry. 1994;164:190–201. [PubMed]
18. Morrison AP. The interpretation of intrusions in psychosis: an integrative cognitive approach to hallucinations and delusions. Behav Cogn Psychother. 2001;29:257–276.
19. Freeman D, Garety PA, Kuipers E. Persecutory delusions: developing the understanding of belief maintenance and emotional distress. Psychol Med. 2001;31:1293–1306. [PubMed]
20. Birchwood M, Meaden A, Trower P, Gilbert P, Plaistow J. The power and omnipotence of voices: subordination and entrapment by voices and significant others. Psychol Med. 2000;30:337–344. [PubMed]
21. Gumley A, Schwannauer M. Staying Well after Psychosis: A Cognitive Interpersonal Approach to Recovery and Relapse Prevention. New York, NY: John Wiley & Sons Ltd; 2006.
22. Gumley A, O'Grady M, McNay L, Reilly J, Power K, Norrie J. Early intervention for relapse in schizophrenia: results of a 12-month randomized controlled trial of cognitive behavioural therapy. Psychol Med. 2003;33:419–431. [PubMed]
23. Hall PL, Tarrier N. The cognitive-behavioural treatment of low self-esteem in psychotic patients: a pilot study. Behav Res Ther. 2003;41:317–332. [PubMed]
24. Iqbal Z, Birchwood M, Chadwick P, Trower P. Cognitive approach to depression and suicidal thinking in psychosis. 2. Testing the validity of a social ranking model. Br J Psychiatry. 2000;177:522–528. [PubMed]
25. Haddock G, McCarron J, Tarrier N, Faragher EB. Scales to measure dimensions of hallucinations and delusions: the psychotic symptom rating scales (PSYRATS) Psychol Med. 1999;29:879–889. [PubMed]
26. Beck AT, Steer RA, Brown GK. The BDI-II Manual. San Antonia, TX: The Psychological Corporation; 1996.
27. Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56:893–897. [PubMed]
28. Chadwick P, Lees S, Birchwood M. The revised Beliefs About Voices Questionnaire (BAVQ-R) Br J Psychiatry. 2000;177:229–232. [PubMed]
29. Chadwick P, Birchwood M. The omnipotence of voices. II: The Beliefs About Voices Questionnaire (BAVQ) Br J Psychiatry. 1995;166:773–776. [PubMed]
30. Beck-Sander A, Birchwood M, Chadwick P. Acting on command hallucinations: a cognitive approach. Br J Clin Psychol. 1997;36(pt 1):139–148. [PubMed]
31. Birchwood M, Mason R, MacMillan F, Healy J. Depression, demoralization and control over psychotic illness: a comparison of depressed and non-depressed patients with a chronic psychosis. Psychol Med. 1993;23:387–395. [PubMed]
32. Rosenberg M. Society and the Adolescent Self-image. Princeton, NJ: Princeton University Press; 1965.
33. Bellack AS. Scientific and consumer models of recovery in schizophrenia: concordance, contrasts, and implications. Schizophr Bull. 2006;32:432–442. [PMC free article] [PubMed]
34. May R. Routes to recovery from psychosis: the roots of a clinical psychologist. Clin Psychol Forum. 2000;146:6–10.
35. Perkins R. What constitutes success? The relative priority of service users’ and clinicians’ views of mental health services. Br J Psychiatry. 2001;179:9–10. [PubMed]
36. Liberman RP, Kopelowicz A. Recovery from schizophrenia: a challenge for the 21st century. Int Rev Psychiatry. 2002;14:245–255.
37. Harrow M, Grossman LS, Jobe TH, Herbener ES. Do patients with schizophrenia ever show periods of recovery? A 15-year multi-follow-up study. Schizophr Bull. 2005;31:723–734. [PubMed]
38. Lecomte T, Wallace C, Caron J, Perreault M, Lecomte J. Further validation of the Client Assessment of Strengths Interests and Goals. Schizophr Res. 2004;66:59–70. [PubMed]
39. Rogers ES, Chamberlin J, Ellison ML, Crean T. A consumer-constructed scale to measure empowerment among users of mental health services. Psychiatr Serv. 1997;48:1042–1047. [PubMed]
40. Auquier P, Simeoni MC, Sapin C, et al. Development and validation of a patient-based health-related quality of life questionnaire in schizophrenia: the S-QoL. Schizophr Res. 2003;63:137–149. [PubMed]
41. EuroQol Group. EuroQol–a new facility for the measurement of health-related quality of life. Health Policy (New York) 1990;16:199–208. [PubMed]
42. Brazier J, Roberts J, Deverill M. The estimation of a preference-based measure of health from the SF-36. J Health Econ. 2002;21:271–292. [PubMed]
43. Brazier J. The SF-36 health survey questionnaire–a tool for economists. Health Econ. 1993;2:213–215. [PubMed]
44. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1:30–46. correction, 1:390.
45. Steel C, Garety PA, Freeman D, et al. The multidimensional measurement of the positive symptoms of psychosis. Int J Methods Psychiatr Res. 2007;16:88–96. [PubMed]
46. Drake R, Haddock G, Tarrier N, Bentall R, Lewis S. The Psychotic Symptom Rating Scales (PSYRATS): their usefulness and properties in first episode psychosis. Schizophr Res. 2007;89:119–122. [PubMed]
47. Priebe S, Huxley P, Knight S, Evans S. Application and results of the Manchester Short Assessment of Quality of Life (MANSA) Int J Soc Psychiatry. 1999;45:7–12. [PubMed]
48. Beck AT, Barch E, Balter JM, Steer RA, Warman DM. A new instrument for measuring insight: the Beck Cognitive Insight Scale. Schizophr Res. 2004;68:319–329. [PubMed]
49. MacCallum RC, Widaman KF, Zhang S, Hong S. Sample size in factor analysis. Psychol Methods. 1999;4:84–99.
50. Tabachnick BG, Fidell LS. Using Multivariate statistics. Chapter 13. 4th ed. New York, NY: Harper Collins; 2001.
51. Kupper Z, Tschacher W. Lack of concordance between subjective improvement and symptom change in psychotic episodes. Br J Clin Psychol. 2008;47(pt 1):75–93. [PubMed]
52. Kuipers E, Garety P, Fowler D, Freeman D, Dunn G, Bebbington P. Cognitive, emotional, and social processes in psychosis: refining cognitive behavioral therapy for persistent positive symptoms. Schizophr Bull. 2006;32(suppl 1):S24–S31. [PMC free article] [PubMed]
53. Ritsner M, Modai I, Endicott J, et al. Differences in quality of life domains and psychopathologic and psychosocial factors in psychiatric patients. J Clin Psychiatry. 2000;61:880–889. [PubMed]
54. Fitzgerald PB, de Castella AR, Filia K, et al. A longitudinal study of patient- and observer-rated quality of life in schizophrenia. Psychiatry Res. 2003;119:55–62. [PubMed]
55. Sitzia J. How valid and reliable are patient satisfaction data? An analysis of 195 studies. Int J Qual Health Care. 1999;11:319–328. [PubMed]
56. Pitt L, Kilbride M, Nothard S, Welford M, Morrison AP. Researching recovery from psychosis: a user-led project. Psychiatr Bull R Coll Psychiatr. 2007;31:55–60.
57. Jacobson N. Greenley D. What is recovery? A conceptual model and explication. Psychiatr Serv. 2007;52:482–485. [PubMed]
58. McGowan JR, Lavender T, Garety PA. Factors in outcome of cognitive-behavioural therapy for psychosis: users’ and clinicians’ views. Psychol Psychother. 2005;78:513–529. [PubMed]

Articles from Schizophrenia Bulletin are provided here courtesy of Oxford University Press