PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of brbLink to Publisher's site
 
Brain Behav. Sep 2012; 2(5): 576–589.
Published online Jul 26, 2012. doi:  10.1002/brb3.76
PMCID: PMC3489810
Sensitivity, specificity, and predictive power of the “Brief Risk-resilience Index for SCreening,” a brief pan-diagnostic web screen for emotional health
Leanne M Williams,1,2 Nicholas J Cooper,3,4 Stephen R Wisniewski,5 Justine M Gatt,2 Stephen H Koslow,1 Jayashri Kulkarni,6 Savannah DeVarney,3 Evian Gordon,2,3 and Augustus John Rush6
1BRAINnet Foundation, 71 Stephenson Street, Suite 400, San Francisco, California, 94105
2University of Sydney Medical School and Westmead Millennium Institute, Sydney, New South Wales, 2145, Australia
3Brain Resource, Level 12, 235 Jones Street, Ultimo, Sydney, 2007, Australia
4Brain Resource, 1000 Sansome Street, San Francisco, California, 94111
5Department of Epidemiology, University of Pittsburgh, 127 Parran Hall, Pittsburgh, Pennsylvania, 15261
6Duke-NUS Graduate Medical School Singapore, 8 College Road, Singapore, 169857
Leanne M. Williams, Westmead Hospital, University of Sydney Medical School, Sydney, NSW 2145, Australia. Tel: +61 2 9845 8195; Fax: +61 2 9845 8190; E-mail: lea.williams/at/sydney.edu.au
Funding Information This research received no specific grant from any funding agency in the public, commercial, or not for profit sectors. It was supported in part by grants DP0773994 and LP0883621 from the Australian Research Council. Brain Resource was the industry partner on LP0883621.
Received September 14, 2011; Revised May 2, 2012; Accepted May 16, 2012.
Few standardized tools are available for time-efficient screening of emotional health status across diagnostic categories, especially in primary care. We evaluated the 45-question Brief Risk-resilience Index for SCreening (BRISC) and the 15-question mini-BRISC in identifying poor emotional health and coping capacity across a range of diagnostic groups – compared with a detailed clinical assessment – in a large sample of adult outpatients. Participants 18–60 years of age (n = 1079) recruited from 12 medical research and clinical sites completed the computerized assessments. Three index scores were derived from the full BRISC and the mini-BRISC: one for risk (negativity–positivity bias) and two for coping (resilience and social capacity). Summed answers were converted to standardized z-scores. BRISC scores were compared with detailed health assessment and diagnostic interview (for current psychiatric, psychological, and neurological conditions) by clinicians at each site according to diagnostic criteria. Clinicians were blinded to BRISC scores. Clinical assessment stratified participants as having “clinical” (n = 435) or “healthy” (n = 644) diagnostic status. Receiver operating characteristic analyses showed that a z-score threshold of −1.57 on the full BRISC index of emotional health provided an optimal classification of “clinical” versus “healthy” status (sensitivity: 81.2%, specificity: 92.7%, positive predictive power: 80.2%, and negative predictive power: 93.1%). Comparable findings were revealed for the mini-BRISC. Negativity–positivity bias index scores contributed the most to prediction. The negativity–positivity index of emotional health was most sensitive to classifying major depressive disorder (100%), posttraumatic stress disorder (95.8%), and panic disorder (88.7%). The BRISC and mini-BRISC both offer a brief, clinically useful screen to identify individuals at risk of disorders characterized by poor emotion regulation, from those with good emotional health and coping.
Keywords: Depression and anxiety, emotional well-being, Internet, mental health screen, risk and resilience, sensitivity and specificity
Emotional dysregulation is a feature of multiple psychiatric, psychological, and neurological conditions, and conversely, effective emotional regulation characterizes positive well-being, coping, and resilience. Our aim was to use these features to identify a broad screen for poor versus good emotional health across diagnostic and community samples.
Approximately 60% of patients who have psychiatric and neurological disorders seek care from primary care physicians (Regier et al. 1978; Ezzati-Rice and Rohde 2008). Clinicians who are not psychiatric or neurological specialists are increasingly expected to serve roles in early identification, management, and ultimately prevention of these disorders. (Druss et al. 2010). To support these roles, there is demand for a quick screen that can be applied across broad populations and provide immediate feedback. Ideally, such screening tools would be time effective for both physician – given typical heavy patient loads – and patient – picking up a broad set of conditions earlier and more effectively. They would provide an objective and accurate way to identify individuals at risk of psychiatric and neurological conditions, and factor in behaviors which contribute to resilience and capacity to cope. Furthermore, they would provide immediate feedback on case identification via automated reporting.
There is currently a dearth of standardized tools that provide a broad screen of this kind. At the population level, mental health-related disorders go unidentified and thus untreated in 50–65% of cases (Nielson and Williams 1980; Kessler et al. 1985; Schulberg et al. 1985; Katon 1987; Barret et al. 1988; Borus et al. 1988; Schulberg and Burns 1988; Andersen and Harthorn 1989; Ormel et al. 1991; Rydon et al. 1992). Of the available self-report screening scales that could be considered brief and comprising sound psychometric properties, the focus is on screening for a particular diagnosis (Mulrow et al. 1995). For example, the Patient Health Questionnaire-9 item (PHQ-9) screens specifically for diagnostic criteria of depressive disorder (Kroenke et al. 2010), and the Quick Inventory of Depressive Symptoms – Self-Report (QIDS-SR) assesses the severity of symptoms in major depressive disorder (Rush et al. 2003). Other scales are focused on health-related outcomes. For example, the Medical Outcomes Study Short Form (SF-36; Ware and Sherbourne 1992) and its even shorter version (SF-12) are a psychometrically sound survey designed to assess quality-of-life outcomes across diagnoses. It is not intended as a screening tool. Other pan-diagnostic scales with robust psychometric qualities are focused on outcomes for a related set of diagnoses. For example, the OASIS is a brief self-report scale for assessing frequency of anxiety, intensity of anxiety, behavioral avoidance, and functional impairment associated with anxiety to determine symptom and functional outcomes across diagnoses of anxiety disorder (Campbell-Sills et al. 2009).
The BRISC is designed to address gaps in these available tools. First, it provides a quick screen for emotional health relative to a wide spectrum of diagnoses and healthy people, which is not available in currently available instruments. This enables identification of cases at risk of poor mental and neurological health across various disorders and practice settings. Second, it includes measures of coping to inform the triage of those most at risk and coping poorly versus those who are resilient and coping well. This information is also not provided by available instruments. The BRISC has been validated against other self-report measures of emotional health, functional outcome measures, and biological susceptibility factors (for details, see Methods). It is designed to provide a time- and cost-effective screen, delivered via the web, with immediate reporting on results.
This study was designed to evaluate the sensitivity, specificity, and predictive power of the 45-item BRISC and the 15-item “mini-BRISC” in distinguishing clinical versus healthy status across a range of disorders in a large sample of adult outpatients and healthy volunteers. BRISC scores were compared with a detailed assessment of clinical status.
The BRISC
The BRISC was developed and validated within a framework called the “INTEGRATE model”, which draws on psychiatric, psychological, physiological, and neuroscience theories (Gordon et al. 2008; Williams et al. 2008). It is designed to measure, by self-report, the spectrum of good versus poor self-regulation of emotional functions, which underlies mental health and has a basis in neurobiology.
The BRISC measures three core domains: negativity bias, emotional resilience, and social skills. Negativity bias represents hypersensitivity to stress and the expectation of negative outcomes, which elevate the risk for poor brain health (Wichers et al. 2007; Williams et al. 2009, 2010). Positivity Bias is the opposing tendency and quantifies a lack of negativity bias and an expectation of positive and/or neutral outcomes. Emotional resilience is the capacity for self-efficacy. It is premised in the notion that having a “thick skin” (or emotional resilience) may offset poor mental functioning and facilitate good functioning. Social skills is the capacity to engage socially and seek support. These attributes contribute to the ability to cope with poor mental functioning and to facilitate good functioning. Development of the BRISC followed a stepwise process which is detailed in its manual (Brain Resource Ltd publishers 2010). The five main validation steps are summarized below:
Construct validation of content domains
These three domains were validated by principal components analyses of an initial pool of 93 items (Rowe et al. 2007; Williams et al. 2008). Factor analysis confirmed the presence of domains reflecting negativity–positivity bias, emotional resilience, and social skill capacity in a healthy volunteer sample of 1000 individuals who spanned nine decades in age (Rowe et al. 2007; Williams et al. 2008). Using regression analysis, we reduced the number of items loading on these factors to the core 45 items needed to predict them (Wichers et al. 2007). This structure was replicated in an independent sample of 1557 (Brain Resource Ltd publishers 2010).
Face validation
To achieve face validity, the phrasing of the 45 items was slightly adjusted so that the tense and format of the questions in each item were consistent, without affecting the content of the question. The questions are listed in Appendix 1
Construct validation using self-report measures of regulation
Sample: A community sample of 55 healthy volunteers participated in this component of the study (mean age 29.13 ± 8.80 years, range: 19–55 years; 76.4% female). Exclusion criteria were Axis 1 criteria for psychiatric disorder (assessed using the Somatic and Psychological Health Report Questionnaire, SPHERES-12; Hickie et al. 2001), Patient Health Questionnaire for indicators of eating disorder (Kroenke et al. 2010), indicators of neurological disorder (assessed using items from the mental status examination; Trzepacz and Baker 1993), and indicators of alcohol and/or substance dependence assessed using the AUDIT and Fagerstrom nicotine dependency questionnaire (Heatherton et al. 1991; Bush et al. 1998).
Assessments:
  • Emotion Regulation Scale (Gross and John 2003): To assess the capacity to regulate one's emotions in terms of both reappraisal and suppression strategies
  • Internal Control Index (ICI; Duttweiler 1984): To assess internal locus of control related to the belief that reinforcement is contingent on one's own behavior, related to self-confidence and autonomy
Validation outcomes: We conducted correlation analyses between the BRISC scales and the ERQ and ICI, using a corrected P-value of 0.005. Results demonstrated convergent construct validation for each BRISC scale as follows:
  • Negative correlations between lower negativity–positivity bias and higher scores on ICI components of internal control (r = −0.51, P < 0.0001) and self-confidence (r = −0.51, P < 0.0001)
  • Positive correlations between higher emotional resilience and higher scores on the ICI components of internal control (r = 0.39, P = 0.003) and self-confidence (r = 0.39, P = 0.003)
  • Positive correlations between higher social skills and higher scores on the ICI perceived control component (r = 0.40, P = 0.003) and the ERQ reappraisal strategy component (r = 0.56, P < 0.0001)
Construct validation of the negativity bias measures using genetic, autonomic, and brain imaging measures
Sample: Three hundred and three healthy volunteers of European ancestry (mean age 32.92 ± 10.73 years, range: 18–54 years, 49.5% female) took part and completed the BRISC, heart rate recording, and genotyping (Williams et al. 2009, 2010). Of these, matched subsets of 39 and 46 also completed functional magnetic resonance imaging (fMRI) (Williams et al. 2009, 2010).
Assessments: Heart rate and fMRI were recorded during a facial emotion viewing task, under both conscious and nonconscious conditions. DNA was extracted from cheek swab samples and genotyped for the polymorphism of the serotonin transporter gene (5-HTT-LPR) and COMT Val108/158Met genotypes (for details of Methods, Williams et al. 2009, 2010).
Validation outcomes: Greater fear reactivity indexed by both heart rate and activation of brainstem, amygdala, and medial prefrontal cortex circuitry was associated with greater negativity relative to positivity bias. This association was pronounced in individuals with the “risk” alleles, 5 HTT-LPR Short and COMT Met. The findings indicate that a higher negativity bias is underpinned by genetic and fear circuitry susceptibility for emotional disorder.
Ecological validation with real-world functional capacities
The association between the full BRISC and proxy measures of real-world functional outcomes was established in the same sample of 55 participants used to assess construct validation against the ERQ and IC. These proxy measures included the following:
  • Quality of Life assessed by the World Health Organizations Qualify of Life scale, brief version (WHOQOL-BREF) scale (World Health Organization Group 1998)
  • Satisfaction with Life Scale (SWLS; Diener et al. 1985)
  • Work productivity, in terms of both absenteeism (hours absent from work) and presenteeism (performance level), assessed using the Health and Work Performance Questionnaire (HPQ; Kessler et al. 2003)
Validation outcomes: Correlation analyses between the BRISC scales and the WHOQOL-BREF, SWLS, and HPQ, at a corrected P-value of 0.01, demonstrated the following associations:
  • Negative correlations between lower negativity–positivity bias and higher WHOQOL-BREF psychological (r = −0.50, P < 0.0001) and social relationships (P = −0.39, P = 0.003) components of quality of life, and the presenteeism component of the productivity on the HPQ (r = −0.39, P = 0.01)
  • Positive correlations between higher emotional resilience and higher scores on the WHOQOL-BREF psychological component (r = 0.52, P < 0.0001) and satisfaction with life on the SWLS (r = 0.34, P = 0.01)
  • Positive correlations between higher social skills and higher scores on the WHOQOL-BREF components of physical health (r = 0.45, P = 0.001) and environment (r = 0.56, P < 0.0001), satisfaction with life (r = 0.42, P = 0.001) and presenteeism on the HPQ (r = 0.37, P = 0.008)
Study research sites
Participants were recruited from 12 medical research or clinical research sites. These sites agreed to collaborate as partners with Brain Resource to evaluate brain health in patients using a standardized set of assessments and contribute the data to a centralized library (the Brain Resource International Database). The medical research sites were located in universities with teaching hospital outpatient clinics in psychiatry and psychology. The clinical research sites were multidisciplinary outpatient clinics that offer brain health assessment and treatment services (such as EEG testing) for any medical condition. Expert clinicians at each site completed diagnostic interviews and were blinded to the results of the BRISC and other self-report assessments.
Recruitment
This retrospective study recruited participants through advertising and self-referral. Inclusion criteria were in regard to the capacity to undergo a computerized test: reading at Year 5 level (equivalent to Year 6 in England and fifth grade in the United States), normal (or corrected to normal) vision, and ability to use a keyboard. The protocol received independent ethics committee or institutional review board approval before recruitment of participants. All participants signed and dated an approved informed consent form. Where participants consented, these data have also been made available for open sharing and secondary analysis by the research community (Gordon et al. 2005, 2008). All research is in compliance with the Code of Ethics of the World Medical Association (Declaration of Helsinki).
Main measures
The assessment of behavioral health status
At the testing site, participants first completed a computer battery of detailed questions to provide an independent determination of behavioral health status. This assessment comprised established items to assess current or lifetime psychiatric and neurological conditions (Table 1). Stepwise stratification logic was used to determine “clinical” versus “healthy” behavioral health according to the criteria summarized in Figure 1.
Table 1
Table 1
Summary items used in the independent assessment of clinical versus healthy status
Figure 1
Figure 1
Summary of the criteria for independent classification of “good” versus “poor” brain health status.
The BRISC
After the assessment of behavioral health status, yet in the same testing session, participants completed the 45-question BRISC (Appendix 1) via computer, which took about 10 min to complete. The results provided one score for risk (negativity bias) and two scores for coping (emotional resilience and social skills; Rowe et al. 2007; Williams et al. 2008). As indicated in Appendix 1, the 15-question mini version of the BRISC is made up of the five highest-loading BRISC items for each of the core content domains: negativity bias, emotional resilience, and social skills.
Responses to each BRISC question were made on a scale of 1–5, with 5 representing higher functioning (less risk, better coping). We summed the responses for negativity bias, for emotional resilience, and for social skills (raw scores are shown in Appendix 2 for the 45-question BRISC and Appendix 3 for the mini-BRISC). These summed responses were converted to standardized z-scores, using norms in 1317 nonclinical participants established for the BRISC (Rowe et al. 2007). The results of both the assessment of health status and the BRISC were not provided to the participant or the investigator at the time of testing.
Diagnostic interview
The clinicians at each site also completed a semistructured diagnostic interview for each participant which included the current status of any psychiatric, psychological, or neurological disorder. The interview provided confirmation of the disorder against diagnostic criteria, as well as the nature of the primary diagnosis. Clinics were psychiatrists, neurologists, and clinical psychologists.
Methods of analysis
Analyses were undertaken using z-scores for negativity bias, emotional resilience, and social skills for the full BRISC and the mini-BRISC. Pearson correlations were used to examine associations between the three BRISC core content domain scores.
Receiver operating characteristic (ROC) curves were then generated using the “Epi” package from the statistical analysis program “R” version 2.10.1 (http://www.r-project.org/; Ihaka and Gentleman 1996). The goal of the ROC curves was to identify the optimal z-score cutpoint at which BRISC scores classified participants who were independently identified as positive for one or more psychiatric-neurological disorders (clinical) versus those identified as negative for these disorders (healthy). The optimal cutpoint was determined algorithmically to maximize sensitivity plus specificity. This threshold was annotated on these curves with a summary of classification performance. A priori z-score thresholds of −0.5, −1.0, −1.5, and −2.0 were also marked on each ROC curve to provide a context for the interpretation of the optimal threshold. The area under the curve (AUC) statistic was also generated in each case, where 1.0 is the maximum possible value. Sensitivity, specificity, positive predictive power, and negative predictive power were tabulated for the results at the optimal and a priori z-score thresholds.
Characteristics of sample
From March 2005 through December 2009, 1079 participants (mean age = 37.0 years; range: 18–60 years, 51.8% female) completed the assessment of behavioral health status, the full 45-question BRISC, and the clinician-administered diagnostic interview. This sample represented a dataset without missing or indeterminate data. Overall, 644 participants were identified as being of “healthy” status as they answered “no” to all trigger questions. The remaining 435 participants were identified as being of “clinical” status as they answered “yes” to one or more of the trigger questions. The clinical diagnostic interview confirmed that all 435 met diagnostic criteria for a primary psychiatric, psychological, or neurological disorder. Of these 435, 260 met criteria for a primary depressive or anxiety disorder, including major depressive disorder (128, 29.4%), posttraumatic stress disorder (79, 18.2%), and panic disorder (53, 12.2%). Other disorders were traumatic brain injury (86, 19.8%), mild cognitive impairment (48, 11.0%), and psychosis (41, 9.4%; specified as first onset by clinicians based on no prior episodes and being within 3 months of first contact with the health service).
Full BRISC
In the total sample (n = 1079), negativity–positivity bias scores correlated negatively and significantly with both emotional resilience (r = −0.499; P < 0.0001) and social skills (r = −0.279; P < 0.0001; Table 2). These correlations are consistent with the theoretical basis of the BRISC: that the marker of risk (negativity bias) will be inversely related to markers of coping (emotional resilience and social skills). Emotional resilience and social skills were found to have a significant overlap (r = 0.312; P < 0.0001). The degree of overlap is consistent with these markers, reflecting partially separable types of protective factors.
Table 2
Table 2
Correlations between scores on the 45-question BRISC and 15-question mini-BRISC*
ROC analyses
In ROC analyses, negativity bias made the largest contribution to classification. Figure 2 shows the breakdown of clinically confirmed diagnoses for negativity bias in the “clinical” group. Sensitivity of the BRISC was highest for depression, posttraumatic stress disorder, and panic disorder, followed by psychosis, brain injury, and mild cognitive impairment.
Figure 2
Figure 2
45-Item BRISC. Breakdown of classification by diagnosis for negativity bias using the ROC determined threshold.
Table 3 shows the ROC curve analysis results across negativity bias, emotional resilience, social skills, and combined total scores for the 45-item BRISC.
Table 3
Table 3
Summary of sensitivity, specificity, and positive and negative predictive power of the 45-question BRISC scores at z-score thresholds of −2, −1.5, −1, and −0.5 and ROC determined optimal score
For the negativity bias score, the optimal z-score threshold for distinguishing clinical status was −1.14. This threshold was both sensitive (84.9%) and specific (87.6%) in classifying the clinical versus healthy groups. In addition to good positive predictive power at this threshold (70.7%), there was also high negative predictive power (94.3%; Table 3). The AUC value of 0.92 indicated a very high discrimination, reflective of overall accuracy.
Emotional resilience scores revealed a lower optimal threshold of z = −0.43 for distinguishing clinical from healthy status. Sensitivity was at 69.3% and specificity was at 70.0%. The results suggested that these scores contribute most to negative predictive power (81.7%) for supporting decisions about confirming good emotional health (Table 3). Overall accuracy was high (AUC was 0.75).
Social skills scores had an optimal threshold of z = −0.50 for classifying clinical from healthy groups. Sensitivity was at 54.6% and specificity was at 68.1%. Results for these scores suggest that they contribute most to negative predictive power (80.9%) relevant to the confirmation of healthy status (Table 3). These scores contributed to a good overall accuracy (AUC was 0.64).
When ROC analysis was run for the three BRISC scores combined, both positive and negative predictive power were maximized (Table 3). The optimal threshold was z = −1.57 for the combined scores, with a sensitivity of 81.2%, specificity of 92.7%, positive predictive power of 80.2%, and negative predictive power of 93.1%. These values generated a high overall accuracy (AUC of 0.93).
Mini-BRISC
Correlations for the mini-BRISC showed very nearly the same pattern of associations for the total sample, and for the clinical and healthy groups, as were found with the full BRISC. The only exception was the lack of a significant inverse association between negativity bias and social skills for the “clinical” participants (Table 2).
ROC analyses
Table 4 summarizes the ROC curve analysis results for the 15-item BRISC. The mini-BRISC showed a very similar pattern of classification to the full BRISC. For the 5-item negativity bias score, the optimal threshold was z = −1.34, with a sensitivity of 79.9%, specificity of 89.2%, positive predictive power of 72.2%, and negative predictive power of 92.7% (Table 4). Overall accuracy remained very high (AUC of 0.92).
Table 4
Table 4
Summary of sensitivity, specificity, and positive and negative predictive power of the 15-question mini-BRISC scores at z-score thresholds of −2, −1.5, −1, and −0.5 and ROC determined optimal score
The 5-item emotional resilience score showed an optimal threshold of z = −0.95. The results suggested that this score contributes most to specificity (83.3%) and negative predictive power (81.2%) for supporting decisions about confirming healthy status, rather than sensitivity to a clinical condition (Table 4). Accuracy was retained at a similarly high level to that for the full BRISC (AUC of 0.69).
For the 5-item social skills score, the optimal threshold was z = −0.61. The results suggest that this score also contributes most to specificity (71.1%) and negative predictive power (78.7%) for classifying good brain health (Table 4). Overall accuracy remained in the moderate to high range (AUC of 0.58).
For the three mini-BRISC scores combined, both positive and negative predictive power were maximized, as they were for the 45-question version (Table 4). The optimal threshold was z = −1.31 for the combined scores, with a sensitivity of 80.0%, specificity of 89.3%, positive predictive power of 73.3%, and negative predictive power of 92.4%. Overall accuracy was similarly high (AUC of 0.92).
This study evaluated the performance of the web-delivered BRISC (full and mini versions) in identifying emotional dysregulation, a hallmark of clinical status in patients with a range of psychiatric and neurological conditions. The study results were consistent across the full- and mini-BRISC versions. For the three BRISC scores combined, the full 45-question BRISC had a high overall accuracy of 0.93 (Fig. 3). The best classification of clinical status was at the threshold of z = −1.57, substantially below the population average of 0. The mini 15-question BRISC showed a similarly high accuracy of 0.92 (Fig. 4). These results support the effectiveness of the BRISC for identifying risk for a clinical disorder, manifested as loss of emotion regulation.
Figure 3
Figure 3
Receiver operating curve results for the 45-item BRISC, for negativity bias (a), emotional resilience (b), social skills (c), and all three scores combined (d).
Figure 4
Figure 4
Receiver operating curve results for the 15-item BRISC, for negativity bias (a), emotional resilience (b), social skills (c), and all three scores combined (d).
Negativity bias scores made the main contribution to the determination of clinical versus healthy status. For the full 45-question BRISC, the negativity bias score on its own detected clinical status best at a z-score of −1.14, consistent with a threshold of clinical meaningfulness. At this threshold, negativity bias scores showed high accuracy for detecting outpatients with a clinical condition. Across diagnostic categories, negativity bias scores showed the highest detection for major depressive disorder, posttraumatic stress disorder, and panic disorder. This profile of accuracy was duplicated for the mini version's negativity bias scores.
Emotional resilience and social skills separated clinical from healthy status at a higher z-score threshold than did negativity bias. Both emotional resilience and social skills scores showed high specificity. These scores are consistent with the view that a higher-than-average coping capacity may offset risk for a clinical condition and thus support screening and triaging decisions. Results were duplicated for the full and mini version of these scores.
These findings suggest that the BRISC functions to effectively assess the spectrum of poor through to effective emotion regulation. It provides a quick and accurate screen for identifying risk of a clinical disorder across multiple diagnostic categories that takes into account both susceptibility and coping factors. These findings support the use of the BRISC as an objective pan-diagnostic screen for multiple populations, from general through specialty. It expands on the current tools that screen for a particular diagnosis such as major depressive disorder (Mulrow et al. 1995; Rush et al. 2003). The sensitivity of the BRISC was highest in participants with diagnoses of depressive and anxiety disorders, consistent with the concept of negativity bias, but also retained a good level of classification across the other diagnostic categories. It also accomplishes the consideration of coping factors, and how they may offset risk factors, which has not been a part of previous instruments.
Strengths of the study include the large sample size, and coverage of multiple diagnostic groups. Future research is needed to extend the findings and address its limitations. The range of clinical participants included in the study was defined by the types of clinics being operated in participating sites. Future studies are needed to extend the evaluation to other diagnostic groups. Validation work with the BRISC has shown it correlates with real-world capacities such as quality of life and work productivity. Here, the cross-sectional design means there was no opportunity to follow up participants to assess the BRISC in relation to real-world functional outcomes over time. A controlled design would be of value, in which the BRISC is evaluated pretreatment and posttreatment. Future research is also needed to evaluate the replicability of the current findings, and their generalizability to additional populations. A prospective study might address this study's limitations involving the range of clinical participants and the lack of participant follow-up in relation to outcomes. Another valuable area for future studies would be to compare the sensitivity/specificity of the BRISC against multiple disorder-specific measures.
The BRISC offers a web-based tool to support the efficient management of mental and neurological health across populations. Its accuracy enables nonspecialist physicians and physician assistants to confidently screen for emotion dysregulation, as a core feature of mental health issues. The mini-BRISC offers an even briefer screen of emotional health that retains high levels of accuracy and may be especially suitable when a heavy patient load constrains the clinician's time. BRISC scores, especially negativity bias, capture maladaptive emotional reactivity to daily events and could be used to identify this feature of risk for depressive and anxiety disorders within other chronic conditions. The coping scores of emotional resilience and social skills may help to determine which patients are best able to cope with clinical issues and engage social support. Using this tool may help support early management of emotional mental health issues and limit the disproportionate flow on effects to disability and loss of productivity.
Acknowledgments
We acknowledge the Brain Resource International Database which provided data for the study. We also acknowledge the contribution of each of the 12 sites which provided data to the database, as well as the editorial support of Jon Kilner, MS, MA (Pittsburgh, Pennsylvania, USA). This research received no specific grant from any funding agency in the public, commercial, or not for profit sectors. It was supported in part by grants DP0773994 and LP0883621 from the Australian Research Council. Brain Resource was the industry partner on LP0883621.
 
Appendix 1
BRISC items that contribute to negativity bias, emotional resilience, and social skills scores. The subset of items that define the mini version of the BRISC is indicated by bold text. Reverse scored items indicated by (–)*
Negativity bias
 I was often stressed, with my nerves on edge
 I felt I have no value as a person
 I often felt annoyed at the way people treated me
 I lost hope and wanted to give up when something went wrong
 I felt out of control of my life and needed others to help me
 I was reliable and could be counted on to keep my word (–)
 I found it hard to wind down
 I was jumpy and agitated
 I found it difficult to relax
 I was intolerant of anything that kept me from getting on with things
 I was rather touchy
 I tended to over-react to situations
 I felt that I had nothing to look forward to
 I felt that life was meaningless
 I felt down-hearted and blue
 I felt I wasn't worth anything
 I couldn't seem to experience any positive feeling at all
 I found it difficult to work up the initiative to do things
 I felt I close to panic
 I was worried about situations in which I might panic and make a fool of myself
Emotional resilience
 I felt very satisfied with the way I look and act
 I responded best to positive feedback about myself
 When receiving negative comments about myself, I looked for positive things to counter balance those comments
 I was often irritable and argued with people around me (–)
 I was able to plan ahead to meet deadlines
 I was proud of myself and some people might have thought I was putting myself first (–)
 I was conscientious about everything I was asked to do
 I was organized and could work toward a goal in a step by step way
 I usually dithered around before I getting down to focusing on a task (–)
 I worked hard, and always tried to achieve my goals
 There were times when people couldn't rely on me as much as they should have be able to (–)
 I usually tried to consider other people's needs and feelings
 I was always successful at completing my tasks, even if I had more tasks than others
 I was always disorganized and in a mess (–)
Social skills
 I could sense the mood of a group and discuss unspoken feelings
 I always tried to put myself into the place of those I was talking with
 I got feedback that I am a sensitive and understanding person
 I usually took the initiative and introduced myself to strangers
 I got enormous satisfaction by getting people to like me
 I tried to build my close relationships with people
 I took part in social groups
 People reacted to me as if I would do whatever it takes to get ahead (–1)
 I enjoyed socializing and chatting to other people
 I tried out exciting places and things to do
 I was unable to become enthusiastic about anything (–)
*These items have been trademarked as the “BRISC”, by Brain Resource, and are available as a web-delivered assessment for clinical and research uses.
Appendix 2
45-Item BRISC summary of the raw scores and their corresponding z and standardized 10 (STEN) scores for the composite markers; negativity bias, emotional resilience, and social skills. Raw scores were converted to standardized z-scores using the nonclinical norm sample of n = 1317*
Negativity bias (20 items)Emotional resilience (14 items)Social skills (11 items)



Raw scorez-ScoreSTENRaw scorez-ScoreSTENRaw scorez-ScoreSTEN
20−6.54114−6.11111−6.161
21−6.44115−5.95112−5.951
22−6.33116−5.78113−5.741
23−6.22117−5.61114−5.531
24−6.11118−5.45115−5.321
25−6.01119−5.28116−5.111
26−5.90120−5.12117−4.901
27−5.79121−4.95118−4.691
28−5.69122−4.79119−4.471
29−5.58123−4.62120−4.261
30−5.47124−4.46121−4.051
31−5.36125−4.29122−3.841
32−5.26126−4.13123−3.631
33−5.15127−3.96124−3.421
34−5.04128−3.79125−3.211
35−4.94129−3.63126−3.001
36−4.83130−3.46127−2.791
37−4.72131−3.30128−2.581
38−4.62132−3.13129−2.371
39−4.51133−2.97130−2.161.2
40−4.40134−2.80131−1.951.6
41−4.29135−2.64132−1.742.0
42−4.19136−2.47133−1.532.4
43−4.08137−2.31134−1.322.9
44−3.97138−2.141.235−1.113.3
45−3.87139−1.971.636−0.903.7
46−3.76140−1.811.937−0.694.1
47−3.65141−1.642.238−0.484.5
48−3.54142−1.482.539−0.275.0
49−3.44143−1.312.940−0.065.4
50−3.33144−1.153.2410.155.8
51−3.22145−0.983.5420.366.2
52−3.12146−0.823.9430.576.6
53−3.01147−0.654.2440.787.1
54−2.90148−0.494.5450.997.5
55−2.79149−0.324.9461.207.9
56−2.69150−0.155.2471.418.3
57−2.581510.015.5481.628.7
58−2.471520.185.9491.839.2
59−2.371530.346.2502.049.6
60−2.261540.516.5512.2510
61−2.151.2550.676.8522.4610
62−2.041.4560.847.2532.6710
63−1.941.6571.007.5542.8810
64−1.831.8581.177.8553.0910
65−1.722.1591.338.2
66−1.622.3601.508.5
67−1.512.5611.678.8
68−1.402.7621.839.2
69−1.292.9632.009.5
70−1.193.1642.169.8
71−1.083.3652.3310
72−0.973.6662.4910
73−0.873.8672.6610
74−0.764.0682.8210
75−0.654.2692.9910
76−0.544.4703.1510
77−0.444.6
78−0.334.8
79−0.225.1
80−0.125.3
81−0.015.5
820.105.7
830.215.9
840.316.1
850.426.3
860.536.6
870.636.8
880.747.0
890.857.2
900.957.4
911.067.6
921.177.8
931.288.1
941.388.3
951.498.5
961.608.7
971.708.9
981.819.1
991.929.3
1002.039.6
*Of these nonclinical norms, 579 were also included in this study and in each case were identified as having good brain health status.
Appendix 3
15-item BRISC summary of the raw scores and their corresponding z and standardized 10 (STEN) scores for the composite markers; negativity bias, emotional resilience, and social skills. Raw scores were converted to standardized z-scores using the nonclinical norm sample of n = 1317*
Negativity bias (5 items)Emotional resilience (5 items)Social skills (5 items)



Raw scorez-ScoreSTENRaw scorez-ScoreSTENRaw scorez-ScoreSTEN
5−5.5415−4.9115−4.331
6−5.1616−4.5216−4.001
7−4.7917−4.1217−3.671
8−4.4218−3.7218−3.331
9−4.0419−3.3319−3.001
10−3.67110−2.93110−2.671
11−3.30111−2.54111−2.331
12−2.92112−2.141.212−2.001.5
13−2.55113−1.742.013−1.672.2
14−2.181.114−1.352.814−1.332.8
15−1.801.915−0.953.615−1.003.5
16−1.432.616−0.564.416−0.674.2
17−1.063.417−0.165.217−0.334.8
18−0.684.1180.246.0180.005.5
19−0.314.9190.636.8190.336.2
200.065.6201.037.6200.676.8
210.446.4211.428.3211.007.5
220.817.1221.829.1221.338.2
231.187.9232.229.9231.678.8
241.558.6242.6110242.009.5
251.939.4253.0110252.3310
*Of these nonclinical norms, 579 were also included in this study and in each case were identified as having good brain health status.
Conflict of Interest
N. J. C. undertook analyses for this work as senior statistician employee with Brain Resource Ltd. E. G. is founder and receives income as Chairman for Brain Resource Ltd. S. D. D. receives income as VP for Productfor Brain Resource Inc, San Francisco. L. M. W., S. H. K., S. R. W., N. J. C., J. K., A. J. R., and E. G. are members of the publication committee for the international Study for Optimizing Treatment in Depression (iSPOT-D), which is sponsored by Brain Resource and uses the BRISC as one of the hypothesized predictors of treatment outcomes. The BRISC is also offered by Brain Resource as a for-profit screening tool, with financial interest for E. G. as employee and E. G., L. W., S. D. D., and J. G. as stockholders.
Clinical Trial Registry
Trial Registry: http://ClinicalTrials.gov;
Registration Number: NCT00693849
  • Andersen SM, Harthorn BH. The recognition, diagnosis, and treatment of mental disorders by primary care physicians. Med. Care. 1989;27:869–886. [PubMed]
  • Barret JE, Barrett JA, Oxman TE, Gerber PD. The prevalence of psychiatric disorders in a primary care practice. Arch. Gen. Psychiatry. 1988;45:1100–1106. [PubMed]
  • Borus JF, Howes MJ, Devins NP, Rosenberg R, Livingston WW. Primary health care providers’ recognition and diagnosis of mental disorders in their patients. Gen. Hosp. Psychiatry. 1988;10:317–321. [PubMed]
  • Brain Resource Ltd publishers. BRISC and WebNeuro assessment manual, v1.7. Sydney and San Francisco: Brain Resource Ltd publishers; 2010.
  • Breslau N, Kessler RC. The stressor criterion in DSM–IV posttraumatic stress disorder: an empirical investigation. Biol. Psychiatry. 2001;50:699–704. [PubMed]
  • Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch. Intern. Med. 1998;158:1789–1795. [PubMed]
  • Campbell-Sills L, Norman SB, Craske MG, Sullivan G, Lang AJ, Chavira DA, et al. Validation of a brief measure of anxiety-related severity and impairment: the Overall Anxiety Severity and Impairment Scale (OASIS) J. Affect. Disord. 2009;112:92–101. [PMC free article] [PubMed]
  • Diener E, Emmons RA, Larsen RJ, Griffin S. The satisfaction with life scale. J. Pers. Assess. 1985;49:71–75. [PubMed]
  • Druss BG, Mays RA, Edwards VJ, Chapman DP. Primary care, public health and mental health. Prev. Chronic Dis. 2010;7:A04. [PMC free article] [PubMed]
  • Duttweiler PC. The internal control index: a newly developed measure of locus of control. Educ. Psychol. Meas. 1984;44:209–221.
  • Ezzati-Rice TM, Rohde F. Variation in ambulatory health care visits and visits for general checkup by demographic characteristics and insurance status, US civilian noninstitutionalized population ages 18–64, 2005. Statistical brief no. 201. Rockville, MD: Agency for Healthcare Research and Quality; 2008.
  • Gordon E, Cooper N, Rennie C, Hermens D, Williams LM. Integrative neuroscience: the role of a standardized database. Clin. EEG Neurosci. 2005;36:64–75. [PubMed]
  • Gordon E, Barnett KJ, Cooper NJ, Tran N, Williams LM. An “Integrative Neuroscience” platform: application to profiles of Negativity and Positivity Bias. J. Integr. Neurosci. 2008;7:345–366. [PubMed]
  • Gross JJ, John OP. Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. J. Pers. Soc. Psychol. 2003;85:348–362. [PubMed]
  • Heatherton TF, Kozlowski LT, Frecker RC, Fagerström KO. The Fagerström Test for Nicotine Dependence: a revision of the Fagerström Tolerance Questionnaire. Br. J. Addict. 1991;86:1119–1127. [PubMed]
  • Hickie I, Davenport T, Hadzi-Pavlovic D, Koschera A, Naismith S, Scott EM, et al. Development of a simple screening tool for common mental disorders in general practice. Med. J. Aust. 2001;175:S10–S17. [PubMed]
  • Ihaka R, Gentleman R. R: a language for data analysis and graphics. J. Comput. Graph. Statist. 1996;5:299–314.
  • Katon W. The epidemiology of depression in medical care. Int. J. Psychiatry Med. 1987;17:93–112. [PubMed]
  • Kessler LG, Cleary PD, Burke JD., Jr Psychiatric disorders in primary care: results of a follow-up study. Arch. Gen. Psychiatry. 1985;42:583–587. [PubMed]
  • Kessler RC, Barber C, Beck A, Berglund P, Cleary PD, McKenas D, et al. The world health organisation health and work performance questionnaire (HPQ) J. Occup. Environ. Med. 2003;45:156–174. [PubMed]
  • Kroenke K, Spitzer RL, Williams JBW, Lowe B. The Patient Health Questionnaire Somatic, Anxiety and Depressive Symptom Scales; a systematic review. Gen. Hosp. Psychiatry. 2010;22:345–359. [PubMed]
  • Maislin G, Pack AI, Kribbs NB, Smith PL, Schwartz AR, Kline LR, et al. A survey screen for prediction of apnea. Sleep. 1995;18:158–166. [PubMed]
  • Mulrow CD, Williams JW, Gerety MB, Ramirez G, Montiel OM, Kerber C. Case-finding instruments for depression in primary care settings. Ann. Intern. Med. 1995;122:913–921. [PubMed]
  • Nielson AC, Williams T. Depression in ambulatory medical patients. Arch. Gen. Psychiatry. 1980;37:999–1004. [PubMed]
  • Ormel J, Koeter MWJ, van den Brink W, van de Willige G. Recognition, management, and course of anxiety and depression in general practice. Arch. Gen. Psychiatry. 1991;48:700–706. [PubMed]
  • Regier DA, Goldberg ID, Taube CA. The de facto US mental health services system. Arch. Gen. Psychiatry. 1978;35:685–693. [PubMed]
  • Rowe DL, Cooper NJ, Liddell BJ, Clark CR, Gordon E, Williams LM. Brain structure and brain function correlates of general and social cognition. J. Integr. Neurosci. 2007;6:35–74. [PubMed]
  • Rush AJ, Trivedi MH, Ibrahim HM, Carmody TJ, Arnow B, Klein DN, et al. The 16-item Quick Inventory of Depressive Symptomatology (QIDS) Clinician Rating (QIDS-C) and Self-Report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biol. Psychiatry. 2003;54:573–583. [PubMed]
  • Rydon P, Redmon S, Sanson-Fisher RW, Reid ALA. Detection of alcohol-related problems in general practice. J. Stud. Alcohol. 1992;53:197–202. [PubMed]
  • Schulberg HC, Burns BJ. Mental disorders in primary care: epidemiologic, diagnostic, and treatment research directions. Gen. Hosp. Psychiatry. 1988;10:79–87. [PubMed]
  • Schulberg HC, Saul M, McClelland M. Assessing depression in primary medical and psychiatric practices. Arch. Gen. Psychiatry. 1985;12:1164–1170. [PubMed]
  • Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA. 1999;282:1737–1744. [PubMed]
  • Trzepacz PT, Baker RW. The psychiatric mental status examination. Oxford, U.K: Oxford Univ. Press; 1993. p. 202.
  • Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36) Med. Care. 1992;30:473–483. [PubMed]
  • Wichers M, Myin-Germeys I, Jacobs N, Peeters F, Kenis G, Derom C, et al. Genetic risk of depression and stress-induced negative affect in daily life. Br. J. Psychiatry. 2007;191:218–223. [PubMed]
  • Williams LM, Gatt JM, Hatch A, Palmer DM, Nagy M, Rennie C, et al. The Integrate model of emotion, thinking, feeling and self regulation. An application to the “paradox of aging” J. Integr. Neurosci. 2008;7:367–404. [PubMed]
  • Williams LM, Gatt JM, Schofield PR, Olivieri G, Peduto A, Gordon E. “Negativity bias” in risk for depression and anxiety: brain–body fear circuitry correlates, 5-HTT-LPR and early life stress. Neuroimage. 2009;47:804–814. [PubMed]
  • Williams LM, Gatt JM, Grieve SM, Dobson-Stone C, Paul RH, Gordon E, et al. COMT Val108/158Met polymorphism effects on emotional brain function and negativity bias. Neuroimage. 2010;53:918–925. [PubMed]
  • World Health Organization Group. Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychol. Med. 1998;28:551–558. [PubMed]
Articles from Brain and Behavior are provided here courtesy of
Wiley-Blackwell