|Home | About | Journals | Submit | Contact Us | Français|
The detection of common mental disorders in humanitarian crisis settings requires a screening tool that is feasible to use as well as sensitive and specific. The Self-Report Questionnaire, developed by the World Health Organization in 1994 to detect presence or absence of common mental health disorders, has frequently been used among conflict-affected and refugee populations. Our goal was to identify a highly predictive and reliable subset of items to serve as a screening tool that can be used in busy, over-crowded, and low-resource primary health care settings to identify women who need mental health attention.
We analyzed the responses on a version of the Self-Report Questionnaire expanded to include two suicidality items from 810 displaced women living in refugee camps in Rwanda. Screening items were selected and evaluated for predictive ability using logistic regression in a cross-validation process, sensitivity and specificity using receiver operating characteristic curve analysis, and internal consistency analysis.
A five-item screening tool resulted. Those items are "Do you feel unhappy?", Do you feel easily nervous, tense, or worried?", "Have you lost interest in things?", "Do you have trouble thinking clearly?" and "Has the thought of ending your life been on your mind?"
The Self-Report Questionnaire-5 may be an important tool for identifying common mental disorders as well as suicide ideation and behaviors when assessing mental health among women in crisis situations. Further evaluation of this tool is warranted.
Mental health problems are a serious and growing public health epidemic, contributing 14% to the global burden of disease (Prince et al., 2007). War and conflict have devastating effects on populations, with women being more acutely affected than their male counterparts (Usta, Farver, & Zein, 2008). Countries affected by humanitarian crises (defined as armed conflict, famine, epidemics, or natural disaster) rank among the lowest in mothers’ and children’s indicators of well-being, including health status, contraceptive use and infant mortality (Women’s Refugee Commission, 2010). Refugees experience depression and posttraumatic stress disorder at more than double the rate of the United States population (Mollica et al., 2004). Prevalence rates of both common mental health disorders and suicidal intention and behavior are high, with one study citing 31% of Darfurian refugee women as meeting criteria for major depression (Kim, Torbay, & Lawry, 2007). Another study with a population of Burmese refugee women awaiting resettlement or repatriation in Thailand found 7.4% had suicidal ideation in the past month (Falb, McCormick, Hemenway, Anfinson, & Silverman, 2013a). Refugee women represent a population with a high potential trauma history, where residing in harsh conditions in addition to experiences of gender-based violence, sexual violence and war-related conflict may increase predisposition to common mental health disorders (Falb et al., 2013a; Falb, McCormick, Hemenway, Anfinson, & Silverman, 2013b; Gupta et al., 2014).
Development efforts have long focused on reproductive health indicators and safe motherhood, while mental health of women (which arguably affects all aspects of women's health), has been neglected, particularly in the Millennium Development Goals. Given that these goals were set for the world's poorest countries where war and human rights violations are endemic (United Nations, 2010), this absence is striking. Frontline approaches to mental health services in post-conflict and post-disaster settings are limited and badly needed (Carter Center, 2012). Detecting women with common mental health disorders that are severe or life-threatening is essential so that they can be connected to what limited psychiatric care is available. Screening that is both sensitive and specific is ideal, but in post-conflict and post-disaster settings where resources are severely constrained, it is particularly important to achieve reasonable specificity so as to focus further assessment on those most likely to be affected. Given the paucity of mental health service providers in these settings, screening via women's health or primary care clinics through a short written or verbal assessment may be an efficient case-finding strategy (Baron, Field, Kafaar, & Honikman, 2014).
The focal measure for this project was the SRQ-SIB, a modified version of the Self Report Questionnaire (SRQ). The original version has long been used in developing countries as a means to measure incidence and prevalence of common mental health disorders, where rather than being predictive of specific mental health diagnoses, the SRQ assesses symptoms of common mental health disorders and uses a cut-point on the total score to indicate the need for further evaluation. Designed by the World Health Organization (1994), the original tool includes 20 items about depression, anxiety, and somatic complaints, and is designed to be either self-administered or administered by a trained interviewer. The SRQ has been psychometrically validated in multiple settings, including conflict and non-conflict situations (Iacoponi & Mari, 1989; Scholte, Verduin, van Lammeren, Rutayisire, & Kamperman, 2011; Ventevogel et al., 2007). Cut-off scores used have varied depending on the population and setting, although a cut-off score of between 6 and 8 has been commonly used to identify presence of common mental health disorders (Harding et al., 1980; Harpham et al., 2003; World Health Organization, 1994).
Our analysis followed steps to develop a set of screening items from a longer instrument. To enhance the rigor of this approach, where no concurrent validation with another measure or a clinical interview was available, we used a random split-halves cross-validation design (Choi et al., 2012; Seng et al., 2010) to select and test items with the “training” half of the dataset first, then repeating the testing with forced coefficients on the “testing” half of the dataset.
The dataset employed for this analysis was from a larger analysis of conflict-affected women using the Reproductive Health Assessment Toolkit for Conflict-Affected Women, which was conducted by the American Refugee Committee with support from the Centers for Disease Control in July and August 2008. The survey was part of a larger parent study that was intended for field staff and management of non-governmental organizations to use to identify and prioritize key women's health needs, translate priorities into programmatic responses, evaluate programs and policies, and to disseminate results for improving the reproductive health of the women in the camps (Division of Reproductive Health, 2007). The sample size of 810 was determined for the original study using the Reproductive Health Assessment for Conflict Affected Women Toolkit guidelines to achieve point estimates within +/− 5% of the true population prevalence, with 95% confidence (Division of Reproductive Health, 2007).
The original study sample was drawn from a population of Congolese refugee women living in long-term temporary camps in Rwanda. Inclusion criteria were women aged 15–49 (deemed to be women of reproductive age) (Division of Reproductive Health, 2007), residing in one of two refugee camps, and with history of reported displacement from their home of origin due to war-related conflict. Exclusion criteria were persons identifying as male, females younger than 15 or older than 49, and those having no reported history of displacement from their home.
This screening tool development project was determined to be exempt from review by the Institutional Review Board at the University of Michigan, Ann Arbor, Michigan, USA, as this was an analysis of previously collected data. This study used de-identified data provided under a data use agreement with the American Refugee Committee.
For the original study, in the absence of a formal institutional review board, the survey data collection received approval from the local refugee councils of both camps, the Rwandan Ministry of Health, and the local office of the United Nations High Commissioner for Refugees.
The Reproductive Health Toolkit questionnaire (Division of Reproductive Health, 2007) uses numerous study-specific items and some scales. For this study we used items related to age, marriage status, and literacy to describe the samples. We also used variables that were available to represent trauma exposures, particularly women who had been forced to have sex during the conflict, women who had been forced to have sex after the conflict and women who had experienced death of a child not due to stillbirth, as a means to evaluate the screening tool’s validity, since common mental disorders and distress are strongly associated with trauma exposure.
The expanded version of the SRQ used in this analysis, the SRQ-SIB, for Self-Report Questionnaire, Suicide Ideation and Behavior, has two added questions about lifetime suicidal behavior. These items were added by the Centers for Disease Control in the Reproductive Health Assessment Toolkit for Conflict Affected Women (Division of Reproductive Health, 2007), to make it a 22-item instrument. For this analysis we opted to use scores of 7 or greater as a middle-ground case definition and because this cut point is a widely used threshold in recent reports (Harpham, Grant, & Rodriguez, 2004; Harpham, Huttly, De Silva, & Abramsky, 2005; Ola et al., 2011; Scholte, Verduin, Kamperman, et al., 2011). Psychometric properties of the SRQ-SIB within this same sample, including factor analysis, have been described elsewhere (Bell, Lori, Redman, & Seng, in press). Since services for mental health are very limited in situations of humanitarian crisis, we aimed to select items that would detect women who most urgently need mental health care and compare the diagnostic accuracy of this subset of questions with the original questionnaire. Our goal was to keep the number of items low so the screening tool could be used in primary care and reproductive health clinics in low-resource settings as a first step in an assessment and referral process.
The parent study was cross-sectional. Because of low literacy levels of many participants, a two-step informed consent process was used. In the first step, a locator went to households selected for inclusion and explained the survey and received verbal consent. In the second step, a community health worker went to the home and explained the survey again, received verbal consent, and administered the survey. Women received a small incentive for participation. No one selected for the survey refused participation and no adverse events from participating were encountered. Individual interviews were conducted in person by a community health worker in the Kinyarwanda language. Because of literacy challenges, the questions were asked verbally and the response set for the SRQ-SIB items was yes, no, or no response.
All analyses were completed in SPSS version 21.0 (IBM, 2012). We first assessed missing data, examined the sample as whole, and described the SRQ-SIB distribution characteristics. As we derived the screening measure on existing data without having another concurrent self-report measure or clinician diagnosis against which to validate the diagnostic utility, we created two random split half datasets to use in the analysis. These training (n=407) and testing (n=403) samples were randomly selected through SPSS via a random seed generator. In order to determine which items were the best predictors of common mental health disorder ‘case’ status, we determined the strength of the association between the case variable (SRQ-SIB score of 7 or greater) and each item in the SRQ-SIB using a series of logistic regressions. We then used three metrics to select the best items: Kuder-Richardson internal consistency analysis, logistic regression percent of variance explained via Nagelkerke's R-squared, and receiver operating characteristic (ROC) analysis to estimate the area under the curve (AUC) of each item. The selected items were assessed for internal consistency reliability and diagnostic utility.
Among the 810 women, 94% responded to all items in the SRQ-SIB. 43 participants skipped one item, and 7 skipped two items. Missing data appeared to be missing at random. In order to preserve all 810 women in the analysis, we opted to count skipped items as “0” or “no,” such that error introduced is in the direction of under estimating common mental health disorders.
The random split into training and testing datasets resulted in samples that were similar in characteristics (Table 1.) The mean age was 29 years. Two-thirds of the sample was married. One in four women were without literacy. Nearly one in four had experienced the death of one or more children. Two-thirds of the sample had experienced some type of victimization during the conflict. There were no characteristics that differed to a statistically significant extent between the training and the testing set. In the total sample, the mean score on the SRQ-SIB was 6.4 (SD=5.3) and the scores were skewed to the right. Using the seven-point cut-off, 42.5% of the women were classified as likely having a common mental health disorder (i.e., as ‘cases’).
Each item of the SRQ-SIB was tested for strength of the association with the case cut point of seven or greater, using the training set and the testing set, with the end goal to identify which items were the most likely to be predictive. Table 2 shows the item-to-total correlations, Nagelkerke R2, and AUC for the top five items. “Are you easily frightened?” and “”Do you feel easily nervous, tense, or worried?” had similar metrics, and we opted to use the more broad query, resulting in a four-item set that demonstrated adequate sensitivity and specificity in relation to case status. We also used these four items to determine if this screening set, using a cut point of two or more positive responses, was able to detect the most distressed cases—those represented by positive reports of past-month suicidal thoughts. Women were classified as having acute or current suicidality based on the SRQ- item, "(In the past four weeks) has the thought of ending your life been on your mind?" 68 out of 810 women (8.4%) answered ‘yes’ to this item. When a cross-tabulation of screening status was conducted using the variable presence or absence of suicidal thoughts in the past four weeks by the top four highest scoring items, 55 (81%) of the suicidal women were identified, but 13 (19%) were overlooked. This clinical validation effort led us to amend our recommendation to include the item asking about current suicidal thoughts. None of the three suicidality questions had performed highly in the item-selection analysis. They had low item-to-total correlational values (ranging from .333 to .507 in the training set), low AUC (.546 to .591), and a lower amount of variance explained (6.4% to 17%). Nevertheless, we considered that the serious implications of suicidal behavior warranted including the item that queries current suicidal thoughts. The five screening items (SRQ-5) are listed in Table 3.
As a validity assessment, we used an independent samples t-test to compare the SRQ-SIB to the SRQ-5 in ability to discriminate between three groups exposed to trauma, women who had been forced to have sex during the conflict, women who had been forced to have sex after the conflict and women who had experienced death of a child not due to stillbirth. There was no substantial overlap between groups. The differences between the two measures reached significance among the forced sex during the conflict group, and were similarly not significant among the other two groups (see Table 4 for full reporting).
Since the screening measure was derived on existing data, we used a cross-validation approach to determine sensitivity and specificity in the absence of another self-report measure or clinician diagnosis. Using the SRQ-5 within the training sample, we predicted common mental disorder case status with a logistic regression model, using the five items of the SRQ-5 as independent variables. We then used those training sample regression coefficients in a parallel logistic regression conducted on the testing sample (Table 5). Receiver operator characteristic curve analysis conducted on the whole sample indicates that sensitivity (88%) and specificity (86%) are adequate for front-line screening in primary care and women’s health settings. The Kuder-Richardson internal consistency for the SRQ-5 is in the good range (.794).
This five-item screener derived from the SRQ-SIB showed good reliability, and indicated a degree of sensitivity and specificity that likely is acceptable for use in low-resource settings. It also demonstrates clinical utility in terms of identifying women with suicidal thoughts as well as women who likely have a common mental disorder. The SRQ-5 met the goals that we and others have sought in using the SRQ-SIB: to determine an acceptable level between a high sensitivity and an agreeable specificity, as found by similar studies (Scholte et al., 2011). While the full SRQ-SIB has an excellent Kuder-Richardson internal consistency of .911, the SRQ-5 is in the good range, most likely due to the small amount of questions. Because the setting of this screening tool development analysis was a crowded refugee camp, the SRQ-5 is consistent with what is useful in very low resource settings where there is not capacity for all who need it to be able to receive mental health services. The SRQ-5 can quickly rule out unaffected women, and it likely will rule in the most at-risk women with common mental disorders, including those with current suicidal thoughts. This type of screening tool may be feasible to use in women’s health service settings, making screening for identification and treatment more feasible (Kagee, Tsai, Lund, & Tomlinson, 2013). We advocate for verbally asking each of the five items and giving further attention to women who give affirmative answers to two or more of the questions.
Suicide ideation and behavior is an important, but understudied, issue in sub-Saharan Africa (Mars, Burrows, Hjelmeland, & Gunnell, 2014). Additionally, trauma exposure has been associated with suicidality in samples of refugees (Jankovic et al., 2013), including in sub-Saharan African women (Robertson et al., 2006).The purpose of developing this screening tool was to identify those who most need clinical intervention for common mental health disorders in a highly trauma-exposed population of women. The clinical relevance and the resultant morbidity and mortality associated with suicide cannot be ignored. Some estimates put lifetime suicide attempts at close to 10% for conflict-affected populations, and higher for women at almost 11% (Akinyemi, Owoaje, Ige, & Popoola, 2012; Kinyanda et al., 2010) in African nations. Approximately 8% of this study’s participants, reported suicidal thoughts in the four weeks prior to taking the survey. Although further validation research is needed, we found indications of good discriminant validity with the SRQ-5 because the women with the trauma exposure of forced sex during the conflict had differences in mean scores on the SRQ-5 that were as strongly different across groups as those on the full SRQ-SIB.
Currently mental health services are often limited in refugee camp settings, but abundant research points to the need to address depression, anxiety, and posttraumatic stress disorder in populations affected by disaster and conflict (Falb et al., 2013a; Scholte, Verduin, Kamperman, et al., 2011; Verduin, Engelhard, Rutayisire, Stronks, & Scholte, 2013). As mental health service delivery in these situations improves, including psychological first aid, it will be important to have practical means to screen those who might be affected. A short, focused, and highly specific screening tool can be of enormous benefit in a refugee camp primary care or women’s health clinic setting. The full self-report questionnaire is an example of a brief assessment scale that is reliable and valid for detecting common mental disorders across many cultures (Scholte et al., 2011). However, this still take considerable time and effort to complete, especially when the population being assessed has low prevalence of literacy. Thus a screening tool with the fewest possible number of items that health care providers could memorize and ask verbally could prove extremely useful. The SRQ-5 can be used verbally or in a check-box format. As this is an initial and brief screening tool, a secondary evaluation of all who screen positive in highly traumatized refugee populations should be considered—possibly by using the full SRQ-SIB, with all who screen positive on the full instrument be referred for additional mental health services.
The SRQ-5 shows promise as a practical tool for screening for common mental health disorders and suicidality among refugee women. More study is needed to build upon the results of this initial study however, including further research to test its utility against standardized epidemiological interview or clinician diagnosis. While this research has shown the utility of a brief screening tool compared to longer self-report questionnaire, research is needed to assess feasibility, acceptability, and practical utility in clinical settings. We envisioned creating a screening tool for women’s health settings, but research also is needed to determine if these five items are similarly useful for screening men. However, screening will not be implemented and its ultimate utility cannot be completely assessed until treatment is available. Mental health supports must be put into place in the refugee camp setting in order for the mental health of the population the camps represent to have access to care, an essential human right.
This screening measure development project was an analysis of existing data. We could not compare the performance of the screening items with clinical diagnosis, but only with performance of the full scale’s cut point as a proxy for diagnosis. The cut-off score of the SRQ-SIB was used in the ROC analysis. It would have been preferable if we could have used diagnosis based on another concurrent, self-administered instrument or based on clinician diagnosis. We acknowledge this as a limitation that could only be partially redressed within this study by use of the cross-validation approach. Additionally, this tool was created with data from a sample of Congolese refugee women, and therefore it is not possible to determine the extent to which these results will generalize to other samples, for example samples that included men or populations exposed to different sorts of conflict. The stigma of mental health is well known in Africa (Bartels et al., 2012; Ola et al., 2011). It is possible that both responses about trauma and mental health symptomatology may have been underreported. Finally, the inherent meanings behind some of the SRQ-SIB questions may vary across cultures and in the context of a refugee camp. An exploration of the cultural and situational relevance of the screening questions warrants further examination.
The SRQ-5 is a sensitive, specific, and reliable screening tool that can be administered—perhaps even verbally—in a far shorter period of time, with very similar results as the full SRQ-SIB from which it was derived. It opens possibilities for clinical assessment that can allow providers in primary care and women’s health settings to rapidly identify women with common mental disorders and suicidality who might otherwise go unnoticed or unassessed.
This work was funded in part by a T32 pre-doctoral training grant from the National Institutes of Health (NIH 5T32NR007073-17/5T32NR007073-18). The authors wish to acknowledge the American Refugee committee, including Leah Elliot and Katie Anfison as the sponsors who allowed us to use their originally collected data.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflict of Interest: None
Sue Anne Bell, University of Michigan School of Nursing.
Jody Lori, University of Michigan School of Nursing.
Richard Redman, University of Michigan School of Nursing.
Julia Seng, University of Michigan School of Nursing, Department of Obstetrics and Gynecology and Institute for Research on Women and Gender.