|Home | About | Journals | Submit | Contact Us | Français|
The present study aimed to utilize a Receiver Operating Characteristic (ROC) approach in order to improve clinical decision-making for adolescents at risk for the development of psychopathology in the aftermath of a natural disaster. Specifically we assessed theoretically-driven individual, interpersonal, and event-related vulnerability factors to determine which indices were most accurate in forecasting PTSD. Furthermore, we aimed to translate these etiological findings by identifying clinical cut-off recommendations for relevant vulnerability factors. Our study consisted of structured phone-based clinical interviews with 2,000 adolescent-parent dyads living within a 5-mile radius of tornados that devastated Joplin, MO, and northern Alabama in Spring 2011. Demographics, tornado incident characteristics, prior trauma, mental health, and family support and conflict were assessed. A subset of youth completed two behavioral assessment tasks online to assess distress tolerance and risk taking behavior. ROC analyses indicated four variables that significantly improved PTSD diagnostic efficiency: Lifetime depression (AUC=.90), trauma history (AUC=.76), social support (AUC=.70), and family conflict (AUC=.72). Youth were 2–3 times more likely to have PTSD if they had elevated scores on any of these variables. Of note, event-related characteristics (e.g., property damage) were not related to PTSD diagnostic status. The present study adds to the literature by making specific recommendations for empirically-based, efficient disaster-related PTSD assessment for adolescents following a natural disaster. Implications for practice and future trauma-related developmental psychopathology research are discussed.
Youth exposed to disasters are vulnerable to trauma-related psychological problems (Udwin, Boyle, Yule, Bolton, & O'Ryan, 2000; Yule et al., 2000). Nearly half of disaster-exposed youth develop clinically elevated posttraumatic stress disorder (PTSD) symptoms (Adams et al., 2014; Furr, Comer, Edmunds, & Kendall, 2010; La Greca et al., 2013). Untreated PTSD in adolescence can persist into adulthood and result in pervasive functional impairment (Morgan, Scourfield, Williams, Jasper, & Lewis, 2003; Yule et al., 2000). It is crucial to identify youth at highest risk for developing PTSD following disasters and stem this deleterious trajectory via intervention. Similarly, given that many youth are also resilient following exposure to a traumatic event, such as a disaster, and may not warrant intervention, a parallel goal for post-disaster assessment is to identify those who should not be referred to unnecessary services. Many promising trauma-focused treatments exist to address youth psychological problems after disasters among those who are identified in need (Pfefferbaum, Sweeton, Newman, Varma, Nitiéma et al., 2014; Pfefferbaum, Sweeton, Newman, Varma, Noffsinger et al., 2014), but disruptions to community infrastructures can impede connecting these adolescents with mental health services. To meet youth mental health needs and guide resource allocation in the post-disaster context, the field needs clinical decision making tools that are efficient to administer and provide sound information for triage, diagnosis, case conceptualization, and treatment planning.
One approach to stratifying youth by PTSD risk is to assess well-established risk factors for each youth (Adams et al., 2014; Furr et al., 2010; La Greca et al., 2013; Norris et al., 2002). Guided by conceptual models rooted in a bioecological, developmental framework (La Greca, Silverman, Vernberg, & Prinstein, 1996; Weems & Overstreet, 2008), vulnerability factors identified in prior studies (Furr et al., 2010; Norris et al., 2002) can be organized across multiple levels of influence. Individual- or youth-level PTSD vulnerability factors include female gender; ethnic minority status; poverty; sustaining personal injury or severe threat to life; living in a highly disrupted community; high levels of secondary stress; prior trauma history; pre-disaster psychiatric problems (e.g., depression), and poor coping. Some youth-level PTSD risk factors can be assessed via computer-based performance tasks, such as distress tolerance (i.e., ability to persist through affective distress; Lejuez, Kahler, & Brown, 2003) and risk-taking propensity (i.e., tendency to engage in potentially harmful behavior with unpredictable rewards and punishments; Lejuez et al., 2002), which have been linked to PTSD symptoms in maltreated youth (Danielson, Ruggiero, Daughters, & Lejuez, 2010) and trauma-exposed adults (Vujanovic, Bonn-Miller, Potter, Marshall, & Zvolensky, 2011). Interpersonal factors include interpersonal conflict (Meiser-Stedman, Yule, Dalgleish, Smith, & Glucksman, 2005; Pelcovitz, Kaplan, DeRosa, Mandel, & Salzinger, 2000; Roussos et al., 2005; Udwin et al., 2000; Wickrama & Kaspar, 2007) and poor social resources (Stallard & Smith, 2007; Udwin et al., 2000; Vernberg, La Greca, Silverman, & Prinstein, 1996). Event-level factors included extreme, widespread damage (Goenjian et al., 1995; Goenjian et al., 2005); serious, ongoing financial hardship (Wickrama & Kaspar, 2007); and high injury and fatality rates (Bryant, Salmon, Sinclair, & Davidson, 2007; Landolt, Vollrath, Timm, Gnehm, & Sennhauser, 2005; Udwin et al., 2000). Prior findings highlight the importance of evaluating multiple types and sources of influence in predicting post-disaster psychopathology in adolescents.
Studies on post-disaster psychopathology typically report correlations summarizing associations between individual, interpersonal, and event level vulnerability factors and PTSD symptoms, or compare samples with versus without PTSD to find associated vulnerability factors. Such results identify candidate etiological factors, but are less helpful in providing post-disaster healthcare workers a risk-assessment framework. Moreover, a comprehensive assessment of all known vulnerability factors is typically not feasible in the post-disaster milieu. Healthcare providers must select variables and assessment tools that balance reliability and decision-making utility against time and resource burdens for patients and themselves. Thus, guidance is needed to develop efficient screening approaches by integrating multiple pieces of data to inform decisions about clinical care.
The use of receiver operating characteristic (ROC) analyses and related data-driven diagnostic efficiency statistics (Kraemer, 1992a; Youngstrom, 2014) represent a strategy for translating basic, post-disaster vulnerability studies into personalized, clinically useful information for decision-making. Rooted in signal detection theory, Bayesian statistics (Swets, Dawes, & Monahan, 2000) and evidence-based medicine (Chambless & Ollendick, 2001; Cohen, Sanborn, & Shiffrin, 2008; Jenkins, Youngstrom, Washburn, & Youngstrom, 2011), ROC analyses facilitate making empirically-derived algorithms to estimate probabilities of meeting diagnostic criteria for each person in light of his or her clinical profile (Straus, Glasziou, Richardson, & Haynes, 2011). This approach to prediction is more consistent, reliable, and efficient than clinical judgment, which often relies on arbitrary or idiosyncratic strategies for weighting multiple pieces of information to make clinical decisions (Meehl, 1954; Rettew, Lynch, Achenbach, Dumenci, & Ivanova, 2009).
Using ROC-based methods in post-disaster settings can reduce assessment-burden by prioritizing the most diagnostically predictive factors within the assessment protocol (Youngstrom & Frazier, 2013). Utilizing an assessment framework with optimized screening tools can address front-line psychiatrists', psychologists', and other disaster health workers' need to efficiently triage large groups of patients and establish clear plans of action for each person. Through the use of ROC procedures, improved assessment protocols have been developed for a wide range of pediatric mental health disorders including bipolar disorder (Youngstrom, Freeman, & Jenkins, 2009), mood disorders (Youngstrom, 2014), and anxiety disorders (Elkins, Carpenter, Pincus, & Comer, 2014; Van Meter et al., 2014). Most relevant to the present study, ROC approaches have also been applied to improving assessment protocols for PTSD in community (Suliman, Kaminer, Seedat, & Stein, 2005) and outpatient settings (You, Youngstrom, Feeny, Youngstrom, & Findling, in press), paving the way for more targeted referrals and interventions for individual who suffer from these symptoms. The present study seeks to build off these past studies by a) assessing PTSD specifically within a post-disaster context and b) including a wide-range of individual, interpersonal, and event-related vulnerabilities as part of the protocol. Broadening the assessment protocol beyond self-report PTSD symptoms may provide a more accurate assessment of post-trauma functioning (O'Donnell et al., 2008).
This study's overarching aim was to inform decision-making for clinicians and health workers following disasters to identify youth at highest risk for PTSD. The first goal was to identify the most potent predictive factors from a set of multi-method (caregiver report, self-report, behavioral tasks), theoretically and empirically justified candidates applied to disaster-exposed adolescents. The second goal was to generate clinical diagnostic likelihood ratios (DLRs; Jaeschke, Guyatt, & Sackett, 1994; Straus, Glasziou et al., 2011) for PTSD risk. As an extension of this latter goal, we aimed to provide a template for how the results can be used clinically. Specifically, we provide an example of how clinicians can apply DLRs by starting with the likelihood of having a diagnosis of PTSD in the general population (the base rate of PTSD in our sample, reported previously in Adams et al., 2014) and adjusting that estimate based on a vulnerable score on the factors found to be most predictive of PTSD risk. Given different base rates of PTSD for boys (6%), girls (8%), younger (ages 12-14; 6%), and older (ages 15-17; 8%) adolescents in our sample (Adams et al., 2014), we provide separate results for the increased likelihood of experiencing PTSD in each of these groups.
Participants (n=2,000; 50.9% female) came from communities affected by the Spring 2011 tornadoes (Joplin, Missouri; Alabama). Participants were 12 to 17 years old (M=14.5, SD=1.7), 71% White/Caucasian, 26% Black/African-American, and 4% other per adolescent self-report. See (Adams et al., 2014) for further demographic and disaster details.
Study design and procedures are detailed in Ruggiero et al. (2015). Selection criteria prioritized identification of families directly impacted by tornadoes. Latitude/longitude coordinates were used to obtain mailing addresses for households within a 5 mile radius of the impact areas (NOAA, 2011). Eligibility criteria included: 1) family residing at their address at the time of the tornado, 2) an adolescent aged 12 to 17 years and legal guardian, and 3) reliable home Internet access.1 Reliable home Internet access was required because data were collected for a randomized controlled trial evaluating a web-based intervention for disaster-related mental health problems (Ruggiero et al., 2015). Addresses matched to a published phone number were contacted and screened by telephone. Addresses unmatched to a phone number were sent a letter explaining the study and a screening questionnaire that assessed inclusion criteria and requested telephone contacts. Adolescent- parent dyads independently completed structured, standardized computer-assisted telephone interviews conducted in a single interview session by well-trained professional interviewers. These interviews assessed PTSD and depression symptoms, substance use, trauma history, disaster impact, social support, and family conflict approximately 9 months (M=8.8, SD=2.6) following the tornado. The overall cooperation rate--(i.e., number screened/number screened + screen-outs + unknown eligibility)--was 61%. Adolescents received a $15 incentive for the interview. All adolescents were provided log-in information to a study website upon completion of the interview.
When adolescents signed in to the study website subsequent to the interview, they were invited to complete two brief game-like computer tasks (BART-Y and BIRD, described below). In total, 352 adolescents completed these tasks. Completers did not differ from noncompleters on any demographic or clinical variables (all ps>.10).
The National Survey of Adolescents-Replication PTSD module (Resnick, Kilpatrick, Dansky, Saunders, & Best, 1993; Ruggiero et al., 2015) assessed the 17 DSM-IV (APA, 2004) PTSD symptom criteria in adolescents. Only symptoms that had been present at least 2 weeks and since the tornado were assessed. Total scores were then dichotomized as either having PTSD or not based on whether symptom counts in each diagnostic cluster were met and functional impairment was endorsed (see Kilpatrick et al., 2003 for psychometric details).
Caregivers reported on the family's experiences with the tornado, including property damage (to house, vehicle, furniture, personal item and pet), loss of services for over a week (water, electricity, clean clothing, food, shelter, transportation, and spending money), and interpersonal impact (present for the tornado, feared death, feared for safety of others, fear of injury to self, feared for pets). Responses were dichotomized (0=no/1=yes) and summed; higher scores meant worse severity (range = 0–17; M=3.71; SD=2.61).
The Social Support for Adolescents Scale (SSAS; Seidman et al., 1995) assessed the extent to which youth could turn to their mothers, fathers, siblings, close friends, and peers for 1) emotional social support; 2) instrumental social support; and 3) fun. Responses used a 3-point Likert scale ranging from Not at all to A great deal. Higher scores indicated elevated social support. Global support scores were summed across relationships and averaged across social support types. Total scores ranged from 1.67 to 10 (M=7.09; SD=1.64). Reliability (α=.80) was good, consistent with past research (Birman, Trickett, & Vinokurov, 2002).
The Conflict Belief Questionnaire Short Form (CBQ; Prinz, Foster, Kent, & O'Leary, 1979; Robin & Foster, 1989). Adolescents completed the CBQ Short Form, a 20-item true/false measure of parent-adolescent conflict. A total score is computed; higher scores indicate greater conflict. Total scores ranged between 0 and 20 across administrations (M=2.78; SD=3.91). Similar to past research using the CBQ Short Form (Kane & Garber, 2009), this scale had excellent internal consistency in our sample (α=.90).
Adolescents reported their experience of five different types of potentially traumatic events including physical assault, physical abuse, witnessing domestic violence, witnessing community violence, and serious accidents. Behaviorally specific prompts probed each trauma type, consistent with reliable and valid questions used in the National Survey of Adolescents (NSA; Kilpatrick et al., 2000). A count of the potentially traumatic event types endorsed by each adolescent in his or her lifetime indexed trauma history severity (range =0–5; M=1.03; SD=1.13).
The Behavioral Indicator of Resiliency to Distress (BIRD; Danielson et al., 2010; Lejuez, Daughters, Danielson, & Ruggiero, 2006) is a behavioral task that measures distress tolerance (DT). Adolescents selected a series of moving targets on a computer screen, getting points (when successful) or loud, unpleasant noises (when unsuccessful). Adolescents could exchange their points for a prize at the end of the task. Trials became faster, more challenging, and more aversive as the task progressed. During the final level of testing, adolescents could elect to stop at any time, but quitting hypothetically influenced the magnitude of their prizes. DT was measured by persistence (i.e., total time in minutes to quit) on the final level (range=0.03–7.30; M=2.51; SD=1.93). Higher times indicated higher levels of DT (Lejuez et al., 2006).
The Balloon Analog Risk Task-Youth (BART-Y; Lejuez et al., 2007) assessed risk-taking propensity by accounting for Participants' inflated virtual balloons via mouse click. Each pump earned the participant one point. If a balloon was pumped past its pre-set explosion point, the participant lost all that balloon's points. Each pump increased the likelihood of popping. Adolescents could stop pumping and bank each trial's points. The total number of pumps across 30 balloon trials quantified risk-taking propensity (range=1–203; M=101.17; SD = 45.05).
The CRAFFT Substance Abuse Screening Tool (Knight, Sherritt, Shrier, Harris, & Chang, 2002) is a six-item (yes/no) measure of risky substance use behavior. Higher summed scores indicate greater risk. The present study assessed for lifetime substance abuse. CRAFFT total scores ranged between 0 and 5 (M=0.18; SD=0.65). The CRAFFT reliability (α=0.54) was consistent with prior work using the CRAFFT (Dhalla, Zumbo, & Poole, 2011).
The National Survey of Adolescents-Replication MDD module (Resnick et al., 1993) assessed the 9 DSM-IV (APA, 2004) depression symptom criteria in adolescents. Symptoms were scored dichotomously (1=present/0=absent). Analyses used lifetime MDD symptom count as a dimensional variable (Range 0–9; M=1.35; SD=1.94). Reliability (α=.80) was comparable to previous research with this measure (Boscarino, Galea, Ahern, Resnick, & Vlahov, 2002).
Receiver Operating Characteristics (ROC) analyses followed recommendations by Youngstrom (2014). Back-to-back histograms investigated the degree of degeneracy across indicators (i.e., high scores on an indicator variable corresponding to no diagnosis; Van Meter et al., 2014). Areas under the curve (AUC) analyses quantified how well each predictor classified PTSD diagnoses (i.e., sensitivity) and identified cases that did not have PTSD (i.e., specificity). Per Swets' (1988) recommendations, AUC values above 0.70 were deemed adequate predictors of PTSD status. Hanley and McNeil's (1983) test of dependent AUCs tested which indicator had the best predictive accuracy for PTSD diagnoses. For the most discriminating predictors (AUC>.70), we created multilevel diagnostic likelihood ratios based on tertiles (i.e., balanced groups of high, medium, and low scores; Straus, Tetroe, & Graham, 2011). DLRs less than 1 indicate that corresponding predictor scores are less likely to lead to a diagnosis of PTSD. For DLR scores between 0.10 and 0.25, one can be moderately certain that the person does not have a PTSD diagnosis in most settings (Straus, Tetroe, & Graham, 2011). Scores above 1 indicate an increased risk for PTSD diagnosis. In cases, where age and sex correlate to the outcome, logistic regressions were used to examine if the influence of predictors varied as a function of these demographic variables. Similarly, as PTSD-risk following a natural disaster is multifaceted (Furr et al., 2010), and indicators for psychological distress should be studied in combination rather than isolation (Beauchaine & Gatzke-Kopp, 2012), interaction terms between predictor variables were created and logistic regression analyses tested if sets of predictors showed incremental value in identifying PTSD (i.e., interactions between predictor variables were tested). Finally, posterior probabilities were calculated for each vulnerable score range on significant indicators for PTSD. Posterior probabilities represent the increased risk for presenting with PTSD once accounting for a vulnerable score on a given indicator. Posterior probabilities were then used to complete standardized nomograms to demonstrate how clinicians could translate ROC analyses into real time decision making regarding patient-risk for PTSD especially for patients who present with more than one indicator of risk.
Descriptive data is provided in Table 1 and Figure 1. Table 1 presents bivariate correlations between all of the study variables. Of note, females and older adolescents were more likely to report a diagnosis of PTSD since the natural disaster (p < .05). Figure 1 illustrates a series of population pyramids comparing the distribution of different vulnerability scores for cases without PTSD versus those with a PTSD diagnosis. Very few examples of degeneracy (i.e., a high vulnerability score corresponding to no diagnosis) were identified across the indicators.
Of the 2,000 adolescents completing baseline assessments, 126 met diagnostic criteria for PTSD, well exceeding the recommended minimum of 20 cases for estimating diagnostic efficiency parameters (Kraemer, 1992b). Table 2 presents AUC statistics and corresponding Cohen's d scores. Distress tolerance and risk taking propensity did not discriminate PTSD significantly better than chance (p > .05). Furthermore, disaster impact (AUC=.62) and lifetime substance abuse (AUC=.68) were deemed poor predictors (i.e., AUC <.70) of PTSD. The remaining four indicators (lifetime depression, trauma history, social support, family conflict) performed fair or better (Swets, 1988), with effect sizes in the medium (d >.50) to large (d >.80) range (Cohen, 1988). Analyses suggested that none of the indicators varied by sex (p > .05) or age (p > .20). Thus, the potential consequences for each significant indicator were consistent for boys, girls, and all adolescents within our population.
It was next tested which of these four constructs (lifetime depression, trauma history, social support, and family conflict) forecasted the greatest risk for PTSD within our sample (Hanley & McNeil, 1983). Lifetime depression predicted PTSD diagnosis significantly better than trauma history (z=7.07, p<.001), social support (z=10.00, p<.001), and family conflict (z=6.04, p<.001). Trauma history outperformed social support (z=2.34, p<.05); however, post hoc correction rendered this finding nonsignificant. Thus, lifetime depression performed best, and all other variables were functionally equivalent in predicting PTSD diagnosis.
We then examined how different score ranges on each significant indicator corresponded to the likelihood of presenting with a diagnosis of PTSD. Specifically, diagnostic likelihood ratios (DLRs) across three equal cutpoints (tertiles) were calculated (Straus et al., 2011). As shown in Table 3, there is a linear pattern of increasing DLRs (i.e., heightened risk for PTSD) with more vulnerable scores. Scores in the low and medium range across variables indicated that adolescents with these scores were less likely to experience PTSD compared to the base rate in the population (DLRs < 1). Meanwhile, adolescents who scored in the high range were at greater risk for presenting with a diagnosis of PTSD (DLRs > 1; Straus et al., 2011).2
To better understand the level of risk associated with a vulnerable score, DLRs from the most vulnerable group were taken and inserted into Table 4. Table 4 demonstrates how the DLRs can be interpreted clinically by transforming the likelihood of having a diagnosis of PTSD in the general population (i.e., prior probability) to the percentage chance of having PTSD with a vulnerable score (posterior probability; Youngstrom, 2014). Because of different base rates for boys, girls, younger (ages 12-14), and older (ages 15-17) adolescents in our sample (Adams et al., 2014), separate results are reported for the increased likelihood of experiencing PTSD in each of these groups in light of vulnerable scores on each indictor. As seen in the posterior probabilities in Table 4, findings were both statistically and clinically significant. Youth were approximately 2 to 3 times more likely to have PTSD if they had elevated scores on any of the four predictor variables.
Finally, two-way interactions between predictors were examined to test whether having a vulnerable score on more than one indicator for PTSD conferred additional risk. The interaction between social support and family conflict (B=.03, SE=.02; W=4.59, p=.03) was the only significant interaction between the indicator variables (all other ps>.10). Results suggested youth who endorsed low social support (score <6.33) and high levels of conflict with their parents (scores ≥3) were at increased risk for PTSD following natural disasters. Similar to Table 4, Figure 2 displays two probability nomograms that can also be used to highlight how DLRs can be used to interpret increased risk for a given diagnosis. In cases where there are multiple vulnerabilities that explain risk above and beyond one of the vulnerabilities independently (i.e., a significant interaction) clinicians can line probability nomograms side by side (Youngstrom, 2014). As seen in Figure 2, the left side of the left nomogram represents the general population's prior probability (i.e., base rate) for PTSD (7% in our study; Adams et al., 2014). The middle line of the nomogram represents the calculated DLR for a given predictor of PTSD. For instance, in the case of family conflict, the highest tertile of family conflict corresponded to a DLR of 2.04. With this information, clinicians would then draw a straight line from the prior probability through the DLR to calculate the posterior probability. In this case, individuals with high family conflict have a 13% of presenting with PTSD (also shown in Table 4). This posterior probability now becomes the prior probability in the second nomogram (displayed on the right). The DLR for this second nomogram comes from the second significant indicator in the interaction, in this case low social support (DLR = 2.20). We conclude by then repeating the step of drawing a line from the prior probability through the DLR to understand the posterior probability of having both vulnerability scores. As shown in Figure 2, an adolescent high in parental conflict goes from having a 13% probability of having a diagnosis of PTSD to a 25% probability if they also report low levels of social support (nearly a four-fold increase).3
Given the deleterious effects of PTSD in adolescence, there is great public health utility in reducing risk for its onset in the aftermath of a disaster. Mental health service resources are scarce in disaster-stricken communities, underscoring the value of empirically-based, efficient clinical-decision making tools. Present data show that, in the wake of a natural disaster, lifetime depression and trauma history, as well as social support and family conflict, effectively predict PTSD status among disaster-exposed youth. Depression emerged as the most diagnostically powerful, increasing the odds of PTSD 3.5-fold among youth reporting more than two depression symptoms. Assessing depression history may be highly informative for post-disaster PTSD risk screening. Results also underscore the importance of assessing other psychosocial factors such as interactions with family members, and peer social support. The prominent roles low social support and high family conflict showed for PTSD risk prediction are consistent with a recent meta-analysis of pediatric PTSD risk factors (Trickey, Siddaway, Meiser-Stedman, Serpell, & Field, 2012), supporting potential protective effects of social connectedness in the aftermath of trauma (Allen, Marcelin, Schmitz, Hausman, & Shultz, 2012; Landau, 2013; Norris & Stevens, 2007). The current study extends research by providing specific cut score ranges and DLRs to promote individualized risk probability estimation for disaster-exposed youth.
Although it is not surprising that trauma and mental health history are clinically useful in predicting post-disaster PTSD onset, it was surprising that individual-related factors would have greater diagnostic utility than impact of the disaster itself (i.e., event-related vulnerability). These findings highlight the utility of the ROC approach. A plethora of research has shown that traumatic event incident characteristics, such as degree of property damage (Goenjian et al., 1995; Goenjian et al., 2005; van den Berg, Wong, van der Velden, Boshuizen, & Grievink, 2012), are significantly related to post-disaster mental health symptoms (Goenjian et al., 2001; Parson, Fussell, Rhodes, & Waters, 2012; Zhang, Shi, Wang, & Liu, 2011). The current study suggests that while such disaster incident characteristics may contribute to the etiology and/or maintenance of PTSD per prior research, assessing these variables in post-disaster screenings may not improve prediction of PTSD beyond chance—and other variables are more effective for identifying those at highest risk.
It was also somewhat surprising that other individual characteristics (distress tolerance and risk-taking propensity) that have been previously empirically linked with PTSD risk did not improve diagnostic accuracy in this post-disaster sample. The extant literature suggests these factors may still be relevant in post-disaster mental health. Assessment of these characteristics may help in intervention tailoring (Zatzick et al., 2011) and/or detecting other mental health outcomes (e.g., externalizing problems; Daughters et al., 2009). There has been a recent emphasis on the utility of behavioral assessment tasks, including those that incorporate physiological measures, to potentially help inform empirically based assessment in mental health (Youngstrom & de Los Reyes, 2015). Additional research is needed to ultimately determine the utility of behavioral tasks and physiological measures in PTSD risk prediction.
Study strengths include the behaviorally-specific and comprehensive assessment of trauma history and the calculation of PTSD rates by gender and age group in our sample. The ROC approach also reminds clinicians to consider base rates of target problems, which vary by setting and context (Meehl, 1954; Straus, Tetroe, & Graham, 2011). For example, the base rate of PTSD in a clinical setting (e.g., 7.2–14.4%; Zimmerman & Mattia, 1999) is higher than community base rates (5%; Kilpatrick et al., 2003; McLaughlin et al., 2013), including among disaster-exposed community populations (6.7% in this sample; Adams et al., 2014). The DLRs provide locally contextualized interpretation in the form of a Bayesian posterior probability.
Despite these strengths, there are also noteworthy limitations. First, the length of time since the tornado for the initial post-disaster assessment in the current study (average of 8 months) prevented assessment of pre-existing vulnerability factors and symptoms immediately following the disaster. It is possible that certain factors such as social support or family dynamics may have been strong before the storm or in its immediate aftermath, then deteriorated by the time the assessments took place. Alternatively, families may have experienced temporary increases or breakdowns in support and conflict immediately following the storm that resolved by the assessments in this study. Thus the utility of these factors in predicting PTSD may vary by timing of the assessment—a hypothesis that should be tested in future studies. Second, a relatively lower number of youth who completed the behavioral assessment tasks in comparison to those who completed the interviews, which may prevent definitive conclusions from being drawn about the utility of behavioral assessment tasks in PTSD risk assessment, as noted above. Relatedly, while the multi-method approach of the study was a strength, findings indicated that only self-report measures predicted self-reported PTSD diagnostic status. Thus, shared method variance between the measures may have contributed to the strong effects seen for many of the self-report measures, and the null findings concerning distress tolerance and risk-taking propensity in predicting PTSD status. Because most assessment batteries rely on self-reported data, understanding which self-report data is most valuable in forecasting PTSD within a post-disaster context carries significant clinical value.
One of the most effective strategies for achieving accurate and efficient mental health risk prediction is the use of psychological measures that are low burden in their cost and administration time and high yield in their output (i.e., low burden:high yield, LB:HY). These considerations are particularly important following disasters, when efficient resource allocation is key during recovery. Empirically-based clinical decision making—ROC analyses in particular—help achieve LB:HY goal by winnowing down the potential list of variables to screen to those most likely to inform PTSD risk status. Our ROC analyses here show that youth with high (problematic) scores across four key risk factors (lifetime depression, trauma history, social support, and family conflict) can be identified via brief sets of questions. Consideration of these four factors also will allow public health workers to triage a large group of people into a low risk category correctly, providing more fine-grained probability estimates for higher risk individuals based on presence or absence of specific risk factors.
Practical considerations for how to assess the four key post-disaster PTSD risk factors identified here (depression, trauma history, social support, family conflict) will depend somewhat on the resources (e.g., time, personnel, cost, computer and/or reliable internet access) available in the particular post-disaster context in which the recommendations are being applied. These four variables were assessed in the current study via phone-based interview. Each of the key risk factors identified here also may be feasibly and validly assessed using paper-and-pencil questionnaires or tablet-based surveys. Once these values are obtained, scores can be compared to the vulnerability scores reported in Tables 3 and and44 to yield a DLR and a Posttest Probability, as described earlier, to guide clinical decisions about intervention and referrals. Future studies should evaluate the correspondence between other publically available standardized measures for assessing these constructs and the measures used in the current study in predicting PTSD risk. Additional work may involve development of tools that can automate the process of calculating risk scores for high volume screening events in the post-disaster context.
Current findings also inform future trauma-related developmental psychopathology research directions to further evaluate and improve PTSD risk assessment among youth. Given the relevance of developmentally sensitive windows with regard to trauma exposure, the four identified potent predictors in this study, and their respective cut-scores, should be evaluated in prospective investigations across different age groups of youth with future disasters and mass traumatic events. Future studies can similarly inform the ideal timing of post-disaster PTSD risk assessment, which is a commonly posed question in practice (Pfefferbaum et al., 2014ab). In addition, given that changes to the PTSD diagnostic criteria have occurred since the execution of the current study (e.g., DSM-5 PTSD includes four clusters versus three clusters of symptoms), it will be important to determine if the results presented here replicate in a post-disaster youth sample assessed with the DSM-5 PTSD criteria. Finally, ROC analyses may be applied to other research data sets and other types of traumatic events to similarly derive clinically applicable information for a variety of problems and populations (Youngstrom, Choukas-Bradley, Calhoun, & Jensen-Doss, 2014).
This work was supported by 1R21MH086313 (PI: Danielson) and 1R01MH081056 (PI: Ruggiero) from the National Institute of Mental Health (NIMH). The preparation of this manuscript was supported by R01DA031285 (PI: Danielson from the National Institute on Drug Abuse (NIDA), P50AA010761 (PI: Becker) from the National Institute on Alcohol Abuse and Alcoholism (NIAAA) and T32MH018869 (PIs: Danielson & Kilpatrick) from the NIMH. Views expressed herein are those of the authors and do not necessarily represent the official views of NIMH, NIDA, or NIAAA.
The authors declare they have no conflict of interest.
Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent: Informed consent was obtained from all individual participants included in the study.
1Only 2.1% of all adolescents contacted were ruled out due to Internet access, suggesting that our inclusion criteria did not have an adverse impact on the generalizability of our results.
2In addition to the methods outlined by Strauss and colleagues regarding tertiles, Kraemer's (1992a) method to optimize thresholds was also examined. Results using Kraemer's approach were remarkably similar to the cutpoint in the high categories (low for social support) using the initial method. The one exception was depressive symptoms, wherein a cutpoint of greater than 3, opposed to 2, symptoms resulted in the highest kappa (0.70).
3It should be noted that this new probability is not a true assessment of the interaction between social support and conflict. Instead, Figure 2 provides a quick heuristic for interested clinicians to calculate an estimated range of the additive consequences of having a significant interaction between two vulnerabilities.