The Short-Term Assessment of Risk and Treatability (START; Webster, Martin, Brink, Nicholls, & Desmarais, 2009
; Webster, Martin, Brink, Nicholls, & Middleton, 2004
) is a clinical guide designed to aid in the assessment and management of short-term risks that adults involved in mental health, forensic, or correctional settings may experience, including risk for violence towards others, self-harm, suicide, victimization, substance abuse, self-neglect, and unauthorized leave. Compared to other adult risk assessment instruments, the START was developed with an emphasis on strength factors in addition to vulnerabilities or risk factors, and dynamic or modifiable factors that are relevant to treatment-planning. Furthermore, rather than focusing solely on an individual’s risk of violence, the START aims to provide a more integrated assessment of multiple adverse events relevant to care (e.g., self-harm, suicide, victimization, substance abuse).
The START uses a structured professional judgment (SPJ) model of risk assessment, meaning that rather than relying on total scores, evaluators are guided to make a final risk estimate of low, moderate, or high risk on each of the outcomes after systematically considering an individual’s current strengths, vulnerabilities, and prior behaviors. In making these final risk estimates, evaluators may place a heavier emphasis on some factors than others and consider case-specific factors that are not included in the START items. In this respect, the START functions as an aide mèmoire
(see Webster, Douglas, Eaves, & Hart, 1997
) orienting professionals to evidence-based factors relevant to risk across the eight domains, while allowing evaluators to add their own knowledge regarding how these factors may be best understood in the context of an individual case.
Although the START was developed for use with adults, the authors received numerous queries pertaining to the use of the START with adolescents. Consequently, the Short-Term Assessment of Risk and Treatability: Adolescent Version (START:AV; Nicholls, Viljoen, Cruise, Desmarais, & Webster, 2010
; Viljoen, Cruise, Nicholls, Desmarais, & Webster, in preparation) was developed to respond to this clinical interest (see Viljoen, Cruise, Nicholls, Desmarais, & Webster, this issue). At the time the START:AV was developed, several well-validated measures, such as the Structured Assessment of Violence Risk in Youth (SAVRY; Borum, Bartel, & Forth, 2006
), existed to assess adolescents’ risk of violence. For that reason the START team was reluctant to add another measure to the field. However, attention to broader adverse outcomes, such as self-harm, suicide, victimization, and substance abuse, was lacking. Despite widespread agreement that risk may be particularly dynamic during adolescence (Borum, 2003
; Grisso, 1998
; Prentky & Righthand, 2003
), there were few adolescent measures that were designed to assess short-term
risk. In addition, although several adolescent risk assessment tools included some protective factors, there remained a need for a more balanced emphasis on risk factors and strength factors. Thus, the adolescent version of the START aimed to complement existing adolescent risk assessment approaches through its focus on multiple adverse outcomes, short-term dynamic assessment, and strength as well as vulnerability factors.
To adapt the START for adolescents, a stepwise, developmentally-informed approach was adopted. First, a measure development team was formed by bringing together members of the adult START (Nicholls, Desmarais, Webster) alongside individuals with clinical and research expertise with adolescents (Cruise, Viljoen). Second, to avoid creating an approach that replicated existing measures, a literature review was conducted to evaluate the potential value of such a measure. Third, a set of developmentally-informed principles were established to guide the adaptation of the START for adolescents. These principles emphasized the need for the adolescent version of the START to a) include developmentally-appropriate risk and protective factors, b) capture the multiple systems that adolescents are embedded within through the inclusion of family, peer, school, community, and individual factors, and c) ground risk assessments in an understanding of normative adolescent development.
Drawing from these principles, the START:AV added several items pertaining to family and peer systems (e.g., Parenting and Home Environment, Social Support from Caretakers and Other Adults, Relationships with Peers), and adjusted the item anchors and outcomes to take into account adolescents’ contexts and lesser maturity compared to adults. For instance, whereas the adult START includes an item on Occupational functioning, this item was revised for adolescents to focus on school as well as employment. Furthermore, the START:AV manual (Viljoen et al., in preparation) compiles adolescent-specific research and highlights developmental issues, thus aiming to facilitate a developmentally-informed assessment. For a more thorough discussion of the rationale and development of the START:AV see Viljoen et al, (this issue).
Research on the Adult START
Although the current study, along with Desmarais et al. (this issue), are the first studies to examine the psychometric properties of the START:AV, a number of studies have examined the adult START. Results on the adult START cannot be generalized to the START:AV, but speaks to the overall framework and may help identify critical research questions to be examined in studies on the START:AV. As such, we briefly summarize findings on the adult START below.
With respect to inter-rater reliability, results generally have found intraclass correlation coefficients (ICCs) to be in the “excellent” range for both Strength and Vulnerability total scores and final risk estimates on the adult version of the START (ICCs > .80s; Desmarais, Nicholls, Wilson, & Brink, 2012
; Nicholls, Brink, Desmarais, Webster, & Martin, 2006
; Wilson, Desmarais, Nicholls, & Brink, 2010
; but see Viljoen, Nicholls, Greaves, de Ruiter, & Brink, 2011
who found somewhat lower inter-rater reliability). Growing research support also has been obtained for the predictive validity of the adult version of the START. Studies have indicated that START final risk estimates and total scores can predict aggression towards others, including physical aggression and violence, verbal aggression, aggression towards objects, and sexually inappropriate behavior (Braithwaite, Charette, Crocker, & Reyes, 2010
; Chu, Thomas, Ogloff, & Daffern, 2011
; Desmarais et al., 2012
; Gray et al., 2011
; Nicholls et al., 2006
; Nonstad et al., 2010
; Wilson et al., 2010
Research on the ability of the adult version of the START to measure other outcomes of interest, such as self-harm and victimization, is more limited but still quite promising. Gray and colleagues (2011)
reported that START final risk estimates predicted self-harm, self-neglect, and victimization in psychiatric inpatients, although Strength and Vulnerability total scores did not. Braithwaite and colleagues (2010)
found that START Strength and Vulnerability total scores predicted aggression towards others, unauthorized leave, and substance abuse, but did not predict self-harm, suicidality, self-neglect, or victimization in a small sample of forensic inpatients (n
= 34). Thus, as research on the START:AV unfolds, it will be particularly important to examine the extent to which the START:AV predicts not only violence but also broader adverse outcomes, as well as its potential utility in non-forensic and community settings.
Given that the assessment of strengths is a major focus of the START, several studies have explored this aspect in greater detail. In user satisfaction surveys, clinicians reported that they found the adult START’s attention to both strengths and vulnerabilities to be clinically useful (Doyle, Lewis, & Brisbane, 2008
; Kroppan et al., 2011
). In addition, research has found Strength total scores to predict lower levels of aggression (Desmarais et al., 2012
; Wilson et al., 2010
) and more successful community reintegration (Viljoen et al., 2011
). Desmarais et al. (2012)
found that the Strength total scores were able to add incremental validity beyond that of the Vulnerabilities total scores in predicting aggression. In contrast, other studies have not obtained evidence for incremental validity of strength ratings, possibly as a result of the high inverse correlations between Strength and Vulnerability total scores (r > −.80; Braithwaite et al., 2010
; Viljoen et al., 2010; Wilson et al., 2010
). Therefore, as research on the START:AV develops, the role of strength factors requires careful attention.
Finally, several studies have examined the dynamic nature and the “shelf-life” of START assessments. Wilson et al. (2010)
found that START Strength and Vulnerability total scores were more effective in predicting risk over a short-term follow-up (i.e., 9 months or shorter) than a long-term follow-up (i.e., 12 months), supporting the START’s emphasis on regular reassessment. In addition, Nonstad et al. (2010)
found increases in mean strength scores, and decreases in mean vulnerability scores among forensic psychiatric patients during the course of treatment. Given that many view adolescents’ risks as particularly dynamic (Borum, 2003
; Grisso, 1998
; Prentky & Righthand, 2003
), it will be important for research to examine the ability of START:AV assessments to capture change.
The Present Study
Although studies with the adult version of the START support the reliability and validity of START assessments completed in both research and practice, these findings cannot be generalized to the newly-developed adolescent version of the measure, the START:AV. The current study was the first to examine the validity of the START:AV, and had five primary aims. First, we tested the reliability of the START:AV, including both inter-rater reliability as well as internal consistency. Second, to evaluate concurrent validity, we examined the association of START:AV assessments to other risk assessment tools and measures of protective factors (i.e., SAVRY, Borum et al., 2006
; Developmental Assets Profile, Search Institute, 2005). Third, we investigated the predictive validity of START:AV assessments, specifically their ability to predict multiple adverse outcomes (i.e., violence towards others, general reoffending, suicidal ideation, non-suicidal self-harm, victimization, and substance abuse) over a short-term (3-month) follow-up period. Fourth, we dedicated specific attention to strengths, including not only the ability of strengths to predict outcomes, but also correlations between strength and vulnerability ratings, the incremental validity of Strength total scores, and the extent to which risk level moderated the relationship between Strength total scores and outcomes (see Fergus & Zimmerman, 2003). Finally, as the START:AV was designed to be dynamic in nature, we investigated 3-month changes in the START:AV ratings through group-based statistics (e.g., t
-tests) as well as individually-oriented approaches (e.g., reliable change index; Jacobson & Truax, 1991