|Home | About | Journals | Submit | Contact Us | Français|
This study examines the predictive validity of sensation seeking as a predictor of adolescent substance use, in order to optimize targeting for substance use prevention programmes.
Random digit dial telephone survey.
6522 U.S. adolescents aged 10–14 years at baseline, resurveyed at 8-month intervals for 3 subsequent waves.
Two outcomes were assessed--onset of binge drinking (more than 5 drinks in a short time) and established smoking (>100 cigarettes lifetime). Sensation seeking level was assessed at baseline. Logistic regression was used to predict onset of substance use at any follow-up wave as a function of sensation seeking. The receiver operating characteristic curve was used to illustrate how well sensation seeking predicted substance use as a function of different cut-off points for defining high sensation seeking, and area under the receiver operating characteristic curve (AROC) was the metric of predictive validity.
Of 5854 participants with one or more follow up assessments, 5634 reported no binge drinking and 5802 were not established smokers at baseline, of whom 717 (12.7% of 5634) reported binge drinking and 144 (2.5% of 5802) reported established smoking at one or more follow up interviews. Sensation seeking predicted binge drinking moderately well (AROC = 0.71 [95% CI 0.69, 0.73]) and was a significantly better predictor of established smoking onset (AROC = 0.80 [0.76, 0.83)). For binge drinking, predictive validity was significantly lower in Blacks; for established smoking it was significantly higher for Hispanics. Implications for two targeting interventions are discussed.
Sensation seeking works moderately well at identifying adolescents at risk for onset of binge drinking and established smoking. This study offers a guide for determining the appropriate targeting cut-off value, based on intervention efficacy, costs and risks.
Sensation seeking is a personality variable that describes an individual’s tendency to seek out novel and exciting stimuli.(1, 2) Sensation seeking tendencies have been linked with substance use in adults some 30 years ago.(3–6) Since then, sensation seeking has been shown to be more strongly associated with substance use than many other personality measures.(7) One problem with the adult literature is the emphasis on cross-sectional analyses, which limits a causal interpretation. Among adolescents, however, sensation seeking has been found to be a longitudinal predictor in two structural models of substance use.(8–10) Two additional longitudinal studies of adolescents showed that sensation seeking at baseline predicted change in substance use over time.(11, 12) Finally, Crawford et.al.(13) used parallel process growth models in two samples to show that sensation seeking predicted baseline level and growth of marijuana use but found less evidence of a relation with alcohol use.
The robust associations between sensation seeking and adolescent substance use have prompted some to promote sensation seeking as a screening test for those at risk. One group proposes tailoring anti-drug use messages to adolescents with high sensation seeking propensity,(14, 15) having shown that sensation seekers are more drawn to messages that are novel, complex; intensely stimulating and arousing.(16) In a strategy they call Sensation Seeking Targeting (SENTAR), SENTAR researchers have used these insights to design interpersonal prevention messages aimed at sensation seekers.(14) One community trial suggested that messages targeted at high sensation seekers could reduce marijuana use in that population.(17) Sensation seeking targeting approaches have also been employed in large anti-drug campaigns, including the National Youth Anti- Drug Media(18–20) and the truth® campaigns.(21) Results were mixed for the National Youth Anti-Drug Media Campaign. Studying the effect of the program overall, Hornick et.al.(18, 19) found no effect, even among high sensation seekers; they also identified possible unfavorable (boomerang) effects of the ad campaign. In contrast, Palmgreen(20) found that the Marijuana Initiative, implemented late in the campaign, reversed upward developmental trends in 30-day marijuana use among high-sensation-seeking adolescents in two communities. The impact of truth® has not been evaluated in the context of high vs. low sensation seeking.(21)
Another research group has proposed using sensation seeking to identify adolescents for Personality-Targeted Interventions (PTI).(22, 23) Conrod and colleagues used assessments of sensation seeking, negative thinking, anxiety sensitivity and impulsivity to identify adolescents at higher risk for alcohol use. For each group, PTI researchers deployed tailored interventions designed to address the identified risk factor. In two randomized trials, they found reductions in binge drinking among sensation seekers that were attributable to the intervention; results were mixed for the other personality assessments. These studies suggest a role for sensation seeking in targeting adolescents for substance use interventions.
Targeting involves choosing a cut-off point (for a continuous risk variable) that identifies a high-risk segment of the population for intervention. Targeting has potential drawbacks that are well described in medical screening and prevention literature;(24) however, these issues are not often addressed in the substance use literature, in part because of the tendency to focus on correlation among continuous measures of risk and substance use. From a population standpoint, targeting high sensation seekers will be an effective prevention strategy only if the screening test defining high sensation seeking adequately identifies a large proportion of eventual substance users. SENTAR researchers use a median split on the sensation seeking scale to identify high sensation seeking youth. PTI researchers use a more restrictive cut-off point, one standard deviation above the mean, which would identify approximately 16% of the sample as at risk on the basis of sensation seeking (assuming a normal distribution). In this study, we assess sensation seeking as a longitudinal predictor of binge drinking and established smoking. We assess predictive validity as a function of different cut points for defining high sensation seekers. Our aim is to determine how well current approaches for classifying high sensation seeking (median split and one standard deviation above the mean) separates eventual substance users from nonusers.
A detailed description of the recruitment methods for study participants has been published previously.(25) Briefly, between June and October, 2003, we conducted a random-digit dial (RDD) telephone survey of 6522 U.S. adolescents aged 10–14 years, designed to capture a representative sample of U.S. adolescents. We obtained parental consent and adolescent assent prior to interviewing each respondent, with a Certificate of Confidentiality protecting information on risk behaviors. All aspects of the survey were approved by the institutional review boards at Dartmouth Medical School and the survey research firm (Westat, Rockville MD). The CASRO response rate for the survey was 32% (which includes estimates of the number of adolescents lost because the household did not answer the phone) and the completion rate was 66%, rates comparable to other contemporary random digit dial surveys that involve consent of the parent and assent of the adolescent.(26) A recent publication by Beiner et al,(26) examined the impact of declining response rates in telephone surveys of smoking. They found that under- and over-representation of population subgroups has not changed and also found no evidence that declining response rates have resulted in biased estimates of smoking behavior.
We compared the distributions of age, sex, and census region in our unweighted sample with those of the 2000 U.S. Census,(25) confirming there was little evidence of under- or over-representation of demographic subgroups in this study and consistent with the results of Biener et al. The 6522 adolescents at baseline were equally distributed by age (between 10 and 14 years) and sex, and represented every U.S. state. Some 62 percent of adolescents described themselves as white, 13.6% as Black, and 18.2% as Hispanic. Seventy four percent of parents were homeowners, and family income (for the 6083 parents who responded) was distributed as follows: 7.7% <=$10,000, 10.4% $10001–20,000, 12.1% $20001–30,000, 20.9% $30001–50,000, 19.3% $50001–75,000, 29.6% >$75,000; these income groupings were almost identical to U.S. 2000 census data on median family income for families with adolescents in this age range.
Three follow-up surveys were conducted at 8-month intervals with 84% (n=5503) retained at 8 months (8M), 77% (n=5019) at 16 months (16M), and 71% (n=4575) at 24 months (24M). Adolescents lost to follow-up were more likely to be of non-white ethnicity (Black, Hispanic, or other) and older age, to have parents with lower educational attainment, and to be higher in sensation seeking. Established smoking status and binge drinking status at baseline did not predict attrition at wave 4. Subjects were included in the analysis if they had data for sensation seeking at baseline and substance use at one or more follow up surveys.
Adolescents were surveyed regarding their use of alcohol and tobacco. Binge drinking was ascertained through the question, “"Have you ever had 5 or more drinks of alcohol in a row, that is, within a couple of hours?" (yes, no). Assessment of established smoking was based on a query about lifetime smoking, "How many cigarettes have you smoked in your life?" (none, a few puffs, 1 to 19 cigarettes, 20 to 100 cigarettes, or more than 100 cigarettes). Any adolescent who stated smoking more than 100 cigarettes was defined as an established smoker, consistent with surveys of adult samples and other studies of adolescents.(26) A dichotomous variable was developed to model onset of each substance use outcome, starting with status at 24M. For any adolescent not in the 24M survey, 16M or 8M substance use status was used if available; for adolescents with multiple assessments, any positive response was taken as an indication of substance use. Thus, both outcome variables were composed of data for 5834 adolescents who had responses for one or more follow up surveys (some adolescents missing from wave 2 were recaptured at waves 3 or 4).
We developed a 4-item sensation seeking scale for younger children, tapping two of four constructs identified by Zuckerman as important components of sensation seeking:(27) thrill/adventure seeking (“I like to do scary things” and “I like to do dangerous things”) and boredom susceptibility (“I often think there is nothing to do”). We also included intensity seeking (“I like to listen to loud music”), a component of the Arnett Inventory of Sensation Seeking (AISS) that was more highly correlated with risk behaviors than novelty seeking.(28) Response categories included “not like you”, “a little like you”, “a lot like you” and “just like you”, with scores for the individual items ranging from 1 to 4; these scores were summed across items (range 4–16). The 4 item scale had alphas of .60, .58, .64, and .63 and average inter-item correlations of .27, .26, .31, and .30 at waves 1 to 4, respectively.
Regarding the normative stability of the measure, the correlation between sensation seeking at baseline and subsequent waves ranged from 0.54 (baseline-wave 4) to 0.73 (wave 3 - wave 4), similar to the ranges for stability of the subscales for the SSSV in a sample of Norwegian adolescents over 20 months (where correlations ranged from 0.59 to 0.74).(11) The means (standard deviation) for sensation seeking for the 4081 adolescents with data from all four waves were 7.8 (2.4), 8.0 (2.4), 8.2 (2.5), and 8.4 (2.5) for baseline, wave 2, wave 3 and wave 4 respectively. A longitudinal generalized least squares random effects regression model showed that the average mean increase of 0.2 was statistically significant (z = 21, p < 0.0001), confirming previous observations on the growth of sensation seeking during adolescence.(13)
The reliabilities for this scale compare well with another 4-item measure of sensation seeking, the Brief Sensation Seeking Scale (BSS4). Stephenson et.al.(29) reported a reliability of 0.66 and average inter-item correlation of 0.33 for the BSSS4. We chose this measure over the BSSS4 because we felt the language in some items was too advanced (e.g., “I prefer friends who are exciting and unpredictable”) for the reading ability of our younger participants (10–11 years). Furthermore, we felt that another two-item scale (developed by Slater) with higher reliability(29) was too narrow because it assessed only the thrill seeking and not other sensation seeking domains.
With regard to correlation with substance use, the cross-sectional correlation between our sensation seeking measure at wave 3 (the first wave in which we assessed alcohol frequency and quantity) was 0.25 for binge drinking, 0.18 for alcohol frequency, and 0.24 for alcohol quantity, comparable with correlations between these substances (0.25, 0.25, 0.26 respectively) and the 20-item AISS in a sample of Australian high school students.(30) Regarding multivariate predictive validity, longitudinal studies of adolescents’ media use have found this sensation seeking measure to be a robust predictor of smoking(31–35) and alcohol use,(36, 37) independent of a number of other confounding social, parenting and personality influences.
Means for sensation seeking were compared across demographic subgroups using multivariate logistic regression; in these regressions, sensation seeking was log transformed to reduce left skewness. Signal detection theory was used as the basis for the analysis of predictive validity.(38, 39) Receiver operating characteristics (ROC) curves were used to examine the effectiveness of sensation seeking as a predictor of substance use onset. In medicine, ROC curves are commonly used to study the ability of laboratory tests to discriminate between disease and non-disease(40) or, for screening/targeting, to discriminate between those at risk for or not at risk for a disease.(41) Key constructs for evaluating a screening test for substance use are shown in Table 1, based on the relationship between the targeting test results and whether or not the person goes on to use substances. True positives (TP) are substance users correctly identified by the test; false positives (FP) are persons identified as at risk who do not go on to use substances; false negatives (FN) are persons not identified by the test who go on to use substances; and true negatives (TN) are persons the test correctly identifies as not at-risk for substance use. The sensitivity of the test is a measure of how well the test correctly identifies true positives, and the false positive rate is a measure of how often the test mistakenly identifies someone as at risk.
Because sensation seeking is a continuous variable, there are a number of cut-off points one could use to target “high” sensation seekers. The ROC curve is a plot of the sensitivity of the test against the false positive rate for each possible cut-off point (see Figure 1 as an example). It illustrates the trade off between more inclusive cut points--which tend to maximize sensitivity but also result in high false positive rates, and more restrictive cut points--which minimize false positive rates but may result in low sensitivity. The area under the ROC curve (AROC) is a measure of the predictive validity of the test; the perfect test has one cut point, a sensitivity of 100%, a false positive rate of 0% and an AROC of 1. A test with no predictive validity has an AROC of 0.5 and a linear ROC curve that parallels the diagonal. ROC curves were fit after logistic regression using STATA 9 (College Station, TX) and compared for demographic subgroups using nonparametric methods.(42) In each regression, the baseline sample consisted of non substance users (232 baseline binge drinkers were removed from the binge drinker predictive model, and 34 baseline established smokers were removed from the established smoker predictive model).
Table 2 shows the distribution for sensation seeking among the 6521 adolescents who completed the baseline assessment (one subject who responded to only one item was omitted), overall and by demographics.
At baseline, the scale values ranged from 4 to 16, with a mean 8.0 (std dev 2.5). The distribution was skewed left, such that one reached the median value at 7.0, where a cut-off level would identify 52% of the population as at risk. Cut off levels of 9.0 and 11.0 would respectively identify 25% or 9% of the population as at risk. With regard to mean sensation seeking scores by demographics, multivariate analysis (with log-transformed sensation seeking) found that baseline sensation seeking means were significantly lower by 2% (95% confidence interval 0.3%, 4%) in Hispanics and by 6% (5%, 7%) in females compared with males. Compared with 10 year olds, sensation seeking was higher by 8% (5%, 10%), 14% (12%, 17%), and 19% (17%, 22%) in 12, 13, and 14 years of age, respectively. These differences are apparent when one examines the distributions in Table 2. For example, whereas a sensation seeking score of 7.0 was the 60th percentile at 10 years, it was only the 33rd percentile at 14 years. There were no other statistically significant comparisons of means.
Of 5834 participants with one or more follow up assessments of substance use, 5634 reported no binge drinking at baseline, of whom 717 (12.7%) reported binge drinking at one or more follow up interviews. Figure 1 shows the relation between level of sensation seeking at baseline, and sensitivity/false positive rate for predicting binge drinking in subsequent surveys. The area under the ROC curve was 0.71 (95% confidence interval 0.69, 0.73), suggesting that sensation seeking at baseline was a moderately good predictor of binge drinking in the future. At the relatively restrictive cut-off point employed by PTI researchers (one standard deviation above the mean), one would identify 16% of the population for the intervention, and this targeted sample would include about one-third of future binge drinkers (sensitivity = 31%) and have a false positive rate of 12%. The median split cut-off point employed by SENTAR researchers would identify half the population for intervention, capturing 77% of future binge drinkers at the cost of a 46% false positive rate. At higher cut-off levels, there was only a small gain in sensitivity at the expense of large increases of the false positive rate.
Table 3 compares the predictive validity of sensation seeking for binge drinking across demographic variables. There were no differences by gender. Sensation seeking was a significantly worse predictor of binge drinking among Black vs. other adolescents (AROC 0.65 and 0.72 respectively), but not among Hispanics. Sensation seeking was also a poorer predictor at extremes of the age bracket.
Of 5834 participants with one or more follow up assessments of substance use, 5802 were not established smokers at baseline, of whom 144 (2.5%) reported established smoking at one or more follow up interviews. Thus, established smoking was a much less common substance use outcome than binge drinking. The area under the ROC curve was 0.80 (95% confidence interval 0.76, 0.83), indicating that sensation seeking at baseline was a better predictor of established smoking than binge drinking in this sample, since the 95% confidence intervals did not overlap. At the relatively restrictive cut-off point employed by PTI researchers, one would identify 16% of the population for the intervention, and this targeted sample would include about half of future established smokers (sensitivity = 53%) and have a false positive rate of 14%. The median split cut-off point employed by SENTAR researchers would identify half the population for intervention, capture 92% of future established smokers, with a higher false positive rate of 50%. At higher cut-off levels, there was only a small gain in sensitivity at the expense of large increases of the false positive rate.
Table 4 compares the predictive validity of sensation seeking for established smoking across demographic variables. There were no differences by gender or age. Sensation seeking was a significantly better predictor among Hispanic vs. other adolescents (AROC 0.87 and 0.79 respectively), and predictive validity was no worse among Blacks compared with other adolescents.
This study addresses the appropriateness of sensation seeking as a risk factor to screen adolescents for binge drinking and smoking onset. To our knowledge, this is a question that has not been addressed, despite the existence of programs that have tailored interventions for high sensation seeking youth. The four-item assessment of sensation seeking used in this study was found to be moderately predictive of binge drinking and more strongly predictive of established smoking.
The ROC analysis identified trade offs inherent in two targeting strategies. SENTAR researchers employ a mass media intervention designed to appeal to adolescents who score above the median on sensation seeking. This strategy, if employed on a sample of non-substance using young adolescents, would be expected to identify three-quarters of future binge drinkers and almost all future established smokers. Thus it offers a very acceptable way to target these individuals, capturing the majority of future substance users. For a mass media intervention, the cost of intervening in a false positive individual is minimal, so there is little disadvantage to false positive rates that approach half of the population. The more restrictive cut-off point used by PTI researchers will identify only one third of future binge drinkers and half of future established smokers. The disadvantage of this approach is that it will miss the majority of future binge drinkers and half of future established smokers. However, their more stringent cut-off point is appropriate because their one-on-one intervention has clear disadvantages if aimed at a false positive individual. First, the intervention diverts the individual from other activities, both scholarly and recreational. Secondly, the intervention diverts teacher resources from these same activities and is associated with costs to the educational institution. Finally, with any more intensive intervention, the possibility of a boomerang effect in a minority of reactive students has to be considered, as it has been documented as a problem that undercuts the effects of some health education programs.(43) A boomerang effect is less likely in the case of the PTI intervention, since it does not address substance use directly. Given the real costs of the more intensive intervention, the cut-off point chosen by PTI researchers seems reasonable.
This study also offers an assessment of the reliability and predictive validity of this sensation seeking measure in demographic subgroups. Valone et al.(44) have questioned the validity of sensation seeking among Black adolescents because they found poorer reliability of the measure in that subgroup; they also found lower means among Black adolescents. The reliability of sensation seeking was lower also among Black adolescents in the present sample (alphas for waves 1–4 = 0.51, 0.48, 0.56, 0.57), consistent with Vallone, but we did not find lower mean scores among Blacks. Instead, we found slightly lower scores among Hispanics and females. Lower reliability may be why the present study finds that sensation seeking had poorer predictive validity for established smoking among Blacks. Notably, it seemed to work well in predicting binge drinking, so low reliability is not always a reason to doubt the predictive validity of sensation seeking among Blacks.
This study confirms findings of other studies(13) by offering cross-sectional and longitudinal support for the notion that sensation seeking increases during adolescence. An important question for substance use researchers is whether environmental factors affect substance use by increasing sensation seeking among some adolescents.(45) Although research on sensation seeking has emphasized biological/genetic determiners and neurotransmitter systems,(27) evidence of important environmental effects is emerging. In one longitudinal study, Stacy found specific paths from family and social support to changes in experience seeking and from depression and exposure to peer violence to thrill and adventure seeking.(46) Additionally, R-rated movie viewing has been shown to affect short and long term growth in sensation seeking in the present longitudinal sample of adolescents.(47) Such research raises the intriguing question of whether interventions to prevent certain environmental exposures could affect the degree to which sensation seeking increases during adolescence.
There are also some limitations to be discussed. Loss to follow up was related to sensation seeking at baseline, with adolescents higher in sensation seeking more likely to be lost to follow up. The difference was small, with completion rates around 80% for low sensation seekers and 70% for high sensation seekers. This could affect results if high sensation seekers lost to follow up were markedly different from those retained in terms of their substance use. Larger differences in attrition were found across race and ethnicity, with follow up rates much better for whites (82%) compared with Blacks (64%) and Hispanics (71%). This could affect our conclusions about the difference in predictive validity by race/ethnicity.
Another limitation is that this sensation seeking measure does not capture the construct of sensation seeking as reliably as longer measures, such as the Zuckerman’s Sensation Seeking Scale Five (SSS-V) or the AISS. However, longer measures would be impractical in a setting (such as a pediatric clinic) where one might choose to target high sensation seekers for an intervention. The scale reliability of this sensation seeking measure is comparable to another commonly used 4-item measure (BSSS 4), and even some of the subscales of the SSS-V. For example, the alpha ranges were 0.61–0.67 and 0.56–0.65 among adults for the experience seeking and boredom susceptibility SSS-V subscales respectively in adult samples for Zuckerman’s Form V questionnaire.(27) Further research could examine how scale reliability affects the predictive validity of a targeting measure.
In summary, within this broad sample of US adolescents, sensation seeking was found to be a moderately strong predictor of binge drinking and a strong predictor of established smoking. Given the interest in developing ways to target high risk groups of adolescents for substance use interventions, these data may be used to determine an appropriate cut-off level to identify high sensation seekers, as appropriate to the character of the intervention. We suggest that the predictive validity of targeting risk factors be assessed in longitudinal samples before they are employed as targeting variables, in order to assess overall predictive validity and determine the appropriate cut-off value. Signal detection theory with ROC curve analysis is a useful way to examine overall predictive validity and determine the screening cut point for any continuous candidate variable.
This work was supported by the National Institutes of Health (CA77026, and AA015591), and The American Legacy Foundation.