Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Subst Abus. Author manuscript; available in PMC 2017 September 22.
Published in final edited form as:
PMCID: PMC5522769

Development of an Opioid-Related Overdose Risk Behavior Scale (ORBS)



Drug overdose has emerged as the leading cause of injury-related death in the U.S., driven by prescription opioid (PO) misuse, polysubstance use and use of heroin. To better understand opioid-related overdose risks that may change over time and across populations there is a need for a more comprehensive assessment of related risk behaviors. We developed the opioid-related Overdose Risk Behavior Scale (ORBS), drawing on existing research, formative interviews, and discussions with community and scientific advisors.


We enrolled military veterans reporting any use of heroin or POs in the past month using venue-based and chain referral recruitment. The final scale consisted of 25 items grouped into 5 subscales eliciting the number of days in the past 30 during which the participant engaged in each behavior. We assessed internal reliability using Cronbach’s alpha; test-retest reliability using intraclass correlation coefficients (ICCs); and criterion validity using Pearson’s correlations with indicators of having overdosed during the past 30 days.


Data for 220 veterans were analyzed. The 5 subscales---A. Adherence to Opioid Dosage and Therapeutic Purposes, B. Alternative Methods of Opioid Administration, C. Solitary Opioid Use; D. Use of Non-prescribed Overdose-associated Drugs and E. Concurrent Use of POs, Other Psychoactive Drugs and Alcohol---generally showed good internal reliability (alpha range = 0.61 to 0.88), test-retest reliability (ICC range = 0.81 to 0.90), and criterion validity (r range = 0.22 to 0.66). The subscales were internally consistent with each other (alpha = 0.84). The scale mean had an ICC value of 0.99, and correlations with validators ranged from 0.44 to 0.56.


These results constitute preliminary evidence for the reliability and validity of the new scale. If further validated, it could help improve overdose prevention and response research, and could help improve the precision of overdose education and prevention efforts.

Keywords: Opioids, overdose, drug use, risk behavior


Drug overdose has emerged as the leading cause of injury-related death in the U.S., driven by prescription opioid (PO) misuse, polysubstance use and use of heroin.16 For overdose prevention and response research a broad assessment capable of capturing behavioral risks in populations with varying substance choices and use patterns is critically important, particularly as we seek to understand the precipitants of changes in overdose risk behaviors among at-risk populations.

Though no comprehensive overdose risk behavior scale exists, a number of clinical screening tools assess some aspects of the potential for PO misuse. For example, the Current Opioid Misuse Measure (COMM), the Screener and Opioid Assessment for Patients with Pain (SOAPP), and the Opioid-Related Behaviors in Treatment (ORBIT) scale were designed to help clinicians assess PO misuse risks among pain patients.711 Reflecting the purpose of clinical screening, these scales include items on drug-seeking behaviors, past history of substance use, and mood states presumed to be related to PO misuse,7,1017 sometimes incorporating criteria to suggest when a clinician should consider modifying (or denying) PO treatment.9,1215,18 However, since these scales were not designed to assess overdose risk they lack items regarding the contraindicated use of opioids (including heroin, methadone and buprenorphine), other drugs and alcohol, and items regarding drug use behaviors that can increase risk, such as injecting opioids, or using opioids alone.1921

To better understand the factors that cause opioid-related overdose a first step is to comprehensively assess overdose risk behaviors, and test their associations with overdose events. One group with elevated overdose risks related to PO and heroin use is military veterans.22,23 Below, we describe the development of an opioid-related Overdose Risk Behavior Scale (ORBS) and present results of preliminary reliability and validity analyses in a community-based sample of US military veterans.


We enrolled 220 U.S. military veterans who reported any heroin or PO use (within past 30-days) at the time of enrollment between August 2014 and June 2016 as part of an ongoing study of opioid misuse and overdose risks.23,24 We recruited participants in veterans’ service agencies, homeless shelters and opioid treatment programs throughout New York City using venue-based and chain referral methods. Enrolled participants completed survey-based assessments of their military service, alcohol, PO and other drug use, biological, psychological, social contextual factors and major life events.

Our aim was to construct a scale to assess known opioid-related overdose risk behaviors. We adapted 4 items from the COMM [“How often have you…taken your medications differently from how they are prescribed? …needed to take pain medications belonging to someone else? …had to take more of your medication than prescribed? …used your pain medicine for symptoms other than for pain?”] regarding POs that were prescribed by the patients’ own doctor.25 Additional items were developed using a literature review,1921 discussions with members of the study’s community advisory board and scientific advisory board, and results of qualitative interviews with 50 veterans who reported experiencing at least one opioid-related overdose at some point in the past.26 Although, theoretically, there was no strong rationale for assuming that these items would be correlated, items were naturally categorized into subscale domains that we hypothesized would be internally consistent. We used the number of days during the past 30 days as the response units for all items. Thus, responses are ratio scaled.27

We pilot tested a preliminary version of the scale with 15 participants who were then interviewed about clarity, comprehensiveness and tone. After this process, the research team reconvened and eliminated 4 items (using borrowed, purchased or stolen POs, and using POs at different intervals than prescribed) because they were not understood consistently and overlapped with other items, and revised item wording for clarity and brevity.

After eligibility screening and informed consent procedures, participants completed a survey assessment using face-to-face methods with a trained and experienced interviewer. Participants provided informed consent, and received $20 for completing the assessment, which also included investigational domains hypothesized to precipitate change in overdose risk behaviors, including mood, physical pain and life events. All procedures and protocols were approved by the Institutional Review Board of the first author. Administration duration of the new scale was typically 5–10 minutes. Only the initial assessment from each participant was analyzed (with the exception of test-retest data). We used IBM SPSS (ver. 22) for analyses. We assessed reliability and validity of the total scale and of subscales. We report scale scores as an unweighted average of constituent items. Thus, the subscale and total scale means reflect the same 0–30 units as the constituent items.

We asked a subset of 38 sequential participants to repeat the assessment the next day, preferably at the same time of day; 34 of these provided retest data within the time frame. Participants were compensated an additional $20 for retests. We compared their responses using intraclass correlation coefficients (ICCs) with a one-way random effects formulation, ICC(1,1).28 We also compared mean differences in test and retest assessments using t-tests for paired samples. To determine whether the participants who provided test-retest reliability data differed significantly from the rest of the sample we compared their characteristics using Student’s t-tests.

We assessed criterion validity retrospectively using Pearson’s correlations with two indicators of having experienced an overdose during the same past-30-day time period. We asked: “On how many days in the past 30 did you lose consciousness or pass out so that you could not really wake up or others could not wake you up?” and “On how many days in the past 30 did you or someone else call for medical assistance because of how sedated, drugged, or high you were after using opioids?”

We examined Pearson’s correlations among the subscales to assess how they were related.


Participant characteristics are shown in Table 1. Most participants were male, never married and unemployed, despite relatively high educational achievement. The majority were black/African American, and almost a quarter were Hispanic/Latino. Ages ranged from 21 to 60, averaging 36.9 years (standard deviation [SD] = 9.5). Almost half the sample reported being homeless or unstably housed at the time of enrollment. Heroin use (during the last 30 days) was reported by 27.3%, and PO use (including those who had been prescribed POs) was reported by 89.1%.

Participant characteristics. N = 220 veterans currently using opioids (past 30-days)

Table 2 lists ORBS items, grouped by subscales. All or some items in 3 of the 5 scales (A, B, and C) are only applicable to participants who reported a current PO prescription. Therefore, the sample size for those scales is restricted to the 104 (47%) participants who met this criterion. Mean values represent the average number of days during the past 30 in which participants engaged in each behavior. As shown in the interquartile range column, many items were infrequently endorsed, with 25th percentile, and sometimes 75th percentile values of 0. The most frequent risk behavior was A.4. Use of POs to aid sleeping, averaging 6.0 days (SD = 9.2). Mean values of items regarding prescribed opioids were generally higher than those for non-prescribed drugs.

Scale items; Cronbach’s alpha values; means, standard deviations and interquartile ranges; item-total correlations; alpha values if item deleted; intraclass correlations; and correlations with overdose criterion validators (N = 220)

There were no significant differences in participant characteristics between participants who provided test-retest data and those who did not, and there were no significant differences in mean values of ORBS items or scales between test and retest assessments.

Subscales A, B and C good internal consistency (alpha range = 0.68 to 0.88), test-retest reliability (ICC range = 0.47 to 0.85), and validity (r with criterion variables range = 0.38 to 0.66). (As shown in Table 2, all correlations of the total ORBS and of ORBS subscales with overdose event criterion variables were significant at the p < 0.05 level.)

Subscale D showed marginally acceptable internal consistency (alpha = 0.61), and good retest reliability (ICC = 0.84). Correlations with validators were moderate (r = 0.22, r = 0.47). We considered dropping the non-therapeutic use of methadone from the subscale because few participants (6.4%) endorsed this item. However, the item had a moderate item-total correlation (0.33), which suggested it was worth retaining. We also asked whether participants used non-prescribed buprenorphine. However, this item was the least frequently endorsed, and had a very low item-total correlation (0.08), so it was not retained.

Subscale E showed good internal consistency (alpha = 0.77) and retest reliability (ICC = 0.90). Some items were infrequently endorsed, but are retained to be consistent with our purpose of developing a more comprehensive assessment of opioid-related risk behaviors than is currently available.29 For example, as shown in Table 2, the mean for E.3. Using buprenorphine and any other POs on the same day was 0.3 (4.1% of participants reported using buprenorphine and any other opioids on the same day once or more during the past 30 days). Correlations with validators were moderate (r = 0.38, r = 0.49).

Averaging the subscales yields a mean of 2.6 (SD = 3.7). This mean was correlated moderately to strongly with criterion validators (r = 0.44, r = 0.56). The alpha value for the Cronbach’s analysis of the 5 ORBS subscale means was 0.84.

Correlations among ORBS subscale means are in Table 3. Correlations ranged from 0.45 to 0.72.

Correlations among subscales.


Our findings provide preliminary evidence that engagement in opioid-related overdose risk behaviors can be assessed comprehensively using a brief scale. Five subscales representing engagement in opioid using behaviors that can increase overdose risk were each, and as an aggregate, correlated with indicators of an opioid overdose event. These subscales included behaviors specific to PO misuse among patients, and behaviors that can apply to heroin use or PO misuse among patients and non-patients, such as solitary use and concurrent use of other drugs or alcohol. As the opioid overdose epidemic continues to worsen, it is all the more important to comprehensively assess risks related to heroin and polysubstance use.30 Though it was designed for research purposes the ORBS also has the potential to be useful in therapeutic contexts where counselors, psychiatrists and social workers stand to aid individuals in developing strategies to minimize engagement in such behaviors. This value may also be realized in community-based settings wherein overdose education and prevention programs might be bolstered by more specific data about the nature and frequency of engagement in overdose risk behaviors.

There are several important limitations to the interpretation of these results, the first of which applies to all research using self-report. We did not collect or analyze biological data, leading to a potential underreporting of opioid use severity due to social desirability bias. Two subscales (C and D) with relatively weak internal reliability results retain moderate correlations with criterion validity indicators, which may suggest that the lower reliability resulted from infrequent responses for some items. For example, items regarding smoking or injecting POs were rarely endorsed, despite being associated with increased risk.31,32 We retain these items as they may represent important risks in other populations, especially considering the evolving nature of the opioid overdose epidemic. Our limitations with regard to sample size and a cross-sectional ascertainment provide guidelines for future research; for example, broader investigations of at-risk cohorts in addition to opioid-using veterans could evaluate generalizability, and a prospective-longitudinal study could evaluate predictive validity.

In conclusion, results from this study show preliminary evidence that a comprehensive set of opioid-related overdose risk behaviors can be assessed reliably using a brief scale. The ORBS’ focus on overdose-associated polysubstance use, modes of administration, and social contexts for use may prove to be useful for better understanding overdose risks that can change over time or may differ across populations.3337 It has the potential to help to improve the precision of overdose prevention and response research, and overdose education and prevention efforts.



This research was supported by National Institute on Drug Abuse grant R01DA036754 (PI: A. Bennett); Career Development Award from the Department of Veterans Affairs Office of Research Development, Clinical Science Research and Development (CSR&D) IK2CX000641 (PI: P.C. Britton); and institutional training award T32DA007233 (B. Wolfson-Stofko). The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.


The authors declare they have no conflicts of interest.


ERP performed the statistical analyses, wrote the initial draft and contributed to the development of the scale; ASB, was responsible for the initial conception of the study and was the Principal Investigator and contributed to the writing and scale development, LE was the Project Director and contributed to the writing and scale development; BW-S contributed to the writing; RA, PCB and AR contributed to the writing and scale development.


1. Johnson NB, Hayes LD, Brown K, Hoo EC, Ethier KA. CDC National Health Report: leading causes of morbidity and mortality and associated behavioral risk and protective factors--United States, 2005–2013. Morbidity and mortality weekly report Surveillance summaries (Washington, DC :2002) 2014;63(Suppl 4):3–27. [PubMed]
2. Calcaterra S, Glanz J, Binswanger IA. National trends in pharmaceutical opioid related overdose deaths compared to other substance related overdose deaths: 199992013;2009. Drug Alcohol Depend. 2013;131(3):263–270. [PMC free article] [PubMed]
3. Lankenau SE, Teti M, Silva K, Bloom JJ, Harocopos A, Treese M. Patterns of prescription drug misuse among young injection drug users. J Urban Health. 2012;89(6):1004–1016. [PMC free article] [PubMed]
4. Chen LHH, Warner M. QuickStats: Rates of Deaths from Drug Poisoning and Drug Poisoning Involving Opioid Analgesics, United States, 1999–2013. Morbidity and Mortality Weekly Report. 2015;64(32)
5. Cicero TJ, Ellis MS, Surratt HL, Kurtz SP. The changing face of heroin use in the United States: a retrospective analysis of the past 50 years. JAMA Psychiatry. 2014;71(7):821–826. [PubMed]
6. Courtwright DT. Preventing and Treating Narcotic Addiction--Century of Federal Drug Control. N Engl J Med. 2015;373(22):2095–2097. [PubMed]
7. Butler SF, Budman SH, Fernandez KC, et al. Development and validation of the Current Opioid Misuse Measure. Pain. 2007;130(1–2):144–156. [PMC free article] [PubMed]
8. Finkelman MD, Kulich RJ, Zoukhri D, Smits N, Butler SF. Shortening the Current Opioid Misuse Measure via computer-based testing: a retrospective proof-of-concept study. BMC Med Res Methodol. 2013;13:126. [PMC free article] [PubMed]
9. Butler SF, Fernandez K, Benoit C, Budman SH, Jamison RN. Validation of the revised Screener and Opioid Assessment for Patients with Pain (SOAPP-R) J Pain. 2008;9(4):360–372. [PMC free article] [PubMed]
10. Akbik H, Butler SF, Budman SH, Fernandez K, Katz NP, Jamison RN. Validation and clinical application of the Screener and Opioid Assessment for Patients with Pain (SOAPP) J Pain Symptom Manage. 2006;32(3):287–293. [PubMed]
11. Larance B, Bruno R, Lintzeris N, et al. Development of a brief tool for monitoring aberrant behaviours among patients receiving long-term opioid therapy: The Opioid-Related Behaviours In Treatment (ORBIT) scale. Drug Alcohol Depend. 2015 [PubMed]
12. Banta-Green CJ, Merrill JO, Doyle SR, Boudreau DM, Calsyn DA. Measurement of opioid problems among chronic pain patients in a general medical population. Drug Alcohol Depend. 2009;104(1–2):43–49. [PMC free article] [PubMed]
13. Adams LL, Gatchel RJ, Robinson RC, et al. Development of a self-report screening instrument for assessing potential opioid medication misuse in chronic pain patients. Journal of pain and symptom management. 2004;27(5):440–459. [PubMed]
14. Jamison RN, Martel MO, Edwards RR, Qian J, Sheehan KA, Ross EL. Validation of a brief Opioid Compliance Checklist for patients with chronic pain. J Pain. 2014;15(11):1092–1101. [PMC free article] [PubMed]
15. Compton PA, Wu SM, Schieffer B, Pham Q, Naliboff BD. Introduction of a self-report version of the Prescription Drug Use Questionnaire and relationship to medication agreement noncompliance. J Pain Symptom Manage. 2008;36(4):383–395. [PMC free article] [PubMed]
16. Webster LR, Webster RM. Predicting aberrant behaviors in opioid-treated patients: preliminary validation of the Opioid Risk Tool. Pain Med. 2005;6(6):432–442. [PubMed]
17. Friedman R, Li V, Mehrotra D. Treating pain patients at risk: evaluation of a screening tool in opioid-treated pain patients with and without addiction. Pain Med. 2003;4(2):182–185. [PubMed]
18. Chabal C, Erjavec MK, Jacobson L, Mariano A, Chaney E. Prescription opiate abuse in chronic pain patients: clinical criteria, incidence, and predictors. Clin J Pain. 1997;13(2):150–155. [PubMed]
19. Wright N, Oldham N, Jones L. Exploring the relationship between homelessness and risk factors for heroin-related death--a qualitative study. Drug and alcohol review. 2005;24(3):245–251. [PubMed]
20. Binswanger IA, Nowels C, Corsi KF, et al. Return to drug use and overdose after release from prison: a qualitative study of risk and protective factors. Addiction Science & Clinical Practice. 2012;7(1):3–3. [PMC free article] [PubMed]
21. Williams AV, Strang J, Marsden J. Development of Opioid Overdose Knowledge (OOKS) and Attitudes (OOAS) Scales for take-home naloxone training evaluation. Drug Alcohol Depend. 2013;132(1–2):383–386. [PubMed]
22. Copeland LA, Finley EP, Bollinger MJ, Amuan ME, Pugh MJ. Comorbidity Correlates of Death Among New Veterans of Iraq and Afghanistan Deployment. Medical care. 2016 [PubMed]
23. Bennett ASEL, Golub A. Veterans’ health and opioid safety–contexts, risks, and outreach implications. Federal Practitioner. 2015;32(6):4–7.
24. Bennett AS, Elliott L, Golub A, Pouget ER, Rosenblum A. Opioid use and overdose risk among OEF/OIF/OND era veterans: a mixed methods analysis. Drug and Alcohol Dependence. 2015;158:e18.
25. Butler SF, Budman SH, Fanciullo GJ, Jamison RN. Cross validation of the current opioid misuse measure to monitor chronic pain patients on opioid therapy. Clin J Pain. 2010;26(9):770–776. [PMC free article] [PubMed]
26. Bennett AS, Elliott L, Golub A, Pouget ER, Rosenblum A. Opioid use and overdose risk among OEF/OIF/OND era veterans: a mixed methods analysis. Annual Meeting of the College on Problems of Drug Dependence; June 13–18, 2015; Phoenix, Arizona.
27. Furr RM, Bacharach VR. Psychometrics : an introduction. Second. Lod Angeles: SAGE; 2014.
28. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–428. [PubMed]
29. Clark LAWD. Constructing Validity: Basic Issues in Objective Scale Development. Psychological assessment. 1995;7(3):309–319.
30. Rudd RA, Aleshire N, Zibbell JE, Gladden RM. Increases in Drug and Opioid Overdose Deaths - United States, 2000–2014. MMWR Morbidity and mortality weekly report. 2016;64(50–51):1378–1382. [PubMed]
31. Lake S, Hayashi K, Buxton J, et al. The effect of prescription opioid injection on the risk of non-fatal overdose among people who inject drugs. Drug Alcohol Depend. 2015;156:297–303. [PMC free article] [PubMed]
32. Lake S, Wood E, Buxton J, Dong H, Montaner J, Kerr T. Prescription opioid use and non-fatal overdose in a cohort of injection drug users. The American journal of drug and alcohol abuse. 2015;41(3):257–263. [PMC free article] [PubMed]
33. Compton WM, Jones CM, Baldwin GT. Relationship between Nonmedical Prescription-Opioid Use and Heroin Use. N Engl J Med. . 2016;374(2):154–163. [PubMed]
34. Jones JD, Mogali S, Comer SD. Polydrug abuse: a review of opioid and benzodiazepine combination use. Drug Alcohol Depend. 2012;125(1–2):8–18. [PMC free article] [PubMed]
35. Han B, Compton WM, Jones CM, Cai R. Nonmedical Prescription Opioid Use and Use Disorders Among Adults Aged 18 Through 64 Years in the United States, 2003–2013. Jama. 2015;314(14):1468–1478. [PubMed]
36. Minett WJ, Moore TL, Juhascik MP, Nields HM, Hull MJ. Concentrations of opiates and psychotropic agents in polydrug overdoses: a surprising correlation between morphine and antidepressants. Journal of forensic sciences. 2010;55(5):1319–1325. [PubMed]
37. Jones C, Paulozzi LJ, Mack KA. Alcohol Involvement in Opioid Pain Reliever and Benzodiazepine Drug Abuse–Related Emergency Department Visits and Drug-Related Deaths — United States, 2010. Morbidity and Mortality Weekly Report (MMWR) 2014;63(40):881–885. [PubMed]