|Home | About | Journals | Submit | Contact Us | Français|
Drug overdose has emerged as the leading cause of injury-related death in the U.S., driven by prescription opioid (PO) misuse, polysubstance use and use of heroin. To better understand opioid-related overdose risks that may change over time and across populations there is a need for a more comprehensive assessment of related risk behaviors. We developed the opioid-related Overdose Risk Behavior Scale (ORBS), drawing on existing research, formative interviews, and discussions with community and scientific advisors.
We enrolled military veterans reporting any use of heroin or POs in the past month using venue-based and chain referral recruitment. The final scale consisted of 25 items grouped into 5 subscales eliciting the number of days in the past 30 during which the participant engaged in each behavior. We assessed internal reliability using Cronbach’s alpha; test-retest reliability using intraclass correlation coefficients (ICCs); and criterion validity using Pearson’s correlations with indicators of having overdosed during the past 30 days.
Data for 220 veterans were analyzed. The 5 subscales---A. Adherence to Opioid Dosage and Therapeutic Purposes, B. Alternative Methods of Opioid Administration, C. Solitary Opioid Use; D. Use of Non-prescribed Overdose-associated Drugs and E. Concurrent Use of POs, Other Psychoactive Drugs and Alcohol---generally showed good internal reliability (alpha range = 0.61 to 0.88), test-retest reliability (ICC range = 0.81 to 0.90), and criterion validity (r range = 0.22 to 0.66). The subscales were internally consistent with each other (alpha = 0.84). The scale mean had an ICC value of 0.99, and correlations with validators ranged from 0.44 to 0.56.
These results constitute preliminary evidence for the reliability and validity of the new scale. If further validated, it could help improve overdose prevention and response research, and could help improve the precision of overdose education and prevention efforts.
Drug overdose has emerged as the leading cause of injury-related death in the U.S., driven by prescription opioid (PO) misuse, polysubstance use and use of heroin.1–6 For overdose prevention and response research a broad assessment capable of capturing behavioral risks in populations with varying substance choices and use patterns is critically important, particularly as we seek to understand the precipitants of changes in overdose risk behaviors among at-risk populations.
Though no comprehensive overdose risk behavior scale exists, a number of clinical screening tools assess some aspects of the potential for PO misuse. For example, the Current Opioid Misuse Measure (COMM), the Screener and Opioid Assessment for Patients with Pain (SOAPP), and the Opioid-Related Behaviors in Treatment (ORBIT) scale were designed to help clinicians assess PO misuse risks among pain patients.7–11 Reflecting the purpose of clinical screening, these scales include items on drug-seeking behaviors, past history of substance use, and mood states presumed to be related to PO misuse,7,10–17 sometimes incorporating criteria to suggest when a clinician should consider modifying (or denying) PO treatment.9,12–15,18 However, since these scales were not designed to assess overdose risk they lack items regarding the contraindicated use of opioids (including heroin, methadone and buprenorphine), other drugs and alcohol, and items regarding drug use behaviors that can increase risk, such as injecting opioids, or using opioids alone.19–21
To better understand the factors that cause opioid-related overdose a first step is to comprehensively assess overdose risk behaviors, and test their associations with overdose events. One group with elevated overdose risks related to PO and heroin use is military veterans.22,23 Below, we describe the development of an opioid-related Overdose Risk Behavior Scale (ORBS) and present results of preliminary reliability and validity analyses in a community-based sample of US military veterans.
We enrolled 220 U.S. military veterans who reported any heroin or PO use (within past 30-days) at the time of enrollment between August 2014 and June 2016 as part of an ongoing study of opioid misuse and overdose risks.23,24 We recruited participants in veterans’ service agencies, homeless shelters and opioid treatment programs throughout New York City using venue-based and chain referral methods. Enrolled participants completed survey-based assessments of their military service, alcohol, PO and other drug use, biological, psychological, social contextual factors and major life events.
Our aim was to construct a scale to assess known opioid-related overdose risk behaviors. We adapted 4 items from the COMM [“How often have you…taken your medications differently from how they are prescribed? …needed to take pain medications belonging to someone else? …had to take more of your medication than prescribed? …used your pain medicine for symptoms other than for pain?”] regarding POs that were prescribed by the patients’ own doctor.25 Additional items were developed using a literature review,19–21 discussions with members of the study’s community advisory board and scientific advisory board, and results of qualitative interviews with 50 veterans who reported experiencing at least one opioid-related overdose at some point in the past.26 Although, theoretically, there was no strong rationale for assuming that these items would be correlated, items were naturally categorized into subscale domains that we hypothesized would be internally consistent. We used the number of days during the past 30 days as the response units for all items. Thus, responses are ratio scaled.27
We pilot tested a preliminary version of the scale with 15 participants who were then interviewed about clarity, comprehensiveness and tone. After this process, the research team reconvened and eliminated 4 items (using borrowed, purchased or stolen POs, and using POs at different intervals than prescribed) because they were not understood consistently and overlapped with other items, and revised item wording for clarity and brevity.
After eligibility screening and informed consent procedures, participants completed a survey assessment using face-to-face methods with a trained and experienced interviewer. Participants provided informed consent, and received $20 for completing the assessment, which also included investigational domains hypothesized to precipitate change in overdose risk behaviors, including mood, physical pain and life events. All procedures and protocols were approved by the Institutional Review Board of the first author. Administration duration of the new scale was typically 5–10 minutes. Only the initial assessment from each participant was analyzed (with the exception of test-retest data). We used IBM SPSS (ver. 22) for analyses. We assessed reliability and validity of the total scale and of subscales. We report scale scores as an unweighted average of constituent items. Thus, the subscale and total scale means reflect the same 0–30 units as the constituent items.
We asked a subset of 38 sequential participants to repeat the assessment the next day, preferably at the same time of day; 34 of these provided retest data within the time frame. Participants were compensated an additional $20 for retests. We compared their responses using intraclass correlation coefficients (ICCs) with a one-way random effects formulation, ICC(1,1).28 We also compared mean differences in test and retest assessments using t-tests for paired samples. To determine whether the participants who provided test-retest reliability data differed significantly from the rest of the sample we compared their characteristics using Student’s t-tests.
We assessed criterion validity retrospectively using Pearson’s correlations with two indicators of having experienced an overdose during the same past-30-day time period. We asked: “On how many days in the past 30 did you lose consciousness or pass out so that you could not really wake up or others could not wake you up?” and “On how many days in the past 30 did you or someone else call for medical assistance because of how sedated, drugged, or high you were after using opioids?”
We examined Pearson’s correlations among the subscales to assess how they were related.
Participant characteristics are shown in Table 1. Most participants were male, never married and unemployed, despite relatively high educational achievement. The majority were black/African American, and almost a quarter were Hispanic/Latino. Ages ranged from 21 to 60, averaging 36.9 years (standard deviation [SD] = 9.5). Almost half the sample reported being homeless or unstably housed at the time of enrollment. Heroin use (during the last 30 days) was reported by 27.3%, and PO use (including those who had been prescribed POs) was reported by 89.1%.
Table 2 lists ORBS items, grouped by subscales. All or some items in 3 of the 5 scales (A, B, and C) are only applicable to participants who reported a current PO prescription. Therefore, the sample size for those scales is restricted to the 104 (47%) participants who met this criterion. Mean values represent the average number of days during the past 30 in which participants engaged in each behavior. As shown in the interquartile range column, many items were infrequently endorsed, with 25th percentile, and sometimes 75th percentile values of 0. The most frequent risk behavior was A.4. Use of POs to aid sleeping, averaging 6.0 days (SD = 9.2). Mean values of items regarding prescribed opioids were generally higher than those for non-prescribed drugs.
There were no significant differences in participant characteristics between participants who provided test-retest data and those who did not, and there were no significant differences in mean values of ORBS items or scales between test and retest assessments.
Subscales A, B and C good internal consistency (alpha range = 0.68 to 0.88), test-retest reliability (ICC range = 0.47 to 0.85), and validity (r with criterion variables range = 0.38 to 0.66). (As shown in Table 2, all correlations of the total ORBS and of ORBS subscales with overdose event criterion variables were significant at the p < 0.05 level.)
Subscale D showed marginally acceptable internal consistency (alpha = 0.61), and good retest reliability (ICC = 0.84). Correlations with validators were moderate (r = 0.22, r = 0.47). We considered dropping the non-therapeutic use of methadone from the subscale because few participants (6.4%) endorsed this item. However, the item had a moderate item-total correlation (0.33), which suggested it was worth retaining. We also asked whether participants used non-prescribed buprenorphine. However, this item was the least frequently endorsed, and had a very low item-total correlation (0.08), so it was not retained.
Subscale E showed good internal consistency (alpha = 0.77) and retest reliability (ICC = 0.90). Some items were infrequently endorsed, but are retained to be consistent with our purpose of developing a more comprehensive assessment of opioid-related risk behaviors than is currently available.29 For example, as shown in Table 2, the mean for E.3. Using buprenorphine and any other POs on the same day was 0.3 (4.1% of participants reported using buprenorphine and any other opioids on the same day once or more during the past 30 days). Correlations with validators were moderate (r = 0.38, r = 0.49).
Averaging the subscales yields a mean of 2.6 (SD = 3.7). This mean was correlated moderately to strongly with criterion validators (r = 0.44, r = 0.56). The alpha value for the Cronbach’s analysis of the 5 ORBS subscale means was 0.84.
Correlations among ORBS subscale means are in Table 3. Correlations ranged from 0.45 to 0.72.
Our findings provide preliminary evidence that engagement in opioid-related overdose risk behaviors can be assessed comprehensively using a brief scale. Five subscales representing engagement in opioid using behaviors that can increase overdose risk were each, and as an aggregate, correlated with indicators of an opioid overdose event. These subscales included behaviors specific to PO misuse among patients, and behaviors that can apply to heroin use or PO misuse among patients and non-patients, such as solitary use and concurrent use of other drugs or alcohol. As the opioid overdose epidemic continues to worsen, it is all the more important to comprehensively assess risks related to heroin and polysubstance use.30 Though it was designed for research purposes the ORBS also has the potential to be useful in therapeutic contexts where counselors, psychiatrists and social workers stand to aid individuals in developing strategies to minimize engagement in such behaviors. This value may also be realized in community-based settings wherein overdose education and prevention programs might be bolstered by more specific data about the nature and frequency of engagement in overdose risk behaviors.
There are several important limitations to the interpretation of these results, the first of which applies to all research using self-report. We did not collect or analyze biological data, leading to a potential underreporting of opioid use severity due to social desirability bias. Two subscales (C and D) with relatively weak internal reliability results retain moderate correlations with criterion validity indicators, which may suggest that the lower reliability resulted from infrequent responses for some items. For example, items regarding smoking or injecting POs were rarely endorsed, despite being associated with increased risk.31,32 We retain these items as they may represent important risks in other populations, especially considering the evolving nature of the opioid overdose epidemic. Our limitations with regard to sample size and a cross-sectional ascertainment provide guidelines for future research; for example, broader investigations of at-risk cohorts in addition to opioid-using veterans could evaluate generalizability, and a prospective-longitudinal study could evaluate predictive validity.
In conclusion, results from this study show preliminary evidence that a comprehensive set of opioid-related overdose risk behaviors can be assessed reliably using a brief scale. The ORBS’ focus on overdose-associated polysubstance use, modes of administration, and social contexts for use may prove to be useful for better understanding overdose risks that can change over time or may differ across populations.33–37 It has the potential to help to improve the precision of overdose prevention and response research, and overdose education and prevention efforts.
This research was supported by National Institute on Drug Abuse grant R01DA036754 (PI: A. Bennett); Career Development Award from the Department of Veterans Affairs Office of Research Development, Clinical Science Research and Development (CSR&D) IK2CX000641 (PI: P.C. Britton); and institutional training award T32DA007233 (B. Wolfson-Stofko). The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
The authors declare they have no conflicts of interest.
AUTHOR CONTRIBUTIONSERP performed the statistical analyses, wrote the initial draft and contributed to the development of the scale; ASB, was responsible for the initial conception of the study and was the Principal Investigator and contributed to the writing and scale development, LE was the Project Director and contributed to the writing and scale development; BW-S contributed to the writing; RA, PCB and AR contributed to the writing and scale development.