|Home | About | Journals | Submit | Contact Us | Français|
Attrition in studies of substance use disorder treatment is problematic, potentially introducing bias into data analysis.
This study aimed to determine the effect of participant compensation amounts on rates of missing data and observed rates of drug use.
We performed a secondary analysis of a clinical trial of buprenorphine/naloxone among 152 treatment-seeking opioid-dependent subjects aged 15–21 during participation in a randomized trial. Subjects were randomized to a 2-week detoxification with buprenorphine/naloxone (DETOX; N = 78) or 12 weeks buprenorphine/naloxone (BUP; N = 74). Participants were compensated $5 for weekly urine drug screens and self-reported drug use information and $75 for more extensive assessments at weeks 4, 8, and 12.
Though BUP assignment decreased the likelihood of missing data, there were significantly less missing data at 4, 8, and 12 weeks than other weeks, and the effect of compensation on the probability of urine screens being positive was more pronounced in DETOX subjects.
These findings suggest that variations in the amount of compensation for completing assessments can differentially affect outcome measurements, depending on treatment group assignment.
Adequate financial compensation may minimize bias when treatment condition is associated with differential dropout and may be a cost-effective way to reduce attrition. Moreover, active users may be more likely than non-active users to drop out if compensation is inadequate, especially in control groups or in groups who are not receiving active treatment.
Attrition in substance abuse treatment research is problematic, with rates of data collection often falling well below 70–80% (1,2). Higher severity, poorer functioning, and younger age are associated with higher attrition in some, but not all, studies (3–5). Similarly, treatment assignment (placebo vs. intervention) has been shown to affect dropout (3).
High rates of attrition in substance abuse research may interfere with outcomes assessment since as little as 30% attrition can result in unpredictable bias (6). The pattern of missingness (e.g., whether measured or unmeasured covariates contribute to attrition) is sometimes not determinable, which can make outcomes analyses complicated (7–9). Problems associated with missing data can sometimes be alleviated by statistical methods including intent-to-treat analyses/imputation, data deletion, mixed effects models with maximum-likelihood estimates, and using dropout status as a time-varying covariate (8–11). If missing data are suspected to be missing not at random, examining the range of possible outcomes and using modified stratified summary statistics may be viable approaches (8,9). Conclusions about treatment efficacy may vary with different analytic strategies, thus increasing the attractiveness of maximizing adherence to follow-up assessments (12).
A variety of strategies have been employed to increase adherence in adult studies including networking and frequent phone calls (3,5,6). In non-research settings, financial incentives increase treatment attendance and abstinence (13–15). The relatively few studies in research subjects that have been published show a preference for cash over vouchers, greater adherence with larger compensation amounts (16–18), and no evidence that greater compensation is coercive or triggers relapse (17,18).
As with adults, follow-up rates in adolescent studies have been similarly problematic (19), although some have successfully adopted more intensive follow-up protocols (20). As with adults, contingency management improves adherence and outcomes in substance abuse treatment (21,22), but to the best of our knowledge, the effects of varying financial compensation amounts on adherence to outcome assessments in treatment studies of substance abusing youth, and their potential effects on bias in outcomes analyses, have not been examined.
Here we address these issues by exploring the effects of different size cash payments on data collection in secondary analyses of a randomized trial of buprenorphine/naloxone treatment for opioid-addicted youth aged 15–21 where subjects were paid $5 for completing weekly assessments and $75 for more detailed assessments at weeks 4, 8, and 12. Our first aim was to investigate the effect of the varying compensation amount on attrition in this understudied population. In addition, we aimed to explore indirectly the kinds of biases that may be present in studies with inadequate compensation amounts by testing whether (1) the strength of the compensation effect differed by treatment group assignment and (2) the strength of the compensation effect differed by urine test results (e.g., whether there was a greater effect of compensation in actively using participants compared with abstinent users).
In the study, 152 subjects aged 15–21 seeking treatment for opioid dependence were randomized to a 2-week detoxification with buprenorphine/naloxone (DETOX; N = 78) or 12 weeks of buprenorphine/naloxone (BUP; N = 74) with a dose taper beginning in week 9 and ending in week 12, each with weekly individual and group drug counseling (23). Subjects were paid $5 for providing a weekly urine drug screen and self-report of drug use, and $75 for more extensive assessments at weeks 4, 8, and 12. Weekly assessments took approximately 30 minutes, and monthly assessments (weeks 4, 8, and 12) took approximately 90 minutes.
Urine samples were tested for morphine/opiates and oxycodone. From these results, three dichotomous variables were created: (1) missing versus non-missing (non-missing samples were defined as positive or negative); (2) positive versus non-positive (non-positive samples were defined as negative or missing); (3) negative versus nonnegative (non-negative samples were defined as positive or missing). The correlations between the three variables across 12 weeks were variable, implying discriminate validity (range of phi coefficients between missing and positive: −.52 to −.36; positive and negative: −.63 to −.16; missing and negative: −.84 to −.46).
To analyze longitudinal dichotomous variables, we used generalized linear mixed modeling (GLMM), an extension of the generalized linear model that can accommodate random effects, as well as the correlated nature of repeated measures data (24). Based on maximum-likelihood estimation, GLMM provides results in terms of logit(x) = log(x)/([1−x]). Logit(x) can be converted into a probability value by applying the formula p(x) = exp(logit[x])/exp(1+logit[x]). GLMM was performed based on the R lme4 package’s lmer function.
For each of the dependent variables, a GLMM was created, in which we included the following time points: weeks 3, 4, and 5; weeks 7, 8, and 9; weeks 11 and 12. We did so in order to model three levels of time, each of which included one high compensation week (week 4, 8, or 12) and the weeks immediately preceding and following (except that there were no data for the week following week 12).
Treatment (DETOX vs. BUP) and Compensation (HIGH: weeks 4, 8, and 12 vs. LOW: weeks 3, 5, 7, 9, and 11) are fixed factors and Time, a continuous variable (Time 1 = weeks 3, 4, and 5; Time 2 = weeks 7, 8, and 9; Time 3 = weeks 11 and 12). Time was centered so that the intercepts would represent average values at Time 1, instead of Time 0. Treatment was a between-subject factor, while Compensation and Time were within-subject factors. All models tested Treatment, Compensation, and Time effects and their interaction.
We chose this particular model in order to maximize the ease of interpretation of results (only one output table per model) and the precision of the estimate of effects (larger number of data points). Although results are not presented here, we also tested for differences in Compensation, Treatment, and their interaction for the three dependent measures at three separate time points, and used the same GLMM approach using only weeks 3, 4, 7, 8, 11, and 12. When subject numbers were high enough to make these data interpretable, results of these analyses were in agreement with the results presented below, indicating that having a smaller number of data points at Time 3 (i.e., 2 weeks instead of 3) did not bias our results.
We first fit the GLMM to the data without allowing subjects to have their own unique slopes (i.e., random slopes). Then we tested whether allowing random slopes would significantly improve the model fit, using likelihood-ratio tests. For the missing versus non-missing dependent variable, adding random slopes did not significantly improve model fit (p = .06), while for the other two dependent variables, it did (p < .01). Thus, the GLMM for the missing versus non-missing variable did not include random slopes, while for the other two dependent variables it did. For the GLMM analysis, the following statistics are reported below: estimate of logit(p) ± standard error of the estimate, z-scores (estimate of logit (p)/standard error), p-value.
While the majority of subjects returned at weeks 1 and 2, attrition slowly increased over the course of the study with the highest attrition at week 11 (Figure 1). Unlike the low compensation weeks, at high compensation weeks (4,8,12), the follow-up rates nearly reached those in the first 2 weeks.
Follow-up rates were also affected by treatment group assignment (Figure 1). At low compensation weeks, the return rate was considerably less among subjects assigned to DETOX (mean = 27%), and declined more rapidly than those assigned to BUP (mean = 67%). At high compensation weeks, the rate was more comparable between the groups (mean = 54% for DETOX vs. mean = 72% for BUP).
As described in Section 2, Treatment, Compensation, and Time variables were included. The Low Compensation and DETOX condition at Time 1 were defined as reference conditions. The model revealed significant positive Treatment and Compensation effects (p < .01, Table 1), indicating that BUP assignment and higher compensation amounts decreased the likelihood of missing data. The model also revealed a negative interaction effect (p < .01), reflecting the greater Compensation effect among DETOX participants. Moreover, the model revealed a significant negative Time effect (p < .01), indicating that non-missing values decreased over time, and a significant positive Compensation × Time interaction (p < .05), indicating an increasing effect of compensation amounts on the probability of non-missing values over time. In other words, this could indicate that the probability of missing data during low compensation increased over time, whereas the probability of missing data during high compensation was steady over time. Neither the Treatment × Time interaction nor the three-way interaction was significant (p > .10).
The model revealed significant positive Compensation and negative Time effects (p < .01, Table 2), indicating that the likelihood of positive urine screen increased with higher compensation amounts and decreased over time. However, the Treatment effect was not significant (p > .10). The model also revealed a significant negative Compensation ×Treatment interaction (p < .01), suggesting that high compensation increased the probability of urine positive samples (as opposed to urine negative or missing samples) to a greater degree in DETOX subjects compared with BUP subjects. Other effects and interactions were not significant (p > .05).
The model revealed significant positive Compensation and Treatment effects (p < .01, Table 3), indicating that the likelihood of negative urine screens increased with higher compensation amounts and BUP assignment. Unlike analyses of the missing or positive urine screen data, the Compensation × Treatment interaction was not significant (p > .10). The only other significant effect was the three-way interaction (p < .05), which could be interpreted to mean that higher compensation is associated with a higher rate of negative urine screens in BUP subjects, whereas in DETOX subjects, there is no comparable change in status associated with compensation.
These results emphasize the importance of giving adequate monetary compensation to participants in treatment studies of substance abusing youth. In this secondary analysis, higher monetary compensation decreased the probability of missing urine screen data. Treatment group assignment was associated with differing rates of missing data, with DETOX being associated with higher rates of missing data. Higher compensation had a larger effect on the probability of urine screen collection in DETOX compared with BUP. This implies that the $5 compensation amount was not adequate to offset the study burden and that attrition varied according to study conditions. This could be viewed also as a treatment phase effect (e.g., active treatment vs. post taper from active treatment), since individuals in DETOX are no longer on active pharmacological treatment.
That the compensation effect on the probability of positive urine screens was greater in DETOX compared with BUP, whereas the compensation effect on the probability of negative urine screens did not differ between DETOX and BUP, suggests that the compensation effect was especially robust in opioid users in the control condition or that active users, as opposed to abstinent participants, were more likely to be missing during low compensation time points. This effect can also be observed qualitatively (Figure 1). This could be interpreted to mean that higher compensation preferentially entices active users to comply with follow-ups to obtain money because they may be more desperate (16,17), giving support to arguments against higher compensation amounts due to the possibility of coercion. On the other hand, higher compensation appears to improve the accuracy of outcome analyses and significantly reduces biases due to poor data collection from active users in a control condition. Had this study been designed with lower compensation amounts across the board, DETOX would have looked more effective than it actually was. Higher compensation amounts may also support the ethical principle of justice, which speaks to the need to gather information about disenfranchised populations (25).
Notably, even with $75 compensation, there was, overall, a 40% attrition rate by week 12, below the accepted 30% standard (6). During week 11, where subjects were paid $5, there was a 68% attrition rate, highlighting the importance of adequate compensation for the primary outcome assessment points (3,5,6,20).
Our goal in this study was to investigate the effects of compensation amounts on attrition and not to investigate the validity of the methods used in the original clinical trial. However, that there was an overall treatment effect on rates of negative urine screens in our analysis gives further support to the findings and validity of the methods used (a random-effects pattern-mixture model and an intent-to-treat analysis in which missing urine screen data were imputed as positive) (23).
Our results demonstrated that the probability of missing urine screen data increased over time, which is supported by other studies (3). Our analyses also showed an increasingly larger effect of higher monetary compensation on the probability of non-missing data over time, emphasizing the importance of adequate compensation amounts at later time points. Whereas the likelihood of urine screens being positive significantly decreased over time, there was no change in rates of urine screens being negative, implying that non-users may be less influenced to drop out over time than active users.
Several significant limitations of this study should be mentioned. This was a secondary analysis of a study that was not designed to assess reimbursement effects. Moreover, we cannot eliminate the possibility that mediating factors (such as differing assessment time or levels of research assistant (RA) effort in scheduling between $5 and $75 time points) were driving these associations. Although demographics and other patient variables could potentially influence drug screen results, we chose not to investigate other predictors of outcomes because no particular subject characteristic has yet been found to reliably predict attrition (3–5), and this was not the primary focus of our study.
Despite these limitations, these data are important as they highlight the importance of adequate monetary compensation to minimize attrition in treatment studies of adolescents with substance use disorders, and to minimize risk of bias in outcomes analyses. Some authors have expressed concerns about the use of positive imputation methods (10). However, in statistical approaches which examine the range of possible outcomes, positive imputation may provide more valid results in cases where there is greater dropout in control groups. Further studies are needed to determine the optimal and ethically acceptable amount of monetary compensation.
Supported by the NIDA Clinical Trials Network grants U10DA15833 (Bogenschutz) and U10DA13043 and KO5 DA-17009 (Woody). Thanks to the following sites and nodes of the NIDA Clinical Trials Network for their help during the clinical trial: Ayundantes, Espanola, NM (Southwest Node); Brandywine Counseling, Newark, DE (Delaware Valley Node); Duke Addictions Program, Durham, NC (North Carolina Node); Mercy Recovery, Westbrook, ME (Northern New England Node); Mountain Manor Treatment Center, Baltimore, MD (Mid-Atlantic Node); and the University of New Mexico Addiction and Substance Abuse Programs, Albuquerque, NM (Southwest Node).
Declaration of Interest
The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.