PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Am J Drug Alcohol Abuse. Author manuscript; available in PMC 2013 August 17.
Published in final edited form as:
PMCID: PMC3745824
NIHMSID: NIHMS498046

Compensation Effects on Clinical Trial Data Collection in Opioid-Dependent Young Adults

Abstract

Background

Attrition in studies of substance use disorder treatment is problematic, potentially introducing bias into data analysis.

Objectives

This study aimed to determine the effect of participant compensation amounts on rates of missing data and observed rates of drug use.

Methods

We performed a secondary analysis of a clinical trial of buprenorphine/naloxone among 152 treatment-seeking opioid-dependent subjects aged 15–21 during participation in a randomized trial. Subjects were randomized to a 2-week detoxification with buprenorphine/naloxone (DETOX; N = 78) or 12 weeks buprenorphine/naloxone (BUP; N = 74). Participants were compensated $5 for weekly urine drug screens and self-reported drug use information and $75 for more extensive assessments at weeks 4, 8, and 12.

Results

Though BUP assignment decreased the likelihood of missing data, there were significantly less missing data at 4, 8, and 12 weeks than other weeks, and the effect of compensation on the probability of urine screens being positive was more pronounced in DETOX subjects.

Conclusion

These findings suggest that variations in the amount of compensation for completing assessments can differentially affect outcome measurements, depending on treatment group assignment.

Scientific Significance

Adequate financial compensation may minimize bias when treatment condition is associated with differential dropout and may be a cost-effective way to reduce attrition. Moreover, active users may be more likely than non-active users to drop out if compensation is inadequate, especially in control groups or in groups who are not receiving active treatment.

Keywords: compensation, incentives, missing data, attrition, substance use disorder treatment

1. INTRODUCTION

Attrition in substance abuse treatment research is problematic, with rates of data collection often falling well below 70–80% (1,2). Higher severity, poorer functioning, and younger age are associated with higher attrition in some, but not all, studies (35). Similarly, treatment assignment (placebo vs. intervention) has been shown to affect dropout (3).

High rates of attrition in substance abuse research may interfere with outcomes assessment since as little as 30% attrition can result in unpredictable bias (6). The pattern of missingness (e.g., whether measured or unmeasured covariates contribute to attrition) is sometimes not determinable, which can make outcomes analyses complicated (79). Problems associated with missing data can sometimes be alleviated by statistical methods including intent-to-treat analyses/imputation, data deletion, mixed effects models with maximum-likelihood estimates, and using dropout status as a time-varying covariate (811). If missing data are suspected to be missing not at random, examining the range of possible outcomes and using modified stratified summary statistics may be viable approaches (8,9). Conclusions about treatment efficacy may vary with different analytic strategies, thus increasing the attractiveness of maximizing adherence to follow-up assessments (12).

A variety of strategies have been employed to increase adherence in adult studies including networking and frequent phone calls (3,5,6). In non-research settings, financial incentives increase treatment attendance and abstinence (1315). The relatively few studies in research subjects that have been published show a preference for cash over vouchers, greater adherence with larger compensation amounts (1618), and no evidence that greater compensation is coercive or triggers relapse (17,18).

As with adults, follow-up rates in adolescent studies have been similarly problematic (19), although some have successfully adopted more intensive follow-up protocols (20). As with adults, contingency management improves adherence and outcomes in substance abuse treatment (21,22), but to the best of our knowledge, the effects of varying financial compensation amounts on adherence to outcome assessments in treatment studies of substance abusing youth, and their potential effects on bias in outcomes analyses, have not been examined.

Here we address these issues by exploring the effects of different size cash payments on data collection in secondary analyses of a randomized trial of buprenorphine/naloxone treatment for opioid-addicted youth aged 15–21 where subjects were paid $5 for completing weekly assessments and $75 for more detailed assessments at weeks 4, 8, and 12. Our first aim was to investigate the effect of the varying compensation amount on attrition in this understudied population. In addition, we aimed to explore indirectly the kinds of biases that may be present in studies with inadequate compensation amounts by testing whether (1) the strength of the compensation effect differed by treatment group assignment and (2) the strength of the compensation effect differed by urine test results (e.g., whether there was a greater effect of compensation in actively using participants compared with abstinent users).

2. MATERIALS AND METHODS

2.1. Participants and Outcomes

In the study, 152 subjects aged 15–21 seeking treatment for opioid dependence were randomized to a 2-week detoxification with buprenorphine/naloxone (DETOX; N = 78) or 12 weeks of buprenorphine/naloxone (BUP; N = 74) with a dose taper beginning in week 9 and ending in week 12, each with weekly individual and group drug counseling (23). Subjects were paid $5 for providing a weekly urine drug screen and self-report of drug use, and $75 for more extensive assessments at weeks 4, 8, and 12. Weekly assessments took approximately 30 minutes, and monthly assessments (weeks 4, 8, and 12) took approximately 90 minutes.

Urine samples were tested for morphine/opiates and oxycodone. From these results, three dichotomous variables were created: (1) missing versus non-missing (non-missing samples were defined as positive or negative); (2) positive versus non-positive (non-positive samples were defined as negative or missing); (3) negative versus nonnegative (non-negative samples were defined as positive or missing). The correlations between the three variables across 12 weeks were variable, implying discriminate validity (range of phi coefficients between missing and positive: −.52 to −.36; positive and negative: −.63 to −.16; missing and negative: −.84 to −.46).

2.2. Statistical Methods

To analyze longitudinal dichotomous variables, we used generalized linear mixed modeling (GLMM), an extension of the generalized linear model that can accommodate random effects, as well as the correlated nature of repeated measures data (24). Based on maximum-likelihood estimation, GLMM provides results in terms of logit(x) = log(x)/([1−x]). Logit(x) can be converted into a probability value by applying the formula p(x) = exp(logit[x])/exp(1+logit[x]). GLMM was performed based on the R lme4 package’s lmer function.

For each of the dependent variables, a GLMM was created, in which we included the following time points: weeks 3, 4, and 5; weeks 7, 8, and 9; weeks 11 and 12. We did so in order to model three levels of time, each of which included one high compensation week (week 4, 8, or 12) and the weeks immediately preceding and following (except that there were no data for the week following week 12).

Treatment (DETOX vs. BUP) and Compensation (HIGH: weeks 4, 8, and 12 vs. LOW: weeks 3, 5, 7, 9, and 11) are fixed factors and Time, a continuous variable (Time 1 = weeks 3, 4, and 5; Time 2 = weeks 7, 8, and 9; Time 3 = weeks 11 and 12). Time was centered so that the intercepts would represent average values at Time 1, instead of Time 0. Treatment was a between-subject factor, while Compensation and Time were within-subject factors. All models tested Treatment, Compensation, and Time effects and their interaction.

We chose this particular model in order to maximize the ease of interpretation of results (only one output table per model) and the precision of the estimate of effects (larger number of data points). Although results are not presented here, we also tested for differences in Compensation, Treatment, and their interaction for the three dependent measures at three separate time points, and used the same GLMM approach using only weeks 3, 4, 7, 8, 11, and 12. When subject numbers were high enough to make these data interpretable, results of these analyses were in agreement with the results presented below, indicating that having a smaller number of data points at Time 3 (i.e., 2 weeks instead of 3) did not bias our results.

We first fit the GLMM to the data without allowing subjects to have their own unique slopes (i.e., random slopes). Then we tested whether allowing random slopes would significantly improve the model fit, using likelihood-ratio tests. For the missing versus non-missing dependent variable, adding random slopes did not significantly improve model fit (p = .06), while for the other two dependent variables, it did (p < .01). Thus, the GLMM for the missing versus non-missing variable did not include random slopes, while for the other two dependent variables it did. For the GLMM analysis, the following statistics are reported below: estimate of logit(p) ± standard error of the estimate, z-scores (estimate of logit (p)/standard error), p-value.

3. RESULTS

3.1. Attrition

While the majority of subjects returned at weeks 1 and 2, attrition slowly increased over the course of the study with the highest attrition at week 11 (Figure 1). Unlike the low compensation weeks, at high compensation weeks (4,8,12), the follow-up rates nearly reached those in the first 2 weeks.

FIGURE 1
The number of participants by urine screen outcome (missing, negative, and positive), by week, and by treatment is illustrated. Subjects are paid $75 at 4, 8, and 12 weeks and $5 during the other weeks for completing assessment.

Follow-up rates were also affected by treatment group assignment (Figure 1). At low compensation weeks, the return rate was considerably less among subjects assigned to DETOX (mean = 27%), and declined more rapidly than those assigned to BUP (mean = 67%). At high compensation weeks, the rate was more comparable between the groups (mean = 54% for DETOX vs. mean = 72% for BUP).

3.2. Missing versus Non-missing

As described in Section 2, Treatment, Compensation, and Time variables were included. The Low Compensation and DETOX condition at Time 1 were defined as reference conditions. The model revealed significant positive Treatment and Compensation effects (p < .01, Table 1), indicating that BUP assignment and higher compensation amounts decreased the likelihood of missing data. The model also revealed a negative interaction effect (p < .01), reflecting the greater Compensation effect among DETOX participants. Moreover, the model revealed a significant negative Time effect (p < .01), indicating that non-missing values decreased over time, and a significant positive Compensation × Time interaction (p < .05), indicating an increasing effect of compensation amounts on the probability of non-missing values over time. In other words, this could indicate that the probability of missing data during low compensation increased over time, whereas the probability of missing data during high compensation was steady over time. Neither the Treatment × Time interaction nor the three-way interaction was significant (p > .10).

TABLE 1
Summary of generalized linear mixed modeling (GLMM) analysis for predicting non-missing values (N = 152).

3.3 Positive versus Non-positive

The model revealed significant positive Compensation and negative Time effects (p < .01, Table 2), indicating that the likelihood of positive urine screen increased with higher compensation amounts and decreased over time. However, the Treatment effect was not significant (p > .10). The model also revealed a significant negative Compensation ×Treatment interaction (p < .01), suggesting that high compensation increased the probability of urine positive samples (as opposed to urine negative or missing samples) to a greater degree in DETOX subjects compared with BUP subjects. Other effects and interactions were not significant (p > .05).

TABLE 2
Summary of generalized linear mixed modeling (GLMM) analysis for predicting positive opioid urine screen with missing values imputed as non-positive (N = 152).

3.4. Negative versus Non-negative

The model revealed significant positive Compensation and Treatment effects (p < .01, Table 3), indicating that the likelihood of negative urine screens increased with higher compensation amounts and BUP assignment. Unlike analyses of the missing or positive urine screen data, the Compensation × Treatment interaction was not significant (p > .10). The only other significant effect was the three-way interaction (p < .05), which could be interpreted to mean that higher compensation is associated with a higher rate of negative urine screens in BUP subjects, whereas in DETOX subjects, there is no comparable change in status associated with compensation.

TABLE 3
Summary of generalized linear mixed modeling (GLMM) analysis for predicting negative opioid urine screen with missing values imputed as non-negative (N = 152).

4. DISCUSSION

These results emphasize the importance of giving adequate monetary compensation to participants in treatment studies of substance abusing youth. In this secondary analysis, higher monetary compensation decreased the probability of missing urine screen data. Treatment group assignment was associated with differing rates of missing data, with DETOX being associated with higher rates of missing data. Higher compensation had a larger effect on the probability of urine screen collection in DETOX compared with BUP. This implies that the $5 compensation amount was not adequate to offset the study burden and that attrition varied according to study conditions. This could be viewed also as a treatment phase effect (e.g., active treatment vs. post taper from active treatment), since individuals in DETOX are no longer on active pharmacological treatment.

That the compensation effect on the probability of positive urine screens was greater in DETOX compared with BUP, whereas the compensation effect on the probability of negative urine screens did not differ between DETOX and BUP, suggests that the compensation effect was especially robust in opioid users in the control condition or that active users, as opposed to abstinent participants, were more likely to be missing during low compensation time points. This effect can also be observed qualitatively (Figure 1). This could be interpreted to mean that higher compensation preferentially entices active users to comply with follow-ups to obtain money because they may be more desperate (16,17), giving support to arguments against higher compensation amounts due to the possibility of coercion. On the other hand, higher compensation appears to improve the accuracy of outcome analyses and significantly reduces biases due to poor data collection from active users in a control condition. Had this study been designed with lower compensation amounts across the board, DETOX would have looked more effective than it actually was. Higher compensation amounts may also support the ethical principle of justice, which speaks to the need to gather information about disenfranchised populations (25).

Notably, even with $75 compensation, there was, overall, a 40% attrition rate by week 12, below the accepted 30% standard (6). During week 11, where subjects were paid $5, there was a 68% attrition rate, highlighting the importance of adequate compensation for the primary outcome assessment points (3,5,6,20).

Our goal in this study was to investigate the effects of compensation amounts on attrition and not to investigate the validity of the methods used in the original clinical trial. However, that there was an overall treatment effect on rates of negative urine screens in our analysis gives further support to the findings and validity of the methods used (a random-effects pattern-mixture model and an intent-to-treat analysis in which missing urine screen data were imputed as positive) (23).

Our results demonstrated that the probability of missing urine screen data increased over time, which is supported by other studies (3). Our analyses also showed an increasingly larger effect of higher monetary compensation on the probability of non-missing data over time, emphasizing the importance of adequate compensation amounts at later time points. Whereas the likelihood of urine screens being positive significantly decreased over time, there was no change in rates of urine screens being negative, implying that non-users may be less influenced to drop out over time than active users.

Several significant limitations of this study should be mentioned. This was a secondary analysis of a study that was not designed to assess reimbursement effects. Moreover, we cannot eliminate the possibility that mediating factors (such as differing assessment time or levels of research assistant (RA) effort in scheduling between $5 and $75 time points) were driving these associations. Although demographics and other patient variables could potentially influence drug screen results, we chose not to investigate other predictors of outcomes because no particular subject characteristic has yet been found to reliably predict attrition (35), and this was not the primary focus of our study.

Despite these limitations, these data are important as they highlight the importance of adequate monetary compensation to minimize attrition in treatment studies of adolescents with substance use disorders, and to minimize risk of bias in outcomes analyses. Some authors have expressed concerns about the use of positive imputation methods (10). However, in statistical approaches which examine the range of possible outcomes, positive imputation may provide more valid results in cases where there is greater dropout in control groups. Further studies are needed to determine the optimal and ethically acceptable amount of monetary compensation.

Acknowledgments

Supported by the NIDA Clinical Trials Network grants U10DA15833 (Bogenschutz) and U10DA13043 and KO5 DA-17009 (Woody). Thanks to the following sites and nodes of the NIDA Clinical Trials Network for their help during the clinical trial: Ayundantes, Espanola, NM (Southwest Node); Brandywine Counseling, Newark, DE (Delaware Valley Node); Duke Addictions Program, Durham, NC (North Carolina Node); Mercy Recovery, Westbrook, ME (Northern New England Node); Mountain Manor Treatment Center, Baltimore, MD (Mid-Atlantic Node); and the University of New Mexico Addiction and Substance Abuse Programs, Albuquerque, NM (Southwest Node).

Footnotes

Declaration of Interest

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

References

1. Dutra L, Stathopoulou G, Basden SL, Leyro TM, Powers MB, Otto MW. A meta-analytic review of psychosocial interventions for substance use disorders. Am J Psychiatry. 2008;165(2):179–187. [PubMed]
2. Mattson ME, Del Boca FK, Carroll KM, Cooney NL, Di Clemente CC, Donovan D, Kadden RM, McRee B, Rice C, Rycharik RG, Zweben A. Compliance with treatment and follow-up protocols in project MATCH: Predictors and relationship to outcome. Alcohol Clin Exp Res. 1998;22(6):1328–1339. [PubMed]
3. Lavori PW, Bloch DA, Bridge PT, Leiderman DB, Locastro JS, Somoza E. Plans, designs, and analyses for clinical trials of anti-cocaine medications: Where we are today, NIDA/VA/SU Working Group on Design and Analysis. J Clin Psychopharmacol. 1999;19(3):246–256. [PubMed]
4. Agosti V, Nunes E, Ocepeck-Welikson K. Patient factors related to early attrition from an outpatient cocaine research clinic. Am J Drug Alcohol Abuse. 1996;22(1):29–39. [PubMed]
5. Cottler LB, Compton WM, Ben-Abdallah A, Horne M, Claverie D. Achieving a 96. 6 percent follow-up rate in a longitudinal study of drug abusers. Drug Alcohol Depend. 1996;41(3):209–217. [PubMed]
6. Scott CK. A replicable model for achieving over 90% follow-up rates in longitudinal studies of substance abusers. Drug Alcohol Depend. 2004;74(1):21–36. [PubMed]
7. Fielding S, Fayers PM, Ramsay CR. Investigating the missing data mechanism in quality of life outcomes: A comparison of approaches. Health Qual Life Outcomes. 2009;7:57. [PMC free article] [PubMed]
8. Hedden SL, Woolson RF, Carter RE, Palesch Y, Upadhyaya HP, Malcolm RJ. The impact of loss to follow-up on hypothesis tests of the treatment effect for several statistical methods in substance abuse clinical trials. J Subst Abuse Treat. 2009;37(1):54–63. [PMC free article] [PubMed]
9. Delucchi KL. Methods for the analysis of binary outcome results in the presence of missing data. J Consult Clin Psychol. 1994;62(3):569–575. [PubMed]
10. Arndt S. Stereotyping and the treatment of missing data for drug and alcohol clinical trials. Subst Abuse Treat Prev Policy. 2009;4:2. [PMC free article] [PubMed]
11. Carroll KM, Rounsaville BJ, Nich C, Gordon LT, Wirtz PW, Gawin F. One-year follow-up of psychotherapy and pharmacotherapy for cocaine dependence. Delayed emergence of psychotherapy effects. Arch Gen Psychiatry. 1994;51(12):989–997. [PubMed]
12. Nich C, Carroll KM. Intention-to-treat meets missing data: Implications of alternate strategies for analyzing clinical trials data. Drug Alcohol Depend. 2002;68(2):121–130. [PMC free article] [PubMed]
13. Petry NM, Martin B, Cooney JL, Kranzler HR. Give them prizes, and they will come: Contingency management for treatment of alcohol dependence. J Consult Clin Psychol. 2000;68(2):250–257. [PubMed]
14. Higgins ST, Alessi SM, Dantona RL. Voucher-based incentives: A substance abuse treatment innovation. Addict Behav. 2002;27(6):887–910. [PubMed]
15. Dallery J, Silverman K, Chutuape MA, Bigelow GE, Stitzer ML. Voucher-based reinforcement of opiate plus cocaine abstinence in treatment-resistant methadone patients: Effects of reinforcer magnitude. Exp Clin Psychopharmacol. 2001;9(3):317–325. [PubMed]
16. Croft JR, Festinger DS, Dugosh KL, Marlowe DB, Rosenwasser BJ. Does size matter? Salience of follow-up payments in drug abuse research. IRB. 2007;29(4):15–19. [PubMed]
17. Festinger DS, Marlowe DB, Dugosh KL, Croft JR, Arabia PL. Higher magnitude cash payments improve research follow-up rates without increasing drug use or perceived coercion. Drug Alcohol Depend. 2008;96(1–2):128–135. [PMC free article] [PubMed]
18. Festinger DS, Marlowe DB, Croft JR, Dugosh KL, Mastro NK, Lee PA, Dematteo DS, Patapis NS. Do research payments precipitate drug use or coerce participation? Drug Alcohol Depend. 2005;78(3):275–281. [PubMed]
19. Winters KC, Stinchfield RD, Opland E, Weller C, Latimer WW. The effectiveness of the Minnesota Model approach in the treatment of adolescent drug abusers. Addiction. 2000;95(4):601–612. [PubMed]
20. Meyers K, Webb A, Frantz J, Randall M. What does it take to retain substance-abusing adolescents in research protocols? Delineation of effort required, strategies undertaken, costs incurred, and 6-month post-treatment differences by retention difficulty. Drug Alcohol Depend. 2003;69(1):73–85. [PubMed]
21. Lott DC, Jencius S. Effectiveness of very low-cost contingency management in a community adolescent treatment program. Drug Alcohol Depend. 2009;102(1–3):162–165. [PubMed]
22. Kaminer Y. Contingency management reinforcement procedures for adolescent substance abuse. J Am Acad Child Adolesc Psychiatry. 2000;39(10):1324–1326. [PubMed]
23. Woody GE, Poole SA, Subramaniam G, Dugosh K, Bogenschutz M, Abbott P, Patkar A, Publicker M, McCain K, Potter JS, Forman R, Vetter V, McNicholas L, Blaine J, Lynch KG, Fudala P. Extended vs short-term buprenorphine-naloxone for treatment of opioid-addicted youth: A randomized trial. JAMA. 2008;300(17):2003–2011. [PMC free article] [PubMed]
24. Zuur AF, Leno EN, Walker NJ, Saveliev AA, Smith GM. Mixed Effects Models and Extensions in Ecology in R. New York: Springer; 2009.
25. Anderson EE, DuBois JM. The need for evidence-based research ethics: A review of the substance abuse literature. Drug Alcohol Depend. 2007;86(2–3):95–105. [PubMed]