In this large, diverse sample of psychiatric outpatients, individuals who agreed to participate in a health promotion study differed from those who declined participation on several psychosocial and demographic characteristics. Relative to patients who declined study participation, patients who agreed to join the study were at higher risk for substance-related problems and were more likely to receive psychiatric care from clinics serving lower functioning patients. The only demographic variable associated with agreement to participate in the study was marital status, indicating that unmarried participants were somewhat more likely to consent for the study than married participants (univariate analysis only). A different set of predictors emerged in analyses of attrition. Among those who agreed to participate, individuals who actually completed the baseline study were older in age and were more likely to report a recent STD diagnosis relative to those who dropped out of the study. Psychiatric diagnosis also emerged as a predictor of study completion, with patients diagnosed with an adjustment disorder being more likely to drop out of the study.
We initially hypothesized that patients experiencing greater psychosocial dysfunction would be less
rather than more willing to participate and follow-through with research participation. In general, findings were contrary to this hypothesis, indicating a modest participation bias towards enrolling and retaining patients with greater impairment. Although researchers generally strive to enroll participants who are similar to non-participants with respect to key study outcomes, we view the present results as encouraging news for investigators working within psychiatric or community-based settings. That is, to the extent that those who are most in need of intervention are more likely to become research participants, investigators and consumers can more confidently conclude that research findings are generalizable to those patients with the greatest need for clinical services. In this respect, results appear to be most consistent with a small body of treatment outcome research indicating that individuals who volunteer for clinical research in mental health or substance abuse settings tend to be more severely impaired than those who decline participation (Rychtarik et al., 1998
; Shadish et al., 2000
). Thus, although the present study was not a treatment program, it is plausible that individuals with more severe psychopathology or substance abuse difficulties were motivated to join the study in hopes of obtaining personal benefit, support, or symptom relief. Given the pattern of findings, we also would have expected that patients with a more disabling psychiatric diagnosis (e.g., schizophrenia or bipolar disorder) would express greater interest in study participation. Although there were no differences in consent rates as a function of diagnosis, patients with the least disabling diagnosis (adjustment disorder) were more likely to discontinue the study.
Several studies involving the general population (Klesges et al., 1999
; Pullen et al., 1992
) raise concerns about the consequences of biases associated with a “healthy participant” effect, in which participants report healthier lifestyles or fewer risk factors than non-participants. In general, we found no evidence of the healthy participant effect, with one exception: younger aged participants were more likely to drop out of the study. To the extent that younger age is often associated with greater health-related risk taking (e.g., Vanable, Ostrow, McKirnan, Taywaditep, & Hope, 2000
), age differences in attrition point to a potential concern associated with under-representation of an important subgroup of participants who could benefit from psychosocial intervention. The other predictors of study completion (past history of an STD and non-adjustment disorder diagnosis) appear to be more consistent with the study consent findings, suggesting that those with a more disabling diagnosis and greater health risks are more likely to be retained. However, because the predictors of study consent and study completion were not identical, it will be important for future research to identify common underlying factors contributing to both study consent and completion.
Findings from the current research underscore the potential for population differences in receptivity to health-related research. Specifically, for research involving psychiatric patients, intervention or treatment effects could potentially be attenuated by the disproportionate enrollment of patients with more severe psychiatric or substance abuse difficulties. Given that these conditions correlate with higher risk behaviors, it is possible that this research appealed to those at highest risk for health-related problems or to those with less access to other health-related educational opportunities. Alternatively, lower functioning patients may have been more responsive to the modest financial incentives offered in exchange for participation. In terms of our own interests in health behavior change in the area of HIV risk, these findings may alleviate some concerns recently expressed in the literature (Auerbach & Coates, 2000
; DiFranceisco et al., 1998
) about problems associated with over-enrollment of participants with fewer HIV-related risk factors. Indeed, for the purposes of conducting health behavior research, we are less troubled by the relatively modest evidence of participation bias reported here because it suggests that those who might benefit most from a psychosocial or health behavior intervention are most likely to enroll. Nonetheless, for research conducted in psychiatric settings to be most broadly generalizable, our findings encourage additional efforts to enroll and retain higher functioning patients.
A major strength of this study concerns the fact the we report on archival chart data for individuals who chose not join the research project. Although not always feasible, results highlight the importance of considering differences between study joiners vs. non-joiners, as there are often a substantial number of individuals who refuse participation in clinical research programs. Other study strengths include the use of psychometrically validated measures, our large sample size, and the fact that patients were recruited from multiple psychiatric clinics within the community. Several limitations are also worth noting. Our use of average GAF scores from each clinic site as a means of categorizing patients as “lower” vs. “higher” functioning provided only a rough approximation of individual psychopathology level. Thus, although findings concerning the role of psychopathology and study participation were intriguing, they require replication and further exploration with improved indices of psychopathology. More generally, we note that our selection of predictor variables was limited to those that were assessed during a brief health screen. Future research should build on the findings reported here by developing and testing a more comprehensive theoretical model to clarify the psychological processes underlying decisions about study participation and attrition. Finally, the study could be improved upon by including follow-up data on participants who decline participation or who discontinue the study to identify reasons for their non-participation. Such qualitative data would help to provide additional context for the findings reported here.
The research described in this study required considerable planning and staffing resources directed towards the goal of recruiting and retaining representative research participants from psychiatric outpatient clinics. These results point toward the importance of considering the potential impact of (and possible remedies for) biases that arise from differential enrollment and dropout patterns. Besides taking steps to assure maximum enrollment and retention of all eligible participants, a number of researchers have moved towards the use of “intent to treat” analyses (Lachin, 2000
), in which efforts are made to report on outcome data from participants who fail to complete an intervention as a means increasing the external validity of findings. If such analyses are not possible, we suggest that researchers should, at a minimum, report observed differences between study completers and non-completers as a means of evaluating the degree to which external validity is threatened by differential enrollment and dropout patterns.