PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Suicide Life Threat Behav. Author manuscript; available in PMC 2012 June 5.
Published in final edited form as:
Suicide Life Threat Behav. 2011 April; 41(2): 203–209.
Published online 2011 February 2. doi:  10.1111/j.1943-278X.2010.00011.x
PMCID: PMC3367501
NIHMSID: NIHMS374916

Participation Bias among Suicidal Adults in a Randomized Controlled Trial

Abstract

Although individuals who attempt suicide have poor compliance rates with treatment recommendations, the nature and degree of participation bias in clinical treatment research among these individuals is virtually unknown. The purpose of this study was to examine participation bias by comparing the demographic and diagnostic characteristics of adult suicide attempters who participated in a randomized controlled trial to a sample of nonparticipants. Results indicated that males and individuals with a diagnosis of substance abuse or dependence were more likely to be participants in the randomized controlled trial. The implications of these findings for suicide intervention research are discussed.

The generalizability of findings from clinical trials is based on the assumption that research participants represent the population from which they are sampled. Biases from differential patterns of study enrollment among patient subgroups can lead to overestimates or underestimates of the effectiveness of an intervention. A critical research aim of effectiveness studies, therefore, is to examine the procedures and characteristics associated with participant recruitment. To understand the current limits of our knowledge of what works with whom, it is important to determine differences that exist between participants and nonparticipants in suicide prevention research. The examination of those factors associated with eligible participants’ refusal to participate in clinical trials, especially among individuals who are at risk for suicide, has important public health significance and can lead to more clinically representative research.

The nature and degree of participation bias among individuals who attempt suicide or intentionally injure themselves and who are recruited for clinical outcome studies is virtually unknown (for review, see Arensman et al., 2001). For example, among 32 randomized controlled trials with individuals who attempted suicide or deliberately harmed themselves, only 10 studies reported the proportion of individuals who refused to participate (Allard, Marshall, & Plante 1992; Battaglia et al., 1999; Evans et al., 1999; Guthrie et al., 2001; Hawton et al., 1981; MacLeod et al., 1998; Rotheram-Borus, Piacentini, Cantwell, Belin, & Song, 2000; Verkes et al., 1998; Waterhouse & Platt, 1990; Welu, 1977). For these studies, the proportion of eligible patients who refused to participate in the clinical trials ranged from 0% (Waterhouse & Platt, 1990) to 49% (Allard et al., 1992). Only one study examined the factors that may be associated with participation bias and found that there were no significant differences in demographic variables between the study participants and the study refuses (Welu, 1977). Additionally, to date, diagnostic differences between participants and nonparticipants in suicide research have not been adequately characterized.

Research on treatment compliance in nonresearch settings indicate that the extent of participation bias for individuals who attempt suicide may be substantial. Adults who attempt suicide and who are evaluated in emergency departments or other hospital settings have a very poor compliance rate, with follow-up outpatient appointments ranging from 20% to 40% (Jauregui, Martinez, Rubio, & Santo-Domingo, 1999; Kreitman, 1979; Morgan, Burns-Cox, Pocock, & Pottle, 1975; O’Brien, Holton, Hurren, & Watt, 1987). Low compliance rates have also been observed with adolescents who attempted suicide (King, Hovey, Brand, & Wilson, 1997). Such findings suggest that many individuals who attempt suicide would be unlikely to seek out or participate fully in interventions that are conducted through research.

To date, there is little evidence that specific diagnoses relate to the decision to participate or decline participation in research. However, research involving individuals being treated for psychiatric problems may be particularly vulnerable to participation bias (Patten, 2000; Vanable, Carey, Carey, & Maisto, 2002). Both symptom presentation and distress level may impact the decision to participate in research, particularly after a suicidal crisis. Those who are in greater distress or struggle with a greater number of psychiatric difficulties might feel more inclined to participate in research if they believe that the treatments being examined may reduce their distress. On the other hand, such individuals may feel overwhelmed and may be less inclined to participate in research that they believe involves too much of their time or energy. Research findings to date have been mixed regarding severity and participation in research. Candilis Geppert, Fletcher, Lidz, and Appelbau (2005) found that individuals with schizophrenia who had lower levels of severity were more likely to participate in clinical trials. Individuals with more severe psychosis were less likely to participate in psychiatric research in one study (Haapea et al., 2007). Other research has indicated that individuals with schizophrenia may be more likely to participate in low-risk research (Jaskiw et al., 2003). Individuals with comorbid diagnoses may be more likely to seek treatment (Tomasson & Vaglum, 2007; Wu, Kouzis, & Leaf, 1999), and participation may increase with severity of disorders (Shadish, Matt, Navarro, & Phillips, 2000). In addition, individuals who abuse substances may be more likely to participate in longitudinal research (Vanable et al., 2002). Higher severity was shown to correspond with more full research participation in an effectiveness study for alcohol dependence (Strohmetz, Alterman, & Walter, 1990).

Although data are scarce regarding differential participation in research among individuals who attempt suicide, demographic factors have been associated with lower rates of study participation in other literatures. Some research has suggested that African Americans are less likely to participate in medical and mental health research than individuals of other racial and ethnic backgrounds, possibly due to mistrust of the medical and research community, poor access to care, cultural barriers, and a failure to actively recruit African Americans (Shavers-Hornaday, Lynch, Burmeister, & Torner, 1997). In a randomized controlled trial of cognitive therapy for individuals who had attempted suicide, however, African Americans were less likely to refuse participation than individuals of other races (Brown et al., 2005), but the researchers were not able to determine reasons for the differential participation rates. There is also evidence to suggest that among individuals with serious mental illness, women may be less likely than men to participate in research (Mohr & Czobor, 2000; Robinson, Woerner, Pollack, & Lerner, 1996). Some research has suggested that symptom severity and social circumstances (e.g., homelessness, poverty, etc.) may lead to differential participation among patient subgroups. Individuals with fewer social resources have been found to be less likely to enroll in mental health by some researchers, (Farmer, Jackson, Camacho, & Hall, 2007; Perez, Ezpeleta & Domenech, 2007). Others have observed lower socioeconomic status among participants in substance abuse treatment trials (Rychtarik, McGillicuddy, Connors, & Whitney, 1998; Strohmetz et al., 1990) relative to individuals who decline participation in such research.

The assessment of bias in clinical trial samples through interview or survey research is challenging, as many nonparticipants refuse to speak with researchers or cite reasons for nonparticipation. In the present study, we sought to extend existing research on participation and selection bias by comparing the diagnostic and demographic characteristics of participants in an intervention study with those of nonparticipants who appeared to be eligible for participation, based on de-identified information obtained from medical records from the site at which recruitment took place.

METHOD

With the approval of the University of Pennsylvania Institutional Review Board, a review was conducted on electronic medical records of adult patients who attempted suicide and presented at a University of Pennsylvania emergency department, crisis response center, or inpatient psychiatric unit during the recruitment period for a study of cognitive therapy for suicide attempters (March 2002–December 2007). Data of individuals whose records indicated a probable suicide attempt were compared with the inclusion and exclusion criteria. The inclusion criteria were: (a) attempted suicide within 48 hours of presenting to an emergency department or trauma care unit, (b) age 16 or older, (c) able to speak English, and (d) able to understand the nature of the study and provide written informed consent. Exclusion criteria included: (a) self-mutilating behavior (e.g., burning oneself with a cigarette) without any intent to commit suicide, (b) an incapacity to participate in the study because of an acute, unstable, or severe Axis III disorder (e.g., organic brain damage), (c) a severe Axis I disorder that would prevent participation in outpatient psychotherapy (e.g., acutely delusional), or (d) Axis II disorder of mental retardation that would prevent an understanding of study procedures and participation requirements. Ninety-nine patients who appeared to meet inclusion criteria based on available information were included in the nonparticipant sample. Diagnostic and demographic data from the medical records of 140 patients who were entered into the clinical trial were used for comparison. After participants or eligible nonparticipants were identified in the medical record review, data were de-identified and the following data were obtained from the charts: age, gender, employment status, marital status, type of employment (if applicable), race, DSM–IV diagnoses (Axis I–IV), past suicide attempts, past hospitalization, housing status (homeless vs. not), recruitment site, and the method of suicide attempt.

RESULTS

The percentages of participants and nonparticipants by characteristics that were hypothesized to relate to study participation are presented in Table 1. The variables that were found to be significantly related to participation were gender, substance dependence, and psychiatric comorbidity (other than substance dependence). No other diagnostic or demographic differences were found between participants and nonparticipants. To determine the factors that were most uniquely associated with participation or nonparticipation, the three factors that were independently related to participation were entered into a logistic regression model. As indicated in Table 1, the overall model was significant, and substance use and gender were found to be significantly associated with participation. Odds ratios resulting from the logistic regression indicate that individuals with substance dependence diagnoses were 64% more likely to be study participants than those without a diagnosis. In addition, females were 97% less likely to be study participants than males.

TABLE 1
Chi-Square and Logistic Regression Results for Study Participants and Nonparticipants (N = 239)

DISCUSSION

This study extends the current literature by examining both diagnostic and demographic differences between participants and nonparticipants in a clinical trial among individuals who have attempted suicide, using a method that is intended to draw a representative sample of nonparticipants who presented for emergency care during the recruitment period of the study. Charts of a diverse sample of individuals who presented following a suicide attempt at a large, urban hospital system were evaluated. Consistent with previous research examining participation bias among nonsuicidal individuals (Mohr & Czobor, 2000; Robinson et al., 1996), females were less likely to become research participants than males. Substance dependence was the only diagnostic variable that was found to be significantly associated with participation. These findings suggest that males and individuals with substance abuse or dependence diagnoses may be overrepresented in suicide intervention studies. Reasons for this overrepresentation cannot be discerned from the available data, but potential reasons include a perceived high need for intervention among individuals who use substances, particularly if problems related to substance use precipitated the attempt. A potential alternative explanation that must be considered is that those who misuse substances had an external incentive to pursue financial compensation for time spent participating in study-related assessments, in which case researchers targeting suicidal individuals in future studies should consider appropriate compensation carefully. The provision of compensation and rewards for participation may yield a sample that is more representative of a broader population in terms of motivation (Sharp, Pelletier, & Levesque, 2006), but may lead to an under-representation of the motivation level and different characteristics of individuals who decline treatment or engage in treatment outside the context of a study after a suicide attempt. Further efforts are warranted to examine the characteristics and reasons for refusal among those eligible individuals who decline participation in suicide research. Qualitative and quantitative data obtained through interviews and other research formats would help to provide additional context for the findings reported here.

We note some important limitations to this study. First, the findings are limited to individuals who were evaluated at an emergency department or trauma care unit of a hospital and may not generalize to those who attempt suicide but do not receive medical evaluations, or those who had such acute injuries that they were not medically stable enough to speak with a researcher within the window of time specified for recruitment. Furthermore, as the current study was conducted in an urban setting, it will be important to examine potential differences between urban and nonurban samples in future suicide research. Complete data were not available for two of the variables studied, homelessness and employment status, resulting in a smaller samples size for these analyses. Further research will be needed to determine whether differences in socioeconomic status exist between those who participate in suicide research and those who do not. Another limitation is that associated with the use of medical records rather than standardized diagnostic interviews to obtain diagnostic information. Despite these limitations, administrative data offer rich opportunities to estimate the applicability of the treatment outcome literature to a large sample of patients, and can inform the development of treatments for under-studied populations, particularly when used in conjunction with other studies that estimate the applicability of the literature using other methods.

Our results indicate that caution should be used in considering the effectiveness of suicide intervention research for underrepresented populations. Additional efforts to increase the likelihood of participation are needed to specifically target females and those with diagnoses other than substance disorders who may be more reluctant to enter suicide intervention trials. To increase the extent to which findings can generalize to clinical populations, it will be important to explore potential differences in secondary analyses and target underrepresented individuals in future research. The development of innovative methods for examining sample bias among clinical populations will be central to making further progress in understanding the nature of sample bias in specific clinical populations.

Acknowledgments

This research was conducted at the Psychopathology Research Unit in the Department of Psychiatry, University of Pennsylvania. This research was supported by grants from the National Institute of Mental Health (R01 MH60915, K99 MH080100, and P20 MH71905) and the Centers for Disease Control and Prevention (R37 CCR316866).

Contributor Information

Shannon Wiltsey Stirman, Women’s Health Sciences Division, National Center for PTSD, Boston University, Boston, MA, USA.

Gregory K. Brown, Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA.

Marjan Ghahramanlou-Holloway, Medical and Clinical Psychology, Uniformed Services University of the Health Sciences, Bethesda, MD, USA.

Allison J. Fox, Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA.

Mariam Zahid Chohan, Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA.

Aaron T. Beck, Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA.

REFERENCES

  • Allard R, Marshall M, Plante MC. Increase follow-up does not decrease the risk of repeat suicide attempts. Suicide and Life-Threatening Behavior. 1992;22:303–315. [PubMed]
  • Arensman E, Townsend E, Hawton K, Bremner S, Feldman E, Goldney R, et al. Psychosocial and pharmacological treatment of patients following deliberate self-harm: The methodological issues involved in evaluating effectiveness. Suicide and Life-Threatening Behavior. 2001;31:169–180. [PubMed]
  • Battaglia J, Wolf TK, Wagner-Johnson DS, Rush AJ, Carmody TJ, Basaco MR. Structured diagnostic assessment and depot fluphenazine treatment of multiple suicide attempters in the emergency department. International Clinical Psychopharmacology. 1999;14:361–372. [PubMed]
  • Brown GK, Ten Have T, Henriques GR, Xie SX, Hollander JE, Beck AT. Cognitive therapy for the prevention of suicide attempts: A randomized controlled trial. Journal of the American Medical Association. 2005;294:563. [PubMed]
  • Candilis PJ, Geppert CMA, Fletcher KE, Lidz CW, Appelbau PS. Willingness of subjects with thought disorder to participate in research. Schizophrenia Bulletin. 2005;32:1. [PMC free article] [PubMed]
  • Evans K, Tyrer J, Catalan J, Schmidt U, Davidson K, Dent J, et al. Manual-assisted cognitive-behavior therapy (MACT): A randomized controlled trial of brief intervention with bibliotherapy in the treatment of recurrent deliberate self-harm. Psychological Medicine. 1999;29:19–25. [PubMed]
  • Farmer DF, Jackson SA, Camacho F, Hall MA. Attitudes of African American and low socioeconomic status white women toward medical research. Journal of Health Care for the Poor and Underserved. 2007;18:85–99. [PubMed]
  • Guthrie E, Kapur N, Mackway-Jones K, Chew-Graham C, Moorey J, Mendel E, et al. Randomized controlled trial of brief psychological intervention after deliberate self-poisoning. British Medical Journal. 2001;323:1–4. [PMC free article] [PubMed]
  • Haapea M, Miettunen J, Veijola J, Lauronen E, Tanskanen P, Isohanni M. Non-participation may bias the results of a psychiatric survey: An analysis from the survey including magnetic resonance imaging within the Northern Finland 1966 Birth Cohort. Social Psychiatry and Psychiatric Epidemiology. 2007;42:403–409. [PubMed]
  • Hawton K, Bancroft J, Catalan J, Kingston B, Stedeford A, Welch N. Domiciliary and out-patient treatment of self-poisoning patients by medical and non-medical staff. Psychological Medicine. 1981;11:169–177. [PubMed]
  • Jaskiw GE, Blumer T, Gutierrezesteinou R, Meltzer HY, Steele V, Strauss ME. Comparison of inpatients with major mental illness who do and do not consent to low-risk research. Psychiatry Research. 2003;119:183–188. [PubMed]
  • Jauregui J, Martinez ML, Rubio G, Santo-Domingo J. Patients who attempted suicide and failed to attend mental health centers. European Psychiatry. 1999;14:205–209. [PubMed]
  • King CA, Hovey JD, Brand E, Wilson R. Suicidal adolescents after hospitalization: Parent and family impacts on treatment follow-through. Journal of the American Academy of Child & Adolescent Psychiatry. 1997;36:85–93. [PubMed]
  • Kreitman N. Reflections on the management of parasuicide. British Journal of Psychiatry. 1979;135:275–277. [PubMed]
  • MacLeod AK, Tata P, Evans K, Tyrer P, Schmidt U, Davidson K, et al. Recovery of positive future thinking within a high-risk parasuicide group: Results from a pilot randomized controlled trial. British Journal of Clinical Psychology. 1998;37:371–379. [PubMed]
  • Mohr P, Czobor P. Subject selection for the placebo- and comparator-controlled trials of neuroleptics in schizophrenia. Journal of Clinical Psychopharmacology. 2000;20:240–245. [PubMed]
  • Morgan HG, Burns-Cox CJ, Pocock H, Pottle S. Deliberate self-harm: Clinical and socio-economic characteristics of 368 patients. British Journal of Psychiatry. 1975;127:564–574. [PubMed]
  • O’brien G, Holton AR, Hurren K, Watt L. Deliberate self-harm and predictors of out-patient attendance. British Journal of Psychiatry. 1987;150:246–247. [PubMed]
  • Perez RG, Ezpeleta L, Domenech JM. Features associated with the nonparticipation and drop out by socially-at-risk children and adolescents in mental-health epidemiological studies. Social Psychiatry and Psychiatric Epidemiology. 2007;42:251–258. [PubMed]
  • Patten SB. Selection bias in studies of major depression using clinical subjects. Journal of Clinical Epidemiology. 2000;53:351–357. [PubMed]
  • Robinson D, Woerner MG, Pollack S, Lerner G. Subject selection biases in clinical trials: Data from a multicenter schizophrenia treatment study. Journal of Clinical Psychopharmacology. 1996;16:170–176. [PubMed]
  • Rotheram-Borus MJ, Piacentini J, Cantwell C, Belin TR, Song J. The 18-month impact of an emergency room intervention for adolescent female suicide attempters. Journal of Consulting and Clinical Psychology. 2000;68:1081–1093. [PubMed]
  • Rychtarik RG, Mcgillicuddy NB, Connors GJ, Whitney RB. Participant selection biases in a randomized clinical trial of alcoholism treatment settings and intensities. Alcoholism, Clinical and Experimental Research. 1998;22:969–973. [PubMed]
  • Shadish WR, Matt GE, Navarro AM, Phillips G. The effects of psychological therapies under clinically representative conditions: A meta-analysis. Psychological Bulletin. 2000;126:512–529. [PubMed]
  • Sharp EC, Pelletier LG, Levesque CS. The double-edged sword of rewards for participation in psychology experiments. Canadian Journal of Behavioural Science/Revue canadienne des Sciences du comportement. 2006;38:269–277.
  • Shavers-Hornaday VL, Lynch CF, Burmeister LF, Torner JC. Why are African Americans under-represented in medical research studies? Impediments to participation. Ethnicity & Health. 1997;2:31–45. [PubMed]
  • Strohmetz DB, Alterman AI, Walter D. Subject selection bias in alcoholics volunteering for a treatment study. Alcoholism, Clinical and Experimental Research. 1990;14:736–738. [PubMed]
  • Tomasson K, Vaglum P. A nationwide representative sample of treatment-seeking alcoholics: A study of psychiatric comorbidity. Acta Psychiatrica Scandinavica. 2007;92:378–385. [PubMed]
  • Vanable PA, Carey MP, Carey KB, Maisto SA. Predictors of participation and attrition in a health promotion study involving psychiatric outpatients. Journal of Consulting and Clinical Psychology. 2002;70:362–368. [PMC free article] [PubMed]
  • Verkes RJ, Van der Mast RC, Hengeveld MW, Tuyl JP, Zwinderman AH, Kempen GM. Reduction by paroxetine of suicidal behavior in patients with repeated suicide attempts but not major depression. American Journal of Psychiatry. 1998;155:543–547. [PubMed]
  • Waterhouse J, Platt S. General hospital admission in the management of parasuicide: A randomized controlled trial. British Journal of Psychiatry. 1990;156:236–242. [PubMed]
  • Welu TC. A follow-up program for suicide attempters: Evaluation of effectiveness. Suicide and Life-Threatening Behavior. 1977;7:17–30. [PubMed]
  • Wu L-T, Kouzis AC, Leaf PJ. Influence of comorbid alcohol and psychiatric disorders on utilization of mental health services in the National Comorbidity Survey. American Journal of Psychiatry. 1999;156:1230–1236. [PubMed]