Repeated Attempts using Different Strategies are Important for Timely Contact with Study Participants

doi:10.1016/j.jclinepi.2010.08.007

J Clin Epidemiol. Author manuscript; available in PMC 2012 October 1.

Published in final edited form as:

J Clin Epidemiol. 2011 October; 64(10): 1144–1151.

Published online 2010 December 15. doi: 10.1016/j.jclinepi.2010.08.007

PMCID: PMC3116960

NIHMSID: NIHMS236511

Repeated Attempts using Different Strategies are Important for Timely Contact with Study Participants

Kuan-Fu Chen, MD, Elizabeth Colantuoni, PhD, Faisal Siddiqi, MD, Victor D. Dinglas, BS, Kristin A. Sepulveda, BA, Eddy Fan, MD, Peter Pronovost, MD, PhD, and Dale M. Needham, MD, PhD

Department of Emergency Medicine, Chang-Gung University and Chang-Gung Memorial Hospital, Taoyuan, Taiwan (KFC), and the Department of Emergency Medicine (KFC); OACIS Group, Division of Pulmonary and Critical Care Medicine (EF, FS, KS, VD, DMN); Department of Anesthesiology and Critical Care Medicine (PJP); and Department of Biostatistics (EC), Johns Hopkins University, Baltimore, MD, USA

Corresponding Author (Reprints Not Available from Author): Dale Needham, MD, PhD, Pulmonary and Critical Care Medicine, Johns Hopkins Medical Institutions, 1830 E. Monument Street, 5^th floor, Baltimore, MD 21205, Phone: 410-955-3467, Fax: 410-955-0036, Email: dale.needham/at/jhmi.edu

The publisher's final edited version of this article is available at J Clin Epidemiol

See other articles in PMC that cite the published article.

Publisher's Disclaimer

Abstract

Objective

To study the effect of different mail- and phone-based strategies, along with patient- and research-related factors, on the time to contact with research participants.

Study Design and Setting

A prospective evaluation of a 12-week standardized protocol (embedded with two randomized trials of mail- and phone-based strategies) for contacting existing research participants for recruitment into a related study.

Result

Of 146 participants, 87 were eligible for contact via the standardized protocol and 63 (72%) of these were successfully contacted within 12 weeks after multiple mail- and phone-based efforts. Using Cox proportional hazards regression analysis, the different mail and phone strategies showed no significant difference in the time to contact with participants. Of 34 patient- and research-related factors evaluated, only 2 were independently associated with time to contact among all 146 participants: (1) participants having their last visit conducted outside of the research clinic due to patient illness/condition had a longer time to contact; and (2) those with a self-reported chronic fatigue history had a shorter time to contact.

Conclusion

Few patient characteristics and research-related factors accurately predict time to contact. Repeated attempts using different strategies are important to successful and timely contact with study participants.

Keywords: patient selection, risk factors, cohort studies, patient participation, respiratory distress syndrome, adult

1. Introduction

Poor recruitment of research subjects often leads to underpowered studies and/or added cost due to extending the timeframe for subject recruitment (1). Approximately 50% of clinical trials, observational studies and laboratory-based experimental studies have difficulties with recruitment (2). Researchers must make timely and successful contact with potential research participants as a first step in achieving the goal of timely and complete recruitment of a representative study population (3-5).

Repeated attempts and use of different strategies may be helpful in contacting participants in both survey-based research (6) and clinical studies (2). However, a recent Cochrane review concludes that it is difficult to predict the effectiveness of many strategies used by trialists to improve subject recruitment. Although only three of 27 trials in the Cochrane review evaluated interventions aimed at recruiting participants within health care research, the authors suggest the following procedures, where feasible, to increase recruitment in clinical trials: telephone reminders; use of opt-out, rather than opt-in, procedures for contacting potential trial participants; and non-masked treatment allocation whereby participants would know which treatment they are receiving. Telephone reminders to non-responders were effective in recruiting subjects in one clinical trial (7), but the use of personalized invitation letters did not have a significant effect in another study (8). Overall, there is little empirical evidence regarding successful methods for contacting participants being recruited for clinical studies.

Several studies have attempted to characterize which potential research subjects participate or respond to research invitations, in order to develop better strategies at identifying and allocating limited resources for study recruitment (4, 9-19). Among these studies, responders tended to be younger (4, 14, 15), male (9, 15), white (14), employed (10, 13, 16), more highly educated (14), less healthy (17), less immobile (9), and less likely to have psychiatric (18), alcohol dependence (19) or problems. However, other studies could not confirm some of these findings (4, 12, 15, 18).

Most of prior studies in this field were conducted retrospectively and are limited by incomplete information on potential subjects who did not participate. As a result, we sought to prospectively study the effect of different mail- and phone-based strategies for contacting participants and the factors affecting the time to contact with them. We designed a study to evaluate a standardized mail- and phone-based contact protocol, embedded with two small randomized trials of contact strategies, applied within an existing population of research subjects who survived critical illness due to acute lung injury/acute respiratory distress syndrome (ALI/ARDS).

2. Methods

2.1. Study population

From 2004 – 2007, consecutive patients with ALI/ARDS from 13 ICUs at four hospitals located predominantly in inner city Baltimore, USA were enrolled into a prospective cohort study evaluating survivors’ long-term physical and mental health outcomes (20). A total of 224 consenting patients who survived until hospital discharge were evaluated at 3, 6, 12, and 24 months after ALI/ARDS, with only 3 patients lost to follow-up during this 2-year study period. However, similar to other studies in this field (21), ongoing mortality after hospital discharge of these ALI/ARDS patients was high with >30% of hospital survivors dying by 24 month follow-up. In September 2008, a new National Institutes of Health grant was awarded to extend this cohort’s follow-up duration until 5 years after ALI/ARDS. Because patients were enrolled prospectively over time, some participants had completed their 24 month visit while others were still awaiting 12 and 24 month visits at the time that this new funding was approved. All subjects were required to sign a new consent form for participation in the extended follow-up study.

In this context, a protocol was designed to prospectively evaluate the factors associated with successfully contacting subjects to discuss the new, extended follow-up study and the effect of different mail- and phone-based strategies for repeatedly contacting subjects. This evaluation specifically focused on the time to contact with participants, rather than the time to informed consent, because: 1) not all participants were required to consent at the time that we contacted them (i.e. 44% were still under-going 12- and 24-month follow-up visits and would not require consent for up to 2 years later), and 2) prior experience with this cohort indicated that a high rate of informed consent was likely provided that we could successfully contact these research subjects on a timely basis. All participating survivors from the original study, as of September 2008, were included in this prospective evaluation.

2.2. Standardized protocol to contact participants

Over a 12-week period, a multi-step protocol was designed and implemented to repeatedly contact existing subjects to discuss potential participation in the extended follow-up study. First, a general letter was sent to the 146 participating survivors remaining in the study as of September 2008. This general letter described the new funding for a 3-year extension to the follow-up duration and requested that the participant contact the study staff to get additional information about this extension. Second, non-responders to the general letter, who were not due for a follow-up visit within the next 90 days, then entered a “mail trial” (as described below). Finally, the remaining non-responders to the “mail trial,” who were not due for a follow-up visit within 90 days, then entered a “phone trial” (as described below). Participants with a follow-up visit due within 90 days were specifically excluded from subsequent steps of the standardized follow-up protocol in order to permit customized, intensive efforts to contact them for consent and enrollment into the study to ensure that their impending follow-up visit was not missed.

2.3. The Mail Trial

The mail trial started one month after the general letter was mailed to subjects. In the mail trial, subjects were randomized to receive one of two different envelope formats which enclosed an identical contact letter: a “business format” in which the address label was typed and a stamping machine added postage to the envelope versus a “personal format” in which the address was hand written and a traditional stamp was affixed to the envelope. Similar to the general letter, in this mail trial, the letter briefly explained the extended follow-up duration to participants and asked them to contact the research staff to get additional information.

2.4. The Phone Trial

The phone trial started one month after the mail trial to permit adequate time for participants to contact the study staff. In the phone trial, participants were telephoned by a single person, once weekly for 4 weeks, until the subject was reached via the phone call or the subject returned the call. When participants could not be reached via this phone call, a brief message (using a written script) was left with the person who answered the phone or via the participant’s voicemail. Participants were randomized to identical messages with an exception that the research staff member said that she was calling on behalf of either “ICAP” (the name of the study) or “Dr. Peter Pronovost” (the principal investigator). At the mid-point of this phone trial, non-respondents crossed over to receive the alternate phone message.

2.5. Measurement of outcome and predictor variables

The outcome variable for this analysis was the time, in days, until phone contact was made with the subject (i.e. the subject called to study staff after receipt of the letter or the phone message, or was directly reached during a call conducted as part of the phone trial). Time zero was defined as the mailing date of the general letter sent to all subjects.

A total of 34 potential predictors of the time to contact evaluated in this analysis were determined on an a priori basis from among all relevant variables available within the parent study including data obtained during the participant’s original intensive care unit (ICU) admission or during their preceding follow-up visits. The predictor variables were grouped into four general categories: demographic/socioeconomic (12 predictors), pre-ALI/ARDS comorbidity (4 predictors), medical status at last follow-up (7 predictors), and research visit history (11 predictors). Demographic/socioeconomic factors included age, sex, race, hospital site from which the participant was recruited, years of education, and measures of the participant’s employment status as at their last follow-up visit. Pre-ALI/ARDS comorbidity data were obtained from medical records at the original ICU admission and included 4 factors: overall comorbidity burden measured using the Charlson Index (22), documented drug abuse, alcohol abuse, or any other psychiatric disorder (e.g., depression, anxiety disorder). Post-ALI/ARDS medical status factors, measured at the last follow-up visit, included 7 potential predictors: immobility (self-reported inability to transfer from bed and/or chair), hearing impairment (self-reported inability to hear the television or radio), use of home oxygen, self-report of chronic dyspnea and chronic fatigue, requirement of assistance with activities of daily living at home, and living in a health care facility. Lastly, potentially relevant data from the participant’s research visit history included 7 factors from the last visit completed: number of contact telephone numbers available (count variable), last visit conducted in clinic (binary variable), last visit not conducted in clinic due to patient illness/condition (binary variable), incomplete data collection or missed visit due to specific patient factors (2 separate binary variables), reason for non-attendance at a scheduled visit (3-level categorical variable) and missed last follow-up visit (binary variable). In addition, from the participant’s entire visit history over the course of the study, 2 binary factors were analyzed: any prior home visit, and any prior missed follow-up visit.

2.6. Statistical Analysis

Cox proportional hazards regression models were used to evaluate the association between the time to contact with the participant and (1) assigned mail- and phone-based contact strategies within the randomized trials and (2) a priori potential predictors described above. Participants who died during the protocol, could not be contacted by the end of the 12-week protocol, or were due for immediate follow-up and did not continue through the remainder of protocol were censored. In the phone trial, patients who answered the phone and never received a phone message were excluded from analysis of the effect of the randomized phone strategies. The planned cross-over of the randomized telephone message during the phone trial was analyzed by treating this exposure as a time-varying variable in the Cox model. The association of individual predictors with the time to contact was evaluated by simple Cox models. Predictors with a p-value ≤0.2 were evaluated in multivariable Cox models. To avoid overfitting the multivariable model, two separate interim models were developed for each of the factors related to participant characteristics and to the research visit history. The final multivariable Cox model was created using predictors with a p-value ≤ 0.2 in either of the two interim models.

The proportional hazards assumption in the final multivariable model was assessed by examining the complementary log-log survival curves for each relevant covariate. Plots of the Martingale residuals versus the continuous covariates in the final multivariable model were used to validate linearity assumptions within the Cox model. Outliers or influential points were assessed using the efficient score residuals. Multicollinearity was evaluated with variance inflation factors. Statistical significance was defined as a 2-sided p-value <0.05. All statistical analyses were performed using Stata 11.0 (Stata Corp, College Station, TX). The institutional review board at Johns Hopkins University approved this research.

3. Results

3.1. Overall recruitment

All 146 survivors eligible to be contacted for recruitment were sent general letters, and 23 (16%) responded (see Figure). Non-responders were then sequentially contacted via the mail trial and the phone trial in which 5 (6%) and 20 (47%) responded, respectively. After excluding one participant who died before contact, one who was unintentionally excluded and 58 who were removed because of a follow-up visit that was due within 90 days (31 in the mail trial and 27 in the phone trial), 63 of 87 (72%) eligible participants were contacted with the standardized protocol described in Methods (see Figure). Of these 63 participants contacted, 60 (95%) provided consent to participate in the extended study. The customized, intensive follow-up efforts conducted for the 58 participants removed from the standardized protocol, resulted in 44 (76%) being contacted by the end of the 12-week period, of whom 38 (86%) provided consent.

Figure

Study flow diagram

Participant characteristics and other proposed predictors of the time to contact are presented in Table 1. In general, participants were middle-aged (median 47 years old), Caucasian (n=84, 57.5%), unemployed (n=106, 72.6%) with at least a high school education (n=108, 74%). Drug or alcohol abuse, and other psychiatric conditions were common, with 64 (44%) of participants having a history of drug and/or alcohol abuse. The most commonly reported medical status issues at the last follow-up visit were patient-reported chronic fatigue (n=57, 39.3%) and chronic dyspnea (n=51, 35.2%). The majority of patients had never missed any follow-up visit (n=101, 69.2%), but 58 patients (39.7%) had required at least one home visit during follow-up. At the last scheduled research visit, most subjects had ≥3 telephone contact numbers available, had attended the visit, and had complete data collection. For 15 (10.3%) patients with non-attendance due to personal factors and 43 (29%) patients with incomplete data collection at the last visit, the most common reasons included: incarceration, being on mechanical ventilation, or being incapable due to fatigue, cognitive, physical or psychiatric issues.

Table 1

Characteristic of study population and associations with time to participant contact

3.2. Randomized mail and phone trials

Baseline characteristics of patients randomized within each of the mail and phone trials showed no important differences (data not shown). The overall rate of contact with participants in the phone trial (n=20, 47%) is higher than the rates from both the preceding general letter mailed to subjects (n=23, 16%) and the mail trial (n=5, 6%) (see Figure). No significant differences in the time to contact could be detected in either of these trials with a non-significantly shorter time to contact (hazard ratio [HR], 95% confidence interval [CI]) for patients receiving envelopes with a “personal” versus “business” format (3.52, 0.39 – 31.5) and for those receiving a phone message with the study name versus the principal investigator’s name (1.29, 0.53 – 3.14).

3.3. Predictors of time to contact

Given non-significant results from the mail and phone trials, analysis of the predictors of time to contact was conducted for all participants together regardless of their randomization status for each of the two trials. In simple Cox regression models, 13 predictors had a potential association (p ≤ 0.20) with the time to contact (Table 1), and three factors were significantly associated (p ≤ 0.05): patient-reported chronic fatigue (HR, 95% CI: 1.69, 1.03-2.79), last visit not conducted in clinic due to patient illness/condition (0.40, 0.19-0.84), and ever missed any follow-up visit (0.52, 0.29-0.94). The 13 predictors with a potential association with the time to contact were further categorized into two interim multivariable Cox models (Table 2) to evaluate factors independently associated with time to contact. In the final multivariable Cox model, only two predictors (HR, 95% CI) were significant: (1) participants with their last visit not conducted in clinic due to patient illness/condition had a longer time to contact (0.40, 0.19-0.84) and (2) those with self-reported chronic fatigue had a shorter time to contact (1.85, 1.12-3.06).

Table 2

Predictors of time to participant contact

4. Discussion

This prospective cohort study incorporated a standardized recruitment protocol, embedded with two small randomized controlled trials of participant contact strategies, to evaluate methods and predictors of time to contact with participants for enrollment into an extended duration longitudinal study. In the two small randomized trials, we could not detect any significant difference between “personal” versus “business” formats of the envelope used for a standard recruitment letter, nor between phone messages using the study name versus the principal investigator’s name. Among 34 potential predictors examined in this study, only two were independently associated with time to contact with participants. Having the participant’s last visit conducted outside of the research clinic due to patient illness/condition, and not having a self-reported history of chronic fatigue were each independently associated with a longer time to contact.

In our study, the overall rate of contact with participants by the end of the 12-week standardized recruitment protocol was relatively high at 72% (of whom 95% provided informed consent), compared to the 6 - 60% rates experienced in other trials of recruitment strategies (8, 23). We attribute this relatively high rate of contact to our repeated contact attempts and the use of different methods to contact patients. However, participant contact rates, within a 1-month time frame, to each of the general recruitment letter and the mail trial were low (16% and 6% respectively). We observed a relatively higher contact rate with the phone versus mail strategies. The low response rates to our two letters may be explained, in part, by 44% of participants still undergoing their 12- or 24-month follow-up visits at the time of the letter (i.e. they may not have responded simply because they had not yet reached the end of the original study period in order to be eligible for enrollment in the extended follow-up study). However, the relatively high prevalence of chronic illness and/or drug or alcohol use in our population also may have played a role in the low rate of participant response to the letters.

To our knowledge, only a single study has evaluated factors predicting the time to participant response. Tennant el al. studied factors associated with the time to response to a postal questionnaire (5) and found that participants who were physically dependent responded sooner. Other research in this field has studied factors associated with subjects consenting to participate in research (with no investigation of the time required to contact subjects and obtain consent). Predictors of consent and its timing were not specifically evaluated in our study for two reasons: (1) not all participants were required to consent at the time that we contacted them (i.e. those still under-going 12- and 24-month follow-up would not require consent for up to 24 months later), and (2) based on experience with our cohort, we believed that the greatest barrier to participation was successfully contacting participants, rather than their reluctance to provide informed consent, as demonstrated by the very high consent rate obtained among all participants who were successfully contacted. Nonetheless, there are a number of potentially relevant studies within this larger body of literature to discuss. For example, Janzon et al. found that patient illness was negatively associated with participation (11) similar to the finding in our evaluation of time to contact with participants. However, our other predictor of time to contact (i.e. a history of patient-reported fatigue) has not been specifically investigated in prior research, but was considered within our analysis based on the high prevalence of this symptom among ALI/ARDS survivors (24).

While other studies have found either positive or negative associations of socioeconomic and health-related factors, such as age, sex, race, educational level, employment status, current health status, immobility, and alcohol addiction or other psychiatric history, with participation in research (4, 8-10, 12-19), we found no association of any of these variables with our time to contact with participants. Moreover, although missing a prior follow-up visit was negatively associated with participation in prior research (10), this factor was not associated with time to contact in our study. Conflicting results within this literature and with our study findings may be due to the different end point studied in our research, or differences in the target participant populations evaluated in each study. Findings may not be readily generalizable outside of the specific participant population.

Regarding our finding that patients with a chronic fatigue history had a shorter time to contact, we speculate that this result may be due to chronically fatigued participants more frequently being at home to receive a phone call or more frequently having assistance at home that may help with phone-based responsiveness. However, these hypotheses need further evaluation to confirm their association with the time to contact of participants.

There are several potential limitations of our study. First, our study was focused on time to contact with members of an existing research cohort for participation in an extended follow-up protocol, which may limit the generalizability of our findings. However, the circumstances of our study (i.e., needing to recruit patients from within an existing cohort) are not unusual in clinical research (21, 25, 26); therefore, the findings from this study still have importance and relevance. Second, our study had a limited sample size and a relatively low response rate to the two randomized trials making the trials likely under-powered to detect an important effect of the interventions evaluated. This low response rate was not anticipated when designing the trials and our available sample size was constrained to those subjects already participating and eligible for the extended duration study. Further randomized trials investigating the effect of different participant contact strategies are needed. Third, some of the predictors evaluated in this study were measured at the participant’s immediately preceding research visit. These predictors may have changed since that time resulting in a misclassification error. However, this potential misclassification is likely infrequent given the relative stability of these medical-related factors in this patient population and the relatively short median time (7 months) between participants’ last visit and our contact with them. Lastly, we focused on the time to participant contact, rather than the rate of contact or recruitment/informed consent because we believed it was the key underlying factor in achieving participation in our cohort and because some participants were still undergoing 12- and 24-month follow-up visits, making consent for the extended follow-up protocol a less relevant end point. Since most studies have a time-sensitive nature for patient contact and recruitment, understanding the time to participant contact and the factors affecting it are important for designing contact strategies.

5. Conclusion

Repeated attempts to contact participants and use of different strategies over a 12-week time period are important for successfully contacting a high proportion of patients regarding study participation. Despite evaluating 34 potentially relevant demographic, socioeconomic, medical, and research-related factors, only 2 were independently associated with a longer time to contact with participants: having participant’s last visit conducted outside of the research clinic due to patient illness/condition and not having a self-reported chronic fatigue history. Thus, repeated attempts using different strategies are important to successful and timely contact with participants, particularly when few patient and research visit factors can be empirically demonstrated to predict time to contact.

What is new?

Key findings

Repeated attempts using both mail- and phone-based strategies are important for successful and timely contact with research participants.
A longer time to contact with participants was associated with only 2 of 34 potentially relevant patient and research visit characteristics evaluated: (1) not having a self-reported history of chronic fatigue; and (2) having last research visit conducted outside of the research clinic due to patient illness/condition.

What was known

Timely and successful contact with potential research participants is important in rapidly recruiting a representative study population, but factors affecting time to contact with participants are unclear.

What is the implication, what should change now

Researchers should consider repeated attempts using different strategies over a prolonged period in contacting potential participants for study recruitment since few patient and research visit characteristics could predict time to participant contact.

Footnotes

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

1. Treweek S, Mitchell E, Pitkethly M, Cook J, Kjeldstrom M, Taskila T, et al. Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev. 2010;(1) MR000013. [PubMed]

2. Easterbrook PJ, Matthews DR. Fate of research studies. J R Soc Med. 1992 Feb;85(2):71–6. [PMC free article] [PubMed]

3. Forthofer RN. Investigation of nonresponse bias in NHANES II. Am J Epidemiol. 1983 Apr;117(4):507–15. [PubMed]

4. Jacobsen SJ, Mahoney DW, Redfield MM, Bailey KR, Burnett JC, Jr., Rodeheffer RJ. Participation bias in a population-based echocardiography study. Ann Epidemiol. 2004 Sep;14(8):579–84. [PubMed]

5. Tennant A, Badley EM. Investigating non-response bias in a survey of disablement in the community: implications for survey methodology. J Epidemiol Community Health. 1991 Sep;45(3):247–50. [PMC free article] [PubMed]

6. Kiezebrink K, Crombie IK, Irvine L, Swanson V, Power K, Wrieden WL, et al. Strategies for achieving a high response rate in a home interview survey. BMC Med Res Methodol. 2009;9:46. [PMC free article] [PubMed]

7. Nystuen P, Hagen KB. Telephone reminders are effective in recruiting nonresponding patients to randomized controlled trials. J Clin Epidemiol. 2004 Aug;57(8):773–6. [PubMed]

8. Kiernan M, Phillips K, Fair JM, King AC. Using direct mail to recruit Hispanic adults into a dietary intervention: an experimental study. Ann Behav Med. 2000 Winter;22(1):89–93. [PubMed]

9. Boshuizen HC, Viet AL, Picavet HS, Botterweck A, van Loon AJ. Non-response in a survey of cardiovascular risk factors in the Dutch population: determinants and resulting biases. Public Health. 2006 Apr;120(4):297–308. [PubMed]

10. Eagan TM, Eide GE, Gulsvik A, Bakke PS. Nonresponse in a community cohort study: predictors and consequences for exposure-disease associations. J Clin Epidemiol. 2002 Aug;55(8):775–81. [PubMed]

11. Janzon L, Hanson BS, Isacsson SO, Lindell SE, Steen B. Factors influencing participation in health surveys. Results from prospective population study ‘Men born in 1914’ in Malmo, Sweden. J Epidemiol Community Health. 1986 Jun;40(2):174–7. [PMC free article] [PubMed]

12. Paul PP, George RJ, Arvind H, Raj M, Augustian, Ramesh SV, et al. A comparison of participants and non-participants in the Chennai Glaucoma Study-rural population. Ophthalmic Epidemiol. 2005 Apr;12(2):125–35. [PubMed]

13. Pietila AM, Rantakallio P, Laara E. Background factors predicting non-response in a health survey of northern Finnish young men. Scand J Soc Med. 1995 Jun;23(2):129–36. [PubMed]

14. Pirzada A, Yan LL, Garside DB, Schiffer L, Dyer AR, Daviglus ML. Response rates to a questionnaire 26 years after baseline examination with minimal interim participant contact and baseline differences between respondents and nonrespondents. Am J Epidemiol. 2004 Jan 1;159(1):94–101. [PubMed]

15. Ramos E, Lopes C, Barros H. Investigating the effect of nonparticipation using a population-based case-control study on myocardial infarction. Ann Epidemiol. 2004 Jul;14(6):437–41. [PubMed]

16. Van Loon AJ, Tijhuis M, Picavet HS, Surtees PG, Ormel J. Survey non-response in the Netherlands: effects on prevalence estimates and associations. Ann Epidemiol. 2003 Feb;13(2):105–10. [PubMed]

17. Hoeymans N, Feskens EJ, Van Den Bos GA, Kromhout D. Non-response bias in a study of cardiovascular diseases, functional status and self-rated health among elderly men. Age Ageing. 1998 Jan;27(1):35–40. [PubMed]

18. Haapea M, Miettunen J, Laara E, Joukamaa MI, Jarvelin MR, Isohanni MK, et al. Non-participation in a field survey with respect to psychiatric disorders. Scand J Public Health. 2008 Sep;36(7):728–36. [PubMed]

19. Rosengren A, Wilhelmsen L, Berglund G, Elmfeldt D. Non-participants in a general population study of men, with special reference to social and alcoholic problems. Acta Med Scand. 1987;221(3):243–51. [PubMed]

20. Needham DM, Dennison CR, Dowdy DW, Mendez-Tellez PA, Ciesla N, Desai SV, et al. Study protocol: The Improving Care of Acute Lung Injury Patients (ICAP) study. Crit Care. 2006 Feb;10(1):R9. [PMC free article] [PubMed]

21. Cheung AM, Tansey CM, Tomlinson G, Diaz-Granados N, Matte A, Barr A, et al. Two-year outcomes, health care use, and costs of survivors of acute respiratory distress syndrome. Am J Respir Crit Care Med. 2006 Sep 1;174(5):538–44. [PubMed]

22. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83. [PubMed]

23. Funkhouser E, Macaluso M, Wang X. Alternative strategies for selecting population controls: comparison of random digit dialing and targeted telephone calls. Ann Epidemiol. 2000 Jan;10(1):59–67. [PubMed]

24. Herridge MS, Cheung AM, Tansey CM, Matte-Martyn A, Diaz-Granados N, Al-Saidi F, et al. One-year outcomes in survivors of the acute respiratory distress syndrome. N Engl J Med. 2003 Feb 20;348(8):683–93. [PubMed]

25. Splansky GL, Corey D, Yang Q, Atwood LD, Cupples LA, Benjamin EJ, et al. The Third Generation Cohort of the National Heart, Lung, and Blood Institute’s Framingham Heart Study: design, recruitment, and initial examination. Am J Epidemiol. 2007 Jun 1;165(11):1328–35. [PubMed]

26. Christen WG, Gaziano JM, Hennekens CH. Design of Physicians’ Health Study II--a randomized trial of beta-carotene, vitamins E and C, and multivitamins, in prevention of cancer, cardiovascular disease, and eye disease, and review of results of completed trials. Ann Epidemiol. 2000 Feb;10(2):125–34. [PubMed]