Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Am J Ind Med. Author manuscript; available in PMC 2011 October 3.
Published in final edited form as:
PMCID: PMC3184944

Characteristics of non-participation and potential for selection bias in a prospective cohort study



We investigated the potential for selection bias due to non-participation in the follow-up of a large prospective cohort study.


Licensed pesticide applicators (52,395 private; 4916 commercial) in the Agricultural Health Study provided demographic, health, and pesticide exposure information at enrollment (1993-1997) and in a five-year follow-up telephone interview. Factors associated with non-participation in the follow-up were identified using multiple logistic regression. Potential for selection bias was evaluated by comparing exposure-disease associations between the entire cohort and the follow-up subset.


Sixty-six percent of private and sixty percent of commercial applicators completed the follow-up interview. Private and commercial applicators who did not complete the follow-up reported at enrollment younger age, less education, lower body mass index, poorer health behaviors but fewer health conditions, and lower pesticide use. Estimates of exposure-disease associations calculated with and without non-participants did not indicate strong selection bias.


Differences between non-participants and participants in the follow-up interview were generally small, and we did not find significant evidence of selection bias. However, the extent of bias may depend on the specific exposure and outcome under study.


In order to provide a basis for drawing valid conclusions, study participants should accurately reflect the exposure and outcome prevalence of the population they represent. Because some participants are inevitably lost to follow-up over the course of a study with multiple follow-up time points, it is important to know whether non-participants differ significantly from participants with respect to either disease or exposure status. Differential loss of participants by exposure and disease status simultaneously can lead to selection bias.[Greenland 1977]

The Agricultural Health Study (AHS) is a longitudinal cohort study that has prospectively collected information on a variety of farm-related exposures and health outcomes for over 57 000 licensed pesticide applicators from North Carolina and Iowa. Having completed the first five-year follow-up, we examine the similarities and differences in characteristics at enrollment between participants and non-participants and determine the extent to which differential losses to follow-up could bias exposure-disease associations.


The AHS was designed to examine potential health effects of farm-related exposures, in particular exposure to pesticides. All pesticide applicators applying between 1993 and 1997 for a license to use restricted-use pesticides in North Carolina or Iowa were invited to participate. Applicators in Iowa included both private and commercial applicators; in North Carolina, all applicators were private applicators. Those who agreed (82% of private applicators, 42% of commercial applicators) completed an enrollment questionnaire, which included information on demographic characteristics, health history, and lifetime pesticide use practices. Consistent with approved informed consent procedures for questionnaire data at the time, returning an enrollment questionnaire was considered informed consent. Almost one-half (47%) of these applicators provided additional information in a more detailed take-home questionnaire, which they returned by mail. Questionnaires are available on the study website ( The study was approved by the Institutional Review Boards of the National Institutes of Health (Bethesda, Maryland) and its contractors.

Between 1999 and 2003, applicators were contacted by phone for the second phase of the study, which used a Computer Assisted Telephone Interview (CATI) to collect updated information on both pesticide exposure and diagnosis of incident disease. Interviews were scheduled to avoid months of peak farm activity. In Iowa, 70% of interviews were completed between November and March, with virtually no interviews completed in April, May, and October. In North Carolina, 75% of interviews were completed between November and April, with relatively few interviews completed in June, August, or September. Of the 50 766 private and 4775 commercial pesticide applicators who enrolled and were still living at the time of the telephone interview, 66% of private and 60% of commercial applicators completed the follow-up CATI after a median follow-up time of 5.4 years. Non-participants were defined as applicators who enrolled in the first phase of the study by completing the enrollment questionnaire but who did not complete the second phase telephone interview. Deaths within the cohort up to the time of follow-up were identified using state mortality files and the National Death Index, and individuals who had died were excluded from all analyses (N = 1629). Proxy interviews with next of kin were not conducted. Incident cancer cases diagnosed before January 01, 2006 were identified by linkage with the state cancer registries in North Carolina and Iowa. Prevalent cases for all health conditions were those reported at enrollment, and incident cases were those reported after enrollment.

Most non-participation occurred because the applicator refused to be interviewed (15%) or could not be reached (14%). Contact information was missing for 1%, another 1% of non-participation was due to chronic illness or language difficulties, and 2% did not participate for other reasons.

Analyses were performed using the P1REL0712.00 and P2REL0612.03 AHS data release files and SAS, version 9.1 (SAS Institute, Inc., Cary, North Carolina). Reported results were restricted to private pesticide applicators; commercial applicators were investigated separately for comparison and differences are noted. Multiple logistic regression was used to calculate the adjusted odds of non-participation; hence an odds ratio greater than one indicates a greater odds of not participating in the follow-up interview.

In order to estimate the effect that potential selection bias may have on estimates of association, we considered three potential exposure-disease associations: chlorpyrifos with prevalent depression, smoking with prevalent chronic lung disease, and smoking with incident cancer. We compared the results for the entire cohort that was originally enrolled with results for the sub-cohort that participated in the follow-up interview. For depression and chronic lung disease we conducted cross-sectional analyses using enrollment data, and for overall cancer we used incident diagnosis information that was collected prospectively. While the association of smoking with cancer and chronic lung disease is well established, an association between chlorpyrifos and depression has been suggested but not confirmed.[Aldridge, et al. 2005, Beseler, et al. 2008]


Demographics and Health

Among private pesticide applicators, nonparticipation in follow-up was associated with younger age, non-White race, fewer years of education, and North Carolina residency (Table 1). With regard to household characteristics, non-participants were less likely to have children, although the number of children had no significant influence, and to be unmarried. Growing up on a farm was not associated with participation.

Table 1
General population characteristics of private pesticide applicators and participation in follow-up in the Agricultural Health Study 1993 - 2003.

Considering health behaviors, non-response was positively associated with smoking and alcohol use and inversely associated with vegetable consumption and vitamin or mineral supplementation. In contrast, non-participants were less likely to be overweight or obese (body mass index over 25). Leisure time physical activity was not significantly associated with participation.

Information on all health conditions was self-reported at enrollment with the exception of incident cancer, which was obtained from cancer registry files (Table 2). For most health conditions, non-participants were less likely to have reported a condition at enrollment than participants, although many differences were not statistically significant. The only health condition for which we had information on incident diagnoses for both participants and non-participants was cancer. Although prevalent cancer cases were more likely to participate in follow-up than applicators with no cancer diagnosis at enrollment (adjusted odds ratio for nonparticipation (OR) 0.74, 95% confidence interval (CI) 0.63 – 0.87), cancer incidence did not differ significantly by follow-up status (OR 0.93, CI 0.85 – 1.01). Results for additional health conditions that were reported on the take-home questionnaire (and thus available for less than half the cohort) are included in Appendix A.

Table 2
Health conditions reported at enrollment and participation in follow-up among private applicators in the Agricultural Health Study 1993 - 2003.

Pesticide Use

Personally mixing or applying pesticides was significantly associated with participation at follow-up (non-response OR 0.52, CI 0.40 .68) (Table 3). Furthermore, applicators in both states who personally applied their own pesticides less than half of the time were less likely to participate (OR 0.85, CI 0.79 – 0.93 in North Carolina; OR 0.86, CI 0.81 – 0.91 in Iowa). In Iowa participation increased with longer lifetime duration of pesticide application but was not associated with the frequency of pesticide application (days applied per year). On the other hand, in North Carolina lifetime years of application was not associated with participation, but a higher frequency of application was associated with non-participation.

Table 3
Participation (%) and pesticide exposure among private pesticide applicators in the Agricultural Health Study

While pesticide use was positively associated with participation, the type of pesticide used appeared to have little impact. One exception was fungicide users in North Carolina, who were slightly more likely to be non-participants in the telephone interview (OR 1.13, CI 1.05 – 1.22). Use of chemical-resistant gloves was associated with decreased odds of non-participation in Iowa but not North Carolina. Larger farm size was associated with increased odds of non-participation in both states, although the association was slightly stronger in North Carolina.

AHS investigators previously developed an algorithm that takes into account factors such as frequency of use, application method, and personal protective equipment to estimate intensity of pesticide exposure.[Dosemeci, et al. 2002, Coble, et al. 2005] Increasing exposure intensity score for all pesticide types was associated with a slightly decreased odds of non-participation in Iowa but not in North Carolina (Appendix B). The method used for pesticide application was not associated with non-participation in North Carolina. In Iowa, nearly all application methods were associated with decreased odds of non-participation, suggesting that this was just a surrogate for applying pesticides.

Characteristics associated with follow-up among commercial applicators were similar to the findings for private applicators although some associations were no longer significant due to the smaller sample size (data not shown). The single exception was that commercial applicators who had grown up on a farm were significantly more likely to participate in the follow-up interview than applicators who had not (OR 0.76, CI 0.63 – 0.93).

Bias Estimation

Exposure-outcome relationships for the original enrolled cohort were compared with results from private pesticide applicators who participated in the follow-up interview (Table 4). We examined two cross-sectional relationships: chlorpyrifos exposure with depression and smoking status with chronic lung disease (excluding asthma). Because we were particularly interested in the effect of losses to follow-up on associations with incident disease in addition to prevalent disease, we also included the association between smoking and incident cancer.

Table 4
Disease-exposure associations comparing follow-up participants with the entire population of private pesticide applicators in the Agricultural Health Study 1993 - 2003.

From the analyses reported above, we know that depression, chronic lung disease, and smoking are associated with the probability of follow-up although the association was only marginally significant for depression. For chlorpyrifos, non-participation was more likely among exposed applicators in North Carolina (OR 1.11, 1.04-1.19) but less likely in Iowa (OR 0.90, 0.85-0.94). We therefore examined the chlorpyrifos-depression association separately by state. If selection bias were to affect the point estimates, we would expect that losing exposed controls in North Carolina would increase the observed odds ratio and that losing unexposed controls in Iowa would decrease the observed odds ratio.

The odds ratio for the association of smoking with either chronic lung disease or incident cancer did not differ significantly between the entire cohort and those who completed the follow-up. Likewise, estimates did not differ for the chlorpyrifos-depression association in Iowa. However, in North Carolina there was a non-significant increase in the association between chlorpyrifos and depression for the follow-up cohort (OR 1.22, CI 0.98 – 1.51) compared to the enrollment cohort (OR 1.07, CI 0.90 – 1.27).


Overall participation of private pesticide applicators in the telephone interview five years after enrollment was 66%. Patterns of response associated with age, education, and marital status were consistent with what has frequently been observed in other studies.[Benfante, et al. 1989, Shahar, et al. 1996, Osler, et al. 2008, Russell, et al. 2001] Participants in the follow-up interview tended to have healthier behaviors with regard to smoking, alcohol intake, vegetable consumption, and vitamin and mineral supplementation than non-participants. In apparent contrast to these healthy behaviors, we found that body mass index was higher among participants and that individuals who reported a health condition at enrollment were more likely to participate in the follow-up.

These findings together support the “worried ill” hypothesis proposed by Veenstra.[Veenstra, et al. 2006] Having been diagnosed with a health condition, these participants were likely instructed to improve their health habits and would therefore be more likely to report having healthier habits than participants without a diagnosis. Furthermore, these participants may have a greater vested interest in the completion of the study than those without any diagnosis. We also know the health conditions reported at enrollment were not severe enough to interfere with participation, and provided that these conditions did not seriously progress in the intervening time, we would expect that participants who reported a condition at enrollment should be capable of participating in the follow-up interview.

One limitation of this analysis is that for health conditions other than overall cancer we were unable to measure the probability of participation among incident cases. However, it was reassuring to find that incident cancer cases were not significantly different from non-cancer cases in their probability of follow-up at interview. Furthermore, we did not observe a significant effect of selection bias when we examined the association of incident cancer with smoking status in the subset of applicators who completed the follow-up interview. It should be noted that while incident cancer was not significantly associated with follow-up, it is still possible for selection bias to occur. [Greenland 1977]

In addition to having limited information on incident disease status, we also could not adequately determine whether the impact of losses due to fatal incident conditions was similar to that from non-fatal health conditions. The loss of deceased participants would be expected to result in an underestimation of disease incidence, and if these participants were more (or less) likely to report pesticide exposure than those with non-fatal health conditions, this could lead to selection bias. Although we were unable to consider specific causes of death because of the small number of deaths between enrollment and follow-up, a comparison of deceased participants with participants who completed the follow-up did not indicate any remarkable differences. Since the percent of participants who died before follow-up was only 3%, we would not expect this exclusion to result in substantial selection bias.

Applicators were more likely not to participate if they had never mixed or applied pesticides or if they personally applied pesticides less than one-half of the time, consistent with the idea that those with more of a connection to the subject of the study would be more likely to participate. Conversely, larger farm size was associated with increased probability of non-participation, suggesting that how busy a participant was had an effect on participation. The relationship of other measures of pesticide use to likelihood of participation differed between the two states. Overall, participation in the telephone interview was greater in Iowa than in North Carolina. Farming activities differ between the two states. North Carolina has a longer growing season and increased crop variety compared to Iowa. Frequency of pesticide use was higher in North Carolina and was associated with increased probability of non-participation, whereas in Iowa there was no significant association. In Iowa applicators reported a significantly greater number of years of pesticide use at enrollment despite being younger on average, and this measure was proportional to the probability of participation. Overall, there did not appear to be a general trend with respect to the level of pesticide exposure and probability of follow-up. Furthermore, the potential for differential non-response or selection bias is likely to vary for specific pesticides.

Farming status at time of interview might also affect participation rates. We did not have information on pesticide license status at time of follow-up and do not know if non-respondents had disproportionately left farming. However, since nearly 20% of those who completed a follow-up interview were no longer farming at follow-up (11.5% in Iowa and 33.5% in North Carolina), leaving farming cannot entirely explain non-response at follow-up.

We investigated the potential for selection bias to affect estimates of exposure-disease associations in the subset of applicators who completed the follow-up by comparing odds ratio estimates between the original cohort and the CATI interview subset. We observed no significant changes in the estimates for any of the three associations. However, the differential loss of participants with no report of depression who were exposed to chlorpyrifos in North Carolina did increase the association from 1.07 to 1.20.

With the exception of cancer, our exploration of selection bias was based on prevalent conditions reported at enrollment. Incident conditions might have a greater impact. The severity of such conditions and the timing with regard to follow-up interview would no doubt influence participation and in turn the possibility of selection bias. For large cohort studies collecting prospective information on self-reported diseases and exposures, it is important to consider the potential for selection bias to occur and to estimate the extent to which it may bias associations.

Supplementary Material

Appendix A

Appendix B


Agricultural Health Study
Computer Assisted Telephone Interview


  • Greenland S. Response and follow-up bias in cohort studies. Am J Epidemiol. 1977;106:184–187. [PubMed]
  • Aldridge JE, Levin ED, Seidler FJ, Slotkin TA. Developmental exposure of rats to chlorpyrifos leads to behavioral alterations in adulthood, involving serotonergic mechanisms and resembling animal models of depression. Environ Health Perspect. 2005;113:527–531. [PMC free article] [PubMed]
  • Beseler CL, Stallones L, Hoppin JA, Alavanja MC, Blair A, Keefe T, Kamel F. Depression and pesticide exposures among private pesticide applicators enrolled in the Agricultural Health Study. Environ Health Perspect. 2008;116:1713–1719. [PMC free article] [PubMed]
  • Dosemeci M, Alavanja MC, Rowland AS, Mage D, Zahm SH, Rothman N, Lubin JH, Hoppin JA, Sandler DP, Blair A. A quantitative approach for estimating exposure to pesticides in the Agricultural Health Study. Ann Occup Hyg. 2002;46:245–260. [PubMed]
  • Coble J, Arbuckle T, Lee W, Alavanja M, Dosemeci M. The validation of a pesticide exposure algorithm using biological monitoring results. J Occup Environ Hyg. 2005;2:194–201. [PubMed]
  • Benfante R, Reed D, MacLean C, Kagan A. Response bias in the Honolulu Heart Program. Am J Epidemiol. 1989;130:1088–1100. [PubMed]
  • Shahar E, Folsom AR, Jackson R, Atherosclerosis Risk in Communities (ARIC) Study Investigators The effect of nonresponse on prevalence estimates for a referent population: insights from a population-based cohort study. Ann Epidemiol. 1996;6:498–506. [PubMed]
  • Osler M, Kriegbaum M, Christensen U, Lund R, Nybo Andersen AM. Loss to follow up did not bias associations between early life factors and adult depression. J Clin Epidemiol. 2008;61:958–963. [PubMed]
  • Russell C, Palmer JR, Adams-Campbell LL, Rosenberg L. Follow-up of a large cohort of Black women. Am J Epidemiol. 2001;154:845–853. [PubMed]
  • Veenstra MY, Friesema IH, Zwietering PJ, Garretsen HF, Knottnerus JA, Lemmens PH. Lower prevalence of heart disease but higher mortality risk during follow-up was found among nonrespondents to a cohort study. J Clin Epidemiol. 2006;59:412–420. [PubMed]