|Home | About | Journals | Submit | Contact Us | Français|
The aim of the study was to collect pilot data on response rates to a follow-up postal questionnaire in a cohort of American Indians living in the Southwestern United States. We tested the effect of questionnaire length on response.
Cohort members were American Indian adults aged 18 and over who completed the baseline study visit. Study participants (N=1587), cohort members who completed the baseline study visit during the first year of enrollment, were randomized into two groups to receive either an 18-page or 3-page follow-up postal questionnaire. Data were collected between October 2005 and March 2006.
The response rates after two questionnaire mailings and a reminder postcard were significantly higher for the short versus the long (56.2%, 48.1% p=<0.01) questionnaire. Being female and being aged 50 or older were associated with returning a completed questionnaire. A reminder postcard and second mailing improved response by 11.7% and 13.4% respectively.
These results show that a postal questionnaire can be used in a cohort of American Indians living in the Southwest, but suggest that questionnaires should be short and repeat mailings are needed.
Achieving a high response rate to a postal follow-up questionnaire in a cohort has two components: 1) maintaining contact with the participants and 2) for those contacted, maximizing participation (Hunt and White, 1998). Evaluation of potential barriers to questionnaire completion and strategies to improve response to postal questionnaires are needed to plan successful follow-up data collection.
The Navajo Education and Research Towards Health (EARTH) study is part of a multi-centered study funded to test the feasibility of establishing and following a cohort of American Indian and Alaska Native (AIAN) people. Past experience with conducting cohort studies in an AIAN population is limited. There are many reasons AIAN communities are not enthusiastic about participating in research including lack of trust in researchers due to past exploitation and abuse from research studies; individuals may not feel as if they are stakeholders in the research, i.e. they are not getting anything out of it; lack of understanding most researchers have for American Indian culture; and lack of a long standing relationship with the community (Davis and Reid, 1999; Sambo, 2001) Other potential barriers to achieving a high response rate to a postal follow-up questionnaire include: 1) the preponderance of postal boxes versus street addresses; 2) mobility of the population; 3) age of the population; 4) isolated and remote geographic location of some communities; 5) economic disparities in some communities (Choudhary, 2000); 6) the wide range of educational level (Choudhary, 2000); 7) scarcity of telephones making follow-up calls difficult (Choudhary, T; White et al., 1997); 8) variable frequency of mail pick-up (White et al., 1997); and 9) language barriers in some communities (White et al., 1997).
The goals of the present study were to collect pilot data on response rates to a postal follow-up questionnaire among individuals who enrolled in a cohort study on the Navajo Nation and to identify factors that might increase response to future follow-up questionnaires mailed to the entire cohort.
Detailed methods have been described elsewhere (Edwards et al., 2007; Slattery et al., 2007, 2008; Edwards et al., 2008). Briefly, Navajo EARTH study participants are 18 years of age or older, self-identify as American Indian or Alaska Native, have given informed consent, understand English or Navajo, and are eligible to receive care at the Indian Health Service. Participants enrolled in the cohort by coming to a study visit site to provide informed consent; complete study questionnaires including diet history and physical activity; and have blood pressure, blood lipids, and body size measurements taken. The Institutional Review Boards of the University of Utah, the Indian Health Service, and the Navajo Nation approved the study. Local health boards and chapters within local health boards also approved and supported the study.
Participants were asked to provide contact information at the baseline study visit. The contact information included name; spouse or partner name; maiden name (women); phone, including home, cell, message, work, and other; street address or physical location of residence for participants with a postal box; name, address, and phone for two friends or relatives; usual medical care provider; and Social Security number. This information was stored in the study tracking system. Initial follow-up contact was made with participants through a study newsletter. Addresses for participants whose newsletters were returned by the post office with change of address information were updated in the tracking system and a newsletter was resent. Participants whose newsletters were returned by the post office without change of address information were put into tracing. Attempts were made to call all telephone numbers, contacts, and obtain updated address information from medical records where the medical record consent was signed. If a new address was obtained, the tracking system was updated and the newsletter resent.
Two follow-up questionnaires were developed. Each questionnaire included a one page medical history questionnaire asking the participant to update his or her health information. The short questionnaire also included the short format International Physical Activity Questionnaire (Booth, 2000) making it three pages in length, while the long questionnaire included a 17 page physical activity log developed for the Navajo EARTH Study physical activity validation study, making it 18 pages in length. Both follow-up questionnaires were in English only.
Participants were selected for the pilot follow-up study if their baseline study visit had been completed over 1 year prior to the mailing of the follow-up questionnaire (Fig. 1). All participants who completed a study visit between March 2004 and October 2004, N=1587, were selected. Participants were randomized into two groups using a computerized random-number table by the study statistician. One group received the long questionnaire, N=794, while the other group received the short questionnaire, N=792. The questionnaires were mailed in October, 2005. The study design did not allow for staff and participants to be blinded.
Following the recommendations of the total design method for implementing mail surveys (Dillman, 1978) a cover letter and stamped return envelope were included with the questionnaire mailout packet, a post card reminder was sent after the first mailing, a letter and second questionnaire were mailed to all nonrespondents approximately 1 month after the post card was sent. The recommended mailing intervals were adjusted due to the potential delays in picking up mail. We included a 5$ incentive with the first mailing and a sharpened pencil with each mailing (White et al., 2005).
Demographic data including age, education, employment status, marital status, and income were collected at the baseline study visit. Other variables collected or calculated from the baseline study visit included: sex, number of years at current residence; language spoken at home; identification with non-Native culture; Diet History Questionnaire calorie outlier; perceived health status; the 12-item Short Form health survey questionnaire (SF-12) summary scores (the Mental Health Component Summary (MCS) score and the Physical Health Component Summary (PCS) score) (Ware et al., 1996); number of medical conditions reported; body mass index; risky behaviors (seatbelt and helmet use and drinking while driving); cigarette smoking status; and parity (women).
Study variables relating to participant contact were generated from the tracking system and included: signed medical records release; provided telephone number at baseline study visit (home and any other, cell and any other but home, other only, no phone); had postal box address; provided Social Security number (SSN); and provided contact information such as name and phone number for at least one friend or relative.
Response rates (RR) were calculated using the “RR1 formula for mailed surveys to specifically named persons” recommended by the American Association For Public Opinion Research (2006).
We calculated the overall response rates for the two questionnaire length groups and compared using chi-square tests. We evaluated the influence of other characteristics including sex and age.
Analysis was performed using Cox regression. Predictors of completion were determined in a univariate model stratified for each questionnaire type. Multivariate analysis was performed using backward stepwise Cox regression, adjusted for questionnaire type, age, provided SSN, and signed medical consent to determine which set of variables best predicted completion of the postal follow-up questionnaire. SAS 9.1 (SAS Institute Inc., Cary, NC) was used for descriptive statistics and related tests, and Intercooled Stata 8.0 (Stata Corporation, College Station, TX) for tests of proportions and statistical analysis of predictors.
Of the 1587 participants selected for the pilot, one was determined to be ineligible because of death. Overall, more women than men were mailed a questionnaire (63.9% versus 36.1%), 49.8% were under age 40, 75.1% had completed high school or beyond, 44.2% were currently employed, and 69.6% reported speaking their “Native” language, that is, the language of their American Indian or Alaska Native tribe, or both “Native” language and English at home. (Table 1) Only 6.7% of participants had a mailing address that was not a post office box. The consent to review medical records was signed by 91.3% of participants, 83.5% provided a SSN, 63.1 % had a “home” telephone number (versus a cell/other phone only or no phone), and 87.8% gave a telephone number for at least one contact. The characteristics of study participants receiving the long and short follow-up questionnaires were similar, although participants receiving the long questionnaire were younger (p=0.01), were more likely to provide an SSN (p=0.01), and signed the medical consent (p=0.05) more often.
Table 2 summarizes response outcomes by questionnaire length and mailing along with the final outcome. As shown, the response rates were significantly higher for the short questionnaire versus the long (56.2%, 48.1% p<0.01). This difference remained significant when adjusted for age and other covariates. The response rate was also significantly higher for the short questionnaire versus the long after the initial mailing. However, upon each subsequent mailing there was no difference in the proportion of questionnaires returned by questionnaire length with an additional 11.7% returning a completed questionnaire after mailing the reminder postcard and 13.4% after mailing a second questionnaire.
When subject characteristics were considered as predictors of questionnaire response, females were 35% more likely to return the long questionnaire (p=<0.01) and 32% (p=<0.01) more likely to return the short questionnaire (data not shown). In the multivariate model, older age and parity were associated with response for women, and older age, not always using a seatbelt, and a higher score on the MCS of the SF-12 were associated with response for men (data not shown).
The results of the study showed that for this population, receiving a shorter questionnaire did increase response. As the overall response rate was 52.1% after two follow-up mailings to non-responders, a reminder postcard and then a second questionnaire, more intensive follow-up would be necessary to achieve higher response rates.
While a focus of the main study was to address issues of lack of trust in research that exists when doing research with American Indians, many of the other potential barriers to response to a postal follow-up questionnaire, e.g. scarcity of telephones, or remote locations, could not be directly addressed in the design of pilot study. However, some of these barriers might have had an impact on response. For instance 93.3% of participants had a post office box making it impossible to know how many of the 677 non-respondents actually opened the questionnaire and decided not to return it. Due to the distance many participants have to travel to their local postal box, other household members could have picked up the mail and potentially the participant might not have ever received the questionnaire. Certified mail might help address this barrier, but due to the travel distance to a post office it would be hard for most participants to sign for mail.
Language is another barrier that might have had an impact on the response to the follow-up questionnaire. It is unlikely that this is the case however. During the first year of data collection, we “deferred” participants who needed their study visit conducted only in Navajo as the translations were not complete. Thus, all the participants in the pilot could read and understand English.
The results of this study show that a postal questionnaire can be used in a cohort of American Indians living in the Southwest, but suggest that future mailings should be kept short. In addition, repeat mailings to increase the likelihood that the postal questionnaire will be received and opened by the participant are important in this population. Although the study was conducted in American Indians living in the Southwest, it is reasonable that these findings may have implications for participant follow-up in other populations.
This study was funded by grant CA 088958 from the National Cancer Institute. The contents of this manuscript are solely the responsibility of the authors and do not necessarily represent the official view of the National Cancer Institute. We acknowledge additional support from the Survey Methods and Data Collection Core supported by the Huntsman Cancer Foundation.
We would like to acknowledge the contributions of Clarina Clark, Kate Hak, Jenovia Plenty, Molly McFadden, Amy Rogers, and the staff at the Navajo Nation field centers to this study.
Conflict of interest statement The authors declare that there are no conflicts of interest.