In Australia telephone surveys have been the method of choice for ongoing jurisdictional population health surveys. Although it was estimated in 2011 that nearly 20% of the Australian population were mobile-only phone users, the inclusion of mobile phone numbers into these existing landline population health surveys has not occurred. This paper describes the methods used for the inclusion of mobile phone numbers into an existing ongoing landline random digit dialling (RDD) health survey in an Australian state, the New South Wales Population Health Survey (NSWPHS). This paper also compares the call outcomes, costs and the representativeness of the resultant sample to that of the previous landline sample.
After examining several mobile phone pilot studies conducted in Australia and possible sample designs (screening dual-frame and overlapping dual-frame), mobile phone numbers were included into the NSWPHS using an overlapping dual-frame design. Data collection was consistent, where possible, with the previous years’ landline RDD phone surveys and between frames. Survey operational data for the frames were compared and combined. Demographic information from the interview data for mobile-only phone users, both, and total were compared to the landline frame using χ2 tests. Demographic information for each frame, landline and the mobile-only (equivalent to a screening dual frame design), and the frames combined (with appropriate overlap adjustment) were compared to the NSW demographic profile from the 2011 census using χ2 tests.
In the first quarter of 2012, 3395 interviews were completed with 2171 respondents (63.9%) from the landline frame (17.6% landline only) and 1224 (36.1%) from the mobile frame (25.8% mobile only). Overall combined response, contact and cooperation rates were 33.1%, 65.1% and 72.2% respectively. As expected from previous research, the demographic profile of the mobile-only phone respondents differed most (more that were young, males, Aboriginal and Torres Strait Islanders, overseas born and single) compared to the landline frame responders. The profile of respondents from the two frames combined, with overlap adjustment, was most similar to the latest New South Wales (NSW) population profile.
The inclusion of the mobile phone numbers, through an overlapping dual-frame design, did not impact negatively on response rates or data collection, and although costing more the design was still cost-effective because of the additional interviews that were conducted with young people, Aboriginal and Torres Strait Islanders and people who were born overseas resulting in a more representative overall sample.
Sample survey; Mobile phone; Sampling frame
To examine the trend of "mobile only" households, and households that have a mobile phone or landline telephone listed in the telephone directory, and to describe these groups by various socio-demographic and health indicators.
Representative face-to-face population health surveys of South Australians, aged 15 years and over, were conducted in 1999, 2004, 2006, 2007 and 2008 (n = 14285, response rates = 51.9% to 70.6%). Self-reported information on mobile phone ownership and usage (1999 to 2008) and listings in White Pages telephone directory (2006 to 2008), and landline telephone connection and listings in the White Pages (1999 to 2008), was provided by participants. Additional information was collected on self-reported health conditions and health-related risk behaviours.
Mobile only households have been steadily increasing from 1.4% in 1999 to 8.7% in 2008. In terms of sampling frame for telephone surveys, 68.7% of South Australian households in 2008 had at least a mobile phone or landline telephone listed in the White Pages (73.8% in 2006; 71.5% in 2007). The proportion of mobile only households was highest among young people, unemployed, people who were separated, divorced or never married, low income households, low SES areas, rural areas, current smokers, current asthma or people in the normal weight range. The proportion with landlines or mobiles telephone numbers listed in the White Pages telephone directory was highest among older people, married or in a defacto relationship or widowed, low SES areas, rural areas, people classified as overweight, or those diagnosed with arthritis or osteoporosis.
The rate of mobile only households has been increasing in Australia and is following worldwide trends, but has not reached the high levels seen internationally (12% to 52%). In general, the impact of mobile telephones on current sampling frames (exclusion or non-listing of mobile only households or not listed in the White Pages directory) may have a low impact on health estimates obtained using telephone surveys. However, researchers need to be aware that mobile only households are distinctly different to households with a landline connection, and the increase in the number of mobile-only households is not uniform across all groups in the community. Listing in the White Pages directory continues to decrease and only a small proportion of mobile only households are listed. Researchers need to be aware of these telephone sampling issues when considering telephone surveys.
Telephone surveys based on samples of landline telephone numbers are widely used to measure the prevalence of health risk behaviours such as smoking, drug use and alcohol consumption. An increasing number of households are relying solely on mobile telephones, creating a potential bias for population estimates derived from landline-based sampling frames which do not incorporate mobile phone numbers. Studies in the US have identified significant differences between landline and mobile telephone users in smoking and alcohol consumption, but there has been little work in other settings or focussed on illicit drugs.
This study examined Australian prevalence estimates of cannabis use, tobacco smoking and risky alcohol consumption based on samples selected using a dual-frame (mobile and landline) approach. Respondents from the landline sample were compared both to the overall mobile sample (including respondents who had access to a landline) and specifically to respondents who lived in mobile-only households. Bivariate comparisons were complemented with multivariate logistic regression models, controlling for the effects of basic demographic variables.
The landline sample reported much lower prevalence of tobacco use, cannabis use and alcohol consumption than the mobile samples. Once demographic variables were adjusted for, there were no significant differences between the landline and mobile respondents on any of the alcohol measures examined. In contrast, the mobile samples had significantly higher rates of cannabis and tobacco use, even after adjustment. Weighted estimates from the dual-frame sample were generally higher than the landline sample across all substances, but only significantly higher for tobacco use.
Landline telephone surveys in Australia are likely to substantially underestimate the prevalence of tobacco smoking by excluding potential respondents who live in mobile-only households. In contrast, estimates of alcohol consumption and cannabis use from landline surveys are likely to be broadly accurate, once basic demographic weighting is undertaken.
To present the design and preliminary results of a pilot study to investigate the use of opt-in Internet panel surveys for behavioral health surveillance.
Today, surveyors in both the private and public sectors are facing considerable challenges with random digit dialed (RDD) landline telephone samples. The population coverage rates for landline telephone surveys are being eroded by wireless-only households, portable telephone numbers, telecommunication barriers (e.g., call forwarding, call blocking and pager connections), technological barriers (call-blocking, busy circuits) and increased refusal rates and privacy concerns. Addressing these issues increasingly drives up the costs associated with dual-frame telephone surveys designed to be representative of the target population as well as hinders their ability to be fully representative of the adult population of each state and territory in the United States.
In an effort to continue to meet these challenges head on and assist state and territorial public health professionals in the continued collection of data that are representative of their respective populations, novel approaches to behavioral health surveillance need continued examination. Both private and public sector researchers are evaluating the use of Internet opt-in panels to augment dual-frame RDD survey methods. Compared to dual-frame RDD, opt-in Internet panels offer lower costs, quick data collection and dissemination, and the ability to gather additional data on panelists over time. However, as with dual-frame RDD, this mode has similar challenges with coverage error and non-response. Nevertheless, survey methodologists are moving forward and exploring ways to reduce or eliminate biases between the sample and the target population.
A collaborative pilot project was designed to assess the feasibility and accuracy of opt-in Internet panel surveys for behavioral health surveillance. This pilot project is a collaboration between the CDC, four state departments of health, opt-in Internet panel providers and the leads of several large surveys and systems such as the Patient-Reported Outcome Measures Information System (PROMIS) and the Cooperative Congressional Election Study (CCES). Pilot projects were conducted in four states (GA, IL, NY, and TX) and four Metropolitan Statistical Areas (Atlanta, Chicago, New York City, and Houston). Data were collected using three different opt-in Internet panels and sampling methods that differ with respect to recruitment strategy, sample selection and sample matching to the adult population of each geography. A question bank consisting of 80 questions was developed to benchmark with other existing surveys used to assess various public health surveillance measures (e.g., the Behavioral Risk Factor Surveillance System, the PROMIS, National Survey on Drug Use and Health, and the CCES).
We present comparative analyses that assess the advantages and disadvantages of different opt-in Internet panels sampling methodologies across a range of parameters including cost, geography, timeliness, usability, and ease of use for technology transfer to states and local communities. Recommendations for future efforts in behavioral health surveillance are given based on these results.
Random Digit Dialing; BRFSS; Survey Methods; Sample Matching; Representativeness
Population-based landline telephone surveys are potentially biased due to inclusion of only people with landline telephones. This article examined the degree of telephone coverage bias in a low-income population.
The Charles County Cancer Survey (CCCS) was conducted to evaluate cancer screening practices and risk behaviors among low-income, rural residents of Charles County, Maryland. We conducted face-to-face interviews with 502 residents aged 18 years and older. We compared the prevalence of health behaviors and cancer screening tests for those with and without landline telephones. We calculated the difference between whole sample estimates and estimates for only those respondents with landline telephones to quantify the magnitude of telephone coverage bias.
Of 499 respondents who gave information on telephone use, 80 (16%) did not have landline telephones. We found differences between those with and without landline telephones for race/ethnicity, health-care access, insurance coverage, and several types of cancer screening. The absolute coverage bias ranged up to 6.5 percentage points. Simulation scenarios showed the magnitude of telephone coverage bias decreases as the percent of the population with landline telephone coverage increases, and as landline telephone coverage increases, the estimates from a landline telephone survey would approximate the estimates from a face-to-face survey.
Our findings highlighted the need for targeted face-to-face surveys to supplement telephone surveys to more fully characterize hard-to-reach subpopulations. Our findings also indicated that landline telephone-based surveys continue to offer a cost-effective method for conducting large-scale population studies in support of policy and public health decision-making.
Households with fixed-line telephones have decreased while mobile (cell) phone ownership has increased. We therefore sought to examine the feasibility of recruiting young women for a national health survey through random digit dialling mobile phones.
Two samples of women aged 18 to 39 years were surveyed by random digit dialling fixed and mobile numbers. We compared participation rates and responses to a questionnaire between women surveyed by each contact method.
After dialling 5,390 fixed-lines and 3,697 mobile numbers, 140 and 128 women were recruited respectively. Among women contacted and found to be eligible, participation rates were 74% for fixed-lines and 88% for mobiles. Taking into account calls to numbers where eligibility was unknown (e.g. unanswered calls) the estimated response rates were 54% and 45% respectively. Of women contacted by fixed-line, 97% reported having a mobile while 61% of those contacted by mobile reported having a fixed-line at home. After adjusting for age, there were no significant differences between mobile-only and fixed-line responders with respect to education, residence, and various health behaviours; however compared to those with fixed-lines, mobile-only women were more likely to identify as Indigenous (OR 4.99, 95%CI 1.52-16.34) and less likely to live at home with their parents (OR 0.09, 95%CI 0.03-0.29).
Random digit dialling mobile phones to conduct a health survey in young Australian women is feasible, gives a comparable response rate and a more representative sample than dialling fixed-lines only. Telephone surveys of young women should include mobile dialling.
Cellular phone; mobile phone; telephone surveys; survey methods; HPV vaccine
The usefulness of landline random digit dialing (RDD) in epidemiologic studies is threatened by the rapid increase in households with only cellular telephone service. This study assessed the feasibility of including cellular telephone numbers in RDD and differences between young adults with landline telephones and those with only cellular telephones. Between 2008 and 2009, a total of 9,023 cellular telephone numbers were called and 43.8% were successfully screened; 248 men and 249 women who resided in 3 Washington State counties, were 20–44 years of age, and used only cellular telephones were interviewed. They were compared with 332 men and 526 women with landline telephones interviewed as controls for 2 case-control studies conducted in parallel with cellular telephone interviewing. Cellular-only users were more likely to be college educated and less likely to have fathered/birthed a child than were their landline counterparts. Male cellular-only users were less likely to be obese and more likely to exercise, to be Hispanic, and to have lower incomes, while female cellular-only users were more likely to be single than landline respondents. Including cellular telephone numbers in RDD is feasible and should be incorporated into epidemiologic studies that rely on this method to ascertain subjects, although low screening rates could hamper the representativeness of such a sample.
bias (epidemiology); case-control studies; epidemiologic methods; selection bias; telephone
In 2012 mobile phone numbers were included into the ongoing New South Wales Population Health Survey (NSWPHS) using an overlapping dual-frame design. Previously in the NSWPHS the sample was selected using random digit dialing (RDD) of landline phone numbers. The survey was undertaken using computer assisted telephone interviewing (CATI). The weighting strategy needed to be significantly expanded to manage the differing probabilities of selection by frame, including that of children of mobile-only phone users, and to adjust for the increased chance of selection of dual-phone users. This paper describes the development of the final weighting strategy to properly combine the data from two overlapping sample frames accounting for the fact that population benchmarks for the different sampling frames were not available at the state or regional level.
Estimates of the number of phone numbers for the landline and mobile phone frames used to calculate the differing probabilities of selection by frame, for New South Wales (NSW) and by stratum, were obtained by apportioning Australian estimates as none were available for NSW. The weighting strategy was then developed by calculating person selection probabilities, selection weights, applying a constant composite factor to the dual-phone users sample weights, and benchmarking to the latest NSW population by age group, sex and stratum.
Data from the NSWPHS for the first quarter of 2012 was used to test the weighting strategy. This consisted of data on 3395 respondents with 2171 (64%) from the landline frame and 1224 (36%) from the mobile frame. However, in order to calculate the weights, data needed to be available for all core weighting variables and so 3378 respondents, 2933 adults and 445 children, had sufficient data to be included. Average person weights were 3.3 times higher for the mobile-only respondents, 1.3 times higher for the landline-only respondents and 1.7 times higher for dual-phone users in the mobile frame compared to the dual-phone users in the landline frame. The overall weight effect for the first quarter of 2012 was 1.93 and the coefficient of variation of the weights was 0.96. The weight effects for 2012 were similar to, and in many cases less than, the effects found in the corresponding quarter of the 2011 NSWPHS when only a landline based sample was used.
The inclusion of mobile phone numbers, through an overlapping dual-frame design, improved the coverage of the survey and an appropriate weighing procedure is feasible, although it added substantially to the complexity of the weighting strategy. Access to accurate Australian, State and Territory estimates of the number of landline and mobile phone numbers and type of phone use by at least age group and sex would greatly assist in the weighting of dual-frame surveys in Australia.
It is crucial for policy makers to monitor the evolution of tobacco smoking prevalence. In France, this monitoring is based on a series of cross-sectional general population surveys, the Health Barometers, conducted every five years and based on random samples. A methodological study has been carried out to assess the reliability of a monitoring system based on regular quota sampling surveys for smoking prevalence.
Design / Outcome Measures
In 2010, current and daily tobacco smoking prevalences obtained in a quota survey on 8,018 people were compared with those of the 2010 Health Barometer carried out on 27,653 people. Prevalences were assessed separately according to the telephone equipment of the interviewee (landline phone owner vs “mobile-only”), and logistic regressions were conducted in the pooled database to assess the impact of the telephone equipment and of the survey mode on the prevalences found. Finally, logistic regressions adjusted for sociodemographic characteristics were conducted in the random sample in order to determine the impact of the needed number of calls to interwiew “hard-to-reach” people on the prevalence found.
Current and daily prevalences were higher in the random sample (respectively 33.9% and 27.5% in 15-75 years-old) than in the quota sample (respectively 30.2% and 25.3%). In both surveys, current and daily prevalences were lower among landline phone owners (respectively 31.8% and 25.5% in the random sample and 28.9% and 24.0% in the quota survey). The required number of calls was slightly related to the smoking status after adjustment for sociodemographic characteristics.
Random sampling appears to be more effective than quota sampling, mainly by making it possible to interview hard-to-reach populations.
Response rates in surveys have been falling over the last 20 years, leading to the need for novel approaches to enhance recruitment. This study describes strategies used to maximise recruitment to a home interview survey of mothers with young children living in areas of high deprivation.
Mothers of two year old children received a letter from their GP inviting them to take part in a survey on diet. Participants were subsequently recruited by a researcher. The researcher first tried to contact potential participants by telephone, to discuss the study and make an appointment to conduct a home interview. Where telephone numbers for women could not be obtained from GP records, web searches of publicly available databases were conducted. After obtaining correct telephone numbers, up to six attempts were made to establish contact by telephone. If this was unsuccessful, a postal request for telephone contact was made. Where no telephone contact was achieved, the researcher sent up to two appointments by post to conduct a home interview.
Participating GPs invited 372 women to take part in a home based interview study. GP practices provided telephone numbers for 162 women, of which 134 were valid numbers. The researcher identified a further 187 numbers from electronic directories. Further searches of GP records by practice staff yielded another 38 telephone numbers. Thus, telephone numbers were obtained for 99% of potential participants.
The recruitment rate from telephone contacts was 77%. Most of the gain was achieved within four calls. For the remaining women, contact by post and home visits resulted in 18 further interviews, corresponding to 35% of the women not recruited by telephone. The final interview rate was 82%. This was possible because personal contact was established with 95% of potential participants.
This study achieved a high response rate in a hard to reach group. This was mainly achieved by first establishing contact by telephone. The use of multiple sources identified the telephone numbers of almost all the sample. Multiple attempts at telephone contact followed by postal approaches led to a high home interview rate.
Incorrect beliefs about memory have wide-ranging implications. We recently reported the results of a survey showing that a substantial proportion of the United States public held beliefs about memory that conflicted with those of memory experts. For that survey, respondents answered recorded questions using their telephone keypad. Although such robotic polling produces reliable results that accurately predicts the results of elections, it suffers from four major drawbacks: (1) telephone polling is costly, (2) typically, less than 10 percent of calls result in a completed survey, (3) calls do not reach households without a landline, and (4) calls oversample the elderly and undersample the young. Here we replicated our telephone survey using Amazon Mechanical Turk (MTurk) to explore the similarities and differences in the sampled demographics as well as the pattern of results. Overall, neither survey closely approximated the demographics of the United States population, but they differed in how they deviated from the 2010 census figures. After weighting the results of each survey to conform to census demographics, though, the two approaches produced remarkably similar results: In both surveys, people averaged over 50% agreement with statements that scientific consensus shows to be false. The results of this study replicate our finding of substantial discrepancies between popular beliefs and those of experts and shows that surveys conducted on MTurk can produce a representative sample of the United States population that generates results in line with more expensive survey techniques.
Examine the effect of including cell-phone numbers in a traditional landline random digit dial (RDD) telephone survey.
The 2007 California Health Interview Survey (CHIS).
Data Collection Methods
CHIS 2007 is an RDD telephone survey supplementing a landline sample in California with a sample of cell-only (CO) adults.
We examined the degree of bias due to exclusion of CO populations and compared a series of demographic and health-related characteristics by telephone usage.
When adjusted for noncoverage in the landline sample through weighting, the potential noncoverage bias due to excluding CO adults in landline telephone surveys is diminished. Both CO adults and adults who have both landline and cell phones but mostly use cell phones appear different from other telephone usage groups. Controlling for demographic differences did not attenuate the significant distinctiveness of cell-mostly adults.
While careful weighting can mitigate noncoverage bias in landline telephone surveys, the rapid growth of cell-phone population and their distinctive characteristics suggest it is important to include a cell-phone sample. Moreover, the threat of noncoverage bias in telephone health survey estimates could mislead policy makers with possibly serious consequences for their ability to address important health policy issues.
Telephone surveys; survey methods; survey noncoverage bias; California Health Interview Survey
STUDY OBJECTIVE--The study aimed to investigate the influence the mode of administration of a questionnaire (telephone or face to face) on reports of sexual behaviour and attitudes of HIV risk among woman of reproductive age. DESIGN--Two cross sectional surveys--one, a modified random digit dialing telephone survey, the second, a face to face street sample--were carried out by the same interviewers using similar questionnaires in the same neighbourhoods. SETTING AND PARTICIPANTS--Two socially deprived, inner city neighbourhoods of Baltimore City were assessed in early 1990 before a community health intervention was carried out in one of them. Women between 17 and 35 years were surveyed. MAIN RESULTS--Altogether 775 and 416 women in the target age group were interviewed by telephone and face to face methods: the response rates were 66.4% and 77% respectively. Telephone respondents tended to be older, had more education, were more often married, were less likely to live in subsidised housing, and were more likely to report HIV testing. The proportions of respondents who reported a previous abortion and had had a surgical sterilisation were higher among the telephone respondents (34.7% v 24.1% and 26.4% v 20.6%, respectively). With regard to sexual risk behaviour, the only statistically significant differences were found in the proportion who reported having used drugs (10.6% of the face to face v 2.4% of the telephone sample) or alcohol (30.5% v 16.3%) at last sexual intercourse. The observed method effect on these variables remained unchanged after adjusting for age, education, employment, and marital status. This effect was even stronger for a subgroup of face to face respondents who reported not having a telephone at home. The adjusted odds ratios for reporting alcohol consumption and use of drugs at the last sexual encounter in this group compared with the telephone respondents were 3.7 (2.1, 6.6) and 14.1 (5.7, 34.5) respectively. CONCLUSIONS--Despite the socioeconomic bias associated with the mode of data collection, there are only a few differences between the telephone and personal survey methods in reports of sexual behaviour. These differences are mostly concentrated in young women (under 20 years), and in a particularly socioeconomically deprived subgroup identified through telephone ownership.
An evaluation was made of the use of telephone survey methods to collect illicit drug use data. Using data from a national survey that collects data by personal interviews, marijuana and cocaine use prevalence rates among households with telephones and those without were compared in order to assess coverage errors in telephone surveys. Drug use rates were substantially higher among households without telephones, with 24.9 percent of those living in households without telephones reporting use of marijuana in the past year, compared with only 9.4 percent of persons living in households with telephones. Trends in drug use were divergent, with substantial decreases in use occurring between 1985 and 1988 in households with telephones, but not in those without. National prevalence patterns and trends among households with telephone appear to be consistent with national patterns and trends in the total household population, because about 93 percent of the population lives in households with telephones. However, surveys conducted by telephone were found to produce underestimates of illicit drug use prevalence. In a 1988 national telephone survey, estimated rates of past year use were 5.2 percent for marijuana and 1.4 percent for cocaine. Comparable data from a personal visit survey (including only households with telephones and reedited and reweighted to control for differences in data collection protocols) were 8.0 percent for marijuana and 3.1 percent for cocaine use. Comparisons with several other telephone surveys collecting illicit drug use data showed similar results. Based on these results, researchers are advised to use caution in using telephone surveys to produce drug use prevalence estimates.
Collecting population data on sensitive issues such as non-suicidal self-injury (NSSI) is problematic. Case note audits or hospital/clinic based presentations only record severe cases and do not distinguish between suicidal and non-suicidal intent. Community surveys have largely been limited to school and university students, resulting in little much needed population-based data on NSSI. Collecting these data via a large scale population survey presents challenges to survey methodologists. This paper addresses the methodological issues associated with collecting this type of data via CATI.
An Australia-wide population survey was funded by the Australian Government to determine prevalence estimates of NSSI and associations, predictors, relationships to suicide attempts and suicide ideation, and outcomes. Computer assisted telephone interviewing (CATI) on a random sample of the Australian population aged 10+ years of age from randomly selected households, was undertaken.
Overall, from 31,216 eligible households, 12,006 interviews were undertaken (response rate 38.5%). The 4-week prevalence of NSSI was 1.1% (95% ci 0.9-1.3%) and lifetime prevalence was 8.1% (95% ci 7.6-8.6).
Methodological concerns and challenges in regard to collection of these data included extensive interviewer training and post interview counselling. Ethical considerations, especially with children as young as 10 years of age being asked sensitive questions, were addressed prior to data collection. The solution required a large amount of information to be sent to each selected household prior to the telephone interview which contributed to a lower than expected response rate. Non-coverage error caused by the population of interest being highly mobile, homeless or institutionalised was also a suspected issue in this low prevalence condition. In many circumstances the numbers missing from the sampling frame are small enough to not cause worry, especially when compared with the population as a whole, but within the population of interest to us, we believe that the most likely direction of bias is towards an underestimation of our prevalence estimates.
Collecting valid and reliable data is a paramount concern of health researchers and survey research methodologists. The challenge is to design cost-effective studies especially those associated with low-prevalence issues, and to balance time and convenience against validity, reliability, sampling, coverage, non-response and measurement error issues.
Epidemiological and other studies that require participants to respond by completing a questionnaire face the growing threat of non-response. Response rates to household telephone surveys are diminishing because of changes in telecommunications, marketing and culture. Accordingly, updated information is required about the rate of telephone listing in directories and optimal strategies to maximise survey participation.
A total of 3426 households in Sydney, Australia were approached to participate in a computer assisted telephone interview (CATI) regarding their domestic (recycled and/or drinking) water usage. Only randomly selected households in the suburb and postcode of interest with a telephone number listed in the Electronic White Pages (EWP) that matched Australian electoral records were approached.
The CATI response rate for eligible households contacted by telephone was 39%. The rate of matching of electoral and EWP records, a measure of telephone directory coverage, was 55%.
The use of a combination of approaches, such as an advance letter, interviewer training, establishment of researcher credentials, increasing call attempts and targeted call times, remains a good strategy to maximise telephone response rates. However, by way of preparation for future technological changes, reduced telephone number listings and people's increasing resistance to unwanted phone calls, alternatives to telephone surveys, such as internet-based approaches, should be investigated.
Epidemiological research often requires collection of data from a representative sample of the community or recruitment of specific groups through broad community approaches. The population coverage of traditional survey methods such as mail-outs to residential addresses, and telephone contact via public directories or random-digit-dialing is declining and survey response rates are falling. There is a need to explore new sampling frames and consider multiple response modes including those offered by changes in telecommunications and internet technology.
We evaluated response rates and cost-effectiveness for three modes of survey administration (postal invitation/postal survey, postal invitation/internet survey and postal invitation/telephone survey) and two styles of contact approach (personalised and generic) in a community survey of greywater use. Potential respondents were contacted only once, with no follow up of non-responders.
The telephone survey produced the highest adjusted response rate (30.2%), followed by the personalised postal survey (10.5%), generic postal survey (7.5%) and then the internet survey (4.7% for the personalised approach and 2.2% for the generic approach). There were some differences in household characteristics and greywater use rates between respondents to different survey modes, and between respondents to personalised and generic approaches. These may be attributable to the differing levels of motivations needed for a response, and varying levels of interest in the survey topic among greywater users and non-users. The generic postal survey had the lowest costs per valid survey received (Australian $22.93), followed by the personalised postal survey ($24.75).
Our findings suggest that postal surveys currently remain the most economic option for population-based studies, with similar costs for personalised and generic approaches. Internet surveys may be effective for specialised groups where email lists are available for initial contact, but barriers other than household internet access still exist for community-based surveys. Given the increasing recruitment challenges facing community-based studies, there is an imperative to gather contemporary comparative data on different survey modes and recruitment approaches in order to determine their strengths, limitations and costs. Researchers also need to document and report on the potential biases in the target and respondent populations and how this may affect the data collected.
Survey methods; Postal survey; Telephone survey; Internet survey; Cost-effectiveness
AIM: To determine the prevalences of symptoms consistent with gastroesophageal reflux disease (GERD) and dyspepsia in South America.
METHODS: A telephone survey was conducted among adult owners of land-based telephones in São Paulo, Brazil, using previously validated computer-assisted sampling and survey protocols. The Portuguese-language survey included (1) sociodemographic characteristics (e.g., weight, height, smoking) and comorbidities; (2) dietary habits; (3) presence of symptoms consistent with GERD or dyspepsia within the prior 3 mo; and (4) use of medications and other therapies to manage symptoms. Data were stratified post-hoc into three homogeneous geographical regions of São Paulo according to the Social Exclusion Indices of the districts and postal codes. Survey response data from each respondent were weighted by the numbers of adults and landline telephones in each household. The analyses were weighted to account for sampling design and to be representative of the São Paulo population according to city census data.
RESULTS: Among 4570 households contacted, an adult from 3050 (66.7%) agreed to participate. The nonresponse rate was 33.3%. The mean (SE) respondent age was 42.6 (16.0) years. More than half of all respondents were women (53.1%), aged 18 through 49 years (66.7%), married or cohabitating (52.5%), and/or above normal-weight standards (i.e., 35.3% overweight and 16.3% obese). A total of 26.5% of women were perimenopausal. More than 20% of respondents reported highly frequent symptoms consistent with GERD (e.g., gastric burning sensation = 20.8%) or dyspepsia (e.g., abdominal swelling/distension = 20.9%) at least once per month. Prevalences of these symptoms were significantly (approximately 1.5- to 2.0-fold) higher among women than men but did not vary significantly as a function of advancing age. For instance, 14.1% of women reported that they experienced stomach burning (symptom of GERD) at least twice per week, compared to 8.4% of men (P = 0.012 by χ2 test). A total of 15.7% of women reported that they experienced abdominal swelling (symptom of dyspepsia) at least twice per week, compared to 6.4% of men (P < 0.001 by χ2 test). Despite frequent manifestations of GERD or dyspepsia, most (≥ 90%) respondents reported that they neither received prescription medications from physicians, nor took behavioral measures (e.g., dietary modifications), to manage symptoms.
CONCLUSION: Symptoms consistent with dyspepsia and GERD are prevalent in Brazil and represent major public-health and clinical challenges.
Dyspepsia; Epidemiology; Esophagus; Gastroesophageal reflux; Prevalence; Stomach
Telephone surveys have been used widely in public health research internationally and are being increasingly used in Ireland and the U.K.
This study compared three telephone surveys conducted on the island of Ireland from 2000 to 2004, examining study methodology, outcome measures and the per unit cost of each completed survey. We critically examined these population-based surveys which all explored health related attitudes and behaviours.
Over the period from 2000 to 2005 the percentage of calls which succeeded in contacting an eligible member of the public fell, from 52.9% to 31.8%. There was a drop in response rates to the surveys (once contact was established) from 58.6% to 17.7%. Costs per completed interview rose from €4.48 to €15.65.
Respondents were prepared to spend 10–15 minutes being surveyed, but longer surveys yielded poorer completion rates. Respondents were willing to discuss issues of a sensitive nature. Interviews after 9 pm were less successful, with complaints about the lateness of the call. Randomisation from electronic residential telephone directory databases excluded all ex-directory numbers and thus was not as representative of the general population as number generation by the hundred-bank method. However the directory database was more efficient in excluding business and fax numbers.
Researchers should take cognisance of under-representativeness of land-line telephone surveys, of the increasing difficulties in contacting the public and of mounting personnel costs. We conclude that telephone surveying now requires additional strategies such as a multimode approach, or incentivisation, to be a useful, cost-effective means of acquiring data on public health matters in Ireland and the U.K.
Web interviewing is becoming increasingly popular worldwide, because it has several advantages over telephone interviewing such as lower costs and shorter fieldwork periods. However, there are also concerns about data quality of web surveys. The aim of this study was to compare the International Tobacco Control (ITC) Netherlands web and telephone samples on demographic and smoking related variables to assess differences in data quality.
Wave 1 of the ITC Netherlands Survey was completed by 1,668 web respondents and 404 telephone respondents of 18 years and older. The two surveys were conducted in parallel among adults who reported smoking at least monthly and had smoked at least 100 cigarettes over their lifetime.
Both the web and telephone survey had a cooperation rate of 78%. Web respondents with a fixed line telephone were significantly more often married, had a lower educational level, and were older than web respondents without a fixed line telephone. Telephone respondents with internet access were significantly more often married, had a higher educational level, and were younger than telephone respondents without internet. Web respondents were significantly less often married and lower educated than the Dutch population of smokers. Telephone respondents were significantly less often married and higher educated than the Dutch population of smokers. Web respondents used the "don't know" options more often than telephone respondents. Telephone respondents were somewhat more negative about smoking, had less intention to quit smoking, and had more self efficacy for quitting. The known association between educational level and self efficacy was present only in the web survey.
Differences between the web and telephone sample were present, but the differences were small and not consistently favourable for either web or telephone interviewing. Our study findings suggested sometimes a better data quality in the web than in the telephone survey. Therefore, web interviewing can be a good alternative to telephone interviewing.
Since 1997, the NSW Population Health Survey (NSWPHS) had selected the sample using random digit dialing of landline telephone numbers. When the survey began coverage of the population by landline phone frames was high (96%). As landline coverage in Australia has declined and continues to do so, in 2012, a sample of mobile telephone numbers was added to the survey using an overlapping dual-frame design. Details of the methodology are published elsewhere. This paper discusses the impacts of the sampling frame change on the time series, and provides possible approaches to handling these impacts.
Prevalence estimates were calculated for type of phone-use, and a range of health indicators. Prevalence ratios (PR) for each of the health indicators were also calculated using Poisson regression analysis with robust variance estimation by type of phone-use. Health estimates for 2012 were compared to 2011. The full time series was examined for selected health indicators.
It was estimated from the 2012 NSWPHS that 20.0% of the NSW population were mobile-only phone users. Looking at the full time series for overweight or obese and current smoking if the NSWPHS had continued to be undertaken only using a landline frame, overweight or obese would have been shown to continue to increase and current smoking would have been shown to continue to decrease. However, with the introduction of the overlapping dual-frame design in 2012, overweight or obese increased until 2011 and then decreased in 2012, and current smoking decreased until 2011, and then increased in 2012. Our examination of these time series showed that the changes were a consequence of the sampling frame change and were not real changes. Both the backcasting method and the minimal coverage method could adequately adjust for the design change and allow for the continuation of the time series.
The inclusion of the mobile telephone numbers, through an overlapping dual-frame design, did impact on the time series for some of the health indicators collected through the NSWPHS, but only in that it corrected the estimates that were being calculated from a sample frame that was progressively covering less of the population.
Telephone health survey; Sample survey; Overlapping dual-frame; Time series
The prevalence of food insecurity is substantially higher among Australians of Aboriginal or Torres Strait Islander descent. The purpose of this study is to explain the relationship between food insecurity and Aboriginal and Torres Islander status in the state of Victoria.
Data were obtained from the 2008 Victorian Population Health Survey; a cross-sectional landline computer-assisted telephone interview survey of 34,168 randomly selected Victorians aged 18 years and older; including 339 Aboriginal and Torres Strait Islanders. We categorised a respondent as food insecure, if in the previous 12 months, they reported having run out of food and not being able to afford to buy more. We used multivariable logistic regression to adjust for age, sex, socioeconomic status (household income), lifestyle risk factors (smoking, alcohol consumption and obesity), social support (ability to get help from family, friends or neighbours), household composition (lone parent status, household with a child, and household size), and geographic location (rurality).
Aboriginal and Torres Strait Islanders (20.3%) were more likely than their non-Aboriginal and Torres Strait Islander counterparts (5.4%) to have experienced food insecurity; odds ratio (OR) = 4.5 (95% CI; 2.7-7.4). Controlling for age, SES, smoking, obesity and inability to get help from family or friends reduced the odds ratio by 38%; ORadjusted = 2.8 (1.6-5.0).
Social determinants and lifestyle risk factors only partially explained the higher prevalence of food insecurity among Aboriginal and Torres Strait Islanders in Victoria. Further research is needed to explain the disparity in food insecurity between the two populations in order to inform and guide corrective action.
Chronic infection with the hepatitis B virus is endemic in Southeast Asian populations, including Vietnamese. Previous research has documented low rates of hepatitis B vaccine coverage among Vietnamese-American children and adolescents ages 3 to 18. To address this problem, we designed and tested in a controlled trial 2 public health outreach “catch-up” campaigns for this population.
In the Houston, Texas metropolitan area, we mounted a media-led information and education campaign, and in the Dallas metropolitan area, we organized a community mobilization strategy. We evaluated the success of these interventions in a controlled trial, using the Washington, DC metropolitan area as a control site. To do so, we conducted computer-assisted telephone interviews with random samples of ~500 Vietnamese-American households in each of the 3 study sites both before and after the interventions. We assessed respondents’ awareness and knowledge of hepatitis B and asked for hepatitis B vaccination dates for a randomly selected child in each household. When possible, we validated vaccination dates through direct contact with each child’s providers.
Awareness of hepatitis B increased significantly between the pre- and postintervention surveys in all 3 areas, and the increase in the media education area (+21.5 percentage points) was significantly larger than in the control area (+9.0 percentage points). At postintervention, significantly more parents knew that free vaccines were available for children in the media education (+31.9 percentage points) and community mobilization (+16.7 percentage points) areas than in the control area (+4.7 percentage points). An increase in knowledge of sexual transmission of hepatitis B virus was significant in the media education area (+14.0 percentage points) and community mobilization (+13.6 percentage points) areas compared with the control area (+5.2 percentage points). Parent- or provider-reported data (n = 783 for pre- and n = 784 for postintervention surveys) suggest that receipt of 3 hepatitis B vaccinations increased significantly in the community mobilization area (from 26.6% at pre- to 38.8% at postintervention) and in the media intervention area (28.5% at pre- and 39.4% at postintervention), but declined slightly in the control community (37.8% at pre- and 33.5% at postintervention). Multiple logistic regression analyses estimated that the odds of receiving 3 hepatitis B vaccine doses were significantly greater for both community mobilization (odds ratio 2.15, 95% confidence interval 1.16–3.97) and media campaign (odds ratio 3.02, 95% confidence interval 1.62–5.64) interventions compared with the control area. The odds of being vaccinated were significantly greater for children who had had at least 1 diphtheria-tetanuspertussis shot, and whose parents were married, knew someone with liver disease, had heard of hepatitis B, and had greater knowledge about hepatitis B. The odds of being vaccinated were significantly lower for older children.
Both community mobilization and media campaigns significantly increased the knowledge of Vietnamese-American parents about hepatitis B vaccination, and the receipt of “catch-up” vaccinations among their children.
hepatitis B vaccination; catch-up; Vietnamese-Americans; HBV, hepatitis B virus; HepB, hepatitis B vaccine; CDC, Centers for Disease Control and Prevention; DTP, diphtheria, tetanus toxoid and pertussis; VFC, Vaccines for Children; EDCC, East Dallas Counseling Center; OR, odds ratio; CI, confidence interval
Random-digit dialing (RDD) using landline telephone numbers is the historical gold standard for control recruitment in population-based epidemiologic research. However, increasing cell-phone usage and diminishing response rates suggest that the effectiveness of RDD in recruiting a random sample of the general population, particularly for younger target populations, is decreasing. In this study, we compared landline RDD with alternative methods of control recruitment, including RDD using cell-phone numbers and address-based sampling (ABS), to recruit primarily white men aged 18–55 years into a study of testicular cancer susceptibility conducted in the Philadelphia, Pennsylvania, metropolitan area between 2009 and 2012. With few exceptions, eligible and enrolled controls recruited by means of RDD and ABS were similar with regard to characteristics for which data were collected on the screening survey. While we find ABS to be a comparably effective method of recruiting young males compared with landline RDD, we acknowledge the potential impact that selection bias may have had on our results because of poor overall response rates, which ranged from 11.4% for landline RDD to 1.7% for ABS.
case-control studies; data collection; epidemiologic methods; postal service; telephone; testicular neoplasms
Low cost, effective interventions are needed to deal with the major global burden of HIV/AIDS. Telephone consultation offers the potential to improve health of people living with HIV/AIDS cost-effectively and to reduce the burden on affected people and health systems. The aim of this systematic review was to assess the effectiveness of telephone consultation for HIV/AIDS care.
We undertook a comprehensive search of peer-reviewed and grey literature. Two authors independently screened citations, extracted data and assessed the quality of randomized controlled trials which compared telephone interventions with control groups for HIV/AIDS care. Telephone interventions were voice calls with landlines or mobile phones. We present a narrative overview of the results as the obtained trials were highly heterogeneous in design and therefore the data could not be pooled for statistical analysis.
The search yielded 3321 citations. Of these, nine studies involving 1162 participants met the inclusion criteria. The telephone was used for giving HIV test results (one trial) and for delivering behavioural interventions aimed at improving mental health (four trials), reducing sexual transmission risk (one trial), improving medication adherence (two trials) and smoking cessation (one trial). Limited effectiveness of the intervention was found in the trial giving HIV test results, in one trial supporting medication adherence and in one trial for smoking cessation by telephone.
We found some evidence of the benefits of interventions delivered by telephone for the health of people living with HIV or at risk of HIV. However, only limited conclusions can be drawn as we only found nine studies for five different interventions and they mainly took place in the United States. Nevertheless, given the high penetration of low-cost mobile phones in countries with high HIV endemicity, more evidence is needed on how telephone consultation can aid in the delivery of HIV prevention, treatment and care.