Our study shows that weighting adjustment may be an effective tool to adjust for noncoverage biases associated with the growing cell-phone population. Estimates from the fully adjusted landline sample are very similar to those from the sample adding cell-phone samples. Although the noncoverage bias examined in this study does raise serious concerns, future telephone surveys should give serious considerations to an inclusion of CO populations for four reasons. First, the proportion of those who are not covered by the landline telephone frame in the population is expected to grow in the future. Noncoverage bias examined in this study was relatively small, partly because this proportion was rather small, at <15 percent. Note that it has been reported that the CO rates were over 25 percents in states such as Oklahoma and Utah as of 2007 (Blumberg et al. 2009
). Given the large differences between the CO and landline adults and the expected growth of the CO group, relying on landline samples alone in population-based surveys may jeopardize the data quality even if sophisticated weighting adjustments are applied.
Second, the study showed the distinctiveness of the CM adults throughout various characteristics. This distinctiveness remained even after controlling for demographic characteristics commonly used in weighting adjustments. As CM people are not likely to answer the landline telephone calls but to answer cell-phone calls, they may be a source of nonresponse bias and, consequently, a threat to representativeness of telephone surveys using landline samples. Weighting adjustments may not be effective for such cases, because distinctiveness of the CM group in health measures remained significant even after controlling for demographic variables.
Third, there may be subgroups who are more affected by the cell-phone usage trends and less amenable to effectiveness of weighting adjustment. Larger problems were found with noncoverage bias in subgroups, such as young adults and low-income populations (Blumberg and Luke 2007
; Blumberg and Luke 2009b
;). Moreover, by including cell samples, telephone surveys may gain the statistical power for analyzing some subgroups that may have higher cell-phone penetration. While additional studies on this aspect will show the actual magnitudes, it is imaginable that cell samples are much more productive in obtaining highly mobile populations than landline samples.
Fourth, weighting itself does not guarantee a removal or decrease in noncoverage bias. Its effectiveness depends on the type of weighting methods, variables and their nature controlled under weighting, and the tightness of weighting variables not only with telephone phone usage status but also survey variables.
Given the results of this study and the cell-phone usage trend, it is important that RDD health surveys, including those conducted by state agencies, continue to explore ways to enhance their data quality by introducing cell-phone samples to the data collection. Obviously, including those who use cell phones mainly in telephone survey data collection is important, and it is worthwhile to discuss how to include them. Some RDD telephone surveys utilize full dual-frame samples—both landline- and cell-phone samples without screening by telephone usage (Kennedy 2007
; Keeter, Dimock, and Christian 2008
;). While this is feasible, there is evidence that those using cell phone more frequently are more accessible and amenable to responding to cell-phone survey interviews (Brick et al. 2006
). This suggests that lower response rates may occur in full dual-frame surveys among those who have both landline and cell phones but do not use cell phones frequently. This hypothesis is supported by two studies by Pew Research that found that those using both dual users sampled from a landline frame are different from those sampled from a cell-phone frame (Kennedy 2007
; Keeter, Dimock, and Christian 2008
;). Of course, data collection costs must also be considered, and including more adults from the cell-phone frame will reduce cost per completed case.
As cell-phone usage grows, noncoverage bias of landline telephone surveys may not be adequately compensated by weighting. Such datasets may produce biased estimates, which may have substantial policy implications. For example, state health telephone surveys typically produce point-in-time estimates of the uninsured rate in those states that average 23 percent lower compared with Current Population Survey (CPS) data (Call, Davern, and Blewett 2007
). Although the state survey–CPS difference is due to multiple factors, including landline telephone coverage, it suggests the problem for policy makers when two credible data sources provide widely different estimates of the problem to be addressed. If the policy maker assumes that the CPS estimate is correct, he or she must plan for an uninsured population that is substantially larger than if he or she were to assume that the state survey estimate is more accurate. In that case, policy makers may be discouraged from attempting to address a problem because it appears to be beyond their resources. A large data gap was a contributing factor in the defeat of health care reform in California in 2008 (see analysis by the California Legislative Analyst's Office
 for the Health Care Security and Cost Reduction Act, ABX1 1). In a similar fashion, as CO households increase as a proportion of all households, the threat of noncoverage bias in health survey estimates could mislead policy makers with possibly serious consequences for their ability to address important health policy issues. Therefore, cell-phone samples should be included in telephone surveys not only to represent the population but also to assess the trends and magnitudes of potential noncoverage bias. Policy makers depend on good data to help them make good policy.